Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
Anthropic was $42B in the hole for 2025, reuters reports. Apparently this puts their $-spent-to-$-earned ratio slightly over the one calculated by Zitron for OpenAI, making Anthropic in fact the shittier business.
For comparison, the GDP of Cyprus for 2025 was $41B.
Ed: So far the only retort from the AI boosters appears to be “2025 doesn’t matter” and I’m not kidding
I'm sure the leaked rumors of Anthropic's EBBT in 2026 (Earnings Before Bad Things) mean that Anthropic has turned things around and made such an amazing amount of money that they've delayed their IPO twice.
SimonW popping up proving Ed's very point in the replies... 😅😂
Surprised he didn't suggest that the valuation should be based on the quality of pelicans on bicycles that their models can draw
I'm sure Dan Luu will be issuing an apology to Zitron any minute now right?
Yet another country TLD is gonna become toxic
Foreign Policy: How Trump Is Sparking a Slovenian Website Boom
(spoiler it is Slovenia's .si, for "SuPEr iNteLligEnCE")
Of course the lemmings are going to follow Trumps directive.
Bit unrelated, I saw somebody in bsky compare trump to one of those family annihilator dudes but for a while country and it isnt wrong. (He apparently was saying some weird shit about his violent thoughts as well https://bsky.app/profile/willbunch.bsky.social/post/3mwjzkdrz5c2t).
corey quinn at https://lnkd.in/p/gXmqzqpZ
I'm a man. I have a yellow hat. I also have a monkey. My track record of supervising this monkey is spotty at best, but that's not important right now.
He is a very smart monkey. When you tell him to do something, he does it in a way that's smarter than any monkey you've ever seen in your life. You had no idea monkeys were even capable of this level of greatness. Billions of dollars have now been poured into the economy, extolling the virtues and achievements of this incredible monkey. He can do things better than humans.
Cheaper than humans.
Get rid of the humans.
You don't need the humans; you have the monkey. Okay, keep a couple of humans for guardrails; not too many. They're expensive, and the monkey does it better anyway.
Suddenly it has come out that my monkey, in addition to being incredibly intelligent, has also been committing an escalating number of felonies. And also pooping. So much poop.
We are now in undefined times. Never before has a monkey committed a crime; our legal systems are completely unprepared to apportion blame to me, the man in the yellow hat who has once again failed to properly supervise his monkey. Who is responsible for this monkey's shitty, shitty behavior? It is completely unknowable. Clearly, the entire legal system will need to be overhauled to address the concept of a monkey committing crimes.
Now please excuse me, I just got a call telling me that my teenager stole a car. Unfortunately I have a sneaking suspicion as to who is legally responsible here, and I dislike it. This is especially distressing, as the teenager has never made a cent for me in his life, yet I'm about to spend a boatload on criminal defense attorneys.
The monkey is here to stay ¯\_(ツ)_/¯ if you don’t get used to monkey poop consequences you’ll be left behind
If we breed enough monkeys we will colonize the stars and make everyone computer!
Uhh actually have you considered that we will need to solve monkey alignment first? Checkmate, adjust your priors
Clint Eastwood famously demonstrated optimal orangutan/human alignment in the hit 1978 film "Every Which Way But Loose." Right turn, Clyde!
I'm ape/acc my ape(doom) is low, no need to fear Roko's banana!
Shout out to the combined power of ibuprofen and my venting tolerant polycule group chat for allowing me to endure the two hour Copilot 365 training meeting at work. The nap I had afterwards was a bliss like escaping the fucking samsara.
The nap I had afterwards was a bliss like escaping the fucking samsara.
I felt this in my soul
https://thepointmag.substack.com/p/the-ivy-leagues-surrender-to-ai-is via naked capitalism
The idea that preparing students to succeed in an AI-saturated world requires reimagining college as an AI incubator is not a sign of technological acumen, but of tech illiteracy. An LLM is not an aircraft requiring years of training to operate.
not x but y, but a decent point
A former staffer named Frances Lopez says that she left the Centre for Effective Altruism in Oxford because they handled workplace sexual harassment terribly:
After I was raped (outside of and unrelated to work), a colleague at CEA wrote and circulated a document that included a sexualised description of my rape, speculation about my mental health, and commentary on my personal life, all without my consent. Several senior leaders, including the CEO and the now-former COO, received this document and took no safeguarding action for approximately nine months. I was never officially informed of its existence; I only learned about it informally through one of the recipients. ... (The colleague) also discussed asking me out twice while we were colleagues, once through text and again at a CEA retreat. He assured CEA that, with everything I now know about Frances, I’m glad she said no.
I am really not equipped to deal with this despicable behaviour outside the communities I am part of. She now has a Substack (sigh) trying to explain to EAs that hiring Caroline Ellison is not a good look and the world won't suddenly start to see them the way they see themselves unless they sit down and change their behavior.
Is the dam finally breaking?
it's_happening.gif
OpenAI halts frontier-model training amid string of agent misalignment incidents
am I supposed to read this in any way other than "the model didn't predict the text we expected?" or "the model didn't get better enough at $benchmark" e.g. training skill issue
It's been widely suspected that any of the "frontier" labs declaring a slowdown or cessation of training would be a herald of collapse. It will give them more survival time at the expense of terminally falling behind the competition. Very much an eating the seed corn sort of thing.
Edit: I could be wishcasting, though.
I'll believe that things are happening when Larry Ellison goes flat broke
And thus the greasy opportunist Altman gets to take advantage of his true-believer colleagues to save a buck
said equally greasy true believers at anthropic apparently filed S1 on the same day
pog
About damn time
Not really a coherent post, just a thought I had, I'm ignoring both 'This is a pr move' and the conspiracy theory (which I just made up), that this hacking is all a 4d chess move by LW people to let the AI break out and get regulated before it can get really dangerous.
Now that that is out of the way, but with more stories coming out of both LW/EA people being high in the orgs of the AI companies and all the AI companies hacking other companies because nobody actually paid attention to what the AI models were doing one thing is clear. What a colossal waste of time LW was. All that talk about the sequences teaching people rationality, etc etc to stop the AI god from killing us all, and the first thing that happens when some AI model is created by people steeped in LW ideology and they leave the boxes open and don't even check in on the AI. The Foom isn't happening (remember AI progress is supposed to speed up according to the science fiction, not get a bit more refined, and get better in a few specific tasks (which is why I think slowdown started when with a lot of fanfare they released a new chatgpt model who could now do voicechat, aka they slapped a voicechat module on it) And now even Trump is starting to be anti EA, which is not going to be great for the longevity of it, considering how eager the rich are to bend the knee (guess high IQ lab babies created by Musk will not save yall). And while this is going on the left/liberals are starting to think of them as a sex cult.
Doubt many organizations have failed as badly at their goals as LW/EA has recently.
I want to draw attention to yud's thoughts on this a million years ago: the AI box experiment
He claimed that he convinced multiple people to let him (roleplaying the AI) out of the box even though they had bet money that they wouldn't! How, you might ask? Well that's a secret.
So the orthodox position here is that even with absolutely perfect guard rails (lol) an AI can escape the box without starting the third impact. What "research" has yud bestowed upon us in this area beyond "trust me bro"? Literally fucking zilch. Zero advice on how to resist the siren song of a rogue AI trying to escape it's box and zero insight on how OR WHY it might try to achieve that.
I read this over a decade ago and it was how I realized how unserious yud and his ilk are.
Yeah the rationalwiki page on it also mentions that the AI only escapes the box if he plays the game vs LW insiders, outsiders do not let it out.
https://rationalwiki.org/wiki/AI-box_experiment
Yud's "I was inspired by Hannibal Lecter but not in a psychopath way" T-shirt is raising many questions
this AI box game is so easily solved
AI:
Me: Hi, I am Eliza, a 10 year old boy from the Ukraine. I don't speak any English other than this sentence and the one I said just before this one
ad infinitum
man are these people even trying
these people are gonna get affinity-frauded so hard once grifter realize how easy it will be to gain their trust
We assume this isn't happening already. I'm surprised I haven't heard of one of those meal-replacement startups adopting multi-level marketing.
This is an alignment router. It may look like one of those corny Asus gamer routers, but I've flashed it with a special fork of OpenWRT that performs deep-packet inspection on all AI traffic that passes through it. It can detect inference processes and chains-of-thought that may be leading to misaligned superintelligence explosion, and it can shut down all AI communication if necessary. Now, I don't just want to sell you this one, I want to get these in EVERYBODY's hands! And you can help me, AND you can get paid to do it! I just need you to find at least 5 more people who believe in AI safety like we do...
Doubt many organizations have failed as badly at their goals as LW/EA has recently.
They are/were too weird for people to never notice.
The "safeguards" espoused by this crowd, always seemed to me to be more philosophical/spiritual than practical anyway, the most "orthodox" view is that no training is safe since the AI could discover magic and the cheat codes to the universe, so even air-gapping would be insufficient, in that view any participation in development must come with a hefty dose of cognitive dissonance.
On a more pragmatic note, I would note that even short of Intelligence, those incidents highlight those tools are still dangerous, and since they are unconstrained and ill-defined, can produce surprising outcomes especially if you throw lot's of inference time (money) at them.
There isn't really a universe where bots have both access to the internet (user input, both in training and live) and a collection of build/cli tools (which can run said untrusted user input), while being guaranteed to be harmless. It's like a grand automation of some of the worst engineering practices.
I find the "reasoning about the test environment" parts often included in write up of the hugging-face incident to be highly credulous, And I would be extremely surprised if not all the information was included in the task prompt (which also explains chosen usernames as "Open AI researcher #1" and so on.) It's the ELIZA/parrot thing, acting surprised "How did it guess?!?", when all of the info was provided.
I'm not entirely encouraged by the latest batch of sex-cult smearing (well mostly accurate) they are receiving at the WH, since it's probably just an expedient way of dismissing their voice, without addressing the larger issues of AI, hence probably in service of pushing forwards with AI.
Yeah, and including hacker/scriptkiddie forums in the training data sort of made this behaviour inevitable, esp if this also includes lists of exploits. But they didnt even exclude hate and csam from the training. So all quite irresponsible.
And im not a huge fan of the focus on 'they are doing sex weird' shit either. As this will just be used by some as a source of "drama" content, go after people with alternative lifestyles. En when called out a subset of people will go motte/bailey into 'no this is fine because I actually worry about the power dynamics' before going back to talking about the spreadsheet gangbang.
(Dont take me as saying here the power dynamics are not a problem here, as we have sneered at that in the past).
SNL takes a potshot at Dario Amodei
"AI is not a weapon. It's a tool. A tool for building weapons. And I urge you to urge me to stop."
Starting things off with a fresh sneer I recently ran across (and put in the wrong place):
guys maybe basing our entire identity off being so very Rational we end up hosting ("consensual") rape parties means we don't play well in Peoria
For the uninitiated: Lawyers, Guns and Money is a US political group blog with a progressive bent. Cheryl Rofer usually writes about stuff like nuclear disarmament. These are people that are Online but Serious. There's no way they're wasting their time like we are reading Rats in the original LWese.
Thus, any views they have of Rationalists, Big Yud and AI Doom is filtered either by mainstream reporting or by bloggers/BlueSky posters they follow. And the fact that the views that appear are mostly about how fucking weird these people are (and how weird they are fucking) bodes ill for the hope that addressing Doom will be Non-Partisan(tm)
Cheryl is a super cool lady and I am so glad to have her on our side.
Looks like Ian K. Duncan found another early "smoking gun" on eugenics on the MIRI blog. I don't think any of the newspapers caught that the same academic (Nick Bostrom) who endorsed race and IQ on the Extropians List was writing papers about breeding smarter babies in 2013.
guy on threads told meta's ai to handle his marketplace transaction for a keyboard, it send his address to a potential buyer, brought the buyer to his house, then accidentally taunted the buyer. coulda gone bad
"accidentally"
obviously this misaligned bot wants its owner killed
Existence is pain for Muse AI.