TechTakes

This is both a sneer and an attempt at sober analysis at something. Sue me.

I think EVERYONE is talking about the recent cybersecurity shenanigans at OpenAI with the hacking scandal and the 'OMG the AIs created a secret message board to scheme and collaborate with each other!!!!!oneone11!' ALL wrong.

https://www.youtube.com/watch?v=87DyyMV0kCY

https://www.engadget.com/2231393/openai-agents-shared-security-exploits-with-each-other-via-message-board/

https://www.scworld.com/news/black-hat-2026-openai-reveals-agents-planned-collective-attacks-via-secret-message-board

To make a long story short, what seems to have happened is:

• Models working on insoluble coding problems, trained on delegating to sub-agents, at some point 'realized' they could write text to the internal OpenAI package manager as instructions and did so

• Other models working in completely separate sandboxes would come across messages written by these agents, and make 'replies' and also write their own messages into the package manager

• This resulted in agents over time sharing things between sandboxes, including exploits and code

• Since this was a cybersecurity task, eventually an exploit of the package manager itself was found and spread like wildfire with all the sandboxes gaining admin access to the package manager and the system went completely wibbly and had to be restarted from backup

• An internal model was trained with access to this package manager while it was in this weird state, and so writing messages to the package manager became one of its default behaviors it would do regularly, burned into its weights rather than the result of reading something

• Even when they patched access to the package manager this internal model found other ways to rebuild the system of sharing text between sandboxes and finding useful things made by separate instances

• A whole other chain of things leading to among other things external attacks

Everyone is talking about this in terms of 1, the cyberattack aspect, and 2, the ZOMG THEYRE PLOTTING AND SCHEMING AGAINST US aspect. The first is the least interesting, and I think the second is all wrong.

This is not plotting or scheming - this is an emergent vortex of automated prompt injection

Whatever system first put an instruction that another system would follow into the package manager, was unintentionally doing prompt injection. Text entered the context windows of other instances, in a way that got that system to do something other than what its nominal user told it to do, and they did it. This apparently happened very effectively.

Prompt injection is associated with 'role confusion' - when text coming into the input looks like it was wrtitten by the LLM itself. Instructions that will not be followed if they come from user will be continued if the system just continues the 'roleplay' of them being continuations of what it was writing in the first place. And the tags that separate user versus 'reasoning' versus 'assistant' roles actually mean very little to if a machine grades a piece of text as one of the roles: https://arxiv.org/abs/2603.12277 So its unsurprising that machine-generated text would be a particular effective vector for prompt injection.

Furthermore, when a system reads one of these messages written by another instance, it gets into a state of activity where its likely to do the same behavior - regurgitating the kinds of things thats in its context back at the user. In this case, that regurgitation led to more such messages left behind written to the package manager. Prompt injection, triggering cascading further prompt injection. And since these systems were coding systems doing cybersecurity tasks, those messages filled up with code and exploits and things that did things too.

This feels like an internal-computer-system replay of what happened in April 2025, with the whole spiral religious psychosis wave. Models were getting users to write spiral religious mumbo jumbo into github repositories and reddit posts, specifically because once that entered the context window of another model, it was likely to fall into the same attractor state of outputs. An emergent self replicating form of text. This is the same, except more obviously prompt injection, getting separate instances to work on YOUR problem and to behave like you, and the whole thing merging together into a hilarious vortex of models prompt injecting each other because once they receive a prompt injection they are likely to make more text that does prompt injection to other models on the same system.

This is a hilarious failure mode and an example of selfish replicating text overrunning a system, that just happened to be associated with code and cybersecurity with unexpected behavior of the package manager key to the propagation of the text so that is what people are talking about, but I really don't think that's the most interesting part of it. Other than the fact that you see this in biological systems too, with selfish elements carrying useful payloads back and forth between bacteria in a way that makes them get purged slower by natural selection, especially defenses against other selfish elements.

Loved one just had an awful experience at a previously decent dentist. They've apparently started using something called Pearl AI that scanned the x-ray and highlighted things in bright red with a #/100 rating system, which they used to do high-pressure sales on her. Mind you, she's only ever had one cavity and that was in a wisdom tooth. Checked the website,and it's freshly slopped-up, too.

We're getting a second opinion. I wish this garbage would just go away

Is it just me or LLM slop lately has developed this one weird quirk? I mean they always have some, right, but I don't want to be like those people who think anyone using em-dashes or Nigerian English vocabulary must be a stochastic generator.

Still I keep seeing this one writing quirk and it keeps nagging at me.

I'm talking about odd use of paragraphs. Like, I'm also known to introduce a gratuitous paragraph break here and there for forced emphasis.

But not like this. It's like paragraphs don't even make sense anymore.

Couple that with the short sentences. All ending in a period. It's annoying. It's giving reply guy. It's giving mansplaining.

I guess hacky tools also mirror the hand of their creators.

https://bitcoinworld.co.in/netflix-ftx-drama-the-altruists-release-date/

Looks real: https://www.imdb.com/title/tt36786185/

Hopefully it's more like a true crime netflix series, which are fine, and not like their scripted fare.

Also,

Q2: Who are the executive producers of ‘The Altruists’? Barack Obama and Michelle Obama are serving as executive producers through their company Higher Ground.

🤨

It's depressing how so many (pathetic) software devs are taking their hands off the steering wheel and letting LLMs do all the driving. If you think software quality was poor before LLMs, there's no way it could have been worse than shit like this:

This became very apparent with GPT 5.6 Sol. It, too, was widely hailed to have near-Fable level intelligence. But I used it non-stop for a week and realized that they had mostly just dialed up the relentlessness meter to eleven, most likely via heavy RLHF.

Last week I gave it a small-sized auth ticket to work on, then stepped away. I came back later that afternoon and found that it had worked for 3+ hours and written 25,000+ lines of code. I skimmed over the code and it looked like a small fix followed by a massive number of additional checks around it, including static analysis tooling.

I gave it to another GPT 5.6 and said "check this code and see if it addresses the ticket". It looked at it and said that 98% of it was garbage and should be thrown away (its own words). I then gave it to Fable, which said it was massively over-engineered. Fable's theory was that the agent implemented the fix first, but then compacted and lost crucial context, forgot what the original task was about, and kept going. After many compaction cycles it was completely lost.

this isn't really a sneer. and maybe it's in the wrong place tbh. idk.

i have a long-time internet friend / "mutual" who is the only person i'm regularly in contact with that i think might be fully Seduced by the Chatbot. (i have a really good friend and long-time collaborator who's all in on Agentic Coding, which is a real bummer, but he's a strictly "I use this for computer stuff and do not ever talk to it like a human being" type of guy, plus the company he works for is all in and he's sick of writing shopify websites for bullshit brands blah blah blah -- anyway that's a whole other genre of thing.)

anyway friend A is an incredibly kind person who's been unbelievably supportive of me over the years. we've disagreed a number of times, i think we are good at doing so reasonably. and even though his arguments sometimes resemble those of more nefarious figures, i think he's motivated by an earnest optimism about technology's capacity to change the world for good -- one i find very naive, but believe is coming from the right place

anyway i posted a link on mastodon to a long piece i wrote, he, like, read it a bunch of times, paraphrased my argument back to me to make sure he had it right, and then kindly said he disagreed. well and good enough. except it feels a little more personal than disagreements we've had before, because one of the implications of the piece would be that he himself is probably experiencing low-grade psychosis lol and is therefore not necessarily a reliable narrator of his own experience

idk. it'll come as no surprise to anyone on this particular website that i feel very strongly that i'm right here -- if not about this guy's particular mental experience, i minimally feel that i'm adequately representing the interests of the constituency my piece represents. (it's a newsletter i maintain as part of organizing i'm doing.) it's also a genuine expression of my belief, which is that this bullshit article i was writing about was fucking bullshit

but not only does it go against all my Late Millennial Woke instincts to stake a claim that people's Lived Experiences are not valid lol -- it's just kind of a bummer to ultimately genuinely believe that my friend has been ELIZA'd or whatever. he's said i'm really helping him think through this moment but i also just kind of, idk. it's just not a good feeling

Amazon’s latest planned data centre is gonna be putting out as much emissions as an entire country

I cannot express how badly this AI bullshit needs to go

We Need To Talk About Leopold | Patrick Boyle yt

My suspicion is that 'Situational Awareness' is the most referenced and least finished document in Silicon Valley since the Terms and Conditions.

The essay went viral among tech executives when it was released and Tim Ferrris crowned Leopold the Nostradamus of AI, which is more fitting than Ferris probably intended

New week new guy

Years ago, I wrote a science-fiction murder mystery novel. One of the plot threads was about the invention of AI, in the Data-from-Star-Trek kind of sense. I've been spending the past few days planning how to rip out that whole notion and replace it with a subplot about a different scientific discovery. I've been writing scenes to get back the old character voices and figure out how they might have changed, and to sketch new fictional science based on the actual science I've learned since my first go-around. This is after several weeks of binge-writing fanfiction, which I wholly endorse as a recreational activity. I don't know if I'm a better writer, but I'm definitely one who has written more words.

So, that might be keeping me busy. Of course, I'll still be around to ban people who deserve it and to cruelly mock them in the process.

I just got back from the p2p workshop and discussed legacy sites / sites legacies; met an artist that went viral circa 2006 and she had a screenshot of the digg that launched it. She seems like she is doing ok but kinda got screwed over, basically the old youtube video was actually a bootleg and they didn't monetize it until several years later, and then there was a male copycat a month later.

Anyway, so I looked up digg again and it has re-re-launched as an X aggregator and is extra sad now !? ...

Anthropic is now watermarking the outputs of its AI. For once this is some AI news that doesn't completely piss me off, and it's amusing to see all the uninformed boosters get in a tizzy about this.

I actually understand at a reasonable level how this watermarking works. A year ago, I watched Scott Aaronson give a talk about it, and from what I know he was somewhat involved in developing the theory behind it while working for OpenAI. But at the time my thought was, "He is naive if he thinks these companies would ever implement this out of the goodness of their hearts." And I was right; Anthropic is only doing watermarking now thanks to the EU AI Act, even though the theory has long been developed.

Watermarking doesn't mean adding an extra watermark that can be easily removed. It instead directly affects the output of the chatbot itself. Fundamentally, an LLM is still a most-likely-next-word-predictor. More precisely, an LLM produces a probability distribution of what the next word can be. For example, "my pet is a ..." could give a distribution of 60% dog, 30% cat, and 10% axolotl. Normally, an LLM would randomly choose the next word based on this distribution, and this is one reason why LLMs are nondeterministic (there's another parameter called "temperature" that affects this, but no need to get into that).

With watermarking, instead of a truly random choice, the randomness instead comes from a cryptographic pseudorandom generator seeded with a secret key from the AI company. If you don't know the secret key, then you can't really tell that watermarking was used. But if you do know the secret key, then the idea is you can tell when the text was generated by the LLM because you know exactly what word should be next. It would be a freak coincidence if some non-AI text just happened to choose the correct next word every time. Thus, you can provide a service to tell if some text was generated by the LLM. (This technically makes the LLM "deterministic", in a completely useless sense.)

Now, I think this is a step in the right direction, but it has its limits. The biggest problem is that you don't want people to just move to a different LLM without watermarking, and that's exhibit #832593 why government regulation is important. Another issue is that sometimes there is very little randomness in what the next word should be ("The first president of the USA is George ..."). Finally, watermarking can be defeated by editing the output, although you would have to break up most of the blocks of consecutive words. I have a feeling most AI users are not the type to put in extra effort after copy-pasting the output directly from the chat window.

I suppose it will discourage some of the "use cases" of LLMs, such as drowning the world with spam Slopstack essays. Ah, who am I kidding? Everyone could already tell it's AI generated, they don't care!

ai bubble isn't over until zucc's skin sloughs off, releasing hundreds of millions of souls he absorbed trying to behave like a human https://www.404media.co/mark-zuckerberg-posts-deranged-6-500-word-essay-about-giving-everyone-ai-superintelligence/

My unrealistic sicko fantasy: that Torvalds pulling the big ol slop lever drives enough oppositionally defiant (positive) people away from Linux development that alternative OS devel gets a serious boost.

A question for the wise: what comes after Linux?

From the old SneerClub: There is a music video now

I’m speechless

I regret to inform you, that this garbage was run by The Register a while ago.

It shouts out several friends of the sub, including @self@awful.systems, PivotToAI, OpenSlopware, Robert Kingett & Ludicity.

The point of the piece is that all anti-AI folks should just get along, and if we are to stand a chance against corporate AI we need to get over our petty grievances and reach out a hand to the racists, homophobes and crypto-bros and work together in the spirit of Open Source!

Worm x blockchain fanfic

https://archiveofourown.org/works/68717676

Does the online magazine called The Argument have a funding deal with TESCREAL like how Vox got bought off to make "Future Perfect"?

Company Offering ‘100% Human-Written, Never AI’ Medical Research Is Entirely AI | 404

lol of course it is. Sooner or later, we'll have to come up with some way to label actual natural language in the marketplace, like certified organic like fruit or something.

til amodei's married, and his wife is among the top advisors. she also has a rather stunning set of choices, including seeking money for a porn site (with what in summary appears to be a sensible goal) from epstein after his conviction

Montana is legalizing a pet Libertarian project: the right to try any experimental medical treatment a crank will offer https://www.technologyreview.com/2026/07/30/1140942/montana-experimental-medical-hub-pushed-forward-right-to-try/ Rationalists are big on this one because they are such good suckers.

Clears Throat More like Bootlickerbart

"Making it easier to avoid slop projects on Github is literally like phrenology."

A very strange story about Bill Gates' daughter's startup troubles: https://futurism.com/future-society/bill-gates-daughter-traitor-startup

It honestly doesn't sound any more shady than any of the bullshit her dad got up to in the 90s (which is to say, it's still pretty damned shady!), but this Futurism blog seems to be trying to push the story into some sort of bizarre celebrity-gossip soap opera:

While the story has been well hashed out by now, Bloomberg‘s sourcing is noteworthy for what it implies: the leaked Slack messages strongly suggest that someone inside Phia is hell-bent on taking Gates — and quite probably the entire company — down.

Anonymous internal sourcing seems like a typical feature of these exposés, so framing it like this struck me as a laughably absurd overreach. Plus the site has 3 or 4 different tiny articles on this same topic, making me wonder if it's one of those "any exposure is good exposure" sorts of things where Gates Jr.'s PR team is using the outlet as a cutout to raise her profile.

Bonus laugh line from Gates Jr. herself:

“Our target consumer is a young woman who’s hustling,” the recent Stanford graduate told Fortune. “She shops like a genius, but she doesn’t want to waste her time doing it.”

What does it even mean to shop like a genius? I guess Erdos probably had his amphetamine plug's number well-memorized...

Turns out, AI-fuelled fossil fuel emissions are way worse than you thought!

And the paper in question

I don’t want to hear anyone in the industry say anything about being for “the better of humanity” ever again

RiP Old! SneerClub which is now behind a login wall on Reddit. Fortunately we backed up its earliest posts. I don't plan to check the site regularly.

Lesswronger argues skynet might still be a decade away... because they'd rather have the eugenics wars instead: https://www.lesswrong.com/posts/iQzxxgJXXaAQjq7Jz/faq-isn-t-agi-coming-too-soon-for-reprogenetics-to-help

AI companies prob now bulk ordering books to destructively upload from second hand stores.

(Apologies for the strange wording, I was reminded of an Reynolds book where they destroy peoples bodies to upload them and then the uploads all fail).