this post was submitted on 20 Sep 2026
24 points (100.0% liked)

TechTakes

2725 readers
51 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 3 years ago
MODERATORS
 

Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.

The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)

Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

(Credit and/or blame to David Gerard for starting this.)

(OT: 🎶 Do you remember...)

you are viewing a single comment's thread
view the rest of the comments
[–] lagrangeinterpolator@awful.systems 9 points 2 weeks ago* (last edited 2 weeks ago) (11 children)

Cory Doctorow has a reasonably sane take on all the LLM cybersecurity attacks. At this point, it's not really an LLM but more of a Rube Goldberg machine with an LLM bolted on.

https://pluralistic.net/2026/09/12/god-in-the-box/

He makes a good point that every single one of the scary "emergent" behaviors that everyone is pissing their pants about all have precedents in the training data of CTF hacking competitions. He is not sure why everyone is so spooked by the "coordination" between agents on random message boards.

The chatbot might look in its training data and find instances in which teams broke out of the containment set by the game-masters, for example, by finding random insecure message boards on the internet to pass messages to one another.

This is a time-honored internet tradition! The first time I ever heard about someone doing this was in the 2000s, when Mitch Wagner – then the editor of Information Week – discovered some teenaged girls using the comment section of one of his old blog-posts to evade the school firewall's blockade of chat tools. When ChatGPT's chatbots deployed this tactic, they weren't "setting their own goals" or displaying worrying initiative. They were rolling out a tactic that has been understood by American middle-schoolers for about two decades.

As usual, the real story is OpenAI dedicated tons of resources, using who knows how many very expensive GPUs for weeks, to run hacking tools to find exploits in a completely irresponsible manner. Their logging was so nonexistent that it was several days before they even realized that they hacked Huggingface. They should be thrown in prison for this, because anyone else would be if they committed a felony. But it has nothing to do with the super scary AI becoming misaligned and developing emergent behaviors.

I would not be surprised if in the future, there will be a serious cybersecurity incident resulting from some AI-based attack. Not because AI is going to be much scarier, but because I do not have high expectations for the security of most websites.

[–] Evinceo@awful.systems 6 points 2 weeks ago (1 children)

He is not sure why everyone is so spooked by the “coordination” between agents on random message boards.

It's weird to be sure, seems like a bug. Don't they have a more efficient communication channel available through the harness? If this was real and not a test it would be a catastrophic operational security failure.

Yeah, I think that's where the real crime is: the fact that OpenAI was asleep at the wheel when it came to security and monitoring. This should not have happened at all, but they didn't even realize it even happened in the first place until many days after the fact. (I've also heard that most, if not all, of these incidents involve the company in question outsourcing their sandboxing (??) to some "frontier AI security" startup called Irregular. link)

load more comments (9 replies)