this post was submitted on 27 Nov 2025
45 points (95.9% liked)
Not the Onion
2874 readers
7 users here now
For true stories that are so ridiculous, that you could have sworn it was an !theonion worthy story.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
TL;DR: Teen used a common jailbreak method to get it to discuss suicide the way he wanted it to. Company argues that it's not responsible because intentional jailbreaking is against the TOS.
I'd not say typing a sentence is a "jailbreak" of any sort - but more so, LLMs should just straight not allow certain topics until some future where it is decided and regulated how they respond. Although at this point, I'd gladly lose my coding assistant in trade for making LLMs go away. Big tech is again being reckless with yet again little to no accountability. They need their toys taken away.
They already don't. It doesn't work. That's what happened here.
I mean, fair enough.