this post was submitted on 30 Apr 2026
76 points (92.2% liked)

Fuck AI

7343 readers
641 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
 

Crossposted from https://lemmy.ca/post/64120740

It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder. PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.

The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models. As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments.

top 13 comments
sorted by: hot top controversial new old
[–] StillAlive@piefed.world 73 points 1 month ago (3 children)

Chatbots can't confess. They are just parroting stuff without meaning it.

[–] CosmoNova@lemmy.world 52 points 1 month ago (1 children)

Humanizing AI really needs to stop. They don‘t communicate because they don‘t have a conscience. They just put out letters based on probability and a little bit of randomness.

[–] lastweakness@lemmy.world 3 points 1 month ago

But then how can a company founder pass on accountability?

[–] horn_e4_beaver@discuss.tchncs.de 2 points 1 month ago (1 children)

You're absolutely right, I did just pipe the output from my confabulation unit through to /dev/stdin.

[–] ptu@sopuli.xyz 1 points 1 month ago

And the reasoning I pulled out from /.local/share/Trash/. It’s a well known bug and you were right to question it!

[–] CultLeader4Hire@lemmy.world -2 points 1 month ago (2 children)

Sure but it will explain what it did/why it did that. It’s not a “confession” but an explanation

[–] jeeva@lemmy.world 5 points 1 month ago

Mild point: is an explanation the same as you expect if it's context is driven by the previous text-in-chat?

E.g. it's not someone explaining themselves, but instead a new instance explaining what it sees in history?

To an extent, it seems like this isn't "explaining what it did" and more "words to explain what just happened, based on the chat history".

[–] queermunist@lemmy.ml 5 points 1 month ago* (last edited 1 month ago)

Okay, but using "confession" instead of "explanation" gives the sentence a different underlying meaning.

Headlines need to be precise. It matters that they used one word instead of the other.

[–] NABDad@lemmy.world 22 points 1 month ago (1 children)

AI doesn't give intelligence to machines. It takes it away from humans.

[–] Agrivar@lemmy.world 2 points 1 month ago (1 children)

Well worded! I'm going to have to use this, is it yours?

[–] NABDad@lemmy.world 2 points 1 month ago (1 children)
[–] Agrivar@lemmy.world 3 points 1 month ago
[–] dylanmorgan@slrpnk.net 9 points 1 month ago

Guarantee if you look at the facts, there’s a history of their devs basically saying “ignore the safety protocols, do [thing]” and someone used that in a prompt with an otherwise imprecise command set.