this post was submitted on 30 Apr 2026
75 points (92.1% liked)

Fuck AI

6892 readers
744 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
 

Crossposted from https://lemmy.ca/post/64120740

It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder. PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.

The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models. As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments.

top 13 comments
sorted by: hot top controversial new old
[–] StillAlive@piefed.world 72 points 3 days ago (3 children)

Chatbots can't confess. They are just parroting stuff without meaning it.

[–] CosmoNova@lemmy.world 52 points 3 days ago (1 children)

Humanizing AI really needs to stop. They don‘t communicate because they don‘t have a conscience. They just put out letters based on probability and a little bit of randomness.

[–] lastweakness@lemmy.world 3 points 2 days ago

But then how can a company founder pass on accountability?

[–] horn_e4_beaver@discuss.tchncs.de 2 points 3 days ago (1 children)

You're absolutely right, I did just pipe the output from my confabulation unit through to /dev/stdin.

[–] ptu@sopuli.xyz 1 points 3 days ago

And the reasoning I pulled out from /.local/share/Trash/. It’s a well known bug and you were right to question it!

[–] CultLeader4Hire@lemmy.world -2 points 3 days ago (2 children)

Sure but it will explain what it did/why it did that. It’s not a “confession” but an explanation

[–] jeeva@lemmy.world 5 points 3 days ago

Mild point: is an explanation the same as you expect if it's context is driven by the previous text-in-chat?

E.g. it's not someone explaining themselves, but instead a new instance explaining what it sees in history?

To an extent, it seems like this isn't "explaining what it did" and more "words to explain what just happened, based on the chat history".

[–] queermunist@lemmy.ml 5 points 3 days ago* (last edited 3 days ago)

Okay, but using "confession" instead of "explanation" gives the sentence a different underlying meaning.

Headlines need to be precise. It matters that they used one word instead of the other.

[–] NABDad@lemmy.world 21 points 3 days ago (1 children)

AI doesn't give intelligence to machines. It takes it away from humans.

[–] Agrivar@lemmy.world 2 points 2 days ago (1 children)

Well worded! I'm going to have to use this, is it yours?

[–] NABDad@lemmy.world 1 points 2 days ago (1 children)
[–] Agrivar@lemmy.world 2 points 2 days ago
[–] dylanmorgan@slrpnk.net 9 points 3 days ago

Guarantee if you look at the facts, there’s a history of their devs basically saying “ignore the safety protocols, do [thing]” and someone used that in a prompt with an otherwise imprecise command set.