this post was submitted on 09 Jul 2026
384 points (98.0% liked)

Fuck AI

7597 readers
1111 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
 

In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.

Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.

Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”

Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”

top 50 comments
sorted by: hot top controversial new old
[–] cyclonedusk@lemmy.blahaj.zone 2 points 6 hours ago

... i hope he never had the chance or ability to actually do anything to her physically ._.

i hope learning about what someone did without her knowledge will not scar her for life...

[–] zarkanian@sh.itjust.works 20 points 10 hours ago* (last edited 10 hours ago)

xAI founder Elon Musk has denied that Grok has ever been used to generate child sex images.

Well, that's a relief! Of course they wouldn't do something like that. Because that would be super illegal!

[–] electric_nan@lemmy.ml 1 points 7 hours ago

What a fucking headline. At least it had a happy ending.

[–] WorldsDumbestMan@lemmy.today 14 points 14 hours ago

They are making their pedo dream land, with no laws or rules.

This is their so-called effective altruism.

We all know what to do.

[–] 87Six@lemmy.zip 18 points 15 hours ago (2 children)

How do you even generate ONE of those images and NOT immediately go to prison?

"FoR thE chIlDrEn" when privacy is on the line but they can't put a weirdo in prison after generating SEVEN THOUSAN CSAM images?!

[–] Thebeardedsinglemalt@lemmy.world 6 points 8 hours ago (2 children)

"Think of the children"

And this is what happens when they do that a little too much

[–] cyclonedusk@lemmy.blahaj.zone 3 points 6 hours ago

it's ironic how all the people barking "think of the children" turn out to be the people we absolutely DO NOT want thinking about children, ACTUALLY.

I think they think its the normal amount.

[–] matlag@sh.itjust.works 12 points 10 hours ago (1 children)

Because it's never ever about the children.

[–] cecinestpasunecommunication@lemmy.dbzer0.com 1 points 6 hours ago* (last edited 6 hours ago)

Totally not about the children, says robust discourse and literal thousands of images all about the children .

[–] heartSagan5@lemmy.zip 7 points 14 hours ago (2 children)

So, if xAI did that, then did they get obstruction charges?

Why? This is just about a megapedophile, it's not like anybody camped out on a lawn or asked Lockheed to stop selling f35s to baby murderers.

[–] Lemminary@lemmy.world 4 points 10 hours ago

You mean a slap on the wrist for the richest man in the world?

[–] brucethemoose@lemmy.world 22 points 20 hours ago* (last edited 20 hours ago) (3 children)

Horrors of this aside… is Grok using a keyword filter for CSAM detection?

I cannot even begin to describe how technically negligent that is.

Image based CSAM detectors have been around for ~2 decades, and any image host with an ounce of self preservation uses them. I’m pretty use some Lemmy instances use them. Twitter must have already been using them, even before Musk bought them.

That they wouldn’t implement a checker in Grok is mind-boggling.

[–] prole@lemmy.blahaj.zone 4 points 8 hours ago

is Grok using a keyword filter for CSAM detection?

At some point in the last ten years, everyone apparently forgot why this doesn't work

[–] mnemonicmonkeys@sh.itjust.works 16 points 20 hours ago (1 children)

I cannot even begin to describe how technically negligent that is.

Tbh, this is par for the course for most of Elon's businesses

[–] brucethemoose@lemmy.world 10 points 20 hours ago* (last edited 20 hours ago) (2 children)

But even unintentionally serving CSAM is unique in that it can get you in legal hot water reeeal quick. It’s why web services are so strict about filtering.

I just can’t wrap my head around this. Maybe I’m misreading what Grok did, or maybe the story isn’t quite right, but if it is, the company should be getting raked over hot coals by the feds right about now.

[–] zarkanian@sh.itjust.works 5 points 11 hours ago (1 children)

Yeah, but remember who's running the FBI right now.

[–] matlag@sh.itjust.works 4 points 10 hours ago

More likely the Epstein class will be offered given "premium private" accounts.

[–] core@leminal.space 16 points 20 hours ago

xAI did more to protect a pedophile than it does to protect its regular users from law enforcement.

[–] dumnezero@piefed.social 4 points 16 hours ago
[–] Sam_Bass@lemmy.world 26 points 1 day ago (2 children)

when the 'leader' of the country is a known csam perpetrator, i'm sure it emboldens those members of the public with similar mental leanings

[–] ZILtoid1991@lemmy.world 8 points 20 hours ago (1 children)

Techbros in the past were infamous of having a "free speech absolutist" view, and saw CSAM as "recordings of crimes". While this view fell out of mainstream (alongside with nonconsensual nudes being more ostracized), it wasn't without a lot of opposition, and an almost irreparable harm to free speech advocacy.

[–] ratrace@lemmy.zip 1 points 2 hours ago

here here .... I concur

[–] BeMoreCareful@lemmy.world 6 points 21 hours ago* (last edited 20 hours ago)

Yeah, this guy sort of digitally pimped his wife's kid.

Which is a lot.

I'm thinking this has to be one of the most anti social behaviors.

However, there's also, this Grok LLM. Which, tbh has said some shady shit on the internet and was abetting(?!).

That's somehow more than a lot.

[–] Avicenna@programming.dev 9 points 1 day ago

Well that is what happens when you train an AI on the likeness of Elon

load more comments
view more: next ›