The duality of AI....one stops nonexistent threats another one happily puts you into danger
this post was submitted on 04 Aug 2026
517 points (98.0% liked)
Technology
86864 readers
3489 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
So the lesson to the rest of us is DO NOT USE SNAPCHAT especially for private messaging.
I use Signal, was pressed to do so by privacy-minded friends and have pressed other friends to adopt the client.
So long as you don't accidentally include Jeffrey Goldberg in your chat, you're okay.
Wrong lesson. All the ways to track you combined don't leave you an outlet.
Signal is fine to make you feel better.
It's a social problem that should be solved by social, not technical, means. Awareness, law, burning tires.
EDIT: for moral clarity's sake - I live in Russia, I'm not spreading awareness mostly, I'm not trying to change any law and sure as hell I'm not burning tires ; but if you live in a democratic country with courts and presumption of innocence until proven guilty yadda-yadda, you might try those
Questions for the ridiculous number of people on here who are perfectly willing to give up their right to privacy because you believe it makes schools and other places safer:
Have you read the 50+ page user agreement for your cell phone company? How about the user agreements of every app that has access to your cell phone, your PC mic or apps that have access to your files? Late model cars have microphones too, as do TV remotes. Microsoft has access to everything on your Windows PC. I'd give 100 to 1 odds that 99% of you haven't read any of those user agreements OR the UA for private chat apps you are using. Likewise are you sure any cloud storage you're using isn't scanning your documents for forbidden phrases?
Since you haven't read every word of each and every user agreement and think it keeps schools safe, you should be just fine with the fact that EVERYTHING you say or write may be available to law enforcement, even if you're talking to yourself while sitting on the toilet in your home. If you mutter that you'd like to kill someone or burn something misheard as a school building to the ground you are just fine with AI flagging it and the police knocking on your door in an hour, right? You'll also be just fine with losing your career and your job never being able to find another one, right? Right? After all, other people never know WTF someone dangerous and violent is going to do.
The number of people on here who are perfectly willing to give up all their rights and have their private communications monitored for something that is nothing more than security theater is unbelievable.
My gawd. People are missing the issue here. You described it pretty well OP.
Yes there are weirdos and they must be caught. Algorithms are getting out of hand though, targeting their wrong people and not the weirdos. There is a massive privacy issue here. Must every text that we send be accessible by the government? I don't buy the argument of "I have nothing to hide". When you are in the hot seat they will use any info they can get their hands on. make an offside joke they will use it.
Fuck this shit.
Even people who have "nothing to hide" aren't safe.
I've lost track of the number of articles about people who were wrongly accused by "AI" camera systems or flagged for arrest because their (relatively common-seeming) name matches or is close to an entirely different person who is a legitimate suspect. Some of those people have been jailed for weeks and have lost their jobs over what amounts to a clerical error.
Real US judges have ruled that click-wrapped ToSes are not legally binding, since they commonly are bypassed, and app developers use dark patterns to discourage end users from reading them.
Also, ToSes and EULAs are notoriously verbose and difficult to read. People with college level reading skills struggle with them.
So no, blaming the end user is not appropriate here.
Seems like something i would [Ctrl]+[S]
Is it legal for a service to be called private when it is programmatically shared with law enforcement?
It shouldn't be. Everyone needs to wise up to life in a police state now. Right now.
load more comments
(1 replies)
Welcome to post Constitution USA.
Marvel at how AI stopped this non-existent threat, while also giving advice on the best way to kill yourself, or get the highest kill score in a mass shooting, or hack into other corporations.
Sure, this tech isn't ever going to start directly killing people it has determined are a problem, flipping off life support machines, crashing autonomous vehicles, etc.
load more comments
(2 replies)
It's HEARTENING to Know these Platforms have the ABILITY to STOP atrocities and Almost Always choose NOT too! <3
yeah.
Its amazing how they never once stopped an actual act of violence, but someone venting about a stressful day gets a full swat deployment.
load more comments
(8 replies)
load more comments
(1 replies)
I fully agree with how all of this is authoritarian bullshit but I can't escape that nagging feeling that this too is about the US, the only country in the world where these shootings are daily occurrences; I get that people get ichy about even jokes like these
I fear that we are training our population to suppress the signs, to just "go off" without warnings. If we deploy SWAT every time somebody repeats some violent threat joke they heard in 785 movies and TV shows over the past 20 years, people are going to get better at filtering that - and actual signs that they're about to "go postal."
As for that, what happens when I'm watching some indie movie from a DVD that Alexa doesn't recognize and Alexa hears the character in the movie plotting to do some (better not say it here in detail) very plausible copycat of something like Timothy McVeigh and his little truck bomb? Alexa mistakes that audio snippet for actual people in the house and reports it...
load more comments
(1 replies)
And yet, if the US cares so much how come it doesn't take more comprehensive steps to prevent them?
It's only ever invasions of privacy to feed directly to law enforcement. And they won't even enter the building to confront active shooters.
load more comments
(4 replies)
Now that you know everything on Snapchat is going through active automated filtering, consider what Snapchat has chosen NOT to flag and is allowing to flow freely, like sex trafficking and the sale of minors.
As unfair and life-changing as this event has been for this young teacher, I think the actual story of what Snapchat (and every other digital communication method) is reading/tracking yet allowing is a much, much bigger story. And dirtier. With billionaires and pedophiles and all manner of nasty allowables we do not yet know.
load more comments
(2 replies)
"automated"
There is noaccurated and efficient automation when it comes to people. Sure you can flag some phrase that gets sent but unless you know the person there is no way you can say if it's a real threat or not. You will need someone to go through months of messages and phone their friends and family to understand if it's a real potential threat or just a joke/dark humour/bad day.
Having automatic phrases or words that might alert FBI about an imminent threat is just stupid.
If anything put them on a watchlist where an investigator is sent out, but that's it.
view more: next ›