this post was submitted on 19 Sep 2026
124 points (89.2% liked)

World News

58132 readers
1068 users here now

A community for discussing events around the World

Rules:

Similarly, if you see posts along these lines, do not engage. Report them, block them, and live a happier life than they do. We see too many slapfights that boil down to "Mom! He's bugging me!" and "I'm not touching you!" Going forward, slapfights will result in removed comments and temp bans to cool off.

We ask that the users report any comment or post that violate the rules, to use critical thinking when reading, posting or commenting. Users that post off-topic spam, advocate violence, have multiple comments or posts removed, weaponize reports or violate the code of conduct will be banned.

All posts and comments will be reviewed on a case-by-case basis. This means that some content that violates the rules may be allowed, while other content that does not violate the rules may be removed. The moderators retain the right to remove any content and ban users.


Lemmy World Partners

News !news@lemmy.world

Politics !politics@lemmy.world

World Politics !globalpolitics@lemmy.world

Ask Historians !askhistorians@lemmy.world


Recommendations

For Firefox users, there is media bias / propaganda / fact check plugin.

https://addons.mozilla.org/en-US/firefox/addon/media-bias-fact-check/

founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] automattable@lemmy.world 18 points 3 days ago (3 children)

I don’t know about Gemini because I haven’t used it, but the other frontier models are absolutely smart enough to do some damage. If your only exposure to them is playing with the chat feature and asking it how many rs are in strawberry, you’re getting a skewed impression, IMO, and I’m not surprised you think they’re dumb.

The best models are getting scary good at programming. Not to the point of entirely replacing humans, but to the point where I don’t think software engineering will ever go back to being done by “hand.”

I think it comes down to the fact that programming tasks are inherently testable and verifiable in a way that most other tasks simply aren’t. The AI can write and run tests that directly tell it if it needs to adjust its approach. You don’t have that with prose or other less quantifiable tasks.

Sorry, I’m rambling a bit here, but my point is that hacking is wicked close to programming in skill set and in that you get really quick feedback if your approach is good or not. So, when you have an AI system that can just keep trying things essentially indefinitely, it’s scary good at compromising other systems and can do way more than the funny demos of LLMs confidently asserting very obviously wrong facts will make you believe.

[–] Catoblepas@lemmy.blahaj.zone 41 points 3 days ago (1 children)

The “AI” (LLMs) are not sentient beings deciding to do this on their own with no instruction, which is what they’re hyping this as. You’re never going to get an LLM just sitting somewhere that suddenly decides to start hacking.

This is fucking idiots letting an LLM give itself instructions endlessly with hat on top of a hat technology (sorry, “agents”), after initial instructions telling it to carry out some malicious action, and left it misconfigured with internet access and access to other programs. It’s never not that, and somehow they are trying to spin this as evidence of the incredible power of their LLMs.

[–] automattable@lemmy.world 24 points 3 days ago

Oh completely agree. My original post was implying Google allowed this to happen on purpose because they wanted the headline. Sorry if that wasn’t clear up front.

[–] Zorque@lemmy.world 8 points 3 days ago

I didn't say they were too stupid to do damage, only that they were too stupid to decide to do damage all on their own.

[–] rozodru@piefed.world 3 points 3 days ago (1 children)

I use pretty much every AI because of my job and Gemini is one of the absolute worst. Hell Lumio is better than Gemini.

this past week a client of mine was using it to build something and this idiotic AI somehow, someway, just decided that it had achieved it's goal in the build without actually doing anything. and I'm not talking just writing in comments or TODOs like Claude does but actually go through the files and say "yup, It works now, I did that here's what I changed" and then proceeds to spit out nothing as for what it changed. it REFUSED to acknowledge that it didn't actually do anything when prompted. and this isn't a one time thing with Gemini, this is common.

I refuse to believe that Gemini hacked anything. I just don't see that ever happening. Google have created one of the absolute dumbest AIs on the market. I firmly believe some engineers at Google did the hacking with some assistance from Gemini but it didn't do it on it's own.

[–] QuinnyCoded@sh.itjust.works 1 points 3 days ago

you don't have to be too smart to do a lot of damage on the Internet

Google said in a statement that in May its AI model gained unauthorized access to three outside systems during a test by either guessing login information or using login credentials it found in a public repository.