this post was submitted on 07 Jun 2026
90 points (83.1% liked)
Technology
85242 readers
3807 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
What these articles never say is how many hallucinated bugs the LLM found that either weren't real or were actually exploitable. The LLM didn't find these with any confidence it highlighted areas of interest that actual security researchers then needed to investigate and confirm or rule out.
It literally wouldn't matter if it did.
The fact that it found exploitable bugs means that these bugs need to be addressed. To be clear, I care much more about the security flaws and fixing them than how they were discovered.
I feel like you missed the forest for the trees.
The question is how many were made up?
I saw that, and you're right, I wasn't answering that question. What I was saying was that I thought the question was irrelevant and ignoring a bigger issue.
I disagree that its ignoring the bigger problem, which is that slop like this is overwhelming devs to get fixes out ASAP faster than they can fix.
So now we have AI big reports feeding AI big fixes in a lot of projects.
The assumption that what AI finds is correct in the first place is.... Probably wrong.
It makes stuff up all the bloody time, so how many of these bugs were made up, or not actually bugs?
In the article it says the ffmpeg vulns were found by an "autonomous" agent and that it produced a proof-of-concept for each. So what do you base your claims on? They seem pretty contrary to that.
Even if there was still a lot of human work involved, it seems that the LLM-Agents can help a lot with security research, as the number of (real) zero-days that are beeing found recently (with the help of AI) seems to spike (telling from what I read, e.g. here on Lemmy, or the number of security updates for my distro).
It's states they were produced which I'm taking to mean found and it's ambigously worded so it's unclear if the article is actually claiming it generated PoC for all of them. It doesn't say how many if any hallucinated results were produced or how much effort was needed to fully confirm, basically down played the human involvement.
It's great that so many bugs are being found and squashed but it's implied LLMs are doing all the work when what they are actually doing is assisting as a tool.
I agree that the wording is a bit ambiguous, I interpreted it the way it seems more natural to me. In the post by the researcher(s) themselves, it says in the tldr paragraph that the "agent produces concrete, reproducible PoC inputs to confirm its findings" but also that they (probably humans) "explored the exploitability of the issues and developed a PoC demonstrating a RCE exploit primitive". Apparently it finds the vulnerabilities very concretely but humans were involved for the full-blown exploit. It also doesn't say much about the number of false-positives.
I'm not in the business, so I can't tell how much of the work such agents are actually saving. Since the articles don't say much about the amount of human involvement, the imagination conveyed by them probably depends strongly on the (knowledge of the) reader. But in my opinion it is a bit of stretch to say this is downplaying it. It should be noted though, that the article probably sources its information from a post by the company selling that AI.
With that information, the "without any confidence" and "area of interest" parts of your previous post still seem misleading.
In my experience, when given real world data to run on, they don't hallucinate that often. Its when you ask it to regurgitate stored info that its off by a wild amount sometimes. Fixing or comparing code is like AI 101, unlike code generation where it may be incorrect.
I strongly disagree. Every response longer than one line of code or longer than 1-2 sentences on a non-trivial task, has at least inaccuracies or mistakes. When working with something AI has created, I had to go in and fix it manually in 100% of those cases. That's why I limit the use of AI to only type-ahead suggestions, as I can easily verify them and don't waste more time than creating the result manually.
I'm frankly quite annoyed by the amount and extend of anti-AI hate in this community. It almost seems like this is a pure anti-AI rage community. The capabilities and the utility of LLMs are basically always denied or downplayed as much as possible without running into obvious contradictions.
It would be so nice to have some differentiated and insightful discussions here, about what can or cannot be done with AI, positive and negative impacts it has, possible new use cases, how AI should or should not be used, how the overall benefits of AI can be maximized and the overall negative effects minimized, what the world with AI could be like in the future, ...
I was trying to have some insightful discussion on the actual capability of LLM which is difficult when the involvement of the human element is played down amd the role of the LLM is played up to feed the hype machine. It's hard to acknowledge the real capabilities and weaknesses when the capabilities are always over reported and the weaknesses down played or denied.
It's great that so many bugs are getting discovered but as I say there is no reporting on what effort was needed to sift and review the LLM output or how functional or understandable any PoC were... The article doesn't directly even state the PoC were directly produced by the LLM and reads very ambigously.
I think some of that is because the reporting is focused on the new stuff, that was previously not possible. That human work is involved and some of the weaknesses are not really new. But also because the information in this case comes from a company that wants to sell their AI. I agree that the reporting is probably biased and not really sharp and therefore limited in usefulness.
Also, my (second) comment was not specifically about your comment but generally about the "vibe" of this community
Imagine this trend line increasing a bit more rapidly https://en.wikipedia.org/wiki/Carbon_dioxide_in_the_atmosphere_of_Earth
CO2 emissions are a huge problem, of course. But it is not specific to AI. Data centers are starting to become a significant factor of energy consumption but I think it will stay very manageable compared to other consumers and given the utility it provides. And since data centers luckily natively require electricity, it is much easier, compared to e.g. transportation, to switch them to renewables. And renewables are very often already the cheapest source of energy anyways. So I think AI is just another thing that humans do that requires energy, and it comes with the same tradeoffs (the utility vs. the cost of sourcing that energy). So in my opinion we should mainly focus on accelerating the transition to green energy.
Here's a good overview about AI carbon emissions I just found: https://www.carbonbrief.org/ai-five-charts-that-put-data-centre-energy-use-and-emissions-into-context/
I expected it to be worse. And the org seems trustworthy. Though that's a tough topic to trust any source, considering the amount of money involved.