218
I Asked AI to Count My Carbs 27,000 Times. It Couldn’t Give Me the Same Answer Twice.
(www.diabettech.com)
this post was submitted on 01 May 2026
218 points (93.6% liked)
Technology
84299 readers
3653 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Dude, they fail that exam with even worse error rates than I see!
When you can verify it, it's OFTEN and REGULARLY wrong. It's stupid to trust if for anything you can't personally verify.
The designed purpose of LLMs is to respond to human interaction, not to be correct. They are the showoff who pretends he can answer every question. They are the confident drunkard at the bar who will tell you anything that pops into their head. Intelligent, knowledgeable people say "I don't know" when they don't know. LLMs don't do that. Ever. Trouble is, they don't "know" anything. They're a chatbot from the bottom up. Chatbot through and through. It's their fundamental nature.
Yes there was knowledge and deep understanding in their training data. Also, I ate chicken curry for tea. However, I am not a chicken, I do not cluck, I haven't started eating worms, I cannot produce any chicken, and my poop is not chicken either. My poop smells faintly of curry. So it is with LLMs and the knowledge and understanding in their training data.
They beat any human on that knowledge benchmark, completely unrelated to your 40% "test". Try to answer any of the example questions on the main page.
I don't need a metaphor I know LLMs are hallucinating, lying, bullshitting. That doesn't invalidate my point.