this post was submitted on 10 Sep 2025
938 points (99.1% liked)
Fuck AI
4437 readers
1303 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Despite what OP and most of the comments here would have you believe, that is actually the crux of what was in OpenAI’s recent paper. They observed that most benchmarks and loss functions used for LLMs had a lower penalty overall for guessing than for admitting ignorance, and called for this to change across the industry.
I suppose answering "I don't know" to every prompt is at least more accurate than what we have now, but I don't think they'll want to risk that.
Of course. What the paper is suggesting is that during training and evaluation you should reward correct answers, punish wrong answers, and treat abstentions as somewhere in between. Current benchmarks punish abstentions and wrong answers equally, therefore models that guess instead of abstaining score higher on average.