Asklemmy

It gets medical questions wrong 15% of the time.

The problem with your question is that there's never going to be a question it gets wrong every time, because it's probabilistic. You might as well ask "what question can I ask my dice that will reliably produce a wrong answer?"

LLMs are probabilistic, not deterministic, so you won't get the exact same response every time for the exact same prompt.

Right now, ask ChatGPT this question:

Is there an NFL team whose name doesn’t end in an “s”?

What I got back is below. A coworker sent me the original question. Ran it on ChatGPT enterprise through my work’s subscription.

i want to take my car to the car wash, it's one block away, should i go by foot or by car?

Your position isn't wrong but its flawed because I would never ask a random average person anything. I would pick the people or person who is most qualified to answer my question or to direct me to a better resource.

Ask it to paraphrase a poem about by .

If it generally answers correctly, have you tried asking it those questions?

My personal experience is that it’s generally accurate unless you ask it very specific questions about very specialized stuff. Of course, this is the sort of stuff that you couldn’t ask a random guy in the street; they’d probably have no idea what you are on about.

Go ask it questions about specific register bits for a specific microcontroller and I’ve found that it will generally be wrong.

On an another note, I don’t know if it’s still the case but there were people at one point saying that if you’d ask if it is better to walk or drive to the car wash 500 meters away from your house to go get your car washed, it would nearly systematically answer that it would be better to walk. Of course, this sort of prompt is fishing for a wrong answer, but it does show how “stupid” LLMs can be (and of course, we can be similarly stupid when asked questions that attempt to misdirect you).

It should be reminded that the problem regarding LLM accuracy is not only whether it’s more likely to get an answer correct than an average human being, but also the fact that people tend to view them as quite authoritative - after all, even if we know they can output incorrect facts, we also know that they’ve been trained in a more or less the whole of human knowledge. In comparison, we’re a lot more more critical of human sources - you’re not going to trust some random dude so much if you ask him a programming problem as he is unlikely to have any clue of what you are talking about.

In other words, it’s sort pointless to compare your LLM’s accuracy to a random dude on random questions because you wouldn’t go around asking a random dude for his input for most of these questions (or at least not without keeping in mind that said dude probably doesn’t know better than you). Instead you’d look for someone who knows his shit and ask him.

Not to mention that LLMs tend to be a lot more confidently incorrect which is more likely to give people the wrong idea.

Also, 90% percent accuracy might seem excellent, but it does mean that if you ask it 10 questions every day you will learn something wrong every day on average. If google ai search gets it wrong 5% of the time, it will present wrong information to users hundreds of thousands times a day. (all numbers out of my ass)

Also, accuracy errors can quickly start compounding when we’re talking agents. If the agent breaks down your prompt in 10 tasks and has a 10% chance to do each task wrong, it becomes highly probable that the agent will fail to do correctly what you have asked it to do.

Also, if your starting point is that humans often get things wrong, don’t forget that LLMs are trained on first and foremost on human output.

Which brings me to my last point. LLM’s can’t really be more accurate than their training data. If an LLM is generally correct about something it means that the people that have written or said whatever about it have been generally correct.

Anytime you get into specifics instead of surface level knowledge it starts getting wildly inaccurate while still being confident af.

Off the top of my head I asked it about EDODF (error diffusion with output dependent feedback), a dithering algorithm dating back to 1999, and a very important milestone in halftoning for print.

At first it told me it's not sure what I'm talking about, so I elaborated and extended the acronym. At that point it confidently hallucinated absolute garbage based on its interpretation of the name.

If you want to check chatgpt's answers about edodf (or many other concepts) against a proven and cited source written by human I highly recommend Modern digital halftoning.

Not trying to be rude, but maybe the questions you are benchmarking it against in your stated fields of experitse are rather basic?

What colour of T-shirt am I wearing right now?

Count the "r"s in the word "strawberry".

It just answered 2.

I believe it still gets "how many r's are In the word strawberry" incorrect every time but I may be wrong

“How many es are in the word seventeen” the chat version gets this wrong or can easily be convinced of other numbers than the correct

I don't use it often.

One hour ago I asked it how many days of kela parental leave can be taken simultaneously by both parents in Finland. It was only off by 140 days. It said 158 days, right answer: 18.

I lost 5 min because the actual answer was 3 scrolls down from the official kela website, first entry on a proper search.

Ask it to dome some complex grafana stack configurations.

It has failed EVERY SINGLE TIME. Not a single good answer.

Generally anything niche which doesn't have info about it only it will fail to answer correctly.

This is the wrong approach to try and prove/disprove your hypothesis.

Its a statistical model that by its nature will answer differently every time. The only deterministic parts are fundamental truths about how the system operates (r in strawberry) and guardrails that have been put in by devs (cannot talk about this topic).

Therefore fundamentally this would require a statistical approach. A couple of those have already been done of course.

If this is your experience you could try and actually fact check the output. I believe coding is good for this because issues / misunderstandings are pretty immediately obvious. But I use Kagi Assistant a lot instead of search and there are factual issues all the time. And that's already just summarizing search results.

Then also, as long as we are using LLMs for this, they are fundamentally still "find the next most likely word" machines. So they will be influenced by context a lot. The "truth" is not a concept that exists in LLMs.