this post was submitted on 04 Sep 2026
13 points (71.0% liked)

Books

8457 readers
14 users here now

A community for all things related to Books.

Rules

  1. Be Nice. No personal attacks or hate speech.
  2. No spam. All posts should be related to discussion or reviews related to books. (Please avoid posts that are just a link to the book and a generic summary)
  3. No self promotion.

Official Bingo Posts:

Related Communities

Community icon by IconsBox (from freepik.com)

founded 3 years ago
MODERATORS
 

I was recently reading Ben Aaronvitch’s ‘Rivers of London’ and I was struggling to follow the plot. So I engaged in discussions with both ChatGPT and Gemini and both really struggled to not make lots of mistakes. Both AIs would regularly give inaccurate info about major plot points, and when I questioned them about those issues, they would apologize profusely and then try to continue on. I basically had to coach them through to get them to remember important points of the plot! They also struggled to not reveal spoilers past a certain chapter. It was interesting to see the AIs repeatedly fail.

you are viewing a single comment's thread
view the rest of the comments
[–] e0qdk@reddthat.com 7 points 1 day ago (1 children)

LLMs don't really know the subject matter they talk about in the same way a person does. Whatever you feed as input biases the responses towards and away from certain sequences of words. If you put in something about Star Wars, it'll weigh its output in favor of saying things like "Luke Skywalker" and "Darth Vader" and away from things like "The best way to use a crockpot is...". When you're trying to get it to recall facts about things that aren't extremely famous, it will jumble some parts or just make things up that sound plausible.

As an example, I liked to prompt them with "Tell me about 'A Valiant Effort (1989)'" as a test -- it doesn't actually exist, but they'd often tell me it was a painting or a movie or an obscure book with hallucinated details until very recently; newer models seem more likely to say "I don't know what that is" or similar, but that's not guaranteed.

To get better results, try feeding it a passage you'd like it to work on directly in the input instead of assuming it just knows the story already, and you'll get output weighted by the actual text (and whatever else you put in context).

[–] MooseInMaine@lemmy.world 2 points 1 day ago

That's a funny way to test it with a fake movie!