BatmanAoD

Exactly: that's tight feedback loops. Agents are also capable of reading docs and source code prior to generating new function calls, so they benefit from both of the solutions that I said people benefit from.

As an even more obvious example: students who put wrong answers on tests are "hallucinating" by the definition we apply to LLMs.

making the same mistakes

This is key, and I feel like a lot of people arguing about "hallucinations" don't recognize it. Human memory is extremely fallible; we "hallucinate" wrong information all the time. If you've ever forgotten the name of a method, or whether that method even exists in the API you're using, and started typing it out to see if your autocompleter recognizes it, you've just "hallucinated" in the same way an LLM would. The solution isn't to require programmers to have perfect memory, but to have easily-searchable reference information (e.g. the ability to actually read or search through a class's method signatures) and tight feedback loops (e.g. the autocompleter and other LSP/IDE features).

This seems like it doesn't really answer OP's question, which is specifically about the practical uses or misuses of LLMs, not about whether the "I" in "AI" is really "intelligent" or not.

One list, two list, red list, blue list

(I genuinely thought that was where you were going with that for a line or two)

Agile Meridian / Post Manager

Thanks for sharing this! I really think that when people see LLM failures and say that such failures demonstrate how fundamentally different LLMs are from human cognition, they tend to overlook how humans actually do exhibit remarkably similar failures modes. Obviously dementia isn't really analogous to generating text while lacking the ability to "see" a rendering based on that text. But it's still pretty interesting that whatever feedback loops did get corrupted in these patients led to such a variety of failure modes.

As an example of what I'm talking about, I appreciated and generally agreed with this recent Octomind post, but I disagree with the list of problems that "wouldn’t trip up a human dev"; these are all things I've seen real humans do, or could imagine a human doing.

That is a pretty lame "poisoning".

This also makes me realize that I sometimes enunciate "the" unvoiced.

Well now you've seen it elsewhere, too.

distributing relay knowledge among chatters (TBD)

This is the core reason that centralization is currently necessary. So admitting that it's an unsolved problem for a federated alternative is basically reinforcing Signal's point.

That's because you haven't unlearned it yet