386
this post was submitted on 15 Sep 2026
386 points (98.0% liked)
Fuck AI
8217 readers
2156 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Say you had a version of chat GPT that hadn't been trained on a corpus that included enough information about the number of Rs the word "strawberry" contains, didn't have workarounds for character counting, and has context sharing.
LLMs, at their core, compute lexical & semantic similarity. When they "learn", they more effectively compute lexical similarity (really contextualized vector transformations, but let'snot get into that). Lexical similarity computation is not the same thing as learning concepts, like the concept of numbers, non-numerical things being ascribed numerical values (ironically, since LLMs function by rurning words into numerical vectors), or counting.
When it responds "Strawberry has 3 Rs" it isn't because it knows what that means or how to gain that knowledge about other words. It is merely parroting back what you've told it because "Strawberry has 3 Rs" in its shared context has very close lexical similarity to your query "How many Rs does Strawberry have?". It also parses Strawberry as ["straw", "berry"] encoded into numerical values representing their relationship in the corpus' vector space (e.g. {[ .420, -.67, .67], [.420, .69, -.69]}) - so unless it has instructions to further break those tokens into characters, it does not have the capability to count. You can actually do some weird math using these vectors and their relationships (their distance apart in the 3D vector space, angles between vectors, etc), but you lose granularity such as the number of letters in a token when you look at words this way.
Learning is more than just computing lexical similarity. Modern LLMs mimic reasoning by expanding the query with intermediary tokens, basically creating a temporary scratch pad of related words on the same vector, but don't actually reason (this is known as chain of thought). Learning is partly computing lexical similarity, but it is also about extrapolating concepts from facts and inferences, applying those concepts to novel simuli, etc. LLMs really don't do reasoning, which is part of learning.
Counterpoint:
“Teach” the LLM the proper procedure for counting the Rs and tell it to commit that to a file (or as a tool call). Then from there it will suceed 100% of the time.
It doesn’t matter if it’s true learning or not. When you teach an employee to do something you give them the procedure and have them do it. If AI can be fed a procedure too, it will do it just fine even if it’s not in its true training. Does it matter if it’s consuming the procedure as prompt context everytime? Hell a lot of humans do read the procedure every time when it’s complex enough.
What the example shows is that you cannot "teach" the llm how to count the letter R, because LLMs don't work that way.
The AI company would have to solve the problem another way, let's say by recognizing that the user is asking for letter-counting, and pass that off to a different kind of algorithm that can count letters.
Yes, and that’s called a harness, which everyone uses these days. The harness increases perceived intelligence (or accuracy) by absurd amounts. You can “teach” (or the equivalent of) LLMs anything with a custom harness.
How is it teaching? Sounds like programming to me.
Well the idea is you don’t need to code it yourself, you can have it do it for you. Sure it’ll have bugs the first few times, but humans also make mistakes the until they get the hang of it.
yes lets have the llm be a blurry frontend for a bunch of invisible proprietary harness programs. that surely is a valuable human endeavor
Because right now LLMs (and potentially other forms of AI) are the only technology capable of doing it. Humans can too, but are much slower and more expensive.
I would also like to add another key difference between human learning and machine learning. When I, as a person, read a book, I can conceptually understand the story. A machine would digest that same book by, for example, counting the numbers of times each word comes after another and assigning probabilities to occurrences or words or basic sentences. The major differences are still being explored. We don’t fully understand how humans learn, fundamentally. We know somewhat mechanisticly that neurons share and process chemical signals, but we don’t know how that information is interpreted or stored. We don’t know the equivalent of a computer language of the neurons (or other tissue types that may also learn). We know that we require input to learn, as do computers, the similarities kind of end there. That said, we definitely learn differently from how LLMs “learn.”