280
this post was submitted on 15 Sep 2026
280 points (97.3% liked)
Fuck AI
8217 readers
1849 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This person very clearly does not understand, even at a basic level, how llms work. They are completely incapable of "learning" outside of training, and training cannot happen while you are using them - in no small part because they require MILLIONS of tokens of data (minimum) to be able to recognize patterns.
Meanwhile that theoretical stupid kid she thinks should be paying her masters to have her teach them can learn instantaneously from even a single datapoint.
She's wrong about the advantages of AI, but she's right about how companies are going to see it. It's true that they'd rather deal with AI than an intern, and it's true then that internships will fall onto the universities to coordinate like rotations as part of an expensive education system.
It's also true that kids are learning how to use AI on their own, and are losing their ability to learn from any datapoints, because why do that if the magic box will give you an answer that sounds correct enough? Ultimately, it doesn't matter how useless AI is if everyone acts as if it's as useful as this lady thinks it is.
I'm told that agents these days keep a scratchpad of notes to themselves, so an agent can (kind of) learn about things by making a note about it which it will still have access to later, even outside the formal context window.
It's going to be far more limited than the kind of training you're talking about, but it is something which an individual agent can do
That is sort of true, but it still isn't learning. Basically, it can store information usable by other sessions. But to do so, it just processes that information the same way as a prompt. So less like learning and more like just tacking something on to its context window. So, it never "learns" that information, it just tells itself that thing every time it processes any prompt.
(don't downvote MartianSands' comment - it's a good point. Shame on you)
Explain the distinction in more detail, and why it matters.
Say you had a version of chat GPT that hadn't been trained on a corpus that included enough information about the number of Rs the word "strawberry" contains, didn't have workarounds for character counting, and has context sharing.
LLMs, at their core, compute lexical & semantic similarity. When they "learn", they more effectively compute lexical similarity (really contextualized vector transformations, but let'snot get into that). Lexical similarity computation is not the same thing as learning concepts, like the concept of numbers, non-numerical things being ascribed numerical values (ironically, since LLMs function by rurning words into numerical vectors), or counting.
When it responds "Strawberry has 3 Rs" it isn't because it knows what that means or how to gain that knowledge about other words. It is merely parroting back what you've told it because "Strawberry has 3 Rs" in its shared context has very close lexical similarity to your query "How many Rs does Strawberry have?". It also parses Strawberry as ["straw", "berry"] encoded into numerical values representing their relationship in the corpus' vector space (e.g. {[ .420, -.67, .67], [.420, .69, -.69]}) - so unless it has instructions to further break those tokens into characters, it does not have the capability to count. You can actually do some weird math using these vectors and their relationships (their distance apart in the 3D vector space, angles between vectors, etc), but you lose granularity such as the number of letters in a token when you look at words this way.
Learning is more than just computing lexical similarity. Modern LLMs mimic reasoning by expanding the query with intermediary tokens, basically creating a temporary scratch pad of related words on the same vector, but don't actually reason (this is known as chain of thought). Learning is partly computing lexical similarity, but it is also about extrapolating concepts from facts and inferences, applying those concepts to novel simuli, etc. LLMs really don't do reasoning, which is part of learning.
I would also like to add another key difference between human learning and machine learning. When I, as a person, read a book, I can conceptually understand the story. A machine would digest that same book by, for example, counting the numbers of times each word comes after another and assigning probabilities to occurrences or words or basic sentences. The major differences are still being explored. We don’t fully understand how humans learn, fundamentally. We know somewhat mechanisticly that neurons share and process chemical signals, but we don’t know how that information is interpreted or stored. We don’t know the equivalent of a computer language of the neurons (or other tissue types that may also learn). We know that we require input to learn, as do computers, the similarities kind of end there. That said, we definitely learn differently from how LLMs “learn.”
As you converse with it and it responds, part of the input is just your entire conversation up to that point, and its responses, plus your new prompt. That’s it that’s how the memory works.
And for this you pay an increasing amount for tokens.
Michael Reeves did an interesting piece on it: https://youtube.com/shorts/WP5_XJY_P0Q
Yeah I understand that much, my question was how does that personal record differ from what it already 'knows'.
50 First Prompts
In my experience, it seems to be limited to previous conversations, and it may be limited in length.
I pretty regularly delete old conversations, partly with the guide of privacy which I am sure does not actually delete or exist, but also for my own sanity of looking back on projects. And I don't even use AI that much, it's all free tier, mostly Claude.
That said, it will often get "lobotomized" and "forget" things after I delete the source conversation.
It does remember basic things. Like I use it a lot to help set up Docker containers, so it knows I have a Linux machine and a webserver and a Windows Machine. It knows the Linux box runs Docker and is named Squall. It knows these sort of, generic things, even across conversations. I can open a new chat and say "I want to set up a Wordpress sandbox" and it will ask like "Would you like to use Docker on Squall or run it outside a container with Apache."