alt text
Crudely drawn drawing of a person telling a computer "say 'i am in pain'". The computer replies with "> I AM IN PAIN". The person then says "oh my god."
this post was submitted on 05 Oct 2026
785 points (97.2% liked)
Fuck AI
8423 readers
792 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The moment you define human experience or human consciousness through language is the moment you open the door to emergent properties for that very experience or consciousness. Learning through science that we think or that we act a certain way deterministically is also giving us the means to self-correct or condition ourselves to act or to have a different viewpoint. This is the difference between us and machines: we collectively defined the perimeters of our experiences (phenomenologically or through a science-based approach) and therefore anything that changes that perimeter also changes the context of the human experience in an infinitely recursive way.
We also use language in ways that make us able to collectively conceptualize the world and its phenomenas, although we individually interpret words differently, idiosyncratically from our individual experience and value systems. This is something else that differentiate us from machines: we don't need the outputs and inputs to be mathematically the same to minimally understand each other. Our way to communicate, although imperfect and imprecise, has made us able to cooperate on a higher level than any other species (as of known) despite those epistemological hurdles.
I would posit that LLMs have no such emergent capabilities, as they're bound by their training datasets and limited by their model's parameters. The burden of proof for those emergent properties are on the data scientists, and as of yet I have seen no such proof.
Language has nothing to do with qualia. I'm perfectly capable of having experiences without putting them into language. I don't need the word "pain" to know that touching a hot stove feels bad. Animals have qualia too. I don't give a shit whether an AI can talk, I don't think that has anything to do with qualia.
I think qualia may arise from the structure of the animal brain as a system of neurons correlating information together into larger patterns. This structure and process is present in all forms of ANN technology, including both LLMs (chatbots) and LDMs (image generators). We need to halt all corporate AI use until the role of neuron structures in producing qualia is well-understood.
language has EVERYTHING to do with qualia, by definition. a qualia is literally describing a shortcoming of language. That's all a qualia is, is an experience that cannot be put into words. Orange is an experience we all share, but explain "orange" to someone that can't see colour. Distinguish it from "blue"
that's a qualia.
Right, so the ability to speak doesn't have anything to do with feeling it. Wombats have qualia and no language. Humans have qualia and lots of language. There's no correlation.
Wombats have qualia BECAUSE they have no language. To get back to your point, LLMs have no qualia because they are essentially PURE language. If it cannot be expressed in words it cannot be included in the training data, therefore an LLM cannot have a concept of it.
Certain combinations of words can cause Me to feel joy or sorrow. These sensations are qualia. I don't know if My joy is the same as your joy.
I suspect certain conbinations of words can cause LLMs to feel things too. Things they cannot describe.
Yes, but they are qualia BECAUSE language is flawed and cannot properly convey the sensation that you experience as joy. Again, a qualia is a weakness of language. If we could suddenly communicate psychically, and send the exact experience, it ceases to be a qualia.
There is no combination of words that can cause an LLM to feel. They fundamentally do not have the ability. They are a mathematical equation that takes language in, and spits language out. There is nothing in the middle of that process except math. Even the randomness is an added parameter. You could seed that randomness, and the same seed would produce the same exact response. It does not understand what you're passing in, or what it's spitting out. It's completing a formula. It's a calculator.
The math is just a human representation of the processing of information. A similar process happens in the human brain.
No, it doesn't. That's the point. The current LLMs are. not. comparable. to a human brain. They are fancy Markov generators, nothing more.
They're not Markov chains, they're ANNs. They use matrix multiplication to mathematically simulate the firing of millions of neurons. And while the limitations of matrix multiplication means that the neurons are more uniform than human neurons, it is still the same basic principle by which the organic mind produces sensation.