this post was submitted on 28 Jun 2025
126 points (94.4% liked)

Technology

38710 readers
233 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.

Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.

Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 6 years ago
MODERATORS
MinutePhrase@lemmy.ml
126
We need to stop pretending AI is intelligent (theconversation.com)
submitted 2 days ago by technocrit@lemmy.dbzer0.com to c/technology@lemmy.ml
31 comments fedilink hide all child comments
 

We are constantly fed a version of AI that looks, sounds and acts suspiciously like us. It speaks in polished sentences, mimics emotions, expresses curiosity, claims to feel compassion, even dabbles in what it calls creativity.

But what we call AI today is nothing more than a statistical machine: a digital parrot regurgitating patterns mined from oceans of human data (the situation hasn’t changed much since it was discussed here five years ago). When it writes an answer to a question, it literally just guesses which letter and word will come next in a sequence – based on the data it’s been trained on.

This means AI has no understanding. No consciousness. No knowledge in any real, human sense. Just pure probability-driven, engineered brilliance — nothing more, and nothing less.

So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure. It doesn’t hunger, desire or fear. And because there is no cognition — not a shred — there’s a fundamental gap between the data it consumes (data born out of human feelings and experience) and what it can do with them.

Philosopher David Chalmers calls the mysterious mechanism underlying the relationship between our physical body and consciousness the “hard problem of consciousness”. Eminent scientists have recently hypothesised that consciousness actually emerges from the integration of internal, mental states with sensory representations (such as changes in heart rate, sweating and much more).

Given the paramount importance of the human senses and emotion for consciousness to “happen”, there is a profound and probably irreconcilable disconnect between general AI, the machine, and consciousness, a human phenomenon.

https://archive.ph/Fapar

top 31 comments
sorted by: hot top controversial new old
[–] tiny_mouse@lemmy.ml 1 points 9 hours ago

I think we need to stop pretending our world leaders are intelligent.

[–] Zaleramancer@beehaw.org 8 points 19 hours ago (1 children)

I'm not sure why so many people begin this argument on solid ground and then hurl themselves off into a void of semantics and assertions without any way of verification.

Saying, "Oh it's not intelligent because it doesn't have senses," shifts your argument to proving that's a prerequisite.

The problem is that LLM isn't made to do cognition. It's not made for analysis. It's made to generate coherent human speech. It's an incredible tool for doing that! Simply astounding, and an excellent example of the power of how a trained model can adapt to a task.

It's ridiculous that we managed to get a probabilistic software tool which generates natural language responses so well that we find it difficult to distinguish them from real human ones.

...but it's also an illusion with regards to consciousness and comprehension. An LLM can't understand things for the same reason your toaster can't heat up your can of soup. It's not for that, but it presents an excellent illusion of doing so. Companies that are making these tools benefit from the fact that we anthropomorphize things, allowing them to straight up lie about what their programs can do because it takes real work to prove they can't.

Average customers will engage with LLM as if it was a doing a Google search, reading the various articles and then summarizing them, even though it's actually just completing the prompt you provided. The proper way to respond to a question is an answer, so they always will unless a hard coded limit overrides that. There will never be a way to make a LLM that won't create fictitious answers to questions because they can't tell the difference between truth or fantasy. It's all just a part of their training data on how to respond to people.

I've gotten LLM to invent books, authors and citations when asking them to discuss historical topics with me. That's not a sign of awareness, it's proof that the model is doing what it's intended to do- which is the problem, because it is being marketed as something that could replace search engines and online research.

[–] QuizzaciousOtter@lemmy.dbzer0.com 2 points 16 hours ago

Excellent explanation. Saving this for discussions with friends.

[–] Zerush@lemmy.ml 2 points 20 hours ago

Always more we need to stop believing that humans are intelligent.

[–] PurpleCat@lemmy.ml 3 points 1 day ago

I don't really understand what the author is afraid of.

People sharing personal data they shouldn't? How is that an AI issue, we leave personal info all over the web. People asking emotional questions of a non thinking machine? Should we toss out our magic eight balls too? Obviously there should be safeguards around the kind of issues a chatbot can answer, but that seems unrelated to the belief/perception of lmm intelligence.

They talk about fictional harms if this technology were to progress, yet there no example of harm present with today?

They say the true fear should be of the corporation or government, what exactly should we be afraid of, and how would stripping the chatbot of it's affect safeguard us?

[–] folaht@lemmy.ml 0 points 19 hours ago* (last edited 19 hours ago) (1 children)

People are missing the point of the author here and the author is missing how far AI has come.

Msot AI today has no senses which is why it's so disjointed.
That doesn't make it a digital parrot without the bird part of the parrot,
but it does make it a superbrain in a vat.

AI in cars and drones however do have the intelligence of insects.

[–] Zaleramancer@beehaw.org 2 points 19 hours ago

Superbrain in a vat??

[–] queermunist@lemmy.ml 38 points 2 days ago (2 children)

So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure. It doesn’t hunger, desire or fear. And because there is no cognition — not a shred — there’s a fundamental gap between the data it consumes (data born out of human feelings and experience) and what it can do with them.

What? No.

Chatbots can't think because they literally aren't designed to think. If you somehow gave a chatbot a body it would be just as mindless because it's just a probability engine.

[–] adespoton@lemmy.ca 5 points 2 days ago (1 children)

Exactly. People see “AI” and think LLMs and diffusion models. Those are both probabilistic translation engines. They’re no more intelligent than an AC/DC converter, just a lot more complex.

However, there are neural networks and sense arrays in the field of AI, and those are designed to replicate the process of thought.

The real route to a thinking AI is likely a combination of the two, where a neural network can call on expert systems including translation engines to do the heavy lifting and then run a more nuanced decision tree over the results.

Thing is, modern LLMs and diffusion models are already more complex than a single human mind can fully comprehend, so we default to internally labelling them as either “like us” or “magic”, even when we theoretically know them to be nothing but really deep predictive models.

[–] Ferk@lemmy.ml 2 points 2 days ago* (last edited 2 days ago) (1 children)

They’re no more intelligent than an AC/DC converter

The problem is in the definition of intelligence.

To me, intelligence is simply problem-solving ability. It does not necessarily imply consciousness, having self-awareness or anything like that. A simple calculator is already displaying intelligence, even if limited to a very narrow situational set of problems, in the sense that it can resolve mathematical questions.

That doesn't mean the calculator is self aware.. it just means it can resolve problems. Biological systems can also resolve problems without necessarily being aware of what they are doing.. does the fungus actually knows it's solving a maze the scientists prepared for it when it just expands following what is preprogrammed by its biological instincts determined by natural selection? Do the ants really know what they are doing when they find the shortest path just by instinctively following a scent of pheromones left by other ants?

Knowing exactly what causes consciousness is an entirely different problem.. and it's one that has not been resolved by any scientist or philosopher in a satisfactory manner. So we simply do not know that.

[–] Mouselemming@sh.itjust.works 0 points 2 days ago* (last edited 2 days ago) (1 children)

Seems to me your definition of intelligence ignores whole aspects of true intelligence, at least of the human kind, such as emotional intelligence and social intelligence and artistic intelligence and moral intelligence...

"Problem solving" is the name for what you described and it doesn't necessarily require intelligence. In fact most intelligent people have encountered situations where it made solving a problem more difficult.

[–] Ferk@lemmy.ml 6 points 2 days ago* (last edited 2 days ago) (1 children)

Yes there there as many types of intelligence as there are types of problems. Emotional intelligence deals with emotional problems, social intelligence deals with social problems. This doesn't conflict with my definition, it's still problem solving.

Just because a being is intelligent does not mean it can solve all the problems of all kinds, it would require general intelligence, and even a generally intelligent being needs the right training... if you are trained wrong or trained for a different kind of problem that does not fit the current one then your current experience might actually get in the way, as you point out.

[–] queermunist@lemmy.ml 2 points 2 days ago (1 children)

Slime mold can solve mazes.

[–] Ferk@lemmy.ml 2 points 23 hours ago* (last edited 20 hours ago) (1 children)

Yes, that's what I meant 2 comments above by "fungus" (though to be fair, whether slime molds are fungi depends on your definition, they used to be classified as one, before "protist kingdom" was made up to mix protozoa, algae & molds, but I keep preferring the traditional autotroph / absorptive heterotroph / digestive heterotroph division).

I also mentioned ants who can find the optimal path by simply following scents left by other ants without understanding how this helps with that.

You can be intelligent without being aware of your intelligence, or you can be stupid without being aware of your stupidity... like how humans are actually creating problems for themselves in many cases.

Intelligence != awareness

[–] queermunist@lemmy.ml 2 points 19 hours ago (1 children)

If your definition of intelligence doesn't include awareness it's not very useful.

[–] Ferk@lemmy.ml 1 points 26 minutes ago* (last edited 4 minutes ago)

I don't know, I feel it's the opposite, if you make it include awareness then it becomes a subjective term since awareness is an experience, it's "qualia", a subjective quality that you cannot measure to really determine whether it's present. IQ ("intelligence" quotient) tests don't test conscious awareness, they test problem solving. Most of the time in science intelligence is defined as problem solving and capacity to adapt, extrapolate.

If you make it include awareness then you simply would not be able to answer the question: "is it intelligent?" ..the only answer would be "we don't know". This is the main problem of philosophy of the mind, what is often called "the hard problem of consciousness". Empirical analysis would not show if something is having (or not) the conscious experience of being aware.

[–] yogthos@lemmy.ml -1 points 2 days ago (2 children)

All the evidence suggests that our own minds are also nothing more than probability engines. The reason we consider humans to be intelligent is because our brains learn to model the events in the physical world that are fed into our brains by the nervous system. The whole purpose of a brain is to try and keep the body in a state of homeostasis. That's the basis for our volition. The brain gets data about about the state of the organism, and interprets it as hunger, pain, fear, and so on. Then it uses its internal world model to figure out actions that will put the body into a more desirable state. From this perspective, embodiment would indeed be a necessary component of human style intelligence.

While LLMs on their own are unlikely to provide a sufficient basis for a reasoning system, its not strictly impossible that a model trained on sensory data from a robot body it inhabits wouldn't be able to build a representation of the world and its body that could be used as the basis for decision making and volition.

[–] queermunist@lemmy.ml 3 points 2 days ago* (last edited 2 days ago) (1 children)

My understanding is that the reason LLMs struggle with solving math and logic problems is that those have certain answers, not probabilistic ones. That seems pretty fundamentally different from humans! In fact, we have a tendency to assign too much certainty to things which are actually probabilistic, which leads to its own reasoning errors. But we can also correctly identify actual truth, prove it through induction and deduction, and then hold onto that truth forever and use it to learn even more things.

We certainly do probabilistic reasoning, but we also do axiomatic reasoning i.e. more than probability engines.

[–] yogthos@lemmy.ml 0 points 2 days ago

I suspect that something like LLMs is part of our toolkit, but I agree that this can't be the whole picture. Ideas like neurosymbolic AI might be on the right track here. The idea here is to leverage LLMs at parsing and classifying noisy input data, which they're good at, then use a symbolic logic engine to operate on the classified data. Something along these lines is much more likely to produce genuine intelligence. We're still in very early stages of both understanding how the brain works and figuring out how to implement artificial reasoning.

[–] cecinestpasunbot@lemmy.ml 0 points 2 days ago (1 children)

All the evidence suggests that our own minds are also nothing more than probability engines.

This completely understates the gulf between what we call AI and how the human brain actually works. The difference is so severe that acting as if they’re quantitatively comparable is basically pseudoscience. You might as well start claiming that we’re not far off from building a Dyson sphere just because we invented solar panels.

Most “AI” these days are built using linear feed forward networks. The brain is constructed using nonlinear recurrent networks which are can do far more with less. Now you could theoretically create the same output from a linear feed forward network but it’s way less efficient and would require many more neurons to achieve such a result. Which is wild when you consider that there are orders of magnitude more synapses in just the regions of the brain associated with language than there are parameters used in even today’s most advanced “AI” models. Now consider that human synapses rely on over a hundred qualitatively different neurotransmitters and not just a single 16-bit number. It’s also not just the scale of the signal that transmits information in a human synapse but the pattern too. Would you be surprised to know that there are a whole variety of signaling patterns neurons use? Because that’s true too. I haven’t even gotten into the differences in complexity in terms of how neurons process the information they receive. As of now there is no “AI” system that comes anywhere close to replicating that kind of complexity. It’s absurd to suggest where dealing with qualitatively similar machines here.

[–] yogthos@lemmy.ml 1 points 2 days ago (1 children)

This completely understates the gulf between what we call AI and how the human brain actually works.

Way to completely misrepresent what I was actually saying. Nowhere was I suggesting that there isn't a huge difference between the two. What I pointed out is that, while undeniably more complex, our brains appear to work on similar principles.

My only point was that the feedback loop from embodiment creates the basis for volition, and that what we call intelligence is our ability to create internal models of the world that we use for decision making. So, this is likely a prerequisite for any artificial system that has any meaningful intelligence.

Maybe try engaging with that instead of writing a wall of text arguing with a straw man.

[–] cecinestpasunbot@lemmy.ml 0 points 2 days ago (1 children)

our brains appear to work on similar principles.

Sure in the same way that a horse and a motorcycle operate on similar principles and serve the same function.

Maybe try engaging with that instead of writing a wall of text arguing with a straw man.

Where the straw man? You’ve missed my point entirely. LLMs and the human mind operate on categorically different principles. All the verbiage used to describe neural network models has little to do with how the brain actually works. That’s honestly wasn’t a problem until Tech companies started purposely misusing those terms and now far too many people seem to think “AI” is something it’s not.

[–] yogthos@lemmy.ml -1 points 2 days ago (1 children)

LLMs and the human mind operate on categorically different principles.

A bold statement given that we don't actually understand how the brain operates exactly and what algorithms that would translate into.

Where the straw man?

The straw man is you continuing to argue against equating LLMs with the functioning of the brain, something I never said here.

All the verbiage used to describe neural network models has little to do with how the brain actually works.

You appear to be conflating the implementation details of how the brain works with the what it's doing in a semantic sense. There is zero evidence that all the complexity of the brain is inherent to the way our reasoning functions. Again, we don't have a full understanding of how the brain accomplishes tasks like reasoning. It may be a lot more complex than what LLMs do, or it may not be. We do not know.

Finally, none of this has anything to do with the point I was actually making which is regarding embodiment. You decided to ignore that to focus on braying about tech companies and LLMs instead.

[–] cecinestpasunbot@lemmy.ml -2 points 1 day ago (1 children)

The straw man is you continuing to argue against equating LLMs with the functioning of the brain, something I never said here.

I’m not claiming you ever said they functioned exactly the same way. Im simply stating that you’re way off base when you claim that they appear to operate using the same principles or that all evidence suggests the human mind is nothing more than a probability machine. That’s not a straw man. You literally said those things.

There is zero evidence that all the complexity of the brain is inherent to the way our reasoning functions.

You’re betraying your own ignorance about neuroscience. The complexity of the brain is absolutely linked with its ability to reason and we have plenty of evidence to show that. The evolutionary process does not just create needless complexity if there is a more efficient path.

Again, we don’t have a full understanding of how the brain accomplishes tasks like reasoning. It may be a lot more complex than what LLMs do, or it may not be. We do not know.

This is such a silly statement especially when you’ve been claiming that both the brain and AI appear to work using the same principles. If you truly believe the mind is such a mystery then stop making that claim.

You decided to ignore that to focus on braying about tech companies and LLMs instead.

I don’t really care about your arguments concerning embodiment because they’re so beside the point when you just blowing right by the most basic principles of neuroscience.

I bring up tech companies because they’ve had a massively distorting effect on how many computer scientists think the world works. You’re not immune to it either simply because you’re a critic of capitalism. A ruthless criticism of that exists includes the very researchers whose work you’re taking at face value.

[–] yogthos@lemmy.ml 1 points 1 day ago (1 children)

Im simply stating that you’re way off base when you claim that they appear to operate using the same principles or that all evidence suggests the human mind is nothing more than a probability machine.

I literally said these things, and you never gave any actual counter argument to either of them.

You’re betraying your own ignorance about neuroscience. The complexity of the brain is absolutely linked with its ability to reason and we have plenty of evidence to show that. The evolutionary process does not just create needless complexity if there is a more efficient path.

You're betraying your ignorance of how biology works and illustrating that you have absolutely no business debating this subject. Efficiency is not the primary fitness function for evolution, it's survivability. And that means having a lot of redundancy baked into the system. Here's a concrete example for you of just how much of the brain isn't actually essential for normal day to day function. https://www.rifters.com/crawl/?p=6116

This is such a silly statement especially when you’ve been claiming that both the brain and AI appear to work using the same principles.

There's nothing silly in stating that the underlying principles are similar, but we don't understand a lot of the mechanics of the brain. If you truly can't understand such basic things there's little point trying to have a meaningful discussion.

I don’t really care about your arguments concerning embodiment because they’re so beside the point when you just blowing right by the most basic principles of neuroscience.

That's literally the whole context for this thread, it just doesn't fit with the straw man you want to argue about.

A ruthless criticism of that exists includes the very researchers whose work you’re taking at face value.

Whose work am I taking at face value specifically? You're just spewing nonsense here without engaging with anything I'm saying.

[–] cecinestpasunbot@lemmy.ml -1 points 1 day ago (1 children)

You’re betraying your ignorance of how biology works and illustrating that you have absolutely no business debating this subject.

Have some humility and willingness to learn.

Efficiency is not the primary fitness function for evolution, it’s survivability.

I didn’t say it was the primary function. I guess all that talk about straw men was just projection. You don’t trust me, fine. Then what about Darwin who literally said, “Natural selection is continually trying to economize every part of the organization.” Now please go and read some introductory texts on biology before trying to explain to me why Darwin is wrong. There’s so much going on when it comes to the thermodynamics of living systems and you’re clearly not ready to have a conversation about it.

Here’s a concrete example for you of just how much of the brain isn’t actually essential for normal day to day function.

You’re baseless assuming that hydrocephalus causes the brain to lose a substantial amount of its complexity. Where is the evidence for that? In most of these cases it seems much of the outer layers of the cerebral cortex are in tact. It’s also really telling that your citation’s first source is an article titled “Is Your Brain Really Necessary” which is followed in the Journal by another article entitled “Math and Sex: Are Girls Born with Less Ability?”. But hey neuroscience hasn’t really advanced at all since 1980 right? The brain is totally redundant right? There’s no possible way a critical and discerning person such as yourself could have been taken in by junk science, right?!!

That’s literally the whole context for this thread, it just doesn’t fit with the straw man you want to argue about.

I took issue with specific statements you made that stand apart from the rest of your comment. That’s not a straw man. Although honestly this is on me. What can I expect from someone who thinks LLMs and the Human Brain are operating on similar principles? You’re so wound up in a pseudoscientific fiction there’s nothing I can do. You might as well start believing in the astrology, crystals, and energy healing. At least those interests will make you seem fun and quirky instead of just an over confident tech bro.

[–] yogthos@lemmy.ml 1 points 1 day ago

Have some humility and willingness to learn.

I have plenty of willingness to learn from people who have a clue on the subject.

I didn’t say it was the primary function.

You literally tried to argue that evolution doesn't create complexity if there's a more efficient path.th.

Then what about Darwin who literally said, “Natural selection is continually trying to economize every part of the organization.” Now please go and read some introductory texts on biology before trying to explain to me why Darwin is wrong. There’s so much going on when it comes to the thermodynamics of living systems and you’re clearly not ready to have a conversation about it.

Again, you're showing a superficial understanding of the subject here. Natural selection selects for overall fitness, and efficiency is only a small part of equation. For example, plants don't use the most efficient wavelength for producing energy, they use the one that's most reliably available. Similarly, living organisms have all kinds of redundancies that allow them to continue to function when they're damaged. Evolution optimizes for survival over efficiency.

You’re baseless assuming that hydrocephalus causes the brain to lose a substantial amount of its complexity.

Maybe read the actual paper linked there?

But hey neuroscience hasn’t really advanced at all since 1980 right? The brain is totally redundant right? There’s no possible way a critical and discerning person such as yourself could have been taken in by junk science, right?!!

What I linked you is a case study of an actual living person who was missing large parts of their brain and had a relatively normal life. But hey why focus on the actual facts when you can just write more word salad right?

I took issue with specific statements you made that stand apart from the rest of your comment.

You took issue with made up straw man arguments that you yourself made and have fuck all with what I actually said. Then you proceeded to demonstrate that you don't actually understand the subject you're debating. You might as well start believing in the astrology, crystals, and energy healing. At least those interests will make you seem fun and quirky instead of just a sad debate bro.

[–] belated_frog_pants@beehaw.org 9 points 2 days ago

"AI" is sparkling autocorrect. Its not smart. Its just manipulation of stats at scale.

[–] LandedGentry@lemmy.zip 11 points 2 days ago

It’s not pretend. People are getting caught in a collective delusion.

[–] some_guy@lemmy.sdf.org 2 points 2 days ago

People who don't like "AI" should check out the newsletter and / or podcast of Ed Zitron. He goes hard on the topic.

[–] Xaphanos@lemmy.world 2 points 2 days ago

What is your opinion of "Conscious" by Annika Harris?