A community to post scientific articles, news, and civil discussion.
dart board;; science bs
rule #1: be kind
LLMs are spicy auto-correct. Anthropomorphizing them isn't going to suddenly change reality...
*auto-complete
Yeah, but it will make rich people even richer. So same difference.
Anthropomorphizing them isn’t going to suddenly change reality…
No more than thought terminating clishes do.
The burden of proof is very much on the people who say a computer is a person (and get paid for it).
Marketing does not care about the validity of its claims.
One activity occurs in a hazily understood chemical system with near completely unobservable states in interplay with activities and systems which have effects on the main system that are barely recognized, much less fully understood. The other is a constructed, completely defined system with near total observability of states affected by probabilities based on inputs defined by human written, completely known tokenization systems.
Will they next speculate about the similarity of 'preferences' between a robot from Japan and the nearest non-terrestrial intelligent species?
One activity occurs in a hazily understood chemical system with near completely unobservable states in interplay with activities and systems which have effects on the main system that are barely recognized, much less fully understood.
Yes, this is more or less the counter argument. If you are familiar with Anil Seth this seems to be his basis as well.
IMO it's basically arguing that we can't know something because it's really complex. Historically those problems do tend to get solved with tech and cleverness.
The other is a constructed, completely defined system with near total observability of states
Yes and no of course. Defined and observable, but profoundly difficult to interpret or understand.
I find it interesting that what known about mechanistic interpretability is partially the result of work done in using LLMs to interpret human brain scans -- scanning LLM layers to match active features.
You can look at a pair of meshed gears and understand it just as readily as you can understand a single transformation by running the math on paper with a pencil, but I don't think anyone will ever be able to comprehend the fullness of any useful-scale model for the same reason they would never comprehend a clockwork with the same number of gears as an LLM has transformers or indices. It's a non-intuitable space with too much complexity which cannot be simplified without oversimplifying.
We have no idea how consciousness works, and also we have no idea how LLMs work. So they must be the same!
we have no idea how LLMs work
Give me a break. LLMs are completely 100% understood. People developed weighted functions using very very basic statistics. There is absolutely no mystery here.
Consciousness gets mentioned the first time at 1 hour and 26 minutes into the episode. The discussion is about cognition - they're not the same thing and nobody is claiming that they are.
Your comment inspired me to actually listen to it and there's a lot more evidence for LLM-mindbrain convergence than I thought. Still seems like we invented something too complex to characterize directly so we're interrogating it the way we do our own cognition... I wonder what kind of biases this introduces.
The discussion is about cognition
Ah yes the foundation of "AI" research - retreating from obvious meanings to redefining words in order to make absurd claims.
I have no idea what you're even talking about. Neither of these people is an AI researcher and neither is making the claims you're here arguing against.
We have no idea how consciousness works
Misleading.
We have many very plausible, evidence-driven models of how consciousness works: global workspace, CTM (Conscious Turing Machine), IIT, ... the problem is figuring out which have the best predictive power. And that problem is being very actively worked.
and also we have no idea how LLMs work
Patently false. Look up "mechanistic interpretability", CLiP, etc.
There is very much a back and forth synergy right now between the communities of researchers interpreting neural network activations, and biological brain patterns.
We have many very plausible, evidence-driven models
"Plausible" counts for jack shit in science.
There are many models because there's no convincing evidence for any of them.
the problem is figuring out which have the best predictive power. And that problem is being very actively worked.
Yes, these models have no experimental or "predictive" use even after decades of "research".
Click bait nonsense based on what a psychiatrist feels. It's not even possible to make this determination because LLMs don't think and have no agency and we don't fully understand how humans think to compare against if they did.
Click bait nonsense based on what a psychiatrist feels
An interview with the Director of an Institute for Cognitive Science run by the University of Michigan. Who does not have a podcast, or anything to sell you, but has an honest opinion based on decades of research.
Something in this conversation is clearly very threatening to peoples' worldview and self-image. It evinces an emotional, visceral response.
Their areas of expertise and research are not areas that offer insight into mechanisms of cognition, they are not a neurobiologist or a computer scientist involved in machine learning. Their high standing in related areas is being used to give credence to an argument that is no better than a gut feeling.
You also have no ideas what conflicts of interest exist here, there may be none or the professor could be heavily invested in LLM companies.
LLMs dont think and the "trains of thought" is just strapping a bunch of LLMs together to try and tidy up and correct if one of the models in the chain outputs nonsense which they are wont to do. They are useful tools, they don't spontaneously generate output or request input. It's a very complicated auto complete that is capable of generating plausible valid output (when considered by humans) to a given natural language input.