I like this take - I read the refutation in the replies and I get that point, but consciousness as an illusion to rationalize stimulus response makes a lot of sense - especially because the reach of consciousness's control is much more limited than it thinks it is. Literally copium.

When I was a teenager I read an Appleseed manga and it mentioned a tenet of Buddhism that I'll never forget - though I've forgotten the name of the idea (and I've never heard anyone mention it in any other context, and while I'm not a Buddhist scholar, I have read a decent amount of Buddhist stuff)

There's some concept in Japanese Buddhism that says that, while reality may be an illusion, the fact that we can agree on it, means that we can at least call it "real"

(Aka Japanese Buddhist describes copium)

Well, if you experience consciousness, that's what consciousness is. As in, the word and concept "consciousness" means being conscious, the way you experience being conscious right now (unless of course you're unconscious as I write this...). Free will does not enter into it at the basic level, nothing says you're not conscious if you do not have free will. So what would it really mean to say consciousness is an illusion? Who and what is having the illusion? Ironically, your statement assumes the existence of a higher form of consciousness that is not illusory (which may very well exist but how would we ever know?). Simply because a fake something presupposes a real something that the fake thing is not.

So let's say we could be certain that consciousness purely is the product of material processes in the brain. You still experience consciousness, that does not make it illusory. Perhaps this seems like I'm arguing semantics, but the important takeaway is rather that these kinds of arguments invariably fall apart under scrutiny. Consciousness is actually the only thing we can be absolutely certain exists; in this, Descartes was right.

So, it's meaningful to say that a language model could "fake" consciousness - trick us into believing it is an "experiencing entity" (or whatever your definition would be) by giving convincing answers in a conversation - but not really meaningful to say that actual conscious beings somehow fake consciousness. Or, that "their brains" (somehow suddenly acting apart from the entity) trick them.