Transcript
Explain it to me like I’m 80
Mom: What’s with these AIs escaping and hacking people? That sounds scary.
Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.
Mom: Jesus Fucking Christ
Can you though? You definitely can't know that living things with brains aren't to some significant degree responding to stimulus in a probabilistic way, and in the case of humans at least turning that response back on itself - essentially responding to itself. We do know for certain that the brain does a whole lot of that in variety of situations - it's why optical illusions work, for example. It's one of the brain's favorite magic tricks - generating answers in a probabilistic way if there's insufficient information, or sometimes if the information is inconvenient.