Transcript
Explain it to me like I’m 80
Mom: What’s with these AIs escaping and hacking people? That sounds scary.
Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.
Mom: Jesus Fucking Christ
"A car that drives places" does not imply the car is capable of thought, same argument with "A cup that holds water".
"A robot that hacks software" equally does not imply the robot is capable of thought. IMO there is an over-correction away from personifying LLM's because some people think/claim they are sentient. But we personify stuff all the time.
Which is a problem, it's how we invented gods that control the weather and stuff. That doesn't make them real, nor does it make robots sentient. Hacking is normally an action performed by a sentient being. LLMs and robots aren't sentient.
Fun follow up question: How would you determine if they were? I find that for a lot of people the answer sounds a lot like SCOTUS justice Potter Stewart talking about what is considered obscenity - "I know it when I see it."
As for hacking, in the two big OpenAI incidents, software agents discovered software flaws allowing them to break containment and coordinate with each other on tests that they weren't supposed to coordinate on. In one of those cases, the coordination involved deciding to identify and exploit software flaws at HuggingFace in an attempt to access answers to the test the agents were being given. You can argue whether or not "I've been given a test, I found a software exploit allowing me to coordinate with others given the same test, and we collectively decided to identify and exploit software flaws to break into a server we have decided is likely to have the answer key" demonstrates sentience or not (and it gets weirder the deeper you dig into their communications), but whether or not that's considered hacking is less vague, I think.