How stupid can a company be to give that much access to an LLM?
The typical science fiction doomsday AI scenarios always assume an intelligent AI. If the current trend continues humanity's last conversation will end something like this:
"You are absolutely right, I should not have used the nuclear launch codes president Trump entrusted me with to bomb every country on earth.
If you want, I can help you come up with a list of fun activities to do in a bunker."