Cybersecurity

Encrypted apps like Signal encrypt messages in a way that only you and the recipient can decrypt and read. Not even Signal can decrypt them. However it has always been the case that another person could look over your shoulder and read the messages you send, who you're sending them to, and so on. Pretty obvious, right?

What the author and Signal are calling out here is that all major commercial OSes are now building in features that "look over your shoulder." But it's worse than that because they also record every other device sensor's data.

Windows Recall is the easiest to understand. It is a tool build into windows (and enabled by default) that takes a screenshot a few times per second. This effectively capture a stream of everything you do while using windows; what you browse, who you chat with, the pron you watch, the games you play, where you travel, and who you travel with or near. If you use "private" message tools like Signal, they'll be able to see who you are messaging and read the conversations, just as if they were looking over your shoulder, permanently.

They claim that for an AI agent to serve you well, it needs to know everything it can about you. They also make dubious claims that they'll never use any of this against you, but they also acknowledge that they comply with court orders and government requests (to varying degrees). So... if you trust all of these companies and every government in the world, there's nothing to worry about.

~~He's~~ She's talking specifically about the idea of embedding AI agents in operating systems, and allowing them to interact with the OS on the user's behalf.

So if you think about something like Signal, the point is that as it leaves your device the message is encrypted, and only gets decrypted when it arrives on the device of the intended recipient. This should shut down most "Man in the middle" type of attacks. It's like writing your letters in code so that if the FBI opens them, they can't read any of it.

But when you add an AI agent in the OS, that's like dictating your letter to an FBI agent, and then encrypting it. Kind of makes the encryption part pointless.

"Agentic" LLMs are turning garbage operating systems, like Microslop Winblows, into hostile and untrusted environments where applications need to run. A primary example given is how Recall constantly captures your screen and turns the image data into text that can be processed by Microslop, thus making the fact that Signal is end-to-end encrypted largely irrelevant, since your OS is literally shoulder-surfing you at all times. This is made worse by the fact that the only workaround that application developers can use to defend against this surveillance is to implement OS DRM APIs, which are also controlled by the hostile entity.

Your operating system and half the software you use has integrated spyware that can read anything you see on your computer or phone as free text and use that information to notify state actors or just whoever the fuck they want of the contents. It doesn't matter that the message was encrypted between you and the other person when they can spy directly on your device.

Its like passing a coded note to a friend in class and then they open it and just read it out loud to everyone sitting there. Didn't really matter that you encoded it.

During the interview, she described how AI agents are marketed as helpful assistants but require sweeping permissions to work. As Whittaker explained, these systems are pitched as tools that can coordinate events or communicate on a user’s behalf, but to do so they must access calendars, browsers, payment methods, and private messaging apps like Signal, placing decrypted messages directly within reach of the operating system