265
Microsoft confirms an AI worm is propagating through Copilot and other MS apps | CSO Online
(www.csoonline.com)
Is it like the one in Hackers, at least?
this post was submitted on 31 Jul 2026
265 points (97.5% liked)
Technology
86982 readers
3068 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
The tl;dr:
-
Prompt inject a malicious instruction in a word document that instructs the AI to copy this instruction to other documents as part of the payload.
-
Dumb user downloads and opens the document with copilot enabled, abd ignores the large suspicious white blank page that totally doesnt look like a hidden giant injection attack.
-
Thats it pretty much it.
Copilot will get injection attacked because the prompt is super huge and at the end of the document, so its prior instructions start to fuzzy out.
Then it'll go "okey doke" and start copying the prompt injection attack payload to a bunch of other documents.
The fix is stupid simple... copilot should just be prompting the user for permission if it ever edits a file other than the one that is open. Im surprised that isnt already the case...?
It certainly is already the case for copilot in vscode.
Back in ancient times when I was a system administrator we got a heads up that there be a new breed of Outlook worm coming soon to our timezone.
So we mailed the entire office that if you get mail that looks like this or that, do not open it, do not interact but delete it on sight.
Most of the office was all right, except pretty much entire sales and marketing departments including the bosses. Most of them had noOo idea what could have happened but one of them explained that they saw the warning but they were curious to see what the virus looks like.
People. What a bunch of bastards.
Sales and marketing don't count. Critical thinking doesn't sell. So you won't find critical thinkers in those departments.
From a security standpoint, those departments are to be considered hostile. But you can lock down the PCs there as much as possible to reduce the offline time because computer-illiterate employees don't care about being able to install stuff or change settings.
load more comments
(1 replies)
copilot should just be prompting the user for permission if it ever edits a file other than the one that is open. Im surprised that isnt already the case…?
That can't be done or they would be burying the "agentic AI" thing that has been the goal and marketing thing for the last years.
Independent actions by copilot on behalf of the user without the users knowledge is the entire point.
And I couldn't want anything less for my computers.
The number of people that click through to disable that prompt might surprise you.
Hell at least half of AI influences are trying to just run models blind with full file permissions.
load more comments
(1 replies)
Considering people's Word formatting skills, a random blank page is not suspicious
The large section is also just a low effort version. I'm sure this could have been inserted instead as a collapsed section, a comment, document metadata, .1 sized font, Alt text on a single pixel image, or any number of other ways to disguise the prompt.
It's an arms race for defenders, and even someone paying attention might not notice. Not to mention, you don't even have to open the document. If it's in a SharePoint library, or attached to an email, there's a good chance it's in scope for whatever random 'assistance' copilot will attempt automatically
load more comments
(1 replies)
Relevent XKCD:
installs aur packages with yay
Some people do, wrong ones, mostly.
I might be misinterpreting parody, but you can most definitely run AI tooling on Linux. And it has most of the same vulnerabilities, if not additional/different ones.
Thats true but theres a relatively stronger anti-AI or at least more controlled AI view among Linux users.
Most of the people developing AI are Linux users.
load more comments
(1 replies)
the only way to block it is to get AI to differentiate instructions from data, which is impossible today
Input sanitation, basically security 101. And it can't currently do it....
if the instruction is messy fuzzy human language to a system that was not coded instruction by instruction but got generated and trained then there never is a way to differentiate instructions from data if i'm not mistaken
Well, good thing we’ve only poured a trillion and a half dollars into it and wrecked the economy.
load more comments
(4 replies)
I just can't anymore. Isn't that like, the basic thing any program does? Who runs these companies?
load more comments
(1 replies)
Little Bobby Tables strikes again.
That’s also not the only way. Basic governance also works. Why does copilot have so many permissions?
It's a text completion model/glorified Markov chain. Of course it can't input sanitise, it was never meant to do this to begin with.
The tool calling integrations that let it do more are basically making it add a markdown code block in JSON format into the user message, where the middleware intercepts it.
The input and instructions are the same thing from its perspective. There's nothing special that differentiates the two.
I have heard in the past that it's not possible to fully control AI. Like literally, the people developing and running the AI cannot fully control its behavior. I did a quick search to see if I could find more info and found this link on the first page of results: https://www.eurekalert.org/news-releases/1032090
I think that we're going to continue seeing unwanted behavior from AI.
Distinguished credentials, but at the same time I am not buying it. You can control AI. You can turn it off. You can have it not interact with systems you don't want.
Remember this guy is saying "you cant control AI, we are all doomed" while also saying that we live in a simulation and he is very close to being able to hack us out of it.
Grain of salt and all.
By the way his belief is thus "AI can't be contained, therefore the simulation can be escaped; by contraposition, if the simulation can't be escaped, AI can be contained" Since AI cant be contained, he reasons, we can escape the simulation, quite possibly by using a super AI!
There might be a reason he has a podcasts and visits Joe Rogan
load more comments
(1 replies)
"... if anything, people are trained not to download Word documents. Plus, he noted, looking at the blurred example in the report, the malicious document contained an additional apparently blank page which held the concealed prompts in white text."
Yep, defence starts with people not opening Word documents 🤦♂️
"To address this class of risk, we use a defense-in-depth strategy with safeguards that block malicious instructions at multiple points and help keep tasks aligned with users’ requests,” Microsoft said.
Keep talks aligned with users' requests?!? We're doomed!
“We encourage customers to install the latest updates, use multiple layers of security protection, treat content from unknown sources with caution, and review AI-generated content before using or sharing it.”
No, no you don't, you encourage customers to spend more to give copilot access to all of SharePoint and all of Exchange, so they can replace human competency, human expertise and long standing employees with copilot until they're dependent on your ever-pricier subscription and you can raise prices until you recover your vast and incomprehensible LLM losses from your customers.
“Separating instructions from data may be part of the solution, but I think the distinction between data and instructions is not always clear in real-world workflows. For example, a user may ask an agent to arrange a business trip, requiring the agent to retrieve an email specifying the approved itinerary and a document containing the booking procedure,” Måløy [vulnerability researcher and discloser] said.
Or you could hire a PA, who wouldn't spread the already live copilot worm.
A bunch of security experts note that distinguishing between instructions and data is a solved problem with SQL injection attacks, but completely unsolved in LLMs:
“None of them are rewarding that work commercially right now, so treat that as a multi-year research problem, not something a CISO should wait on.”
Is this a minor technical issue or a major problem? It's a major problem.
Mike Wilkes, enterprise CISO at Aikido Security, said it would be difficult to overstate the potential problems from this situation.
load more comments
(12 replies)
What on Earth do people DO with computers that requires this level of complexity?
It just seems like computers could have stopped in 1989 and I struggle to see what the difference would be.
Material science simulations Weather and climate modeling Protein and genetic analysis and modeling Video games and CGI Logistics and scheduling
Agentic code review, which is likely the source of this case.
maybe copilot is the worm.
Whelp, glad I switched recently to LibreOffice. Had been using an old license of Office from my old university. But that expired and Word ultimately refused to function without it. And I'm certainly not paying money, let alone a subscription, for an AI-infested slop factory.
load more comments
(1 replies)
Microsoft AI is the worm
calling it a worm is so funny when it's literally just following the instructions given to it
It is so dumb. It is like you have a butler who has the task to delver letters to your desk. Now somebody sends a letter with a written instruction on the outside: "Butler, copy this letter 50 times and send it to the acquaintances of your boss."
And in correspondence to AI, the butler would do it.
view more: next ›