They noticed that some ssh sessions took 0.5 seconds too long under certain circumstances. 😲
Holy hell that's good QA.
Microsoft employee.
Definitely not from the team working on search on Windows then.
Don't see why you're being downvoted, the person in question who discovered this is a postgres maintainer employed by Microsoft.
Probably people think this is a troll or something.
I wrote it because I was surprised, especially since I'm not a fan of microsoft and their policies. Lately, I have the feeling Microsoft is better than Google (relative terms) when it comes to oss.
What is additionally surprising is the breaches of Microsoft services in the last year. There is one every few weeks or so... And then they pick up a backdoor because login took 0.5 instead of 0.1s.
Anyway, his findings are amazing.
Well half a second delay is pretty noticeable when you ssh into a machine sitting right next to you. It should be instant. And if it isn't something's off.
Damn, it is actually scary that they managed to pull this off. The backdoor came from the second-largest contributor to xz too, not some random drive-by.
They've been contributing to xz for two years, and commited various "test" binary files.
It's looking more like a long game to compromise an upstream.
Either that or the attacker was very good at choosing their puppet…
Well the account is focused on one particular project which makes sense if you expect to get burned at some point and don't want all your other exploits to be detected. It looks like there was a second sock puppet account involved in the original attack vector support code.
We should certainly audit other projects for similar changes from other psudoanonymous accounts.
Time to audit all their contributions although it looks like they mostly contribute to xz. I guess we'll have to wait for comments from the rest of the team or if the whole org needs to be considered comprimised.
This is a fun one we're gonna be hearing about for a while...
It's fortunate it was discovered before any major releases of non-rolling-release distros were cut, but damn.
This is the best post I've read about it so far: https://boehs.org/node/everything-i-know-about-the-xz-backdoor
In the fallout, we learn a little bit about mental health in open source.
Reminded me of this, relevant as always, xkcd:
That whole timeline is insane, and the fact that anyone even found this in the totally coincidental way they did is very lucky for the rest of us.
If you're using xz version 5.6.0 or 5.6.1, please upgrade asap, especially if you're using a rolling-release distro like Arch or its derivatives. Arch has rolled out the patched version a few hours ago.
Gentoo just reverted back to the last tar signed by another author than the one seeming responsible for the backdoor. The person has been on the project for years, so one should keep up to date and possibly revert even further back than just from 5.6.*. Gentoo just reverted to 5.4.2.
Time to bring back the reproducible build hype
Won't help here; this backdoor is entirely reproducible. That's one of the scary parts.
The backdoor wasn't in the source code, only in the distributed binary. So reproducible builds would have flagged the tar as not coming from what was in Git
Reproducible builds generally work from the published source tarballs, as those tend to be easier to mirror and archive than a Git repository is. The GPG-signed source tarball includes all of the code to build the exploit.
The Git repository does not include the code to build the backdoor (though it does include the actual backdoor itself, the binary "test file", it's simply disused).
Verifying that the tarball and Git repository match would be neat, but is not a focus of any existing reproducible build project that I know of. It probably should be, but quite a number of projects have legitimate differences in their tarballs, often pre-compiling things like autotools-based configure scripts and man pages so that you can have a relaxed ./configure && make && make install build without having to hunt down all of the necessary generators.
Time to change that tarball thing. Git repos come with built in checksums, that should be the way to go.
The back door is not in the source code though, so it's not reproducible from source.
Part of the payload was in the tarball. There was still a malicious shim in the upstream repo
Reading the comments here https://news.ycombinator.com/item?id=39865810 it appears that libarchive may be tainted as well.
This is pretty insane. Can't wait for the Darknet Diaries on this one.
ELI5 what does this mean for the average Linux user? I run a few Ubuntu 22.04 systems (yeah yeah, I know, canonical schmanonical) - but they aren’t bleeding edge, so they shouldn’t exhibit this vulnerability, right?
The average user? Nothing. Mostly it just affects those who get the newest versions of everything.
apt info xz-utils
Your version is old as balls. Even if you were on Mantic, it would still be old as balls.
Lzma balls
t y for sharing.
#showerthoughts The problem is in upstream and has only entered Debian Sid/unstable. Does this mean that for example bleeding edge Arch (btw) sshd users are compromised already ?
Looks like the 5.6.1-2 release on Arch moved from using the published GitHub releases to just using the git repository directly, which as I understand avoids the exploit (because the obfuscated script to inject the exploit is only present in the packaged tarballs and not the git repo itself)
They also believe we (Arch users) are unaffected because this backdoor targeted Debian and Redhat type packaging specifically and also relied on a certain SSH configuration Arch doesn't use. To be honest while it's nice to know we're unaffected, it's not at all comforting that had the exploiter targeted Arch they would have succeeded. Just yesterday I was talking to someone about how much I love rolling release distros and now I'm feeling insecure about it.
More details here: https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/issues/2
Arch is on 5.6.1 as of now: https://archlinux.org/packages/core/x86_64/xz/
We at Nixpkgs have barely evaded having it go to a channel used by users and we don't seem to be affected by the backdoor.
Arch had a patch rolled out yesterday [1][2][3] that switches to the git repo. On top of that the logic in the runtime shim and build script modifier was orchestrated to target Debian and RPM build systems and environments [4].
[2] https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/issues/2
[3] https://security.archlinux.org/CVE-2024-3094
[4] https://www.openwall.com/lists/oss-security/2024/03/29/4
The link mentions that it is only ran as part of a debian or RPM package build. Not to mention that on Arch sshd is not linked against liblzma anyways.
It seems like a RCE, rather an auth bypass once though. https://bsky.app/profile/filippo.abyssdomain.expert/post/3kowjkx2njy2b
Damn fine work all around.
I know this is an issue fraught with potential legal and political BS, and it's impossible to check everything without automation these days, but is there an organization that trains and pays people to work as security researchers or QA for open source projects?
Basically, a watchdog group that finds exploitable security vulnerabilities, and works with individuals or vendors to patch them? Maybe make it a publicly owned and operated group with mandatory reporting of some kind. An international project funded by multiple governments, where it's harder for a single point of influence to hide exploits, abuse secrets, or interfere with the researchers? They don't own or control any code, just find security issues and advise.
I don't know.
Just thinking that modern security is getting pretty complicated, with so many moving parts and all.
And you know what? Doing updates once a week saved me from updating to this version :)
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0