this post was submitted on 22 Oct 2026
128 points (99.2% liked)

Linux

68239 readers
572 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 7 years ago
MODERATORS
nooter692@lemmy.ml
AgreeableLandscape@lemmy.ml
MarcellusDrum@lemmy.ml
cypherpunks@lemmy.ml
cyclohexane@lemmy.ml
d3Xt3r@lemmy.nz
128
Red Hat Funded a Report That Estimates 25% of PRs Are AI Slop (itsfoss.com)
submitted 1 day ago by SocialistVibes01@lemmy.ml to c/linux@lemmy.ml
18 comments fedilink hide all child comments
 

LF Research is the Linux Foundation's research arm. It was set up in 2021 and publishes studies on how open source is built and used, drawing on surveys and project data.

Its latest report, Navigating Open Source Software Development in the Generative AI Era, surveyed 234 contributors and maintainers on how generative AI is affecting the way open source gets made.

The survey estimates that about 25% of incoming pull requests are low-quality AI slop, with the estimate climbing to 31% on larger projects.

Red Hat sponsored the report, with additional contributions from OpenSSF, LF AI & Data, and Futurewei.

The findings

Large projects, the ones with more than 50 contributors, are affected the most.

Among their maintainers, 69% report more incoming contributions, against 59% across all maintainers, and 64% say reviews now take longer. Burnout is their top concern at 63%, narrowly ahead of contribution quality at 60%.

You will also find that policies have not caught up to the AI-assisted coding and vulnerability reporting trend.

Of the 119 respondents who answered the question, 35% said their project has a formal policy permitting AI use. Another 32% have none and no plans for one, 23% plan to write one, and 10% ban AI contributions outright.

You might remember that some projects, like LLVM, allow AI-generated code with conditions. Separately, 26% of projects have no way of verifying AI-assisted contributions at all.

The report calls on contributors to give effort to written policies, disclosure of AI use, and testing and documentation.

Interestingly, AI does seem to save developers' time. Respondents estimate that code authoring takes 52% less time on average, while on more complex tasks, 21% said the work took longer with AI.

Most maintainers (71%) say AI makes it easier for newcomers to start contributing. Just over half (53%) say it has not improved community trust, while only 13% say it has.

Some caveats do apply.

Respondents were recruited through Linux Foundation newsletters and social channels, plus partner channels that include Red Hat. As for the 25% slop figure, it is an estimate from 111 respondents, not a measured count.

That sounds fair, but disclosure is mostly left to individual choice right now, and a quarter of projects have no way to check anything. It only works if the people sending slop make an effort to do it.

And as we have already seen time and again, the ones sending slop rarely bother.

top 18 comments
sorted by: hot top controversial new old
[–] CameronDev@programming.dev 14 points 23 hours ago (1 children)

I'm curious to know what the quality rate was like before AI. Anecdotally, my marginally popular repo got a few PRs, but many of them were pretty much garbage as well. No tests, broken tests, half baked code. Human written didnt mean good

[–] stsquad@lemmy.ml 11 points 15 hours ago (1 children)

Slop has existed before AI but it has lowered the barrier to generating it. Having no AI policies isn't really working because the slop posters don't declare it and as a reviewer you don't want to be second guessing a potential contributor.

We are probably going to move to a policy that allows narrow use of AI tools but emphasises human understanding and ownership as mandatory components of contribution.

[–] CameronDev@programming.dev 1 points 13 hours ago

I'm lucky, so far no AI slop PRs yet, so I'm in a different boat. I'm happy to keep my little thing going and hope that it slowly dies away with no further attention.

emphasises human understanding and ownership as mandatory components of contribution.

That really the best you can hope for, although I dunno how you can force ownership. Its not easy to hunt down the original developer if there is a bug in their code.

[–] dil@lemmy.zip 5 points 19 hours ago (3 children)

Are they slop or bugs getting fixed faster that would otherwise beignroed because they aren't that bad and known

[–] communism@lemmy.ml 9 points 12 hours ago

They are slop according to the maintainers, which I would say should be the arbiter of that.

[–] lengau@midwest.social 2 points 11 hours ago

I maintain several small open source projects. I sometimes use AI for my own PRs, especially ones that are almost mechanical but not quite. A great use for AI is "fix the new listing issues from this litter update."

I've been getting a moderate amount of AI slop PRs recently, though it has also coincided with a similar increase in PRs that are probably written by (or an least partially written by) AI but are actually useful. The ones I'd call "AI slop" tend to be huge, change a lot of tests, and may or may not even achieve their goal. A recent AI-written PR I got from an external first-time contributor fixed a bug by refactoring some code into a function, ending up with 4 calls to that function, of which 3 of the previous implementations had been correct and one had missed a step back in 2021. Net code added? 1 line, and only because it added a test.

The main problem is how much time I end up spending on these slop PRs. It can take a while to even describe why you're rejecting the change, though ironically AI has made this easier too.

[–] alottafagina@lemmy.ml 4 points 17 hours ago

It looks like it was more of a survey and afaict, participants weren't given an opportunity to distinguish between "AI slop" and high quality AI assisted contributions (so were all AI contributions reported as "slop"?!)

To answer your other questions, this is the relevant part:

Across the board, genAI reduces estimated task time across all six measured activities in the pooled sample, with the largest gains in code authoring (−52%) and code review & feedback (−44%), and smaller but still meaningful reductions in technical discussion (−20%) and discussion on proposals (−19%)

Productivity gains are most often reinvested in addressing technical debt [66%], adding new features [63%], and learning/upskilling [58%] ... genAI productivity is invested right back into OSS projects. In other words, genAI is not shrinking the overall OSS workload so much as reallocating it toward highervalue and longer-deferred work, from debt paydown to skill-building.

It's not all clear weather and blue skies though. My interpretation of the findings is that there definitely are challenges to address, but also that those that take the effort to address and integrate AI usage see the most in terms of net positives.

You can download the full report here.

[–] Fizz@lemmy.nz 3 points 23 hours ago

Thats better than I thought it would be.

[–] gary_host_laptop@lemmy.ml 2 points 23 hours ago

i think these results are very balanced, specially keeping in mind projects are still adapting to the new ways.

[–] Sims@lemmy.ml -2 points 19 hours ago (1 children)

Given that these pr's will only get better and better, this already sounds like a huge win for all oss users. Maintainers might need a special maintainer agent that filters and guides new agent contributors, but after that we'll see all oss projects boom with features, quality and constant development, or maintenance as long as anyone uses it. New types of projects - huge, or very complex ones - will start to appear also. It have already started.

At some point even the most negative user will have to stop using the 'slop' pejorative for ai driven pr's, as they'll just be better than hooman pr's. One commenter already noted that Hoomans create 'slop' too. AI will only do that temporarily, maybe longer than I think, but definitely temporarily.

[–] communism@lemmy.ml 4 points 12 hours ago (1 children)

If you actually read the article, you'll see that "slop" in the survey does not refer to AI-driven PRs in general, but to specifically low-quality AI-generated PRs. The % of PRs with AI-generated code would be higher. 25% of all PRs being low-quality AI-generated code is a lot.

[–] alottafagina@lemmy.ml 4 points 12 hours ago (1 children)

you’ll see that “slop” in the survey does not refer to AI-driven PRs in general, but to specifically low-quality AI-generated PRs.

This part is actually unclear. The question posed to survey participants that allowed researchers to determine a PR was "AI slop" is left out of the report. And this line under the chart showing "25% slop etc." is troubling:

Average estimated percentage of incoming PRs that are low-quality and/or AI-generated

How this particular question was framed is of critical importance here and without it, it's easy to assume that all reported AI use was broadly categorized as "slop"

Sadly I could not find a way to access the full list of questions used in the survey to confirm this.

[–] communism@lemmy.ml 2 points 11 hours ago (2 children)

They aren't specific about how the question was posed, no, but page 9's wording does state that it is maintainer-estimated, and specifies the terms "low-quality" and "slop" as the qualifier for the 25% figure, so unless they've made a huge logical leap, I think it is reasonable to conclude that maintainers were specifically asked about low-quality AI-generated PRs (or "AI slop"). I think people are sensible enough these days to not deny that AIs sometimes produce acceptable code.

[–] alottafagina@lemmy.ml 1 points 9 hours ago

page 9’s wording does state that it is maintainer-estimated, and specifies the terms “low-quality” and “slop” as the qualifier for the 25% figure

I'm not seeing it. This is the sole relevant statement on pg 9 where they simply define AI slop in a general way that avoids tying it in with their measurements:

“AI slop” are low quality AI-generated contributions that are unlikely to be worth the review time they demand.

In addition to the explicit 'and/or general AI use' statement, there are several red flags here that make me think this is a low quality study to be taken with salt. Or maybe I'm cynical and using the term 'slop' to push a research study is both unusual and suspect.

[–] atzanteol@sh.itjust.works 0 points 8 hours ago (2 children)

I think people are sensible enough these days to not deny that AIs sometimes produce acceptable code.

New to the forum?

There are plenty of people who call all ai generated code "slop".

[–] communism@lemmy.ml 1 points 7 hours ago (1 children)

It's a shorthand. It doesn't mean that AI is categorically incapable of producing code that is acceptable. If you ask an LLM to write a hello world program I'm sure everyone here can agree it would most likely produce a correct program.

[–] atzanteol@sh.itjust.works 1 points 6 hours ago* (last edited 6 hours ago)

But that's the problem - there is no definition for slop. Some people call ALL ai code slop, some just call bad code slop. It's a messy term that's up to the user to define.

[–] GiorgioBoymoder@lemmy.ml 1 points 7 hours ago

And also, according to Alex

Slop has existed before AI

What are we even talking about anymore? Has language always evolved this fast?