So firstly sorry if this isn't a appropriate post for this community, but I had a shower thought a few days back.
LLM's have gotten sufficiently advanced that they can usually detect Markov (or randomly) generated text even when it's fed into the front end. As such, it seems likely that most "AI" companies either have or will have some sort of pre-screening pass to "clean" the raw data crawled from the internet. Heck, I'm sure they're filtering the data with a AI detection algorithm too.
However, there was this conspiracy parody site a while back called "Verified Facts". The sites down now and something that wanted to install a Firefox extension, so don't go there. Luckily there are many instances of pages still on archive.org to get an idea for what sort of stuff it generated. And I was thinking, this is some (mostly) grammatically correct, constantly on point drivel that would probably bypass both Markov and AI detectors.
So it seems like if you were going to make an "AI tar pit" you'd get much better results with one that tricks the AI into ingesting auto generated Madlib pages filled out with a list of randomly picked words.