Büt whāt æbœùt typīñg lîke thìß?
this post was submitted on 30 Dec 2025
427 points (99.1% liked)
Microblog Memes
9981 readers
1489 users here now
A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.
Created as an evolution of White People Twitter and other tweet-capture subreddits.
Rules:
- Please put at least one word relevant to the post in the post title.
- Be nice.
- No advertising, brand promotion or guerilla marketing.
- Posters are encouraged to link to the toot or tweet etc in the description of posts.
Related communities:
founded 2 years ago
MODERATORS
You didn't use the thorn!
I wonder how effective they are. When I first heard about ssh targets (like endlessh) I thought it was an awesome idea. But as I started to look at some analyzed logged data it turns out they are either slightly effective to not at all effective. If simple logic can be written so a dumb ssh bot programed to find vulnerable ssh servers can easily avoid a tar pit, I would think it is pretty trivial for an AI crawler to do the same thing. I am interested to see some analyzed data on something like this after several months on the open internet.
The reality is that depending on the crawling architecture someone is watching.
As aggressive as the LLM crawlers are there still have limits so a competently written one will have a budget for each host/site as well as a heuristic for the quality of results. It may dig for a bit and periodically return but if you're site is not one that is known to generate high quality data it may only get crawled when there isn't something better in the queue.
I am so confused by the low link lol.
- "AI haters build tarpits to trap and trick AI (!)" - Ohmy god poor AI :<
- "...that ignore robots.txt!" - ...oh, so illegal AI...?
- "Attackers explain-" - YEAH! THE EVIL AGRESSIVE
- "how anti-spamdefense became an AI weapon" - ...folk trying to defend from spam...?
FFS they try to paint people protecting themselves as evil but are keeping facts too much and it becomes an absolute confusing mess xD
It's not really that confusing.
The software equivalent of armed masked men are illegally breaking in to your personal property, stealing everything that isn't nailed down and ripping all the nails out of everything that is, and then leaving with it in order to reuse it for personal profit. It is, in all ways, similar to a home invasion. These invaders are then telling you that you're a bad person because you don't want them invading your property and stealing all your shit.
Its highly illegal, everyone involved with it knows for a fact that it's highly illegal, so they best they can do is try and spin propaganda around it because nobody has the balls to try and arrest Sam Altman, et al about it.
If you pick the lock on my front door and enter my home without permission I am going to put a 12 gauge slug through your solar plexus. If I could do the same to an AI crawler I would.
This is a terrible analogy.
First off, robots.txt has no force of law. It’s just a curtesy. You are free to ignore it (except where prohibited by EULA or contract).
Secondly, this is more similar to a supermarket hanging a sign that you can only access 3 of their 11 aisles.
What this is doing is if you try to access the 7 aisles they requested you not to use, you have to solve a math problem or two.
Ai scrapers are obnoxious loud drunk people who take way more than their fair share.
If you truly have something private (like your house) you should not expose it publically on the internet.
Well, let's turn this situation around then and see how it changes.
I hammer Meta's backend services with 6.8m requests per second, ignoring all posted guidelines, absorbing all the data I can get my hands on from them and feeding it to my machine which is busy trying to build BaseFook based on Meta's data that I've harvested from them.
Criminal DDOS? What's that?
Copyright law? Surely this doesn't apply to this.
Unauthorized access to backend systems? Nah, we'll be fine, that's definitely legal.
....
It is currently true that robots.txt doesn't have legal teeth and relies on voluntary compliance, but there have been court cases involving it in the past, and in my opinion they should have resulted in an established legal precedent. Check these out (courtesy of Wikipedia:)
The robots.txt played a role in the 1999 legal case of eBay v. Bidder's Edge,[12] where eBay attempted to block a bot that did not comply with robots.txt, and in May 2000 a court ordered the company operating the bot to stop crawling eBay's servers using any automatic means, by legal injunction on the basis of trespassing.[13][14][12] Bidder's Edge appealed the ruling, but agreed in March 2001 to drop the appeal, pay an undisclosed amount to eBay, and stop accessing eBay's auction information.[15][16]
In 2007 Healthcare Advocates v. Harding, a company was sued for accessing protected web pages archived via The Wayback Machine, despite robots.txt rules denying those pages from the archive. A Pennsylvania court ruled "in this situation, the robots.txt file qualifies as a technological measure" under the DMCA. Due to a malfunction at Internet Archive, Harding could temporarly access these pages from the archive and thus the court found "the Harding firm did not circumvent the protective measure".[17][18][19]
In 2013 Associated Press v. Meltwater U.S. Holdings, Inc. the Associated Press sued Meltwater for copyright infringement and misappropriation over copying of AP news items. Meltwater claimed that they did not require a license and that it was fair use, because the content was freely available and not protected by robots.txt. The court decided in March 2013 that "Meltwater’s copying is not protected by the fair use doctrine", mentioning among several factors that "failure […] to employ the robots.txt protocol did not give Meltwater […] license to copy and publish AP content".[20]
More like clogging the entry to your exhibition for making copies of your licensed produce, no?
Nowhere in the article or start of the readme did I find how this works. How does it differentiate between a human visitor and a scraper?
Here is a demo for anyone interested. It's deliberately slow to load.
It's deliberately slow to load
That kinda defeats the goal of feeding AI as much garbage as possible. They will just fetch a page from a different site in that time, instead of spending cycles on this page. It's not like the crawler works strictly serially.
The idea is to protect own server from unnecessary loads. You're welcome to provide a faster AI tar pit, just mind that ultimately this is a waste of resources.
I'm guessing that Markov chains are pretty efficient computationally compared to AI training. Don't have a site currently, but I'd love to see a bot rip through thousands of pages a minute.
Hell yeah, that rules
Cyberpunk as fuck.
Please, someone make us super easy to implement version of this.
Finally! Some good fucking news!
not a bad idea, might have to get on this bandwagon
I started experimenting with it a while ago, but I am currenty busy with other things.
That's so cool