706
Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.
(i.imgur.com)
this post was submitted on 17 Aug 2025
706 points (99.7% liked)
Technology
607 readers
900 users here now
Share interesting Technology news and links.
Rules:
- No paywalled sites at all.
- News articles has to be recent, not older than 2 weeks (14 days).
- No external video links, only native(.mp4,...etc) links under 5 mins.
- Post only direct links.
To encourage more original sources and keep this space commercial free as much as I could, the following websites are Blacklisted:
- Al Jazeera;
- NBC;
- CNBC;
- Substack;
- Tom's Hardware;
- ZDNet;
- TechSpot;
- Ars Technica;
- Vox Media outlets, with exception for Axios;
- Engadget;
- TechCrunch;
- Gizmodo;
- Futurism;
- PCWorld;
- ComputerWorld;
- Mashable;
- Hackaday;
- WCCFTECH;
- Neowin.
More sites will be added to the blacklist as needed.
Encouraged:
- Archive links in the body of the post.
- Linking to the direct source, instead of linking to an article talking about the source.
Misc:
Relevant Communities:
- Beehaw Technology- Technology Related Discussions.
- lemmy.zip Technology- Hard Tech news.
founded 4 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I would hope not. Kinda pointless if they become public
On the contrary. Open community based block lists can be very effective. Everyone can contribute to them and asphyxiate people with malicious intents.
If you think something like, "if the blocklist is available then malicious agents simply won't use that ips" I don't think if that makes a lot of sense. As the malicious agent will know any of their IPs being blocked as soon as they use them.
Im sure there are many but I just learned of Crowdsecs WAF this year which is has a shared ban list. Its pretty cool. Im using it in prod right now. Im not saying it's the be all end all, but as part of a multilayered approach, it works pretty well.
It doesn't seem to be free though, right?
I dont know if they have a paid version but their free version has worked for me.