this post was submitted on 08 Jun 2025
220 points (98.2% liked)

Fuck AI

5068 readers
1963 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
 

It's impossible, i got this instance to just see lemmy from my own instance, but no, it was slow as hell the whole week, i got new pods, put postgres on a different pod, pictrs on another, etc.

But it was slow as hell. I didn't know what it was until a few hours before now. 500 GETs in a MINUTE by ClaudeBot and GPTBot, wth is this? why? I blocked the user agents, etc, using a blocking extension on NGINX and now it works.

WHY? So google can say that you should eat glass?

Life is now hell, if before at least someone could upload a website, now even that is painfull.

Sorry for the rant.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] carrylex@lemmy.world 8 points 7 months ago (2 children)

So I just had a look at your robots.txt:

User-Agent: *
  Disallow: /login
  Disallow: /login_reset
  Disallow: /settings
  Disallow: /create_community
  Disallow: /create_post
  Disallow: /create_private_message
  Disallow: /inbox
  Disallow: /setup
  Disallow: /admin
  Disallow: /password_change
  Disallow: /search/
  Disallow: /modlog
  Crawl-delay: 60

You explicitly allow searching your content by bots... That's likely one of the reasons why you get bot traffic.

[โ€“] plz1@lemmy.world 8 points 6 months ago

AI crawlers ignore robots.txt. The only way to get them to stop is with active counter measures.