Lemmy

14638 readers

25 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 6 years ago

MODERATORS

nutomic@lemmy.ml

Anyone with experience using anubis on their lemmy server, have a good config? (github.com)

submitted 2 months ago by dessalines@lemmy.ml to c/lemmy@lemmy.ml

16 comments fedilink hide all child comments

I've recently added anubis to lemmy.ml, and it seems to be working well.

I have a PR to add anubis to lemmy-ansible (our main installation method), and I could use some help tweaking / optimizing its botPolicy.yaml config, for federated services.

Anyone with experience running anubis, this would be much appreciated.

you are viewing a single comment's thread
view the rest of the comments

[–] adam@doomscroll.n8e.dev 4 points 2 months ago (1 children)

It would be nice to have a much more aggressive anti-bot stance for communities/content that aren't local. If google or any other crawler wants to crawl c/lemmy@lemmy.ml then it should do it on the source instance. Doing it on mine makes no sense.

[–] bjoern_tantau@swg-empire.de 1 points 2 months ago

Well behaved bots should see the origin in the headers and only crawl those.