Security

6095 readers

3 users here now

Confidentiality Integrity Availability

founded 5 years ago

MODERATORS

528

submitted 1 year ago by lemmyreader@lemmy.ml to c/security@lemmy.ml

76 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] AmbiguousProps@lemmy.today 7 points 1 year ago

robots.txt doesn't have to be followed. It doesn't block crawling.