snix/third_party/sources
Florian Klink 853754d25f feat(ops/modules/www/git.snix.dev): block AI scrapers
This blocks a bunch of AI scrapers from Forgejo, which seems to be
particularly attractive.

Especially meta-externalagent has been scraping very excessively.

The list comes from https://github.com/ai-robots-txt/ai.robots.txt,
let's see how often this needs updating.

Change-Id: I55ae7c42c6a3eeff6f0457411a8b05d55cb24f65
Reviewed-on: https://cl.snix.dev/c/snix/+/30370
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: edef <edef@edef.eu>
2025-05-01 14:57:44 +00:00
..
default.nix chore(3p/sources): update niv scaffolding 2025-01-03 12:14:08 +00:00
sources.json feat(ops/modules/www/git.snix.dev): block AI scrapers 2025-05-01 14:57:44 +00:00