maurice/snix
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
snix/ops/modules/www
History
Florian Klink 853754d25f feat(ops/modules/www/git.snix.dev): block AI scrapers 
This blocks a bunch of AI scrapers from Forgejo, which seems to be
particularly attractive.

Especially meta-externalagent has been scraping very excessively.

The list comes from https://github.com/ai-robots-txt/ai.robots.txt,
let's see how often this needs updating.

Change-Id: I55ae7c42c6a3eeff6f0457411a8b05d55cb24f65
Reviewed-on: https://cl.snix.dev/c/snix/+/30370
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: edef <edef@edef.eu>
2025-05-01 14:57:44 +00:00
..
auth.snix.dev.nix feat(*): initialize new Snix infrastructure 2025-03-17 17:15:07 +00:00
base.nix feat(*): initialize new Snix infrastructure 2025-03-17 17:15:07 +00:00
bolt.snix.dev.nix chore(web/snixbolt): move to contrib 2025-03-17 17:15:08 +00:00
cache.snix.dev.nix feat(ops): Deploy harmonia on cache.snix.dev 2025-03-31 12:39:21 +00:00
cl.snix.dev.nix feat(ops/www/gerrit): backward compatibility to TVL shortlinks 2025-03-17 17:15:08 +00:00
git.snix.dev.nix feat(ops/modules/www/git.snix.dev): block AI scrapers 2025-05-01 14:57:44 +00:00
loki.snix.dev.nix feat(*): initialize new Snix infrastructure 2025-03-17 17:15:07 +00:00
mimir.snix.dev.nix feat(*): initialize new Snix infrastructure 2025-03-17 17:15:07 +00:00
snix.dev.nix fix(ops/modules/www): fix rustdoc redirect 2025-03-17 21:50:54 +00:00
status.snix.dev.nix feat(*): initialize new Snix infrastructure 2025-03-17 17:15:07 +00:00
tempo.snix.dev.nix fix(ops/modules/tempo): drop keepalive from extraConfig 2025-04-30 20:21:42 +00:00