Commit graph

1069 commits

Author SHA1 Message Date
Florian Klink
d99819280a feat(ops): configure email for Forgejo
This configures Forgejo to use the "Forgejo" Message Stream on our "Snix"
server in Postmark.

Change-Id: I298966a8b43b55b0f1992a8fedf0fffcd6dde472
Reviewed-on: https://cl.snix.dev/c/snix/+/30206
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-21 01:40:52 +00:00
Florian Klink
3191a6c8d0 feat(ops): configure sendemail for gerrit
This configures Gerrit to use the "Gerrit" Message Stream on our "Snix"
server in Postmark.

Change-Id: I4d021919c666aabc94008f9f705163cb9639f1aa
Reviewed-on: https://cl.snix.dev/c/snix/+/30205
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-21 01:40:52 +00:00
adisbladis
7aef14c57f chore(ops/besadii): switch from buildGo to buildGoModule
Change-Id: I0457419d6b74d4f4c3c999a656a22ddd6c9d9ac3
Reviewed-on: https://cl.snix.dev/c/snix/+/30186
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Reviewed-by: Florian Klink <flokli@flokli.de>
2025-03-20 21:25:25 +00:00
adisbladis
b69cd940cf feat(ops/secrets): Use korora for type checking secrets
Type checking of secrets was removed in cff6575948 to get rid of yants.
This adds back type checking using Korora.

Fixes https://git.snix.dev/snix/snix/issues/71
Change-Id: I27cd47b7e1810be5c4cd5d86366e860ca217f9c4
Reviewed-on: https://cl.snix.dev/c/snix/+/30118
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Reviewed-by: Florian Klink <flokli@flokli.de>
2025-03-20 21:25:05 +00:00
Florian Klink
cfe842effa feat(ops/dns): setup Postmark DNS records
This configures the DNS records necessary to send emails from Postmark.

Change-Id: I2e55151f40c4f5e54f6d7f06ae24f2e863b7c656
Reviewed-on: https://cl.snix.dev/c/snix/+/30204
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
2025-03-20 21:18:40 +00:00
Florian Klink
29b4d0367b chore(ops/kontemplate): drop
We don't use this.

Change-Id: I3e6825521928f444a50426e493f448400c752b4e
Reviewed-on: https://cl.snix.dev/c/snix/+/30183
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
2025-03-20 13:57:33 +00:00
Raito Bezarius
dd7372782a feat(ops/modules/monorepo-gerrit): link to Forgejo issues
They are linked to Snix repo because this is the only one we are using.

Fixes #81.

Change-Id: I3c47547128a7dc5e1fe67a8fbe87b17c7e94f153
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
Reviewed-on: https://cl.snix.dev/c/snix/+/30144
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
2025-03-20 13:46:01 +00:00
Raito Bezarius
77669c14d2 fix(ops/modules/monorepo-gerrit): fix CL links
They were not going to q/ but just the root of the website, this was not
working.

Change-Id: I1acda0bb630198a8eef5b6fe991a395f1be1f796
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
Reviewed-on: https://cl.snix.dev/c/snix/+/30170
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
2025-03-20 13:44:00 +00:00
Raito Bezarius
7643267eb2 feat(ops/modules/monorepo-gerrit): re-enable Code Owners
Code Owners was disabled because it's very sensitive about the identity
of the committers and while pushing the original history, this was a
distraction.

Now that the history has been pushed and everyone is back to their
normal identity, it's fine to enable it again.

Fixes #83.

Change-Id: I4181d6af4eca489d4827b1c1ee606dfbb28a05c9
Reviewed-on: https://cl.snix.dev/c/snix/+/30173
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
Autosubmit: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 13:33:28 +00:00
Raito Bezarius
5d789a3ef4 chore(ops/modules/monorepo-gerrit): drop more unnecessary code
We are not going to use Panettone neither r/ revisions.

Change-Id: Icc037fc02861cfbe53690ca6641eb7ea777f7b74
Reviewed-on: https://cl.snix.dev/c/snix/+/30172
Autosubmit: Ryan Lahfa <masterancpp@gmail.com>
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
2025-03-20 13:33:28 +00:00
Florian Klink
91c752549c fix(ops/machines): set build-dir Nix setting
It looks like setting TMPDIR did not have the desired effect. I still
see a bunch of recent nix-build directories in /tmp.

Let's use the dedicated nix.conf setting, maybe it does do the job.

Change-Id: I17dc1e33bd0f20707adfbf9ad925251ac9aa77a5
Reviewed-on: https://cl.snix.dev/c/snix/+/30171
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
2025-03-20 12:36:48 +00:00
Florian Klink
3bd4674179 refactor(ops): use ops.users for ssh keys consistently
Add other keys used in the snix-cache VM to //ops/users, and drop the
`all` alias.

Change-Id: I030d0d49e8a6d9e3d8f1e1c2fc19f17ecb7ecb93
Reviewed-on: https://cl.snix.dev/c/snix/+/30165
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 12:21:44 +00:00
Florian Klink
ae4d967288 chore(ops): move archivist machine to ops and contrib
contrib/ gets the clickhouse patching, the bucket log parsing code and
the awscli setup and shell.

ops/ gets the machine config itself.

Change-Id: If8b8f8cce5ca9c2b4d19e17be9a8b895ac35e84a
Reviewed-on: https://cl.snix.dev/c/snix/+/30163
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 12:21:44 +00:00
Florian Klink
c3de9e21eb chore(ops): move nixos-tvix-cache to ops/machines
Change-Id: Id112f4e9ef195f9366a11f7b0dce326e7951fb49
Reviewed-on: https://cl.snix.dev/c/snix/+/30142
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
2025-03-20 12:21:44 +00:00
Raito Bezarius
8eca846d09 fix(ops/machines/build01): put Nix builds temp dir in /var/tmp
This solves issues such as
```
cargo:warning=Fatal error: can't create
/build/source/target/debug/build/zstd-sys-fa4cde6de82f89a8/out/88f362f13b0528ed-zstd_decompress_block.o:
No space left on device
```

on the Buildkite CI.

Fixes #82.

Change-Id: Iee9516d8d595b718824c3e7b28c01c3ef9e9d090
Signed-off-by: Raito Bezarius <raito@lix.systems>
Reviewed-on: https://cl.snix.dev/c/snix/+/30143
Autosubmit: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
2025-03-19 20:55:02 +00:00
Florian Klink
9e7cadeded fix(ops): delete email config for now
We don't have an email server configured (yet), we can resurrect it once
we do.

Change-Id: I568075154c6169d031462f39b43ce5897a754f19
Reviewed-on: https://cl.snix.dev/c/snix/+/30109
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-18 21:52:06 +00:00
Florian Klink
512cbb0813 fix(ops/machines/build01): stop using Nix 2.3 in CI
There's been a few deadlock problems with Nix 2.3, as discusssed in the
commit message of https://cl.tvl.fyi/c/depot/+/12334.

However, since the fork nothing prevents us from dropping the Nix 2.3
requirement for CI.

Change-Id: Ib00603597dbc11dc1b619fdeee264d7d519eaa02
Reviewed-on: https://cl.snix.dev/c/snix/+/30108
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 17:29:38 +00:00
Florian Klink
a59e95a287 fix(ops/nixos): don't set nixpkgs.pkgs explicitly
As soon as you pass in an already-instantiated nixpkgs version, it will
cause nixpkgs.hostPlatform etc. to be not applied.

This means it's impossible to describe the architecture of a VM closure
you're deploying, and have it deviate from the machine you're evaluating
from, making it quite hard to deploy that x86_64-linux machine from
aarch64-linux (where I'm writing this commit message from).

Drop explicitly passing in nixpkgs.path, and set nixpkgs.hostPlatform
explicitly for all remaining system configurations in the repository
where not already set.

Change-Id: Ie2a596e0826da54674b4f02fcd8fed3569fee0a4
Reviewed-on: https://cl.snix.dev/c/snix/+/30104
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 17:23:37 +00:00
Ilan Joselevich
580f03f6fd chore(ops/modules): Cleanup leftovers from TVL
Change-Id: I979cb18f3b8d461d21424e8dae6b0b2d7407809d
Reviewed-on: https://cl.snix.dev/c/snix/+/30106
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-18 17:12:35 +00:00
Florian Klink
7b36b8e285 chore(ops/keycloak): move oauth application to snix-project org
This was a personal application before, now it's at the `snix-project`
org.

Change-Id: I6df9393f23593f58739f331e73103022301b4f11
Reviewed-on: https://cl.snix.dev/c/snix/+/30101
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 15:28:54 +00:00
Ilan Joselevich
aefa1eaa28 chore(ops/secrets): remove old secrets from TVL
Change-Id: Id7ffd405bbc7cf1d5b09a9a90941e0f3e7ebd574
Reviewed-on: https://cl.snix.dev/c/snix/+/30100
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-18 14:51:11 +00:00
Florian Klink
bc62fc0354 chore(ops/builderball): drop
This is unused.

Change-Id: Ida0764680ff128d80580418a8b1a8bc6576c0f07
Reviewed-on: https://cl.snix.dev/c/snix/+/30081
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 00:24:20 +00:00
Florian Klink
afc5c152b8 fix(ops/modules/www): fix rustdoc redirect
Change-Id: Icd78c2209c20aee0931622bfb914e9dc5978d23d
2025-03-17 21:50:54 +00:00
Florian Klink
68896423b9 fix(ops/pipelines): drop snix trigger
This was previously only used to fix the Tvix subtree pipeline, not the
depot pipeline.

Running it for refs/heads/canon in snix is gonna trigger a lot of
unnecessary builds.

Change-Id: I1b5fdfcc0fe5408cda27053beb317cfeecdc6ca4
2025-03-17 21:31:38 +00:00
Florian Klink
a740b08027 fix(ops/pipelines): fix trigger step
The slug of the pipeline to trigger is snix.

Change-Id: I31941451ffc7680ea316c9adf5c933a562f819bc
2025-03-17 21:07:29 +00:00
Florian Klink
3ca022e5c1 fix(ops/pipelines/depot): fix targeting for anchor step
There is no nevsky here.

Change-Id: Id92356e1a401b24b958694cee1268358a58975cd
2025-03-17 21:07:29 +00:00
Ryan Lahfa
a2d77189df fix(ops/secrets/gerrit-autosubmit): rekey
Change-Id: I8591fddbb7583ae51d67269ae6b055ddb619ddb6
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:16:18 +00:00
Ryan Lahfa
b6516a9605 fix(ops/modules/buildkite): merge tags appropriately
Everything was large=true and then nothing was small=true and did not
have a hostname.

This is fixed.

Change-Id: Id90e6246f9ab44ce020d999e975dd8b4cd4492c9
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:16:18 +00:00
Ryan Lahfa
9aea7dc6df feat(ops/modules/gerrit): push r/* as well
Change-Id: I367d5136749286c377b35dd05c242a65d75b5100
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:15:08 +00:00
Florian Klink
2f3fbf854d feat(ops/machines/build01): bump nrBuildUsers and max-jobs
Change-Id: I8aa4e87eb41483164e284cd5649953081af92711
2025-03-17 17:15:08 +00:00
Florian Klink
c9eae6d8d8 feat(ops/machines/build01): use large slots
build01 can deal with llama.

Change-Id: I9c01dabfb4dfff0061fd16ea37ef8dc0693ad453
2025-03-17 17:15:08 +00:00
Ryan Lahfa
a7916624dc feat(ops/users): move ops' keys in ops/
Change-Id: Ia51eaea658030a893e36d9d6b0c999ca7e71133e
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:15:08 +00:00
Ryan Lahfa
073142f796 feat(ops/www/gerrit): backward compatibility to TVL shortlinks
cl.snix.fyi/q/$ID where $ID ≤ 30K will redirect (301) to
cl.tvl.fyi/q/$ID to keep the old links working.

Change-Id: I27b496a1c52a3de3d106292ba7a2931b0f15fa49
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:15:08 +00:00
Florian Klink
dd392ef054 feat(ops/keycloak): add GitLab SSO
Change-Id: I41ee3cb2988288e6b282d85b111c41064f09eaec
2025-03-17 17:15:08 +00:00
Florian Klink
97f22e0ea6 fix(ops/modules/forgejo): disable downloading source archives
We're probably getting crawled by LLM scrapers, and this unnecessarily
fills up disk space.

Change-Id: Ib20d04337aa26a73889c97d12fb109261b8da56d
2025-03-17 17:15:08 +00:00
Florian Klink
4c65a325a8 public01: deploy snix.dev
Change-Id: Ia0a439dd1628299569503370c21a0bbf9552830e
2025-03-17 17:15:08 +00:00
Florian Klink
9ac1b9a798 fix(ops/deploy-machine): drop
This doesn't look like it's actually being used.

Change-Id: I06e15d01b793748d9ed216b23b95a757bc2d0f0e
2025-03-17 17:15:08 +00:00
Florian Klink
944acead3e chore(web/snixbolt): move to contrib
Change-Id: Ic69648c526e6c1a0b769b26642b3c799872eb87c
2025-03-17 17:15:08 +00:00
Florian Klink
cff6575948 refactor(nix/*): drop yants and consumers, and some more
Change-Id: I96ab5890518c7bb0d4a676adbad20e4c49699b63
2025-03-17 17:15:08 +00:00
Florian Klink
a52ea3675c feat(*): initialize new Snix infrastructure
Co-Authored-By: edef <edef@edef.eu>
Co-Authored-by: Ryan Lahfa <raito@lix.systems>
Change-Id: Ica1cda177a236814de900f50a8a61d288f58f519
2025-03-17 17:15:07 +00:00
Florian Klink
067eff3427 fix(ops/pipelines): allow git+https://
We need this to fetch some third-party sources.

Change-Id: If542f88c5d517d20b530486fafbc7f6c391f072a
2025-03-17 16:18:26 +00:00
Florian Klink
df4500ea2b chore(*): drop everything that is not required for Tvix
Co-Authored-By: edef <edef@edef.eu>
Co-Authored-By: Ryan Lahfa <raito@lix.systems>
Change-Id: I9817214c3122e49d694c5e41818622a08d9dfe45
2025-03-17 16:18:26 +00:00
Florian Klink
fc7f013066 refactor(ops/clbot): use escapeSystemdPath
This is definitely faster than doing a roundtrip via a build.

Change-Id: I7a02b828462def735fdb241ce729143e90bc5c75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13236
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-03-10 15:09:14 +00:00
sterni
3f94a09648 chore(ops/modules): use smtprelay from nixpkgs
This upgrades smtprelay from 1.7.0 -> 1.11.2:

- https://github.com/decke/smtprelay/releases/tag/v1.8.0
- https://github.com/decke/smtprelay/releases/tag/v1.9.0
- https://github.com/decke/smtprelay/releases/tag/v1.10.0
- https://github.com/decke/smtprelay/releases/tag/v1.11.1
- https://github.com/decke/smtprelay/releases/tag/v1.11.2

Change-Id: Ia0641f67fcc4672302a51b3ad422e447ea32b203
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13115
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
2025-03-02 21:16:00 +00:00
sterni
11681612c3 chore: lib.types.string -> lib.types.str
lib.types.string concatenates conflicting definitions on module merge
which is counter intuitive and hard to debug behavior.

Change-Id: I8ccdca0e8895fb5cc4600c367f8d52a9ab80ff75
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13177
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: aspen <root@gws.fyi>
2025-03-02 13:55:21 +00:00
Vincent Ambo
dddcc6ef01 fix(ops/livegrep): adapt reindexer command for podman
Something recently caused us to replace Docker with Podman (I guess a default
changed in nixpkgs? I don't remember making the change explicitly), which broke
the reindexing unit.

Change-Id: I1d3453ed970e536abb540c6ef79765cfda271810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13173
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2025-02-24 11:35:36 +00:00
Vincent Ambo
27f8a8367a fix(ops/www): block ClaudeBot/GPTBot/Amazonbot from cgit
These bots are doing unthrottled requests to cgit 24/7, and it's starting to
annoy me.

Change-Id: I6b7d7a68e9becb8ed4b5c52b376c2a60febc6ec6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13145
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-16 16:10:56 +00:00
Vincent Ambo
eeb6fdd7d4 refactor(ops/pipelines): use large build slots for eval
Change-Id: I22b3ebb91695a3d43696196c0a189d3b1656df8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13128
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2025-02-15 08:21:38 +00:00
Vincent Ambo
84940c7cee chore(ops/nevsky): increase buildkite agent slot count
Normal agents can easily go from 16 -> 24 (proportionally to whitby, this makes
more sense).

I've kind of randomly decided to label 6 agents as large ones. We will filter
things like eval, or building tvix tests (until b/431 is resolved).

Change-Id: Ib38d2c56410c2ad9d86a01546c00192f87274bb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13121
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-09 20:59:31 +00:00
Vincent Ambo
2d989ef6cb refactor(ops/tvl-buildkite): add concept of small/large slots
Adds a new tagging system to Buildkite agents, where agents are tagged with
large/small slots. All agents have small slots, only some agents have large
slots. The small slots are purely informative - nothing targets them, whereas
large slots will be used for filtering agents.

This allows us to target large slots in some builds and minimise the concurrent
execution of extremely large builds, while keeping a large number of small slots
around for all the light targets.

This will need some tuning over time (also because tagging is a manual process).

Change-Id: I15aa657773ed874d84d98e55238fb31c75d4efa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13120
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-09 20:59:31 +00:00