Commit graph

200 commits

Author SHA1 Message Date
Florian Klink
ca23b17680 refactor(ops/machines): switch from grafana-agent to alloy
grafana-agent has been removed, but the failing eval was missed due
to #80.

Change-Id: I87cfc71c8c98e27e32f4e95e4d85901195cb5b75
Reviewed-on: https://cl.snix.dev/c/snix/+/30347
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
2025-04-30 20:21:42 +00:00
Florian Klink
b2fa87f344 fix(ops/machines/*): fix leftover usages of depot.automatic-gc
This was missed, due to #80.

Change-Id: I3b10fa615c09fdd9887c63c847cfd70f5a80d277
Reviewed-on: https://cl.snix.dev/c/snix/+/30346
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
2025-04-30 20:20:12 +00:00
Florian Klink
7ffd2ea502 feat(ops/machines/build01): enable automatic GC
Fixes #109.

Change-Id: I8bcf4f9900a34b6d07f1e70ada22de6e398f6203
Reviewed-on: https://cl.snix.dev/c/snix/+/30339
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2025-04-29 10:06:23 +00:00
Florian Klink
6e45456fec fix(ops/machines/snix-cache): support old /nar/tvix-castore URLs
Nix clients still might have old .narinfo files cached, containing old
NAR URLs. Send a redirect to the new URL.

Fixes: #103
Change-Id: Ie3b77e4fdc4be0f982e023f2a2acd3f9f0257f9b
Reviewed-on: https://cl.snix.dev/c/snix/+/30291
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: edef <edef@edef.eu>
2025-04-02 17:20:10 +00:00
Ilan Joselevich
5551d0ea5e feat(ops): Deploy harmonia on cache.snix.dev
Deploys Harmonia on build01, proxied through public01.
We cannot serve from build01 directly because it only supports IPv6.

Closes: https://git.snix.dev/snix/snix/issues/66
Change-Id: Iff3c16366d60c0fbfd1315a18c27fcd636a0261a
Reviewed-on: https://cl.snix.dev/c/snix/+/30274
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-31 12:39:21 +00:00
Florian Klink
00950aa91d fix(ops): add +x for /nix/var/nix/gcroots
Previously, the buildkite users were not able to traverse there.

Removing /nix/var/nix/gcroots/buildkite/canon might not be needed, and
is racy with other anchor step - the first one might still be building
`ci.gcroot` (and didn't create the new symlink), so the second one will
fail trying to remove the non-existing symlink.

Change-Id: I0449447f7193113d807d597750b26c7beb48a3a6
Reviewed-on: https://cl.snix.dev/c/snix/+/30257
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
2025-03-23 15:02:22 +00:00
Florian Klink
8c4f447ec7 fix(ops/pipelines): fix anchor steps
Every buildkite user needs to be able to update these symlinks, and the
directory doesn't exist. It was probably created imperatively on whitby.

Use a tmpfiles rule creating a /nix/var/nix/gcroots/buildkite directory,
and add a `canon` symlink in there.

Change-Id: Ic4d67fbb69f77cebe891b0fff9b824713ebec87c
Reviewed-on: https://cl.snix.dev/c/snix/+/30247
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-23 00:48:45 +00:00
Florian Klink
22c2770f42 fix(ops/machines/build01): switch to Lix in nix.package
There's been a lot of

```
nix-daemon[2039685]: unexpected Nix daemon error: error: writing to file: Broken pipe
```

log messages, and failed builds in CI.

These don't seem to occur with Lix.

Change-Id: Ida277064282905154ea9265f935a221bf8006c8d
Reviewed-on: https://cl.snix.dev/c/snix/+/30225
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-21 13:46:35 +00:00
Florian Klink
d99819280a feat(ops): configure email for Forgejo
This configures Forgejo to use the "Forgejo" Message Stream on our "Snix"
server in Postmark.

Change-Id: I298966a8b43b55b0f1992a8fedf0fffcd6dde472
Reviewed-on: https://cl.snix.dev/c/snix/+/30206
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-21 01:40:52 +00:00
Florian Klink
3191a6c8d0 feat(ops): configure sendemail for gerrit
This configures Gerrit to use the "Gerrit" Message Stream on our "Snix"
server in Postmark.

Change-Id: I4d021919c666aabc94008f9f705163cb9639f1aa
Reviewed-on: https://cl.snix.dev/c/snix/+/30205
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-21 01:40:52 +00:00
Florian Klink
91c752549c fix(ops/machines): set build-dir Nix setting
It looks like setting TMPDIR did not have the desired effect. I still
see a bunch of recent nix-build directories in /tmp.

Let's use the dedicated nix.conf setting, maybe it does do the job.

Change-Id: I17dc1e33bd0f20707adfbf9ad925251ac9aa77a5
Reviewed-on: https://cl.snix.dev/c/snix/+/30171
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
2025-03-20 12:36:48 +00:00
Florian Klink
3bd4674179 refactor(ops): use ops.users for ssh keys consistently
Add other keys used in the snix-cache VM to //ops/users, and drop the
`all` alias.

Change-Id: I030d0d49e8a6d9e3d8f1e1c2fc19f17ecb7ecb93
Reviewed-on: https://cl.snix.dev/c/snix/+/30165
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 12:21:44 +00:00
Florian Klink
ae4d967288 chore(ops): move archivist machine to ops and contrib
contrib/ gets the clickhouse patching, the bucket log parsing code and
the awscli setup and shell.

ops/ gets the machine config itself.

Change-Id: If8b8f8cce5ca9c2b4d19e17be9a8b895ac35e84a
Reviewed-on: https://cl.snix.dev/c/snix/+/30163
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-20 12:21:44 +00:00
Florian Klink
c3de9e21eb chore(ops): move nixos-tvix-cache to ops/machines
Change-Id: Id112f4e9ef195f9366a11f7b0dce326e7951fb49
Reviewed-on: https://cl.snix.dev/c/snix/+/30142
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
2025-03-20 12:21:44 +00:00
Raito Bezarius
8eca846d09 fix(ops/machines/build01): put Nix builds temp dir in /var/tmp
This solves issues such as
```
cargo:warning=Fatal error: can't create
/build/source/target/debug/build/zstd-sys-fa4cde6de82f89a8/out/88f362f13b0528ed-zstd_decompress_block.o:
No space left on device
```

on the Buildkite CI.

Fixes #82.

Change-Id: Iee9516d8d595b718824c3e7b28c01c3ef9e9d090
Signed-off-by: Raito Bezarius <raito@lix.systems>
Reviewed-on: https://cl.snix.dev/c/snix/+/30143
Autosubmit: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
2025-03-19 20:55:02 +00:00
Florian Klink
9e7cadeded fix(ops): delete email config for now
We don't have an email server configured (yet), we can resurrect it once
we do.

Change-Id: I568075154c6169d031462f39b43ce5897a754f19
Reviewed-on: https://cl.snix.dev/c/snix/+/30109
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2025-03-18 21:52:06 +00:00
Florian Klink
512cbb0813 fix(ops/machines/build01): stop using Nix 2.3 in CI
There's been a few deadlock problems with Nix 2.3, as discusssed in the
commit message of https://cl.tvl.fyi/c/depot/+/12334.

However, since the fork nothing prevents us from dropping the Nix 2.3
requirement for CI.

Change-Id: Ib00603597dbc11dc1b619fdeee264d7d519eaa02
Reviewed-on: https://cl.snix.dev/c/snix/+/30108
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 17:29:38 +00:00
Florian Klink
a59e95a287 fix(ops/nixos): don't set nixpkgs.pkgs explicitly
As soon as you pass in an already-instantiated nixpkgs version, it will
cause nixpkgs.hostPlatform etc. to be not applied.

This means it's impossible to describe the architecture of a VM closure
you're deploying, and have it deviate from the machine you're evaluating
from, making it quite hard to deploy that x86_64-linux machine from
aarch64-linux (where I'm writing this commit message from).

Drop explicitly passing in nixpkgs.path, and set nixpkgs.hostPlatform
explicitly for all remaining system configurations in the repository
where not already set.

Change-Id: Ie2a596e0826da54674b4f02fcd8fed3569fee0a4
Reviewed-on: https://cl.snix.dev/c/snix/+/30104
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
2025-03-18 17:23:37 +00:00
Florian Klink
2f3fbf854d feat(ops/machines/build01): bump nrBuildUsers and max-jobs
Change-Id: I8aa4e87eb41483164e284cd5649953081af92711
2025-03-17 17:15:08 +00:00
Florian Klink
c9eae6d8d8 feat(ops/machines/build01): use large slots
build01 can deal with llama.

Change-Id: I9c01dabfb4dfff0061fd16ea37ef8dc0693ad453
2025-03-17 17:15:08 +00:00
Ryan Lahfa
a7916624dc feat(ops/users): move ops' keys in ops/
Change-Id: Ia51eaea658030a893e36d9d6b0c999ca7e71133e
Signed-off-by: Ryan Lahfa <raito@lix.systems>
2025-03-17 17:15:08 +00:00
Florian Klink
4c65a325a8 public01: deploy snix.dev
Change-Id: Ia0a439dd1628299569503370c21a0bbf9552830e
2025-03-17 17:15:08 +00:00
Florian Klink
9ac1b9a798 fix(ops/deploy-machine): drop
This doesn't look like it's actually being used.

Change-Id: I06e15d01b793748d9ed216b23b95a757bc2d0f0e
2025-03-17 17:15:08 +00:00
Florian Klink
a52ea3675c feat(*): initialize new Snix infrastructure
Co-Authored-By: edef <edef@edef.eu>
Co-Authored-by: Ryan Lahfa <raito@lix.systems>
Change-Id: Ica1cda177a236814de900f50a8a61d288f58f519
2025-03-17 17:15:07 +00:00
Florian Klink
df4500ea2b chore(*): drop everything that is not required for Tvix
Co-Authored-By: edef <edef@edef.eu>
Co-Authored-By: Ryan Lahfa <raito@lix.systems>
Change-Id: I9817214c3122e49d694c5e41818622a08d9dfe45
2025-03-17 16:18:26 +00:00
Vincent Ambo
84940c7cee chore(ops/nevsky): increase buildkite agent slot count
Normal agents can easily go from 16 -> 24 (proportionally to whitby, this makes
more sense).

I've kind of randomly decided to label 6 agents as large ones. We will filter
things like eval, or building tvix tests (until b/431 is resolved).

Change-Id: Ib38d2c56410c2ad9d86a01546c00192f87274bb3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13121
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-09 20:59:31 +00:00
Vincent Ambo
c88fae5277 chore: remove whitby configuration
Removes whitby DNS records and other related configuration that is no longer
required now that whitby is gone.

whitby served us well. RIP.

This resolves b/433.

Change-Id: I56fe6f88cde9112fc3bfc79758ac33e88a743422
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13117
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2025-02-07 22:18:00 +00:00
Vincent Ambo
7851917ebf chore(ops/whitby): retire most services on whitby
This turns off almost all of the lights. The server will be decomissioned on
2025-02-05. Until then we can keep running the Buildkite builders there for
extra capacity.

Stuff that was left in the whitby config has been migrated to nevsky.

This relates to b/433.

Change-Id: I84953e9d5e912f75b8884cb9d8edd5a1b7d5c85d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13095
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 15:01:24 +00:00
Vincent Ambo
afcdcc6338 feat(ops/modules): factor grafana & prometheus setup into a module
... that is then promptly enabled on nevsky.

Change-Id: Ie51037cec810bb7f81099a67ebd2581dcf710bd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13093
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
75fe623d97 feat(ops/nevsky): run keycloak/panettone/paroxysm
These are the postgres-database using services.

Change-Id: I4e8d854e798d85e1b14bfa78aae8827ac0881c7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13092
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
3a92f43ee5 feat(ops/nevsky): run TVL OpenLDAP server
Change-Id: I9afce1500e0888f523fe8b775edaa7a2c3ab53a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13091
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
64ae639cc7 fix(ops/nevsky): ensure CPU is not power-saving unnecessarily
Change-Id: I5ea6f2fdbf3ccbf993787b1c592539b1fdee151f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13090
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:58:16 +00:00
Vincent Ambo
f2257abc5d feat(ops/nevsky): run livegrep instance
Change-Id: Icc17ff919aaf23964b6f35160aaeb437e69ee7bf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13087
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
610cfeda25 feat(ops/nevsky): run cgit & josh on nevsky
Change-Id: If62177d19c0c4e708dce7a20974f53827a133247
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13086
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:58:16 +00:00
Vincent Ambo
c6f2f6924b feat(ops/nevsky): run TVL Gerrit instance
Runs the Gerrit instance with the same config as previously on whitby. Data has
been migrated manually using `tailscale file` (which worked surprisingly well).

Change-Id: I6e85f932c834b2c36fc40327ae081ee396c5e16f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13077
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
4a1a271644 feat(ops/nevsky): run smtprelay instance for gerrit
Change-Id: I856fafb4c13a876bb6d9cfdb0cdf554d9d0a6b11
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13076
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:58:16 +00:00
Vincent Ambo
2b828fbb8e feat(ops/nevsky): run clbot
Change-Id: I2e71ca70b5035041d354a2ba4fa088efb5182d2b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13075
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:58:16 +00:00
Vincent Ambo
4783d83686 feat(ops/nevsky): sync remaining whitby secrets to nevsky
Change-Id: I604426d8e9e91417607eed71f0dbcaf93e88c31d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13074
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
749a01b681 feat(ops/nevsky): run owothia & irccat on nevsky
Change-Id: I9234a77967634c9b472151ea5ac7ef4e76c2d7e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13073
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:58:16 +00:00
Vincent Ambo
119df03a50 style(ops/nevsky): centralise depot services in a config block
Change-Id: I46ceb8fdbcb49e98772cb272a7b775761f9d1ed0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13072
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:58:16 +00:00
Vincent Ambo
899142ed2c feat(ops/nevsky): run ZNC bouncer
Same as whitby, with the difference that there is now a listener on the
tailnet (just in case).

Change-Id: I841b2283112a0fea54f3c35a2dc4d2dd393b2612
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13071
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-02 14:18:10 +00:00
Vincent Ambo
cf919a02b2 feat(ops/nevsky): run postgresql server
All the postgres-dependent services are going to migrate here.

Change-Id: Ie2a25395f6fe6e3c9f7a45f21cf90c635e208cdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13070
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-02 14:18:10 +00:00
Vincent Ambo
01016828b8 feat(bugry): run static & mostly static websites on bugry
Change-Id: Ie4c723f68ce5a07e2c7ab1f10a9d652ad442f44a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13067
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-01 20:34:24 +00:00
Vincent Ambo
11e1b9ffbd feat(ops/nevsky): configure automatic GC module
Change-Id: I6c89129206773f4855cdda7ddc944ecb5437ec8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13061
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
2025-02-01 19:34:55 +00:00
Vincent Ambo
658f07a50e fix(ops/nevsky): fix NDP entry for bugry
Apparently I set this up manually before, and failed to commit it ...

Change-Id: I550a2cd9e1fcc8b508bafc2fd06ddab2a915b597
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13060
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-01 19:34:55 +00:00
Vincent Ambo
86483c7908 feat(ops/bugry): configure self redirect for the machine
Change-Id: I25b8541cc9bd66d4c9db6531ce960d224b5e73c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13059
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-02-01 19:34:55 +00:00
Vincent Ambo
7824f8a91f feat(ops/bugry): configure depot replication to bugry
Change-Id: I3ee35e76c9ec6d7a175801c45eee073ce23d3dfd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13020
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
2025-01-18 22:31:05 +00:00
Vincent Ambo
de8ffb723c chore(ops): remove nixery-01 VM completely
This is no longer needed; Nixery is now served by bugry.

Change-Id: Idd072505c4da1e6af636224e092b6fb21eff9250
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13001
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2025-01-15 15:38:48 +00:00
Vincent Ambo
98aee9f84a fix(ops/machines): add missing bugry & nevsky entries
Why are we doing this manually again?

Change-Id: I5a941d677e7c6e762f97d8b607d8409b6e9badb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13000
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
2025-01-15 15:38:48 +00:00
Vincent Ambo
5a33dd7ec3 feat(ops/bugry): run nixery instance
Running Nixery on bugry is much more cost efficient (better traffic economics
than on a cloud provider, and Nixery is mostly a traffic-heavy service), and
frees up my Yandex Cloud credits for adding another builder.

Change-Id: Id6c8c76b28a5ce13cc8b743ad6e72fffd19353fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12997
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
2025-01-15 15:38:48 +00:00