Adds a new tagging system to Buildkite agents, where agents are tagged with
large/small slots. All agents have small slots, only some agents have large
slots. The small slots are purely informative - nothing targets them, whereas
large slots will be used for filtering agents.
This allows us to target large slots in some builds and minimise the concurrent
execution of extremely large builds, while keeping a large number of small slots
around for all the light targets.
This will need some tuning over time (also because tagging is a manual process).
Change-Id: I15aa657773ed874d84d98e55238fb31c75d4efa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13120
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Included changes/fixes:
* bumped all `wasm-bindgen` usages again
* regenerated protobuf files
* keycloak terraform provider has been migrated to new name
This also included a state migration in the bucket, which I've already
performed.
* tvix/boot: disable tests that are broken in CI
* users/aspen/yeren: avoid upgrading kernel to 6.12
digimend depends on a fix: https://github.com/NixOS/nixpkgs/pull/378830/
Change-Id: I657dcf5c4d0d08f231bfe30e37c8062bfcfaaa32
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13098
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
This replaces the previous Buildkite agent token with one that is tied to the
default agent "cluster".
We haven't used clusters so far, they're a relatively new Buildkite
feature (https://buildkite.com/docs/pipelines/clusters), but I have a suspicion
that weird scheduling behaviour recently has been related to our non-usage of
these clusters.
Change-Id: I30e9c0cf49fe1fc4e263a4dc7d3855c166349939
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13118
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Removes whitby DNS records and other related configuration that is no longer
required now that whitby is gone.
whitby served us well. RIP.
This resolves b/433.
Change-Id: I56fe6f88cde9112fc3bfc79758ac33e88a743422
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13117
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Backups are moving from GleSYS to Yandex Cloud (is this motivated by me not
having to pay for them in that case? Maybe!); this changes the default backup
location to accommodate that.
I also noticed that we previously manually placed the backup key on whitby, so
the new key is going into agenix instead, as well as the secrets for protecting
the repositories.
Change-Id: Ibe5dbfec6784345f020a8b4d92bb01c6ad719a89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13096
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This turns off almost all of the lights. The server will be decomissioned on
2025-02-05. Until then we can keep running the Buildkite builders there for
extra capacity.
Stuff that was left in the whitby config has been migrated to nevsky.
This relates to b/433.
Change-Id: I84953e9d5e912f75b8884cb9d8edd5a1b7d5c85d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13095
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
... that is then promptly enabled on nevsky.
Change-Id: Ie51037cec810bb7f81099a67ebd2581dcf710bd5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13093
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
These are the postgres-database using services.
Change-Id: I4e8d854e798d85e1b14bfa78aae8827ac0881c7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13092
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Runs the Gerrit instance with the same config as previously on whitby. Data has
been migrated manually using `tailscale file` (which worked surprisingly well).
Change-Id: I6e85f932c834b2c36fc40327ae081ee396c5e16f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13077
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Same as whitby, with the difference that there is now a listener on the
tailnet (just in case).
Change-Id: I841b2283112a0fea54f3c35a2dc4d2dd393b2612
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13071
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
All the postgres-dependent services are going to migrate here.
Change-Id: Ie2a25395f6fe6e3c9f7a45f21cf90c635e208cdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13070
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This grants access to all relevant keys to the new machines.
Change-Id: I78a7003181ee9977e548fbfe0d78afb67357b56b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13064
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
🦙 is the blocking step on which everything else is always waiting, so to
avoid a situation where we have idle builders we should prioritise running
llamas within the available slots to spawn other stuff in the build queue.
Change-Id: I76836275edd0b0aedaf702d2626dacc31ced9fe2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13069
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reduces the whitby record TTLs everywhere, and sets up the scaffolding for
adding nevsky/bugry records.
This relates to b/433.
Change-Id: I31b5503fa4fcf5463c989f4cf47a3d403d34c684
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13066
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This is an old unused name.
Change-Id: I63082bb1b0daa38ab63400e956151f1aad53b41a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13065
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Apparently I set this up manually before, and failed to commit it ...
Change-Id: I550a2cd9e1fcc8b508bafc2fd06ddab2a915b597
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13060
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
I've checked the new links manually.
- //web/tvl: changed the target path of links to the tvix docs since
they were moved in r/2378.
- //users/aspen/resume: Tvix is no longer //third_party/nix.
Change-Id: I419bae1a46bdccc7baa7327215aa2368ffc0f01c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13043
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: fogti <fogti+devel@ytrizja.de>
Reviewed-by: tazjin <tazjin@tvl.su>
This is no longer needed; Nixery is now served by bugry.
Change-Id: Idd072505c4da1e6af636224e092b6fb21eff9250
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13001
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
We switched to stable temporarily in 2023 (!) because of some breakage that has
long been fixed.
In general, running nixery against stable is probably advisable, but because of
our Lisp package situation updating stable is not possible at the moment.
Change-Id: I122ac63d6307cab76a3069101682fc5f8f985914
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12999
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Running Nixery on bugry is much more cost efficient (better traffic economics
than on a cloud provider, and Nixery is mostly a traffic-heavy service), and
frees up my Yandex Cloud credits for adding another builder.
Change-Id: Id6c8c76b28a5ce13cc8b743ad6e72fffd19353fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12997
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
I thought this was enabled and got confused when deploying ... cache should
always be enabled on machines that don't build themselves.
Change-Id: Ie52b27c44db4c26387b05553dbe36f7693628e89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12993
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Configures an experimental setup for a builderball-based public cache.
This cache only includes the two build machines (whitby & nevsky), for the time
period where both of them exist simultaneously.
The idea is this:
All participating hosts run a harmonia binary cache locally (whitby already
does). They then run builderball instances pointing at each other's harmonia
caches (through dedicated public hostnames).
When a request comes in, the first matching cache address is returned and Nix
will substitute from there.
Change-Id: Ia7d5357fd5e04f77b460205544fa24e82b100230
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12975
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Otherwise pushes to Github from CI will fail.
Change-Id: Ib3eb3165577cb98c5a7d5f2055b09dbf118da6c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12994
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Adman (the hoster) have not provided an ETA for native v6 on bugry yet, so we
establish a public v6 connection through nevsky for now.
In traffic flows going West->East the overhead is minimal (a few ms), though I
guess it might be worse if you're in the middle (Yekaterinburg or something).
The prefix was chosen by the bugry public v4 address encoded in hex, and
appended to the nevsky prefix.
Change-Id: I133622c17bd02eade0a6febc6bdf97f403fed14c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12974
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
