feat(tvix/nar-bridge): avoid unnecessary NAR uploads

When uploading a Store Path to a Nix HTTP Binary Cache, Nix first does a
HEAD request for $outhash.narinfo, and if that's not found, for
`{narhash}.nar[.compression_suffix]`.

If the NAR is already present, only the NARInfo is uploaded.

Even though we don't have a service allowing to globally look up from
NARHash to root node, `root_nodes` in `AppState` at least contains
recently uploaded NARHashes.

If we find it in there, we can prevent Nix unnecessarily uploading NARs
if the same contents have already been recently uploaded.
We also promote this key, chances are high Nix will subsequently upload
a NARInfo referring to this NARHash.

Change-Id: I34e3fd9b334b695abe945e64cd291e30f303c2a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12752
Tested-by: BuildkiteCI
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
This commit is contained in:
Florian Klink 2024-11-09 15:45:06 +00:00 committed by clbot
parent 8400e523ce
commit 85de9b8dab
2 changed files with 32 additions and 8 deletions

View file

@ -53,10 +53,8 @@ impl AppState {
pub fn gen_router(priority: u64) -> Router<AppState> {
Router::new()
.route("/", get(root))
// FUTUREWORK: respond for NARs that we still have in root_nodes (at least HEAD)
// This avoids some unnecessary NAR uploading from multiple concurrent clients, and is cheap.
.route("/nar/:nar_str", get(four_o_four))
.route("/nar/:nar_str", head(four_o_four))
.route("/nar/:nar_str", head(nar::head_root_nodes))
.route("/nar/:nar_str", put(nar::put))
.route("/nar/tvix-castore/:root_node_enc", get(nar::get_head))
.route("/nar/tvix-castore/:root_node_enc", head(nar::get_head))