feat(tvix/castore/directory): add bigtable backend

This adds a Directory service using
https://cloud.google.com/bigtable/docs/ as a K/V store.

Directory (closures) are put in individual keys.

We don't do any bucketed upload of directory closures (yet), as castore/
fs does query individually, does not request recursively (and buffers).
This will be addressed by store composition at some point.

Change-Id: I7fada45bf386a78b7ec93be38c5f03879a2a6e22
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11212
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
This commit is contained in:
Florian Klink 2024-03-19 12:12:03 +02:00 committed by clbot
parent 84ad8a0bbd
commit 17849c5c00
12 changed files with 2767 additions and 22 deletions

View file

@ -19,10 +19,12 @@ pkgs.mkShell {
pkgs.cargo
pkgs.cargo-machete
pkgs.cargo-expand
pkgs.cbtemulator
pkgs.clippy
pkgs.evans
pkgs.fuse
pkgs.go
pkgs.google-cloud-bigtable-tool
pkgs.grpcurl
pkgs.hyperfine
pkgs.mdbook