snix/tvix/castore/src
Florian Klink c92ef2df64 feat(tvix/castore/directory): add ClosureValidator
This can be used to validate a Directory closure (connected DAG of
Directories), and their insertion order.

Directories need to be inserted (via `add`), in an order from the leaves
to the root. During insertion, we validate as much as we can at that
time:

 - individual validation of Directory messages
 - validation of insertion order (no upload of not-yet-known Directories)
 - validation of size fields of referred Directories

Internally it keeps all received Directories (and their sizes) in a HashMap,
keyed by digest.

Once all Directories have been inserted, a drain() function can be
called to get a (deduplicated and) validated list of directories, in
from-leaves-to-root order (to be stored somewhere).

While assembling that list, a check for graph connectivity is performed
too, to ensure there's no separate components being sent (and only one
root).

It adds a test suite for these cases, which is much nicer to test than
where we previously had these checks (only in the gRPC server wrapper).

Followup CLs will move the existing putters to use this.

Change-Id: Ie88c832924c170a24626e9e3e91d868497b5d7a4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11220
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
2024-03-24 17:39:49 +00:00
..
blobservice feat(tvix/castore/blob): document missing objectstore+*:// URL 2024-03-20 12:17:42 +00:00
directoryservice feat(tvix/castore/directory): add ClosureValidator 2024-03-24 17:39:49 +00:00
fs feat(tvix/castore): Compile fix for Darwin 2024-02-19 17:14:24 +00:00
proto docs(tvix/castore): fix missing slash in docstring 2024-03-23 22:06:27 +00:00
tests refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …> 2024-01-09 14:08:22 +00:00
digests.rs feat(tvix/castore): impl Debug for B3Digest 2024-03-17 22:18:20 +00:00
errors.rs feat(tvix/castore): impl From<std::io::Error> for Error 2023-11-18 19:23:47 +00:00
fixtures.rs refactor(tvix): condense long bytestrings 2023-10-27 07:37:26 +00:00
hashing_reader.rs feat(tvix/castore): add HashingReader, B3HashingReader 2024-03-03 15:31:31 +00:00
import.rs fix(tvix/castore): don't emit ret as INFO 2024-02-18 07:12:27 +00:00
lib.rs feat(tvix/castore): add HashingReader, B3HashingReader 2024-03-03 15:31:31 +00:00
tonic.rs refactor(tvix/castore/tonic): use match in channel_from_url 2023-11-17 14:01:26 +00:00
utils.rs refactor(tvix/castore/tests): let gen_*_service return Boxes 2024-01-01 14:45:17 +00:00