Commit graph

16 commits

Author SHA1 Message Date
Florian Klink
8400e523ce refactor(tvix/castore/blob): use near/far for CombinedBlobService
Align this naming with CachePathInfoService.

Change-Id: Ib9a0d73b8ca57a93e9fc027ae907fc6ed370842a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12751
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-11-11 18:46:33 +00:00
Florian Klink
a9f453f6da docs(tvix/[ca]store): improve docstrings, remove wildcard imports
Extend the docstrings of `add_default_services`, and add one for
`addrs_to_configs` as well as the module-wide one at
`tvix_store::composition`.

Change-Id: Ie9b449988eb210cd65b19b174094bbe0c4af2fd6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12748
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-11-11 18:46:33 +00:00
Florian Klink
e71a857ec8 refactor(tvix/[ca]store): rename store composition feature flags
tvix-castore already supports composition without any additional feature
flags, the only thing that can be explicitly enabled is referring to
other stores via an anonymous url. Rename that feature flag to
"xp-composition-url-refs".

tvix-store effectively only controls the CLI surface, so rename this to
"xp-composition-cli".

The "store" in the feature name was dropped, as it's already apparent
from being in the tvix-[ca]store crate.

Change-Id: I1175dc6280cbba4cbcdfb7fd4b35fce713b45fc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12747
Tested-by: BuildkiteCI
Reviewed-by: yuka <yuka@yuka.dev>
2024-11-11 18:46:33 +00:00
Florian Klink
d505f03e00 refactor(tvix/store/composition): rename 'default' to 'root'
This becomes the root of the composition. `default` implies we can
directly access anything else, which we cannot. `root` makes this more
understandable, and it's all internal only anyways.

Change-Id: I297511bc05a7c32c59510b9d192b40d1bd937b5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12746
Reviewed-by: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-11-11 18:46:33 +00:00
Florian Klink
f0d594789e refactor(tvix/castore): remove remaining lazy_static usage
Change-Id: I86480cf625a457c4aa8153262f829d34c230b084
Co-authored-by: edef <edef@edef.eu>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12613
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
2024-10-17 21:57:13 +00:00
Yureka
52bb3c6d02 feat(tvix/composition): allow urls as anonymous stores
This allows specifying an url in place of a named reference to another
composition entry, if the castore crate has been compiled with the
 xp-store-composition feature.

Example: `--directory-service-addr cache://?near=memory://&far=memory://`

This would be equivalent to the instantiation via toml file:

```toml
[memory1]
type = "memory"

[memory2]
type = "memory"

[default]
type = "cache"
near = "memory1"
far = "memory2"
```

Note that each anonymous url causes a distinct instance to be created.

Change-Id: Iee5a07a94b063b5e767c704d9cad0114fa843164
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12146
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-10-10 12:53:08 +00:00
Yureka
8342f70c60 feat(tvix/composition): improve error message for unknown tag
Change-Id: I61a31488de17725ae4311f5f5bf8e02edb752cf9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12017
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-22 13:36:08 +00:00
Yureka
e408783bac test(tvix/composition): add recursion and concurrent tests
Change-Id: Ic10773b08d940b45c3067bd514c3080cadac9606
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12003
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 21:18:17 +00:00
Yureka
eea9f3d9ee docs(tvix/composition): add comment about stack
Change-Id: I3888b5034c53728e2f9cfe24213f8854eb38bfe1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12005
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 18:37:50 +00:00
Yureka
76394daca3 feat(tvix/composition): downcast boxed error if possible
We still have the unique store name to identify which instantiation caused the error. For recursion errors, the full chain is still retained inside the CompositionError.

Change-Id: Iaddcece445a5df331e578d7c69d710db3d5f8dcd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12002
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
Reviewed-by: flokli <flokli@flokli.de>
2024-07-21 17:03:27 +00:00
Yureka
a805a80b48 fix(tvix/composition): include typeid in recursion check
Change-Id: Icc279d41a4980d4b57acbb4243bbd509039b753f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12000
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-21 13:20:00 +00:00
Yureka
8b77c7fcd7 refactor(tvix/store): use composition in tvix_store crate
Change-Id: Ie6290b296baba2b987f1a61c9bb4c78549ac11f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11983
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-07-20 19:37:27 +00:00
Yureka
4801d6bf85 feat(tvix): different service types in one composition
This will be necessary for the PathInfoService composition, as some
PathInfoService implementations require a BlobService & DirectoryService
to ingest into.

Using the Extend trait for creating compositions allows extending the same
composition with configs of various types e.g. BlobStore, DirectoryStore
Generics are moved from the Composition struct to the functions.The storage of
the InstantiatonStates uses the TypeId in the key and a Box<dyn Any> in the
value, which is downcasted to InstantiatonState<T>.

Change-Id: I2af11f26c535029adfb1c62905e0e7c4aaed7b51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11980
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
2024-07-19 15:13:37 +00:00
Yureka
168e4fda59 refactor(tvix): use composition & registry for from_addr
Change-Id: I3c94ecb5958294b5973c6fcdf5ee9c0d37fa54ad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11976
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: yuka <yuka@yuka.dev>
2024-07-18 19:19:26 +00:00
Yureka
b8415a94d9 feat(tvix/composition): allow creating a blank CompositionContext
this is useful when oneshot-instantiating a store from a single config

Change-Id: I08538fdee1d0bb26b3ae2da7d3b2339b2e93bc0a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11975
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: yuka <yuka@yuka.dev>
Tested-by: BuildkiteCI
2024-07-18 18:47:42 +00:00
Yureka
1a6b6e3ef3 feat(tvix/castore): add composition module
Change-Id: I0868f3278db85ae5fe030089ee9033837bc08748
Signed-off-by: Yureka <tvl@yuka.dev>
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11853
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-07-18 10:42:11 +00:00