chore(tvix): move store/fs to castore/fs

With the recent introduction of the RootNodes trait, there's nothing in
the fs module pulling in tvix-store dependencies, so it can live in
tvix-castore.

This allows other crates to make use of TvixStoreFS, without having to
pull in tvix-store.

For example, a tvix-build using a fuse mountpoint at /nix/store doesn't
need a PathInfoService to hold the root nodes that should be present,
but just a list.

tvix-store now has a pathinfoservice/fs module, which contains the
necessary glue logic to implement the RootNodes trait for a
PathInfoService.

To satisfy Rust orphan rules for trait implementations, we had to add a
small wrapper struct. It's mostly hidden away by the make_fs helper
function returning a TvixStoreFs.

It can't be entirely private, as its still leaking into the concrete
type of TvixStoreFS.

tvix-store still has `fuse` and `virtiofs` features, but they now simply
enable these features in the `tvix-castore` crate they depend on.

The tests for the fuse functionality stay in tvix-store for now, as
they populate the root nodes through a PathInfoService.

Once above mentioned "list of root nodes" implementation exists, we
might want to shuffle this around one more time.

Fixes b/341.

Change-Id: I989f664827a5a361b23b34368d242d10c157c756
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
This commit is contained in:
Florian Klink 2023-12-17 00:16:22 +02:00 committed by flokli
parent 52cad86195
commit a5167c508c
18 changed files with 257 additions and 199 deletions

View file

@ -0,0 +1,84 @@
use futures::Stream;
use futures::StreamExt;
use std::ops::Deref;
use std::pin::Pin;
use tonic::async_trait;
use tvix_castore::fs::{RootNodes, TvixStoreFs};
use tvix_castore::proto as castorepb;
use tvix_castore::Error;
use tvix_castore::{blobservice::BlobService, directoryservice::DirectoryService};
use super::PathInfoService;
#[cfg(test)]
mod tests;
/// Helper to construct a [TvixStoreFs] from a [BlobService], [DirectoryService]
/// and [PathInfoService].
/// This avoids users to have to interact with the wrapper struct directly, as
/// it leaks into the type signature of TvixStoreFS.
pub fn make_fs<BS, DS, PS>(
blob_service: BS,
directory_service: DS,
path_info_service: PS,
list_root: bool,
) -> TvixStoreFs<BS, DS, RootNodesWrapper<PS>>
where
BS: Deref<Target = dyn BlobService> + Send + Clone + 'static,
DS: Deref<Target = dyn DirectoryService> + Send + Clone + 'static,
PS: Deref<Target = dyn PathInfoService> + Send + Sync + Clone + 'static,
{
TvixStoreFs::new(
blob_service,
directory_service,
RootNodesWrapper(path_info_service),
list_root,
)
}
/// Wrapper to satisfy Rust's orphan rules for trait implementations, as
/// RootNodes is coming from the [tvix-castore] crate.
#[doc(hidden)]
#[derive(Clone, Debug)]
pub struct RootNodesWrapper<T>(pub(crate) T);
/// Implements root node lookup for any [PathInfoService]. This represents a flat
/// directory structure like /nix/store where each entry in the root filesystem
/// directory corresponds to a CA node.
#[cfg(any(feature = "fuse", feature = "virtiofs"))]
#[async_trait]
impl<T> RootNodes for RootNodesWrapper<T>
where
T: Deref<Target = dyn PathInfoService> + Send + Sync,
{
async fn get_by_basename(&self, name: &[u8]) -> Result<Option<castorepb::node::Node>, Error> {
let Ok(store_path) = nix_compat::store_path::StorePath::from_bytes(name) else {
return Ok(None);
};
Ok(self
.0
.deref()
.get(*store_path.digest())
.await?
.map(|path_info| {
path_info
.node
.expect("missing root node")
.node
.expect("empty node")
}))
}
fn list(&self) -> Pin<Box<dyn Stream<Item = Result<castorepb::node::Node, Error>> + Send>> {
Box::pin(self.0.deref().list().map(|result| {
result.map(|path_info| {
path_info
.node
.expect("missing root node")
.node
.expect("empty node")
})
}))
}
}

File diff suppressed because it is too large Load diff

View file

@ -4,6 +4,9 @@ mod memory;
mod nix_http;
mod sled;
#[cfg(any(feature = "fuse", feature = "virtiofs"))]
mod fs;
use futures::Stream;
use std::pin::Pin;
use tonic::async_trait;
@ -18,6 +21,9 @@ pub use self::memory::MemoryPathInfoService;
pub use self::nix_http::NixHTTPPathInfoService;
pub use self::sled::SledPathInfoService;
#[cfg(any(feature = "fuse", feature = "virtiofs"))]
pub use self::fs::make_fs;
/// The base trait all PathInfo services need to implement.
#[async_trait]
pub trait PathInfoService: Send + Sync {