maurice/snix
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(tvix/castore): simple filesystem blob service

Browse source 
The simple filesystem `BlobService` enable a user to write blob store
on an existing filesystem using a prefix-style layout in the provided root directory,
e.g. the two first bytes of the blake3 hashes are used as directories prefixes.

Change-Id: I3451a688a6f39027b9c6517d853b95a87adb3a52
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10071
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This commit is contained in:
Ryan Lahfa 2023-12-17 01:22:01 +01:00 • committed by clbot
parent 923a5737e6
commit 0ae32d45f6
6 changed files with 316 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

11
tvix/castore/src/blobservice/from_addr.rs
View file

@ -3,7 +3,9 @@ use url::Url;
use crate::{proto::blob_service_client::BlobServiceClient, Error};
use super::{BlobService, GRPCBlobService, MemoryBlobService, SledBlobService};
use super::{
BlobService, GRPCBlobService, MemoryBlobService, SimpleFilesystemBlobService, SledBlobService,
};
/// Constructs a new instance of a [BlobService] from an URI.
///
@ -11,6 +13,7 @@ use super::{BlobService, GRPCBlobService, MemoryBlobService, SledBlobService};
/// - `memory://` ([MemoryBlobService])
/// - `sled://` ([SledBlobService])
/// - `grpc+*://` ([GRPCBlobService])
/// - `simplefs://` ([SimpleFilesystemBlobService])
///
/// See their `from_url` methods for more details about their syntax.
pub async fn from_addr(uri: &str) -> Result<Arc<dyn BlobService>, crate::Error> {
@ -54,6 +57,12 @@ pub async fn from_addr(uri: &str) -> Result<Arc<dyn BlobService>, crate::Error>
// Constructing the channel is handled by tvix_castore::channel::from_url.
let client = BlobServiceClient::new(crate::tonic::channel_from_url(&url).await?);
Arc::new(GRPCBlobService::from_client(client))
} else if url.scheme() == "simplefs" {
if url.path().is_empty() {
return Err(Error::StorageError("Invalid filesystem path".to_string()));
}
Arc::new(SimpleFilesystemBlobService::new(url.path().into()).await?)
} else {
Err(crate::Error::StorageError(format!(
"unknown scheme: {}",