feat(tvix/castore/blobsvc): add chunks method

This adds support to retrieve a list of chunks for a given blob to the
BlobService interface.

While theoretically all chunk-awareness could be kept private inside
each BlobService reader, we'd not be able to resolve individual chunks
from different Blobservices - and due to this, not able to substitute
chunks we already have in a more local store.

This function allows asking a BlobService for the list of chunks,
leaving any actual fetching up to the caller (be it through individual
calls to open_read), or asking another store for it.

Change-Id: I1d33c591195ed494be3aec71a8c804743cbe0dca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10586
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
This commit is contained in:
Florian Klink 2024-01-09 15:00:01 +02:00 committed by clbot
parent 9596c5caff
commit 719cbad871
2 changed files with 47 additions and 2 deletions

View file

@ -1,6 +1,7 @@
use std::io;
use tonic::async_trait;
use crate::proto::stat_blob_response::ChunkMeta;
use crate::B3Digest;
mod from_addr;
@ -35,6 +36,22 @@ pub trait BlobService: Send + Sync {
/// Insert a new blob into the store. Returns a [BlobWriter], which
/// implements [io::Write] and a [BlobWriter::close].
async fn open_write(&self) -> Box<dyn BlobWriter>;
/// Return a list of chunks for a given blob.
/// There's a distinction between returning Ok(None) and Ok(Some(vec![])).
/// The former return value is sent in case the blob is not present at all,
/// while the second one is sent in case there's no more granular chunks (or
/// the backend does not support chunking).
/// A default implementation signalling the backend does not support
/// chunking is provided.
async fn chunks(&self, digest: &B3Digest) -> io::Result<Option<Vec<ChunkMeta>>> {
if !self.has(digest).await? {
return Ok(None);
} else {
// default implementation, signalling the backend does not support chunking.
return Ok(Some(vec![]));
}
}
}
/// A [tokio::io::AsyncWrite] that you need to close() afterwards, and get back