refactor(tvix/glue): move Fetch[er] into its own types, fetch lazily
We actually want to delay fetching until we actually need the file. A simple evaluation asking for `.outPath` or `.drvPath` should work even in a pure offline environment. Before this CL, the fetching logic was quite distributed between tvix_store_io, and builtins/fetchers.rs. Rather than having various functions and conversions between structs, describe a Fetch as an enum type, with the fields describing the fetch. Define a store_path() function on top of `Fetch` which can be used to ask for the calculated store path (if the digest has been provided upfront). Have a `Fetcher` struct, and give it a `fetch_and_persist` function, taking a `Fetch` as well as a desired name, and have it deal with all the logic of persisting the PathInfos. It also returns a StorePathRef, similar to the `.store_path()` method on a `Fetch` struct. In a followup CL, we can extend KnownPaths to track fetches AND derivations, and then use `Fetcher` when we need to do IO into that store path. Change-Id: Ib39a96baeb661750a8706b461f8ba4abb342e777 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11500 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
This commit is contained in:
parent
dc444e55dc
commit
091de12a9a
8 changed files with 538 additions and 399 deletions
|
|
@ -41,17 +41,17 @@ pub enum FetcherError {
|
|||
#[error("Invalid hash type '{0}' for fetcher")]
|
||||
InvalidHashType(&'static str),
|
||||
|
||||
#[error("Error in store path for fetcher output: {0}")]
|
||||
StorePath(#[from] BuildStorePathError),
|
||||
|
||||
#[error(transparent)]
|
||||
Http(#[from] reqwest::Error),
|
||||
}
|
||||
|
||||
impl From<FetcherError> for tvix_eval::ErrorKind {
|
||||
fn from(err: FetcherError) -> Self {
|
||||
tvix_eval::ErrorKind::TvixError(Rc::new(err))
|
||||
}
|
||||
#[error(transparent)]
|
||||
Io(#[from] std::io::Error),
|
||||
|
||||
#[error(transparent)]
|
||||
Import(#[from] tvix_castore::import::Error),
|
||||
|
||||
#[error("Error calculating store path for fetcher output: {0}")]
|
||||
StorePath(#[from] BuildStorePathError),
|
||||
}
|
||||
|
||||
/// Errors related to `builtins.path` and `builtins.filterSource`,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue