This error type is a bit less annoying to use in other errors, as it
doesn't hang onto bytes from the input.
Change-Id: I4b969717d443f5e04dd2086baeb946a98c4f9d7c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13206
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
This makes it possible for external consumers to deal with this error.
Change-Id: I70b16efb99c6a98c75b1672ed0618ac963f5f79d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13194
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
This adds code allowing to parse (and emit) the "internal-json" log
format that Nixcpp emits.
Move the Verbosity struct from worker_protocol.rs out into
src/log/mod.rs, it's not only used there.
Change-Id: Ia35e739c83e06080a677b576616bdd0d4a821cf4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13167
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
Reviewed-by: edef <edef@edef.eu>
See 72dd5818b7/CHANGELOG.md
for the nom changelog.
Most notably, there's now a .parse() to be added:
`combinator(arg)(input)` -> `combinator(arg).parse(input)`
There also doesn't need to be a tuple combinator (it's implemented on
tuples directly).
This also refactors the string / byte field parsing parts, to make them
more concise.
Change-Id: I9e8a3cedd07d6705be391898eb6a486fb8164069
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13193
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: Brian Olsen <me@griff.name>
We're not saying 2.3 here anymore in all cases, but use the version
specified in the `nix_version` function argument.
Change-Id: I86c34f97eea4141b8431de7c6a784750d6b7c2cb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13166
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Brian Olsen <me@griff.name>
This is probably a nicer default, and also aligns more with Serialize.
Change-Id: I3a8f18607c305dbf9fb93a2e2dd02ad07cd99f24
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13161
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Included changes:
* targeted primary NixOS channel back at `nixos-unstable`
* //3p/chicago95: disable new broken symlink check
* //tvix: omit unnecessary lifetime clippy recommends
* //users/sterni/blipqn: wait a bit for data to arrive in test so it
succeeds under load.
* //fun/paroxysm: force pkg-config flag of pq-sys by adding a bogus
dependency on it. Otherwise, pq-sys will try to use pg_config
which does not work correctly in pkgs.libpq at the moment.
* //users/flokli/keyboards/dilemma: disable temporarily
Change-Id: I6d53bd7bca6886f3457e1f41505e97314f4cd191
Reviewed-on: https://cl.tvl.fyi/c/depot/+/13119
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: aspen <root@gws.fyi>
Reviewed-by: sterni <sternenseemann@systemli.org>
We aim to produce bit-identical output when roundtripping, and this
applies to legacy formats as well.
Change-Id: Iaec7d6bb5c5e305ec5e1b78c6968226dee9a0d90
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12943
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
We know these names statically, so we can just use &'static str.
Change-Id: I81cb7ecc4d7553f57baca74464c120a143586fe6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12941
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Last one of the year! С наступающим)
Fixes:
* users/wpcarro: remove use-package from emacs packages (it has been built-in
for a while now)
* users/sterni: the same thing
* users/aspen: remove `coz`, forwardport `gdmap` from stable
* users/flokli: dropped corneish_zen firmware from CI
This firmware depends on a non-reproducible FOD which, when updated, causes
build failures. We have worked around this repeatedly, but it needs to be
fixed properly.
* tvix: regenerate Go protobufs
* tvix: address new clippy lints
* tvix/{castore,store,build}-go: update grpc/protobuf libraries
* tvix/eval: formatting fixes
* 3p/overlays/tvl: work around GCC 14 -Werrors
Change-Id: Ice5948ca7780192fb7d2abc6a48971fb875f03c9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12933
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Use dotted keys where we'd else have maps with a single key.
Change-Id: I9389e0fedddad1cf65f870a3a68415a0defaa259
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12929
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
There's more than one result in there.
Change-Id: I5d519db51fda050ed293bfb52215a643882e0116
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12879
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This operation is particularly used when invoking the following
nix commands:
```
nix-store --add-fixed some-path
nix-store --add-fixed --recursive some-path
```
Change-Id: I0f9b129c838c00e10415881f1e6e0d7bc1d7a3a6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12800
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
When sending nars over the wire to the nix-daemon, nix protocol versions
>= 1.23 use this framing protocol.
This change implements an AsyncRead for this protocol, to be used in
AddToStoreNar and any other operations when necessary.
Change-Id: I5f7972fe1c9ea145780bf449321bd3efeb833d18
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12814
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
When sending nars over the wire to the nix-daemon, nix protocol versions
1.21 to 1.23 use this framing protocol.
This change implements an AsyncRead for this protocol, to be used in
AddToStoreNar and any other operations when necessary.
Change-Id: I571f1adbb2343c14c98503d1a2c12eea4c783ec9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12813
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
These are required to support certain nix's local-overlay store
operations, it's safer to return empty results for these
operations than failing with "operation not implemented" errors.
Change-Id: Ic9b69d75dd52af5a826bfb6a8b283b082a0f6bcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12766
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This change contains 2 improvements to the tvix-store copy command:
1. Allows reading the reference graph from stdin, using `-` argument
2. Supports json representation produced by `nix path-info --json`
command.
In general it makes is easier and faster to import arbitrary closures
from an existing nix store with e.g the following command:
```
nix path-info ./result --json --closure-size --recursive | \
jq -s '{closure: add}' | \
tvix-store copy -
```
Change-Id: Id6eea2993da233ecfbdc186f1a8c37735b686264
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12765
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
By manually writing out the async function, and moving the owned String
we can avoid for S to be Sync in the NixSerialize for StorePath<S> impl.
Co-Authored-By: edef@edef.eu
Change-Id: I8427b38d2bd61de2818088728cbad79cda69d17d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12764
Reviewed-by: Vladimir Kryachko <v.kryachko@gmail.com>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Implement PartialEq/Eq ourselves instead of deriving, by proxying to
name.as_ref() (and digest of course).
Also implement Hash on our own, clippy doesn't like this to be derived,
while Eq/PartialEq is not.
Change-Id: Idbe289a23ba3bc8dabf893d4d8752792ae2778c3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12744
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
This change includes only the basic nix handshake protocol handling and
sets up a client session. The only supported operation at this point is
SetOptions.
Additional operations will be implemented in subsequent cls.
Change-Id: I3eccd9e0ceb270c3865929543c702f1491768852
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12743
Autosubmit: Vladimir Kryachko <v.kryachko@gmail.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Reviewed-by: Brian Olsen <me@griff.name>
This groups most `wire` feature gated logic into a single module.
The nix_daemon module will be gated by a feature that adds
nix-compat-derive as a dependency.
All of this is a way to break the crate2nix dependency cycle between
nix-compat and nix-compat-derive(which depends on nix-compat for its
doctests).
Change-Id: I95938a6f280c11967371ff21f8b5a19e6d3d3805
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This change implements the serialization part that is needed to
implement the nix daemon protocol. Previously was add deserialization
and derivers for that and this then adds the other part of that equation
so that you can write types that can then be read using deserialization.
Change-Id: I2917de634980a93822a4f5a8ad38897b9ce16d89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12729
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Existing handshake behavior assumed that the server version is always
at least as new as the client. Meaning that the client's version was
always picked the handshake details as well as for further communication
This change removes that assumption and correctly uses
min(server_version, client_version).
Change-Id: Ia5dad4613dd5f69a0aeb6c9d86982f1f36fe1a4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12722
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This is the only (remaining) occurence of it, and not really
more code than just calling store_path::build_ca_path with
`CAHash::Nar(NixHash::Sha256(…))`, especially considering we need the
CAHash in the PathInfo struct later anyways - so let's remove this
function.
Change-Id: Ia82212086062c366e0280ca0823d9e68a3f91d3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12632
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
These are not necessarily strings, and making it paths allows us to stop
converting them to lossy strings.
Change-Id: I11366c721dc5da1778aafe89092a1966b5a43178
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12617
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Make this generic on the StorePath<SP> that's being used, similar to the
other functions in there.
Change-Id: I453d1fd3749053d4e5aca156abc18da1f95ca264
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12616
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Jörg Thalheim <joerg@thalheim.io>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
This is now supported in the standard library via std::sync::LazyLock, but
requires some manual shuffling around of code.
I found at least one dead variable along the way, which I deleted.
Change-Id: I8600c87c49078fb5ff72671994c77b919259e67b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12608
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
We also use S in other places in the same file, but that's for the
string-like references.
SP is now consistently used as the type parameter for StorePath<_> (and
build_output_path) gets support for it).
By being a bit more careful in the order of assignments in nix-compat/
src/derivation, we can nudge the compiler to use the type we want.
Change-Id: Ia7c298e110dff98d3b113d2388674ce9e22b80e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12590
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com>
Tested-by: BuildkiteCI
This switches the PathInfoService trait from using the proto-derived
PathInfo struct to a more restrictive struct, and updates all
implementations to use it.
It removes a lot of the previous conversion and checks, as invalid
states became nonrepresentable, and validations are expressed on the
type level.
PathInfoService implementations consuming protobuf need to convert and
do the verification internally, and can only return the strongly typed
variant.
The nix_compat::narinfo::NarInfo conversions for the proto PathInfo
are removed, we only keep a version showing a NarInfo representation for
the strong struct.
Converting back to a PathInfo requires the root node now, but is
otherwise trivial, so left to the users.
Co-Authored-By: Florian Klink <flokli@flokli.de>
Change-Id: I6fdfdb44063efebb44a8f0097b6b81a828717e03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12588
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This is useful for building other NAR writers which use custom (async or optimized) I/O to write the blob parts of the NAR.
Change-Id: I447c09914fb0c99044e2fa910d4213660dc51c64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12437
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
We always read the length before returning a BytesReader, so len()
cannot be called before the length is read.
Change-Id: Ifa1cbc2c0923dd24d59e0af5e135ab8ed7314aa9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12453
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This will later be used to allow the user to access the original writer.
Change-Id: I1734600c5d580d18b16727f892cd2f6335cc9459
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12442
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
In https://cl.tvl.fyi/c/depot/+/12389 I accidentally deleted some
comments. Also fixed some useless no-default-features with futures crate, making it match the previous feature combination.
Change-Id: I72bb2cfb88719ff2f8812d90193de2bd49149cce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12395
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
From now on we will add the dependencies and their version in the root
Cargo.toml and in order to enable the dependency for a workspace member
we set `workspace = true` in the member's Cargo.toml.
Change-Id: I9738c1cf99810b7ace87ca712c3ea965ba846e25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12389
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
This updates all the dependencies and their "minimum" versions in
Cargo.{lock,toml} to the latest compatible version using `cargo-edit`'s
`cargo upgrade` command that will eventually be merged into `cargo
update`.
Change-Id: Iccb2aa4a1c84a0465222244a0bd0cafe2a82e781
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12388
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Tested-by: BuildkiteCI
The external feature flag was there because I couldn't find a way to
refer to crate and nix-compat with the same name so that the generated
code could be the same.
In essence `use nix_compat::nix_daemon:🇩🇪:NixDeserialize` is an error
when used inside nix_compat crate.
So my best fix was the external feature flag until I found the solution
used here which also removes the flag completely.
Change-Id: Ia3e89c6c350c3fb22ca87f974a39c21542aae152
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12376
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Brian Olsen <me@griff.name>
When only the wire feature was enabled two methods were unused and so
would faild `cargo check --no-default-features --features wire`.
This feature flags those two methods on async feature since that is the
only place they are used.
Change-Id: I6ec18a670e3c6e3ecee8d1417c99f1a5084e0ae7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12346
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
This adds a nix-compat-derive derive crate that implements a deriver
for NixDeserialize implementations. This is to reduce the amount of
code needed to implement deserialization for all the types used by
the Nix daemon protocol.
Change-Id: I484724b550e8a1d5e9adad9555d9dc1374ae95c2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12022
Autosubmit: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
Add a trait for deserializing a type from a daemon worker connection.
This adds the NixDeserialize trait which is kind of like the serde
Deserialize trait in that individual types are meant to implement it
and it can potentially be derived in the future.
The NixDeserialize trait takes something that implements NixRead as
input so that you can among other things mock the reader.
Change-Id: Ibb59e3562dfc822652f7d18039f00a1c0d422997
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11990
Autosubmit: Brian Olsen <me@griff.name>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
This adds the MIME types Nix uses when uploading files to a S3 bucket /
HTTP endpoint. It ignores them when retrieving.
Change-Id: I75fa96d5e53349f0470bbe8cbba93f961fbd5f05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12266
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
This moves the URL component parsing code we had in nar-bridge to
nix-compat.
We change the function signature to return an Option, not a
Result<_, StatusCode>.
This allows returning more appropriate error codes, as we can
ok_or(…) at the callsite, which we now do: on an upload to an
invalid path, we now return "unauthorized", while on a GET/HEAD, we
return "not found".
This also adds support to parse compression suffixes. While not
supported in nar-bridge, other users of nix-compat might very well want
to parse these paths.
Also fix the error message when parsing NAR urls, it mentioned 32, not
52, which is a copypasta error from the narinfo URL parsing code.
Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
