We previously used to calculate the transitive input closure of builds
using eval state, i.e. based on known_paths from the drv that is being
built. This caused had 2 issues:
* The build included a lot of unnecessary build time dependencies of drv's that
the build depends on in addition to runtime dependencies.
* Some runtime dependencies were missing causing the build to fail, see #106
This implementation uses only runtime dependencies of direct
dependencies and makes sure to include any transitive runtime dependencies,
this is achieved by querying path_info_service for "references".
fixed#106
Change-Id: Id734bed7b0cf50e2dac96501a9bc70655ed15054
Reviewed-on: https://cl.snix.dev/c/snix/+/30308
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
builtins.toXML relies on attr keys being sorted, which is an
implementation detail.
Cppnix creates sorted xml, and tests expect it too, so changed that
iter call site.
builtins.attrNames and attrValues should also returns sorted lists, so
changed that too.
Change-Id: Ic706afc6855d5d28932ade7482449c965b0ae94f
Reviewed-on: https://cl.snix.dev/c/snix/+/30307
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
This is a small utility that allows ingesting a given path or .tar
file content into the snix-castore and returns the B3Digest of the
root node. Another subcommand takes this hash to mount the content
back as a virtiofs or FUSE drive.
This works as-is, but I discovered issue #107 while working on it.
Change-Id: I11df73e39ab0db6f3868effab9bde4f090eadcb5
Reviewed-on: https://cl.snix.dev/c/snix/+/30293
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
subuid/subgids used to be hardcoded, which resulted in build failures
if those did not match the ones of the effective user.
fixes#86
Change-Id: I3b0c3e9ef710aa9e3de998891abe10fd1a893189
Reviewed-on: https://cl.snix.dev/c/snix/+/30301
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
This provides a canonical way to parse a `B3Digest`, to be used in,
e.g., a CLI like `snix-castore`.
Change-Id: If6c7858c1fcd7645721002be7789f5ce2ff97a5c
Reviewed-on: https://cl.snix.dev/c/snix/+/30303
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
This prevents the SVG from overflowing into the "On this page" sidebar
on the right. We apparently still need to set a height.
Change-Id: Iae4183f92115b3190195a5f524bff18b18ad4d39
Reviewed-on: https://cl.snix.dev/c/snix/+/30297
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: Florian Klink <flokli@flokli.de>
This is sufficiently covered in //web, or simply outdated.
Change-Id: Ic84c97ca88dc1351d564861c573dc80939442f1b
Reviewed-on: https://cl.snix.dev/c/snix/+/30300
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: Florian Klink <flokli@flokli.de>
This doesn't document any of Snix, but is a leftover from the TVL.
It probably should have been deleted as part of
df4500ea2b.
Change-Id: I1be5a743b66d786a1b6861d2f0d9e325b7be5f7c
Reviewed-on: https://cl.snix.dev/c/snix/+/30298
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
In the castore Rust types, nodes don't have names, they only get names
my the mapping happening in a Directory struct.
However, in proto land, they do, as Directories are not maps, but three
(individually sorted) lists.
This has caused some confusion in the past. Let's fix this, by renaming
what used to be proto nodes to "Entry".
We also use this "entry" in a very similar fashion in the NAR reader,
describing something with a name, so this should be more consistent
and understandable. There's no change in the wire representation.
Change-Id: Ie6184d9a6e00c8114fc2a46bfd2bc90208a1d623
Reviewed-on: https://cl.snix.dev/c/snix/+/30296
Tested-by: besadii
Reviewed-by: Vova Kryachko <v.kryachko@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
This mostly matches the default configuration, but notably does not
make the lastName field mandatory, in order to accommodate mononymy.
Change-Id: I47ca86a179eb9b7dcf5f3e761681c78e22f5265c
Fixes: https://git.snix.dev/snix/snix/issues/104
Reviewed-on: https://cl.snix.dev/c/snix/+/30289
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Just a small cosmetic change to update the ASCII art in init
script and README!
I refrained myself from trying to draw an energy bar as a logo ^^'
Change-Id: Ifbc6e05f2ab3d8686ba844aa0b91ab1644b29e46
Reviewed-on: https://cl.snix.dev/c/snix/+/30294
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
This makes mounting the content of a just ingested archive or
directory more straightforward.
Change-Id: I64589aad3dc1885c1266310200cdace21e5810e9
Reviewed-on: https://cl.snix.dev/c/snix/+/30292
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Nix clients still might have old .narinfo files cached, containing old
NAR URLs. Send a redirect to the new URL.
Fixes: #103
Change-Id: Ie3b77e4fdc4be0f982e023f2a2acd3f9f0257f9b
Reviewed-on: https://cl.snix.dev/c/snix/+/30291
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: edef <edef@edef.eu>
Due to Keycloak not being set up for mononymy just yet, Gerrit currently
submits changes as "edef .", so we mailmap it back to the proper form.
Change-Id: I8a84e9310dde709bc9c2710c02db99986f14e84a
Reviewed-on: https://cl.snix.dev/c/snix/+/30283
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
Deploys Harmonia on build01, proxied through public01.
We cannot serve from build01 directly because it only supports IPv6.
Closes: https://git.snix.dev/snix/snix/issues/66
Change-Id: Iff3c16366d60c0fbfd1315a18c27fcd636a0261a
Reviewed-on: https://cl.snix.dev/c/snix/+/30274
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
[se-StorePathName] was missing in the footnotes, and links to sets were
wrong.
Also rename serialization.md to types.md to match the title.
Change-Id: I8e602df1958cf5354c6fd3a5c16cbe859a0549b9
Reviewed-on: https://cl.snix.dev/c/snix/+/30275
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Brian Olsen <brian@maven-group.org>
While processing `AddToStore` operation, it is possible to receive a
framed derivation as a response instead of a NAR.
Due to how the wire protocol works, it's not easy to know if we are done
with pulling more bytes (i.e. incomplete derivation) or not (i.e.
incorrect derivation).
To solve this, a streaming parser would propagate the incompleteness of
the parse as a valid response and the reader would be requested more
bytes until it returns EOF.
Change-Id: Icd2b311f03ad68010a9b48e883f6dbee2fbd2c3e
Signed-off-by: Raito Bezarius <raito@lix.systems>
Reviewed-on: https://cl.snix.dev/c/snix/+/30278
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Autosubmit: Ryan Lahfa <masterancpp@gmail.com>
Previously drv's were fed into the build as opposed to their
dependencies.
Additionally this change refactors input propagation in a way that
ensures both:
* We only construct inputs *once* and use for both build inputs and
refscan needles
* Inputs and refscan needles have consistent ordering and fixes#89
Change-Id: Id53701cea48598a0f73c6abd01293a02b71fb2d8
Reviewed-on: https://cl.snix.dev/c/snix/+/30240
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
We only use the OAuth flow (with Keycloak), and the native login
mechanism is an unnecessary source of user confusion.
Change-Id: I819e0b6ac507013c903c55a28f0db52e8706d8dc
Reviewed-on: https://cl.snix.dev/c/snix/+/30282
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Reviewed-by: Florian Klink <flokli@flokli.de>
Autosubmit: edef . <edef@edef.eu>
We use hashbrown directly (instead of through std::collections::HashMap)
so that we can use HashMap::entry_ref, which only allocates if the entry
is not yet occupied.
This implicitly switches our hash to a less long-term DoS-resistant one
(foldhash rather than SipHash), but we don't usually face HashDoS risks
when ingesting archives.
Change-Id: I3e7fc2cd08d96380cd9fd62bfcfe6cd24698bc9a
Reviewed-on: https://cl.snix.dev/c/snix/+/30277
Reviewed-by: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Destructuring by value gives us ownership of the data we need,
without needing unnecessary clones. We previously avoided this
because we wanted to call IngestionEntry::path, but inlining
that into our match is cleaner.
Change-Id: Id58075a98929306e99706746b6e4a9f961a24faf
Reviewed-on: https://cl.snix.dev/c/snix/+/30239
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
This moves the daemon protocol reference to the "Reference" section in
our new docs.
It also updates them a bit, we now use `{{< relref "path/to/file.md" >}}`
to refer to other files, and fixes the (previously broken) references
in handshake.md.
Change-Id: I114e40622dc504e7a5f75946895900e797a1e722
Reviewed-on: https://cl.snix.dev/c/snix/+/30273
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Brian Olsen <brian@maven-group.org>
These only contain placeholder text for now.
Change-Id: I377dc4f2bdec8911ece74c3914608289f3e51fdc
Reviewed-on: https://cl.snix.dev/c/snix/+/30272
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Brian Olsen <brian@maven-group.org>
This has already been migrated to //web.
Change-Id: I1a6e1cbe2866be9519928f77acfabc96988ccf25
Reviewed-on: https://cl.snix.dev/c/snix/+/30271
Tested-by: besadii
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
This has already been migrated to //web.
Change-Id: Ica2ac77bdaad6ac2906ab3c93ac6d541c7abd6fc
Reviewed-on: https://cl.snix.dev/c/snix/+/30270
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
This has already been migrated to //web.
Change-Id: If816a24104ece30ae0826529a9d4df4c4ec3c442
Reviewed-on: https://cl.snix.dev/c/snix/+/30269
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
This has already been migrated to //web.
Change-Id: I09f4b405795d94f3c2d6542610d6e38c53b13531
Reviewed-on: https://cl.snix.dev/c/snix/+/30268
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
This is already in contact.md in //web.
Change-Id: Ic8112318fdb2ea7c8f61f0c72f24b81c6a629d61
Reviewed-on: https://cl.snix.dev/c/snix/+/30266
Reviewed-by: Ryan Lahfa <masterancpp@gmail.com>
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
Absorb some of the paragraphs from snix/docs/src/architecture.md, as
well as the blog post around the builder protocol, and extend it with a
bit more context.
The parts about the Evaluator are largely outdated (see the warning
above in that file), and otherwise already covered.
Update and restructure protocol.md to describe the goals of the builder.
Change-Id: I26ed625f7b6fcfded3e65cf87b00ceb33b75f229
Reviewed-on: https://cl.snix.dev/c/snix/+/30264
Reviewed-by: edef . <edef@edef.eu>
Tested-by: besadii
Autosubmit: Florian Klink <flokli@flokli.de>
It seems this is its own Markdown context of some sort, so that
hyperlink url needs to be moved inside to work.
Change-Id: Id35fdf97ccc7a5ed087360d27a2cda3a7df0c850
Reviewed-on: https://cl.snix.dev/c/snix/+/30265
Tested-by: besadii
Reviewed-by: Vova Kryachko <v.kryachko@gmail.com>
Autosubmit: Florian Klink <flokli@flokli.de>
What we're doing is safe on POSIX, but not elsewhere, so we should use
the Unix-only safe OsStr::from_bytes method rather than adding unsafety.
Change-Id: Ib53edaec77a1a0aa2663287d973d398274fde79a
Reviewed-on: https://cl.snix.dev/c/snix/+/30263
Tested-by: besadii
Reviewed-by: Florian Klink <flokli@flokli.de>
Previously, the buildkite users were not able to traverse there.
Removing /nix/var/nix/gcroots/buildkite/canon might not be needed, and
is racy with other anchor step - the first one might still be building
`ci.gcroot` (and didn't create the new symlink), so the second one will
fail trying to remove the non-existing symlink.
Change-Id: I0449447f7193113d807d597750b26c7beb48a3a6
Reviewed-on: https://cl.snix.dev/c/snix/+/30257
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Add a bit more information, callouts to prevent people potentially
bricking their laptops.
Also change the performance section, we will
Change-Id: Id516c4bb0f0c2cbe99db86199b91c7d68becdd44
Reviewed-on: https://cl.snix.dev/c/snix/+/30254
Autosubmit: Florian Klink <flokli@flokli.de>
Tested-by: besadii
Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
