As described in issue #14, the registry API does not allow image names
with uppercase-characters in them.
However, the Nix package set has several top-level keys with uppercase
characters in them which could previously not be retrieved using
Nixery.
This change implements a method for retrieving those keys, but it is
explicitly only working for the top-level package set as nested
sets (such as `haskellPackages`) often contain packages that differ in
case only.
Google Cloud Storage supports granting access to protected objects via
time-restricted URLs that are cryptographically signed.
This makes it possible to store private data in buckets and to
distribute it to eligible clients without having to make those clients
aware of GCS authentication methods.
Nixery now uses this feature to sign URLs for GCS buckets when
returning layer URLs to clients on image pulls. This means that a
private Nixery instance can run a bucket with restricted access just
fine.
Under the hood Nixery uses a key provided via environment
variables to sign the URL with a 5 minute expiration time.
This can be set up by adding the following two environment variables:
* GCS_SIGNING_KEY: Path to the PEM file containing the signing key.
* GCS_SIGNING_ACCOUNT: Account ("e-mail" address) to use for signing.
If the variables are not set, the previous behaviour is not modified.
Previously the acknowledgement calls from Docker were receiving a
404 (which apparently doesn't bother it?!). This corrects the URL,
which meant that acknowledgement had to move inside of the
registryHandler.
POSIX file locks are essentially incompatible with multithreading. BSD
locks have much saner semantics. We need this now that there can be
multiple concurrent LocalStore::buildPaths() invocations.
This currently fails because we're using POSIX file locks. So when the
garbage collector opens and closes its own temproots file, it causes
the lock to be released and then deleted by another GC instance.
Shiny, new domain is much better and eliminates the TLS redirect issue
because there is a HSTS preload for the entire .dev TLD (which, by the
way, is awesome!)
This might not yet be the final version, but it's going in the right
direction.
Additionally the favicon has been reduced to just the coloured Nix
logo, because details are pretty much invisible at that size anyways.
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:
DRV_PATH=/nix/store/drv-that-has-been-built.drv
OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev
This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).
This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:
[nix]$ ./inst/bin/nix-build ./test.nix
these derivations will be built:
/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
hello!
bye!
running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
post-build-hook: + sleep 1
post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
post-build-hook: + sleep 1
post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
post-build-hook: + sleep 1
post-build-hook: + printf 'very important stuff'
/nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
[nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
my-example-derivation> hello!
my-example-derivation> bye!
my-example-derivation (post)> + sleep 1
my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
my-example-derivation (post)> + sleep 1
my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
my-example-derivation (post)> + sleep 1
my-example-derivation (post)> + printf 'very important stuff'
[1 built, 0.0 MiB DL]
Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
People should not start depending on the demo instance. There have
been discussions around making a NixOS-official instance, but the
project needs to mature a little bit first.
The MD5 sum is used for verifying contents in the layer cache before
accidentally re-uploading, but the syntax of the hash invocation was
incorrect leading to a cache-bust on the manifest layer on every
single build (even for identical images).
Uses the structured errors feature introduced in the Nix code to
return more sensible errors to clients. For now this is quite limited,
but already a lot better than before:
* packages that could not be found result in 404s
* all other errors result in 500s
This way the registry clients will not attempt to interpret the
returned garbage data/empty response as something useful.
Changes the return format of Nixery's build procedure to return a JSON
structure that can indicate which errors have occured.
The server can use this information to send appropriate status codes
back to clients.
The highlighter expects that all code blocks are valid expressions (to
some degree) and highlights mismatches in dark red, which is not
particularly pleasant for the reader.
This introduces a "fake" attribute set in the `inherit` section to
please the highlighter.
These are not strictly speaking language features, but my intention
for this document is that someone can - after reading it - navigate a
large chunk of existing Nix code.
Adds the initial version originally published as a gist. This is
already slightly different from the gist in that it contains a section
on derivations.
This is not yet complete.
--
8f685654a7d04eb8a0cb82d31e44e391e906b609 by Derek Mauro <dmauro@google.com>:
Support constexpr construction of absl::string_view from a
string literal in MSVC 2017+.
Fixes https://github.com/abseil/abseil-cpp/issues/352
PiperOrigin-RevId: 260853160
--
a3c4c5168ce2a491134d7c87cf7fdc75d1ee2533 by Derek Mauro <dmauro@google.com>:
Make SwissTable's at() throw when exceptions are enabled
Fixes https://github.com/abseil/abseil-cpp/issues/355
PiperOrigin-RevId: 260788026
GitOrigin-RevId: 8f685654a7d04eb8a0cb82d31e44e391e906b609
Change-Id: I9ed498e181faa9c9d16e9b1b01404969d99b8ea9
Adds environment variables with which users can configure the package
set source to use. Not setting a source lets Nix default to a recent
NixOS channel (currently nixos-19.03).
This extends the package set import mechanism in
build-registry-image.nix with several different options:
1. Importing a nixpkgs channel from Github (the default, pinned to
nixos-19.03)
2. Importing a custom Nix git repository. This uses builtins.fetchGit
and can thus rely on git/SSH configuration in the environment (such
as keys)
3. Importing a local filesystem path
As long as the repository pointed at is either a checkout of nixpkgs,
or nixpkgs overlaid with custom packages this will work.
A special syntax has been defined for how these three options are
passed in, but users should not need to concern themselves with it as
it will be taken care of by the server component.
This relates to #3.
Adds git & SSH as part of the Nixery image, which are required to use
Nix's builtins.fetchGit.
The dependency on interactive tools is dropped, as it was only
required during development when debugging the image building process
itself.
Introduce an empty runtime configuration object in each built layer.
This is required because Kubernetes expects the configuration to be
present (even if it's just empty values).
Providing an empty configuration will make Docker's API return a full
configuration struct with default (i.e. empty) values rather than
`null`, which works for Kubernetes.
This fixes issue #1. See the issue for additional details.
startProcess does not appear to send the exit code to the helper
correctly. Not sure why this is, but it is probably safe to just
fallback on all sandbox errors.
--
788c948f698afe5998738cbb40b0629668640d73 by Derek Mauro <dmauro@google.com>:
Add an empty raw_logging_internal library. The raw_logging sources
will eventually migrate here.
This target will only contain internal sources. Abseil compatibility
guidelines require users not depend on internals.
https://abseil.io/about/compatibility
PiperOrigin-RevId: 260712817
--
8ea947ec54f47fb02e97597d243cb63aa4c5fe0b by Abseil Team <absl-team@google.com>:
Fixed an incorrect version number test for optional,etc. availability
in iOS. Added tests for watchOS and tvOS.
PiperOrigin-RevId: 260490390
--
53af0544277dc3020b81d16ce110e7bc89f5cf6e by Jorg Brown <jorg@google.com>:
Test the error-string path a bit more.
PiperOrigin-RevId: 260175886
GitOrigin-RevId: 788c948f698afe5998738cbb40b0629668640d73
Change-Id: I366b73331857dc5a1db843b650c2ba27a69a141e
Instead of just dispatching on URL regexes, use handlers to split the
routes into registry-related handlers and otherwise(tm).
For now the otherwise(tm) consists of a file server serving the static
directory, rather than just a plain match on the index route.
