From 57033490de2ecf0de854d97ed192f8c54b5fe1b5 Mon Sep 17 00:00:00 2001 From: Vova Kryachko Date: Sat, 22 Mar 2025 00:01:15 +0000 Subject: [PATCH] feat(blog): Add nix-deamon blog and guide. Change-Id: I46b8632ac2bfa43ccf95bd63d37e08a8f6b0869e Reviewed-on: https://cl.snix.dev/c/snix/+/30244 Autosubmit: Vova Kryachko Tested-by: besadii Reviewed-by: Florian Klink --- .../blog/2025-03-21-overlay-store/index.md | 69 ++++++++++ web/content/docs/guides/local-overlay.md | 118 ++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 web/content/blog/2025-03-21-overlay-store/index.md create mode 100644 web/content/docs/guides/local-overlay.md diff --git a/web/content/blog/2025-03-21-overlay-store/index.md b/web/content/blog/2025-03-21-overlay-store/index.md new file mode 100644 index 000000000..ca5ac2bbb --- /dev/null +++ b/web/content/blog/2025-03-21-overlay-store/index.md @@ -0,0 +1,69 @@ +--- +title: "Using Snix as Lower Nix Overlay Store" +description: "Using Snix as Lower Nix Store" +date: 2025-03-21T22:40:33+00:00 +lastmod: 2025-03-21T22:40:33+00:00 +draft: false +weight: 51 +categories: [] +tags: [] +contributors: [] +pinned: false +homepage: false +--- + +## Background + +About a year ago, nix introduced a new experimental store type: +[Local Overlay Store]. This store allows having multiple physical stores acting +as a single logical store. + +With this approach, stores are layered on top of each other with all but the +top-most layer being writable, the others are read-only. The feature is +described in the [Upstream Nix Documentation][Local Overlay Store]. Work is +[ongoing][lix-local-overlay] to bring the feature to Lix aswell. + +The main use-case is for it is having a large nix store mounted onto a machine +as read-only and configure nix to use data stored in it to avoid +rebuilding/substituting from `cache.nixos.org`. + +Due to implementation details of this feature, the lower layer(s) can not only +be a location on the file system but also have another `nix-daemon` back the +lower layer(s). + +## Meet the new `snix` nix-daemon + +Implementing `nix-daemon` protocol is a lot of effort, due to being entirely +custom and undocumented. On the other hand, mounting a large nix store is a +great fit for `snix` as our [content-addressed store][castore] is much more +space efficient than conventional filesystem storage. A great example of this +can be found [here][replit]. + +What's interesting about Local Overlay Store, is that it uses only a small +subset of operations when talking to the `nix-daemon`. And this seemed like a +good opportunity to make using `nix` backed by our `castore` more seamless. + +So we are happy to announce that as of today, `snix` has implemented **all** of +the operations required to operate as a lower layer in nix's overlay store. + +Check out our [Guide]({{\< ref "/docs/guides/local-overlay" >}}) on how to set +it up. + +Please test it out, let us know what you think and report [bugs]. + +## What's next for Local Overlay Store + +We have plans expand our `nix-daemon` to support non-readonly mode and have all +possible substitutions go into your local `castore` and only use the upper nix +store for local builds. + +We also want to make `snix` nix-daemon easier to use without having to run +multiple `snix` components. + +Stay tuned. + +[castore]: https://snix.dev/docs/components/overview/ +[bugs]: https://git.snix.dev/snix/snix/issues +[Local Overlay Store]: https://nix.dev/manual/nix/2.26/store/types/experimental-local-overlay-store.html +[replit]: https://blog.replit.com/tvix-store +[lix-local-overlay]: https://gerrit.lix.systems/c/lix/+/2859 diff --git a/web/content/docs/guides/local-overlay.md b/web/content/docs/guides/local-overlay.md new file mode 100644 index 000000000..45dd3482d --- /dev/null +++ b/web/content/docs/guides/local-overlay.md @@ -0,0 +1,118 @@ +--- +title: "Use as a lower store with nix" +slug: local-overlay +description: "" +summary: "" +date: 2025-03-21T22:40:33+00:00 +lastmod: 2025-03-21T22:40:33+00:00 +draft: false +weight: 14 +toc: true +--- + +This document describes how to configure `snix` as the lower layer in your +[Local Overlay] nix store. + +### Build required `snix` components + +To use this feature you will need to 2 `snix` compontents, for detailed building +instructions see [Building]({{< ref "building" >}}). + +```console +$ nix-build -A snix.store +$ nix-build -A snix.nix-daemon +``` + +These will provide `snix-store` and `nix-daemon` binaries. + +### Run the `snix` daemon + +`snix daemon` is the component exposing `castore` and `store` data. By default, +these live inside `/var/lib/snix`, so make sure it's writable for the user +you're executing it with. See `snix-store daemon --help` for customization +options. `/var/lib/snix`, you can run `snix-store daemon --help` for +customization instructions. + +You can run the daemon with: + +```console +$ $(nix-build -A snix.snix-store)/bin/snix-store daemon +``` + +### Mount the castore onto your file system + +To expose the store paths and their contents as a file system, if can be +FUSE-mounted with the following command: + +```console +$ $(nix-build -A snix.snix-store)/bin/snix-store mount /path/to/mount +``` + +This mount will talk to the previously invoked daemon. + +Note that by default, this mount won't allow listing files and directories at +the root of the store, if you want to enable it, use the `--list-root` flag, but +be careful with it if your store is really large. + +### Run `snix` nix-daemon + +```console +$ $(nix-build -A snix.nix-daemon)/bin/nix-daemon -l /tmp/snix-daemon.sock \ + --unix-listen-unlink +``` + +This will launch the `snix` nix-daemon listening on a unix domain socket + +### Create an overlayfs mount + +Bind mount your real /nix store on the side, so that nix has direct access to +it, this is optional but allows you to have access to your real nix store +without unmounting: + +```console +$ mount --bind /nix /opt/nix +``` + +```console +$ mount -t overlay overlay \ + -o lowerdir=/path/to/mount \ + -o upperdir=/opt/nix \ + /nix +``` + +### Configure nix to use the daemon + +With all of the above out of the way, we are ready to configure nix. In the +proposed setup we will configure nix-daemon with an overlay store but for the +Nix CLI you can just configure nix with the overlay store. + +#### nix-daemon + +The daemon can be configured in the following way: + +Add the following line to your `/etc/nix.conf` + +``` +store = local-overlay://?state=/opt/nix/var/nix&upper-layer=/opt/nix/store&check-mount=false&lower-store=unix%3A%2F%2F%2Ftmp%2Fsnix-daemon.sock +``` + +#### Personal nix config + +With the above configuration in your `/etc/nix.conf`, we need to tell nix not to +use it but instead use `store = daemon` so that only the daemon is aware of the +Local Overlay Store. + +This can be achieved by either setting the env variable +`NIX_CONFIG='store = daemon` or by adding `store = daemon` to your +`$XDG_CONFIG_HOME/nix.conf` file. + +### Profit + +With the above setup you should now be able to have nix use Snix Castore as its +lower store. + +Note that snix's FUSE mount might be performing slower than the native +file-system depending on your workload. Please file bugs if you notice obviously +bad performance. + +[local overlay]: https://nix.dev/manual/nix/2.26/store/types/experimental-local-overlay-store.html