Scatter blog post ideas for "Let's Learn Nix"
I may not use any of these. I'm just scrawling notes as blog posts to see if anything sticks.
This commit is contained in:
		
							parent
							
								
									209fdf5726
								
							
						
					
					
						commit
						79b5fce68a
					
				
					 3 changed files with 211 additions and 0 deletions
				
			
		
							
								
								
									
										49
									
								
								blog/content/english/lets-learn-nix-caching.md
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										49
									
								
								blog/content/english/lets-learn-nix-caching.md
									
										
									
									
									
										Normal file
									
								
							| 
						 | 
					@ -0,0 +1,49 @@
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					title: "Lets Learn Nix Caching"
 | 
				
			||||||
 | 
					date: 2020-03-17T18:05:38Z
 | 
				
			||||||
 | 
					draft: true
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## TL;DR
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					1. I use `NixOS/nixpkgs-channels` instead of `NixOS/nixpkgs` and avoid
 | 
				
			||||||
 | 
					   `nix-channel`.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## More information
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- By default the Nix package manager uses cache.nixos.org as a binary cache.
 | 
				
			||||||
 | 
					- Visit status.nixos.org
 | 
				
			||||||
 | 
					- `git clone git@github.com:NixOS/nixpkgs-channels` instead of
 | 
				
			||||||
 | 
					  `NixOS/nixpkgs`. The former mirrors the latter and uses Git branches to track
 | 
				
			||||||
 | 
					  the published channels.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## What is a Nix channel
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					If you run...
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ git clone git@github.com:NixOS/nixpkgs ~/nixpkgs
 | 
				
			||||||
 | 
					$ export NIX_PATH="nixpkgs=$(realpath ~/nixpkgs)"
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					One benefit to cloning nixpkgs is that you can browse the source code on your
 | 
				
			||||||
 | 
					machine using tools like `git` and `emacs`. You can also experimentally patch
 | 
				
			||||||
 | 
					and test Nix code this way.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					If any of the above appeals to you, clone `nixpkgs-channels` instead.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					The Nix maintainers build and test the commits from `nixpkgs` using Hydra. Tests
 | 
				
			||||||
 | 
					include reproducibility tests, etc.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Various channels have different verification phases.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					The cache at cache.nixos.org is populate the cache at cache.nixos.org.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					You want to increase the likelihood that you are hitting this cache. For
 | 
				
			||||||
 | 
					example, `google-chrome` takes hours to build.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## What is a binary cache?
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## What is Hydra (Nix CI)?
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## What is Cachix?
 | 
				
			||||||
| 
						 | 
					@ -0,0 +1,121 @@
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					title: "Lets Learn Nix: Reproducibility"
 | 
				
			||||||
 | 
					date: 2020-03-17T12:06:47Z
 | 
				
			||||||
 | 
					draft: true
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					I am dedicating this page to defining and disambiguating some terminology. I
 | 
				
			||||||
 | 
					think it is important to use these terms precisely, so it may be worthwhile to
 | 
				
			||||||
 | 
					memorize these definitions and ensure that you are clarifying the discourse
 | 
				
			||||||
 | 
					rather than muddying it.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Terms
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- repeatable build:
 | 
				
			||||||
 | 
					- reproducible build:
 | 
				
			||||||
 | 
					- deterministic build:
 | 
				
			||||||
 | 
					- pure function:
 | 
				
			||||||
 | 
					- impure function:
 | 
				
			||||||
 | 
					- idempotent function:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					TODO(wpcarro): Consistently and deliberately use reproducible and
 | 
				
			||||||
 | 
					deterministic.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Repeatable vs. Reproducible
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Is NixOS reproducible? Visit [@grhmc][who-grhmc]'s website,
 | 
				
			||||||
 | 
					[r13y.com](https://r13y.com), to find out.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					At the time of this writing, 1519 of 1568 (i.e. 96.9%) of the paths in the
 | 
				
			||||||
 | 
					`nixos.iso_minimal.x86_64-linux` installation image are reproducible.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## What hinders reproducibility?
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Timestamps.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					If package A encodes a timestamp into its build artifact, then we can
 | 
				
			||||||
 | 
					demonstrate that package A is *not reproducible* simply by building it at two
 | 
				
			||||||
 | 
					different times and doing a byte-for-byte comparison of the build artifacts.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Does Nix protect developers against non-determinism
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Yes. But not entirely. How?
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Deterministic Nix derivation
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```nix
 | 
				
			||||||
 | 
					{ pkgs ? import <nixpkgs> {}, ... }:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					with pkgs;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					stdenv.mkDerivation {
 | 
				
			||||||
 | 
					  name = "reproducible";
 | 
				
			||||||
 | 
					  phases = [ "buildPhase" ];
 | 
				
			||||||
 | 
					  buildPhase = "echo reproducible >$out";
 | 
				
			||||||
 | 
					}
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Non-deterministic Nix derivation
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We can introduce some non-determinism into our build using the `date` function.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```nix
 | 
				
			||||||
 | 
					# file: /tmp/test.nix
 | 
				
			||||||
 | 
					{ pkgs ? import <nixpkgs> {}, ... }:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					with pkgs;
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					stdenv.mkDerivation {
 | 
				
			||||||
 | 
					  name = "non-reproducible";
 | 
				
			||||||
 | 
					  phases = [ "buildPhase" ];
 | 
				
			||||||
 | 
					  buildPhase = "date >$out";
 | 
				
			||||||
 | 
					}
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Then run...
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ nix-build /tmp/test.nix
 | 
				
			||||||
 | 
					$ nix-build /tmp/test.nix --check --keep-failed
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## How do you test reproducibility?
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					We can use `cmp` to compare files byte-for-byte. The following comparison should
 | 
				
			||||||
 | 
					fail:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ echo foo >/tmp/a
 | 
				
			||||||
 | 
					$ echo bar >/tmp/b
 | 
				
			||||||
 | 
					$ cmp --silent /tmp/{a,b}
 | 
				
			||||||
 | 
					$ echo $?
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					And the following comparison should succeed:
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ echo hello >/tmp/a
 | 
				
			||||||
 | 
					$ echo hello >/tmp/b
 | 
				
			||||||
 | 
					$ cmp --silent /tmp/{a,b}
 | 
				
			||||||
 | 
					$ echo $?
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Reproducible vs. deterministic
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Reproducible builds *are* deterministic builds and deterministic build
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Deterministic, Reproducible, Pure, Idempotent, oh my
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- A pure function has no side-effects.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- An idempotent function can be executed more than once with the same arguments
 | 
				
			||||||
 | 
					  without altering the side-effects.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					- A deterministic function ensures that
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Deterministic vs. Reproducible
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					I can check if a build is reproducible using [these tools][wtf-repro-tools].
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					[wtf-repro-tools]: https://reproducible-builds.org/tools/
 | 
				
			||||||
 | 
					[who-grhmc]: https://twitter.com/grhmc
 | 
				
			||||||
| 
						 | 
					@ -0,0 +1,41 @@
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					title: "Lets Learn Nix: Tutorial Reproducibility"
 | 
				
			||||||
 | 
					date: 2020-03-17T18:34:58Z
 | 
				
			||||||
 | 
					draft: true
 | 
				
			||||||
 | 
					---
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## Install Nix
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Link to nixos page.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					## The rest
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					Start with this...
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ mkdir ~/lets-learn-nix
 | 
				
			||||||
 | 
					$ cd ~/lets-learn-nix
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					...done. Copy the following and paste it into a file name `shell.nix`.
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```nix
 | 
				
			||||||
 | 
					# file: shell.nix
 | 
				
			||||||
 | 
					let
 | 
				
			||||||
 | 
					  pkgs = import (builtins.fetchGit {
 | 
				
			||||||
 | 
					    url = "https://github.com/NixOS/nixpkgs-channels";
 | 
				
			||||||
 | 
					    ref = "refs/heads/nixos-19.09";
 | 
				
			||||||
 | 
					  }) {}
 | 
				
			||||||
 | 
					in pkgs.mkShell {
 | 
				
			||||||
 | 
					  buildInputs = with pkgs; [
 | 
				
			||||||
 | 
					    git
 | 
				
			||||||
 | 
					  ];
 | 
				
			||||||
 | 
					  NIX_PATH = "nixpkgs=${pkgs}";
 | 
				
			||||||
 | 
					};
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					...then...
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					```shell
 | 
				
			||||||
 | 
					$ nix-shell
 | 
				
			||||||
 | 
					```
 | 
				
			||||||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue