Scatter blog post ideas for "Let's Learn Nix"
I may not use any of these. I'm just scrawling notes as blog posts to see if anything sticks.
This commit is contained in:
		
							parent
							
								
									209fdf5726
								
							
						
					
					
						commit
						79b5fce68a
					
				
					 3 changed files with 211 additions and 0 deletions
				
			
		
							
								
								
									
										49
									
								
								blog/content/english/lets-learn-nix-caching.md
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										49
									
								
								blog/content/english/lets-learn-nix-caching.md
									
										
									
									
									
										Normal file
									
								
							| 
						 | 
				
			
			@ -0,0 +1,49 @@
 | 
			
		|||
---
 | 
			
		||||
title: "Lets Learn Nix Caching"
 | 
			
		||||
date: 2020-03-17T18:05:38Z
 | 
			
		||||
draft: true
 | 
			
		||||
---
 | 
			
		||||
 | 
			
		||||
## TL;DR
 | 
			
		||||
 | 
			
		||||
1. I use `NixOS/nixpkgs-channels` instead of `NixOS/nixpkgs` and avoid
 | 
			
		||||
   `nix-channel`.
 | 
			
		||||
 | 
			
		||||
## More information
 | 
			
		||||
 | 
			
		||||
- By default the Nix package manager uses cache.nixos.org as a binary cache.
 | 
			
		||||
- Visit status.nixos.org
 | 
			
		||||
- `git clone git@github.com:NixOS/nixpkgs-channels` instead of
 | 
			
		||||
  `NixOS/nixpkgs`. The former mirrors the latter and uses Git branches to track
 | 
			
		||||
  the published channels.
 | 
			
		||||
 | 
			
		||||
## What is a Nix channel
 | 
			
		||||
 | 
			
		||||
If you run...
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ git clone git@github.com:NixOS/nixpkgs ~/nixpkgs
 | 
			
		||||
$ export NIX_PATH="nixpkgs=$(realpath ~/nixpkgs)"
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
One benefit to cloning nixpkgs is that you can browse the source code on your
 | 
			
		||||
machine using tools like `git` and `emacs`. You can also experimentally patch
 | 
			
		||||
and test Nix code this way.
 | 
			
		||||
 | 
			
		||||
If any of the above appeals to you, clone `nixpkgs-channels` instead.
 | 
			
		||||
 | 
			
		||||
The Nix maintainers build and test the commits from `nixpkgs` using Hydra. Tests
 | 
			
		||||
include reproducibility tests, etc.
 | 
			
		||||
 | 
			
		||||
Various channels have different verification phases.
 | 
			
		||||
 | 
			
		||||
The cache at cache.nixos.org is populate the cache at cache.nixos.org.
 | 
			
		||||
 | 
			
		||||
You want to increase the likelihood that you are hitting this cache. For
 | 
			
		||||
example, `google-chrome` takes hours to build.
 | 
			
		||||
 | 
			
		||||
## What is a binary cache?
 | 
			
		||||
 | 
			
		||||
## What is Hydra (Nix CI)?
 | 
			
		||||
 | 
			
		||||
## What is Cachix?
 | 
			
		||||
| 
						 | 
				
			
			@ -0,0 +1,121 @@
 | 
			
		|||
---
 | 
			
		||||
title: "Lets Learn Nix: Reproducibility"
 | 
			
		||||
date: 2020-03-17T12:06:47Z
 | 
			
		||||
draft: true
 | 
			
		||||
---
 | 
			
		||||
 | 
			
		||||
I am dedicating this page to defining and disambiguating some terminology. I
 | 
			
		||||
think it is important to use these terms precisely, so it may be worthwhile to
 | 
			
		||||
memorize these definitions and ensure that you are clarifying the discourse
 | 
			
		||||
rather than muddying it.
 | 
			
		||||
 | 
			
		||||
## Terms
 | 
			
		||||
 | 
			
		||||
- repeatable build:
 | 
			
		||||
- reproducible build:
 | 
			
		||||
- deterministic build:
 | 
			
		||||
- pure function:
 | 
			
		||||
- impure function:
 | 
			
		||||
- idempotent function:
 | 
			
		||||
 | 
			
		||||
TODO(wpcarro): Consistently and deliberately use reproducible and
 | 
			
		||||
deterministic.
 | 
			
		||||
 | 
			
		||||
## Repeatable vs. Reproducible
 | 
			
		||||
 | 
			
		||||
Is NixOS reproducible? Visit [@grhmc][who-grhmc]'s website,
 | 
			
		||||
[r13y.com](https://r13y.com), to find out.
 | 
			
		||||
 | 
			
		||||
At the time of this writing, 1519 of 1568 (i.e. 96.9%) of the paths in the
 | 
			
		||||
`nixos.iso_minimal.x86_64-linux` installation image are reproducible.
 | 
			
		||||
 | 
			
		||||
## What hinders reproducibility?
 | 
			
		||||
 | 
			
		||||
Timestamps.
 | 
			
		||||
 | 
			
		||||
If package A encodes a timestamp into its build artifact, then we can
 | 
			
		||||
demonstrate that package A is *not reproducible* simply by building it at two
 | 
			
		||||
different times and doing a byte-for-byte comparison of the build artifacts.
 | 
			
		||||
 | 
			
		||||
## Does Nix protect developers against non-determinism
 | 
			
		||||
 | 
			
		||||
Yes. But not entirely. How?
 | 
			
		||||
 | 
			
		||||
## Deterministic Nix derivation
 | 
			
		||||
 | 
			
		||||
```nix
 | 
			
		||||
{ pkgs ? import <nixpkgs> {}, ... }:
 | 
			
		||||
 | 
			
		||||
with pkgs;
 | 
			
		||||
 | 
			
		||||
stdenv.mkDerivation {
 | 
			
		||||
  name = "reproducible";
 | 
			
		||||
  phases = [ "buildPhase" ];
 | 
			
		||||
  buildPhase = "echo reproducible >$out";
 | 
			
		||||
}
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
## Non-deterministic Nix derivation
 | 
			
		||||
 | 
			
		||||
We can introduce some non-determinism into our build using the `date` function.
 | 
			
		||||
 | 
			
		||||
```nix
 | 
			
		||||
# file: /tmp/test.nix
 | 
			
		||||
{ pkgs ? import <nixpkgs> {}, ... }:
 | 
			
		||||
 | 
			
		||||
with pkgs;
 | 
			
		||||
 | 
			
		||||
stdenv.mkDerivation {
 | 
			
		||||
  name = "non-reproducible";
 | 
			
		||||
  phases = [ "buildPhase" ];
 | 
			
		||||
  buildPhase = "date >$out";
 | 
			
		||||
}
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
Then run...
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ nix-build /tmp/test.nix
 | 
			
		||||
$ nix-build /tmp/test.nix --check --keep-failed
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
## How do you test reproducibility?
 | 
			
		||||
 | 
			
		||||
We can use `cmp` to compare files byte-for-byte. The following comparison should
 | 
			
		||||
fail:
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ echo foo >/tmp/a
 | 
			
		||||
$ echo bar >/tmp/b
 | 
			
		||||
$ cmp --silent /tmp/{a,b}
 | 
			
		||||
$ echo $?
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
And the following comparison should succeed:
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ echo hello >/tmp/a
 | 
			
		||||
$ echo hello >/tmp/b
 | 
			
		||||
$ cmp --silent /tmp/{a,b}
 | 
			
		||||
$ echo $?
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
## Reproducible vs. deterministic
 | 
			
		||||
 | 
			
		||||
Reproducible builds *are* deterministic builds and deterministic build
 | 
			
		||||
 | 
			
		||||
## Deterministic, Reproducible, Pure, Idempotent, oh my
 | 
			
		||||
 | 
			
		||||
- A pure function has no side-effects.
 | 
			
		||||
 | 
			
		||||
- An idempotent function can be executed more than once with the same arguments
 | 
			
		||||
  without altering the side-effects.
 | 
			
		||||
 | 
			
		||||
- A deterministic function ensures that
 | 
			
		||||
 | 
			
		||||
## Deterministic vs. Reproducible
 | 
			
		||||
 | 
			
		||||
I can check if a build is reproducible using [these tools][wtf-repro-tools].
 | 
			
		||||
 | 
			
		||||
[wtf-repro-tools]: https://reproducible-builds.org/tools/
 | 
			
		||||
[who-grhmc]: https://twitter.com/grhmc
 | 
			
		||||
| 
						 | 
				
			
			@ -0,0 +1,41 @@
 | 
			
		|||
---
 | 
			
		||||
title: "Lets Learn Nix: Tutorial Reproducibility"
 | 
			
		||||
date: 2020-03-17T18:34:58Z
 | 
			
		||||
draft: true
 | 
			
		||||
---
 | 
			
		||||
 | 
			
		||||
## Install Nix
 | 
			
		||||
 | 
			
		||||
Link to nixos page.
 | 
			
		||||
 | 
			
		||||
## The rest
 | 
			
		||||
 | 
			
		||||
Start with this...
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ mkdir ~/lets-learn-nix
 | 
			
		||||
$ cd ~/lets-learn-nix
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
...done. Copy the following and paste it into a file name `shell.nix`.
 | 
			
		||||
 | 
			
		||||
```nix
 | 
			
		||||
# file: shell.nix
 | 
			
		||||
let
 | 
			
		||||
  pkgs = import (builtins.fetchGit {
 | 
			
		||||
    url = "https://github.com/NixOS/nixpkgs-channels";
 | 
			
		||||
    ref = "refs/heads/nixos-19.09";
 | 
			
		||||
  }) {}
 | 
			
		||||
in pkgs.mkShell {
 | 
			
		||||
  buildInputs = with pkgs; [
 | 
			
		||||
    git
 | 
			
		||||
  ];
 | 
			
		||||
  NIX_PATH = "nixpkgs=${pkgs}";
 | 
			
		||||
};
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
...then...
 | 
			
		||||
 | 
			
		||||
```shell
 | 
			
		||||
$ nix-shell
 | 
			
		||||
```
 | 
			
		||||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue