feat(users/flokli): add ASG Lightning talk presentation
This adds the materials for the lightning talk held at All Systems Go Conference 2023, Berlin. Talk lives at https://media.ccc.de/v/all-systems-go-2023-245-tvix-store Change-Id: I114b1aec9f1953c148dd29ca88888c16b9fc655d Reviewed-on: https://cl.tvl.fyi/c/depot/+/9751 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
This commit is contained in:
parent
b8ef3e5263
commit
8b09ae54b1
3 changed files with 187 additions and 0 deletions
|
|
@ -0,0 +1,32 @@
|
||||||
|
{ depot, pkgs, ... }:
|
||||||
|
|
||||||
|
let
|
||||||
|
inherit (pkgs)
|
||||||
|
fontconfig qrencode runCommand stdenv;
|
||||||
|
mkQr = url: runCommand "qrcode.png" { } ''
|
||||||
|
${qrencode}/bin/qrencode -o $out -t SVG -s 5 \
|
||||||
|
--background=fafafa \
|
||||||
|
--foreground=000000 \
|
||||||
|
${url}
|
||||||
|
'';
|
||||||
|
in
|
||||||
|
stdenv.mkDerivation {
|
||||||
|
name = "2023-asg-tvix-store";
|
||||||
|
src = ./.;
|
||||||
|
|
||||||
|
FONTCONFIG_FILE = pkgs.makeFontsConf {
|
||||||
|
fontDirectories = with pkgs; [ jetbrains-mono fira fira-code fira-mono lato ];
|
||||||
|
};
|
||||||
|
|
||||||
|
nativeBuildInputs = [ pkgs.reveal-md pkgs.graphviz ];
|
||||||
|
|
||||||
|
buildPhase = ''
|
||||||
|
cp ${depot.tvix.logo}/logo.png tvix-logo.png
|
||||||
|
cp ${mkQr "https://flokli.de"} qrcode-flokli.svg
|
||||||
|
cp ${mkQr "https://tvix.dev"} qrcode-tvix.svg
|
||||||
|
|
||||||
|
mkdir -p $out
|
||||||
|
cp tvix-store-graph-blob-directory.svg $out/
|
||||||
|
reveal-md --static $out presentation.md
|
||||||
|
'';
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,138 @@
|
||||||
|
---
|
||||||
|
author:
|
||||||
|
- Florian Klink
|
||||||
|
date: 2023-09-09
|
||||||
|
title: "tvix-store: A content-addressed file system and sync protocol"
|
||||||
|
theme: moon
|
||||||
|
revealOptions:
|
||||||
|
transition: 'fade'
|
||||||
|
---
|
||||||
|
|
||||||
|
## tvix-store
|
||||||
|
### A content-addressed file system and sync protocol
|
||||||
|
|
||||||
|
2023-09-13
|
||||||
|
|
||||||
|
Florian Klink / flokli
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Whoami
|
||||||
|
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
flokli
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Nix/NixOS contributor
|
||||||
|
- maintain systemd, nss and more low-level stuff there
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Freelance Nix/DevOps consultant
|
||||||
|
|
||||||
|
Note: more Kubernetes/DevOps exposure with work
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What is tvix-store?
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
A new implementation of a content-addressed "storage system"
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
part of the Tvix Project, a (WIP) reimplementation of Nix and auxillary components in Rust
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Storage model: think about git trees and its Merkle DAG…
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
… but with nicer wire format (`.proto`) and hash function (blake3)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Storage model
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Once you know the root: everything else is content-addressed
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
No timestamps, no uid/gid, no xattrs, only one way to represent the same tree
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Automatic dedup of identical subtrees in different file system trees
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Automatic dedup of identical blobs (and you can do more chunking underneath too)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Storage model (cont.)
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Granular seekable access into blobs
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
verified streaming thanks to BLAKE3 and Bao, faulty data is detected early on
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Everything below can be retrieved from anyone without having to trust (P2P substitution, CDNs, …)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usecases
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
File system tree delivery
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Container image delivery
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Backing store for VCS
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Granular access into large datasets
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Status
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
In-memory backend, a local K/V backend (Sled)
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
FUSE filesystem
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
A gRPC API to transfer things, bindings for golang and rust
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
some object storage backends in development (GCS, NATS)
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
FUTUREWORK: more storage backends / store composition / in-kernel module?
|
||||||
|
|
||||||
|
Notes: of course you can use your own network protocol too, like HTTP CAS or iroh....plug different stores together to represent caches, blobfs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Join the IRC channel (`#tvl` on `hackint`), bridged to Matrix and XMPP
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Check our issue tracker
|
||||||
|
- <!-- .element: class="fragment" -->
|
||||||
|
Try to use it and tell us how you broke it!
|
||||||
|
|
||||||
|
Note: if this sounds useful to you, reach out!
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Thanks!
|
||||||
|
|
||||||
|
<style>
|
||||||
|
.container{
|
||||||
|
display: flex;
|
||||||
|
}
|
||||||
|
.col{
|
||||||
|
flex: 1;
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
||||||
|
<div class="container">
|
||||||
|
|
||||||
|
<div class="col">
|
||||||
|
Florian Klink / <a href="https://flokli.de">flokli.de</a><br />
|
||||||
|
<img src="qrcode-flokli.svg" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="col">
|
||||||
|
Tvix / <a href="https://tvix.dev">tvix.dev</a><br />
|
||||||
|
<img src="qrcode-tvix.svg" />
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
[tvix-store graph](tvix-store-graph-blob-directory.svg)
|
||||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 67 KiB |
Loading…
Add table
Add a link
Reference in a new issue