feat(tvix/store/bin): add progress bar infrastructure

This adds the tracing-indicatif crate, and configures it as a layer in
our tracing_subscriber pipeline to emit progress for every span that's
configured so.

It also moves from using std::io::stderr to write logs to using their
writer, to avoid clobbering output.

Progress bar styles are defined in a lazy_static, moving this into a
general tracing is left for later.

This adds some usage of this to the `imports` and `copy` commands.

The output can still be improved a bit - we should  probably split each
task up into a smaller (instrumented) helper functions, so we can create
a progress bar for each task.

Change-Id: I59a1915aa4e0caa89c911632dec59c4cbeba1b89
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11747
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
This commit is contained in:
Florian Klink 2024-04-17 19:44:07 +03:00 committed by flokli
parent 9b77ce9f8f
commit 20513e7a52
8 changed files with 452 additions and 19 deletions

View file

@ -14,6 +14,7 @@ data-encoding = "2.3.3"
digest = "0.10.7"
fastcdc = { version = "3.1.0", features = ["tokio"] }
futures = "0.3.30"
indicatif = "0.17.8"
lazy_static = "1.4.0"
object_store = { version = "0.9.1", features = ["http"] }
parking_lot = "0.12.1"
@ -28,6 +29,7 @@ tokio = { version = "1.32.0", features = ["fs", "macros", "net", "rt", "rt-multi
tonic = "0.11.0"
tower = "0.4.13"
tracing = "0.1.37"
tracing-indicatif = "0.3.6"
url = "2.4.0"
walkdir = "2.4.0"
zstd = "0.13.0"