Commit graph

4 commits

Author SHA1 Message Date
edef
201d8f0cf2 feat(users/edef/weave): use FxHashSet and dedupe early
Deduping early saves a fair bit of memory, but the extra hashing is
costly.

We switch to FxHash, since we don't need a DoS-proof hash, but we do
need it to be *fast*.

Change-Id: Ic6b7010874c417862baa9b882593208c8dd1d5e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12648
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
2024-10-19 17:01:38 +00:00
edef
06d2536eec feat(users/edef/weave): ingest roots in Parquet format
Parsing of store-paths.xz is now handled by //users/edef/fetchroots.

Change-Id: I78be5aada0c0a321ed79d80c9b615e5f997ac3e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12670
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-10-19 17:01:38 +00:00
edef
549c51f200 refactor(users/edef/weave): use swap rather than CAS
We only care about ordering on the same variable, and we rely on the
release barrier provided by rayon's thread joining.

The comparison failing is always an error path, and we're indifferent
about which thread it fails on.

Change-Id: I592a7eaae5b6935cf424c6576a49786f39909fb6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12666
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
2024-10-19 13:49:02 +00:00
Vincent Ambo
bb34210a64 chore(tvix/tools): move weave to //users/edef
This is not a core Tvix tool, it's a tool that uses a Tvix component.

Change-Id: I705f2c4ab87f1512e005007c933e16b84ed4279f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/12605
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 23:11:25 +00:00
Renamed from tvix/tools/weave/src/main.rs (Browse further)