feat(users/edef/weave): use FxHashSet and dedupe early
Deduping early saves a fair bit of memory, but the extra hashing is costly. We switch to FxHash, since we don't need a DoS-proof hash, but we do need it to be *fast*. Change-Id: Ic6b7010874c417862baa9b882593208c8dd1d5e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12648 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
This commit is contained in:
parent
06d2536eec
commit
201d8f0cf2
4 changed files with 33 additions and 4 deletions
7
users/edef/weave/Cargo.lock
generated
7
users/edef/weave/Cargo.lock
generated
|
|
@ -1556,6 +1556,12 @@ version = "0.1.24"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "719b953e2095829ee67db738b3bfa9fa368c94900df327b3f07fe6e794d2fe1f"
|
||||
|
||||
[[package]]
|
||||
name = "rustc-hash"
|
||||
version = "2.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "583034fd73374156e66797ed8e5b0d5690409c9226b22d87cb7f19821c05d152"
|
||||
|
||||
[[package]]
|
||||
name = "rustc_version"
|
||||
version = "0.4.1"
|
||||
|
|
@ -2027,6 +2033,7 @@ dependencies = [
|
|||
"nix-compat",
|
||||
"polars",
|
||||
"rayon",
|
||||
"rustc-hash",
|
||||
"safer_owning_ref",
|
||||
]
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue