test(tvix/eval): Add the start of a nix oracle test suite

Add the start of a test suite that compares tvix eval results against
nix, using the string repr of the value as the comparison. This shells
out to a nix-instantiate binary, which is configurable as an environment
variable, to eval - there's some extra machinery there to setup a new
nix store as a tempdir to allow running this test inside the nix build
for tvix-eval itself.

Currently this has a macro that'll allow writing lots and lots of
hardcoded tests, but going forward I'm also going to be looking into
adding proptest-based generation of expressions to compare.

Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
This commit is contained in:
Griffin Smith 2022-09-03 11:42:27 -04:00 committed by clbot
parent ef80d00b06
commit 67e796b2e1
4 changed files with 165 additions and 30 deletions

View file

@ -33,6 +33,7 @@ criterion = "0.3.6"
test-generator = "0.3.0"
pretty_assertions = "1.2.1"
itertools = "0.10.3"
tempdir = "0.3.7"
[features]
default = [ "repl" ]