Commit graph

41 commits

Author SHA1 Message Date
Kane York
2c19bd6662 refactor(3p/tvix): convert NixList usage to shared_ptr
Starting to reclaim memory. Fairly simple mechanical replacement.

Change-Id: I6b4c5c5596729470d1a049eba61e69e8097decf4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1971
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-09-15 21:42:54 +00:00
Eelco Dolstra
785cb3a754 refactor(tvix): getEnv(): Return std::optional
This allows distinguishing between an empty value and no value.

Patch ported from upstream at
ba87b08f85

Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-01 02:21:53 +00:00
Vincent Ambo
06681c35a9 refactor(tvix): Replace several DiscardLogsSink() with std::cerr
Introduces the actual log sink in several places where we actually
want the build logs to thread through correctly.

This should cover *most* build paths.

Change-Id: I735dff8a79f7e35a5874eb89b4abb980f9703dc2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1827
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-21 01:35:08 +00:00
Vincent Ambo
33e8b0f975 chore(tvix): Thread a std::ostream through Store::buildPaths
This part of the store API needs to carry a handle to the log sink
from now on, so that it can be passed in as appropriate from the gRPC
handlers.

In all places where there is no such handler available at the moment,
the discarding log sink has been inserted. This can be used as a
convenient grep target in the future.

Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 11:48:40 +00:00
Vincent Ambo
1a48f39078 fix(tvix): Ensure prim_sort actually uses the right outlist
Previously the outlist would not be used, and it would sort god knows
what in the out value.

This was probably introduced by the std::vector refactoring, and the
language test for builtins.sort was disabled. Whatever reason there
was for disabling it seems to be gone, so we're re-enabling it.

Change-Id: I98941c2cad78df58ff7bea1ece3aaa4133e94bf8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1757
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
2020-08-15 01:58:44 +00:00
Griffin Smith
d1c38d9597 refactor(tvix): Make Store::buildPaths return a Status
Make Store::buildPaths return a Status with [[nodiscard]] rather than
throwing exceptions to signal failure. This is the beginning of a long
road to refactor the entire store API to be status/statusor based
instead of using exceptions.

Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-14 03:06:46 +00:00
Kane York
72e61aa584 refactor(tvix): completely remove boehm gc
We have decided that leaking memory is a better fate than random,
non-debuggable memory corruption. Future CLs will begin changing
various fields to std::unique_ptr and std::shared_ptr.

It turns out that disabling the GC does not have disasterous impact.
The Nix evaluator only runs on the client CLI, never in any long-
running process. Even the REPL does not leak too badly under this
change, because it uses one EvalState for the duration of the REPL.

Building an explicitly tracing garbage collector is likely in the
future of this project, but that giant amount of work cannot be
done under a nix evaluator that is constantly crashing. We need to
restore development velocity here, and this is the best way we've
figured out to do it.

Change-Id: I2fcda8fcee853c15a9a5e22eca7c5a784bc2bf76
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1720
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-14 00:35:18 +00:00
Kane York
1de00e6c42 chore(3p/nix): apply google-readability-casting
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix

Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc

Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-08 22:16:02 +00:00
Kane York
a5dae62e85 fix(3p/nix): Use SkipEmpty in all calls to absl::StrSplit
The behavior to return a list containing a single empty string when
provided an empty string is a behavior that absl inherited from legacy
code. However, the behavior expected by legacy code in Nix is the
behavior provided by the SkipEmpty option. Switch all calls to use
SkipEmpty, except for the call already using SkipWhitespace.

See also commit 26a59482d2, with the
partly-prophetic message: "there may be other places we need to
fix this as well."

Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 20:50:47 +00:00
Kane York
72fc2fd27e fix(3p/nix): revert "apply all clang-tidy fixes"
This reverts commit ef54f5da9f.

Resolved conflicts:
	third_party/nix/src/libexpr/eval.cc
	third_party/nix/src/libstore/builtins/fetchurl.cc
	third_party/nix/src/libstore/references.cc
	third_party/nix/src/libutil/hash.cc
	third_party/nix/src/nix-daemon/nix-daemon.cc

Change-Id: Ib9cf6e96a79a23bde3983579ced3f92e530cb011
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1547
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
2020-08-01 22:45:15 +00:00
Kane York
64f6bb6951 feat(3p/nix): remove External values feature
External values are only useful when using the plugin framework, which we are not interested in carrying forward.

Reverts commit 320659b0cd

Change-Id: Ib4929c349bbb33f16224fc674e94c7b7d5953c6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1505
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
2020-08-01 18:54:50 +00:00
Kane York
1cbffe21f3 chore(3p/nix/hash): prefer StatusOr over throwing constructor
The use of `unwrap_throw` can be used as a later grep target.

Change-Id: I8c54ed90c4289f07aecb8a1393dd10204c8bce4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1493
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
2020-08-01 01:15:29 +00:00
Kane York
ef54f5da9f fix(3p/nix): apply all clang-tidy fixes
Change-Id: I265e763393422ee1881653527c91024458060825
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1432
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
2020-07-27 21:16:39 +00:00
Alyssa Ross
855995325e feat(3p/nix): remove builtins.importNative
This is the shared object equivalent of builtins.exec, or a plugins
equivalent accessible from the Nix language.  Either way, since we
don't have builtins.exec or plugins any more, I think it makes sense
to remove this builtin.

This will also allow us to drop the
allow-unsafe-native-code-during-evaluation option, which formerly
controlled whether builtins.exec and builtins.importNative were
enabled.

Cc: Griffin Smith <grfn@gws.fyi>
Cc: Profpatsch <mail@profpatsch.de>
Change-Id: I8993a8a79d559c102647308a2684c089bbc06713
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1340
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
2020-07-23 04:53:23 +00:00
Vincent Ambo
15afa8472e fix(3p/nix): Fix all remaining compiler warnings
This compiles under `-Wall -Werror`.

The largest chunk of this change is `final` qualifiers for the various
Nix CLI command structs, which inherit from a Command class that has
more virtual functions than are implemented by each command.

Change-Id: I0925e6e1a39013f026773db5816e4a77d50f3b4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1294
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-07-19 22:00:59 +00:00
Vincent Ambo
324d385b29 refactor(3p/nix): Rename & undeprecate Bindings::lexicographicOrder
The function is renamed to `SortedByKeys`, which is more descriptive,
and annotated with a comment about what it is used for.

The deprecation warning has been removed because this function is
currently functionally required.

Change-Id: I0ee3a76deff05f366feca9ddac8f38ab34bffbd0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1288
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-19 19:02:06 +00:00
Vincent Ambo
cdc1a0bb6e refactor(3p/nix/libexpr): Back Nix lists with std::vector
This change does away with the previous special-casing of lists of
certain element sizes, and the use of raw C-style arrays.

Lists are now backed by a std::vector of nix::Value*, which uses the
traceable GC allocator.

This change is unfortunately quite noisy because the accessor methods
were updated/removed accordingly, so all callsites of Nix-related
lists have changed.

For some operations in primops.cc where keeping the previous code
structure would have been more difficult with a "proper" vector, the
implementation has been replaced with std::vector methods. For
example, list concatenation now uses appropriate range inserts.

Anecdotally the performance of this is about equal, to even slightly
better, than the previous implementation.

All language tests pass and the depot paths I've used for testing
still evaluate.

Change-Id: Ib5eca6c0207429cb323a330c838c3a2200b2c693
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1266
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-18 18:08:42 +00:00
Vincent Ambo
8d5857abbc chore(3p/nix): Remove builtins.exec
This is a bad idea, it shouldn't exist, nixpkgs doesn't use it.

Change-Id: Ic4d1b936d8f059d5c40f0567af165b02427d7e36
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1241
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-17 19:03:59 +00:00
Vincent Ambo
fa161e9a38 refactor(3p/nix/libexpr): Remove the nix::Symbol default constructor
Having a default constructor for this causes a variety of annoying
situations across the codebase in which this is initialised to an
unexpected value, leading to constant guarding against those
conditions.

It turns out there's actually no intrinsic reason that this default
constructor needs to exist. The biggest one was addressed in CL/1138
and this commit cleans up the remaining bits.

Change-Id: I4a847f50bc90e72f028598196592a7d8730a4e01
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1139
Reviewed-by: isomer <isomer@tvl.fyi>
Tested-by: BuildkiteCI
2020-07-13 21:06:43 +00:00
Griffin Smith
b94d9f89dc fix(3p/nix): Make traces log, not dlog
We don't want traces compiled out since they're an actual language
feature that're used in userspace - also their absence is breaking the
tests

Change-Id: Icaefca8f52e94001785f724fdc0c10a7586b24e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/562
Reviewed-by: lukegb <lukegb@tvl.fyi>
Reviewed-by: Kane York <rikingcoding@gmail.com>
Reviewed-by: lukegbot <bot@lukegb.com>
2020-06-23 12:37:48 +00:00
Vincent Ambo
3652326ed2 refactor(3p/nix): Anchor local includes at src/
Previously all includes were anchored in one global mess of header
files. This moves the includes into filesystem "namespaces" (if you
will) for each sub-package of Nix.

Note: This commit does not introduce the relevant build system changes.
2020-05-27 21:56:34 +01:00
Vincent Ambo
bf452cbc2a refactor(3p/nix): Replace tokenizeStrings with absl::StrSplit
This function was a custom (and inefficient in the case of
single-character delimiters) string splitter which was used all over
the codebase. Abseil provides an appropriate replacement function.
2020-05-25 15:54:14 +01:00
Vincent Ambo
838f86b0fd style(3p/nix): Remove 'using std::*' from types.hh
It is considered bad form to use things from includes in headers, as
these directives propagate to everywhere else and can make it
confusing.

types.hh (which is includes almost literally everywhere) had some of
these directives, which this commit removes.
2020-05-24 22:29:21 +01:00
Vincent Ambo
3b903193be fix(3p/nix/libexpr): Fix attrNames/attrValues builtins for btree_map
Replaces the previous implementations which performed sorting with one
that instead walks through the map (which is already sorted) and
yields values from it.

This fixes a handful of language tests because the previous
implementation did not actually yield useful values on the new implementation.
2020-05-23 06:18:20 +01:00
Vincent Ambo
68e6e92a20 chore(3p/nix/libexpr): Delete Bindings::sort
This function does nothing anymore since the attributes are always
in-order.
2020-05-22 03:35:07 +01:00
Vincent Ambo
986a8f6b75 fix(3p/nix): Update for usage of new attribute set API
The new attribute set API uses the iterators of the btree_map
directly. This requires changes in various files because the internals
of libexpr are very entangled.

This code runs and compiles, but there is a bug causing empty
attribute sets to be assigned incorrectly.
2020-05-22 01:59:39 +01:00
Vincent Ambo
a162f4e825 refactor(3p/nix/libexpr): Use std::string as qualified type
Replaces most uses of `string` with `std::string`.

This came up because I removed the "types.hh" import from
"symbol-table.hh", which percolated through a bunch of files where
`string` was suddenly no longer defined ... *sigh*
2020-05-21 05:43:22 +01:00
Vincent Ambo
c395a48be2 fix(3p/nix): Compatibility with updated SymbolTable
The functions in SymbolTable have been renamed to match the Google
Style guide, and some debug-only functions have been removed.
2020-05-21 05:09:23 +01:00
Vincent Ambo
43677021e3 refactor(3p/nix): Apply clang-tidy's performance-* fixes
This applies the performance fixes listed here:

https://clang.llvm.org/extra/clang-tidy/checks/list.html
2020-05-20 22:58:43 +01:00
Vincent Ambo
689ef502f5 refactor(3p/nix): Apply clang-tidy's readability-* fixes
This applies the readability fixes listed here:

https://clang.llvm.org/extra/clang-tidy/checks/list.html
2020-05-20 22:27:37 +01:00
Vincent Ambo
d331d3a0b5 refactor(3p/nix): Apply clang-tidy's modernize-* fixes
This applies the modernization fixes listed here:

https://clang.llvm.org/extra/clang-tidy/checks/list.html

The 'modernize-use-trailing-return-type' fix was excluded due to my
personal preference (more specifically, I think the 'auto' keyword is
misleading in that position).
2020-05-20 04:33:07 +01:00
Vincent Ambo
3908732181 style(3p/nix): Final act in the brace-wrapping saga
This last change set was generated by a full clang-tidy run (including
compilation):

    clang-tidy -p ~/projects/nix-build/ \
      -checks=-*,readability-braces-around-statements -fix src/*/*.cc

Actually running clang-tidy requires some massaging to make it play
nice with Nix + meson, I'll be adding a wrapper or something for that soon.
2020-05-19 20:51:32 +01:00
Vincent Ambo
1841d93ccb style(3p/nix): Add braces around single-line for-loops
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:

    ambr --regex 'for (\(.+\))\s([a-z].*;)' 'for $1 { $2 }'

[0]: https://github.com/dalance/amber
2020-05-19 19:04:08 +01:00
Vincent Ambo
867055133d style(3p/nix): Add braces around single-line conditionals
These were not caught by the previous clang-tidy invocation, but were
instead sorted out using amber[0] as such:

    ambr --regex 'if (\(.+\))\s([a-z].*;)' 'if $1 { $2 }'

[0]: https://github.com/dalance/amber
2020-05-19 18:55:58 +01:00
Vincent Ambo
09cbc431cc fix(3p/nix): Fix incorrectly braced conditionals and loops
Fixes mistakes introduced by clang-tidy in the previous commit.
2020-05-19 18:01:12 +01:00
Vincent Ambo
b490742a51 style(3p/nix): Enforce braces around loops and conditionals
This change was generated with:

  fd -e cc -e hh | xargs -I{} clang-tidy {} -p ~/projects/nix-build/ \
    --checks='-*,readability-braces-around-statements' --fix \
    -fix-errors

Some manual fixes were applied because some convoluted unbraced
statements couldn't be untangled by clang-tidy.

This commit still includes invalid files, but I decided to clean them
up in a subsequent commit so that it becomes more obvious where
clang-tidy failed. Maybe this will allow for a bug-report to
clang-tidy.
2020-05-19 17:38:04 +01:00
Vincent Ambo
c758de9d22 style(3p/nix): Reformat all includes to match new style 2020-05-19 15:55:11 +01:00
Vincent Ambo
c7be956f3f style(3p/nix/libexpr): Reformat with clang-format 2020-05-19 04:51:01 +01:00
Vincent Ambo
939dd9f817 refactor(3p/nix/libexpr): Replace logging.h with glog 2020-05-19 01:24:50 +01:00
Vincent Ambo
0f2cf531f7 style(3p/nix): Reformat project in Google C++ style
Reformatted with:

    fd . -e hh -e cc | xargs clang-format -i
2020-05-17 16:31:57 +01:00
Vincent Ambo
7994fd1d54 Add 'third_party/nix/' from commit 'be66c7a6b24e3c3c6157fd37b86c7203d14acf10'
git-subtree-dir: third_party/nix
git-subtree-mainline: cf8cd640c1
git-subtree-split: be66c7a6b24e3c3c6157fd37b86c7203d14acf10
2020-05-17 15:52:38 +01:00