c39cc00404 has added assertions for
all Value accesses and the following case has started failing with
an `unreachable`:
(/tmp/fun.nix):
```nix
{a}: a
```
```
$ nix eval --impure --expr 'import /tmp/fun.nix {a="a";b="b";}'
```
This would crash:
```
terminating due to unexpected unrecoverable internal error: Unexpected condition in getStorage at ../include/nix/expr/value.hh:844
```
This is not a regression, but rather surfaces an existing problem, which previously
was left undiagnosed. In the case of an import `fun` is the `import` primOp, so that read is invalid
and previously this resulted in an access into an inactive union member, which is UB.
The correct thing to use is `vCur`. Identical problem also affected the case of a missing argument.
Add previously failing test cases to the functional/lang test suite.
Fixes#13448.
Fixes:
[261/394] Linking target src/libexpr/libnixexpr.so
In function ‘copy’,
inlined from ‘__ct ’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:688:23,
inlined from ‘operator+’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:3735:43,
inlined from ‘operator()’ at ../src/libexpr/primops/fetchClosure.cc:127:58,
inlined from ‘prim_fetchClosure’ at ../src/libexpr/primops/fetchClosure.cc:132:88:
/nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/char_traits.h:427:56: warning: ‘__builtin_memcpy’ writing 74 bytes into a region of size 16 overflows the destination [-Wstringop-overflow=]
427 | return static_cast<char_type*>(__builtin_memcpy(__s1, __s2, __n));
| ^
../src/libexpr/primops/fetchClosure.cc: In function ‘prim_fetchClosure’:
../src/libexpr/primops/fetchClosure.cc:132:88: note: at offset 16 into destination object ‘<anonymous>’ of size 32
132 | fromPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
| ^
This failed on macOS:
nix-store-tests-run> C++ exception with description "../nix_api_store.cc:33: nix_err_code(ctx) != NIX_OK, message: error: getting status of '/nix/var/nix/daemon-socket/socket': Operation not permitted" thrown in the test body.
This changes makes nix detect a machines available cores automatically whenever build-cores is set to 0.
So far, nix simply passed NIX_BUILD_CORES=0 whenever build-cores is set to 0. (only when build-cores is unset it was detecting cores automatically)
The behavior of passing NIX_BUILD_CORES=0 leads to a performance penalty when sourcing nixpkgs' generic builder's `setup.sh`, as setup.sh has to execute `nproc`. This significantly slows down sourcing of setup.sh
The use of R"(...)" added a bunch of unnecessary whitespace, e.g.
error:
Unable to start any build;
either increase '--max-jobs' or enable remote builds.
For more information run 'man nix.conf' and search for '/machines'.
Now we get
error: Unable to start any build; either increase '--max-jobs' or enable remote builds.
For more information run 'man nix.conf' and search for '/machines'.
This shaves off a very significand amount of memory used
for evaluation as well as reduces the GC-managed heap.
Previously the union discriminator (InternalType) was
stored as a separate field in the Value, which takes up
whole 8 bytes due to padding needed for member alignment.
This effectively wasted 7 whole bytes of memory. Instead
of doing that InternalType is instead packed into pointer
alignment niches. As it turns out, there's more than enough
unused bits there for the bit packing to be effective.
See the doxygen comment in the ValueStorage specialization
for more details.
This does not add any performance overhead, even though
we now consistently assert the InternalType in all getters.
This can also be made atomic with a double width compare-and-swap
instruction on x86_64 (CMPXCHG16B instruction) for parallel evaluation.
This factors out most of the value representation into a mixin class.
`finishValue` is now gone for good and replaced with a simple template
function `setStorage` which derives the type information/disriminator from
the type of the argument. Likewise, reading of the value goes through function
template `getStorage`.
An empty type `Null` is introduced to make the bijection InternalType <-> C++ type
complete.
Previous use of symlink_status() always translated into a stat call, leading
to huge performance penalties for by-name-overlay in nixpkgs. The comment
below references the possible caching, but that seemed to be erroneous, since
the correct way to make use of the caching API is by calling a bunch of `is_*`
functions [1]. For example, here's how libstdc++ does that [2], [3].
This translates to great nixpkgs eval performance improvements:
```
Benchmark 1: GC_INITIAL_HEAP_SIZE=4G result/bin/nix-instantiate ../nixpkgs -A hello --readonly-mode
Time (mean ± σ): 186.7 ms ± 6.7 ms [User: 121.3 ms, System: 64.9 ms]
Range (min … max): 179.4 ms … 201.6 ms 16 runs
Benchmark 2: GC_INITIAL_HEAP_SIZE=4G nix-instantiate ../nixpkgs -A hello --readonly-mode
Time (mean ± σ): 230.6 ms ± 5.0 ms [User: 126.9 ms, System: 103.1 ms]
Range (min … max): 225.1 ms … 241.4 ms 13 runs
```
[1]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0317r1.html
[2]: 8ea555b7b4/libstdc%2B%2B-v3/include/bits/fs_dir.h (L341-L348)
[3]: 8ea555b7b4/libstdc%2B%2B-v3/include/bits/fs_dir.h (L161-L163)
These wrappers don't seem to be used anywhere in and out of tree.
Also the declaration in the header has an incorrect function name `maybeParseFlake`.
Closes#11948
Fixes
error:
… while processing sandbox path '/private/tmp/nix-shell.0MDgyx/nix-test/ca/repl/store/nix/var/nix/builds/nix-build-simple.drv-65916-3910734210' (/private/tmp/nix-shell.0MDgyx/nix-test/ca/repl/store)
error: 'nix' is too short to be a valid store path
which happened because we were now putting the build directory
underneath the store directory.
If a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. Usually this is only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).
(cherry picked from pennae Lix commit 55b416f6897fb0d8a9315a530a9b7f0914458ded)
(store setting done by roberth)
