Now that Symbols are statically allocated at compile time with known IDs,
we can use switch statements instead of if-else chains for Symbol comparisons.
This provides better performance through compiler optimizations like jump tables.
Changes:
- Add public getId() method to Symbol class to access the internal ID
- Convert if-else chains comparing Symbol values to switch statements
in primops.cc's derivationStrictInternal function
- Simplify control flow by removing the 'handled' flag and moving the
default attribute handling into the switch's default case
The static and runtime Symbol IDs are guaranteed to match by the
copyIntoSymbolTable implementation which asserts this invariant.
Co-authored-by: John Ericson <git@JohnEricson.me>
The motivation for this change is two-fold:
1. Commonly used Symbol values can be referred to
quite often and they can be assigned at compile-time
rather than runtime.
2. This also unclutters EvalState constructor, which was
getting very long and unreadable.
Spiritually similar to https://gerrit.lix.systems/c/lix/+/2218,
though that patch doesn't allocate the Symbol at compile time.
Co-authored-by: eldritch horrors <pennae@lix.systems>
This allows us to replace some very hacky and not correct string
concatentation in `HttpBinaryCacheStore`. It will especially be useful
with #13752, when today's hacks started to cause problems in practice,
not just theory.
Also make `fixGitURL` returned a `ParsedURL`.
This is needed to rearrange include order, but I also think it is a good
thing anyways, as we seek to reduce the use of global settings variables
over time.
This avoids problems with older versions of Nix that don't put the
caches in WAL mode. That's generally not a problem, until you do something like
nix build --print-out-paths ... | cachix
which deadlocks because cachix tries to switch the caches to truncate
mode, which requires exclusive access. But the first process cannot
make progress because the cachix process isn't reading from the pipe.
Compilers in nixpkgs have caught up and major distros
should also have recent enough compilers. It would be
nice to have newer features like more full featured
ranges and deducing this.
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].
1. Added more regression tests for timestamp formats.
Special attention needs to be paid to the precision
of the subsecond range for local-time. Prior versions select the closest
(upwards) multiple of 3 with a hard cap of 9 digits.
2. Normalize local datetime and offset datetime to always
use the uppercase separator `T`. This is actually the issue
surfaced in [2]. This canonicalization is basically a requirement
by (a certain reading) of rfc3339 section 5.6 [3].
3. If using toml11 >= 4.0 also keep the old behavior wrt
to the number of digits used for subsecond part of the local-time.
Newer versions cap it at 6 digits unconditionally.
[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339
This was removed in https://github.com/NixOS/nix/pull/11152. However,
we need it for the multi-threaded evaluator, because otherwise Boehm
GC will crash while scanning the thread stack:
#0 GC_push_all_eager (bottom=<optimized out>, top=<optimized out>) at extra/../mark.c:1488
#1 0x00007ffff74691d5 in GC_push_all_stack_sections (lo=<optimized out>, hi=<optimized out>, traced_stack_sect=0x0) at extra/../mark_rts.c:704
#2 GC_push_all_stacks () at extra/../pthread_stop_world.c:876
#3 GC_default_push_other_roots () at extra/../os_dep.c:2893
#4 0x00007ffff746235c in GC_mark_some (cold_gc_frame=0x7ffee8ecaa50 "`\304G\367\377\177") at extra/../mark.c:374
#5 0x00007ffff7465a8d in GC_stopped_mark (stop_func=stop_func@entry=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:875
#6 0x00007ffff7466724 in GC_try_to_collect_inner (stop_func=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:624
#7 0x00007ffff7466a22 in GC_collect_or_expand (needed_blocks=needed_blocks@entry=1, ignore_off_page=ignore_off_page@entry=0, retry=retry@entry=0) at extra/../alloc.c:1688
#8 0x00007ffff746878f in GC_allocobj (gran=<optimized out>, kind=<optimized out>) at extra/../alloc.c:1798
#9 GC_generic_malloc_inner (lb=<optimized out>, k=k@entry=1) at extra/../malloc.c:193
#10 0x00007ffff746cd40 in GC_generic_malloc_many (lb=<optimized out>, k=<optimized out>, result=<optimized out>) at extra/../mallocx.c:477
#11 0x00007ffff746cf35 in GC_malloc_kind (bytes=120, kind=1) at extra/../thread_local_alloc.c:187
#12 0x00007ffff796ede5 in nix::allocBytes (n=<optimized out>, n=<optimized out>) at ../src/libexpr/include/nix/expr/eval-inline.hh:19
This is because it will use the stack pointer of the coroutine, so it
will scan a region of memory that doesn't exist, e.g.
Stack for thread 0x7ffea4ff96c0 is [0x7ffe80197af0w,0x7ffea4ffa000)
(where 0x7ffe80197af0w is the sp of the coroutine and 0x7ffea4ffa000
is the base of the thread stack).
We don't scan coroutine stacks, because currently they don't have GC
roots (there is no evaluation happening in coroutines). So there is
currently no need to restore the other parts of the original patch,
such as BoehmGCStackAllocator.
GCC doesn't really benefit as much as Clang does from
using precompiled headers. Another aspect to consider is that
clangd doesn't really like GCC's PCH flags in the compilation database,
so GCC based devshells would continue to work with clangd.
This also has the slight advantage of ensuring that our includes are in
order, since we build with both Clang and GCC.
Instead of parsing a structured attrs at some later point, we parsed it
right away when parsing the A-Term format, and likewise serialize it to
`__json = <JSON dump>` when serializing a derivation to A-Term.
The JSON format can directly contain the JSON structured attrs without
so encoding it, so we just do that.
Add a new setting to warn about path literals that don't start with "." or "/". When enabled,
expressions like `foo/bar` will emit a warning suggesting to use `./foo/bar` instead.
A functional test is included.
The setting defaults to false for backward compatibility but could eventually default to true in
the future.
Closes: #13374
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
* It is tough to contribute to a project that doesn't use a formatter,
* It is extra hard to contribute to a project which has configured the formatter, but ignores it for some files
* Code formatting makes it harder to hide obscure / weird bugs by accident or on purpose,
Let's rip the bandaid off?
Note that PRs currently in flight should be able to be merged relatively easily by applying `clang-format` to their tip prior to merge.
The default constructor for Attr was not initializing the value pointer,
which could lead to undefined behavior when the uninitialized pointer is
accessed. This was caught by clang-tidy's UninitializedObject check.
This fixes the warning:
1 uninitialized field at the end of the constructor call
[clang-analyzer-optin.cplusplus.UninitializedObject]
c39cc00404 has added assertions for
all Value accesses and the following case has started failing with
an `unreachable`:
(/tmp/fun.nix):
```nix
{a}: a
```
```
$ nix eval --impure --expr 'import /tmp/fun.nix {a="a";b="b";}'
```
This would crash:
```
terminating due to unexpected unrecoverable internal error: Unexpected condition in getStorage at ../include/nix/expr/value.hh:844
```
This is not a regression, but rather surfaces an existing problem, which previously
was left undiagnosed. In the case of an import `fun` is the `import` primOp, so that read is invalid
and previously this resulted in an access into an inactive union member, which is UB.
The correct thing to use is `vCur`. Identical problem also affected the case of a missing argument.
Add previously failing test cases to the functional/lang test suite.
Fixes#13448.
Fixes:
[261/394] Linking target src/libexpr/libnixexpr.so
In function ‘copy’,
inlined from ‘__ct ’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:688:23,
inlined from ‘operator+’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:3735:43,
inlined from ‘operator()’ at ../src/libexpr/primops/fetchClosure.cc:127:58,
inlined from ‘prim_fetchClosure’ at ../src/libexpr/primops/fetchClosure.cc:132:88:
/nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/char_traits.h:427:56: warning: ‘__builtin_memcpy’ writing 74 bytes into a region of size 16 overflows the destination [-Wstringop-overflow=]
427 | return static_cast<char_type*>(__builtin_memcpy(__s1, __s2, __n));
| ^
../src/libexpr/primops/fetchClosure.cc: In function ‘prim_fetchClosure’:
../src/libexpr/primops/fetchClosure.cc:132:88: note: at offset 16 into destination object ‘<anonymous>’ of size 32
132 | fromPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
| ^
This shaves off a very significand amount of memory used
for evaluation as well as reduces the GC-managed heap.
Previously the union discriminator (InternalType) was
stored as a separate field in the Value, which takes up
whole 8 bytes due to padding needed for member alignment.
This effectively wasted 7 whole bytes of memory. Instead
of doing that InternalType is instead packed into pointer
alignment niches. As it turns out, there's more than enough
unused bits there for the bit packing to be effective.
See the doxygen comment in the ValueStorage specialization
for more details.
This does not add any performance overhead, even though
we now consistently assert the InternalType in all getters.
This can also be made atomic with a double width compare-and-swap
instruction on x86_64 (CMPXCHG16B instruction) for parallel evaluation.
This factors out most of the value representation into a mixin class.
`finishValue` is now gone for good and replaced with a simple template
function `setStorage` which derives the type information/disriminator from
the type of the argument. Likewise, reading of the value goes through function
template `getStorage`.
An empty type `Null` is introduced to make the bijection InternalType <-> C++ type
complete.
