1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-11 21:16:02 +01:00
Commit graph

3373 commits

Author SHA1 Message Date
Sergei Zimmerman
92df96543c
Merge pull request #13972 from xokdvium/no-string-values-in-evalstate 2025-09-12 21:19:09 +00:00
Sergei Zimmerman
f4c38278ca
libexpr: Remove vString* Values from EvalState
EvalState is too big and cluttered. These strings
can be private constant statics.
2025-09-12 23:44:52 +03:00
Sergei Zimmerman
0db2b8c8fe
Revert "meson: add soversion to libraries (#13960)"
This reverts commit bdbc739d6e.

Such a change needs more thought put into it. By versioning
shared libraries we'd make a false impression that libraries
themselves are actually versioned and have some sort of stable
ABI, which is not the case.

This will be useful when C bindings become stable, but as long
as they are experimental it does not make sense to set SONAME.

Also this change should not have been backported, since it's
severely breaking.
2025-09-12 20:43:34 +03:00
Jörg Thalheim
377b60ee9b
Merge pull request #13926 from NaN-git/opt_boost-unordered
replace all occurences of std::unordered_* by equivalents from boost
2025-09-12 11:46:42 +02:00
Jörg Thalheim
429812cdb8
Merge pull request #13966 from juhp/patch-1
meson: add soversion to libraries (#13960)
2025-09-12 08:25:12 +02:00
Jens Petersen
bdbc739d6e meson: add soversion to libraries (#13960) 2025-09-12 14:06:39 +08:00
Sergei Zimmerman
c0b35c71cd
libexpr: Fix build without Boehm
This should have been placed under the ifdef.
2025-09-12 04:02:07 +03:00
Sergei Zimmerman
5db4b0699c
libexpr: Make constant Values global constants, move out of EvalState
These constant Values have no business being in the EvalState in the
first place. The ultimate goal is to get rid of the ugly `getBuiltins`
and its relience (in `createBaseEnv`) on these global constants is getting in the way.

Same idea as in f017f9ddd3.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-09-11 01:53:41 +03:00
Sergei Zimmerman
462b9ac49c
libexpr: Make Value::isa and Value::getStorage private methods
This was always intended to be the case, but accidentally left
in the public interface.
2025-09-11 01:52:17 +03:00
Sergei Zimmerman
4df1a3ca76
libexpr: Make emptyBindings a global constant
This object is always constant and will never get modified.
Having it as a global (constant) static is much easier and
unclutters the EvalState.

Same idea as in f017f9ddd3.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-09-11 01:51:48 +03:00
Philipp Otterbein
9dbc2cae4f hashmaps with string keys: add transparent lookups 2025-09-10 23:04:44 +02:00
Philipp Otterbein
9f2b6a1b94 replace more std::unordered_* types by faster boost hash maps 2025-09-10 23:04:44 +02:00
Philipp Otterbein
4f8c50fb77 libexpr: replace std::unordered_* types by faster boost hash maps 2025-09-10 23:04:44 +02:00
Sergei Zimmerman
2ed2c79721
libexpr: Fix Value::mkList for empty lists
This code used to save the pointer to a small
list allocated on the stack to the Value, which
is unintended.
2025-09-10 01:37:39 +03:00
Sergei Zimmerman
4524235af4
libexpr: Overalign Value to 16 bytes
This is necessary to make use of 128 bit atomics on x86_64 [1],
since MOVAPD, MOVAPS, and MOVDQA need memory operands to be 16-byte
aligned. We are not losing anything here, because Value is already 16-byte
wide and Boehm allocates memory in granules that are 16 bytes by default
on 64 bit systems [2].

[1]: https://patchwork.sourceware.org/project/gcc/patch/YhxkfzGEEQ9KHbBC@tucnak/
[2]: 54ac18ccbc/include/gc/gc_tiny_fl.h (L31-L33)
2025-09-09 22:18:52 +03:00
Sergei Zimmerman
bbdabe4973
libexpr: Remove decl for undefined overload of Value::mkPath 2025-09-06 16:36:16 +03:00
Jörg Thalheim
1d62ccdb3d
Merge pull request #13767 from ethanavatar/master
libutil, libexpr: #10542 abstract over getrusage for getting cpuTime stat and implement windows version
2025-09-06 09:26:13 +02:00
Sergei Zimmerman
738924b705
libexpr: Slim down Bindings to 8 bytes (on 64 bit systems)
Since the only construction and push_back() calls
to Bindings happen through the `BindingsBuilder` [1] we don't
need to keep `capacity` around on the heap anymore. This saves 8 bytes
(because of the member alignment padding)
per one Bindings allocation. This isn't that much, but it does
save significant memory.

This also shows that the Bindings don't necessarily have to
be mutable, which opens up opportunities for doing small bindings
optimization and storing a 1-element Bindings directly in Value.

For the following scenario:

nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux

(nixpkgs revision: ddcddd7b09a417ca9a88899f4bd43a8edb72308d)

This patch results in reduction of `sets.bytes` 13115104016 -> 12653087640,
which amounts to 462 MB less bytes allocated for Bindings.

[1]: Not actually, `getBuiltins` does mutate bindings, but this is pretty
     inconsequential and doesn't lead to problems.
2025-09-06 00:23:54 +03:00
John Ericson
49e9c14e2f
Merge pull request #13900 from NixOS/fix-mingw-windows-build
Fix downstream MinGW build by not looking for Boost Regex
2025-09-04 21:05:12 -04:00
John Ericson
819bf13607
Merge pull request #13880 from Mic92/static-alloc-symbol-ids
libexpr: Convert Symbol comparisons to switch statements
2025-09-03 17:13:12 -04:00
Jörg Thalheim
cbcb434cb3 libexpr: Convert Symbol comparisons to switch statements
Now that Symbols are statically allocated at compile time with known IDs,
we can use switch statements instead of if-else chains for Symbol comparisons.
This provides better performance through compiler optimizations like jump tables.

Changes:
- Add public getId() method to Symbol class to access the internal ID
- Convert if-else chains comparing Symbol values to switch statements
  in primops.cc's derivationStrictInternal function
- Simplify control flow by removing the 'handled' flag and moving the
  default attribute handling into the switch's default case

The static and runtime Symbol IDs are guaranteed to match by the
copyIntoSymbolTable implementation which asserts this invariant.

Co-authored-by: John Ericson <git@JohnEricson.me>
2025-09-03 10:13:12 +02:00
John Ericson
6bdb5e8e09 Fix downstream MinGW build by not looking for Boost Regex 2025-09-02 10:41:39 -04:00
Sergei Zimmerman
34181afc6a
libexpr: Use mkStringNoCopy in prim_typeOf
This would lead to an unnecessary allocation. Not
a significant issue by any means, but it doesn't
have to allocate for most cases.
2025-09-02 00:16:11 +03:00
Eelco Dolstra
d62cfc1c97
Re-introduce mkStringNoCopy (revised)
In b70d22b `mkStringNoCopy()` was renamed to
`mkString()`, but this is a bit risky since in code like

    vStringRegular.mkString("regular");

we want to be sure that the right overload is picked. (This is
especially problematic since the overload that takes an
`std::string_view` *does* allocate.)  So let's be explicit.

(Rebased from https://github.com/NixOS/nix/pull/11551)
2025-09-02 00:16:06 +03:00
Jörg Thalheim
fea4a29c0a
Merge pull request #13883 from xokdvium/toml-timestamps-reapply
Reapply "Merge pull request #13741 from xokdvium/toml-timestamps"
2025-09-01 09:12:33 +02:00
Sergei Zimmerman
8251305aff
Reapply "Merge pull request #13741 from xokdvium/toml-timestamps"
This reverts commit 75740fbd75.
2025-09-01 01:26:14 +03:00
Sergei Zimmerman
363620dd24 libexpr: Statically allocate commonly used symbols
The motivation for this change is two-fold:

1. Commonly used Symbol values can be referred to
   quite often and they can be assigned at compile-time
   rather than runtime.

2. This also unclutters EvalState constructor, which was
   getting very long and unreadable.

Spiritually similar to https://gerrit.lix.systems/c/lix/+/2218,
though that patch doesn't allocate the Symbol at compile time.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-08-31 13:24:06 +02:00
John Ericson
e82210b3b2 Implement parseURLRelative, use in HttpBinaryCacheStore
This allows us to replace some very hacky and not correct string
concatentation in `HttpBinaryCacheStore`. It will especially be useful
with #13752, when today's hacks started to cause problems in practice,
not just theory.

Also make `fixGitURL` returned a `ParsedURL`.
2025-08-26 19:45:10 -04:00
Sergei Zimmerman
f0e4af4365
libexpr: Fix weird formatting after treewide reformat 2025-08-25 22:09:18 +03:00
Ethan Evans
7b8ceb5d2d libutil, libexpr: #10542 abstract over getrusage for getting cpuTime stat and implement windows version
Update src/libutil/windows/current-process.cc

Prefer `nullptr` over `NULL`

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>

Update src/libutil/unix/current-process.cc

Prefer C++ type casts

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>

Update src/libutil/windows/current-process.cc

Prefer C++ type casts

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>

Update src/libutil/unix/current-process.cc

Don't allocate exception

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-08-24 18:45:33 -07:00
John Ericson
72a548ed6a Limit to lenient parsing of non-standard URLs only where needed
This allows us to put `parseURL` in more spots without furthering
technical debt.
2025-08-22 12:37:37 -04:00
Eelco Dolstra
615b10cb44
Merge pull request #13800 from NixOS/concurrent-eval-cache
Use WAL mode for SQLite cache databases
2025-08-21 11:42:15 +02:00
John Ericson
52212635db No more globals.hh in headers
This is needed to rearrange include order, but I also think it is a good
thing anyways, as we seek to reduce the use of global settings variables
over time.
2025-08-20 16:24:37 -04:00
Eelco Dolstra
0df147b145 Bump the version of the SQLite caches
This avoids problems with older versions of Nix that don't put the
caches in WAL mode. That's generally not a problem, until you do something like

  nix build --print-out-paths ... | cachix

which deadlocks because cachix tries to switch the caches to truncate
mode, which requires exclusive access. But the first process cannot
make progress because the cachix process isn't reading from the pipe.
2025-08-20 20:44:58 +02:00
Eelco Dolstra
4ab8ff5b4c SQLite: Use std::filesystem::path 2025-08-20 20:44:21 +02:00
Sergei Zimmerman
03101cc839
treewide: Bump C++ standard version to C++23
Compilers in nixpkgs have caught up and major distros
should also have recent enough compilers. It would be
nice to have newer features like more full featured
ranges and deducing this.
2025-08-19 15:33:27 +03:00
Sergei Zimmerman
1d943f5819
flake: Apply nixfmt 1.0.0 2025-08-18 20:29:45 +03:00
Jörg Thalheim
75740fbd75 Revert "Merge pull request #13741 from xokdvium/toml-timestamps"
This reverts commit 53ac49f72c, reversing
changes made to 8e5ca787f4.

This broke nixpkgs eval test that was depending overflowing integers...
2025-08-13 22:46:09 +02:00
Sergei Zimmerman
dc769d72cb
libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339
2025-08-12 21:49:30 +03:00
Sergei Zimmerman
d8fc55a46e
libexpr: Use table.size() instead of unnecessary loop 2025-08-12 14:58:53 +03:00
Sergei Zimmerman
a80a5c4dba
libexpr: Use recursive lambda instead of std::function
There's no reason to use a std::function for recursive lambdas
since there are polymorphic lambdas.
2025-08-12 14:55:29 +03:00
Sergei Zimmerman
df4e55ffc1
libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.
2025-08-12 14:31:38 +03:00
Sergei Zimmerman
2c38ad2cfa
libutil: Include necessary headers in config-impl.hh
These headers need to be included before everything else
(at least in GCC).
2025-08-11 20:44:27 +03:00
Jörg Thalheim
46b6907346 Revert "Merge pull request #13709 from NixOS/boehm-coroutines-sp"
This reverts commit 4b3ca9bd80, reversing
changes made to 867b69f533.

Since this commit we get reproducible segfaults building Nix ci in macos github runners:
https://github.com/NixOS/nix/actions/runs/16885882321/job/47837390248
2025-08-11 19:18:04 +02:00
Jörg Thalheim
b2fb421386
Merge pull request #13490 from Mic92/clang-tidy-simple-warnings
Fix simple clang-tidy warnings
2025-08-11 18:21:46 +02:00
Jörg Thalheim
4b3ca9bd80
Merge pull request #13709 from NixOS/boehm-coroutines-sp
Restore fixupBoehmStackPointer
2025-08-11 16:17:14 +02:00
Eelco Dolstra
3ba103865d Restore fixupBoehmStackPointer
This was removed in https://github.com/NixOS/nix/pull/11152. However,
we need it for the multi-threaded evaluator, because otherwise Boehm
GC will crash while scanning the thread stack:

  #0  GC_push_all_eager (bottom=<optimized out>, top=<optimized out>) at extra/../mark.c:1488
  #1  0x00007ffff74691d5 in GC_push_all_stack_sections (lo=<optimized out>, hi=<optimized out>, traced_stack_sect=0x0) at extra/../mark_rts.c:704
  #2  GC_push_all_stacks () at extra/../pthread_stop_world.c:876
  #3  GC_default_push_other_roots () at extra/../os_dep.c:2893
  #4  0x00007ffff746235c in GC_mark_some (cold_gc_frame=0x7ffee8ecaa50 "`\304G\367\377\177") at extra/../mark.c:374
  #5  0x00007ffff7465a8d in GC_stopped_mark (stop_func=stop_func@entry=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:875
  #6  0x00007ffff7466724 in GC_try_to_collect_inner (stop_func=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:624
  #7  0x00007ffff7466a22 in GC_collect_or_expand (needed_blocks=needed_blocks@entry=1, ignore_off_page=ignore_off_page@entry=0, retry=retry@entry=0) at extra/../alloc.c:1688
  #8  0x00007ffff746878f in GC_allocobj (gran=<optimized out>, kind=<optimized out>) at extra/../alloc.c:1798
  #9  GC_generic_malloc_inner (lb=<optimized out>, k=k@entry=1) at extra/../malloc.c:193
  #10 0x00007ffff746cd40 in GC_generic_malloc_many (lb=<optimized out>, k=<optimized out>, result=<optimized out>) at extra/../mallocx.c:477
  #11 0x00007ffff746cf35 in GC_malloc_kind (bytes=120, kind=1) at extra/../thread_local_alloc.c:187
  #12 0x00007ffff796ede5 in nix::allocBytes (n=<optimized out>, n=<optimized out>) at ../src/libexpr/include/nix/expr/eval-inline.hh:19

This is because it will use the stack pointer of the coroutine, so it
will scan a region of memory that doesn't exist, e.g.

  Stack for thread 0x7ffea4ff96c0 is [0x7ffe80197af0w,0x7ffea4ffa000)

(where 0x7ffe80197af0w is the sp of the coroutine and 0x7ffea4ffa000
is the base of the thread stack).

We don't scan coroutine stacks, because currently they don't have GC
roots (there is no evaluation happening in coroutines). So there is
currently no need to restore the other parts of the original patch,
such as BoehmGCStackAllocator.
2025-08-07 12:09:34 +02:00
Eelco Dolstra
4c1c4f79a3 Move setting GC_THREADS into eval-gc.hh 2025-08-07 11:44:20 +02:00
Eelco Dolstra
a3ff648f90 Call GC_allow_register_threads() to enable parallel marking in Boehm GC 2025-08-07 11:44:07 +02:00
Sergei Zimmerman
385e2c3542
meson: Apply formatting universally
Now that we have applied the [1] patch, the diff is much
nicer and less noisy.

[1]: https://www.github.com/mesonbuild/meson/pull/14861
2025-08-07 02:58:29 +03:00