Jörg Thalheim
6721ef5feb
Merge branch 'master' into fix-fetch-to-store-caching
2025-09-26 20:44:39 +02:00
Taeer Bar-Yam
7b3c193bd3
libexpr: move eval memory allocation to own struct
...
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
See original commit on lix:
f5754dc90a
2025-09-26 00:40:43 +03:00
Eelco Dolstra
1d130492d7
Mount inputs on storeFS to restore fetchToStore() caching
...
fetchToStore() caching was broken because it uses the fingerprint of
the accessor, but now that the accessor (typically storeFS) is a
composite (like MountedSourceAccessor or AllowListSourceAccessor),
there was no fingerprint anymore. So fetchToStore now uses the new
getFingerprint() method to get the specific fingerprint for the
subpath.
2025-09-25 11:30:11 -04:00
Eelco Dolstra
55c7ef9d40
SourceAccessor: Make lstat() virtual
...
With FilteringSourceAccessor, lstat() needs to throw a different
exception if the path is inaccessible than if it doesn't exist.
2025-09-25 11:20:00 -04:00
John Ericson
46095284f1
Merge pull request #14080 from NixOS/storeFS-prep
...
Some `storeFS` and similar cleanup
2025-09-25 10:50:25 -04:00
Eelco Dolstra
e8f951289f
EvalState: Don't maintain stats by default
...
These counters are extremely expensive in a multi-threaded
program. For instance, disabling them speeds up evaluation of the
NixOS/nix/2.21.2 from 32.6s to 17.8s.
2025-09-25 08:03:24 +02:00
Eelco Dolstra
8d257f5510
EvalState: Make the counters atomic
2025-09-25 08:03:24 +02:00
John Ericson
9b2f282af5
Simplify the definition of rootFS
...
It was getting very hard to follow.
2025-09-25 00:14:14 -04:00
Eelco Dolstra
35189c0ae0
Expose the fact that storeFS is a MountedSourceAccessor
...
This will become useful.
2025-09-25 00:14:14 -04:00
John Ericson
8ef70ef522
Rename one overload to allowPathLegacy
...
Makes it easier to tell when it is isued.
2025-09-25 00:14:14 -04:00
Eelco Dolstra
339338e166
MountedSourceAccessor: Move into a separate header, add mount method
2025-09-25 00:14:14 -04:00
Sergei Zimmerman
97ce7759d0
libexpr: Use same naive iterative merging but with evalForUpdate
2025-09-24 21:47:59 +03:00
Sergei Zimmerman
9789019a50
libexpr: Move *StackReservation constants to gc-small-vector.hh
...
There are other places where it's useful to use these constants
(notably in eval.hh).
2025-09-24 01:05:18 +03:00
Sergei Zimmerman
b7c6cf900f
libexpr: Explicitly define ExprOpUpdate
2025-09-24 01:04:26 +03:00
Sergei Zimmerman
e282175f48
libexpr: Split out MakeBinOpMembers from MakeBinOp
2025-09-24 01:04:23 +03:00
Robert Hensing
169a368459
Merge pull request #14040 from NixOS/import-thunk
...
Ensure that files are parsed/evaluated only once (2nd attempt)
2025-09-22 18:33:00 +02:00
David McFarland
32d4ea8140
fix cross-build for cygwin
2025-09-22 12:27:04 -03:00
Eelco Dolstra
d32d77f4d4
Allocate ExprParseFile on the heap for now
...
https://github.com/NixOS/nix/pull/14013#issuecomment-3308085755
2025-09-22 12:06:51 +02:00
Eelco Dolstra
5f60602875
Reapply "Merge pull request #13938 from NixOS/import-thunk"
...
This reverts commit fd034814dc
2025-09-22 11:48:58 +02:00
Robert Hensing
71b27774f0
libexpr: Document {eval,maybeThunk} methods
2025-09-22 01:41:02 +03:00
Sergei Zimmerman
94d37e62fc
treewide: Support builds with ASAN, enable in CI
...
Enables builds with ASAN to catch memory corruption
bugs faster and in CI. This is an incredibly valuable
instrument that must be used as much as possible.
Somewhat based on jade's work from Lix, though there's a lot that
we have to do differently:
19ae87e5ce
2025-09-19 01:33:57 +03:00
Sergei Zimmerman
fd034814dc
Revert "Merge pull request #13938 from NixOS/import-thunk"
...
This has multiple dangling pointer issues that lead to segfaults in e.g.:
nix eval --expr '(builtins.getFlake "github:nixos/nixpkgs/25.05")' --impure
This reverts commit ad175727e4d314750174
2025-09-18 01:52:46 +03:00
Sergei Zimmerman
6138bc3de3
libexpr: Structural sharing of attrsets
...
This changes the implementation of Bindings to allow
for a more space-efficient implementation of attribute
set merges. This is accomplished by "layering" over the "base" Bindings.
The top "layer" is naturally the right-hand-side of the update operator //.
Such an implementation leads to significantly better memory usage on
something like nixpkgs:
nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux > /dev/null
Comparison against 2b0fd88324
2025-09-17 23:54:45 +03:00
Jens Petersen
86bb7c958a
meson: refactor nix_soversion into nix-meson-build-support/common
...
This is a follow-on to #13995 which added soversion to the libraries
2025-09-16 12:54:30 +08:00
Jörg Thalheim
0b401e2199
Merge pull request #13995 from juhp/soversion
...
meson: add soversion with nix version to give SONAME to libs
2025-09-15 20:36:31 +02:00
Eelco Dolstra
ad175727e4
Merge pull request #13938 from NixOS/import-thunk
...
Ensure that files are parsed/evaluated only once
2025-09-15 19:03:18 +02:00
Jens Petersen
dd1a554aba
meson: add soversion with nix version to give SONAME to libs ( #13960 , #13979 )
...
remove 'pre' version suffix for non-releases (chokes Darwin ld)
2025-09-15 22:08:26 +08:00
Sergei Zimmerman
d830840433
libexpr: Remove Bindings::find
...
A follow-up optimization will make it impossible to make a find function
that returns an iterator in an efficient manner. All consumer code can
easily use the `get` variant.
2025-09-14 23:29:44 +03:00
Sergei Zimmerman
ddabd94f82
libexpr: Make Bindings::iterator a proper strong type instead of pointer
...
As evident from the number of tests that were holding this API completely
wrong (the end() iterator returned from find() is NEVER nullptr) we should
not have this footgun. A proper strong type guarantees that this confusion
will not happen again.
Also this will be helpful down the road when Bindings becomes something
smarter than an array of Attr.
2025-09-14 22:52:37 +03:00
Sergei Zimmerman
e75501da3e
libexpr: Remove non-const iterators of Bindings
2025-09-13 23:21:24 +03:00
Sergei Zimmerman
92df96543c
Merge pull request #13972 from xokdvium/no-string-values-in-evalstate
2025-09-12 21:19:09 +00:00
Sergei Zimmerman
f4c38278ca
libexpr: Remove vString* Values from EvalState
...
EvalState is too big and cluttered. These strings
can be private constant statics.
2025-09-12 23:44:52 +03:00
Sergei Zimmerman
0db2b8c8fe
Revert "meson: add soversion to libraries ( #13960 )"
...
This reverts commit bdbc739d6e
2025-09-12 20:43:34 +03:00
Eelco Dolstra
8d8f49cb5a
Use concurrent_flat_map_fwd.hpp
2025-09-12 17:49:15 +02:00
Eelco Dolstra
4b63ff63a4
Remove some unnecessary hash template arguments
2025-09-12 17:26:29 +02:00
Eelco Dolstra
ad6eb22368
Ensure that files are parsed/evaluated only once
...
When doing multithreaded evaluation, we want to ensure that any Nix
file is parsed and evaluated only once. The easiest way to do this is
to rely on thunks, since those ensure locking in the multithreaded
evaluator. `fileEvalCache` is now a mapping from `SourcePath` to a
`Value *`. The value is initially a thunk (pointing to a
`ExprParseFile` helper object) that can be forced to parse and
evaluate the file. So a subsequent thread requesting the same file
will see a thunk that is possibly locked and wait for it.
The parser cache is gone since it's no longer needed. However, there
is a new `importResolutionCache` that maps `SourcePath`s to
`SourcePath`s (e.g. `/foo` to `/foo/default.nix`). Previously we put
multiple entries in `fileEvalCache`, which was ugly and could result
in work duplication.
2025-09-12 17:05:21 +02:00
Jörg Thalheim
377b60ee9b
Merge pull request #13926 from NaN-git/opt_boost-unordered
...
replace all occurences of std::unordered_* by equivalents from boost
2025-09-12 11:46:42 +02:00
Jörg Thalheim
429812cdb8
Merge pull request #13966 from juhp/patch-1
...
meson: add soversion to libraries (#13960 )
2025-09-12 08:25:12 +02:00
Jens Petersen
bdbc739d6e
meson: add soversion to libraries ( #13960 )
2025-09-12 14:06:39 +08:00
Sergei Zimmerman
c0b35c71cd
libexpr: Fix build without Boehm
...
This should have been placed under the ifdef.
2025-09-12 04:02:07 +03:00
Sergei Zimmerman
5db4b0699c
libexpr: Make constant Values global constants, move out of EvalState
...
These constant Values have no business being in the EvalState in the
first place. The ultimate goal is to get rid of the ugly `getBuiltins`
and its relience (in `createBaseEnv`) on these global constants is getting in the way.
Same idea as in f017f9ddd3
2025-09-11 01:53:41 +03:00
Sergei Zimmerman
462b9ac49c
libexpr: Make Value::isa and Value::getStorage private methods
...
This was always intended to be the case, but accidentally left
in the public interface.
2025-09-11 01:52:17 +03:00
Sergei Zimmerman
4df1a3ca76
libexpr: Make emptyBindings a global constant
...
This object is always constant and will never get modified.
Having it as a global (constant) static is much easier and
unclutters the EvalState.
Same idea as in f017f9ddd3
2025-09-11 01:51:48 +03:00
Philipp Otterbein
9dbc2cae4f
hashmaps with string keys: add transparent lookups
2025-09-10 23:04:44 +02:00
Philipp Otterbein
9f2b6a1b94
replace more std::unordered_* types by faster boost hash maps
2025-09-10 23:04:44 +02:00
Philipp Otterbein
4f8c50fb77
libexpr: replace std::unordered_* types by faster boost hash maps
2025-09-10 23:04:44 +02:00
Sergei Zimmerman
2ed2c79721
libexpr: Fix Value::mkList for empty lists
...
This code used to save the pointer to a small
list allocated on the stack to the Value, which
is unintended.
2025-09-10 01:37:39 +03:00
Sergei Zimmerman
4524235af4
libexpr: Overalign Value to 16 bytes
...
This is necessary to make use of 128 bit atomics on x86_64 [1],
since MOVAPD, MOVAPS, and MOVDQA need memory operands to be 16-byte
aligned. We are not losing anything here, because Value is already 16-byte
wide and Boehm allocates memory in granules that are 16 bytes by default
on 64 bit systems [2].
[1]: https://patchwork.sourceware.org/project/gcc/patch/YhxkfzGEEQ9KHbBC@tucnak/
[2]: 54ac18ccbc/include/gc/gc_tiny_fl.h (L31-L33)
2025-09-09 22:18:52 +03:00
Sergei Zimmerman
bbdabe4973
libexpr: Remove decl for undefined overload of Value::mkPath
2025-09-06 16:36:16 +03:00
Jörg Thalheim
1d62ccdb3d
Merge pull request #13767 from ethanavatar/master
...
libutil, libexpr: #10542 abstract over getrusage for getting cpuTime stat and implement windows version
2025-09-06 09:26:13 +02:00
