1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-09 03:56:01 +01:00
Commit graph

13028 commits

Author SHA1 Message Date
Sergei Zimmerman
0f08feaa58
libexpr: Remove unused members from ParserLocation 2025-09-28 22:57:11 +03:00
Taeer Bar-Yam
eab467ecfb libexpr: introduce arena to hold ExprString strings
1. Saves 24-32 bytes per string (size of std::string)
2. Saves additional bytes by not over-allocating strings (in total we
save ~1% memory)
3. Sets us up to perform a similar transformation on the other Expr
subclasses
4. Makes ExprString trivially moveable (before the string data might
move, causing the Value's pointer to become invalid). This is important
so we can put ExprStrings in an std::vector and refer to them by index

We have introduced a string copy in ParserState::stripIndentation().
This could be removed by pre-allocating the right sized string in the
arena, but this adds complexity and doesn't seem to improve performance,
so for now we've left the copy in.
2025-09-28 14:23:13 -04:00
Sergei Zimmerman
c43ea09b9b
Merge pull request #14100 from obsidiansystems/add-missing-pragma-once
Add `#pragma once` to `dummy-store.hh`
2025-09-28 16:41:28 +00:00
John Ericson
582d3ee611 Add #pragma once to dummy-store.hh
We should have a lint for this.

In later (yet to be merged at this time) commits, this started causing
problems that only the sanitzer caught.
2025-09-28 12:12:24 -04:00
John Ericson
f7d35dc1dc
Merge pull request #14099 from xokdvium/fix-assert-failure
libstore: Call canonPath for constructing LocalFSStoreConfig::rootDir
2025-09-28 12:06:02 -04:00
Sergei Zimmerman
0866ba0b4a
libstore: Deduplicate LocalFSStoreConfig::rootDir initializers
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2025-09-28 18:38:57 +03:00
John Ericson
a01d52e57b
Merge pull request #14063 from obsidiansystems/test-less-macro
Minimize the use of C Macros for characterization tests
2025-09-28 11:14:29 -04:00
Sergei Zimmerman
3a64d3c0da
libstore: Call canonPath for constructing LocalFSStoreConfig::rootDir
This mirrors what OptionalPathSetting does. Otherwise we run into
an assertion failure for relative paths specified as the authority + path:

nix build nixpkgs#hello --store "local://a/b"
nix: ../posix-source-accessor.cc:13: nix::PosixSourceAccessor::PosixSourceAccessor(std::filesystem::__cxx11::path&&): Assertion `root.empty() || root.is_absolute()' failed.

This is now diagnosed properly:

error: not an absolute path: 'a/b'

Just as you'd specify the root via a query parameter:

nix build nixpkgs#hello --store "local?root=a/b"
2025-09-28 17:42:19 +03:00
John Ericson
01b2037bc0 Minimize the use of C Macros for characterization tests
Fewer macros is better!

Introduce a new `JsonChacterizationTest` mixin class to help with this.

Also, avoid some needless copies with `GetParam`.

Part of my effort shoring up the JSON formats with #13570.
2025-09-28 09:54:46 -04:00
Sergei Zimmerman
426a72c9cf
libstore: Make all StoreConfig::getReference implementations return store parameters
These stragglers have been accidentally left out when implementing the StoreConfig::getReference.
Also HttpBinaryCacheStore::getReference now returns the actual store parameters, not the cacheUri
parameters.
2025-09-28 16:29:12 +03:00
John Ericson
e731c43eae Use std::variant to enforce BuildResult invariants
There is now a clean separation between successful and failing build
results.
2025-09-27 15:56:06 -04:00
John Ericson
43550e8edb Lock down BuildResult::Status enum values
This allows refactoring without changing wire protocol by mistake.
2025-09-27 15:15:20 -04:00
John Ericson
3c610df550 Delete scratch data for CA derivation that produced already-extant output
In the case where the store object doesn't exist, we do correctly move
(rather than copy) the scratch data into place. In this case, the
destination store object already exists, but we still want to clean up
after ourselves.
2025-09-27 15:14:33 -04:00
John Ericson
3bf1268ac6
Merge pull request #14096 from Mic92/concurrency-bugs-2
document thread-unsafe mutation in PosixSourceAccessor
2025-09-27 11:18:56 -04:00
Yaroslav Bolyukin
0a3eb22360
fix: wait on incomplete assignment in REPL
Fixes: https://github.com/NixOS/nix/issues/13507
2025-09-27 13:25:27 +03:00
Jörg Thalheim
866c9179a0 document thread-unsafe mutation in PosixSourceAccessor 2025-09-26 23:31:56 +02:00
Jörg Thalheim
6721ef5feb
Merge branch 'master' into fix-fetch-to-store-caching 2025-09-26 20:44:39 +02:00
Jörg Thalheim
b5f765b7eb
Merge pull request #14047 from Radvendii/eval-memory
libexpr: move eval memory allocation to own struct
2025-09-26 19:24:30 +02:00
Tristan Ross
bb6a4dccdf
libutil-c: add nix_set_verbosity function 2025-09-26 08:31:23 -07:00
Eelco Dolstra
eec4dece33 Args::Flag: Add required attribute 2025-09-26 16:01:59 +02:00
Seth Flynn
ff82de86da
nix-cli: use pure/restricted eval for help pages
This avoids any complications that can arise from the environment
affecting evaluation of the help pages (which don't need to be calling
out to anything external anyways)

A recent example of one of these problems is
https://github.com/NixOS/nix/issues/14085, which would break help pages
by causing them to make invalid calls to the dummy store they're
evaluated with

Fixes: https://github.com/NixOS/nix/issues/14062
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-26 02:05:58 -04:00
Taeer Bar-Yam
7b3c193bd3
libexpr: move eval memory allocation to own struct
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>

See original commit on lix:
f5754dc90a
2025-09-26 00:40:43 +03:00
Tristan Ross
cf595b81d5
libmain-c: add nix_set_log_format function 2025-09-25 10:57:48 -07:00
Eelco Dolstra
4b9735b761 Test against uncacheable paths
This is to test the non-functional property that most paths should be
cacheable. We've had frequent cases where caching broken but we didn't
notice.
2025-09-25 11:30:11 -04:00
Eelco Dolstra
1d130492d7 Mount inputs on storeFS to restore fetchToStore() caching
fetchToStore() caching was broken because it uses the fingerprint of
the accessor, but now that the accessor (typically storeFS) is a
composite (like MountedSourceAccessor or AllowListSourceAccessor),
there was no fingerprint anymore. So fetchToStore now uses the new
getFingerprint() method to get the specific fingerprint for the
subpath.
2025-09-25 11:30:11 -04:00
Eelco Dolstra
ec6d5c7de3 Path fetcher: Simplify fingerprint computation 2025-09-25 11:21:14 -04:00
Eelco Dolstra
3450a72ba0 Git fetcher: Make dirty repos with no commits cacheable 2025-09-25 11:20:00 -04:00
Eelco Dolstra
28d11c5bcc Add SourceAccessor::getFingerprint()
This returns the fingerprint for a specific subpath. This is intended
for "composite" accessors like MountedSourceAccessor, where different
subdirectories can have different fingerprints.
2025-09-25 11:20:00 -04:00
Eelco Dolstra
55c7ef9d40 SourceAccessor: Make lstat() virtual
With FilteringSourceAccessor, lstat() needs to throw a different
exception if the path is inaccessible than if it doesn't exist.
2025-09-25 11:20:00 -04:00
John Ericson
46095284f1
Merge pull request #14080 from NixOS/storeFS-prep
Some `storeFS` and similar cleanup
2025-09-25 10:50:25 -04:00
Jörg Thalheim
099a74e9f4
Merge pull request #14041 from getchoo-contrib/getchoo/cache-substituted-inputs
libfetchers: avoid re-copying substituted inputs
2025-09-25 13:27:02 +02:00
Seth Flynn
74305d5260
libfetchers: avoid re-copying substituted inputs
Previously, Nix would not create a cache entry for substituted/cached
inputs

This led to severe slowdowns in some scenarios where a large input (like
Nixpkgs) had already been unpacked to the store but didn't exist in a
users cache, as described in https://github.com/NixOS/nix/issues/11228

Using the same method as https://github.com/NixOS/nix/pull/12911, we can
create a cache entry for the fingerprint of substituted/cached inputs
and avoid this problem entirely
2025-09-25 04:04:57 -04:00
Jörg Thalheim
bdd3eb400a
Merge pull request #14030 from roberth/c-api-item-access
C API: Various improvements to attribute/item access
2025-09-25 09:06:11 +02:00
Jörg Thalheim
eb04a6d3e3
Merge pull request #14079 from obsidiansystems/clean-up-why-depends
Clean up `nix why-depends` store accessor usage, and put back store dir in output
2025-09-25 08:49:12 +02:00
Eelco Dolstra
e8f951289f EvalState: Don't maintain stats by default
These counters are extremely expensive in a multi-threaded
program. For instance, disabling them speeds up evaluation of the
NixOS/nix/2.21.2 from 32.6s to 17.8s.
2025-09-25 08:03:24 +02:00
Eelco Dolstra
8d257f5510 EvalState: Make the counters atomic 2025-09-25 08:03:24 +02:00
John Ericson
9b2f282af5 Simplify the definition of rootFS
It was getting very hard to follow.
2025-09-25 00:14:14 -04:00
Eelco Dolstra
35189c0ae0 Expose the fact that storeFS is a MountedSourceAccessor
This will become useful.
2025-09-25 00:14:14 -04:00
John Ericson
8ef70ef522 Rename one overload to allowPathLegacy
Makes it easier to tell when it is isued.
2025-09-25 00:14:14 -04:00
Eelco Dolstra
339338e166 MountedSourceAccessor: Move into a separate header, add mount method 2025-09-25 00:14:14 -04:00
John Ericson
d26dee20b2 Clean up nix why-depends store accessor usage, and put back store dir in output
With this change, the store-wide `getFSAccessor` has only one usage left
--- the evaluator. If we get rid of that (as is planned), we can then
remove that method altogether, simplifying `Store`. Hurray!

I removed the store dir by mistake from the pretty-printed (for humans)
output in eb643d034f. That change was not
supposed to change output.
2025-09-24 23:06:03 -04:00
John Ericson
30691c38c2 Add JSON tests for Realisation 2025-09-24 18:09:24 -04:00
John Ericson
9f26c20ebd
Merge pull request #14057 from obsidiansystems/derived-path-json
Modernize and test derived path JSON
2025-09-24 18:08:06 -04:00
John Ericson
f2cd6235a7
Merge pull request #14056 from obsidiansystems/better-output-spec-json-tests
Convert `{Extended,}OutputsSpec` JSON tests to characterization tests
2025-09-24 17:41:25 -04:00
Sergei Zimmerman
05279f2ba0
Merge pull request #14061 from xokdvium/k-way-update
libexpr: Preparation for more k-way updates of attribute sets (NFC)
2025-09-25 00:29:54 +03:00
John Ericson
a97d6d89d8 Create a second Store::getFSAccessor for a single store object
This is sometimes easier / more performant to implement, and
independently it is also a more convenient interface for many callers.

The existing store-wide `getFSAccessor` is only used for

 - `nix why-depends`
 - the evaluator

I hope we can get rid of it for those, too, and then we have the option
of getting rid of the store-wide method.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-24 15:49:14 -04:00
Sergei Zimmerman
97ce7759d0
libexpr: Use same naive iterative merging but with evalForUpdate 2025-09-24 21:47:59 +03:00
Sergei Zimmerman
9789019a50
libexpr: Move *StackReservation constants to gc-small-vector.hh
There are other places where it's useful to use these constants
(notably in eval.hh).
2025-09-24 01:05:18 +03:00
Sergei Zimmerman
b7c6cf900f
libexpr: Explicitly define ExprOpUpdate 2025-09-24 01:04:26 +03:00
Sergei Zimmerman
e282175f48
libexpr: Split out MakeBinOpMembers from MakeBinOp 2025-09-24 01:04:23 +03:00