1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-27 20:51:00 +01:00
Commit graph

22075 commits

Author SHA1 Message Date
John Ericson
0275b64b81 JSON impl and Schema for DummyStore
This is the "keystone" that puts most of the other store-layer JSON
formats together.

Also, add some documentation for JSON testing.
2025-11-24 17:04:24 -05:00
John Ericson
622a5cd1bf Add DummyStore::operator==
Will need it for tests.
2025-11-24 17:04:24 -05:00
John Ericson
b0c016ae7d DummyStore build trace holds UnkeyedRealisation by value
Otherwise the equality instance we need to add will be messed up.
2025-11-24 17:04:24 -05:00
John Ericson
f78e88c973 Add some infrastructure changes for better JSON ref<T> impls
Also skip a trailing semicolon inside a macro so the caller can use it
instead, which is generally nicer to the formatter.
2025-11-24 17:04:23 -05:00
John Ericson
f198e9a0b3 Document the JSON Schema testing a bit 2025-11-24 17:03:42 -05:00
John Ericson
3bac0d7aa2
Merge pull request #14635 from Radvendii/alloc-exprlet-exprattrs
libexpr: move the ExprLet::attrs allocations into the arena
2025-11-24 21:14:52 +00:00
John Ericson
36419a6ccb
Merge pull request #14507 from obsidiansystems/derivation-options-json-schema
JSON Schema for `DerivationOptions`
2025-11-24 21:11:07 +00:00
John Ericson
3ba51bf61b
Merge pull request #14560 from obsidiansystems/fill-in-outputs
Dedup some derivation initialization logic, and test
2025-11-24 21:10:38 +00:00
John Ericson
209f413e80 JSON Schema for DerivationOutputs
Progress on #13570
2025-11-24 15:23:50 -05:00
John Ericson
b8d32388bc Move derivation JSON doc to index.md in dir
This prepares for more structure.
2025-11-24 15:23:50 -05:00
John Ericson
eb53e61e08 Fix stray derivation "v3" in manual
It's commented out, but we should still update it to "v4" to match the
link target.
2025-11-24 15:23:50 -05:00
Taeer Bar-Yam
60f09928d1 libexpr: move ExprLet::attrs data to arena as well
I missed this because I assumed all Exprs were recursed into by
bindVars, but ExprLet's ExprAttrs field is not really its own AST node,
so it doesn't get recursed into.
2025-11-24 21:14:13 +01:00
Taeer Bar-Yam
43a183120a libexpr: factor out functions for moving data to a new allocator 2025-11-24 21:14:13 +01:00
John Ericson
0c786f3a3c
Merge pull request #14617 from vinayakankugoyal/path
Update profiles to use `std::filesystem::path`
2025-11-24 19:31:25 +00:00
John Ericson
504c5e7cf9 Convert profiles to use std::filesystem::path
Co-authored-by: Vinayak Goyal <vinayakankugoyal@gmail.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2025-11-24 13:38:01 -05:00
John Ericson
5d066386b5
Merge pull request #14260 from roberth/ulimit
Clarify setStackSize error message and warn if not possible
2025-11-24 17:12:31 +00:00
John Ericson
c7b61f3d13
Merge pull request #14631 from obsidiansystems/use-serialisation-abstraction
Use `WorkerProto::Serialise` abstraction for `DrvOutput`
2025-11-24 16:33:18 +00:00
John Ericson
d689b764f3 Use WorkerProto::Serialise abstraction for DrvOutput
It's better to consistently use the abstraction, rather than code which
happens to do the same thing.

See also d782c5e586 for the same sort of
change.
2025-11-24 10:44:45 -05:00
John Ericson
487c6b6c46
Merge pull request #14630 from NixOS/prefetch-fixes
nix/prefetch: Be honest about when path name is derived from URL
2025-11-23 22:24:17 +00:00
Sergei Zimmerman
28fac9fe4d
nix/prefetch: Be honest about when path name is derived from URL
Only add the message to trace when name is really derived from URL.
2025-11-24 00:25:48 +03:00
Sergei Zimmerman
2594e417b5
Merge pull request #14627 from jonhermansen/libstore-curl-version-maximum
libstore: fix curl version check to allow 8.17.0
2025-11-23 09:57:09 +00:00
Jon Hermansen
76ed967f79
libstore: fix curl version check to allow 8.17.0
The single-string syntax '>=8.16.0 <8.17.0' only applied the lower
bound, causing curl 8.17.0 to be incorrectly rejected. Split into two
separate version_compare() calls for compatibility with Meson 1.1,
since multi-argument syntax requires Meson 1.8+.
2025-11-23 12:13:05 +03:00
John Ericson
327e8babf7
Merge pull request #14584 from Radvendii/allocbytes-stringdata
libexpr: use allocBytes() to allocate StringData
2025-11-23 00:38:50 +00:00
John Ericson
d5d4bafc2a
Merge pull request #14620 from NixOS/revert-shared-tarball-cache
libfetchers: Don't have a single shared tarball cache
2025-11-23 00:33:51 +00:00
John Ericson
bd11043c67
Merge pull request #14623 from Radvendii/exprcall-alloc-shvach
libexpr: plug ExprCall memory leak
2025-11-23 00:08:10 +00:00
Taeer Bar-Yam
dbfe6318b3 libexpr: move ExprCall storage to the arena 2025-11-23 00:06:10 +01:00
Taeer Bar-Yam
484f40fc64 libexpr: make ExprCall::args an std::optional 2025-11-23 00:06:10 +01:00
Taeer Bar-Yam
43fc6c314d libexpr: ExprCall use std::pmr::vector 2025-11-23 00:06:10 +01:00
Sergei Zimmerman
2bbec7d573
Merge pull request #14622 from roberth/meson-commandlet-deps
src/nix: Make meson compile <cmdlet> valid
2025-11-22 19:55:02 +00:00
Sergei Zimmerman
385d7e77bd
libfetchers: Don't have a single shared tarball cache
This partially reverts commit bc6b9ce.

This transformation is unsound and thread unsafe. Internal libgit2
structures must *never* be shared between threads. This causes
internal odb corruption with e.g.:

nix flake prefetch-inputs:

error:
       … while fetching the input 'github:nixos/nixpkgs/89c2b2330e733d6cdb5eae7b899326930c2c0648?narHash=sha256-Stk9ZYRkGrnnpyJ4eqt9eQtdFWRRIvMxpNRf4sIegnw%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - upload-image.sh
error:
       … while fetching the input 'github:NixOS/nixpkgs/a8d610af3f1a5fb71e23e08434d8d61a466fc942?narHash=sha256-v5afmLjn/uyD9EQuPBn7nZuaZVV9r%2BJerayK/4wvdWA%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - outline.nix
double free or corruption (!prev)

Thread 21 "nix" received signal SIGABRT, Aborted.
2025-11-22 22:48:40 +03:00
Robert Hensing
67f6a24171 src/nix: Make meson compile <cmdlet> valid
Without this dependency, e.g. `meson compile nix-instantiate`
would produce a broken symlink, or the `nix` it points to may be
stale.
With the dependency in place, `meson compile nix-instantiate`
produces a reliable outcome.
2025-11-22 20:19:34 +01:00
Sergei Zimmerman
8cdeab8f2e
Merge pull request #14613 from roberth/deepSeq-stack-overflow
`deepSeq`, json: handle stack overflow, report list index
2025-11-22 17:49:32 +00:00
Sergei Zimmerman
ed176cb42e
Merge pull request #14618 from jonhermansen/freebsd-path-null-terminator
fix(FreeBSD): remove null terminator from executable path
2025-11-22 11:51:01 +00:00
Jon Hermansen
3ff8d0ece4 fix(FreeBSD): remove null terminator from executable path
On FreeBSD, sysctl(KERN_PROC_PATHNAME) returns a null-terminated
string with pathLen including the terminator. This causes Nix to
fail during manual generation with:

  error:
         … while calling the 'concatStringsSep' builtin
           at /nix/var/nix/builds/nix-63232-402489527/source/doc/manual/generate-settings.nix:99:1:
             98| in
             99| concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
               | ^
            100|

         error: input string '/nix/store/gq89cj02b5zs67cbd85vzg5cgsgnd8mj-nix-2.31.2/bin/nix␀'
                cannot be represented as Nix string because it contains null bytes

The issue occurs because generate-settings.nix reads the nix binary
path from JSON and evaluates it as a Nix string, which cannot contain
null bytes. Normal C++ string operations don't trigger this since they
handle null-terminated strings correctly.

Strip the null terminator on FreeBSD to match other platforms (Linux
uses /proc/self/exe, macOS uses _NSGetExecutablePath).

Credit: @wahjava (FreeBSD ports and Nixpkgs contributor)
2025-11-22 03:59:29 -05:00
John Ericson
c9fe290b30
Merge pull request #14616 from vinayakankugoyal/patch-1
Clarify build options in debugging documentation
2025-11-22 06:28:56 +00:00
Vinayak Goyal
48c800f7ef
Clarify build options in debugging documentation
Updated documentation to clarify that building without optimization can lead to faster builds.
2025-11-22 01:00:35 -05:00
John Ericson
79dcc094b0
Merge pull request #14614 from NixOS/libcurl-pause
libstore/filetransfer: Pause transfers instead of stalling the download thread
2025-11-22 05:41:18 +00:00
Sergei Zimmerman
be28ad92fd
rl-next: Add docs for libcurl pausing 2025-11-22 04:25:59 +03:00
Sergei Zimmerman
a2d6a69d45
libstore: Reduce the default download-buffer-size down to 1 MiB
Since the root cause (the lack of backpressure control) has
been fixed in the previous commit we can revert the change from
8ffea0a018 and make the default size much
smaller.
2025-11-22 04:23:25 +03:00
Sergei Zimmerman
4307420c44
libstore/filetransfer: Pause transfers instead of stalling the download thread
Instead of naively stalling the download thread we can instead stop the transfer.
This allows the other multiplexed connections to continue downloading (and unpacking),
if the result of the download gets piped into a GitFileSystemObjectSink.

Prior art in lix project:

- 4ae6fb5a8f
- 12156d3beb

This patch is very different from the lix one, since we are using a decompression sink
in the middle of the pipeline but the co-authored-by is there since I was motivated to
implement this by looking at the lix side of things.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-11-22 04:23:24 +03:00
Sergei Zimmerman
ec0b270c6c
libstore/filetransfer: Return an opaque handle from enqueueFileTransfer
This is necessary to make pausing/unpausing possible in a follow-up commit.
2025-11-22 03:33:13 +03:00
Sergei Zimmerman
3f8474a62f
libstore/filetransfer: Use ref instead of std::shared_ptr
Those can never be nullptr, so we should use the type system
to ensure this invariant.
2025-11-22 03:33:12 +03:00
Robert Hensing
c7e1c612eb libexpr: fix stack overflow in toJSON on deeply nested structures
Similar to the deepSeq fix, toJSON on deeply nested structures caused
an uncontrolled OS-level stack overflow.

Fix by adding call depth tracking to printValueAsJSON.
2025-11-22 00:17:26 +01:00
Robert Hensing
a812b6c6e6 libexpr: add list index to deepSeq error traces
When deepSeq encounters an error while evaluating a list element, the
error trace now includes the list index, making it easier to locate
the problematic element.
2025-11-21 23:51:07 +01:00
Robert Hensing
59a566db13 libexpr: fix stack overflow in deepSeq on deeply nested structures
builtins.deepSeq on deeply nested structures (e.g., a linked list with
100,000 elements) caused an uncontrolled OS-level stack overflow with
no Nix stack trace.

Fix by adding call depth tracking to forceValueDeep, integrating with
Nix's existing max-call-depth mechanism. Now produces a controlled
"stack overflow; max-call-depth exceeded" error with a proper stack
trace.

Closes: https://github.com/NixOS/nix/issues/7816
2025-11-21 23:50:47 +01:00
John Ericson
eb654acdd1
Merge pull request #14610 from NixOS/git-accessor-options
Introduce GitAccessorOptions
2025-11-21 22:13:52 +00:00
Taeer Bar-Yam
7cd3252946 libexpr: use allocBytes() to allocate StringData 2025-11-21 21:26:23 +01:00
Taeer Bar-Yam
9b9446e860 c api: shovel EvalMemory * into nix_value
this is a painful change. we should really add EvalState or EvalMemory
as an argument to various functions as we need it, but because we want
to preserve the stablity API, we hack it in as a field of nix_value.
2025-11-21 21:26:23 +01:00
Eelco Dolstra
6c4d2a7d11 Introduce GitAccessorOptions 2025-11-21 20:29:47 +01:00
John Ericson
152e7e48c1
Merge pull request #14607 from NixOS/open-directory-cloexec
libutil/unix: Add O_CLOEXEC to openDirectory
2025-11-21 01:23:57 +00:00