1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-28 05:00:58 +01:00
Commit graph

13310 commits

Author SHA1 Message Date
John Ericson
84079e10cf No more Path in libnixcmd
Co-authored-by: Vinayak Goyal <vinayakankugoyal@gmail.com>
2025-11-25 05:00:09 +00:00
John Ericson
88c9c6d89d
Merge pull request #14636 from NixOS/openat2-wrapper
libutil/file-descriptor: Add safer utilities for opening files relati…
2025-11-24 23:23:51 +00:00
John Ericson
4f4da90513
Merge pull request #13942 from NixOS/json-no-store-dir
JSON impl and Schema for `DummyStore`
2025-11-24 23:06:13 +00:00
Jörg Thalheim
3e9104c9ca
Merge pull request #14637 from lovesegfault/aws-crt-cpp-log-level
feat(libstore): tie AWS CRT logging to Nix verbosity level
2025-11-24 22:45:45 +00:00
Sergei Zimmerman
3a9be9fd2f
libutil: Use openFileEnsureBeneathNoSymlinks in RestoreSink::createDirectory
Starts using the new function.
2025-11-25 01:10:35 +03:00
John Ericson
0275b64b81 JSON impl and Schema for DummyStore
This is the "keystone" that puts most of the other store-layer JSON
formats together.

Also, add some documentation for JSON testing.
2025-11-24 17:04:24 -05:00
John Ericson
622a5cd1bf Add DummyStore::operator==
Will need it for tests.
2025-11-24 17:04:24 -05:00
John Ericson
b0c016ae7d DummyStore build trace holds UnkeyedRealisation by value
Otherwise the equality instance we need to add will be messed up.
2025-11-24 17:04:24 -05:00
John Ericson
f78e88c973 Add some infrastructure changes for better JSON ref<T> impls
Also skip a trailing semicolon inside a macro so the caller can use it
instead, which is generally nicer to the formatter.
2025-11-24 17:04:23 -05:00
Jörg Thalheim
439af1dca1 feat(libstore): tie AWS CRT logging to Nix verbosity level
Map Nix's verbosity levels to AWS CRT log levels so users can
debug SSO authentication issues without modifying code:

- Default/warn: AWS Warn (errors/warnings only)
- Chatty (-vvv): AWS Info (credential provider actions)
- Debug (-vvvv): AWS Debug (detailed auth flow)
- Vomit (-vvvvv): AWS Trace (full CRT internal tracing)

This makes it easy to diagnose SSO issues with:
  nix copy -vvvv --to s3://bucket?profile=foo ...
2025-11-24 17:02:19 -05:00
Sergei Zimmerman
77990e7cca
libutil/file-descriptor: Add safer utilities for opening files relative to dirFd
Implements a safe no symlink following primitive operation for opening file descriptors.
This is unix-only for the time being, since windows doesn't really suffer from symlink
races, since they are admin-only.

Tested with enosys --syscall openat2 as well.
2025-11-25 00:42:57 +03:00
John Ericson
3bac0d7aa2
Merge pull request #14635 from Radvendii/alloc-exprlet-exprattrs
libexpr: move the ExprLet::attrs allocations into the arena
2025-11-24 21:14:52 +00:00
John Ericson
36419a6ccb
Merge pull request #14507 from obsidiansystems/derivation-options-json-schema
JSON Schema for `DerivationOptions`
2025-11-24 21:11:07 +00:00
John Ericson
3ba51bf61b
Merge pull request #14560 from obsidiansystems/fill-in-outputs
Dedup some derivation initialization logic, and test
2025-11-24 21:10:38 +00:00
John Ericson
209f413e80 JSON Schema for DerivationOutputs
Progress on #13570
2025-11-24 15:23:50 -05:00
John Ericson
b8d32388bc Move derivation JSON doc to index.md in dir
This prepares for more structure.
2025-11-24 15:23:50 -05:00
Taeer Bar-Yam
60f09928d1 libexpr: move ExprLet::attrs data to arena as well
I missed this because I assumed all Exprs were recursed into by
bindVars, but ExprLet's ExprAttrs field is not really its own AST node,
so it doesn't get recursed into.
2025-11-24 21:14:13 +01:00
Taeer Bar-Yam
43a183120a libexpr: factor out functions for moving data to a new allocator 2025-11-24 21:14:13 +01:00
John Ericson
0c786f3a3c
Merge pull request #14617 from vinayakankugoyal/path
Update profiles to use `std::filesystem::path`
2025-11-24 19:31:25 +00:00
John Ericson
504c5e7cf9 Convert profiles to use std::filesystem::path
Co-authored-by: Vinayak Goyal <vinayakankugoyal@gmail.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2025-11-24 13:38:01 -05:00
John Ericson
5d066386b5
Merge pull request #14260 from roberth/ulimit
Clarify setStackSize error message and warn if not possible
2025-11-24 17:12:31 +00:00
John Ericson
d689b764f3 Use WorkerProto::Serialise abstraction for DrvOutput
It's better to consistently use the abstraction, rather than code which
happens to do the same thing.

See also d782c5e586 for the same sort of
change.
2025-11-24 10:44:45 -05:00
Sergei Zimmerman
28fac9fe4d
nix/prefetch: Be honest about when path name is derived from URL
Only add the message to trace when name is really derived from URL.
2025-11-24 00:25:48 +03:00
Jon Hermansen
76ed967f79
libstore: fix curl version check to allow 8.17.0
The single-string syntax '>=8.16.0 <8.17.0' only applied the lower
bound, causing curl 8.17.0 to be incorrectly rejected. Split into two
separate version_compare() calls for compatibility with Meson 1.1,
since multi-argument syntax requires Meson 1.8+.
2025-11-23 12:13:05 +03:00
John Ericson
327e8babf7
Merge pull request #14584 from Radvendii/allocbytes-stringdata
libexpr: use allocBytes() to allocate StringData
2025-11-23 00:38:50 +00:00
John Ericson
d5d4bafc2a
Merge pull request #14620 from NixOS/revert-shared-tarball-cache
libfetchers: Don't have a single shared tarball cache
2025-11-23 00:33:51 +00:00
Taeer Bar-Yam
dbfe6318b3 libexpr: move ExprCall storage to the arena 2025-11-23 00:06:10 +01:00
Taeer Bar-Yam
484f40fc64 libexpr: make ExprCall::args an std::optional 2025-11-23 00:06:10 +01:00
Taeer Bar-Yam
43fc6c314d libexpr: ExprCall use std::pmr::vector 2025-11-23 00:06:10 +01:00
Sergei Zimmerman
385d7e77bd
libfetchers: Don't have a single shared tarball cache
This partially reverts commit bc6b9ce.

This transformation is unsound and thread unsafe. Internal libgit2
structures must *never* be shared between threads. This causes
internal odb corruption with e.g.:

nix flake prefetch-inputs:

error:
       … while fetching the input 'github:nixos/nixpkgs/89c2b2330e733d6cdb5eae7b899326930c2c0648?narHash=sha256-Stk9ZYRkGrnnpyJ4eqt9eQtdFWRRIvMxpNRf4sIegnw%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - upload-image.sh
error:
       … while fetching the input 'github:NixOS/nixpkgs/a8d610af3f1a5fb71e23e08434d8d61a466fc942?narHash=sha256-v5afmLjn/uyD9EQuPBn7nZuaZVV9r%2BJerayK/4wvdWA%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - outline.nix
double free or corruption (!prev)

Thread 21 "nix" received signal SIGABRT, Aborted.
2025-11-22 22:48:40 +03:00
Robert Hensing
67f6a24171 src/nix: Make meson compile <cmdlet> valid
Without this dependency, e.g. `meson compile nix-instantiate`
would produce a broken symlink, or the `nix` it points to may be
stale.
With the dependency in place, `meson compile nix-instantiate`
produces a reliable outcome.
2025-11-22 20:19:34 +01:00
Sergei Zimmerman
8cdeab8f2e
Merge pull request #14613 from roberth/deepSeq-stack-overflow
`deepSeq`, json: handle stack overflow, report list index
2025-11-22 17:49:32 +00:00
Jon Hermansen
3ff8d0ece4 fix(FreeBSD): remove null terminator from executable path
On FreeBSD, sysctl(KERN_PROC_PATHNAME) returns a null-terminated
string with pathLen including the terminator. This causes Nix to
fail during manual generation with:

  error:
         … while calling the 'concatStringsSep' builtin
           at /nix/var/nix/builds/nix-63232-402489527/source/doc/manual/generate-settings.nix:99:1:
             98| in
             99| concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
               | ^
            100|

         error: input string '/nix/store/gq89cj02b5zs67cbd85vzg5cgsgnd8mj-nix-2.31.2/bin/nix␀'
                cannot be represented as Nix string because it contains null bytes

The issue occurs because generate-settings.nix reads the nix binary
path from JSON and evaluates it as a Nix string, which cannot contain
null bytes. Normal C++ string operations don't trigger this since they
handle null-terminated strings correctly.

Strip the null terminator on FreeBSD to match other platforms (Linux
uses /proc/self/exe, macOS uses _NSGetExecutablePath).

Credit: @wahjava (FreeBSD ports and Nixpkgs contributor)
2025-11-22 03:59:29 -05:00
Sergei Zimmerman
a2d6a69d45
libstore: Reduce the default download-buffer-size down to 1 MiB
Since the root cause (the lack of backpressure control) has
been fixed in the previous commit we can revert the change from
8ffea0a018 and make the default size much
smaller.
2025-11-22 04:23:25 +03:00
Sergei Zimmerman
4307420c44
libstore/filetransfer: Pause transfers instead of stalling the download thread
Instead of naively stalling the download thread we can instead stop the transfer.
This allows the other multiplexed connections to continue downloading (and unpacking),
if the result of the download gets piped into a GitFileSystemObjectSink.

Prior art in lix project:

- 4ae6fb5a8f
- 12156d3beb

This patch is very different from the lix one, since we are using a decompression sink
in the middle of the pipeline but the co-authored-by is there since I was motivated to
implement this by looking at the lix side of things.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-11-22 04:23:24 +03:00
Sergei Zimmerman
ec0b270c6c
libstore/filetransfer: Return an opaque handle from enqueueFileTransfer
This is necessary to make pausing/unpausing possible in a follow-up commit.
2025-11-22 03:33:13 +03:00
Sergei Zimmerman
3f8474a62f
libstore/filetransfer: Use ref instead of std::shared_ptr
Those can never be nullptr, so we should use the type system
to ensure this invariant.
2025-11-22 03:33:12 +03:00
Robert Hensing
c7e1c612eb libexpr: fix stack overflow in toJSON on deeply nested structures
Similar to the deepSeq fix, toJSON on deeply nested structures caused
an uncontrolled OS-level stack overflow.

Fix by adding call depth tracking to printValueAsJSON.
2025-11-22 00:17:26 +01:00
Robert Hensing
a812b6c6e6 libexpr: add list index to deepSeq error traces
When deepSeq encounters an error while evaluating a list element, the
error trace now includes the list index, making it easier to locate
the problematic element.
2025-11-21 23:51:07 +01:00
Robert Hensing
59a566db13 libexpr: fix stack overflow in deepSeq on deeply nested structures
builtins.deepSeq on deeply nested structures (e.g., a linked list with
100,000 elements) caused an uncontrolled OS-level stack overflow with
no Nix stack trace.

Fix by adding call depth tracking to forceValueDeep, integrating with
Nix's existing max-call-depth mechanism. Now produces a controlled
"stack overflow; max-call-depth exceeded" error with a proper stack
trace.

Closes: https://github.com/NixOS/nix/issues/7816
2025-11-21 23:50:47 +01:00
Taeer Bar-Yam
7cd3252946 libexpr: use allocBytes() to allocate StringData 2025-11-21 21:26:23 +01:00
Taeer Bar-Yam
9b9446e860 c api: shovel EvalMemory * into nix_value
this is a painful change. we should really add EvalState or EvalMemory
as an argument to various functions as we need it, but because we want
to preserve the stablity API, we hack it in as a field of nix_value.
2025-11-21 21:26:23 +01:00
Eelco Dolstra
6c4d2a7d11 Introduce GitAccessorOptions 2025-11-21 20:29:47 +01:00
John Ericson
152e7e48c1
Merge pull request #14607 from NixOS/open-directory-cloexec
libutil/unix: Add O_CLOEXEC to openDirectory
2025-11-21 01:23:57 +00:00
Sergei Zimmerman
ea4854fda1
libutil/unix: Add O_CLOEXEC to openDirectory
As a precaution. This function might get used for some long persisted
file descriptor and we need good defaults.
2025-11-21 02:43:26 +03:00
John Ericson
d3ff01cb2e
Merge pull request #14606 from NixOS/fix-copy-recursive
libutil: Fix copyRecursive and use for nix flake clone
2025-11-20 22:28:45 +00:00
John Ericson
a835d6ad2a
Merge pull request #14319 from obsidiansystems/json-schema-fso
`nlohmann::json` instance and JSON Schema for `MemorySourceAccessor`
2025-11-20 21:52:57 +00:00
John Ericson
ec3c93f17f
Merge pull request #14603 from NixOS/safe-cast
Turn one unsafe C cast into a safe `static_cast`
2025-11-20 21:26:00 +00:00
Sergei Zimmerman
6d0f4fa666
libutil: Fix copyRecursive and use for nix flake clone
The use of sourceToSink is an unnecessary serialization bottleneck.
While we are at it, fix the copyRecursive implementation to actually copy
the whole directory. It wasn't used for anything prior, but now it has a use
and accompanying tests for flake clone.
2025-11-21 00:21:23 +03:00
John Ericson
b2ead92791 Turn one unsafe C cast into a safe static_cast 2025-11-20 15:58:31 -05:00