1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-16 23:42:43 +01:00
Commit graph

13159 commits

Author SHA1 Message Date
Jami Kettunen
140b08ae3e libstore: Include missing header to fix compile with libc++ 20
https://en.cppreference.com/w/cpp/thread.html

src/libstore/gc.cc:121:39: error: no member named 'sleep_for' in namespace 'std::this_thread'
  121 |                     std::this_thread::sleep_for(std::chrono::milliseconds(100));
      |                     ~~~~~~~~~~~~~~~~~~^
2025-10-01 22:19:08 +03:00
Bernardo Meurer Costa
b72898b2aa
refactor(libstore): extract S3 URL parsing into separate files
Move ParsedS3URL from s3.cc/.hh into dedicated s3-url.cc/.hh files.
This separates URL parsing utilities (which are protocol-agnostic) from
the AWS SDK-specific S3Helper implementation, making the code cleaner
and enabling reuse by future curl-based S3 implementation.
2025-10-01 16:11:38 +00:00
John Ericson
9ac306c4df Expose some core implementation details and write a basic unit test for the dummy store
This test currently doesn't use the new-exposed functionality, but with
future changes the tests will be expanded and they will be used.
2025-09-30 14:52:32 -04:00
John Ericson
88bd0c25f2 Store::registerDrvOutput make pure virtual
It should be the responsibility of implementations that don't implement
it to say so.

See also PR #9799, and issue #5729
2025-09-30 14:13:04 -04:00
John Ericson
c97b050a6c Fix ca/eval-store.sh test
The refactor in the last commit fixed the bug it was supposed to fix,
but introduced a new bug in that sometimes we tried to write a resolved
derivation to a store before all its `inputSrcs` were in that store.

The solution is to defer writing the derivation until inside
`DerivationBuildingGoal`, just before we do an actual build. At this
point, we are sure that all inputs in are the store.

This does have the side effect of meaning we don't write down the
resolved derivation in the substituting case, only the building case,
but I think that is actually fine. The store that actually does the
building should make a record of what it built by storing the resolved
derivation. Other stores that just substitute from that store don't
necessary want that derivation however. They can trust the substituter
to keep the record around, or baring that, they can attempt to re
resolve everything, if they need to be audited.
2025-09-30 11:29:21 -04:00
John Ericson
39f6fd9b46 Fix #13247
Resolve the derivation before creating a building goal, in a context
where we know what output(s) we want. That way we have a chance just to
download the outputs we want.

Fix #13247
2025-09-30 11:29:19 -04:00
John Ericson
8f4a739d0f Split out DerivationResolutionGoal
This prepares the way for fixing a few issues.
2025-09-30 11:25:52 -04:00
John Ericson
d76dc2406f
Merge pull request #14060 from obsidiansystems/build-result-variant
Use `std::variant` to enforce `BuildResult` invariants
2025-09-30 11:02:13 -04:00
Jörg Thalheim
bc66e131f8
Merge pull request #14120 from lovesegfault/http-binary-cache-compression
feat(libstore/http-binary-cache-store): narinfo/ls/log compression
2025-09-30 12:50:10 +02:00
Jörg Thalheim
a5facbd2d1
Merge pull request #14121 from obsidiansystems/file-transfer-quit
Some Curl file transfer cleanups
2025-09-30 09:12:08 +02:00
Jörg Thalheim
a5b35ec129
Merge pull request #14106 from Radvendii/exprpath-alloc
libexpr: allocate ExprPath strings in the allocator
2025-09-30 09:04:24 +02:00
Sergei Zimmerman
a8670e8a7d
libexpr-tests: Add unit tests for broken readDir /. for pure eval
A very unfortunate interaction of current filtering with pure eval is
that the following actually leads to `lib.a = {}`. This just adds a unit
test for this broken behavior. This is really good to be done as a unit test
via the in-memory store.

{
  outputs =
    { ... }:
    {
      lib.a = builtins.readDir /.;
    };
}
2025-09-30 03:16:35 +03:00
John Ericson
86fb5b24a9 curlFileTransfer::workerThreadEntry Only call quit if we need to. 2025-09-29 18:10:34 -04:00
John Ericson
1f65b08d94 curlFileTransfer::State:quit emptys the queue
Whoever first calls `quit` now empties the queue, instead of waiting for
the worker thread to do it.

(Note that in the unwinding case, the worker thread is still the first
to call `quit`, though.)
2025-09-29 18:10:34 -04:00
John Ericson
d5402b8527 Encapsulate curlFileTransfer::State:quit
It is allowed to read it, and to set it to `false`, but not to set it
to `true`.
2025-09-29 18:10:34 -04:00
Bernardo Meurer Costa
689fa81dc9
feat(libstore/http-binary-cache-store): narinfo/ls/log compression 2025-09-29 21:53:40 +00:00
Taeer Bar-Yam
f70b0b599c libexpr: allocate ExprPath strings in the allocator 2025-09-29 17:02:05 -04:00
Sergei Zimmerman
1830f5f967
libutil: Create empty directory at the root for makeEmptySourceAccessor
This is my SNAFU. Accidentally broken in 02c9ac445f.

There's very dubious behavior for 'builtins.readDir /.':

{
  outputs =
    { ... }:
    {
      lib.a = builtins.readDir /.;
    };
}

nix eval /tmp/test-flake#lib.a

Starting from 2.27 this now returns an empty set. This really isn't supposed
to happen, but this change in the semantics of makeEmptySourceAccessor accidentally
changed the behavior of this.
2025-09-29 23:16:28 +03:00
Jörg Thalheim
fca6d8f1cc
Merge pull request #14112 from EphraimSiegfried/make-content-addressed-doc-fix
docs: fix build command in make-content-addressed.md
2025-09-29 18:45:14 +02:00
Ephraim Siegfried
121dda0f1f docs: fix build command in make-content-addressed.md 2025-09-29 14:07:26 +02:00
Jörg Thalheim
b6f4788a8f
Merge pull request #14110 from Mic92/ptsname
Fix thread-safety issue with ptsname() usage
2025-09-29 13:49:58 +02:00
Jörg Thalheim
f816b9bcb8
Merge pull request #14111 from Mic92/symlinks
Prevent infinite symlink loop in followLinksToStore()
2025-09-29 13:49:19 +02:00
Jörg Thalheim
5ec9138179 Prevent infinite symlink loop in followLinksToStore()
The followLinksToStore() function could hang indefinitely when encountering
symlink cycles outside the Nix store, causing 100% CPU usage and blocking
any operations that use this function.

This affects multiple commands including nix-store --query, --delete,
--verify, nix-env, and nix-copy-closure when given paths with symlink cycles.

The fix adds a maximum limit of 1024 symlink follows (matching the limit
used by canonPath) and throws an error when exceeded, preventing the
infinite loop while preserving the original semantics of stopping at
the first path inside the store.
2025-09-29 12:22:43 +02:00
Jörg Thalheim
a9ffa42dda Fix thread-safety issue with ptsname() usage
Replace non-thread-safe ptsname() calls with a new getPtsName() helper
function that:
- Uses thread-safe ptsname_r() on Linux/BSD platforms
- Uses mutex-protected ptsname() on macOS (which lacks ptsname_r())
2025-09-29 12:01:49 +02:00
Eelco Dolstra
b0431a76f5
Merge pull request #14058 from DeterminateSystems/upstream-RossComputerGuy/feat/expose-logfmt
C API: add log format and verbosity functions
2025-09-29 11:21:43 +02:00
Jörg Thalheim
5e65fa7069
Merge pull request #14109 from Mic92/mingw
fix mingw build
2025-09-29 10:57:43 +02:00
Jörg Thalheim
69eae7770a fix mingw build 2025-09-29 10:29:37 +02:00
Jörg Thalheim
7cbc0f97e7
Merge pull request #14090 from Radvendii/expr-slim
nixexpr: introduce arena to hold ExprString strings
2025-09-29 08:33:10 +02:00
Jörg Thalheim
c5b3567512
Merge pull request #14105 from xokdvium/cpp-bison
libexpr: Switch parser.y to %skeleton lalr1.cc
2025-09-29 08:23:03 +02:00
John Ericson
676e885f8d
Merge pull request #14084 from obsidiansystems/issue-13247-test
Create test for Issue 13247
2025-09-28 19:12:44 -04:00
Sergei Zimmerman
a8715a2d6e
libexpr: Switch parser.y to %skeleton lalr1.cc
Since the parser is now LALR we can easily switch
over to the less ugly sketelon than the default C one.
This would allow us to switch from %union to %define api.value.type variant
in the future to avoid the need for triviall POD types.
2025-09-29 00:58:41 +03:00
Sergei Zimmerman
0f08feaa58
libexpr: Remove unused members from ParserLocation 2025-09-28 22:57:11 +03:00
Taeer Bar-Yam
eab467ecfb libexpr: introduce arena to hold ExprString strings
1. Saves 24-32 bytes per string (size of std::string)
2. Saves additional bytes by not over-allocating strings (in total we
save ~1% memory)
3. Sets us up to perform a similar transformation on the other Expr
subclasses
4. Makes ExprString trivially moveable (before the string data might
move, causing the Value's pointer to become invalid). This is important
so we can put ExprStrings in an std::vector and refer to them by index

We have introduced a string copy in ParserState::stripIndentation().
This could be removed by pre-allocating the right sized string in the
arena, but this adds complexity and doesn't seem to improve performance,
so for now we've left the copy in.
2025-09-28 14:23:13 -04:00
Sergei Zimmerman
c43ea09b9b
Merge pull request #14100 from obsidiansystems/add-missing-pragma-once
Add `#pragma once` to `dummy-store.hh`
2025-09-28 16:41:28 +00:00
John Ericson
582d3ee611 Add #pragma once to dummy-store.hh
We should have a lint for this.

In later (yet to be merged at this time) commits, this started causing
problems that only the sanitzer caught.
2025-09-28 12:12:24 -04:00
John Ericson
f7d35dc1dc
Merge pull request #14099 from xokdvium/fix-assert-failure
libstore: Call canonPath for constructing LocalFSStoreConfig::rootDir
2025-09-28 12:06:02 -04:00
Sergei Zimmerman
0866ba0b4a
libstore: Deduplicate LocalFSStoreConfig::rootDir initializers
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2025-09-28 18:38:57 +03:00
John Ericson
a01d52e57b
Merge pull request #14063 from obsidiansystems/test-less-macro
Minimize the use of C Macros for characterization tests
2025-09-28 11:14:29 -04:00
Sergei Zimmerman
3a64d3c0da
libstore: Call canonPath for constructing LocalFSStoreConfig::rootDir
This mirrors what OptionalPathSetting does. Otherwise we run into
an assertion failure for relative paths specified as the authority + path:

nix build nixpkgs#hello --store "local://a/b"
nix: ../posix-source-accessor.cc:13: nix::PosixSourceAccessor::PosixSourceAccessor(std::filesystem::__cxx11::path&&): Assertion `root.empty() || root.is_absolute()' failed.

This is now diagnosed properly:

error: not an absolute path: 'a/b'

Just as you'd specify the root via a query parameter:

nix build nixpkgs#hello --store "local?root=a/b"
2025-09-28 17:42:19 +03:00
John Ericson
01b2037bc0 Minimize the use of C Macros for characterization tests
Fewer macros is better!

Introduce a new `JsonChacterizationTest` mixin class to help with this.

Also, avoid some needless copies with `GetParam`.

Part of my effort shoring up the JSON formats with #13570.
2025-09-28 09:54:46 -04:00
Sergei Zimmerman
426a72c9cf
libstore: Make all StoreConfig::getReference implementations return store parameters
These stragglers have been accidentally left out when implementing the StoreConfig::getReference.
Also HttpBinaryCacheStore::getReference now returns the actual store parameters, not the cacheUri
parameters.
2025-09-28 16:29:12 +03:00
John Ericson
e731c43eae Use std::variant to enforce BuildResult invariants
There is now a clean separation between successful and failing build
results.
2025-09-27 15:56:06 -04:00
John Ericson
43550e8edb Lock down BuildResult::Status enum values
This allows refactoring without changing wire protocol by mistake.
2025-09-27 15:15:20 -04:00
John Ericson
3c610df550 Delete scratch data for CA derivation that produced already-extant output
In the case where the store object doesn't exist, we do correctly move
(rather than copy) the scratch data into place. In this case, the
destination store object already exists, but we still want to clean up
after ourselves.
2025-09-27 15:14:33 -04:00
John Ericson
3bf1268ac6
Merge pull request #14096 from Mic92/concurrency-bugs-2
document thread-unsafe mutation in PosixSourceAccessor
2025-09-27 11:18:56 -04:00
Yaroslav Bolyukin
0a3eb22360
fix: wait on incomplete assignment in REPL
Fixes: https://github.com/NixOS/nix/issues/13507
2025-09-27 13:25:27 +03:00
Jörg Thalheim
866c9179a0 document thread-unsafe mutation in PosixSourceAccessor 2025-09-26 23:31:56 +02:00
Jörg Thalheim
6721ef5feb
Merge branch 'master' into fix-fetch-to-store-caching 2025-09-26 20:44:39 +02:00
Jörg Thalheim
b5f765b7eb
Merge pull request #14047 from Radvendii/eval-memory
libexpr: move eval memory allocation to own struct
2025-09-26 19:24:30 +02:00
Tristan Ross
bb6a4dccdf
libutil-c: add nix_set_verbosity function 2025-09-26 08:31:23 -07:00