nixos/nix
1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-12-14 21:11:04 +01:00
Code Activity
21612 commits 193 branches 220 tags 158 MiB
Commit graph

21612 commits

This branch
This branch
All branches
Author SHA1 Message Date
John Ericson
d7464ecd96 Change JSON derivation format to use ContentAddress JSON 
Keeps it more consistent.
 2025-10-23 13:43:51 -04:00
John Ericson
82a736bcbb Merge branch 'dummy-store-derivations-separately' into HEAD 2025-10-23 13:43:46 -04:00
John Ericson
e79b1f5384 Merge branch 'json-schema-fso' into HEAD 2025-10-23 13:42:51 -04:00
John Ericson
044b21fd31 Merge branch 'json-schema-content-address' into HEAD 2025-10-23 13:42:09 -04:00
Sergei Zimmerman
36ee38efd1
Merge pull request #14338 from lovesegfault/s3-docs-listbucket 
docs: add s3:ListBucket to S3 read permissions
 2025-10-23 08:43:01 +00:00
Sergei Zimmerman
5d7912eb18
Merge pull request #14335 from lovesegfault/extract-getcompressionmethod 
refactor(libstore): extract getCompressionMethod() in HttpBinaryCacheStore
 2025-10-23 08:30:08 +00:00
Bernardo Meurer Costa
78888ec8a8
docs: add s3:ListBucket to S3 read permissions 
The s3:ListBucket permission is required for read operations on S3
binary caches, not just for writes. Without this permission, users get
"Access Denied" errors when running nix-build.
 2025-10-23 06:03:00 +00:00
Bernardo Meurer Costa
b047cecf5c
refactor(libstore): extract getCompressionMethod() in HttpBinaryCacheStore 
Extract the path-based compression method determination logic into a
protected method that returns std::optional<std::string>. This allows
subclasses to reuse the logic and makes the semantics clearer (nullopt
means no compression, not empty string).

This prepares for S3BinaryCacheStore to apply the same compression
rules when implementing multipart uploads.
 2025-10-23 05:03:02 +00:00
John Ericson
d0217ec180
Merge pull request #14331 from NixOS/debug-build-fix 
meson: Only enable b_lto for nixexpr-parser when b_lto is enabled glo…
 2025-10-23 04:52:55 +00:00
Sergei Zimmerman
350d602832
meson: Only enable b_lto for nixexpr-parser when b_lto is enabled globally 2025-10-23 01:49:31 +03:00
John Ericson
115dea10b2
Merge pull request #14320 from roberth/open-manual-app 
flake.nix: Add nix run .#open-manual
 2025-10-22 21:37:21 +00:00
Eelco Dolstra
ddb8830c97
Merge pull request #14326 from adeci/githint 
fetchers: add helpful hint for file+git URL scheme error
 2025-10-22 20:39:16 +00:00
Sergei Zimmerman
5390bba920
Merge pull request #14314 from Radvendii/parser-cpp-variant 
libexpr: parser.y: use api.value.type variant
 2025-10-22 18:49:14 +00:00
adeci
387eceff45 fetchers: Add helpful hint for file+git URL scheme error 
At least one user has probably used `file+git://` when they mean `git+file://`, maybe thinking of it as "a file-based git repository". This adds a specific error message to hint at the correct URL scheme format and may save some users from resorting to `path:///` and copying an entire repo.
 2025-10-22 13:57:51 -04:00
Sergei Zimmerman
96c8cc550f libexpr/meson: Rice the compiler inlining heuristics to improve perf of the bison generated parser 
Turns out both GCC and Clang need a bit of hand-holding to optimize the bison generated
code well, otherwise parser performance tanks.

(Comparisons against baseline in 7e8db2eb59):

For GCC:

Benchmark 1 (15 runs): result/bin/nix-instantiate --parse ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           335ms ± 2.89ms     332ms …  342ms          0 ( 0%)        0%

Benchmark 2 (16 runs): result-old/bin/nix-instantiate --parse ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           330ms ± 2.87ms     326ms …  337ms          0 ( 0%)          -  1.4% ±  0.6%

For Clang:

Benchmark 1 (15 runs): result-clang/bin/nix-instantiate --parse ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           340ms ± 1.43ms     338ms …  343ms          0 ( 0%)        0%

Benchmark 2 (15 runs): result-old-clang/bin/nix-instantiate --parse ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           334ms ± 1.61ms     332ms …  338ms          0 ( 0%)        ⚡-  1.7% ±  0.3%
 2025-10-22 02:25:11 +02:00
Taeer Bar-Yam
32b286e5d6 libexpr: parser.y: api.value.type variant 2025-10-22 02:25:11 +02:00
Robert Hensing
b558dac7a9 flake.nix: Add nix run .#open-manual 
Great for reviewing the rendered manual
 2025-10-22 00:42:18 +02:00
John Ericson
0bec9d0716 nlohmann::json instance and JSON Schema for ContentAddress 
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
 2025-10-21 14:32:12 -04:00
Sergei Zimmerman
7e8db2eb59
Merge pull request #14318 from cole-h/remove-useless-fmt 
libstore: remove useless fmt
 2025-10-21 17:50:56 +00:00
John Ericson
6ca2efc7d4
Merge pull request #14254 from roberth/upstream-RossComputerGuy/feat/expose-computefsclosure 
libstore-c: add nix_store_get_fs_closure #14025 with tests and realise error fix
 2025-10-21 17:41:29 +00:00
John Ericson
34de68d260 Optimize DummyStore::isValidPathUncached 
See the API docs for the rationale of why this is needed.
 2025-10-21 13:34:29 -04:00
John Ericson
2d4c7d3d03 Make Dummy store store derivations separately 
This makes for more efficiency. Once we have JSON for the dummy store,
it will also make for better JSON, too.
 2025-10-21 13:34:27 -04:00
John Ericson
9afb7e5082 nlohmann::json instance and JSON Schema for MemorySourceAccessor 
As documented, this for file system objects themselves, since
`MemorySourceAccessor` is an implementation detail.
 2025-10-21 13:30:32 -04:00
Cole Helbling
62247af363
libstore: remove useless fmt 2025-10-21 10:06:35 -07:00
John Ericson
2a194aa29e Make reading and writing derivations store methods 
This allows for different representations.
 2025-10-21 11:25:57 -04:00
John Ericson
a4a49a9dae
Merge pull request #14316 from NixOS/fix-computeStorePath-arg 
Fix computeStorePath() default argument
 2025-10-21 14:48:56 +00:00
Eelco Dolstra
606c258c6f Fix computeStorePath() default argument 2025-10-21 15:58:44 +02:00
John Ericson
ef8218f2e3
Merge pull request #14307 from NixOS/json-schema-hash 
`nlohmann::json` instance and JSON Schema for `Hash`
 2025-10-21 06:03:20 +00:00
John Ericson
ada008a795
Merge pull request #14310 from obsidiansystems/inline-drv-output-subst-goal 
Inline `realisationFetched`
 2025-10-21 06:03:06 +00:00
John Ericson
2a2bb8330d
Merge pull request #14312 from corngood/develop-structuredAttrs-fix 
tests/functional/flakes/develop.sh: Add test for outputChecks stripping
 2025-10-20 22:48:39 +00:00
David McFarland
645794b458 tests/functional/flakes/develop.sh: Add test for outputChecks stripping 2025-10-20 19:16:20 -03:00
John Ericson
1121f0d8ec Inline realisationFetched 
Now that we are using coroutines, we don't need this to be a separate
method of `DrvOutputSubstitutionGoal`.
 2025-10-20 16:45:41 -04:00
Sergei Zimmerman
6420879728
Merge pull request #14296 from lovesegfault/nix-s3-more-tests 
fix(nix-prefetch-url): correctly extract filename from URLs with query parameters
 2025-10-20 19:42:22 +00:00
Sergei Zimmerman
67f5cb97a3
Merge pull request #14306 from corngood/develop-structuredAttrs-fix 
nix/develop: Strip outputChecks when structuredAttrs is enabled
 2025-10-20 19:38:19 +00:00
John Ericson
5e7ee808de nlohmann::json instance and JSON Schema for Hash 
Improving and codifying our experimental JSON interfacing.

Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
 2025-10-20 15:21:07 -04:00
John Ericson
270f20a505
Merge pull request #14305 from NixOS/alignment-utils 
libutil: Add alignUp helper function, use in archive.cc
 2025-10-20 19:08:20 +00:00
Bernardo Meurer Costa
1b1d7e3047
test(nixos): add nix-prefetch-url test for S3 URLs with query parameters 
Adds a comprehensive test to verify that `nix-prefetch-url` correctly
handles S3 URLs with query parameters (e.g., custom endpoints and regions).

Previously, nix-prefetch-url would fail with "invalid store
path" errors when given S3 URLs with query parameters like
`?endpoint=http://server:9000&region=eu-west-1`, because it incorrectly
extracted the filename from the query parameters instead of the path.
 2025-10-20 21:45:37 +03:00
David McFarland
0f28c76a44 nix/develop: Strip outputChecks when structuredAttrs is enabled 2025-10-20 15:40:05 -03:00
Bernardo Meurer Costa
e3b3f05e5d
fix(nix-prefetch-url): correctly extract filename from URLs with query parameters 
Previously, `prefetchFile()` used `baseNameOf()` directly on the URL string
to extract the filename. This caused issues with URLs containing query
parameters that include slashes, such as S3 URLs with custom endpoints:

```
s3://bucket/file.txt?endpoint=http://server:9000
```

The `baseNameOf()` function naively searches for the rightmost `/` in the
entire string, which would find the `/` in `http://server:9000` and extract
`server:9000&region=...` as the filename. This resulted in invalid store
path names containing illegal characters like `:`.

This commit fixes the issue by:

1. Adding a `VerbatimURL::lastPathSegment()` method that extracts the last
   non-empty path segment from a URL, using `pathSegments(true)` to filter
   empty segments
2. Changing `prefetchFile()` to accept `const VerbatimURL &` and use the new
   `lastPathSegment()` method instead of manual path parsing
3. Adding early validation with `checkName()` to fail quickly on invalid
   filenames
4. Maintains backward compatibility by falling back to `baseNameOf()` for
   unparsable `VerbatimURL`s
 2025-10-20 21:40:03 +03:00
John Ericson
f05d240222
Merge pull request #14278 from obsidiansystems/adl-serializer-xp 
Cleanup and JSON serializer and XP feature interations
 2025-10-20 18:22:21 +00:00
Sergei Zimmerman
22c73868c3
libutil/archive: Use alignUp 
With this change it's much more apparent what's going on.
 2025-10-20 21:15:11 +03:00
Sergei Zimmerman
a91b787524
libutil: Add alignUp helper function 2025-10-20 21:11:00 +03:00
Eelco Dolstra
ddf7de0a76
Merge pull request #14291 from NixOS/skip-source 
Add skip() method to Source interface to allow efficient seeks
 2025-10-20 15:04:36 +00:00
Sergei Zimmerman
1fabed18b6
Merge pull request #14301 from NixOS/s3-terminate-unknown-profile 
libstore: Fix reentrancy in AwsCredentialProviderImpl::getCredentialsRaw
 2025-10-20 14:28:16 +00:00
Eelco Dolstra
6c9083db2c Use a smaller buffer 2025-10-20 13:40:19 +02:00
Sergei Zimmerman
c663f7ec79
libstore: Fix reentrancy in AwsCredentialProviderImpl::getCredentialsRaw 
Old code would do very much incorrect reentrancy crimes (trying to do an
erase inside the emplace callback). This would fail miserably with an assertion
in Boost:

terminating due to unexpected unrecoverable internal error: Assertion '(!find(px))&&("reentrancy not allowed")' failed in boost::unordered::detail::foa::entry_trace::entry_trace(const void *) at include/boost/unordered/detail/foa/reentrancy_check.hpp:33

This is trivially reproduced by using any S3 URL with a non-empty profile:

nix-prefetch-url "s3://happy/crash?profile=default"
 2025-10-19 21:03:13 +03:00
Sergei Zimmerman
d0fb03c35d
Merge pull request #14282 from NixOS/s3-cleanup 
Simplify meson for S3 support via aws-crt-cpp
 2025-10-19 17:00:46 +00:00
Sergei Zimmerman
c847cd87f1
Merge pull request #14297 from lovesegfault/nix-s3-test-public 
test(nixos/s3-binary-cache-store): misc improvements
 2025-10-19 16:53:40 +00:00
tomberek
dbbdae926b
Merge pull request #14299 from roberth/unlocked-msg 
Clarify unlocked input warning message
 2025-10-19 16:50:16 +00:00
Eelco Dolstra
3c03050cd6
Merge pull request #14290 from NixOS/dont-write-nar-to-tty 
nix store dump-path: Refuse to write NARs to the terminal
 2025-10-19 12:41:55 +00:00