21834 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Bernardo Meurer Costa	040d1aae41	feat(libstore/s3-binary-cache-store): implement uploadMultipart() ... Implement `uploadMultipart()`, the main method that orchestrates S3 multipart uploads	2025-11-03 01:15:46 +00:00
Bernardo Meurer Costa	bf947bfc26	feat(libstore/s3-binary-cache-store): add multipart upload config settings ... Add three configuration settings to `S3BinaryCacheStoreConfig` to control multipart upload behavior: - `bool multipart-upload` (default `false`): Enable/disable multipart uploads - `uint64_t multipart-chunk-size` (default 5 MiB): Size of each upload part - `uint64_t multipart-threshold` (default 100 MiB): Minimum file size for multipart The feature is disabled by default.	2025-11-02 18:41:48 +00:00
John Ericson	2d83bc6b83	Merge pull request #14446 from mjoerg/fix-docs ... fix documentation issues	2025-11-02 16:59:33 +00:00
Sergei Zimmerman	e0debd61d5	Merge pull request #14449 from roberth/error-resolution-failed ... Improve "resolution failed" error	2025-11-02 15:00:56 +00:00
Robert Hensing	233bd250d1	flake: Update, nixos-25.05-small -> nixos-25.05 ... Flake lock file updates: • Updated input 'nixpkgs': 'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26) → 'github:NixOS/nixpkgs/daf6dc47aa4b44791372d6139ab7b25269184d55?narHash=sha256-wxX7u6D2rpkJLWkZ2E932SIvDJW8%2BON/0Yy8%2Ba5vsDU%3D' (2025-10-27)	2025-11-02 14:10:36 +01:00
Robert Hensing	4ea32d0b03	Improve "resolution failed" error ... Previously: error: Cannot build '/nix/store/cqc798lwy2njwbdzgd0319z4r19j2d1w-nix-manual-2.33.0pre20251101_e4e4063.drv'. Reason: 1 dependency failed. Output paths: /nix/store/f1kln1c6z9r7rlhj0h9shcpch7j5g1fj-nix-manual-2.33.0pre20251101_e4e4063-man /nix/store/k65203rx5g1kcagpcz3c3a09bghcj92a-nix-manual-2.33.0pre20251101_e4e4063 error: Cannot build '/nix/store/ajk2fb6r7ijn2fc5c3h85n6zdi36xlfl-nixops-manual.drv'. Reason: 1 dependency failed. Output paths: /nix/store/0anr0998as8ry4hr5g3f3iarszx5aisx-nixops-manual error: resolution failed Now: error: Cannot build '/nix/store/cqc798lwy2njwbdzgd0319z4r19j2d1w-nix-manual-2.33.0pre20251101_e4e4063.drv'. Reason: 1 dependency failed. Output paths: /nix/store/f1kln1c6z9r7rlhj0h9shcpch7j5g1fj-nix-manual-2.33.0pre20251101_e4e4063-man /nix/store/k65203rx5g1kcagpcz3c3a09bghcj92a-nix-manual-2.33.0pre20251101_e4e4063 error: Cannot build '/nix/store/ajk2fb6r7ijn2fc5c3h85n6zdi36xlfl-nixops-manual.drv'. Reason: 1 dependency failed. Output paths: /nix/store/0anr0998as8ry4hr5g3f3iarszx5aisx-nixops-manual error: Build failed due to failed dependency	2025-11-02 14:03:27 +01:00
Martin Joerg	892eba4944	fix documentation issues	2025-11-02 09:17:40 +00:00
Sergei Zimmerman	e4e4063f16	Merge pull request #14443 from NixOS/inline-unreused-lambda ... Inline only-used-once closures in `ExprConcatStrings::eval`	2025-11-01 21:55:03 +00:00
Aspen Smith	b67c2f1572	Inline only-used-once closures in ExprConcatStrings::eval ... Refactor `ExprConcatStrings::eval` by inlining two only-called-once closures into the call-site, so that the code is easier to reason about locally (especially since the variables that were closed over were mutated all over the place within this function). Also use curly braces with each branch for consistency in the the resulting code. This is a pure refactor, but also arguably causes us to depend less on the optimizer; now, we don't have to make sure that this closure is inlined.	2025-11-01 16:35:54 -04:00
John Ericson	ca9fde1b88	In EvalState::concatLists, local variable s -> strings ... It deserves a better name. Co-Authored-By: Aspen Smith <root@gws.fyi>	2025-11-01 13:41:50 -04:00
John Ericson	0ba1aa34dc	Merge pull request #14440 from lovesegfault/cleanup-aws-sdk ... chore(libstore/package): remove lingering aws-sdk-cpp	2025-11-01 15:08:21 +00:00
John Ericson	6fa7510055	Merge pull request #14439 from NixOS/no-buffer-overflows ... libexpr: Do not overflow heap buffer when there are too many formal a…	2025-11-01 14:48:11 +00:00
Bernardo Meurer Costa	8151afb345	chore(libstore/package): remove lingering aws-sdk-cpp ... This was left behind during the great s3 refactoring of 2025	2025-11-01 14:42:07 +00:00
Sergei Zimmerman	134613e885	libexpr: Do not overflow heap buffer when there are too many formal arguments ... 3a3c062982 introduced a buffer overflow for the case when there are more than 65535 formal arguments. It is a perfectly reasonable limitation, but we *must* not crash, corrupt memory or otherwise crash the process. Add a test for the graceful behavior and switch to using an explicit uninitialized_copy_n to further guard against buffer overflows.	2025-11-01 12:53:53 +03:00
John Ericson	9d1907fff7	Merge pull request #14434 from NixOS/improve-ipv6-zoneid-backcompat ... libstore: Improve store-reference back-compat with IPv6 ZoneId literals	2025-10-31 23:10:40 +00:00
John Ericson	c29411ada9	Merge pull request #14431 from NixOS/git-url-fixes ... libfetchers: Restore plain git inputs recognition	2025-10-31 22:28:14 +00:00
Sergei Zimmerman	8dbc2475f7	libstore: Improve store-reference back-compat with IPv6 ZoneId literals ... This restores the pre-2.31 handling of ZoneID identifiers in store references. It's the only place we reasonably care about this back-compat.	2025-11-01 00:36:15 +03:00
John Ericson	9e79e83cb5	Merge pull request #14384 from Radvendii/exprlambda-alloc ... libexpr: store ExprLambda data in Expr::alloc	2025-10-31 21:12:30 +00:00
John Ericson	937a6df809	Merge pull request #14432 from NixOS/meson-darwin-soname ... meson: Also split version string at '+' for Darwin	2025-10-31 21:03:47 +00:00
Sergei Zimmerman	1ca6e9ef54	meson: Also split version string at '+' for Darwin	2025-10-31 23:12:54 +03:00
Sergei Zimmerman	ade3d5d746	libfetchers: Restore plain git inputs recognition ... Accidentally broken in dbc235cc62. Adds a bit of tests for this, even though this protocol is mostly deprecated everywhere.	2025-10-31 22:42:43 +03:00
John Ericson	d035d8ba8d	Merge pull request #14428 from obsidiansystems/path-info-parse-json-cleanup ... Clean up `PathInfo::fromJSON` using recent JSON utils changes	2025-10-31 19:27:09 +00:00
Taeer Bar-Yam	67be2df174	remove unnecessary constructor argument	2025-10-31 16:54:59 +01:00
Taeer Bar-Yam	34f780d747	safer interface for ExprLambda's formals	2025-10-31 16:54:59 +01:00
Taeer Bar-Yam	e43888890f	restore proper handling of no formals vs. 0 formals ... e.g. (foo@{}: 1) { a = 3; } should error, but wasn't with the previous commit	2025-10-31 16:54:59 +01:00
Taeer Bar-Yam	4a80c92a4d	add test for no formals case	2025-10-31 16:54:59 +01:00
Taeer Bar-Yam	3a3c062982	libexpr: store ExprLambda data in Expr::alloc	2025-10-31 16:54:59 +01:00
John Ericson	4a2fb18ba0	Merge pull request #14137 from lovesegfault/nix-debug-14130 ... fix(libstore/build/derivation-goal): don't assert on partially valid outputs	2025-10-31 02:45:50 +00:00
Bernardo Meurer Costa	9eecee3d4e	fix(libstore/build/derivation-goal): don't assert on partially valid outputs ... Fixes: #14130	2025-10-31 01:58:02 +00:00
John Ericson	089a222111	Clean up PathInfo::fromJSON using recent JSON utils changes ... `optionalValueAt` and then `optionalValueAt` avoids looking up twice.	2025-10-30 18:38:27 -04:00
John Ericson	c2609df08c	Better JSON schema for derivation outputs ... It now uses a `oneOf` and properly models each type of output (corresponding to each type of derivation) separately.	2025-10-30 17:05:00 -04:00
John Ericson	37c1ef52e6	Merge pull request #14412 from NixOS/recursive-lambdas ... Cleanup: Use C++23 "this auto" for recursive lambdas	2025-10-30 20:41:52 +00:00
Sergei Zimmerman	e776a10db3	Merge pull request #14356 from lovesegfault/s3-upload ... refactor(libstore/s3-binary-cache-store): implement `upload()`	2025-10-30 20:19:59 +00:00
Eelco Dolstra	1507843f6c	Cleanup: Use C++23 "explicit this" for recursive lambdas ... Try to pass by reference where possible. Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>	2025-10-30 15:56:06 -04:00
Bernardo Meurer Costa	e636888a09	refactor(libstore/s3-binary-cache-store): implement upload() ... Stop delegating to `HttpBinaryCacheStore::upsertFile` and instead handle compression in the S3 store's `upsertFile` override, then call our own `upload()` method. This separation is necessary for future multipart upload support.	2025-10-30 19:01:05 +00:00
Eelco Dolstra	3b2186e1c8	Merge pull request #14397 from fzakaria/fzakaria/issue-14315 ... Move docker documentation to docker.io	2025-10-30 17:38:22 +00:00
John Ericson	7e2d2db8ef	Merge pull request #14399 from obsidiansystems/json-schema-path-info ... Convert store path info JSON docs to formal JSON Schema, and test	2025-10-30 17:35:20 +00:00
Eelco Dolstra	2cc53201eb	Merge pull request #14418 from lovesegfault/fix-curl-interrupt ... fix(libstore/filetransfer): prevent double callback invocation on interrupt during retry	2025-10-30 17:12:15 +00:00
Eelco Dolstra	720f693627	Merge pull request #14416 from lovesegfault/fix-lexer-warn ... fix(libexpr/lexer): fix flex warning about default rule	2025-10-30 17:11:23 +00:00
John Ericson	49084a7e9e	Merge pull request #14421 from lovesegfault/http-upload ... refactor(libstore): add `HttpBinaryCacheStore::upload` method	2025-10-30 15:11:58 +00:00
tomberek	6d87184a52	Merge pull request #14363 from cootshk/patch-1 ... fix(libstore): Rewrite hard linking message to be more clear	2025-10-30 09:23:00 +00:00
Henry	6985e9f2c3	fix(libstore): Rewrite hard linking message to be more clear	2025-10-30 03:05:06 -05:00
John Ericson	e6f0dd8df5	Merge pull request #14420 from lovesegfault/compressed-source ... refactor(libutil): add `CompressedSource`	2025-10-30 05:21:41 +00:00
Bernardo Meurer Costa	d857a4be50	refactor(libstore): add HttpBinaryCacheStore::upload method ... Introduce protected `upload` method overloads in `HttpBinaryCacheStore` that handle the actual upload after compression has been applied. This separates compression concerns (in `upsertFile`) from upload mechanics (in `upload`). Two overloads are provided: 1. `upload(path, RestartableSource &, sizeHint, mimeType, contentEncoding)` 2. `upload(path, CompressedSource &, mimeType)`	2025-10-30 04:35:43 +00:00
Bernardo Meurer Costa	93fe3354b5	refactor(libutil): add CompressedSource ... Introduce a `CompressedSource` class in libutil's `serialise.hh` that compresses a `RestartableSource` and owns the compressed data. This is a general-purpose utility that can be used anywhere compressed data needs to be treated as a source.	2025-10-30 04:35:27 +00:00
Bernardo Meurer Costa	8b3af40006	fix(libexpr/lexer): fix flex warning about default rule ... We were getting this flex lexer warning during build: ``` ../src/libexpr/lexer.l:333: warning, -s option given but default rule can be matched ``` The lexer uses `%option nodefault` but the `PATH_START` state only had rules for specific patterns (`PATH_SEG` and `HPATH_START`) without a catch-all rule to handle unexpected input. Added a catch-all rule with `unreachable()`. This code path should never be reached in normal operation since `PATH_START` is only entered after matching `PATH_SEG` or `HPATH_START`, and we immediately rewind to re-parse those same patterns. The catch-all exists solely to satisfy flex's `%option nodefault` requirement.	2025-10-29 23:55:37 +00:00
John Ericson	bffbdcfddc	Merge pull request #14390 from NixOS/constant-memory-uploads ... libstore: Make HTTP binary cache uploads run in constant memory	2025-10-29 23:14:42 +00:00
John Ericson	495d1b8435	Merge pull request #14393 from lovesegfault/s3-multipart-tests ... test(nixos): add S3 multipart upload integration tests	2025-10-29 22:56:21 +00:00
John Ericson	66d7b8fe1b	Merge pull request #14396 from roberth/c-api-docs ... doc: Improve libexpr-c docs	2025-10-29 22:38:04 +00:00
Sergei Zimmerman	cf75079bd8	libstore: Make uploads with filetransfer.cc consume a RestartableSource ... Make uploads run in constant memory. Also change the callbacks to be noexcept, since we really don't want to be unwinding the stack in the curl thread. That will definitely corrupt that stack and make nix/curl crash in very bad ways.	2025-10-29 18:34:56 -04:00