From 68a802d253daef01ba4ecd21d033d4d023fd1db7 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 9 Dec 2025 15:26:59 +0100 Subject: [PATCH] release notes: 2.33.0 --- doc/manual/rl-next/channels-subdomain.md | 9 - doc/manual/rl-next/json-format-changes.md | 88 -------- doc/manual/rl-next/libcurl-pausing.md | 12 -- doc/manual/rl-next/repl-interrupt.md | 8 - doc/manual/rl-next/s3-curl-implementation.md | 40 ---- doc/manual/rl-next/s3-object-versioning.md | 14 -- doc/manual/rl-next/s3-storage-class.md | 21 -- doc/manual/source/SUMMARY.md.in | 1 + doc/manual/source/release-notes/rl-2.33.md | 214 +++++++++++++++++++ 9 files changed, 215 insertions(+), 192 deletions(-) delete mode 100644 doc/manual/rl-next/channels-subdomain.md delete mode 100644 doc/manual/rl-next/json-format-changes.md delete mode 100644 doc/manual/rl-next/libcurl-pausing.md delete mode 100644 doc/manual/rl-next/repl-interrupt.md delete mode 100644 doc/manual/rl-next/s3-curl-implementation.md delete mode 100644 doc/manual/rl-next/s3-object-versioning.md delete mode 100644 doc/manual/rl-next/s3-storage-class.md create mode 100644 doc/manual/source/release-notes/rl-2.33.md diff --git a/doc/manual/rl-next/channels-subdomain.md b/doc/manual/rl-next/channels-subdomain.md deleted file mode 100644 index 38dc7bb33..000000000 --- a/doc/manual/rl-next/channels-subdomain.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -synopsis: Channel URLs migrated to channels.nixos.org subdomain -prs: [14518] -issues: [14517] ---- - -Channel URLs have been updated from `https://nixos.org/channels/` to `https://channels.nixos.org/` throughout Nix. - -The subdomain provides better reliability with IPv6 support and improved CDN distribution. The old domain apex (`nixos.org/channels/`) currently redirects to the new location but may be deprecated in the future. diff --git a/doc/manual/rl-next/json-format-changes.md b/doc/manual/rl-next/json-format-changes.md deleted file mode 100644 index 78e543512..000000000 --- a/doc/manual/rl-next/json-format-changes.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -synopsis: "JSON format changes for store path info and derivations" -prs: [] -issues: [] ---- - -JSON formats for store path info and derivations have been updated with new versions and structured fields. - -## Store Path Info JSON - -`nix path-info --json` now requires a `--json-format` flag to specify the output format version. -Using `--json` without `--json-format` is deprecated and will become an error in a future release. -For now, it defaults to version 1 with a warning, for a smoother migration. - -### Version 1 (`--json-format 1`) - -This is the legacy format, preserved for backwards compatibility: - -- String-based hash values (e.g., `"narHash": "sha256:FePFYIlM..."`) -- String-based content addresses (e.g., `"ca": "fixed:r:sha256:1abc..."`) -- Full store paths for map keys and references (e.g., `"/nix/store/abc...-foo"`) -- Now includes `"storeDir"` field at the top level - -### Version 2 (`--json-format 2`) - -The new structured format follows the [JSON guidelines](@docroot@/development/json-guideline.md) with the following changes: - -- **Nested structure with top-level metadata**: - - The output is now wrapped in an object with `version`, `storeDir`, and `info` fields: - - ```json - { - "version": 2, - "storeDir": "/nix/store", - "info": { ... } - } - ``` - - The map from store bath base names to store object info is nested under the `info` field. - -- **Store path base names instead of full paths**: - - Map keys and references use store path base names (e.g., `"abc...-foo"`) instead of full absolute store paths. - Combined with `storeDir`, the full path can be reconstructed. - -- **Structured `ca` field**: - - Content address is now a structured JSON object instead of a string: - - - Old: `"ca": "fixed:r:sha256:1abc..."` - - New: `"ca": {"method": "nar", "hash": "sha256-ungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0="}` - - Still `null` values for input-addressed store objects - - The `hash` field uses the [SRI](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) format like other hashes. - -Nix currently only produces, and doesn't consume this format. - -Additionally the following field is added to both formats. -(The `version` tracks breaking changes, and adding fields to outputted JSON is not a breaking change.) - -- **`version` field**: - - All store path info JSON now includes `"version": <1|2>`. - -- **`storeDir` field**: - - Top-level `"storeDir"` field contains the store directory path (e.g., `"/nix/store"`). - -## Derivation JSON (Version 4) - -The derivation JSON format has been updated from version 3 to version 4: - -- **Restructured inputs**: - - Inputs are now nested under an `inputs` object: - - - Old: `"inputSrcs": [...], "inputDrvs": {...}` - - New: `"inputs": {"srcs": [...], "drvs": {...}}` - -- **Consistent content addresses**: - - Fixed content-addressed outputs now use structured JSON format. - This is the same format as `ca` in store path info (after the new version). - -Version 3 and earlier formats are *not* accepted when reading. - -**Affected command**: `nix derivation`, namely its `show` and `add` sub-commands. diff --git a/doc/manual/rl-next/libcurl-pausing.md b/doc/manual/rl-next/libcurl-pausing.md deleted file mode 100644 index 68a0b5ecf..000000000 --- a/doc/manual/rl-next/libcurl-pausing.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -synopsis: Fix "download buffer is full; consider increasing the 'download-buffer-size' setting" warning -prs: [14614] -issues: [11728] ---- - -The underlying issue that led to [#11728](https://github.com/NixOS/nix/issues/11728) has been resolved by utilizing -[libcurl write pausing functionality](https://curl.se/libcurl/c/curl_easy_pause.html) to control backpressure when unpacking to slow destinations like the git-backed tarball cache. The default value of `download-buffer-size` is now 1 MiB and it's no longer recommended to increase it, since the root cause has been fixed. - -This is expected to improve download performance on fast connections, since previously a single slow download consumer would stall the thread and prevent any other transfers from progressing. - -Many thanks go out to the [Lix project](https://lix.systems/) for the [implementation](https://git.lix.systems/lix-project/lix/commit/4ae6fb5a8f0d456b8d2ba2aaca3712b4e49057fc) that served as inspiration for this change and for triaging libcurl [issues with pausing](https://github.com/curl/curl/issues/19334). diff --git a/doc/manual/rl-next/repl-interrupt.md b/doc/manual/rl-next/repl-interrupt.md deleted file mode 100644 index 7e7e45ed6..000000000 --- a/doc/manual/rl-next/repl-interrupt.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -synopsis: Interrupting REPL commands works more than once -issues: [13481] ---- - -Previously, this only worked once per REPL session; further attempts would be ignored. -This issue is now fixed, so REPL commands such as `:b` or `:p` can be canceled consistently. -This is a cherry-pick of the change from the [Lix project](https://gerrit.lix.systems/c/lix/+/1097). diff --git a/doc/manual/rl-next/s3-curl-implementation.md b/doc/manual/rl-next/s3-curl-implementation.md deleted file mode 100644 index 2647ac581..000000000 --- a/doc/manual/rl-next/s3-curl-implementation.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -synopsis: "Improved S3 binary cache support via HTTP" -prs: [13752, 13823, 14026, 14120, 14131, 14135, 14144, 14170, 14190, 14198, 14206, 14209, 14222, 14223, 14330, 14333, 14335, 14336, 14337, 14350, 14356, 14357, 14374, 14375, 14376, 14377, 14391, 14393, 14420, 14421] -issues: [13084, 12671, 11748, 12403] ---- - -S3 binary cache operations now happen via HTTP, leveraging `libcurl`'s native -AWS SigV4 authentication instead of the AWS C++ SDK, providing significant -improvements: - -- **Reduced memory usage**: Eliminates memory buffering issues that caused - segfaults with large files -- **Fixed upload reliability**: Resolves AWS SDK chunking errors - (`InvalidChunkSizeError`) -- **Lighter dependencies**: Uses lightweight `aws-crt-cpp` instead of full - `aws-cpp-sdk`, reducing build complexity - -The new implementation requires curl >= 7.75.0 and `aws-crt-cpp` for credential -management. - -All existing S3 URL formats and parameters remain supported, however the store -settings for configuring multipart uploads have changed: - -- **`multipart-upload`** (default: `false`): Enable multipart uploads for large - files. When enabled, files exceeding the multipart threshold will be uploaded - in multiple parts. - -- **`multipart-threshold`** (default: `100 MiB`): Minimum file size for using - multipart uploads. Files smaller than this will use regular PUT requests. - Only takes effect when `multipart-upload` is enabled. - -- **`multipart-chunk-size`** (default: `5 MiB`): Size of each part in multipart - uploads. Must be at least 5 MiB (AWS S3 requirement). Larger chunk sizes - reduce the number of requests but use more memory. - -- **`buffer-size`**: Has been replaced by `multipart-chunk-size` and is now an alias to it. - -Note that this change also means Nix now supports S3 binary cache stores even -if built without `aws-crt-cpp`, but only for public buckets which do not -require authentication. diff --git a/doc/manual/rl-next/s3-object-versioning.md b/doc/manual/rl-next/s3-object-versioning.md deleted file mode 100644 index 3b85e0926..000000000 --- a/doc/manual/rl-next/s3-object-versioning.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -synopsis: "S3 URLs now support object versioning via versionId parameter" -prs: [14274] -issues: [13955] ---- - -S3 URLs now support a `versionId` query parameter to fetch specific versions -of objects from S3 buckets with versioning enabled. This allows pinning to -exact object versions for reproducibility and protection against unexpected -changes: - -``` -s3://bucket/key?region=us-east-1&versionId=abc123def456 -``` diff --git a/doc/manual/rl-next/s3-storage-class.md b/doc/manual/rl-next/s3-storage-class.md deleted file mode 100644 index d742b5747..000000000 --- a/doc/manual/rl-next/s3-storage-class.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -synopsis: "S3 binary cache stores now support storage class configuration" -prs: [14464] -issues: [7015] ---- - -S3 binary cache stores now support configuring the storage class for uploaded objects via the `storage-class` parameter. This allows users to optimize costs by selecting appropriate storage tiers based on access patterns. - -Example usage: - -```bash -# Use Glacier storage for long-term archival -nix copy --to 's3://my-bucket?storage-class=GLACIER' /nix/store/... - -# Use Intelligent Tiering for automatic cost optimization -nix copy --to 's3://my-bucket?storage-class=INTELLIGENT_TIERING' /nix/store/... -``` - -The storage class applies to both regular uploads and multipart uploads. When not specified, objects use the bucket's default storage class. - -See the [S3 storage classes documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html) for available storage classes and their characteristics. diff --git a/doc/manual/source/SUMMARY.md.in b/doc/manual/source/SUMMARY.md.in index a8e286314..8b6b29f6a 100644 --- a/doc/manual/source/SUMMARY.md.in +++ b/doc/manual/source/SUMMARY.md.in @@ -151,6 +151,7 @@ - [Contributing](development/contributing.md) - [Releases](release-notes/index.md) {{#include ./SUMMARY-rl-next.md}} + - [Release 2.33 (2025-12-09)](release-notes/rl-2.33.md) - [Release 2.32 (2025-10-06)](release-notes/rl-2.32.md) - [Release 2.31 (2025-08-21)](release-notes/rl-2.31.md) - [Release 2.30 (2025-07-07)](release-notes/rl-2.30.md) diff --git a/doc/manual/source/release-notes/rl-2.33.md b/doc/manual/source/release-notes/rl-2.33.md new file mode 100644 index 000000000..47217aa08 --- /dev/null +++ b/doc/manual/source/release-notes/rl-2.33.md @@ -0,0 +1,214 @@ +# Release 2.33.0 (2025-12-09) + +- Channel URLs migrated to channels.nixos.org subdomain [#14517](https://github.com/NixOS/nix/issues/14517) [#14518](https://github.com/NixOS/nix/pull/14518) + + Channel URLs have been updated from `https://nixos.org/channels/` to `https://channels.nixos.org/` throughout Nix. + + The subdomain provides better reliability with IPv6 support and improved CDN distribution. The old domain apex (`nixos.org/channels/`) currently redirects to the new location but may be deprecated in the future. + +- JSON format changes for store path info and derivations + + JSON formats for store path info and derivations have been updated with new versions and structured fields. + + ## Store Path Info JSON + + `nix path-info --json` now requires a `--json-format` flag to specify the output format version. + Using `--json` without `--json-format` is deprecated and will become an error in a future release. + For now, it defaults to version 1 with a warning, for a smoother migration. + + ### Version 1 (`--json-format 1`) + + This is the legacy format, preserved for backwards compatibility: + + - String-based hash values (e.g., `"narHash": "sha256:FePFYIlM..."`) + - String-based content addresses (e.g., `"ca": "fixed:r:sha256:1abc..."`) + - Full store paths for map keys and references (e.g., `"/nix/store/abc...-foo"`) + - Now includes `"storeDir"` field at the top level + + ### Version 2 (`--json-format 2`) + + The new structured format follows the [JSON guidelines](@docroot@/development/json-guideline.md) with the following changes: + + - **Nested structure with top-level metadata**: + + The output is now wrapped in an object with `version`, `storeDir`, and `info` fields: + + ```json + { + "version": 2, + "storeDir": "/nix/store", + "info": { ... } + } + ``` + + The map from store bath base names to store object info is nested under the `info` field. + + - **Store path base names instead of full paths**: + + Map keys and references use store path base names (e.g., `"abc...-foo"`) instead of full absolute store paths. + Combined with `storeDir`, the full path can be reconstructed. + + - **Structured `ca` field**: + + Content address is now a structured JSON object instead of a string: + + - Old: `"ca": "fixed:r:sha256:1abc..."` + - New: `"ca": {"method": "nar", "hash": "sha256-ungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0="}` + - Still `null` values for input-addressed store objects + + The `hash` field uses the [SRI](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) format like other hashes. + + Nix currently only produces, and doesn't consume this format. + + Additionally the following field is added to both formats. + (The `version` tracks breaking changes, and adding fields to outputted JSON is not a breaking change.) + + - **`version` field**: + + All store path info JSON now includes `"version": <1|2>`. + + - **`storeDir` field**: + + Top-level `"storeDir"` field contains the store directory path (e.g., `"/nix/store"`). + + ## Derivation JSON (Version 4) + + The derivation JSON format has been updated from version 3 to version 4: + + - **Restructured inputs**: + + Inputs are now nested under an `inputs` object: + + - Old: `"inputSrcs": [...], "inputDrvs": {...}` + - New: `"inputs": {"srcs": [...], "drvs": {...}}` + + - **Consistent content addresses**: + + Fixed content-addressed outputs now use structured JSON format. + This is the same format as `ca` in store path info (after the new version). + + Version 3 and earlier formats are *not* accepted when reading. + + **Affected command**: `nix derivation`, namely its `show` and `add` sub-commands. + +- Fix "download buffer is full; consider increasing the 'download-buffer-size' setting" warning [#11728](https://github.com/NixOS/nix/issues/11728) [#14614](https://github.com/NixOS/nix/pull/14614) + + The underlying issue that led to [#11728](https://github.com/NixOS/nix/issues/11728) has been resolved by utilizing + [libcurl write pausing functionality](https://curl.se/libcurl/c/curl_easy_pause.html) to control backpressure when unpacking to slow destinations like the git-backed tarball cache. The default value of `download-buffer-size` is now 1 MiB and it's no longer recommended to increase it, since the root cause has been fixed. + + This is expected to improve download performance on fast connections, since previously a single slow download consumer would stall the thread and prevent any other transfers from progressing. + + Many thanks go out to the [Lix project](https://lix.systems/) for the [implementation](https://git.lix.systems/lix-project/lix/commit/4ae6fb5a8f0d456b8d2ba2aaca3712b4e49057fc) that served as inspiration for this change and for triaging libcurl [issues with pausing](https://github.com/curl/curl/issues/19334). + +- Interrupting REPL commands works more than once [#13481](https://github.com/NixOS/nix/issues/13481) + + Previously, this only worked once per REPL session; further attempts would be ignored. + This issue is now fixed, so REPL commands such as `:b` or `:p` can be canceled consistently. + This is a cherry-pick of the change from the [Lix project](https://gerrit.lix.systems/c/lix/+/1097). + +- Improved S3 binary cache support via HTTP [#11748](https://github.com/NixOS/nix/issues/11748) [#12403](https://github.com/NixOS/nix/issues/12403) [#12671](https://github.com/NixOS/nix/issues/12671) [#13084](https://github.com/NixOS/nix/issues/13084) [#13752](https://github.com/NixOS/nix/pull/13752) [#13823](https://github.com/NixOS/nix/pull/13823) [#14026](https://github.com/NixOS/nix/pull/14026) [#14120](https://github.com/NixOS/nix/pull/14120) [#14131](https://github.com/NixOS/nix/pull/14131) [#14135](https://github.com/NixOS/nix/pull/14135) [#14144](https://github.com/NixOS/nix/pull/14144) [#14170](https://github.com/NixOS/nix/pull/14170) [#14190](https://github.com/NixOS/nix/pull/14190) [#14198](https://github.com/NixOS/nix/pull/14198) [#14206](https://github.com/NixOS/nix/pull/14206) [#14209](https://github.com/NixOS/nix/pull/14209) [#14222](https://github.com/NixOS/nix/pull/14222) [#14223](https://github.com/NixOS/nix/pull/14223) [#14330](https://github.com/NixOS/nix/pull/14330) [#14333](https://github.com/NixOS/nix/pull/14333) [#14335](https://github.com/NixOS/nix/pull/14335) [#14336](https://github.com/NixOS/nix/pull/14336) [#14337](https://github.com/NixOS/nix/pull/14337) [#14350](https://github.com/NixOS/nix/pull/14350) [#14356](https://github.com/NixOS/nix/pull/14356) [#14357](https://github.com/NixOS/nix/pull/14357) [#14374](https://github.com/NixOS/nix/pull/14374) [#14375](https://github.com/NixOS/nix/pull/14375) [#14376](https://github.com/NixOS/nix/pull/14376) [#14377](https://github.com/NixOS/nix/pull/14377) [#14391](https://github.com/NixOS/nix/pull/14391) [#14393](https://github.com/NixOS/nix/pull/14393) [#14420](https://github.com/NixOS/nix/pull/14420) [#14421](https://github.com/NixOS/nix/pull/14421) + + S3 binary cache operations now happen via HTTP, leveraging `libcurl`'s native + AWS SigV4 authentication instead of the AWS C++ SDK, providing significant + improvements: + + - **Reduced memory usage**: Eliminates memory buffering issues that caused + segfaults with large files + - **Fixed upload reliability**: Resolves AWS SDK chunking errors + (`InvalidChunkSizeError`) + - **Lighter dependencies**: Uses lightweight `aws-crt-cpp` instead of full + `aws-cpp-sdk`, reducing build complexity + + The new implementation requires curl >= 7.75.0 and `aws-crt-cpp` for credential + management. + + All existing S3 URL formats and parameters remain supported, however the store + settings for configuring multipart uploads have changed: + + - **`multipart-upload`** (default: `false`): Enable multipart uploads for large + files. When enabled, files exceeding the multipart threshold will be uploaded + in multiple parts. + + - **`multipart-threshold`** (default: `100 MiB`): Minimum file size for using + multipart uploads. Files smaller than this will use regular PUT requests. + Only takes effect when `multipart-upload` is enabled. + + - **`multipart-chunk-size`** (default: `5 MiB`): Size of each part in multipart + uploads. Must be at least 5 MiB (AWS S3 requirement). Larger chunk sizes + reduce the number of requests but use more memory. + + - **`buffer-size`**: Has been replaced by `multipart-chunk-size` and is now an alias to it. + + Note that this change also means Nix now supports S3 binary cache stores even + if built without `aws-crt-cpp`, but only for public buckets which do not + require authentication. + +- S3 URLs now support object versioning via versionId parameter [#13955](https://github.com/NixOS/nix/issues/13955) [#14274](https://github.com/NixOS/nix/pull/14274) + + S3 URLs now support a `versionId` query parameter to fetch specific versions + of objects from S3 buckets with versioning enabled. This allows pinning to + exact object versions for reproducibility and protection against unexpected + changes: + + ``` + s3://bucket/key?region=us-east-1&versionId=abc123def456 + ``` + +- S3 binary cache stores now support storage class configuration [#7015](https://github.com/NixOS/nix/issues/7015) [#14464](https://github.com/NixOS/nix/pull/14464) + + S3 binary cache stores now support configuring the storage class for uploaded objects via the `storage-class` parameter. This allows users to optimize costs by selecting appropriate storage tiers based on access patterns. + + Example usage: + + ```bash + # Use Glacier storage for long-term archival + nix copy --to 's3://my-bucket?storage-class=GLACIER' /nix/store/... + + # Use Intelligent Tiering for automatic cost optimization + nix copy --to 's3://my-bucket?storage-class=INTELLIGENT_TIERING' /nix/store/... + ``` + + The storage class applies to both regular uploads and multipart uploads. When not specified, objects use the bucket's default storage class. + + See the [S3 storage classes documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html) for available storage classes and their characteristics. + + +## Contributors + + +This release was made possible by the following 33 contributors: + +- Adam Dinwoodie [**(@me-and)**](https://github.com/me-and) +- jonhermansen [**(@jonhermansen)**](https://github.com/jonhermansen) +- Arnout Engelen [**(@raboof)**](https://github.com/raboof) +- Jean-François Roche [**(@jfroche)**](https://github.com/jfroche) +- tomberek [**(@tomberek)**](https://github.com/tomberek) +- Eelco Dolstra [**(@edolstra)**](https://github.com/edolstra) +- Marcel [**(@MarcelCoding)**](https://github.com/MarcelCoding) +- David McFarland [**(@corngood)**](https://github.com/corngood) +- Soumyadip Sarkar [**(@neuralsorcerer)**](https://github.com/neuralsorcerer) +- Cole Helbling [**(@cole-h)**](https://github.com/cole-h) +- John Ericson [**(@Ericson2314)**](https://github.com/Ericson2314) +- Tristan Ross [**(@RossComputerGuy)**](https://github.com/RossComputerGuy) +- Alex Auvolat [**(@Alexis211)**](https://github.com/Alexis211) +- edef [**(@edef1c)**](https://github.com/edef1c) +- Sergei Zimmerman [**(@xokdvium)**](https://github.com/xokdvium) +- Vinayak Goyal [**(@vinayakankugoyal)**](https://github.com/vinayakankugoyal) +- Graham Dennis [**(@GrahamDennis)**](https://github.com/GrahamDennis) +- Aspen Smith [**(@glittershark)**](https://github.com/glittershark) +- Jens Petersen [**(@juhp)**](https://github.com/juhp) +- Bernardo Meurer [**(@lovesegfault)**](https://github.com/lovesegfault) +- Peter Bynum [**(@pkpbynum)**](https://github.com/pkpbynum) +- Jörg Thalheim [**(@Mic92)**](https://github.com/Mic92) +- Alex Decious [**(@adeci)**](https://github.com/adeci) +- Matthieu Coudron [**(@teto)**](https://github.com/teto) +- Domen Kožar [**(@domenkozar)**](https://github.com/domenkozar) +- Taeer Bar-Yam [**(@Radvendii)**](https://github.com/Radvendii) +- Seth Flynn [**(@getchoo)**](https://github.com/getchoo) +- Robert Hensing [**(@roberth)**](https://github.com/roberth) +- Vladimir Panteleev [**(@CyberShadow)**](https://github.com/CyberShadow) +- bryango [**(@bryango)**](https://github.com/bryango) +- Henry [**(@cootshk)**](https://github.com/cootshk) +- Martin Joerg [**(@mjoerg)**](https://github.com/mjoerg) +- Farid Zakaria [**(@fzakaria)**](https://github.com/fzakaria) +- Graham Dennis