1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-12-08 18:11:02 +01:00
Commit graph

22175 commits

Author SHA1 Message Date
John Ericson
bec3c5cfcd JSON for Hash now has to be Base16
Fix #14532.

As discussed on the call today:

1. We'll stick with `format = "base16"` and `hash = "<hash>"`, not do
   `base16 = "<hash>"`, in order to be forward compatible with
   supporting more versioning formats.

   The motivation we discussed for someday *possibly* doing this is
   making it easier to write very slap-dash lang2nix tools that create
   (not consume) derivations with dynamic derivations.

2. We will remove support for non-base16 (and make that the default, not
   base64) in `Hash`, so this is strictly forward contingency, *not*
   yet something we support. (And also not something we have concrete
   plans to start supporting.)
2025-12-03 16:08:05 -05:00
Eelco Dolstra
5b95745bc9
Merge pull request #14702 from NixOS/fix-mingw
Fix mingw build
2025-12-03 19:55:03 +00:00
Eelco Dolstra
8d0e289fb9 Fix FdSource::restart() warning 2025-12-03 20:09:33 +01:00
Eelco Dolstra
c338f9cc5d Fix mingw build 2025-12-03 20:09:33 +01:00
John Ericson
c7801fc347
Merge pull request #14701 from NixOS/print-table
Move printTable() into libutil
2025-12-03 18:49:51 +00:00
Eelco Dolstra
c4dc42f306 printTable(): Make destination stream explicit 2025-12-03 18:43:56 +01:00
Eelco Dolstra
ab6dcf2047 Table: Use std::vectors 2025-12-03 18:43:51 +01:00
Eelco Dolstra
863f6811e4 Move table stuff into libutil 2025-12-03 18:42:11 +01:00
John Ericson
96d8b54e42
Merge pull request #14696 from NixOS/even-faster-tarball-cache
libfetchers/git-utils: Do not refresh pack files in GitFileSystemObje…
2025-12-03 15:31:43 +00:00
Sergei Zimmerman
d1f9fe984b
libfetchers/git-utils: Do not refresh pack files in GitFileSystemObjectSink
This leads to incredibly wasteful refreshes (see [^]) when oids are not found.
Since we are writing the pack files only once per unpacking we should not bother
with this refreshing at all.

This brings down the number of syscalls during `nix flake metadata "https://releases.nixos.org/nixos/25.05/nixos-25.05.813095.1c8ba8d3f763/nixexprs.tar.xz" --store "dummy://?read-only=false"`

Down from 576334 to just 6235 (100x less syscalls):

(Before)

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 32.98    0.625288           3    162898           getdents64
 29.58    0.560686           3    163514     81917 openat
 15.01    0.284509           3     81819       186 newfstatat
 10.99    0.208349           2     81601           close
 10.56    0.200145           2     81552           fstat

All these are coming from [2] and are totally useless.

(After)

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 76.47    0.108558         247       438        20 futex
  6.55    0.009292          18       513           munmap
  3.30    0.004680           7       639       492 openat
  2.68    0.003803          10       359           write
  2.30    0.003268           2      1146           read
  2.26    0.003215           3       870           mmap

[^]: 58d9363f02/include/git2/sys/odb_backend.h (L68-L75)
[2]: 58d9363f02/src/libgit2/odb_pack.c (L517-L546)
2025-12-03 03:23:12 +03:00
John Ericson
ec6789f9da
Merge pull request #14690 from roberth/mdbook-0.5
Support mdbook 0.5
2025-12-02 13:40:03 +00:00
John Ericson
e67c97b5f0
Merge pull request #14689 from NixOS/tarball-cache-faster
libfetchers/git-utils: Avoid using git_writestream for small files
2025-12-02 03:53:54 +00:00
Sergei Zimmerman
1b2cb1d75c
libfetchers/git-utils: Only create pack and mempack backends for the tarball cache
Now the unnecessary utimensat syscalls from the previous commit
are completely gone:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 33.39    0.646359           3    162898           getdents64
 29.34    0.567866           3    163523     81934 openat
 14.81    0.286739           3     81835       203 newfstatat
 10.98    0.212550           2     81593           close
 10.56    0.204458           2     81544           fstat
  0.15    0.002814           3       870           mmap

The rather crazy amount of getdents64 is still there though.
2025-12-02 06:09:03 +03:00
John Ericson
7f3ad17ac2
Merge pull request #14687 from NixOS/repl-print-interrupt
libutil/signals: Get rid of setInterruptThrown
2025-12-02 02:50:01 +00:00
Sergei Zimmerman
2f6550b7a7
libfetchers/git-utils: Avoid using git_writestream for small files
It turns out that libgit2 is incredibly naive and each git_writestream creates
a new temporary file like .cache/nix/tarball-cache/objects/streamed_git2_6a82bb68dc0a3918
that it reads from afterwards. It doesn't do any internal buffering.

Doing (with a fresh fetcher cache) a simple:

strace -c nix flake metadata "https://releases.nixos.org/nixos/25.05/nixos-25.05.813095.1c8ba8d3f763/nixexprs.tar.xz" --store "dummy://?read-only=false"

(Before)

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 31.05    2.372728           9    259790     81917 openat
 19.21    1.467784          30     48157           unlink
 10.43    0.796793           4    162898           getdents64
  7.75    0.592637           4    145969           read
  7.67    0.585976           3    177877           close
  7.11    0.543032           4    129970       190 newfstatat
  6.98    0.533211          10     48488           write
  4.09    0.312585           3     81443     81443 utimensat
  3.22    0.246158           3     81552           fstat

(After)

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 29.61    0.639393           3    162898           getdents64
 26.26    0.567119           3    163523     81934 openat
 12.50    0.269835           3     81848       207 newfstatat
 11.60    0.250429           3     81443     81443 utimensat
  9.82    0.212053           2     81593           close
  9.33    0.201390           2     81544           fstat
  0.18    0.003814           9       406        17 futex
2025-12-02 04:48:43 +03:00
Robert Hensing
0aef1ddb9e maint: Fix lowdown override compatibility with newer nixpkgs
Use `or ""` fallback for postInstall attribute which may not exist in
newer nixpkgs versions of lowdown.
2025-12-02 02:38:33 +01:00
Robert Hensing
2636f50dd4 maint: Remove mdbook-linkcheck and support mdbook 0.5.x
Fixes #14628

- Remove mdbook-linkcheck dependency and configuration (was blocking
  upgrades to mdbook 0.5.0+, configured with warning-policy = "ignore"
  due to false positives, and redundant with lychee-based link checking)
- Update substitute.py and anchors.jq to handle 'items' (mdbook 0.5.x)
  in addition to 'sections' (mdbook 0.4.x), as per mdbook 0.5.0
  changelog: "Book::sections was renamed to Book::items"
  https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#05-migration-guide
2025-12-02 02:38:30 +01:00
Sergei Zimmerman
0ec93e7ae7
libfetchers/git-utils: Clean up
Makes private functions static and removes dead code that was used
for fetching, but is currently dead.
2025-12-02 04:11:57 +03:00
Sergei Zimmerman
c0c1bde506
libutil/signals: Get rid of setInterruptThrown
The interrupting code is no longer relevant. Since
054be50257 logging no longer checks for interrupts
and in general logging should be noexcept.

Co-authored-by: Alois Wohlschlager <alois1@gmx-topmail.de>
Cherry-picked-from: https://gerrit.lix.systems/c/lix/+/1097
2025-12-02 00:59:49 +03:00
Sergei Zimmerman
d2615571e2
Merge pull request #14669 from NixOS/bump-nixpkgs
flake: Bump nixpkgs
2025-12-01 21:07:05 +00:00
Eelco Dolstra
18e31d404b
Merge pull request #14682 from NixOS/autodelete-move
AutoDelete: Add move constructor
2025-12-01 13:33:06 +00:00
Eelco Dolstra
0f4c7204f7
Mark move constructor as noexcept
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-12-01 13:46:59 +01:00
Eelco Dolstra
34e92724d6 AutoDelete: Add move constructor 2025-12-01 13:39:03 +01:00
John Ericson
890a4e980a
Merge pull request #14677 from NixOS/restartable-source-no-path
libutil: Get rid of restartableSourceFromFactory, add createAnonymousTempFile
2025-12-01 03:46:48 +00:00
John Ericson
a922a42b33
Merge pull request #14678 from NixOS/fix-error-message-download-filetransfer
libstore/filetransfer: Fix error message for interrupted requests
2025-12-01 03:39:25 +00:00
Sergei Zimmerman
7eab0bf9aa
libstore/filetransfer: Fix error message for interrupted requests
Sometimes we are uploading and that's confusing.
2025-12-01 05:34:29 +03:00
John Ericson
1e9b1ff851
Merge pull request #14676 from NixOS/fs-fixes
libstore: Use makeTempPath in optimizePath_, assorted fs fixes
2025-12-01 01:55:20 +00:00
Sergei Zimmerman
4b3536e092
libutil: Get rid of restartableSourceFromFactory
Instead we can just seek back in the file - duh. Also this makes use
of the anonymous temp file facility, since that is much safer (no need
window where the we don't have an open file descriptor for it).
2025-12-01 04:49:27 +03:00
Sergei Zimmerman
4ad272015e
libutil: Implement createAnonymousTempFile
There are a lot of cases where we don't care about having
the temporary file linked anywhere at all -- just a descriptor is more
than enough.
2025-12-01 04:49:26 +03:00
Sergei Zimmerman
40e3f5c0a4
libutil: Make AutoDelete non-copyable and non-movable
This is a good precaution, since we don't want to delete
directories twice accidentally.
2025-12-01 03:09:20 +03:00
Sergei Zimmerman
bf7c53f2d3
libutil: Propagate error code in createSymlink 2025-12-01 03:00:45 +03:00
Sergei Zimmerman
1cc337bb5f
libstore: Actually correctly call remove in case rename fails 2025-12-01 02:56:44 +03:00
Sergei Zimmerman
d888846b68
libstore: Use makeTempPath in optimizePath_
This was intended to be cherry-picked in 6aed9d877c,
but was left hanging. This is actually important for fixing [^]. emilazy let me know
of this bad cherry-pick and its significance.

[^]: https://github.com/NixOS/nix/issues/7273

Originally fixed by Lily Ballard <lily@ballards.net> in https://gerrit.lix.systems/c/lix/+/2100.
2025-12-01 02:51:37 +03:00
tomberek
8be9507a88
Merge pull request #14670 from juhp/RLO-chars
release-notes/rl-2.26.md: remove hidden Unicode RLO control chars
2025-11-30 20:00:38 +00:00
John Ericson
5b175ace18
Merge pull request #14675 from NixOS/cleanup-verb-filetransfer
libstore: Split FileTransferRequest::verb into verb + noun
2025-11-30 16:59:56 +00:00
Sergei Zimmerman
430bcda3ea
libstore: Split FileTransferRequest::verb into verb + noun
With the addition of "delete" method we can no longer rely on
just concatenating "ing" to get the continuous form of the verb.
Also some use-cases actually need a noun instead.
2025-11-30 18:49:11 +03:00
John Ericson
d7c29383c6
Merge pull request #14674 from Mic92/ca-derivation
Fix crash when querying realisations without ca-derivations enabled
2025-11-30 14:30:53 +00:00
Jörg Thalheim
ee5860f542 Fix crash when querying realisations without ca-derivations enabled
queryRealisationUncached was crashing with an assertion failure when
ca-derivations experimental feature is not enabled, because the SQLite
statements for realisations are only initialized when ca-derivations
is enabled.

Return nullptr (no realisation found) when ca-derivations is disabled,
matching the behavior of other CA-related functions like registerDrvOutput
which check for the feature before proceeding.
2025-11-30 14:25:11 +01:00
Jens Petersen
dacd5eac64 release-notes/rl-2.26.md: remove hidden Unicode RLO control chars (#14666)
They are flagged by Fedora CI checks as a potential security issue.
Use of such raw Right-to-Left control characters in source code is
strongly discouraged

also update release-credits-handle-to-name.json
2025-11-30 18:38:18 +08:00
John Ericson
3a32039508
Merge pull request #14672 from NixOS/fix-13948
libfetchers: Fix fetchGit with ref = "HEAD"
2025-11-30 01:03:01 +00:00
John Ericson
01dbbc926f
Merge pull request #14540 from lovesegfault/pre-compute-outputgraph
perf(libstore/derivation-builder): pre-compute outputGraph for linear complexity
2025-11-29 21:46:21 +00:00
Sergei Zimmerman
18f3598d57
libfetchers: Fix fetchGit with ref = "HEAD"
This seems to have been broken in ee9fa0d360.
Adding the HEAD:HEAD refspec looks like the correct solution.

Suggested-by: hxtmdev on github
2025-11-29 05:39:04 +03:00
John Ericson
c33b2c5834 perf(libstore/derivation-builder): Futher simplify / maybe optimize
We can precompute the exact information we need for topo sorting and
store it in `PerhapsNeedToRegister`. Depending on how `topoSort` works,
this is easy a performance improvement or just completely harmless.

Co-Authored-By: Bernardo Meurer Costa <beme@anthropic.com>
2025-11-28 21:38:59 -05:00
John Ericson
686ad9b052 perf(libstore/derivation-builder): pre-compute outputGraph for linear complexity
Build the inverse of `scratchOuputs` before running topoSort, avoiding
quadratic complexity when determining which outputs reference each
other. This fixes the FIXME comment about building the inverted map up
front.

Inspired by Lix commit 10c04ce84 / Change Id
Ibdd46e7b2e895bfeeebc173046d1297b41998181, but ended up being completely
different code.

Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Bernardo Meurer Costa <beme@anthropic.com>
2025-11-28 21:38:59 -05:00
John Ericson
13b4512cbe topoSort: Optimize templating
- No `std::function` overhead

- Don't copy if not necessary

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-11-28 21:38:54 -05:00
Sergei Zimmerman
0903b0aa7d
Merge pull request #14671 from NixOS/fix-asan-stack-overload-repl-doc
tests/functional: Work around stack overflows under ASAN in doc-funct…
2025-11-29 00:56:10 +00:00
Sergei Zimmerman
a2acb6d7aa
tests/functional: Work around stack overflows under ASAN in doc-functor tests
This was failing under ASAN in https://hydra.nixos.org/build/315173638/nixlog/1.
ASAN uses a bit more stack space and the default max call depth is not enough.
Not sure what's so special about this particular test.
2025-11-29 01:10:26 +03:00
John Ericson
048a58d331
Merge pull request #14668 from NixOS/fix-i686-expr-tests
libexpr: Fix tests on 32 bit systems
2025-11-28 02:33:29 +00:00
Sergei Zimmerman
7dfad3dba7
libexpr: Fix tests on 32 bit systems
This test is now pointless and the comment is outdated. Also the test fails
on 32 bit systems with:

../nix_api_value_internal.cc:22: Failure
Expected equality of these values:
  sizeof(nix::Value)
    Which is: 12
  sizeof(nix_value)
    Which is: 8

It just happeneded to work because Value is 16 bytes and nix_value was also 16 bytes.

Also get rid of a pointless inline in new_nix_value, since it's already static and
inline there does nothing.
2025-11-28 00:35:56 +03:00
Sergei Zimmerman
140c5f69f0
flake: Bump nixpkgs
Updates nixpkgs flake input. Also switches the input type to the channel
tarballs, since infra now supports the lockable tarball protocol.
2025-11-27 23:43:41 +03:00