1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-17 07:52:43 +01:00
Commit graph

2405 commits

Author SHA1 Message Date
Jörg Thalheim
e72f19eb28
Merge pull request #13300 from NixOS/deletePath-keep-going
deletePath(): Keep going when encountering an undeletable file
2025-06-02 18:00:10 +02:00
Eelco Dolstra
6b6d3dcf34 deletePath(): Keep going when encountering an undeletable file
This should reduce the impact of #5207.
2025-06-02 14:54:01 +02:00
Sergei Zimmerman
cdb8567473
libutil: Don't explicitly default special member functions
Since all of the member types are copyable/movable the compiler
will generate all of those by default anyway.
2025-06-01 20:55:28 +00:00
Sergei Zimmerman
b73e706589
libutil: Use std::shared_ptr<const Pos> instead of std::shared_ptr<Pos>
There's actually no mutation happening so there's no point in using
a mutable shared_ptr. Furthermore, this makes it much more evident to
the reader that no actual mutation (especially in multithreaded case)
is happening.

Also get rid of redundant constructor that isn't actually used anywhere
other than `Pos::operator std::shared_ptr<Pos>` which just passes in &*this,
(identical to just `this`), which can't be nullptr.
2025-06-01 20:55:26 +00:00
Sergei Zimmerman
fba1bb0c13
Clear displayPrefix in makeEmptySourceAccessor
Judging by the comment for `makeEmptySourceAccessor` the prefix has
to be empty:

> Return a source accessor that contains only an empty root directory.

Fixes #13295.
2025-05-29 19:35:12 +00:00
John Ericson
8d725fdcb0 Fix FreeBSD builds 2025-05-28 12:47:33 -04:00
Eelco Dolstra
6686b54077 Fix mingw build
https://hydra.nixos.org/build/298331457
2025-05-28 00:48:10 +02:00
John Ericson
625dce659a Prepare for FreeBSD sandboxing support
This is the utility changes from #9968, which were easier to rebase
first.

I (@Ericson2314) didn't write this code; I just rebased it.

Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Audrey Dutcher <audrey@rhelmot.io>
2025-05-27 14:54:50 -04:00
Sergei Zimmerman
114de63d88
Fix various typos in source code
This only touches code comments, class names, documentation,
enumeration names and tests.
2025-05-25 20:14:11 +00:00
Jörg Thalheim
7cef4559fe util/json-log-path: document unix sockets and concurrency issues 2025-05-23 10:45:12 +02:00
Eelco Dolstra
7240fb198f Add json-log-path setting
This setting specifies a path (which can be a regular file or Unix
domain socket) that receives a copy of all Nix log messages (in JSON
format).
2025-05-23 10:45:12 +02:00
Eelco Dolstra
0087188d47 Add convenience function for connecting to a Unix domain socket 2025-05-23 10:45:12 +02:00
Eelco Dolstra
9f680874c5 Make the JSON logger more robust
We now ignore connection / write errors.
2025-05-23 03:27:57 +02:00
Farid Zakaria
6c85a90b87 Change to atomic<uint32_t> 2025-05-21 10:07:04 -07:00
Farid Zakaria
8fd15ac228 Add canon for path 2025-05-21 10:05:23 -07:00
Farid Zakaria
9e26549c2b Consolidate tempName and makeTempPath 2025-05-21 09:01:49 -07:00
Farid Zakaria
6aed9d877c cherry-pick https://gerrit.lix.systems/c/lix/+/2100
Cherry-pick https://gerrit.lix.systems/c/lix/+/2100

This change fixes a potential concurrency failure when accessing random
which is not thread safe.

Co-authored-by: Lily Ballard <lily@ballards.net>
2025-05-21 08:49:09 -07:00
Sergei Zimmerman
49d026083b
Use transparent comparators for StringMap and StringPairs 2025-05-19 20:38:19 +00:00
Sergei Zimmerman
8ee513379a
Use StringMap instead of std::map<std::string, std::string> throughout the codebase 2025-05-19 20:33:28 +00:00
Eelco Dolstra
7d89e46f65 Sync: Support moving out of another Sync 2025-05-17 19:38:36 +02:00
Sergei Zimmerman
5ea81f5b8f
libexpr: Actually cache line information in PosTable
Previous code had a sneaky bug due to which no caching
actually happened:

```cpp
auto linesForInput = (*lines)[origin->offset];
```

That should have been:
```cpp
auto & linesForInput = (*lines)[origin->offset];
```

See [1].

Now that it also makes sense to make the cache bound in side
in order not to memoize all the sources without freeing any memory.
The default cache size has been chosen somewhat arbitrarily to be ~64k
origins. For reference, 25.05 nixpkgs has ~50k .nix files.

Simple benchmark:

```nix
let
  pkgs = import <nixpkgs> { };
in
builtins.foldl' (acc: el: acc + el.line) 0 (
  builtins.genList (x: builtins.unsafeGetAttrPos "gcc" pkgs) 10000
)
```

(After)

```
$ hyperfine "result/bin/nix eval -f ./test.nix"
Benchmark 1: result/bin/nix eval -f ./test.nix
  Time (mean ± σ):     292.7 ms ±   3.9 ms    [User: 131.0 ms, System: 120.5 ms]
  Range (min … max):   288.1 ms … 300.5 ms    10 runs
```

(Before)

```
hyperfine "nix eval -f ./test.nix"
Benchmark 1: nix eval -f ./test.nix
  Time (mean ± σ):     666.7 ms ±   6.4 ms    [User: 428.3 ms, System: 191.2 ms]
  Range (min … max):   659.7 ms … 681.3 ms    10 runs
```

If the origin happens to be a `all-packages.nix` or similar in size then the
difference is much more dramatic.

[1]: 22e3f0e987
2025-05-15 23:07:25 +00:00
Sergei Zimmerman
4711720efe
libutil: Add LRUCache::getOrNullptr
For heavier objects it doesn't make sense to return
a std::optional with the copy of the data, when it
can be used by const reference.
2025-05-15 22:28:41 +00:00
Sergei Zimmerman
d955b401a7
libutil: Sprinkle some noexcept on LRUCache::{size,clear}
The underlying containers are already noexcept to destroy and dtors are
noexcept in general.
2025-05-14 22:14:46 +00:00
Sergei Zimmerman
2f2e04142e
libutil: Simplify LRUCache::get by using list splice
Splicing the list element to the back can be done in
a much simpler and concise way without the need for
erasing and re-inserting the element. Doing it this
way is equivalent to just moving node pointers around,
whereas inserting/erasing allocates/deallocates new nodes.
2025-05-14 22:05:53 +00:00
Sergei Zimmerman
cd61e922ff
libutil: Use heterogeneous lookup for LRUCache
This gets rid of some ugly std::string_view -> std::string
conversions, which are an eye-sore and lead to extra copying.
2025-05-14 21:42:35 +00:00
Sergei Zimmerman
90d70aa4c9
libutil: Format lru-cache.hh
Rip off the band-aid for further refactors. The diff is
very small, so it makes to get it out of the way first.
2025-05-14 21:23:13 +00:00
Jörg Thalheim
f0f196cef0
Merge pull request #13175 from xokdvium/optimise-fetchtarball
libutil/tarfile: Create the scratch `std::vector` only once
2025-05-13 14:21:26 +02:00
Sergei Zimmerman
7628155d2b
libutil/tarfile: Create the scratch std::vector only once
I can't find a good way to benchmark in isolation from the
git cache, but common sense dictates that creating (and destroying)
a 131KiB std::vector for each regular file from the archive imposes
quite a significant overhead regardless of the IO bound git cache.

AFAICT there is no reason to keep a copy of the data since
it always gets fed into the sink and there are no coroutines/threads
in sight.
2025-05-13 10:50:46 +00:00
Eelco Dolstra
bdb3f613dd Improve 'cannot read file from tarball' error
It now says e.g.

  error: cannot read file from tarball: Truncated tar archive detected while reading data
2025-05-13 10:32:12 +02:00
John Ericson
1594d4b879 Fix windows warning 2025-05-06 15:02:10 -04:00
Eelco Dolstra
47989a2124 Simplify ConfigRegistrations 2025-05-05 09:42:16 +02:00
Jörg Thalheim
bd80a4f176
Merge pull request #13137 from xokdvium/regex-cache-transparent
libexpr: Use C++20 heterogeneous lookup for RegexCache
2025-05-04 22:18:07 +02:00
Sergei Zimmerman
36c583dae0
libexpr: Use C++20 heterogeneous lookup for RegexCache 2025-05-04 16:03:57 +00:00
John Ericson
2676ae7ca6
Merge pull request #12676 from silvanshade/blake3-tbb
Implement memory-mapped IO and multi-threading for BLAKE3 hashing
2025-05-04 10:58:53 -04:00
Jörg Thalheim
7808aa2eee
Merge pull request #13129 from xokdvium/transparent-comparator
Use transparent comparators for `std::set<std::string>` (NFC)
2025-05-03 09:04:52 +02:00
silvanshade
7db388f597
Implement multi-threaded BLAKE3 hashing 2025-05-02 15:29:22 -06:00
silvanshade
b1783ff615
Implement memory-mapped IO for Sinks 2025-05-02 15:29:22 -06:00
Sergei Zimmerman
161c5dbf39
libexpr: Remove unused field from SymbolTable::symbols and emplace into the ChunkedVector
Remove outdated and no longer relevant TODO. It's more confusing
now, since symbol table must now be addressed by uint32_t indices
in order to keep Attr size down to 16 bytes on 64 bit machines.
2025-05-02 20:42:47 +00:00
Sergei Zimmerman
ebb836d499
Use transparent comparators for std::set<std::string> (NFC)
This patch finally applies the transition to std::less<>,
which is a transparent comparator. There's no functional
change and string lookups in sets are now more efficient
and don't produce temporaries (e.g. set.find(std::string_view{"key"})).
2025-05-02 17:43:02 +00:00
Sergei Zimmerman
d8c97d8073
treewide: Use StringSet alias consistently instead of std::set<std::string>
The intention is to switch to transparent comparators from N3657 for
ordered set containers for strings and using the alias consistently
would simplify things.
2025-05-02 17:40:29 +00:00
Jörg Thalheim
979d5a7cae Drop fs alias in favour of std::filesystem
Since we dropped fs::symlink_exists, we no longer have a need for the fs
namespace. Having less abstractions makes it easier to lookup the
functions in reference documentations.
2025-05-01 14:24:34 +02:00
Jörg Thalheim
5b59be914d Replace symlink_exists with pathExists
As it turns out the orignal implementation of symlink_exists cannot be
used in Nix because it did now std::filesystem::filesystem_error.
The new implementation fixes that but is now actually the same as
pathExists except for the path type.
2025-05-01 14:22:35 +02:00
Jörg Thalheim
143fb88ceb
Merge pull request #13122 from Mic92/directory-iterator
Replace all instances of std::filesystem::directory_iterator with DirectoryIterator
2025-05-01 14:22:21 +02:00
Jörg Thalheim
1c4496f4e5 replace all instances of std::filesystem::directory_iterator with DirectoryIterator 2025-05-01 11:54:26 +02:00
Jörg Thalheim
7ccc0d591f add DirectoryIterator to re-throw std::filesystem::filesystem_error
Co-authored-by: Sergei Zimmerman <145775305+xokdvium@users.noreply.github.com>
2025-05-01 11:54:13 +02:00
Jörg Thalheim
408746cba6 replaceSymlink: fix quoting in error message 2025-05-01 08:19:18 +02:00
mergify[bot]
4548dd1abb
Merge pull request #13112 from NaN-git/fix-json-getInteger
bugfix in getInteger(const nlohmann::json &) and add bounds checks
2025-04-30 21:20:49 +00:00
Philipp Otterbein
788be3f964 bugfix in getInteger(const nlohmann::json &) and add bounds checks
improve error messages, too
2025-04-30 01:38:48 +02:00
Sergei Trofimovich
e322b714dc libutil: amend OSC 8 escape stripping for xterm-style separator
Before the change `nix` was stripping warning flags
reported by `gcc-14` too eagerly:

    $ nix build -f. texinfo4
    error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
           last 25 log lines:
           >  1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
           >       |                                            ~~~~~~~~^~~~
           > window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type []
           >  1887 |                                       &replen);
           >       |                                       ^~~~~~~
           >       |                                       |
           >       |                                       int *

After the change the compiler flag remains:

    $ ~/patched.nix build -f. texinfo4
    error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
       last 25 log lines:
       >  1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
       >       |                                            ~~~~~~~~^~~~
       > window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type [-Wincompatible-pointer-types]
       >  1887 |                                       &replen);
       >       |                                       ^~~~~~~
       >       |                                       |
       >       |                                       int *

Note the difference in flag rendering around the warning.

https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda has a
good sumamry of why it happens. Befomre the change `nix` was handling
just one form or URL separator:

    $ printf '\e]8;;http://example.com\e\\This is a link\e]8;;\e\\\n'

Now it also handled another for (used by gcc-14`):

    printf '\e]8;;http://example.com\aThis is a link\e]8;;\a\n'

While at it fixed accumulation of trailing escape `\e\\` symbol.
2025-04-29 16:12:18 +01:00
Eelco Dolstra
5565679b45
Merge pull request #13100 from NaN-git/fix-musl-deadlock
allocate SimpleLogger before forking
2025-04-28 09:01:06 +02:00