First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.
This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.
Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.
(cherry picked from commit ced8d311a5)
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.
Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.
It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.
By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.
[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]
Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
(cherry picked from commit afac093b34)
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.
nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298
would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.
So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).
This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).
(cherry picked from commit 39daa4a0d3)
This leads to confusion about what the command does.
E.g. https://github.com/NixOS/nix/issues/9359
- Move the description up
- Remove details about the individual formatters
(cherry picked from commit 8c4842de8c)
The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.
(cherry picked from commit d2f627dda9)
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)
Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory. The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.
(cherry picked from commit 5a794d9366)
Unfortunately `StringSource` class is very easy was very easy to misuse
because the ctor took a plain `std::string_view` which has a bad habit
of being implicitly convertible from an rvalue `std::string`. This lead
to unintentional use-after-free bugs.
This patch makes `StringSource` much harder to misuse by disabling the ctor
from a `std::string &&` (but `const std::string &` is ok).
Fix affected tests from libstore-tests.
Reformat those tests with clangd's range formatting since the diff is tiny
and it seems appropriate.
(cherry picked from commit 5bc8957c73)
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:
../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'
(cherry picked from commit b9f8c4af40)
