1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-22 10:19:36 +01:00
Commit graph

20903 commits

Author SHA1 Message Date
Sergei Zimmerman
b540c2419f {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:52 +00:00
Robert Hensing
1bff2aeec0
Merge pull request #12645 from xokdvium/debugger-perf
Make debugger significantly faster
2025-03-14 13:21:12 +00:00
Robert Hensing
fe00dfbd56 nix-cli: Add --json --pretty / --no-pretty
Default: istty(stdout)

This refactors `nix develop` internals a bit to use the `json` type
more. The assertion now operates in the in-memory json instead of
re-parsing it. While this is technically a weaker guarantee, we
should be able to rely on the library to get this right. It's its
most essential purpose.
2025-03-14 12:39:13 +00:00
Eelco Dolstra
762114b7c4 Log BuildResult 2025-03-13 19:42:52 +01:00
Eelco Dolstra
c515bc66f1 Provide a structured JSON serialisation of hashes 2025-03-13 18:52:29 +01:00
Eelco Dolstra
2a2af3f72f Logger::result(): Support logging arbitrary JSON objects 2025-03-13 18:39:46 +01:00
Sergei Zimmerman
adbd08399c {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```
2025-03-13 16:24:30 +00:00
Jörg Thalheim
e9af7a0749
Merge pull request #12647 from DeterminateSystems/lock-json-logger
JSONLogger: Acquire a lock to prevent log messages from clobbering each other
2025-03-13 16:43:09 +01:00
Eelco Dolstra
d0227f8d02 JSONLogger: Acquire a lock to prevent log messages from clobbering each other 2025-03-13 16:02:10 +01:00
Eelco Dolstra
220000dc1a makeJSONLogger(): Support logging to a Unix domain socket 2025-03-13 15:48:52 +01:00
Eelco Dolstra
d9730fc93b Fix fd check 2025-03-13 15:42:17 +01:00
Sergei Zimmerman
50123f2a56 libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}
2025-03-13 13:29:08 +00:00
Sergei Zimmerman
bf12aedf2e libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.
2025-03-13 13:29:08 +00:00
Sergei Zimmerman
a53b184e63 {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.
2025-03-13 13:29:08 +00:00
Eelco Dolstra
1efccf34b1 JSONLogger: Acquire a lock to prevent log messages from clobbering each other 2025-03-13 13:58:35 +01:00
Eelco Dolstra
29a9e638c1 Remove "@nix" prefix from json-log-path output 2025-03-13 13:37:38 +01:00
Eelco Dolstra
2972e73946 Turn NIX_LOG_FILE into a setting 2025-03-13 13:15:14 +01:00
Eelco Dolstra
1f702cdb01 Allow separate JSON logging
If the NIX_LOG_FILE environment variable is set, Nix will write JSON
log messages to that file in addition to the regular logger (e.g. the
progress bar).
2025-03-13 12:52:15 +01:00
Eelco Dolstra
8ef94c1114 Add a structured log message for FOD hash mismatches 2025-03-13 12:52:15 +01:00
Eelco Dolstra
1212b1fbfe JSONLogger: Log to a file descriptor instead of another Logger
Logging to another Logger was kind of nonsensical - it was really just
an easy way to get it to write its output to stderr, but that only
works if the underlying logger writes to stderr.

This change is needed to make it easy to log JSON output somewhere
else (like a file or socket).
2025-03-13 12:52:15 +01:00
John Ericson
b8eaf1b322
Merge pull request #12643 from obsidiansystems/delete-dead-goal-var
Remove unused parameter to the goal constructor
2025-03-12 16:45:26 -07:00
John Ericson
ecdcba27c5 Remove unused parameter to the goal constructor
It has been unused since 37fca662b0.
2025-03-12 19:09:54 -04:00
John Ericson
1055c9fd14
Merge pull request #12630 from L-as/me/clean-up-drv-goal
Clean up derivation goals a bit
2025-03-12 15:52:05 -07:00
Jörg Thalheim
af4c587ae3
Merge pull request #12596 from obsidiansystems/adv-attrs-organize
Advanced attributes organize
2025-03-12 23:21:48 +01:00
Jörg Thalheim
cba1a2155a
Merge pull request #12567 from obsidiansystems/slightly-rework-drv-resolution
Rework derivation input resolution
2025-03-12 23:15:45 +01:00
Jörg Thalheim
f4fd570ae3
Merge pull request #12642 from Mic92/shallow-clone
libfetchers/git: fix caching head when using shallow clones
2025-03-12 23:10:31 +01:00
John Ericson
dc0bc7f0a3 Make debug message more precise 2025-03-12 18:09:38 -04:00
John Ericson
99d0dd3a43 Simplify hook error status logic
The simplification here is due to a long-standing bug, but it is not
worth fixing the bug at this time. Instead we've finally written up an
issue for the bug, and referenced the issue number in the code.
2025-03-12 18:09:38 -04:00
John Ericson
06af9cb532 Inline the try-catch BuildError in the hook case
In the local building case, there is many things which can through
`BuildError`, but in the hook case there is just this one. We can
therefore simplify the code by "cinching" down the logic just to the
spot the error is thrown.

There is other code outside `libstore/build` which also uses
`BuildError`, but I believe those cases are mistakes. The point of
`BuildError` is the narrow technical use-cases of "errors which should
not be fatal with `--keep-going`". Using it outside the
building/scheduling code doesn't really make sense in that regard. It
seems likely that those usages were instead merely because "oh, this
error has something to do with building, so I guess `BuildError` is
better than `Error`".

It is quite likely that I myself used `BuildError` incorrectly as
described above :).
2025-03-12 18:09:38 -04:00
John Ericson
a39ed67180 Do no store timestamps in the build result in the build hook case
The variables are only set by CGroup mechanisms in `killSandbox` in the
local build. In the build hook case, these variables will not be set, so
there is nothing to do.
2025-03-12 18:09:38 -04:00
Las
db8439c328 Remove signRealisation from drv goal
We can move this method from `LocalStore` to `Store` --- even if we only
want the actual builder to sign things in many cases, there is no reason
to try to enforce this policy by spurious moving the method to a
subclass.

Now, we might technically sign class, but CA derivations is
experimental, and @Ericson2314 is going to revisit all this stuff with
issue #11896 anyways.
2025-03-12 18:09:38 -04:00
Las
0e7e1f5b57 Remove registerOutputs from drv goal
Easy to inline in one spot, and assert in the other.
2025-03-12 18:09:38 -04:00
Las
a87589a035 Simplify local drv goal a bit more
- `chrootParentDir` can be a local variable instead of a class variable.

- `getChildStatus` can be inlined. Again, we have the `assert(!hook);`
  in the local building case, which makes for a simpler thing inlined.
2025-03-12 18:09:38 -04:00
Las
4b521f14ac Remove privateNetwork variable from local drv goal
Can just inline its definition, it was immutable.
2025-03-12 18:09:38 -04:00
John Ericson
87824bca6b Avoid pointless mutation
The code that was in between is now gone. We can just set `st` correctly
the first time.
2025-03-12 18:08:58 -04:00
John Ericson
145aa2f118 Remove dead hook code in LocalDerivationGoal::tryLocalBuild
The `assert` above proves that `hook` is not set.
2025-03-12 18:08:10 -04:00
Las
75feeecd5d Start simplifying {Local,}DerivationGoal cleanup code
Thanks to the previous commit, we can inline all these small callbacks.
In the build-hook case, they were empty, and now they disappear
entirely.

While `LocalDerivationGoal` can be used in the hook case (we use it
based on whether we have a local store, not based on whether we are
using the build hook, a decision which comes later), the previous
commit's inline moved the code into a spot where we know we are cleaning
up after local building, *not* after running the build hook. This allows
for much more simplification.
2025-03-12 18:05:08 -04:00
Las
e87ba85705 Inline buildDone from DerivationGoal into use sites
The basic idea is that while we have duplicated this function, we now
have one call-site in the local build case, and one call site in the
build hook case. This unlocks big opportunities to specialize each copy,
since they really shouldn't be doing the same things. By the time we are
are done, there should not be much duplication left.

See #12628 for further info.
2025-03-12 18:00:07 -04:00
John Ericson
1de97bbe2e Factor out "last 10 log lines" error message code
This will help avoid duplication later. In particular, the next commit
will not need to duplicate as much.
2025-03-12 18:00:07 -04:00
John Ericson
637aa0944d Advanced attributes organize
This is supposed to firstly improve the docs as they are, and secondly
hint at how the core conceptual information ought to be moved to the
store derivation section of the manual.

Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2025-03-12 17:45:51 -04:00
Jörg Thalheim
e9f1761597 libfetchers/git: fix caching head when using shallow clones
the old code was using the wrong cache directory, which lead to a
warning on every nix flake update

Update src/libfetchers/git.cc
2025-03-12 22:32:05 +01:00
Robert Hensing
0068e225e0
Merge pull request #12639 from DeterminateSystems/remove-NIX_HELD_LOCKS
Remove use of $NIX_HELD_LOCKS
2025-03-12 21:10:08 +00:00
Eelco Dolstra
78eed85ba3 Remove use of $NIX_HELD_LOCKS
This variable was once used to communicate already acquired store path
locks between Nix and the build hook, but this hasn't been the case
since 9bcb4d2dd9. So let's get rid of
it.
2025-03-12 21:28:56 +01:00
Eelco Dolstra
9c5a8e9352
Merge pull request #12636 from Mic92/crash-handler
port crash-handler from lix to nix
2025-03-12 20:59:20 +01:00
Jörg Thalheim
cac1168afd
Merge pull request #12424 from ilya-bobyr/fish-profile-unify-ca-bundle.crt-search
nix-profile.fish: Look for ca-bundle.crt in $NIX_PROFILES
2025-03-12 17:07:57 +01:00
Graham Christensen
f988d9edbf
Merge pull request #12 from DeterminateSystems/update-upgrade-docs
Update upgrade docs
2025-03-12 13:23:41 +00:00
Luc Perkins
70f08ee91c
Merge pull request #11 from DeterminateSystems/revamp-introduction
Change title in introduction
2025-03-12 13:22:54 +00:00
Jörg Thalheim
c9a3101bff
Merge pull request #12603 from NaN-git/fix-curl-retry
libstore: curl retry: reset content-encoding and don't use string after move
2025-03-12 13:47:51 +01:00
Jade Lovelace
163f94412a port crash-handler from lix to nix
It was first introduced in 19e0ce2c03

In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.

Sample output:

Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]:  0# 0x000000000076876A in nix
                                   1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
                                   9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
                                  10# 0x000000000087A563 in nix
                                  11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  13# 0x00000000006F4DF5 in nix

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-03-12 10:23:02 +01:00
Dmitry Bogatov
affd9bbab7
Update doc/manual/source/protocols/store-path.md
Co-authored-by: John Ericson <git@JohnEricson.me>
2025-03-11 12:30:21 -04:00