There are two big changes:
1. Public and private config is now separated. Configuration variables
that are only used internally do not go in a header which is
installed.
(Additionally, libutil has a unix-specific private config header,
which should only be used in unix-specific code. This keeps things a
bit more organized, in a purely private implementation-internal way.)
2. Secondly, there is no more `-include`. There are very few config
items that need to be publically exposed, so now it is feasible to
just make the headers that need them just including the (public)
configuration header.
And there are also a few more small cleanups on top of those:
- The configuration files have better names.
- The few CPP variables that remain exposed in the public headers are
now also renamed to always start with `NIX_`. This ensures they should
not conflict with variables defined elsewhere.
- We now always use `#if` and not `#ifdef`/`#ifndef` for our
configuration variables, which helps avoid bugs by requiring that
variables must be defined in all cases.
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.
Today's `nixpkgs` example:
Before the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation contains an illegal reference specifier 'man'
After the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])
Perhaps more significantly, it no longer knows about
`LocalDerivationGoal`, and without any effort it also compiles on
Windows just fine. (`local-derivation-goal.{cc,hh}` is currently skipped
on Windows.)
We can move this method from `LocalStore` to `Store` --- even if we only
want the actual builder to sign things in many cases, there is no reason
to try to enforce this policy by spurious moving the method to a
subclass.
Now, we might technically sign class, but CA derivations is
experimental, and @Ericson2314 is going to revisit all this stuff with
issue #11896 anyways.
- `chrootParentDir` can be a local variable instead of a class variable.
- `getChildStatus` can be inlined. Again, we have the `assert(!hook);`
in the local building case, which makes for a simpler thing inlined.
Thanks to the previous commit, we can inline all these small callbacks.
In the build-hook case, they were empty, and now they disappear
entirely.
While `LocalDerivationGoal` can be used in the hook case (we use it
based on whether we have a local store, not based on whether we are
using the build hook, a decision which comes later), the previous
commit's inline moved the code into a spot where we know we are cleaning
up after local building, *not* after running the build hook. This allows
for much more simplification.
The basic idea is that while we have duplicated this function, we now
have one call-site in the local build case, and one call site in the
build hook case. This unlocks big opportunities to specialize each copy,
since they really shouldn't be doing the same things. By the time we are
are done, there should not be much duplication left.
See #12628 for further info.
Logging to another Logger was kind of nonsensical - it was really just
an easy way to get it to write its output to stderr, but that only
works if the underlying logger writes to stderr.
This change is needed to make it easy to log JSON output somewhere
else (like a file or socket).
This is a first step towards PR #10760, and the issues it addresses.
See the Doxygen for details.
Thanks to these changes, we are able to drastically restrict how the
rest of the code-base uses `ParseDerivation`.
Co-Authored-By: HaeNoe <git@haenoe.party>
"content-address*ed*" derivation is misleading because all derivations
are *themselves* content-addressed. What may or may not be
content-addressed is not derivation itself, but the *output* of the
derivation.
The outputs are not *part* of the derivation (for then the derivation
wouldn't be complete before we built it) but rather separate entities
produced by the derivation.
"content-adddress*ed*" is not correctly because it can only describe
what the derivation *is*, and that is not what we are trying to do.
"content-address*ing*" is correct because it describes what the
derivation *does* --- it produces content-addressed data.
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.
Looks like some cruft has been left over from previous refactorings.
This removes dead variables, which should not have side effects in their
constructors. In cases where the variable initialization has a purpose
[[maybe_unused]] is inserted to silence compiler warnings.
This works because the `builder` and `args` variables are only used
in the non-builtin code path.
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
Otherwise, if checkInterrupt() in any of the supported store operations
would catch onto a user interrupt, the exception would bubble to the thread
start and be handled by std::terminate(): a crash.
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)
Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory. The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.
This patch has been manually adapted from
14dc84ed03
Tested with:
$ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 16:57:50 after 1s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1
Now returns:
nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 17:05:48 after 0s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv':
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Caused by 1d3696f0fb
Without this fix the kept build directory is readable only by root
```
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5
drwx------ root root 60 B Wed Sep 11 00:09:48 2024 /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build
drwxr-xr-x nixbld1 nixbld 80 B Wed Sep 11 00:09:58 2024 /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build/
```
The daemon process is now moved into a new sub-cgroup called nix-daemon when the
daemon starts. This is necessary to abide by the no-processes-in-inner-nodes
rule, because the service cgroup becomes an inner node when the child cgroups
for the build are created (see LocalDerivationGoal::startBuilder()).
See #9675
This reverts commit 43e82c9446, reversing
changes made to d79b9bdec0.
Since /proc/homeless-shelter returns a different errno than /homeless-shelter (ENOENT vs EACCES), we need to revert this change.
Software depends on this error code i.e. cargo and therefore breaks.
this is only used to close non-stdio files in derivation sandboxes. we
may as well encode that in its name, drop the unnecessary integer set,
and use close_range to deal with the actual closing of files. not only
is this clearer, it also makes sandbox setup on linux fast by 1ms each
(cherry-picked and adapted from
c7d97802e4)
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
