This method does *not* create a new type of goal. We instead just make
`DerivationGoal` more sophisticated, which is much easier to do now that
`DerivationBuildingGoal` has been split from it (and so many fields are
gone, or or local variables instead).
This avoids the need for a secondarily trampoline goal that interacted
poorly with `addWantedOutputs`. That, I hope, will mean the bugs from
before do not reappear.
There may in fact be a reason to introduce such a trampoline in the
future, but it would only happen in conjunction with getting rid of
`addWantedOutputs`.
Restores the functionality (and tests) that was reverted in
f4f28cdd0e.
Cherry-pick https://gerrit.lix.systems/c/lix/+/2100
This change fixes a potential concurrency failure when accessing random
which is not thread safe.
Co-authored-by: Lily Ballard <lily@ballards.net>
Try to make `DerivationGoal` care less whether we're working from an
in-memory derivation or not.
It's a clean-up in its own right, but it will also help with other
cleanups under the umbrella of #12628.
Now, each class provides the initial coroutine by value. This avoids
some sketchy virtual function stuff, and will also be further put to
good use in the next commit.
As summarized in
https://github.com/NixOS/nix/issues/77#issuecomment-2843228280 the
motivation is that the complicated retry logic this introduced was
making the cleanup task #12628 harder to accomplish. It was not easy to
ascertain just what policy / semantics the extra control-flow was
implementing, in order to figure out a different way to implementing it
either.
After talking to Eelco about it, he decided we could just....get rid of
the feature entirely! It's a bit scary removing a decade+ old feature,
but I think he is right. See the release notes for more explanation.
This reverts commit 299141ecbd.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
The existing header is a bit too big. Now the following use-cases are
separated, and get their own headers:
- Using or implementing an arbitrary store: remaining `store-api.hh`
This is closer to just being about the `Store` (and `StoreConfig`)
classes, as one would expect.
- Opening a store from a textual description: `store-open.hh`
Opening an aribtrary store implementation like this requires some sort
of store registration mechanism to exists, but the caller doesn't need
to know how it works. This just exposes the functions which use such a
mechanism, without exposing the mechanism itself
- Registering a store implementation: `store-registration.hh`
This requires understanding how the mechanism actually works, and the
mechanism in question involves templated machinery in headers we
rather not expose to things that don't need it, as it would slow down
compilation for no reason.
I split it out before to try to separate the building logic, but now we
have the much better `DerivationBuilder` abstraction for that. With that
change, I think `LocalDerivationGoal` has outlived its usefulness.
We just inline it back into `DerivationGoal`, and do so with minimal
`#ifdef` for Windows.
Note that the order of statements in `~DerivationGoal` is different than
it was after the `~LocalDerivationGoal` split, but it is *restored* to
the way it original was before --- evidently I did the split slightly
wrong, but nobody noticed, probably because the order doesn't actually
matter.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Only a much smaller `StructuredAttrs` remains, the rest is is now moved
to `DerivationOptions`.
This gets us quite close to `std::optional<StructuredAttrs>` and
`DerivationOptions` being included in `Derivation` as fields.
It is just use for adding context to errors, but we have `addTrace` to
do that. Let the callers do that instead.
The callers doing so is a bit duplicated, yes, but this will get better
once `DerivationOptions` is included in `Derivation`.
Rather than "mounting" the store inside an empty virtual filesystem,
just return the store as a virtual filesystem. This is more modular.
(FWIW, it also supports two long term hopes of mind:
1. More capability-based Nix language mode. I dream of a "super pure
eval" where you can only use relative path literals (See #8738), and
any `fetchTree`-fetched stuff + the store are all disjoint (none is
mounted in another) file systems.
2. Windows, where the store dir may include drive letters, etc., and is
thus unsuitable to be the prefix of any `CanonPath`s.
)
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
Instead of calling `worker.waitForAWhile(shared_from_this())` etc.,
the subclasses of Goal instead call protected functions defined in Goal
that abstract over these.
The code for awaiting has also been heavily simplified.
Instead of calling `addWaitee`, then suspending,
`co_await await(waitees)` is called once, which also handles the suspend.
The end-goal is to remove all manual `co_await Suspend{}`s.
Now that we have coroutines, we can go back to loops and regular RAII,
which is must less error-proone!
I look forward to removing the other instances!
The simplification here is due to a long-standing bug, but it is not
worth fixing the bug at this time. Instead we've finally written up an
issue for the bug, and referenced the issue number in the code.
In the local building case, there is many things which can through
`BuildError`, but in the hook case there is just this one. We can
therefore simplify the code by "cinching" down the logic just to the
spot the error is thrown.
There is other code outside `libstore/build` which also uses
`BuildError`, but I believe those cases are mistakes. The point of
`BuildError` is the narrow technical use-cases of "errors which should
not be fatal with `--keep-going`". Using it outside the
building/scheduling code doesn't really make sense in that regard. It
seems likely that those usages were instead merely because "oh, this
error has something to do with building, so I guess `BuildError` is
better than `Error`".
It is quite likely that I myself used `BuildError` incorrectly as
described above :).
The variables are only set by CGroup mechanisms in `killSandbox` in the
local build. In the build hook case, these variables will not be set, so
there is nothing to do.
We can move this method from `LocalStore` to `Store` --- even if we only
want the actual builder to sign things in many cases, there is no reason
to try to enforce this policy by spurious moving the method to a
subclass.
Now, we might technically sign class, but CA derivations is
experimental, and @Ericson2314 is going to revisit all this stuff with
issue #11896 anyways.
- `chrootParentDir` can be a local variable instead of a class variable.
- `getChildStatus` can be inlined. Again, we have the `assert(!hook);`
in the local building case, which makes for a simpler thing inlined.
Thanks to the previous commit, we can inline all these small callbacks.
In the build-hook case, they were empty, and now they disappear
entirely.
While `LocalDerivationGoal` can be used in the hook case (we use it
based on whether we have a local store, not based on whether we are
using the build hook, a decision which comes later), the previous
commit's inline moved the code into a spot where we know we are cleaning
up after local building, *not* after running the build hook. This allows
for much more simplification.
The basic idea is that while we have duplicated this function, we now
have one call-site in the local build case, and one call site in the
build hook case. This unlocks big opportunities to specialize each copy,
since they really shouldn't be doing the same things. By the time we are
are done, there should not be much duplication left.
See #12628 for further info.
I refactored the way that input resolution works in `DerivationGoal`. To
be honest, it is probably unclear to the reader whether this new way is
better or worse. I suppose *intrinsic* motivation, I can say that
- the more structured use of `inputGoal` (a local variable) is better
than the shotgrun approach with `inputDrvOutputs`
- A virtual `waiteeDone` was a hack, and now it's gone.
However, the *real* motivation of this is not the above things, but that
it is needed for my mammoth refactor fixing #11897 and #11928.
It is nice that this step could come first, rather than making that
refactor even bigger.
