nixos/nix
1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-23 02:39:37 +01:00
Code Activity

Make the Derived Path family of types inductive for dynamic derivations

Browse source 
We want to be able to write down `foo.drv^bar.drv^baz`:
`foo.drv^bar.drv` is the dynamic derivation (since it is itself a
derivation output, `bar.drv` from `foo.drv`).

To that end, we create `Single{Derivation,BuiltPath}` types, that are
very similar except instead of having multiple outputs (in a set or
map), they have a single one. This is for everything to the left of the
rightmost `^`.

`NixStringContextElem` has an analogous change, and now can reuse
`SingleDerivedPath` at the top level. In fact, if we ever get rid of
`DrvDeep`, `NixStringContextElem` could be replaced with
`SingleDerivedPath` entirely!

Important note: some JSON formats have changed.

We already can *produce* dynamic derivations, but we can't refer to them
directly. Today, we can merely express building or example at the top
imperatively over time by building `foo.drv^bar.drv`, and then with a
second nix invocation doing `<result-from-first>^baz`, but this is not
declarative. The ethos of Nix of being able to write down the full plan
everything you want to do, and then execute than plan with a single
command, and for that we need the new inductive form of these types.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
This commit is contained in:
John Ericson 2023-01-15 17:39:04 -05:00
parent d00fe5f225
commit 60b7121d2c
48 changed files with 1136 additions and 267 deletions
Download patch file Download diff file Expand all files Collapse all files

222
src/libstore/derived-path.hh
View file

@ -24,28 +24,37 @@ class Store;
struct DerivedPathOpaque {
StorePath path;
nlohmann::json toJSON(ref<Store> store) const;
std::string to_string(const Store & store) const;
static DerivedPathOpaque parse(const Store & store, std::string_view);
nlohmann::json toJSON(const Store & store) const;
GENERATE_CMP(DerivedPathOpaque, me->path);
};
struct SingleDerivedPath;
/**
* A derived path that is built from a derivation
* A single derived path that is built from a derivation
*
* Built derived paths are pair of a derivation and some output names.
* They are evaluated by building the derivation, and then replacing the
* output names with the resulting outputs.
*
* Note that does mean a derived store paths evaluates to multiple
* opaque paths, which is sort of icky as expressions are supposed to
* evaluate to single values. Perhaps this should have just a single
* output name.
* Built derived paths are pair of a derivation and an output name. They are
* evaluated by building the derivation, and then taking the resulting output
* path of the given output name.
*/
struct DerivedPathBuilt {
StorePath drvPath;
OutputsSpec outputs;
struct SingleDerivedPathBuilt {
ref<SingleDerivedPath> drvPath;
std::string output;
/**
* Get the store path this is ultimately derived from (by realising
* and projecting outputs).
*
* Note that this is *not* a property of the store object being
* referred to, but just of this path --- how we happened to be
* referring to that store object. In other words, this means this
* function breaks "referential transparency". It should therefore
* be used only with great care.
*/
const StorePath & getBaseStorePath() const;
/**
* Uses `^` as the separator
@ -57,11 +66,139 @@ struct DerivedPathBuilt {
std::string to_string_legacy(const Store & store) const;
/**
* The caller splits on the separator, so it works for both variants.
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static DerivedPathBuilt parse(const Store & store, std::string_view drvPath, std::string_view outputs);
nlohmann::json toJSON(ref<Store> store) const;
static SingleDerivedPathBuilt parse(
const Store & store, ref<SingleDerivedPath> drvPath,
std::string_view outputs,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
nlohmann::json toJSON(Store & store) const;
GENERATE_CMP(DerivedPathBuilt, me->drvPath, me->outputs);
DECLARE_CMP(SingleDerivedPathBuilt);
};
using _SingleDerivedPathRaw = std::variant<
DerivedPathOpaque,
SingleDerivedPathBuilt
>;
/**
* A "derived path" is a very simple sort of expression (not a Nix
* language expression! But an expression in a the general sense) that
* evaluates to (concrete) store path. It is either:
*
* - opaque, in which case it is just a concrete store path with
* possibly no known derivation
*
* - built, in which case it is a pair of a derivation path and an
* output name.
*/
struct SingleDerivedPath : _SingleDerivedPathRaw {
using Raw = _SingleDerivedPathRaw;
using Raw::Raw;
using Opaque = DerivedPathOpaque;
using Built = SingleDerivedPathBuilt;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
/**
* Get the store path this is ultimately derived from (by realising
* and projecting outputs).
*
* Note that this is *not* a property of the store object being
* referred to, but just of this path --- how we happened to be
* referring to that store object. In other words, this means this
* function breaks "referential transparency". It should therefore
* be used only with great care.
*/
const StorePath & getBaseStorePath() const;
/**
* Uses `^` as the separator
*/
std::string to_string(const Store & store) const;
/**
* Uses `!` as the separator
*/
std::string to_string_legacy(const Store & store) const;
/**
* Uses `^` as the separator
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static SingleDerivedPath parse(
const Store & store,
std::string_view,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Uses `!` as the separator
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static SingleDerivedPath parseLegacy(
const Store & store,
std::string_view,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
nlohmann::json toJSON(Store & store) const;
};
static inline ref<SingleDerivedPath> makeConstantStorePathRef(StorePath drvPath)
{
return make_ref<SingleDerivedPath>(SingleDerivedPath::Opaque { drvPath });
}
/**
* A set of derived paths that are built from a derivation
*
* Built derived paths are pair of a derivation and some output names.
* They are evaluated by building the derivation, and then replacing the
* output names with the resulting outputs.
*
* Note that does mean a derived store paths evaluates to multiple
* opaque paths, which is sort of icky as expressions are supposed to
* evaluate to single values. Perhaps this should have just a single
* output name.
*/
struct DerivedPathBuilt {
ref<SingleDerivedPath> drvPath;
OutputsSpec outputs;
/**
* Get the store path this is ultimately derived from (by realising
* and projecting outputs).
*
* Note that this is *not* a property of the store object being
* referred to, but just of this path --- how we happened to be
* referring to that store object. In other words, this means this
* function breaks "referential transparency". It should therefore
* be used only with great care.
*/
const StorePath & getBaseStorePath() const;
/**
* Uses `^` as the separator
*/
std::string to_string(const Store & store) const;
/**
* Uses `!` as the separator
*/
std::string to_string_legacy(const Store & store) const;
/**
* The caller splits on the separator, so it works for both variants.
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static DerivedPathBuilt parse(
const Store & store, ref<SingleDerivedPath>,
std::string_view,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
nlohmann::json toJSON(Store & store) const;
DECLARE_CMP(DerivedPathBuilt);
};
using _DerivedPathRaw = std::variant<
@ -71,13 +208,13 @@ using _DerivedPathRaw = std::variant<
/**
* A "derived path" is a very simple sort of expression that evaluates
* to (concrete) store path. It is either:
* to one or more (concrete) store paths. It is either:
*
* - opaque, in which case it is just a concrete store path with
* - opaque, in which case it is just a single concrete store path with
* possibly no known derivation
*
* - built, in which case it is a pair of a derivation path and an
* output name.
* - built, in which case it is a pair of a derivation path and some
* output names.
*/
struct DerivedPath : _DerivedPathRaw {
using Raw = _DerivedPathRaw;
@ -90,6 +227,18 @@ struct DerivedPath : _DerivedPathRaw {
return static_cast<const Raw &>(*this);
}
/**
* Get the store path this is ultimately derived from (by realising
* and projecting outputs).
*
* Note that this is *not* a property of the store object being
* referred to, but just of this path --- how we happened to be
* referring to that store object. In other words, this means this
* function breaks "referential transparency". It should therefore
* be used only with great care.
*/
const StorePath & getBaseStorePath() const;
/**
* Uses `^` as the separator
*/
@ -100,14 +249,43 @@ struct DerivedPath : _DerivedPathRaw {
std::string to_string_legacy(const Store & store) const;
/**
* Uses `^` as the separator
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static DerivedPath parse(const Store & store, std::string_view);
static DerivedPath parse(
const Store & store,
std::string_view,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Uses `!` as the separator
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
static DerivedPath parseLegacy(const Store & store, std::string_view);
static DerivedPath parseLegacy(
const Store & store,
std::string_view,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Convert a `SingleDerivedPath` to a `DerivedPath`.
*/
static DerivedPath fromSingle(const SingleDerivedPath &);
nlohmann::json toJSON(Store & store) const;
};
typedef std::vector<DerivedPath> DerivedPaths;
/**
* Used by various parser functions to require experimental features as
* needed.
*
* Somewhat unfortunate this cannot just be an implementation detail for
* this module.
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*/
void drvRequireExperiment(
const SingleDerivedPath & drv,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
}