We can always compute the store path from the content hash, but not
vice versa. Storing the content hash allows `hashPath()` to be
replaced by `fetchToStore(...FetchMode::DryRun...)`, which gets us
caching in lazy-trees mode.
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
(cherry picked from commit cc24766fa6)
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
(cherry picked from commit f3e1c47f47)
The ability to substitute inputs was removed in #10612 because it was
broken: with user-specified inputs containing a `narHash` attribute,
substitution resulted in an input that lacked the attributes returned
by the real fetcher (such as `lastModified`).
To fix this, we introduce a new input attribute `final`. If `final =
true`, fetching the input cannot add or change any attributes.
We only attempt to substitute inputs that have `final = true`. This is
implied by lock file entries; we only write a lock file if all its
entries are "final".
The user can specified `final = true` in `fetchTree`, in which case it
is their responsibility to ensure that all attributes returned by the
fetcher are included in the `fetchTree` call. For example,
nix eval --impure --expr 'builtins.fetchTree { type = "github"; owner = "NixOS"; repo = "patchelf"; final = true; narHash = "sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM="; }'
succeeds in a store path with the specified NAR hash exists or is
substitutable, but fails with
error: fetching final input '{"final":true,"narHash":"sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM=","owner":"NixOS","repo":"patchelf","type":"github"}' resulted in different input '{"final":true,"lastModified":1718457448,"narHash":"sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM=","owner":"NixOS","repo":"patchelf","rev":"a0f54334df36770b335c051e540ba40afcbf8378","type":"github"}'
Progress on #5638
There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.
Continuing process pioneered in
52bfccf8d8.
Subflakes are flakes in the same tree, accessed in flake inputs via
relative paths (e.g. `inputs.foo.url = "path:./subdir"`). Previously
these didn't work very well because they would be separately copied to
the store, which is inefficient and makes references to parent
directories tricky or impossible. Furthermore, they had their own NAR
hash in the lock file, which is superfluous since the parent is
already locked.
Now subflakes are accessed via the accessor of the calling flake. This
avoids the unnecessary copy and makes it possible for subflakes to
depend on flakes in a parent directory (so long as they're in the same
tree).
Lock file nodes for relative flake inputs now have a new `parent` field:
{
"locked": {
"path": "./subdir",
"type": "path"
},
"original": {
"path": "./subdir",
"type": "path"
},
"parent": [
"foo",
"bar"
]
}
which denotes that `./subdir` is to be interpreted relative to the
directory of the `bar` input of the `foo` input of the root flake.
Extracted from the lazy-trees branch.
After the removal of the InputAccessor::fetchToStore() method, the
only remaining functionality in InputAccessor was `fingerprint` and
`getLastModified()`, and there is no reason to keep those in a
separate class.
This wasn't caught by CI because #10149 and #10152 pass
individually... It doesn't happen on lazy-trees either because we
never try to fetch relative path flakes (#10089).
* Convert all InputScheme::fetch() methods to getAccessor().
* Add checkLocks() method for checking lock attributes.
* Rename fetch() to fetchToStore().
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).
Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.
The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
We now have `schemeName` and `allowedAttrs` functions for this purpose.
We look up the schema with the former; we restrict the set of input
attributes with the latter.
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.
Special-casing the file name is rather ugly, so we shouldn't do
that. So now any {file,http,https} URL is handled by
TarballInputScheme, except for non-flake inputs (i.e. inputs that have
the attribute `flake = false`).
When importing e.g. a local `nixpkgs` in a flake to test a change like
{
inputs.nixpkgs.url = path:/home/ma27/Projects/nixpkgs;
outputs = /* ... */
}
then the input is missing a `lastModified`-field that's e.g. used in
`nixpkgs.lib.nixosSystem`. Due to the missing `lastMoified`-field, the
mtime is set to 19700101:
result -> /nix/store/b7dg1lmmsill2rsgyv2w7b6cnmixkvc1-nixos-system-nixos-22.05.19700101.dirty
With this change, the `path`-fetcher now sets a `lastModified` attribute
to the `mtime` just like it's the case in the `tarball`-fetcher already.
When building NixOS systems with `nixpkgs` being a `path`-input and this
patch, the output-path now looks like this:
result -> /nix/store/ld2qf9c1s98dxmiwcaq5vn9k5ylzrm1s-nixos-system-nixos-22.05.20220217.dirty
- Previous to this commit the boundary was exclusive of the
top level flake.
- This is wrong since the top level flake is still a valid
relative reference.
- Now, the check boundary is inclusive of the top level flake.
Signed-off-by: Timothy DeHerrera <tim.deh@pm.me>
