1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-29 05:31:00 +01:00
Commit graph

9491 commits

Author SHA1 Message Date
Graham Dennis
89cadf5d64 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.

(cherry picked from commit 7fd0de38c6)
2024-03-06 17:56:26 -08:00
John Rinehart
5d534dc30f feat: show status bar with 'store copy-sigs' 2024-03-05 18:21:43 -08:00
Olmo Kramer
44ef603335 Accept multiple inputs in nix flake update
(cherry picked from commit 9f11b1b0c4)
2024-03-04 08:53:58 +00:00
John Ericson
5c1fa89f78 Make StoreConfig::getDefaultSystemFeatures a static method
This makes something in Hydra bit simpler. If someday the default
depends on the other config options, we can always change it back.

(cherry picked from commit a9e10a1dbd)
2024-01-25 16:58:00 +00:00
Eelco Dolstra
91e60868bd Use BackedStringView
(cherry picked from commit 1fe8f54bd3)
2024-01-19 10:14:17 +01:00
Eelco Dolstra
8bb4cb0565 Print a more helpful message if the daemon crashes
Instead of

   error: unexpected end-of-file

you now get

   error: Nix daemon disconnected unexpectedly (maybe it crashed?)

(cherry picked from commit a3cf27ca47)
2024-01-19 10:14:12 +01:00
Eelco Dolstra
fdf5313e7e copyStorePath(): Bail out early if the store path already exists
In rare cases (e.g. when using allowSubstitutes = false), it's
possible that we simultaneously have a DerivationGoal *and* a
SubstitutionGoal building the same path. So if a DerivationGoal
already built the path while the SubstitutionGoal was waiting for a
download slot, it saves us a superfluous download to exit early.

(cherry picked from commit dca0a80240)
2024-01-19 08:39:09 +00:00
Eelco Dolstra
cc94ea5a17 LocalStore::addToStore(): Ignore exceptions from parseDump()
In the "discard" case (i.e. when the store path already exists
locally), when we call parseDump() from a Finally and it throws an
exception (e.g. if the download of the NAR fails), Nix crashes:

   terminate called after throwing an instance of 'nix::SubstituteGone'
     what():  error: file 'nar/06br3254rx4gz4cvjzxlv028jrx80zg5i4jr62vjmn416dqihgr7.nar.xz' does not exist in binary cache 'http://localhost'
   Aborted (core dumped)

(cherry picked from commit a18d8d688a)
2024-01-19 08:39:09 +00:00
Eelco Dolstra
3cb2740721 Show what goal is waiting for a build slot
(cherry picked from commit ab786e22f1)
2024-01-19 08:39:09 +00:00
Shea Levy
2e4239f9e3
Merge branch '2.19-maintenance' into ifd-buildStore-2.19 2024-01-11 07:21:51 -05:00
Shea Levy
e7c2b35827
Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.

(cherry picked from commit c3942ef85f)
2024-01-11 06:34:27 -05:00
Eelco Dolstra
6af94c431b Make some more threads receive interrupts
Shouldn't hurt to do this. In particular, this should speed up
shutting down the PathSubstitutionGoal thread if it's copying from a
remote store.

(cherry picked from commit 295a2ff8bd)
2024-01-04 16:06:41 +00:00
Eelco Dolstra
4b38ebb009 withFramedSink(): Receive interrupts on the stderr thread
Otherwise Nix deadlocks when Ctrl-C is received in withFramedSink():
the parent thread will wait forever for the stderr thread to shut
down.

Fixes the hang reported in https://github.com/NixOS/nix/issues/7245#issuecomment-1770560923.

(cherry picked from commit 24e70489e5)
2024-01-04 16:06:41 +00:00
Robert Hensing
01cf57703a Revert "Add nix::isASCII*, locale-independent"
This reverts commit 79eb2920bb.

Not used at this time.

(cherry picked from commit 0b87ba50c0)
2023-12-13 21:09:33 +01:00
Robert Hensing
ebdb6926fd isValidSchemeName: Use regex
As requested by Eelco Dolstra. I think it used to be simpler.

(cherry picked from commit 4eaeda6604)
2023-12-13 21:09:31 +01:00
Robert Hensing
598b0e2317 schemeRegex -> schemeNameRegex
Scheme could be understood to include the typical `:` separator.

(cherry picked from commit 2e451a663e)
2023-12-13 21:09:22 +01:00
Robert Hensing
ffb6246650 allowed-uris: Match whole schemes also when scheme is not followed by slashes
(cherry picked from commit a05bc9eb92)
2023-12-13 21:09:20 +01:00
Robert Hensing
2116ee2454 isValidSchemeName: Add function
(cherry picked from commit d3a85b6834)
2023-12-13 21:08:13 +01:00
Robert Hensing
772a8efff4 Add nix::isASCII*, locale-independent
(cherry picked from commit 79eb2920bb)
2023-12-13 21:07:54 +01:00
Robert Hensing
4795569bf7 isAllowedURI: Format
(cherry picked from commit 1fa958dda1)
2023-12-13 21:07:54 +01:00
Robert Hensing
ec5e4041ba isAllowedURI: Remove incorrect note
(cherry picked from commit 6cbba914a7)
2023-12-13 21:07:54 +01:00
Robert Hensing
90c7904abf isAllowedURI: Extract function and test
(cherry picked from commit 91ba7b2307)
2023-12-13 21:07:50 +01:00
Fabian Möller
f72b0b5b00 Fix query parsing for path-like flakes
(cherry picked from commit f45d2ee2b7)
2023-12-12 18:08:08 +00:00
Robert Hensing
ae451e2247
Merge pull request #9595 from NixOS/backport-9588-to-2.19-maintenance
[Backport 2.19-maintenance] Give `Store::queryDerivationOutputMap` and `evalStore` argument
2023-12-11 18:13:58 +01:00
John Ericson
5fc116a620 Give Store::queryDerivationOutputMap and evalStore argument
Picking up where https://github.com/NixOS/nix/pull/9563 left off.

(cherry picked from commit 5f30c8acc7)
2023-12-11 16:16:40 +00:00
John Ericson
e6a03920ad Give Derivation::tryResolve an evalStore argument
*N.B. Backport is modified not to change any call sites / behavior.*

This is needed for building CA deriations with a src store / dest store
split. In particular it is needed for Hydra.

https://github.com/NixOS/hydra/issues/838 currently puts realizations,
and thus build outputs, in the local store, but it should not.

(cherry picked with modifications from commit 96dd757b0c)
2023-12-11 10:42:06 -05:00
Valentin Gagarin
9c42b2c954 reword description for the fetch-tree experimental feature
without knowing a lot of context, it's not clear who "we" are in that
text. I'm also strongly opposed to adding procedural notes into
a reference manual; it just won't age well.

this change leaves a factual description of the experimental feature and
its purpose.

(cherry picked from commit 3c6244b55e)
2023-12-04 16:56:46 +01:00
John Ericson
a61e42adb5 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

(cherry picked from commit 91b6833686)
2023-12-01 11:37:01 -05:00
Robert Hensing
28f0322307 libexpr/local.mk: Make eval compile deps regular
Dependency is now entirely through the eval.cc rule.
All gen.hh deps are now there.

(cherry picked from commit 68c48756fe)
2023-11-27 16:26:12 +00:00
Sergei Trofimovich
94b2401138 libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
Without the change build for `eval.o` fails occasionally as:

    $ make src/libexpr/eval.o
      GEN    Makefile.config
      GEN    src/libexpr/primops/derivation.nix.gen.hh
      GEN    src/libexpr/fetchurl.nix.gen.hh
      GEN    src/libexpr/parser-tab.cc
      GEN    src/libexpr/lexer-tab.cc
    src/libexpr/lexer.l:314: warning, -s option given but default rule can be matched
      CXX    src/libexpr/eval.o
    src/libexpr/eval.cc:519:18: fatal error: flake/call-flake.nix.gen.hh: No such file or directory
      519 |         #include "flake/call-flake.nix.gen.hh"
          |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    make: *** [mk/patterns.mk:3: src/libexpr/eval.o] Error 1

Noticed in https://github.com/NixOS/nixpkgs/pull/269439

(cherry picked from commit 75134b7513)
2023-11-27 16:26:12 +00:00
Valentin Gagarin
92f3598a16 add deprecation warnings in documentation
this is hacky, but can serve as a stopgap until we can do it
programmatically.

(cherry picked from commit 7e08bdefcc)
2023-11-27 14:08:16 +01:00
Moritz Angermann
819eda4615 nix flake update add deprecation warnings.
This builds on #8817, to add additional UX help for people with existing
muscle memory (or shell history) with --update-input and tries to gently
guide them towards the newly evolved CLI UI.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
(cherry picked from commit af00298587)
2023-11-27 14:07:26 +01:00
John Ericson
914309c35d Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).

(cherry picked from commit 46131567da)
2023-11-22 04:48:14 +00:00
roblabla
a5c6ba3edc Fix bad_format_string error when builder stdout contains %
(cherry picked from commit e2b6821ca0)
2023-11-20 16:22:58 +00:00
DavHau
2778b218c3 fetchTree: clarify docs for shallow flag
(cherry picked from commit 796a7eb92d)
2023-11-20 14:16:08 +00:00
Eelco Dolstra
1d6abec993 Revert use of boost::container::small_vector in the evaluator
It caused random crashes (https://hydra.nixos.org/build/241514506,
https://hydra.nixos.org/build/241443330) because the heap allocation
done by small_vector in the not-small case is not scanned for GC
roots.
2023-11-20 12:35:35 +01:00
tomberek
fb68699456
Merge pull request #9370 from hercules-ci/add-value-types
refactor: Add `Value` types, use `std::span` for list iteration
2023-11-20 01:32:32 -05:00
nicoo
d5928085d5 builtins.concatMap: Fix typo in error message 2023-11-19 19:57:07 +01:00
John Ericson
c4a74d6dcc
Merge pull request #9382 from tweag/filterSource-link
doc: Add link to filterSource from path
2023-11-19 00:06:18 -05:00
Silvan Mosberger
70ddf298e0 doc: Add link to filterSource from path 2023-11-19 04:09:14 +01:00
Robert Hensing
251fb23aea Shebang parser: add virtual destructor
Fixes:

    warning: destructor called on non-final 'nix::ParseUnquoted' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
2023-11-19 01:48:08 +01:00
Robert Hensing
1d86bb4f70
Merge pull request #9373 from obsidiansystems/fix-make-check
Fix `make check`
2023-11-17 19:09:38 +01:00
Robert Hensing
4a539ac3ea Fix buildNoGc
Fixes https://hydra.nixos.org/build/241067941/nixlog/1

    src/libexpr/eval.cc:1776:54: error: variable 'boost::container::small_vector<nix::Value*, 4> vArgs' has initializer but incomplete type
2023-11-17 17:38:08 +01:00
John Ericson
293ae59257 Fix make check
After 9c7749e135, `libutil-tests_RUN`
doesn't exist. It needs to become `libutil-tests-exe_RUN`.
2023-11-17 11:26:45 -05:00
Théophane Hufschmitt
b1e7d7cad6
Merge pull request #9280 from R-VdP/rvdp/fix_remote_logging_phase_reporting
Include phase reporting in log file for ssh-ng builds
2023-11-17 14:37:09 +01:00
Eelco Dolstra
3a7f024a9c
Merge pull request #9363 from edolstra/symlink-regression
Fix symlink handling
2023-11-17 14:11:50 +01:00
Robert Hensing
121665f377 nix-env: Use state.mkList, required for correct stats 2023-11-17 10:23:32 +01:00
Robert Hensing
260c614762 Value: use std::span, change use of const
**`Value` and `const`**

These two deserve some explanation. We'll get to lists later.

Values can normally be thought of as immutable, except they are
are also the vehicle for call by need, which must be implemented
using mutation.

This circumstance makes a `const Value` a rather useless thing:

 - If it's a thunk, you can't evaluate it, except by copying, but
   that would not be call by need.

 - If it's not a thunk, you know the type, so the method that
   acquired it for you should have returned something more specific,
   such as a `const Bindings &` (which actually does make sense
   because that's an immutable span of pointers to mutable `Value`s.

 - If you don't care about the type yet, you might establish the
   convention that `const Value` means `deepSeq`-ed data, but
   this is hardly useful and not actually as safe as you would
   supposedly want to trust it to be - just convention.

**Lists**

`std::span` is a tuple of pointer and size - just what we need.

We don't return them as `const Value`, because considering the
first bullet point we discussed before, we'd have to force all
the list values, which isn't what we want.

So what we end up with is a nice representation of a list in
weak head normal form: the spine is immutable, but the
items may need some evaluation later.
2023-11-17 10:19:03 +01:00
Robert Hensing
7055c65285 Value: extract Value::Lambda 2023-11-17 10:19:03 +01:00
Robert Hensing
6af1d9f7b9 Value: extract Value::FunctionApplicationThunk 2023-11-17 10:19:03 +01:00