1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-08 19:46:02 +01:00
Commit graph

20975 commits

Author SHA1 Message Date
John Ericson
9bc218ca3f Add new C API for working with derivations
Also test the APIs we just added.
2025-09-16 13:25:36 -04:00
Jörg Thalheim
1a69fc6ab5
Merge pull request #14001 from juhp/common-soversion
meson: refactor nix_soversion into nix-meson-build-support/common
2025-09-16 08:50:05 +02:00
Jens Petersen
86bb7c958a meson: refactor nix_soversion into nix-meson-build-support/common
This is a follow-on to #13995 which added soversion to the libraries
2025-09-16 12:54:30 +08:00
Jörg Thalheim
5e17a3f81c
Merge pull request #13998 from Mic92/fast-flake-check
nix flake check: Skip substitutable derivations
2025-09-15 21:44:11 +02:00
Seth Flynn
ecdda5798c nix flake check: Skip substitutable derivations
Since `nix flake check` doesn't produce a `result` symlink, it doesn't
actually need to build/substitute derivations that are already known
to have succeeded, i.e. that are substitutable.

This can speed up CI jobs in cases where the derivations have already
been built by other jobs. For instance, a command like

  nix flake check github:NixOS/hydra/aa62c7f7db31753f0cde690f8654dd1907fc0ce2

should no longer build anything because the outputs are already in
cache.nixos.org.

Based-on: https://github.com/DeterminateSystems/nix-src/pull/134
Based-on: https://gerrit.lix.systems/c/lix/+/3841
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2025-09-15 21:31:03 +02:00
Jörg Thalheim
0b401e2199
Merge pull request #13995 from juhp/soversion
meson: add soversion with nix version to give SONAME to libs
2025-09-15 20:36:31 +02:00
Eelco Dolstra
ad175727e4
Merge pull request #13938 from NixOS/import-thunk
Ensure that files are parsed/evaluated only once
2025-09-15 19:03:18 +02:00
John Ericson
d314750174
Merge pull request #13999 from obsidiansystems/more-get-improvements
More `get` / `getOr` improvements
2025-09-15 12:55:40 -04:00
John Ericson
f3e3f75838 More get / getOr improvements
- Use `const K`, not `K`, otherwise we don't get auto referencing of
  rvalues.

- Generalized the deleted overloads, because we don't care what the key
  type is --- we want to get rid of anything that has an rvalue map
  type.
2025-09-15 12:25:37 -04:00
Jens Petersen
dd1a554aba meson: add soversion with nix version to give SONAME to libs (#13960, #13979)
remove 'pre' version suffix for non-releases (chokes Darwin ld)
2025-09-15 22:08:26 +08:00
Sergei Zimmerman
2b0fd88324
Merge pull request #13991 from xokdvium/bindings-remove-find
libexpr: Remove Bindings::find
2025-09-14 21:32:31 +00:00
John Ericson
ffc14ac91b
Merge pull request #13983 from xokdvium/bindings-fixes
libexpr: Make Bindings::iterator a proper strong type instead of pointer
2025-09-14 17:31:48 -04:00
Sergei Zimmerman
d830840433
libexpr: Remove Bindings::find
A follow-up optimization will make it impossible to make a find function
that returns an iterator in an efficient manner. All consumer code can
easily use the `get` variant.
2025-09-14 23:29:44 +03:00
Sergei Zimmerman
ddabd94f82
libexpr: Make Bindings::iterator a proper strong type instead of pointer
As evident from the number of tests that were holding this API completely
wrong (the end() iterator returned from find() is NEVER nullptr) we should
not have this footgun. A proper strong type guarantees that this confusion
will not happen again.

Also this will be helpful down the road when Bindings becomes something
smarter than an array of Attr.
2025-09-14 22:52:37 +03:00
Sergei Zimmerman
b974b7dc1e
Merge pull request #13985 from dramforever/master 2025-09-14 11:19:10 +00:00
dramforever
7295034362 libstore: Raise default connect-timeout to 15 secs
This allows the weird network or DNS server fallback mechanism inside
glibc to work, and prevents a "Resolving timed out after 5000
milliseconds" error. Read on for details.

The DNS request stuff (dns-hosts) in glibc uses this fallback procedure
to minimize network RTT in the ideal case while dealing with
ill-behaving networks and DNS servers gracefully (see resolv.conf(5)):

- Use sendmmsg() to send UDP DNS requests for IPv4 and IPv6 in parallel
- If that times out (meaning that none or only one of the responses have
  been received), send the requests one by one, waiting for the response
  before sending the next request ("single-request")
- If that still times out, try to use a different socket (hence
  different address) for each request ("single-request-reopen")

The default timeout inside glibc is 5 seconds. Therefore, setting
connect-timeout, and therefore CURLOPT_CONNECTTIMEOUT to 5 seconds
prevents the single-request fallback, and setting it to even 10 seconds
prevents the single-request-reopen fallback as well.

The fallback decision is saved by glibc, but only thread-locally, and
libcurl starts a new thread for getaddrinfo() for each connection.
Therefore for every connection the fallback starts from sendmmsg() all
over again. And since these are considered to have timed out by libcurl,
even though getaddrinfo() might return a successful result, it is not
cached in libcurl.

While a user could tweak these with resolv.conf(5) options (e.g. using
networking.resolvconf.extraOptions in NixOS), and indeed that is
probably needed to avoid annoying delays, it still means that the
default connect-timeout of 5 is too low. Raise it to give fallback a
chance.
2025-09-14 08:48:29 +08:00
John Ericson
5bc96798b1
Merge pull request #13982 from obsidiansystems/path-info-static-function
`ValidPathInfo`, `NarInfo`, turn funky constructor into static method
2025-09-13 20:10:45 -04:00
Sergei Zimmerman
e75501da3e
libexpr: Remove non-const iterators of Bindings 2025-09-13 23:21:24 +03:00
John Ericson
74be28820c ValidPathInfo, NarInfo, turn funky constructor into static method
This is more flexible, and needed for me to be able to reshuffle the
inheritance bureaucracy to make the JSON instances more precise.
2025-09-13 13:17:14 -04:00
Jörg Thalheim
465d627f7f
Merge pull request #13978 from xokdvium/fix-warn
libutil: Fix missing return warning
2025-09-13 10:30:32 +02:00
Sergei Zimmerman
298ea97c12
libutil: Fix missing return warning
../hash.cc: In function 'nix::{anonymous}::DecodeNamePair nix::baseExplicit(HashFormat)':
../hash.cc:114:1: warning: control reaches end of non-void function [-Wreturn-type]
  114 | }
      | ^
2025-09-13 09:19:07 +03:00
John Ericson
1907a3300f
Merge pull request #13811 from hgl/patch-2
doc: Rephrase store-object.md
2025-09-12 23:34:52 -04:00
John Ericson
1710fd09f3
Merge pull request #13975 from obsidiansystems/prep-json-0
Misc prep changes for #13942
2025-09-12 23:32:20 -04:00
Glen Huang
a0b633dd2b doc: Rephrase store-object.md 2025-09-13 10:17:12 +08:00
John Ericson
f78062d2fb
Merge pull request #13976 from xokdvium/darwin-packaging
packaging: Drop legacy apple sdk pattern
2025-09-12 18:37:36 -04:00
Sergei Zimmerman
20b532eab0
packaging: Drop legacy apple sdk pattern
This has been dropped on unstable an nix no longer
compiled with overridden nixpkgs input. On 25.05 these
overrides already do nothing.

Tested with:

nix build .#packages.x86_64-darwin.nix-cli -L --override-input nixpkgs https://releases.nixos.org/nixos/unstable/nixos-25.11pre859555.ab0f3607a6c7/nixexprs.tar.xz

Default deployment target on 25.05 is 11.3, so 10.13
sdk override doesn't have to be updated at all as evident
from the fact that we didn't observe any issues with it.
2025-09-13 01:07:42 +03:00
John Ericson
095ac66d4c Introduce Hash::parseExplicitFormatUnprefixed 2025-09-12 18:04:29 -04:00
John Ericson
c6d06ce486 Fix hash error message
Wrong number of arguments was causing a format assertion.
2025-09-12 18:04:29 -04:00
John Ericson
c242706319 Move json_avoids_null to its own header
This is because we need it in declarations where we should not be
including the full `nlohmann/json.hpp`.

Already can clean up by moving the experimental feature "instance".

Also, make the `std::map` instance better by allowing for other
comparison functions.
2025-09-12 18:04:29 -04:00
Philip Wilk
aef431fbd1
bugfix/3514: do not throw on substituter errors if other substituters are still enabled (#13301)
## Motivation

Nix currently hard fails if a substituter is inaccessible, even when they are other substituters available, unless `fallback = true`. 
This breaks nix build, run, shell et al entirely. 
This would modify the default behaviour so that nix would actually use the other available substituters and not hard error.

Here is an example before vs after when using dotenv where I have manually stopped my own cache to trigger this issue, before and after the patch. The initial error is really frustrating because there is other caches available.
![image](https://github.com/user-attachments/assets/b4aec474-52d1-497d-b4e8-6f5737d6acc7)
![image](https://github.com/user-attachments/assets/ee91fcd4-4a1a-4c33-bf88-3aee67ad3cc9)

## Context

https://github.com/NixOS/nix/issues/3514#issuecomment-2905056198 is the earliest issue I could find, but there are many duplicates.

There is an initial PR at https://github.com/NixOS/nix/pull/7188, but this appears to have been abandoned - over 2 years with no activity, then a no comment review in jan. There was a subsequent PR at https://github.com/NixOS/nix/pull/8983 but this was closed without merge - over a year without activity.
<!-- Non-trivial change: Briefly outline the implementation strategy. -->
I have visualised the current and proposed flows. I believe my logic flows line up with what is suggested in https://github.com/NixOS/nix/pull/7188#issuecomment-1375652870 but correct me if I am wrong.
Current behaviour:
![current](https://github.com/user-attachments/assets/d9501b34-274c-4eb3-88c3-9021a482e364)
Proposed behaviour:
![proposed](https://github.com/user-attachments/assets/8236e4f4-21ef-45d7-87e1-6c8d416e8c1c)

[Charts in lucid](https://lucid.app/lucidchart/1b51b08d-6c4f-40e0-bf54-480df322cccf/view)
<!-- Invasive change: Discuss alternative designs or approaches you considered. -->

Possible issues to think about:
- I could not figure out where the curl error is created... I can't figure out how to swallow it and turn it into a warn or better yet, a debug log.
- Unfortunately, in contrast with the previous point, I'm not sure how verbose we want to warns/traces to be - personally I think that the warn that a substituter has been disabled (when it happens) is sufficient, and that the next one is being used, but this is personal preference.
2025-09-12 17:29:34 -04:00
Sergei Zimmerman
92df96543c
Merge pull request #13972 from xokdvium/no-string-values-in-evalstate 2025-09-12 21:19:09 +00:00
Jörg Thalheim
5772c207d8
Merge pull request #13970 from xokdvium/no-soname
Revert "meson: add soversion to libraries (#13960)"
2025-09-12 23:17:17 +02:00
Sergei Zimmerman
f4c38278ca
libexpr: Remove vString* Values from EvalState
EvalState is too big and cluttered. These strings
can be private constant statics.
2025-09-12 23:44:52 +03:00
Sergei Zimmerman
0db2b8c8fe
Revert "meson: add soversion to libraries (#13960)"
This reverts commit bdbc739d6e.

Such a change needs more thought put into it. By versioning
shared libraries we'd make a false impression that libraries
themselves are actually versioned and have some sort of stable
ABI, which is not the case.

This will be useful when C bindings become stable, but as long
as they are experimental it does not make sense to set SONAME.

Also this change should not have been backported, since it's
severely breaking.
2025-09-12 20:43:34 +03:00
Eelco Dolstra
8d8f49cb5a Use concurrent_flat_map_fwd.hpp 2025-09-12 17:49:15 +02:00
Eelco Dolstra
4b63ff63a4 Remove some unnecessary hash template arguments 2025-09-12 17:26:29 +02:00
Eelco Dolstra
ad6eb22368 Ensure that files are parsed/evaluated only once
When doing multithreaded evaluation, we want to ensure that any Nix
file is parsed and evaluated only once. The easiest way to do this is
to rely on thunks, since those ensure locking in the multithreaded
evaluator. `fileEvalCache` is now a mapping from `SourcePath` to a
`Value *`. The value is initially a thunk (pointing to a
`ExprParseFile` helper object) that can be forced to parse and
evaluate the file. So a subsequent thread requesting the same file
will see a thunk that is possibly locked and wait for it.

The parser cache is gone since it's no longer needed. However, there
is a new `importResolutionCache` that maps `SourcePath`s to
`SourcePath`s (e.g. `/foo` to `/foo/default.nix`). Previously we put
multiple entries in `fileEvalCache`, which was ugly and could result
in work duplication.
2025-09-12 17:05:21 +02:00
Eelco Dolstra
47c16fc4bd SourcePath: Implement boost::hash 2025-09-12 17:05:21 +02:00
Eelco Dolstra
8fbf4b9427 CanonPath: Implement boost::hash 2025-09-12 17:05:21 +02:00
Eelco Dolstra
7f9b5226af Add getConcurrent helper function 2025-09-12 16:49:25 +02:00
Jörg Thalheim
377b60ee9b
Merge pull request #13926 from NaN-git/opt_boost-unordered
replace all occurences of std::unordered_* by equivalents from boost
2025-09-12 11:46:42 +02:00
Jörg Thalheim
429812cdb8
Merge pull request #13966 from juhp/patch-1
meson: add soversion to libraries (#13960)
2025-09-12 08:25:12 +02:00
Jörg Thalheim
f6802a8ccf
Merge pull request #13963 from xokdvium/fix-no-gc
libexpr: Fix build without Boehm
2025-09-12 08:23:19 +02:00
Jens Petersen
bdbc739d6e meson: add soversion to libraries (#13960) 2025-09-12 14:06:39 +08:00
Sergei Zimmerman
c0b35c71cd
libexpr: Fix build without Boehm
This should have been placed under the ifdef.
2025-09-12 04:02:07 +03:00
Sergei Zimmerman
ef5fedbc0d
Merge pull request #13936 from xokdvium/empty-list-bindings
libexpr: Make constant Values global constants, move out of EvalState
2025-09-10 23:12:20 +00:00
Sergei Zimmerman
5db4b0699c
libexpr: Make constant Values global constants, move out of EvalState
These constant Values have no business being in the EvalState in the
first place. The ultimate goal is to get rid of the ugly `getBuiltins`
and its relience (in `createBaseEnv`) on these global constants is getting in the way.

Same idea as in f017f9ddd3.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-09-11 01:53:41 +03:00
Sergei Zimmerman
462b9ac49c
libexpr: Make Value::isa and Value::getStorage private methods
This was always intended to be the case, but accidentally left
in the public interface.
2025-09-11 01:52:17 +03:00
Sergei Zimmerman
4df1a3ca76
libexpr: Make emptyBindings a global constant
This object is always constant and will never get modified.
Having it as a global (constant) static is much easier and
unclutters the EvalState.

Same idea as in f017f9ddd3.

Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-09-11 01:51:48 +03:00
Philipp Otterbein
9dbc2cae4f hashmaps with string keys: add transparent lookups 2025-09-10 23:04:44 +02:00