Typically PosixSourceAccessor can be used from multiple threads,
but mtime is not updated atomically (i.e. with compare_exchange_weak),
so mtime gets raced. It's only needed in dumpPathAndGetMtime and mtime
tracking can be gated behind that.
Also start using getLastModified interface instead of dynamic casts.
The variant has on the left-hand side the topologically sorted vector
and the right-hand side is a pair showing the path and its parent that
represent a cycle in the graph making the sort impossible.
This change prepares for enhanced cycle error messages that can provide
more context about the cycle. The variant approach allows callers to
handle cycles more flexibly, enabling better error reporting that shows
the full cycle path and which files are involved.
Adapted from Lix commit f7871fcb5.
Change-Id: I70a987f470437df8beb3b1cc203ff88701d0aa1b
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Otherwise PosTable grows indefinitely for each reload. Since
the total input size is limited to 4GB (uint32_t for byte offset PosIdx)
it can get exhausted pretty. This ensures that we don't waste memory
on reloads as well.
It turns out this code path is only used for unit tests (to ensure our
JSON formats are possible to parse by other code, elsewhere). No
user-facing functionality consumes this format.
Therefore, let's drop the old version parsing support.
This was already dropped in `inputFromURL()`, but not in
`inputFromAttrs()`. Now it's done in `fixGitURL()`, which is used by
both.
In principle, `git+` shouldn't be used in the `url` attribute, since
we already know that it's a Git URL. But since it currently works, we
don't want to break it.
Fixes#14429.
... via _NIX_TEST_NO_ENVIRONMENT_WARNINGS
This environment variable suppresses warnings that depend on the test
environment (such as ulimit warnings in builds on systems with lower
limits, which may well succeed if it weren't for the warning).
This prevents non-deterministic test failures in golden/characterization
tests.
Alternative considered: filtering stderr in test scripts, but that approach
is fragile with binary test output, and potentially multiple call sites.
Show the actual attempted stack size value (capped at hard limit)
separately from the desired value, making it clearer what's happening
when the hard limit is lower than requested.
Introduce a `CompressedSource` class in libutil's `serialise.hh` that
compresses a `RestartableSource` and owns the compressed data. This is a
general-purpose utility that can be used anywhere compressed data needs
to be treated as a source.
The `showBytes()` function was redundant with `renderSize()` as the
latter automatically selects the appropriate unit (KiB, MiB, GiB, etc.)
based on the value, whereas `showBytes()` always formatted as MiB
regardless of size.
Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
In particular
- Remove `get`, it is redundant with `valueAt` and the `get` in
`util.hh`.
- Remove `nullableValueAt`. It is morally just the function composition
`getNullable . valueAt`, not an orthogonal combinator like the others.
- `optionalValueAt` return a pointer, not `std::optional`. This also
expresses optionality, but without creating a needless copy. This
brings it in line with the other combinators which also return
references.
- Delete `valueAt` and `optionalValueAt` taking the map by value, as we
did for `get` in 408c09a120, which
prevents bugs / unnecessary copies.
`adl_serializer<DerivationOptions::OutputChecks>::from_json` was the one
use of `getNullable`. I give it a little static function for the
ultimate creation of a `std::optional` it does need to do (after
switching it to using `getNullable . valueAt`. That could go in
`json-utils.hh` eventually, but I didn't bother for now since only one
things needs it.
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
addToStore(): Don't parse the NAR
* StringSource: Implement skip()
This is slightly faster than doing a read() into a buffer just to
discard the data.
* LocalStore::addToStore(): Skip unnecessary NARs rather than parsing them
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Since 2.32, nix now needs boost 1.87 or later to build,
due to using unordered::concurrent_flat_map try_emplace_and_cvisit
../src/libexpr/eval.cc: In member function ‘void nix::EvalState::evalFile(const nix::SourcePath&, nix::Value&, bool)’:
../src/libexpr/eval.cc:1096:20: error: ‘class boost::unordered::concurrent_flat_map<nix::SourcePath, nix::Value*, std::hash<nix::SourcePath>, std::equal_to<nix::SourcePath>, traceable_allocator<std::pair<const nix::SourcePath, nix::Value*> > >’ has no member named ‘try_emplace_and_cvisit’; did you mean ‘try_emplace_or_cvisit’?
1096 | fileEvalCache->try_emplace_and_cvisit(
| ^~~~~~~~~~~~~~~~~~~~~~
| try_emplace_or_cvisit
See 834580b539
Previously, `prefetchFile()` used `baseNameOf()` directly on the URL string
to extract the filename. This caused issues with URLs containing query
parameters that include slashes, such as S3 URLs with custom endpoints:
```
s3://bucket/file.txt?endpoint=http://server:9000
```
The `baseNameOf()` function naively searches for the rightmost `/` in the
entire string, which would find the `/` in `http://server:9000` and extract
`server:9000®ion=...` as the filename. This resulted in invalid store
path names containing illegal characters like `:`.
This commit fixes the issue by:
1. Adding a `VerbatimURL::lastPathSegment()` method that extracts the last
non-empty path segment from a URL, using `pathSegments(true)` to filter
empty segments
2. Changing `prefetchFile()` to accept `const VerbatimURL &` and use the new
`lastPathSegment()` method instead of manual path parsing
3. Adding early validation with `checkName()` to fail quickly on invalid
filenames
4. Maintains backward compatibility by falling back to `baseNameOf()` for
unparsable `VerbatimURL`s
I realized that we can actually do this thing, even though it is not
what nlohmann expects at all, because the extra parameter has a default
argument so nlohmann doesn't need to care. Sneaky!
Wrap fmt() calls in lambdas to defer string formatting until the
feature check fails. This avoids unnecessary string formatting in
the common case where the feature is enabled.
Addresses performance concern raised by xokdvium in PR review.
This, alongside the other invariants of the CanonPath is important
to uphold. std::filesystem happily crashes on NUL bytes in the constructor,
as we've seen with `path:%00` prior to c436b7a32a.
Best to stay clear of NUL bytes when we're talking about syscalls, especially
on Unix where strings are null terminated.
Very nice to have if we decide to switch over to pascal-style strings.
