This commit replaces the AWS C++ SDK with a lighter curl-based approach
for S3 binary cache operations.
- Removed dependency on the heavy aws-cpp-sdk-s3 and aws-cpp-sdk-transfer
- Added lightweight aws-crt-cpp for credential resolution only
- Leverages curl's native AWS SigV4 authentication (requires curl >= 7.75.0)
- S3BinaryCacheStore now delegates to HttpBinaryCacheStore
- Function s3ToHttpsUrl converts ParsedS3URL to ParsedURL
- Multipart uploads are no longer supported (may be reimplemented later)
- Build now requires curl >= 7.75.0 for AWS SigV4 support
Fixes: #13084, #12671, #11748, #12403, #5947
This has been dropped on unstable an nix no longer
compiled with overridden nixpkgs input. On 25.05 these
overrides already do nothing.
Tested with:
nix build .#packages.x86_64-darwin.nix-cli -L --override-input nixpkgs https://releases.nixos.org/nixos/unstable/nixos-25.11pre859555.ab0f3607a6c7/nixexprs.tar.xz
Default deployment target on 25.05 is 11.3, so 10.13
sdk override doesn't have to be updated at all as evident
from the fact that we didn't observe any issues with it.
The myriad of hand-rolled URL parsing and validation code
is a constant source of problems. Regexes are not a great way
of writing parsers and there's a history of getting them wrong.
Boost.URL is a good library we can outsource most of the heavy
lifting to.
The STSProfileCredentialsProviders allows to assume a specific IAM role
when accessing an S3 repository. Sometimes this is needed to obtain the
permissions to operate on the bucket.
... and remove a few unused arguments.
This adds pkg-config to a two or three packages that don't use it,
but we shouldn't let that bother us. It's like our personal stdenv.
libgit2 didn't write thin ones, hence the patch.
This should improve performance on systems with weak I/O in ~/.cache,
especially in terms of operations per second, or where system calls
are slower. (macOS, VMs?)
As discussed in our meeting, we should use a simplified version for the
libraries without the date or commit hash. This will make rebuilding a
lot faster in many cases.
Progress on #10379
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.
Changes:
- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)
> nix: disable big-parallel for aws-sdk-cpp
> aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
> which does not justify the requirement on big parallel.
> Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.
> I can finally build nix without requiring a big-parallel machine.
- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)
> nix: use [ ] instead null to empty requiredSystemFeatures
> fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
In addition to adding the missing thread deps in the last commit, we
also appear to need to skip `-Wl,--as-needed` flags that Meson wants to
use, but doesn't work with our *BSD toolchains.
See https://github.com/mesonbuild/meson/issues/3593
