1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-12-02 07:00:59 +01:00
Commit graph

25 commits

Author SHA1 Message Date
Bernardo Meurer Costa
965d6be7c1
tests(nixos/s3-binary-cache-store): enable multipart 2025-11-03 01:15:46 +00:00
Bernardo Meurer Costa
94965a3a3e
test(nixos): add S3 multipart upload integration tests 2025-10-28 06:17:41 +00:00
Bernardo Meurer Costa
e38128b90d
feat(libstore): support S3 object versioning via versionId parameter
S3 buckets support object versioning to prevent unexpected changes,
but Nix previously lacked the ability to fetch specific versions of
S3 objects. This adds support for a `versionId` query parameter in S3
URLs, enabling users to pin to specific object versions:

```
s3://bucket/key?region=us-east-1&versionId=abc123
```
2025-10-25 07:57:58 +00:00
Bernardo Meurer Costa
1b1d7e3047
test(nixos): add nix-prefetch-url test for S3 URLs with query parameters
Adds a comprehensive test to verify that `nix-prefetch-url` correctly
handles S3 URLs with query parameters (e.g., custom endpoints and regions).

Previously, nix-prefetch-url would fail with "invalid store
path" errors when given S3 URLs with query parameters like
`?endpoint=http://server:9000&region=eu-west-1`, because it incorrectly
extracted the filename from the query parameters instead of the path.
2025-10-20 21:45:37 +03:00
Bernardo Meurer Costa
d9c808f8a7
refactor(tests/nixos/s3-binary-cache-store): add verify_packages_in_store helper 2025-10-19 00:21:54 +00:00
Bernardo Meurer Costa
55ea3d3476
test(tests/nixos/s3-binary-cache-store): test public bucket operations
Add `test_public_bucket_operations` to validate that store operations
work correctly on public S3 buckets without requiring credentials.
Tests nix store info and nix copy operations.
2025-10-19 00:04:33 +00:00
Bernardo Meurer Costa
7d0c06f921
feat(tests/nixos/s3-binary-cache-store): add public parameter to setup_s3
Add optional 'public' parameter to setup_s3 decorator. When set to True,
the bucket will be made publicly accessible using mc anonymous set.
2025-10-18 23:57:51 +00:00
Bernardo Meurer Costa
5b4bd5bcb8
refactor(tests/nixos/s3-binary-cache-store): inline make_http_url fn
Remove make_http_url helper function and inline its single usage.
2025-10-18 23:51:44 +00:00
Bernardo Meurer Costa
4ae6c65bc5
test(tests/nixos/s3-binary-cache-store): verify credential caching in concurrent fetches
Add assertion to test_concurrent_fetches to verify that only one
credential provider is created even with 5 concurrent fetches.
2025-10-18 23:48:55 +00:00
Bernardo Meurer Costa
4f19e63a8f
refactor(tests/nixos/s3-binary-cache-store): add --no-link to nix build commands
Prevent creation of result symlinks in all nix build commands by
adding the --no-link flag.
2025-10-18 23:44:13 +00:00
Bernardo Meurer Costa
f88c3055f8
refactor(tests/nixos/s3-binary-cache-store): clean client store in setup_s3
Add cleanup of client store in the finally block of setup_s3 decorator.
Uses `nix store delete --ignore-liveness` to properly handle GC roots
and only attempts deletion if the path exists.
2025-10-18 23:36:48 +00:00
Bernardo Meurer Costa
9058d90ab2
refactor(tests/nixos/s3-binary-cache-store): rename populate_with to populate_bucket 2025-10-18 23:27:03 +00:00
Bernardo Meurer Costa
c1a15d1a26
refactor(tests/nixos/s3-binary-cache-store): rename with_test_bucket to setup_s3 2025-10-18 23:24:30 +00:00
Bernardo Meurer Costa
22f4cccc71
refactor(tests/nixos/s3-binary-cache-store): use a PKGS dict
Replace individual PKG_A, PKG_B, and PKG_C variables with a PKGS
dictionary. This will enable `@with_clean_client_store` in the future.
2025-10-18 23:23:50 +00:00
Sergei Zimmerman
b1d067c9bb
tests/nixos: Rename back S3 store nixos test 2025-10-16 21:49:35 +03:00
Bernardo Meurer Costa
9295c14a35
refactor(libstore): replace AWS SDK with curl-based S3 implementation
This commit replaces the AWS C++ SDK with a lighter curl-based approach
for S3 binary cache operations.

- Removed dependency on the heavy aws-cpp-sdk-s3 and aws-cpp-sdk-transfer
- Added lightweight aws-crt-cpp for credential resolution only
- Leverages curl's native AWS SigV4 authentication (requires curl >= 7.75.0)
- S3BinaryCacheStore now delegates to HttpBinaryCacheStore
- Function s3ToHttpsUrl converts ParsedS3URL to ParsedURL
- Multipart uploads are no longer supported (may be reimplemented later)
- Build now requires curl >= 7.75.0 for AWS SigV4 support

Fixes: #13084, #12671, #11748, #12403, #5947
2025-10-15 18:23:55 +00:00
Eelco Dolstra
ca9696748a Make the S3 test more robust
Waiting for the minio unit is apparently not reliable enough, so let's
also wait for the port.
2025-05-24 01:30:51 +02:00
Robert Hensing
58b657b976 tests/nixos: Work around network-online.target inactivity 2025-04-01 16:36:47 +02:00
Robert Hensing
96e550efc5 Format .nix files
... with nixfmt (rfc style)
2025-01-24 17:04:02 +01:00
Robert Hensing
6f0bdd9ae7 tests: Wait for network *online* targets
More prevalent than I thought in 15073e86a

See also https://github.com/NixOS/nix/actions/runs/12872412321/job/35887830320?pr=12310
which is a failed github-flakes test without "Network is Online"
2025-01-20 18:24:22 +01:00
Cole Helbling
f0c1262d23 tests/nixos/s3-binary-cache-store: disable default substituter so it runs faster
Since networking is disabled in these VMs, trying to talk to the default
cache.nixos.org slows the test down (since it can't resolve it).
2024-12-19 11:16:34 -08:00
Cole Helbling
535724fd79 tests/nixos/s3-binary-cache-store: test that "object does not exist" error message is properly formatted 2024-12-19 11:16:30 -08:00
Eelco Dolstra
e09666d314 Fix test name 2024-10-22 22:05:48 +02:00
Eelco Dolstra
0500fba56a builtins.fetchurl: Fix segfault on s3:// URLs
Also, add an activity to show that we're downloading an s3:// file.

Fixes #11674.
2024-10-11 14:32:34 +02:00
Eelco Dolstra
2950f9e18a Add a VM test for S3BinaryCacheStore
Fixes #11238.
2024-08-05 11:48:50 +02:00