1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-21 01:39:36 +01:00
nix/src/libstore
Bernardo Meurer Costa 18ec3d1094
libstore: Avoid copying derivations to the store if they are already valid
This avoids the quite costly copying of derivations to the daemon over the
wire in case it already exists in the eval store.

For a fresh instantiatation (after running nix-collect-garbage) this doesn't
significantly slow down eval:

taskset -c 2,3 hyperfine --reference "result-old/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket" --prepare "nix-collect-garbage --store /tmp/store1111 --no-keep-derivations" "result/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket"
Benchmark 1: result-old/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket
  Time (mean ± σ):     388.7 ms ±  10.5 ms    [User: 157.0 ms, System: 61.3 ms]
  Range (min … max):   379.4 ms … 415.9 ms    10 runs

Benchmark 2: result/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket
  Time (mean ± σ):     389.2 ms ±   4.8 ms    [User: 158.5 ms, System: 60.7 ms]
  Range (min … max):   381.2 ms … 397.6 ms    10 runs

But if the derivations are already instantiated this shows a pretty neat speedup:

taskset -c 2,3 hyperfine --reference "result-old/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket" "result/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket"
Benchmark 1: result-old/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket
  Time (mean ± σ):     240.4 ms ±   3.1 ms    [User: 148.1 ms, System: 57.0 ms]
  Range (min … max):   233.8 ms … 245.0 ms    12 runs

Benchmark 2: result/bin/nix eval -f ../nixpkgs hello --store unix:///tmp/nix_socket
  Time (mean ± σ):     226.5 ms ±   4.5 ms    [User: 147.8 ms, System: 55.2 ms]
  Range (min … max):   214.0 ms … 231.2 ms    13 runs

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-10-13 01:59:38 +03:00
..
build Fix windows build 2025-10-12 13:22:35 -04:00
builtins lib{store,fetchers}: Pass URLs specified directly verbatim to FileTransferRequest 2025-09-01 02:22:23 +03:00
include/nix/store Split out DerivationResolutionGoal 2025-10-11 19:52:37 -04:00
linux Merge pull request #13490 from Mic92/clang-tidy-simple-warnings 2025-08-11 18:21:46 +02:00
pch libstore: Speed up builds by using precompiled headers 2025-07-20 22:01:53 +03:00
unix Consolidate logic choosing where we can/should build a bit 2025-10-10 17:28:57 -04:00
windows meson: Apply formatting universally 2025-08-07 02:58:29 +03:00
.version Build nix-store with Meson 2024-06-14 10:25:14 -04:00
aws-creds.cc feat(libstore/filetransfer): add S3 signing support 2025-10-08 21:31:34 +00:00
binary-cache-store.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
build-result.cc Use std::variant to enforce BuildResult invariants 2025-09-27 15:56:06 -04:00
ca-specific-schema.sql Rename to "content-address*ing* derivation" 2025-02-10 01:12:56 -05:00
common-protocol.cc Convert Realisation JSON logic to standard style 2025-09-22 12:59:37 -04:00
common-ssh-store-config.cc Add user@address:port support 2025-08-06 23:48:14 +03:00
content-address.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
daemon.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
derivation-options.cc exportReferencesGraph: Handle heterogeneous arrays 2025-10-08 22:15:33 +02:00
derivations.cc libstore: Avoid copying derivations to the store if they are already valid 2025-10-13 01:59:38 +03:00
derived-path-map.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
derived-path.cc Modernize and test derived path JSON 2025-09-23 15:05:56 -04:00
downstream-placeholder.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
dummy-store.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
dummy-store.md libstore: Fix typo in description of dummy store 2025-09-21 13:19:49 +03:00
export-import.cc Move exportPaths() / importPaths() out of the Store class 2025-09-10 14:22:46 +02:00
filetransfer.cc feat(libstore/filetransfer): add S3 signing support 2025-10-08 21:31:34 +00:00
gc.cc libstore: Include missing header to fix compile with libc++ 20 2025-10-01 22:19:08 +03:00
globals.cc Consolidate logic choosing where we can/should build a bit 2025-10-10 17:28:57 -04:00
http-binary-cache-store.cc fix(libstore): improve http-binary-cache-store S3 compatibility 2025-10-10 22:05:57 +00:00
http-binary-cache-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
indirect-root-store.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
keys.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
legacy-ssh-store.cc Use std::variant to enforce BuildResult invariants 2025-09-27 15:56:06 -04:00
legacy-ssh-store.md Add user@address:port support 2025-08-06 23:48:14 +03:00
local-binary-cache-store.cc Rewrite StoreConfig::getUri in terms of new StoreConfig::getReference 2025-08-13 19:06:59 -04:00
local-binary-cache-store.md Support per-store Markdown documentation 2023-03-21 14:03:40 +01:00
local-fs-store.cc libstore: Deduplicate LocalFSStoreConfig::rootDir initializers 2025-09-28 18:38:57 +03:00
local-overlay-store.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
local-overlay-store.md docs: Fix miscellaneous typos and formatting issues 2025-05-13 22:20:11 +00:00
local-store.cc libstore: Fix double-quoting of paths in logs 2025-10-11 00:25:51 +03:00
local-store.md Build the local store on Windows 2024-05-10 13:05:23 -04:00
log-store.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
machines.cc Clean up Base* code 2025-08-06 11:51:44 -04:00
make-content-addressed.cc ValidPathInfo, NarInfo, turn funky constructor into static method 2025-09-13 13:17:14 -04:00
meson.build Split out DerivationResolutionGoal 2025-10-11 19:52:37 -04:00
meson.options build(libstore): add NIX_WITH_CURL_S3 build option 2025-10-03 03:34:57 +00:00
misc.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
mounted-ssh-store.md MountedSSHStore: stores on shared filesystems 2023-11-21 13:34:01 -05:00
names.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
nar-accessor.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
nar-info-disk-cache.cc Convert Realisation JSON logic to standard style 2025-09-22 12:59:37 -04:00
nar-info.cc Make a few more things use StoreDirConfig instead of Store 2025-08-15 15:35:51 -04:00
nix-meson-build-support rename: build-utils-meson -> nix-meson-build-support 2024-12-09 16:54:42 +01:00
optimise-store.cc libstore: Fix double-quoting of paths in logs 2025-10-11 00:25:51 +03:00
outputs-spec.cc Fix JSON_IMPL macro to avoid extraneous copies 2025-09-23 13:05:12 -04:00
package.nix build(libstore): assert withAWS xor withCurlS3 2025-10-06 16:22:01 +00:00
parsed-derivations.cc Deduplicate "export reference graph" logic a bit 2025-08-20 16:54:17 -04:00
path-info.cc ValidPathInfo, NarInfo, turn funky constructor into static method 2025-09-13 13:17:14 -04:00
path-references.cc libstore: Remove unused overload of scanForReferences 2025-08-08 10:30:07 +03:00
path-with-outputs.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
path.cc Fix JSON_IMPL macro to avoid extraneous copies 2025-09-23 13:05:12 -04:00
pathlocks.cc Apply clang-format universally. 2025-07-18 12:47:27 -04:00
posix-fs-canonicalise.cc Use std::variant to enforce BuildResult invariants 2025-09-27 15:56:06 -04:00
profiles.cc No more globals.hh in headers 2025-08-20 16:24:37 -04:00
realisation.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
references.cc libutil: Move references.{hh,cc} to libstore 2025-08-08 10:30:09 +03:00
remote-fs-accessor.cc Create a second Store::getFSAccessor for a single store object 2025-09-24 15:49:14 -04:00
remote-store.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
restricted-store.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
s3-binary-cache-store.cc feat(libstore): add curl-based S3 store implementation 2025-10-10 00:13:29 +00:00
s3-binary-cache-store.md fix location 2024-09-27 11:07:04 +02:00
s3-url.cc build(libstore): add NIX_WITH_CURL_S3 build option 2025-10-03 03:34:57 +00:00
schema.sql Build the local store on Windows 2024-05-10 13:05:23 -04:00
serve-protocol-connection.cc Remove support for serve protocol < 5 2025-09-10 10:57:15 +02:00
serve-protocol.cc Use std::variant to enforce BuildResult invariants 2025-09-27 15:56:06 -04:00
sqlite.cc SQLite: fsync db.sqlite-shm before opening the database 2025-08-26 00:42:18 +03:00
ssh-store.cc Create a second Store::getFSAccessor for a single store object 2025-09-24 15:49:14 -04:00
ssh-store.md Add user@address:port support 2025-08-06 23:48:14 +03:00
ssh.cc libstore: fixup fakeSSH check 2025-10-03 12:17:17 -07:00
store-api.cc Revert "Merge pull request #14097 from obsidiansystems/light-realisation-improvements" 2025-10-05 21:54:32 +03:00
store-dir-config.cc Simplify "Store dir" superclass 2025-08-15 14:12:37 -04:00
store-reference.cc libstore: Reallow unbracketed IPv6 addresses in store references 2025-09-09 00:41:03 +03:00
store-registration.cc No more globals.hh in headers 2025-08-20 16:24:37 -04:00
uds-remote-store.cc libstore: Make all StoreConfig::getReference implementations return store parameters 2025-09-28 16:29:12 +03:00
uds-remote-store.md Enable the unix:// store on Windows 2024-04-18 16:58:32 -04:00
worker-protocol-connection.cc Remove WorkerProto::Op::ImportPaths 2025-09-09 15:41:17 +02:00
worker-protocol.cc Use std::variant to enforce BuildResult invariants 2025-09-27 15:56:06 -04:00