nixos/nix
1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-12-16 14:01:05 +01:00
Code Activity
nix/src
History
Sergei Zimmerman 6138bc3de3
libexpr: Structural sharing of attrsets 
This changes the implementation of Bindings to allow
for a more space-efficient implementation of attribute
set merges. This is accomplished by "layering" over the "base" Bindings.
The top "layer" is naturally the right-hand-side of the update operator //.

Such an implementation leads to significantly better memory usage on
something like nixpkgs:

nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux > /dev/null

Comparison against 2b0fd88324 for x86_64-linux on nixpkgs f06c7c3b6f5074dbffcf02542fb86af3a5526afa:

| metric                 | mean_before     | mean_after      | mean_diff       | mean_%_change | p_value | t_stat  |
| -                      | -               | -               | -               | -             | -       | -       |
| cpuTime                | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| envs.bytes             | 461451951.6190  | 461451951.6190  | -               | -             | -       | -       |
| envs.elements          | 34344544.8571   | 34344544.8571   | -               | -             | -       | -       |
| envs.number            | 23336949.0952   | 23336949.0952   | -               | -             | -       | -       |
| gc.cycles              | 7.5238          | 7.2857          | -0.2381         | -4.6825       | 0.0565  | -2.0244 |
| gc.heapSize            | 1777848124.9524 | 1252162023.6190 | -525686101.3333 | -29.9472      | 0.0000  | -8.7041 |
| gc.totalBytes          | 3102787383.6190 | 2498431578.6667 | -604355804.9524 | -19.7704      | 0.0000  | -9.3502 |
| list.bytes             | 59928225.9048   | 59928225.9048   | -               | -             | -       | -       |
| list.concats           | 1240028.2857    | 1240028.2857    | -               | -             | -       | -       |
| list.elements          | 7491028.2381    | 7491028.2381    | -               | -             | -       | -       |
| nrAvoided              | 28165342.2381   | 28165342.2381   | -               | -             | -       | -       |
| nrExprs                | 1577412.9524    | 1577412.9524    | -               | -             | -       | -       |
| nrFunctionCalls        | 20970743.4286   | 20970743.4286   | -               | -             | -       | -       |
| nrLookups              | 10867306.0952   | 10867306.0952   | -               | -             | -       | -       |
| nrOpUpdateValuesCopied | 61206062.0000   | 25748169.5238   | -35457892.4762  | -58.8145      | 0.0000  | -8.9189 |
| nrOpUpdates            | 2167097.4286    | 2167097.4286    | -               | -             | -       | -       |
| nrPrimOpCalls          | 12337423.4286   | 12337423.4286   | -               | -             | -       | -       |
| nrThunks               | 29361806.7619   | 29361806.7619   | -               | -             | -       | -       |
| sets.bytes             | 1393822818.6667 | 897587655.2381  | -496235163.4286 | -36.7168      | 0.0000  | -9.1115 |
| sets.elements          | 84504465.3333   | 48270845.9524   | -36233619.3810  | -43.8698      | 0.0000  | -8.9181 |
| sets.number            | 5218921.6667    | 5218921.6667    | -               | -             | -       | -       |
| sizes.Attr             | 16.0000         | 16.0000         | -               | -             | -       | -       |
| sizes.Bindings         | 8.0000          | 24.0000         | 16.0000         | 200.0000      | -       | inf     |
| sizes.Env              | 8.0000          | 8.0000          | -               | -             | -       | -       |
| sizes.Value            | 16.0000         | 16.0000         | -               | -             | -       | -       |
| symbols.bytes          | 1368494.0952    | 1368494.0952    | -               | -             | -       | -       |
| symbols.number         | 109147.1905     | 109147.1905     | -               | -             | -       | -       |
| time.cpu               | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| time.gc                | 1.6011          | 0.8508          | -0.7503         | -37.1507      | 0.0017  | -3.6328 |
| time.gcFraction        | 0.0849          | 0.0399          | -0.0450         | -37.4504      | 0.0035  | -3.3116 |
| values.bytes           | 615968144.7619  | 615968144.7619  | -               | -             | -       | -       |
| values.number          | 38498009.0476   | 38498009.0476   | -               | -             | -       | -       |

Overall this does slow down the evaluator slightly (no more than ~10% in most cases),
but this seems like a very decent tradeoff for shaving off 33% of memory usage.
2025-09-17 23:54:45 +03:00
..
external-api-docs meson: Apply formatting universally 2025-08-07 02:58:29 +03:00
internal-api-docs meson: Apply formatting universally 2025-08-07 02:58:29 +03:00
libcmd meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libexpr libexpr: Structural sharing of attrsets 2025-09-17 23:54:45 +03:00
libexpr-c libexpr: Structural sharing of attrsets 2025-09-17 23:54:45 +03:00
libexpr-test-support meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libexpr-tests libexpr: Structural sharing of attrsets 2025-09-17 23:54:45 +03:00
libfetchers meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libfetchers-c meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libfetchers-tests Use MemorySourceAccessor in DummyStore 2025-09-17 16:09:53 -04:00
libflake meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libflake-c meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libflake-tests libfetchers: Restore path separator ignoring behavior for indirect and git-archive flakerefs 2025-08-30 14:40:56 +03:00
libmain meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libmain-c meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libstore Use MemorySourceAccessor in DummyStore 2025-09-17 16:09:53 -04:00
libstore-c Add new C API for working with derivations 2025-09-16 13:25:36 -04:00
libstore-test-support Add new C API for working with derivations 2025-09-16 13:25:36 -04:00
libstore-tests Add new C API for working with derivations 2025-09-16 13:25:36 -04:00
libutil Add missing #pragma once 2025-09-17 15:21:21 -04:00
libutil-c meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libutil-test-support meson: refactor nix_soversion into nix-meson-build-support/common 2025-09-16 12:54:30 +08:00
libutil-tests Introduce Hash::parseExplicitFormatUnprefixed 2025-09-12 18:04:29 -04:00
nix Merge pull request #13998 from Mic92/fast-flake-check 2025-09-15 21:44:11 +02:00
perl Move exportPaths() / importPaths() out of the Store class 2025-09-10 14:22:46 +02:00
nix-functional-tests Build Functional tests with Meson 2024-08-14 15:35:40 -04:00
nix-manual Build the manual with Meson 2024-10-09 11:58:17 -04:00