nixos/nix
1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-09 03:56:01 +01:00
Code Activity

libexpr: Structural sharing of attrsets

Browse source 
This changes the implementation of Bindings to allow
for a more space-efficient implementation of attribute
set merges. This is accomplished by "layering" over the "base" Bindings.
The top "layer" is naturally the right-hand-side of the update operator //.

Such an implementation leads to significantly better memory usage on
something like nixpkgs:

nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux > /dev/null

Comparison against 2b0fd88324 for x86_64-linux on nixpkgs f06c7c3b6f5074dbffcf02542fb86af3a5526afa:

| metric                 | mean_before     | mean_after      | mean_diff       | mean_%_change | p_value | t_stat  |
| -                      | -               | -               | -               | -             | -       | -       |
| cpuTime                | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| envs.bytes             | 461451951.6190  | 461451951.6190  | -               | -             | -       | -       |
| envs.elements          | 34344544.8571   | 34344544.8571   | -               | -             | -       | -       |
| envs.number            | 23336949.0952   | 23336949.0952   | -               | -             | -       | -       |
| gc.cycles              | 7.5238          | 7.2857          | -0.2381         | -4.6825       | 0.0565  | -2.0244 |
| gc.heapSize            | 1777848124.9524 | 1252162023.6190 | -525686101.3333 | -29.9472      | 0.0000  | -8.7041 |
| gc.totalBytes          | 3102787383.6190 | 2498431578.6667 | -604355804.9524 | -19.7704      | 0.0000  | -9.3502 |
| list.bytes             | 59928225.9048   | 59928225.9048   | -               | -             | -       | -       |
| list.concats           | 1240028.2857    | 1240028.2857    | -               | -             | -       | -       |
| list.elements          | 7491028.2381    | 7491028.2381    | -               | -             | -       | -       |
| nrAvoided              | 28165342.2381   | 28165342.2381   | -               | -             | -       | -       |
| nrExprs                | 1577412.9524    | 1577412.9524    | -               | -             | -       | -       |
| nrFunctionCalls        | 20970743.4286   | 20970743.4286   | -               | -             | -       | -       |
| nrLookups              | 10867306.0952   | 10867306.0952   | -               | -             | -       | -       |
| nrOpUpdateValuesCopied | 61206062.0000   | 25748169.5238   | -35457892.4762  | -58.8145      | 0.0000  | -8.9189 |
| nrOpUpdates            | 2167097.4286    | 2167097.4286    | -               | -             | -       | -       |
| nrPrimOpCalls          | 12337423.4286   | 12337423.4286   | -               | -             | -       | -       |
| nrThunks               | 29361806.7619   | 29361806.7619   | -               | -             | -       | -       |
| sets.bytes             | 1393822818.6667 | 897587655.2381  | -496235163.4286 | -36.7168      | 0.0000  | -9.1115 |
| sets.elements          | 84504465.3333   | 48270845.9524   | -36233619.3810  | -43.8698      | 0.0000  | -8.9181 |
| sets.number            | 5218921.6667    | 5218921.6667    | -               | -             | -       | -       |
| sizes.Attr             | 16.0000         | 16.0000         | -               | -             | -       | -       |
| sizes.Bindings         | 8.0000          | 24.0000         | 16.0000         | 200.0000      | -       | inf     |
| sizes.Env              | 8.0000          | 8.0000          | -               | -             | -       | -       |
| sizes.Value            | 16.0000         | 16.0000         | -               | -             | -       | -       |
| symbols.bytes          | 1368494.0952    | 1368494.0952    | -               | -             | -       | -       |
| symbols.number         | 109147.1905     | 109147.1905     | -               | -             | -       | -       |
| time.cpu               | 21.1520         | 21.3414         | 0.1894          | 0.7784        | 0.3190  | 1.0219  |
| time.gc                | 1.6011          | 0.8508          | -0.7503         | -37.1507      | 0.0017  | -3.6328 |
| time.gcFraction        | 0.0849          | 0.0399          | -0.0450         | -37.4504      | 0.0035  | -3.3116 |
| values.bytes           | 615968144.7619  | 615968144.7619  | -               | -             | -       | -       |
| values.number          | 38498009.0476   | 38498009.0476   | -               | -             | -       | -       |

Overall this does slow down the evaluator slightly (no more than ~10% in most cases),
but this seems like a very decent tradeoff for shaving off 33% of memory usage.
This commit is contained in:
Sergei Zimmerman 2025-09-15 22:16:17 +03:00
parent 3eb223f4bb
commit 6138bc3de3
No known key found for this signature in database
7 changed files with 424 additions and 63 deletions
Download patch file Download diff file Expand all files Collapse all files

27
src/libexpr-tests/nix_api_expr.cc
View file

@ -437,4 +437,31 @@ TEST_F(nix_api_expr_test, nix_value_call_multi_no_args)
assert_ctx_ok();
ASSERT_EQ(3, rInt);
}
TEST_F(nix_api_expr_test, nix_expr_attrset_update)
{
nix_expr_eval_from_string(ctx, state, "{ a = 0; b = 2; } // { a = 1; b = 3; } // { a = 2; }", ".", value);
assert_ctx_ok();
ASSERT_EQ(nix_get_attrs_size(ctx, value), 2);
assert_ctx_ok();
std::array<std::pair<std::string_view, nix_value *>, 2> values;
for (unsigned int i = 0; i < 2; ++i) {
const char * name;
values[i].second = nix_get_attr_byidx(ctx, value, state, i, &name);
assert_ctx_ok();
values[i].first = name;
}
std::sort(values.begin(), values.end(), [](const auto & lhs, const auto & rhs) { return lhs.first < rhs.first; });
nix_value * a = values[0].second;
ASSERT_EQ("a", values[0].first);
ASSERT_EQ(nix_get_int(ctx, a), 2);
assert_ctx_ok();
nix_value * b = values[1].second;
ASSERT_EQ("b", values[1].first);
ASSERT_EQ(nix_get_int(ctx, b), 3);
assert_ctx_ok();
}
} // namespace nixC