1
1
Fork 0
mirror of https://github.com/NixOS/nix.git synced 2025-11-29 21:50:58 +01:00
nix/src/libexpr
Sergei Zimmerman 86a3fad085
libexpr: Improve lexer performance by using full scanner tables (-Cf)
This trades off some executable size for measurable lexer performance
improvements.

Note on the explicitly enabling 8bit scanner.
This is needed due to the default behavior of flex (excerpt from the manual [1]):

> Flex’s default behavior is to generate an 8-bit scanner unless you
> use the ‘-Cf’ or ‘-CF’, in which case flex defaults to generating
> 7-bit scanners unless your site was always configured to generate 8-bit
> scanners.

Some quantifyable metrics:

Nixpkgs revision: a6e3f45acf4e817532a861ab0eda4ab5485fecc1
Parsing the largest file in nixpkgs: pkgs/development/haskell-modules/hackage-packages.nix.

(Before this patch)

```
$ nix build github:nixos/nix/9fe3077d4#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2518    result/lib/libnixexpr.so
$ nix build github:nixos/nix/9fe3077d4#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
    result/bin/nix-instantiate --parse               \
    ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
  Time (mean ± σ):     375.5 ms ±   6.3 ms    [User: 316.9 ms, System: 56.7 ms]
  Range (min … max):   368.5 ms … 388.3 ms    10 runs
```

(After the patch)

```
$ nix build .#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2685    result/lib/libnixexpr.so
$ nix build .#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
    result/bin/nix-instantiate --parse               \
    ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
  Time (mean ± σ):     326.8 ms ±   4.9 ms    [User: 269.5 ms, System: 55.3 ms]
  Range (min … max):   319.7 ms … 335.5 ms    10 runs
```

Overall, the change is roughly:

- 2518KiB -> 2685KiB ~ 150 KiB of machine code
- 375ms -> 325ms ~ 50ms

The perf uplift for eval-heavy test cases is obviously less noticeable,
but it doesn't make sense not to take this free perf win.

[1]: https://westes.github.io/flex/manual/Options-Affecting-Scanner-Behavior.html#Options-Affecting-Scanner-Behavior
2025-05-01 23:10:04 +00:00
..
include/nix/expr Move the InputCache to EvalState 2025-04-14 14:29:14 +02:00
primops lockFlake(): Allow registry lookups for the top-level flake 2025-04-24 18:59:10 +02:00
value Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
.version Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
attr-path.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
attr-set.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
eval-cache.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
eval-error.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
eval-gc.cc Add -Wundef to make #if FOO an error if not defined 2025-04-05 00:45:19 +02:00
eval-settings.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
eval.cc Move the InputCache to EvalState 2025-04-14 14:29:14 +02:00
fetchurl.nix Format .nix files 2025-01-24 17:04:02 +01:00
function-trace.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
get-drvs.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
imported-drv-to-derivation.nix Format .nix files 2025-01-24 17:04:02 +01:00
json-to-value.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
lexer-helpers.cc Make lexer-helpers.hh internal to fix a clang-tidy error 2025-04-07 18:21:08 +02:00
lexer-helpers.hh Make lexer-helpers.hh internal to fix a clang-tidy error 2025-04-07 18:21:08 +02:00
lexer.l libexpr: Improve lexer performance by using full scanner tables (-Cf) 2025-05-01 23:10:04 +00:00
meson.build libexpr: Improve lexer performance by using full scanner tables (-Cf) 2025-05-01 23:10:04 +00:00
meson.options Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
nix-meson-build-support rename: build-utils-meson -> nix-meson-build-support 2024-12-09 16:54:42 +01:00
nixexpr.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
package.nix Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
parser.y Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
paths.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
primops.cc libexpr: fix UB in builtins.ceil and builtins.floor 2025-04-13 04:36:09 +02:00
print-ambiguous.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
print.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
search-path.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00
value-to-json.cc Docs 2025-04-02 18:02:32 +02:00
value-to-xml.cc Expose the nix component in header include paths 2025-04-01 11:40:42 -04:00