Since we dropped fs::symlink_exists, we no longer have a need for the fs
namespace. Having less abstractions makes it easier to lookup the
functions in reference documentations.
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
There are two big changes:
1. Public and private config is now separated. Configuration variables
that are only used internally do not go in a header which is
installed.
(Additionally, libutil has a unix-specific private config header,
which should only be used in unix-specific code. This keeps things a
bit more organized, in a purely private implementation-internal way.)
2. Secondly, there is no more `-include`. There are very few config
items that need to be publically exposed, so now it is feasible to
just make the headers that need them just including the (public)
configuration header.
And there are also a few more small cleanups on top of those:
- The configuration files have better names.
- The few CPP variables that remain exposed in the public headers are
now also renamed to always start with `NIX_`. This ensures they should
not conflict with variables defined elsewhere.
- We now always use `#if` and not `#ifdef`/`#ifndef` for our
configuration variables, which helps avoid bugs by requiring that
variables must be defined in all cases.
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.
It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.
Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843
Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.
We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.
Co-authored-by: Mic92 <joerg@thalheim.io>
Fixes https://github.com/NixOS/nix/issues/8946
See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.
We now see exception beeing thrown when remote building in master
because of writing to a non-blocking file descriptor from our json logger.
> #0 0x00007f2ea97aea9c in __pthread_kill_implementation () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #1 0x00007f2ea975c576 in raise () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #2 0x00007f2ea9744935 in abort () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #3 0x00007f2ea99e8c2b in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #4 0x00007f2ea99f820a in __cxxabiv1::__terminate(void (*)()) () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #5 0x00007f2ea99f8275 in std::terminate() () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #6 0x00007f2ea99f84c7 in __cxa_throw () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #7 0x00007f2eaa5035c2 in nix::writeFull (fd=2, s=..., allowInterrupts=true) at ../unix/file-descriptor.cc:43
> #8 0x00007f2eaa5633c4 in nix::JSONLogger::write (this=this@entry=0x249a7d40, json=...) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
> #9 0x00007f2eaa5658d7 in nix::JSONLogger::logEI (this=<optimized out>, ei=...) at ../logging.cc:242
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> #11 nix::handleExceptions (programName="nix", fun=...) at ../shared.cc:343
> #12 0x0000000000465b1f in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/allocator.h:163
> (gdb) frame 10
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> 108 logEI(ei);
So far only drainFD sets the non-blocking flag on a "readable" file descriptor,
while this is a "writeable" file descriptor.
It's not clear to me yet, why we see logs after that point, but it's
also not that bad to handle EAGAIN in read/write functions after all.
We're not replacing `Path` in exposed definitions in many cases, but
just adding alternatives. This will allow us to "top down" change `Path`
to `std::fileysystem::path`, and then we can remove the `Path`-using
utilities which will become unused.
Also add some test files which we forgot to include in the libutil unit
tests `meson.build`.
Co-Authored-By: siddhantCodes <siddhantk232@gmail.com>
this is only used to close non-stdio files in derivation sandboxes. we
may as well encode that in its name, drop the unnecessary integer set,
and use close_range to deal with the actual closing of files. not only
is this clearer, it also makes sandbox setup on linux fast by 1ms each
(cherry-picked and adapted from
c7d97802e4)
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
This ended up motivating a good deal of other infra improvements in
order to get Windows right:
- `OsString` to complement `std::filesystem::path`
- env var code for working with the underlying `OsString`s
- Rename `PATHNG_LITERAL` to `OS_STR`
- `NativePathTrait` renamed to `OsPathTrait`, given a character template
parameter until #9205 is complete.
Split `tests.cc` matching split of `util.{cc,hh}` last year.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
The idea is two-fold:
- Replace autotools with Meson
- Build each library in its own derivation
The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.
I tested the Linux native build, and NetBSD and Windows cross builds.
Also do some clean ups of the Flake in the process of supporting new
jobs.
Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.
Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
- Get a rump derivation goal: hook instance will come later, local
derivation goal will come after that.
- Start cleaning up the channel / waiting code with an abstraction.
This makes for shorter and more portable code.
The only tricky part is catching exceptions: I just searched for near by
`catch (Error &)` or `catch (SysError &)` and adjusted them to `catch
(std::filesystem::filesystem_error &)` according to my human judgement.
Good for windows portability; will help @siddhantk232 with his GSOC
project.
Windows now has some basic Unix Domain Socket support, see
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/
Building `nix daemon` on Windows I've left for later, because the daemon
currently forks per connection but this is not an option on Windows. But
we can get the client part working right away.
At this point many features are stripped out, but this works:
- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands
Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
