3984 commits

Author	SHA1	Message	Date
John Ericson	73c9fc7ab1	Merge remote-tracking branch 'upstream/master' into overlayfs-store	2023-08-02 14:14:11 -04:00
John Ericson	66550878df	Add comment explaining the use of readDirectory(realStoreDir)	2023-08-02 12:46:07 -04:00
John Ericson	770d50e49c	local-store verifying: Rename store to something more clear ... It is not a `Store` but a `StorePathSet`.	2023-08-02 12:40:04 -04:00
Ben Radford	8225b7a011	Merge branch 'master' into overlayfs-store	2023-08-01 12:52:53 +01:00
Ben Radford	c712369ec5	Document remount-hook store parameter.	2023-08-01 12:48:02 +01:00
Ben Radford	c409a753db	Fix new lines in comment.	2023-08-01 12:48:02 +01:00
Ben Radford	1255866e16	Update src/libstore/local-overlay-store.hh ... Co-authored-by: John Ericson <git@JohnEricson.me>	2023-08-01 12:48:02 +01:00
Ben Radford	6a8de4c9dc	Avoid enumerating entire overlay store dir upfront. ... As an optimisation for LocalStore, we read all the store directory entries into a set. Checking for membership of this set is much faster than a stat syscall. However for LocalOverlayStore, the lower store directory is expected to contain a vast number of entries and reading them all can take a very long time. So instead of enumerating them all upfront, we call pathExists as needed. This means making stat syscalls for each store path, but the upper layer is expected to be relatively small compared to the lower store so that should be okay.	2023-08-01 12:48:02 +01:00
Ben Radford	50ce8d15eb	Preparatory refactor of LocalStore::verifyStore.	2023-08-01 12:48:02 +01:00
John Ericson	6525265f46	LocalStore::verifyPath: Try to clarify data flow with more scopes ... It was initially unclear to me which of these are temporary state for the verify paths computation, and which of these are the results of that computation to be used in the rest of the function. Now, it is clear, and enforced.	2023-07-31 12:44:27 -04:00
John Ericson	2a5f5fbb17	LocalStore::verifyPath: Use StorePathSet for store local var ... We don't care about non-store-paths in there (things like `.links`, are, in fact, allowed). So let's just skip them up front and be more strongly typed.	2023-07-31 12:44:18 -04:00
Ben Radford	c9a87ce7ca	Refactor verifyPath to take StorePath instead of Path. ... This way we avoid having to convert from Path to StorePath and vice versa in the body of verifyPath.	2023-07-31 11:17:55 -04:00
John Ericson	621bdbdb31	Merge branch 'overlayfs-store' into delete	2023-07-26 12:01:08 -04:00
Ben Radford	c2d54496a0	Forgot to check flag and early out.	2023-07-26 16:01:20 +01:00
Ben Radford	3a9fe1a085	Made remountRequired atomic to avoid concurrency issues.	2023-07-26 16:01:20 +01:00
Ben Radford	5744a500d6	Use debug instead of writing directly to stderr.	2023-07-26 16:01:20 +01:00
Ben Radford	ed14286924	Invoke remount-hook program when necessary.	2023-07-26 16:01:20 +01:00
Ben Radford	33ebae75ca	Reuse deletion logic for optimiseStore and rename method.	2023-07-26 16:01:20 +01:00
Ben Radford	11c493f8fa	Avoid creating whiteout for duplicate store paths.	2023-07-26 16:01:20 +01:00
Ben Radford	d9688ba708	Add new remount-hook store parameter.	2023-07-26 16:01:20 +01:00
John Ericson	07b34edc44	Fix deletion test ... Lower layer references are ignored for deleting just in the upper layer.	2023-07-25 18:09:02 -04:00
John Ericson	9ef0a9e8aa	Fix hard linking issue causing overlay fs copy-ups	2023-07-25 10:28:11 -04:00
John Ericson	73a55e4005	Merge remote-tracking branch 'layered-store/overlayfs-store' into gc	2023-07-25 09:48:06 -04:00
Ben Radford	497464f494	Extend verify test to check that repair is supported.	2023-07-25 13:33:37 +01:00
John Ericson	0e595a52a3	Remove trailing whitespace	2023-07-24 15:39:59 -04:00
John Ericson	2cabf85b53	Merge remote-tracking branch 'upstream/master' into overlayfs-store	2023-07-24 15:39:36 -04:00
John Ericson	60d8dd7aea	Clean up store hierarchy with IndirectRootStore ... See the API doc comments for details.	2023-07-24 09:19:44 -04:00
John Ericson	13269ba93b	Make RemoteStore::ConnectionHandle part of class and expose ... Will need to do subclass-specific implementations in the next commit. This isn't because there will be multiple variations of the daemon protocol (whew!) but because different clients pick and choose different parts to use.	2023-07-24 09:17:01 -04:00
John Ericson	0a30b07277	Move Store::Params typedef to StoreConfig::Params ... This is because `StoreConfig` also uses it.	2023-07-24 09:03:44 -04:00
John Ericson	fe1fbdb5a1	Merge pull request #8724 from obsidiansystems/queryPartialDerivationOutputMap-evalStore ... Give `queryPartialDerivationOutputMap` an `evalStore` parameter	2023-07-21 08:53:18 -04:00
Eelco Dolstra	7ac24d9525	Merge pull request #8650 from obsidiansystems/content-address-simpler ... Simplify `ContentAddress`	2023-07-21 13:46:53 +02:00
John Ericson	6bc98c7fba	Give queryPartialDerivationOutputMap an evalStore parameter ... This makes it more useful. In general, the derivation will be in one store, and the realisation info is in another. This also helps us avoid duplication. See how `resolveDerivedPath` is now simpler because it uses `queryPartialDerivationOutputMap`. In #8369 we get more flavors of derived path, and need more code to resolve them all, and this problem only gets worse. The fact that we need a new method to deal with the multiple dispatch is unfortunate, but this generally relates to the fact that `Store` is a sub-par interface, too bulky/unwieldy and conflating separate concerns. Solving that is out of scope of this PR. This is part of the RFC 92 work. See tracking issue #6316	2023-07-20 15:59:52 -04:00
John Ericson	f62543fe1c	Remove unneeded copy ... It appeared in 8eb73a8724 (by me!) without justification.	2023-07-20 15:42:06 -04:00
John Ericson	7b30293d38	Tighten #includes: DerivedPath doesn't care about Realisation	2023-07-20 15:42:06 -04:00
Ben Radford	8ddbcb736a	Implement overlay store deduplication.	2023-07-20 10:42:39 +01:00
Ben Radford	614efc1240	Add test for store optimise path deduplication.	2023-07-20 10:42:39 +01:00
Ben Radford	d5cd74a401	Override verifyStore to always pass NoRepair for LocalOverlayStore.	2023-07-20 10:42:39 +01:00
Ben Radford	21b9e15d25	Merge branch 'master' into overlayfs-store	2023-07-19 15:00:47 +01:00
Robert Hensing	32494cbb29	Merge pull request #7973 from fricklerhandwerk/remove-channels ... remove the Channels section	2023-07-19 14:02:26 +02:00
Robert Hensing	eca07b2a47	Merge pull request #8315 from fricklerhandwerk/doc-system ... add information on the system type string	2023-07-19 12:07:23 +02:00
Valentin Gagarin	6c3cd429a6	fix broken links	2023-07-19 11:01:48 +02:00
Valentin Gagarin	0779005f49	expand on the extra-platforms option	2023-07-19 10:57:37 +02:00
Valentin Gagarin	fcadac0a02	mention extra-platforms	2023-07-19 10:38:42 +02:00
Valentin Gagarin	aba32def73	fix wording ... Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>	2023-07-19 10:38:30 +02:00
Valentin Gagarin	32de11923e	add cross-links	2023-07-19 10:37:40 +02:00
Valentin Gagarin	3763c7bb5e	shorten system setting description	2023-07-19 10:37:40 +02:00
Valentin Gagarin	c8a42039ea	move docs of the current system to the system setting ... add information what happens when Nix itself is cross-compiled	2023-07-19 10:37:40 +02:00
Valentin Gagarin	0751c1bfc6	one line per sentence for easier review	2023-07-19 10:37:40 +02:00
Robert Hensing	0e3a7e34a0	Merge pull request #8506 from corngood/ssh-master ... Pass NIX_SSHOPTS when checking for an ssh master connection.	2023-07-18 15:47:57 +02:00
Théophane Hufschmitt	a8d5bb5e7e	Merge pull request #8342 from NixLayeredStore/best-effort-supplementary-groups ... Best effort supplementary groups	2023-07-17 20:58:17 +02:00