FROMLIST: rcu: Provide a boot time parameter to control lazy RCU

To allow more flexible arrangements while still provide a single kernel
for distros, provide a boot time parameter to enable/disable lazy RCU.

Specify:

	rcutree.enable_rcu_lazy=[y|1|n|0]

Which also requires

	rcu_nocbs=all

at boot time to enable/disable lazy RCU.

To disable it by default at build time when CONFIG_RCU_LAZY=y, the new
CONFIG_RCU_LAZY_DEFAULT_OFF can be used.

Bug: 258241771
Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io>
Tested-by: Andrea Righi <andrea.righi@canonical.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Link: https://lore.kernel.org/lkml/20231203011252.233748-1-qyousef@layalina.io/
[Fix trivial conflicts rejecting newer code that doesn't exist on 6.1]
Signed-off-by: Qais Yousef <qyousef@google.com>
Change-Id: Ib5585ae717a2ba7749f2802101b785c4e5de8a90
This commit is contained in:
Qais Yousef 2023-12-03 01:12:52 +00:00 committed by Todd Kjos
parent a079cc5876
commit 90d68cedd1
3 changed files with 23 additions and 1 deletions

View file

@ -4882,6 +4882,11 @@
rcu_node tree with an eye towards determining rcu_node tree with an eye towards determining
why a new grace period has not yet started. why a new grace period has not yet started.
rcutree.enable_rcu_lazy= [KNL]
To save power, batch RCU callbacks and flush after
delay, memory pressure or callback list growing too
big.
rcuscale.gp_async= [KNL] rcuscale.gp_async= [KNL]
Measure performance of asynchronous Measure performance of asynchronous
grace-period primitives such as call_rcu(). grace-period primitives such as call_rcu().

View file

@ -318,6 +318,18 @@ config RCU_LAZY
help help
To save power, batch RCU callbacks and flush after delay, memory To save power, batch RCU callbacks and flush after delay, memory
pressure, or callback list growing too big. pressure, or callback list growing too big.
Requires rcu_nocbs=all to be set.
Use rcutree.enable_rcu_lazy=0 to turn it off at boot time.
config RCU_LAZY_DEFAULT_OFF
bool "Turn RCU lazy invocation off by default"
depends on RCU_LAZY
default n
help
Allows building the kernel with CONFIG_RCU_LAZY=y yet keep it default
off. Boot time param rcutree.enable_rcu_lazy=1 can be used to switch
it back on.
config RCU_BOOT_END_DELAY config RCU_BOOT_END_DELAY
int "Minimum time before RCU may consider in-kernel boot as completed" int "Minimum time before RCU may consider in-kernel boot as completed"

View file

@ -2799,6 +2799,9 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in)
} }
#ifdef CONFIG_RCU_LAZY #ifdef CONFIG_RCU_LAZY
static bool enable_rcu_lazy __read_mostly = !IS_ENABLED(CONFIG_RCU_LAZY_DEFAULT_OFF);
module_param(enable_rcu_lazy, bool, 0444);
/** /**
* call_rcu_hurry() - Queue RCU callback for invocation after grace period, and * call_rcu_hurry() - Queue RCU callback for invocation after grace period, and
* flush all lazy callbacks (including the new one) to the main ->cblist while * flush all lazy callbacks (including the new one) to the main ->cblist while
@ -2824,6 +2827,8 @@ void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func)
return __call_rcu_common(head, func, false); return __call_rcu_common(head, func, false);
} }
EXPORT_SYMBOL_GPL(call_rcu_hurry); EXPORT_SYMBOL_GPL(call_rcu_hurry);
#else
#define enable_rcu_lazy false
#endif #endif
/** /**
@ -2872,7 +2877,7 @@ EXPORT_SYMBOL_GPL(call_rcu_hurry);
*/ */
void call_rcu(struct rcu_head *head, rcu_callback_t func) void call_rcu(struct rcu_head *head, rcu_callback_t func)
{ {
return __call_rcu_common(head, func, IS_ENABLED(CONFIG_RCU_LAZY)); __call_rcu_common(head, func, enable_rcu_lazy);
} }
EXPORT_SYMBOL_GPL(call_rcu); EXPORT_SYMBOL_GPL(call_rcu);