From 500484f5be022b113cdfb97287e3491a149435cd Mon Sep 17 00:00:00 2001 From: Kalesh Singh Date: Tue, 23 May 2023 13:59:21 -0700 Subject: [PATCH] BACKPORT: FROMGIT: Multi-gen LRU: fix workingset accounting On Android app cycle workloads, MGLRU showed a significant reduction in workingset refaults although pgpgin/pswpin remained relatively unchanged. This indicated MGLRU may be undercounting workingset refaults. This has impact on userspace programs, like Android's LMKD, that monitor workingset refault statistics to detect thrashing. It was found that refaults were only accounted if the MGLRU shadow entry was for a recently evicted folio. However, recently evicted folios should be accounted as workingset activation, and refaults should be accounted regardless of recency. Fix MGLRU's workingset refault and activation accounting to more closely match that of the conventional active/inactive LRU. Link: https://lkml.kernel.org/r/20230523205922.3852731-1-kaleshsingh@google.com Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation") Signed-off-by: Kalesh Singh Reported-by: Charan Teja Kalla Acked-by: Yu Zhao Cc: Brian Geffon Cc: Jan Alexander Steffens (heftig) Cc: Oleksandr Natalenko Cc: Suren Baghdasaryan Signed-off-by: Andrew Morton (cherry picked from commit 02ad728453d2ddb09d7ce5e59854ebb27544d488 https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-unstable) Bug: 284043217 [ Kalesh Singh - Fix conflicts in mm/workingset.c ] Signed-off-by: Kalesh Singh Change-Id: I6d42cca9064e66099fbbc20aa2143961f84b2003 --- mm/vmscan.c | 1 - mm/workingset.c | 4 +++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index c2ca5bec1f92..1c289d07255f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4859,7 +4859,6 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) WRITE_ONCE(lrugen->protected[hist][type][tier - 1], lrugen->protected[hist][type][tier - 1] + delta); - __mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta); return true; } diff --git a/mm/workingset.c b/mm/workingset.c index 688aaa73f64e..d9f2edd6686e 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -272,6 +272,8 @@ static void lru_gen_refault(struct folio *folio, void *shadow) lruvec = mem_cgroup_lruvec(memcg, pgdat); lrugen = &lruvec->lrugen; + mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta); + min_seq = READ_ONCE(lrugen->min_seq[type]); if ((token >> LRU_REFS_WIDTH) != (min_seq & (EVICTION_MASK >> LRU_REFS_WIDTH))) goto unlock; @@ -282,7 +284,7 @@ static void lru_gen_refault(struct folio *folio, void *shadow) tier = lru_tier_from_refs(refs); atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]); - mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta); + mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta); /* * Count the following two cases as stalls: