NothingOSS/android_kernel_msm-6.1_nothing_sm7635
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Kernel Source and devicetree for NOTHING Phone(3a) and Phone(3a)Pro
1140962 commits 1 branch 0 tags 2.6 GiB
Find a file
Mateusz Guzik 448e5fbbf9 lockref: stop doing cpu_relax in the cmpxchg loop 
[ Upstream commit f5fe24ef17b5fbe6db49534163e77499fb10ae8c ]

On the x86-64 architecture even a failing cmpxchg grants exclusive
access to the cacheline, making it preferable to retry the failed op
immediately instead of stalling with the pause instruction.

To illustrate the impact, below are benchmark results obtained by
running various will-it-scale tests on top of the 6.2-rc3 kernel and
Cascade Lake (2 sockets * 24 cores * 2 threads) CPU.

All results in ops/s.  Note there is some variance in re-runs, but the
code is consistently faster when contention is present.

  open3 ("Same file open/close"):
  proc          stock       no-pause
     1         805603         814942       (+%1)
     2        1054980        1054781       (-0%)
     8        1544802        1822858      (+18%)
    24        1191064        2199665      (+84%)
    48         851582        1469860      (+72%)
    96         609481        1427170     (+134%)

  fstat2 ("Same file fstat"):
  proc          stock       no-pause
     1        3013872        3047636       (+1%)
     2        4284687        4400421       (+2%)
     8        3257721        5530156      (+69%)
    24        2239819        5466127     (+144%)
    48        1701072        5256609     (+209%)
    96        1269157        6649326     (+423%)

Additionally, a kernel with a private patch to help access() scalability:
access2 ("Same file access"):

  proc          stock        patched      patched
                                         +nopause
    24        2378041        2005501      5370335  (-15% / +125%)

That is, fixing the problems in access itself *reduces* scalability
after the cacheline ping-pong only happens in lockref with the pause
instruction.

Note that fstat and access benchmarks are not currently integrated into
will-it-scale, but interested parties can find them in pull requests to
said project.

Code at hand has a rather tortured history.  First modification showed
up in commit d472d9d98b ("lockref: Relax in cmpxchg loop"), written
with Itanium in mind.  Later it got patched up to use an arch-dependent
macro to stop doing it on s390 where it caused a significant regression.
Said macro had undergone revisions and was ultimately eliminated later,
going back to cpu_relax.

While I intended to only remove cpu_relax for x86-64, I got the
following comment from Linus:

    I would actually prefer just removing it entirely and see if
    somebody else hollers. You have the numbers to prove it hurts on
    real hardware, and I don't think we have any numbers to the
    contrary.

    So I think it's better to trust the numbers and remove it as a
    failure, than say "let's just remove it on x86-64 and leave
    everybody else with the potentially broken code"

Additionally, Will Deacon (maintainer of the arm64 port, one of the
architectures previously benchmarked):

    So, from the arm64 side of the fence, I'm perfectly happy just
    removing the cpu_relax() calls from lockref.

As such, come back full circle in history and whack it altogether.

Signed-off-by: Mateusz Guzik <mjguzik@gmail.com>
Link: https://lore.kernel.org/all/CAGudoHHx0Nqg6DE70zAVA75eV-HXfWyhVMWZ-aSeOofkA_=WdA@mail.gmail.com/
Acked-by: Tony Luck <tony.luck@intel.com> # ia64
Acked-by: Nicholas Piggin <npiggin@gmail.com> # powerpc
Acked-by: Will Deacon <will@kernel.org> # arm64
Acked-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2023-02-01 08:34:34 +01:00
arch KVM: s390: interrupt: use READ_ONCE() before cmpxchg() 2023-02-01 08:34:32 +01:00
block block: mq-deadline: Rename deadline_is_seq_writes() 2023-01-24 07:24:44 +01:00
certs certs: make system keyring depend on built-in x509 parser 2022-09-24 04:31:18 +09:00
crypto crypto: tcrypt - Fix multibuffer skcipher speed test mem leak 2022-12-31 13:32:34 +01:00
Documentation docs: Fix path paste-o for /sys/kernel/warn_count 2023-01-24 07:24:41 +01:00
drivers platform/x86: simatic-ipc: add another model 2023-02-01 08:34:33 +01:00
fs cifs: fix potential memory leaks in session setup 2023-02-01 08:34:32 +01:00
include platform/x86: simatic-ipc: add another model 2023-02-01 08:34:33 +01:00
init gcc: disable -Warray-bounds for gcc-11 too 2023-01-14 10:33:43 +01:00
io_uring io_uring/poll: don't reissue in case of poll race on multishot request 2023-01-24 07:24:37 +01:00
ipc ipc: fix memory leak in init_mqueue_fs() 2022-12-31 13:32:01 +01:00
kernel kcsan: test: don't put the expect array on the stack 2023-02-01 08:34:29 +01:00
lib lockref: stop doing cpu_relax in the cmpxchg loop 2023-02-01 08:34:34 +01:00
LICENSES LICENSES/LGPL-2.1: Add LGPL-2.1-or-later as valid identifiers 2021-12-16 14:33:10 +01:00
mm panic: Consolidate open-coded panic_on_warn checks 2023-01-24 07:24:41 +01:00
net tcp: fix rate_app_limited to default to 1 2023-02-01 08:34:27 +01:00
rust Kbuild: add Rust support 2022-09-28 09:02:20 +02:00
samples samples: vfio-mdev: Fix missing pci_disable_device() in mdpy_fb_probe() 2022-12-31 13:32:42 +01:00
scripts - Handle different output of readelf on different distros running 2022-11-27 12:08:17 -08:00
security tomoyo: fix broken dependency on *.conf.default 2023-02-01 08:34:06 +01:00
sound ASoC: fsl-asoc-card: Fix naming of AC'97 CODEC widgets 2023-02-01 08:34:31 +01:00
tools tools/nolibc: prevent gcc from making memset() loop over itself 2023-02-01 08:34:31 +01:00
usr usr/gen_init_cpio.c: remove unnecessary -1 values from int file 2022-10-03 14:21:44 -07:00
virt Merge branch 'kvm-dwmw2-fixes' into HEAD 2022-11-23 18:59:45 -05:00
.clang-format inet: ping: use hlist_nulls rcu iterator during lookup 2022-12-01 12:42:46 +01:00
.cocciconfig
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore Kbuild: add Rust support 2022-09-28 09:02:20 +02:00
.mailmap 9 hotfixes. 6 for MM, 3 for other areas. Four of these patches address 2022-12-10 17:10:52 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Remove Michal Marek from Kbuild maintainers 2022-11-16 14:53:00 +09:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS panic: Expose "warn_count" to sysfs 2023-01-24 07:24:41 +01:00
Makefile kbuild: fix 'make modules' error when CONFIG_DEBUG_INFO_BTF_MODULES=y 2023-02-01 08:34:08 +01:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.