Commit graph

571 commits

Author SHA1 Message Date
Jiewen Wang
1a40d683e8 ANDROID: vendor_hooks: Add hook in try_to_unmap_one()
Add hook in try_to_unmap_one() to trace this function for debug memory
swap bugs.

Bug: 198385827
Change-Id: I1fdbe60e09bb491b949e06a07133710453ecca03
Signed-off-by: Jiewen Wang <jiewen.wang@vivo.com>
(cherry picked from commit 5a70133febf92e1269f98b533ce9f188c2e3b135)
2023-05-17 10:04:17 +00:00
Jiewen Wang
190af40844 ANDROID: vendor_hooks: Add hook in mmap_region()
Add hook in mmap_region() to record the vma and address information
of monitored processes.

Bug: 198385827
Change-Id: I0bde29113b47ca7f4a9f5d42a54188e791ca3b7e
Signed-off-by: Jiewen Wang <jiewen.wang@vivo.com>
(cherry picked from commit 73c9d4a9d575107b90a6d9f415fa56f963264d06)
2023-05-17 10:04:17 +00:00
heshuai1
a9a44851ec ANDROID: freezer: Add vendor hook to freezer for GKI purpose.
Add the vendor hook to freezer.c so that OEM's logic can be executed
when the process is about to be frozen. We need to clear the flag for
some tasks and rebind task dependencies for optimization purposes.

Bug: 187458531
Bug: 281920779

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: Iea42fd9604d6b33ccd6502425416f0dd28eecebb
(cherry picked from commit a1580311c36ca28344b2f03b3c8a72d9f8db5bde)
2023-05-17 00:25:38 +00:00
Zhuguangqing
17fff41db8 ANDROID: Add vendor hooks for binder perf tuning
Add some hooks in the binder module so that we can do task dependency analysis and statistical work in OEM's module for further optimization.

Bug: 235925535
Bug: 281920779

Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: Id47e59c4e3ccd07b26eef758ada147b98cd1964e
Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
[ cmllamas: don't export complete private definition struct binder_alloc
  in vendor hooks, instead just pass member alloc->free_async_space as
  implemented by heshuai1 and squashed here ]
(cherry picked from commit 254fb1f4034fde523378ee58a501d212a59047b7)
2023-05-16 21:44:18 +00:00
Zhuguangqing
bf4922727c ANDROID: Add vendor hooks to signal.
This hook will allow us to get signal messages so that we can set
limitations for certain tasks and restore them when receiving important signals.

Bug: 184898838
Bug: 281920779

Signed-off-by: Zhuguangqing <zhuguangqing@xiaomi.com>
Change-Id: I83a28b0a6eb413976f4c57f2314d008ad792fa0d
(cherry picked from commit 58e3f869fc3fe84fb7062496ccd049db47f3ed7f)
2023-05-16 21:44:18 +00:00
lijianzhong
8947e06ff7 ANDROID: cgroup: Add vendor hook for cpuset.
This hook allows us to capture information when a process is forked so
that we can stat and set some key task's CPU affinity in the ko module
later. This patch, along with aosp/2548175, is necessary for our
affinity settings.

Bug: 183674818
Signed-off-by: lijianzhong <lijianzhong@xiaomi.com>
Change-Id: Ib93e05e5f6c338c5f7ada56bfebdd705f87f1f66
(cherry picked from commit a188361628461c58a4dfc72869d9acb1dfa2542f)
2023-05-11 13:10:00 +00:00
xieliujie
22fd6b3625 ANDROID: vendor_hooks: Add hooks for futex
We want to use this hook to record the sleeping time due to Futex

Bug: 210947226
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
Change-Id: I637f889dce42937116d10979e0c40fddf96cd1a2
(cherry picked from commit a7ab784f601a93a78c1c22cd0aacc2af64d8e3c8)
2023-05-11 05:22:29 +00:00
xieliujie
16f2ad77c7 ANDROID: vendor_hooks: Add hooks for oem futex optimization
If an important task is going to sleep through do_futex(),
find out it's futex-owner by the pid comes from userspace,
and boost the owner by some means to shorten the sleep time.
How to boost? Depends on these hooks:
commit 53e809978443 ("ANDROID: vendor_hooks: Add hooks for scheduler")

Bug: 243110112
Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I9a315cfb414fd34e0ef7a2cf9d57df50d4dd984f
(cherry picked from commit 548da5d23d98b796cf9a478675622a606b3307c8)
2023-05-11 05:22:29 +00:00
heshuai1
0ea0d6a7a2 ANDROID: power: Add vendor hook to qos for GKI purpose.
Add vendor hooks in add/update/remove frequency QoS request process to
ensure that we can access the OEM's "frequency watchdog" logic for
abnormal frequency monitoring. This is necessary for our power tuning
policy.

Bug: 187458531

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I1fb8fd6134432ecfb44ad242c66ccd8280ab9b43
(cherry picked from commit c445fe4dc67ad74dacfa548bc78876a7ce057086)
2023-05-11 05:22:29 +00:00
heshuai1
4b87d7254b ANDROID: user: Add vendor hook to user for GKI purpose
In order to implement our scheduling tuning policy in certain cases, we
need to initialize the variables that we have defined in the
user_struct. To achieve this, we will add a vendor hook to user.c at
alloc_uid, which will ensure that our own logic is executed during the
initialization of the user_struct.

Bug: 187458531

Signed-off-by: heshuai1 <heshuai1@xiaomi.com>
Change-Id: I078484aac2c3d396aba5971d6d0f491652f3781c
(cherry picked from commit c9b8fa644f45e9c99da85d8947f6c7e06771f205)
(cherry picked from commit 9ac0923ef565e4de4e1f35edcba6fcb7e45948c9)
2023-05-11 05:22:29 +00:00
Youngmin Nam
b8bdb49ef4 Revert "Revert "ANDROID: cpufreq: Add a restricted vendor hook for freq transition""
This reverts commit 3a7f9f00f0.

Some module driver need to know current cpufreq change status
to profile cpu operation.

Bug: 280531400
Change-Id: Ibd7fa6c7e04695d2efc39775767b77b522ba6c06
Signed-off-by: Youngmin Nam <youngmin.nam@samsung.com>
2023-05-09 08:09:49 +00:00
Carlos Llamas
1ca1130ec6 FROMLIST: binder: fix UAF caused by faulty buffer cleanup
In binder_transaction_buffer_release() the 'failed_at' offset indicates
the number of objects to clean up. However, this function was changed by
commit 44d8047f1d ("binder: use standard functions to allocate fds"),
to release all the objects in the buffer when 'failed_at' is zero.

This introduced an issue when a transaction buffer is released without
any objects having been processed so far. In this case, 'failed_at' is
indeed zero yet it is misinterpreted as releasing the entire buffer.

This leads to use-after-free errors where nodes are incorrectly freed
and subsequently accessed. Such is the case in the following KASAN
report:

  ==================================================================
  BUG: KASAN: slab-use-after-free in binder_thread_read+0xc40/0x1f30
  Read of size 8 at addr ffff4faf037cfc58 by task poc/474

  CPU: 6 PID: 474 Comm: poc Not tainted 6.3.0-12570-g7df047b3f0aa #5
  Hardware name: linux,dummy-virt (DT)
  Call trace:
   dump_backtrace+0x94/0xec
   show_stack+0x18/0x24
   dump_stack_lvl+0x48/0x60
   print_report+0xf8/0x5b8
   kasan_report+0xb8/0xfc
   __asan_load8+0x9c/0xb8
   binder_thread_read+0xc40/0x1f30
   binder_ioctl+0xd9c/0x1768
   __arm64_sys_ioctl+0xd4/0x118
   invoke_syscall+0x60/0x188
  [...]

  Allocated by task 474:
   kasan_save_stack+0x3c/0x64
   kasan_set_track+0x2c/0x40
   kasan_save_alloc_info+0x24/0x34
   __kasan_kmalloc+0xb8/0xbc
   kmalloc_trace+0x48/0x5c
   binder_new_node+0x3c/0x3a4
   binder_transaction+0x2b58/0x36f0
   binder_thread_write+0x8e0/0x1b78
   binder_ioctl+0x14a0/0x1768
   __arm64_sys_ioctl+0xd4/0x118
   invoke_syscall+0x60/0x188
  [...]

  Freed by task 475:
   kasan_save_stack+0x3c/0x64
   kasan_set_track+0x2c/0x40
   kasan_save_free_info+0x38/0x5c
   __kasan_slab_free+0xe8/0x154
   __kmem_cache_free+0x128/0x2bc
   kfree+0x58/0x70
   binder_dec_node_tmpref+0x178/0x1fc
   binder_transaction_buffer_release+0x430/0x628
   binder_transaction+0x1954/0x36f0
   binder_thread_write+0x8e0/0x1b78
   binder_ioctl+0x14a0/0x1768
   __arm64_sys_ioctl+0xd4/0x118
   invoke_syscall+0x60/0x188
  [...]
  ==================================================================

In order to avoid these issues, let's always calculate the intended
'failed_at' offset beforehand. This is renamed and wrapped in a helper
function to make it clear and convenient.

Fixes: 32e9f56a96 ("binder: don't detect sender/target during buffer cleanup")
Reported-by: Zi Fan Tan <zifantan@google.com>
Link: https://b.corp.google.com/issues/275041864
Cc: stable@vger.kernel.org
Signed-off-by: Carlos Llamas <cmllamas@google.com>

Bug: 275041864
Link: https://lore.kernel.org/all/20230505203020.4101154-1-cmllamas@google.com
Change-Id: I4bcc8bde77a8118872237d100cccb5caf95d99a1
Signed-off-by: Carlos Llamas <cmllamas@google.com>
2023-05-05 23:19:06 +00:00
Liujie Xie
e0d5caf5c9 ANDROID: vendor_hooks: Add hooks to select binder worklist
trace_android_vh_binder_proc_transaction_entry:
We need change binder thread so that this work can be added in
proc->todo, if we found the binder thread, skip native logic.

trace_android_vh_binder_select_worklist_ilocked:
we need this because we can't change list point in ”trace_android_vh_binder_thread_read“,
otherwise, If a work has beed added in our own defined list before,
current may goto retry and loop again and again.

Bug: 219898723
Change-Id: Ifdb3429c9ddac521bc75c1d21740ee7cc4b8f143
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit acefa91e517b9321295429d3a0e534908e2939c1)
Signed-off-by: Carlos Llamas <cmllamas@google.com>
(cherry picked from commit d26c0e1c409179ea74aaa89f748006bd76000cc3)
2023-05-05 23:07:06 +00:00
Liujie Xie
4619ee1a42 ANDROID: vendor_hooks: Add hooks for binder
We want to add some hooks in the binder module so that we can reduce
block time until binder thread is available
Here are what new hooks do for:
1、android_vh_binder_looper_state_registered: choose a binder thread(do proc work) as a low-level thread.Only this thread has power to excute background binder transaction.
2、android_vh_binder_thread_read: let binder thread do works which come from
our list.
3、android_vh_binder_free_proc: free some pointers and variable.
4、android_vh_binder_thread_release: free the list that we create before.
5、android_vh_binder_has_work_ilocked: to check if our list has work.
6、android_vh_binder_read_done: because of we add hook in binder_has_work_ilocked,
7、android_vh_binder_preset: mark target proc's binder threads.
binder_has_work_ilocked may return true, so we try to wake up low-level thread immediately.

Bug: 212483521
Change-Id: Ic40f452cc4dcf8fc85422e23e6f1a7ad77547309
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit 8d6074509e4ec7e10acf6ee78277ff8cd19e49d0)
2023-05-05 23:07:06 +00:00
Liujie Xie
e80c937cd0 ANDROID: vendor_hooks: Add hooks for rwsem and mutex
Add hooks to apply oem's optimization of rwsem and mutex

Bug: 182237112
Signed-off-by: xieliujie <xieliujie@oppo.com>
(cherry picked from commit 80b4341d0520dce2ef1e33aef06c6a2bc8ada518)

Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I36895c432e5b6d6bff8781b4a7872badb693284c
Signed-off-by: Carlos Llamas <cmllamas@google.com>
[cmllamas: completes the cherry-pick of original commit 80b4341d0520
since commit 0902cc73b793 was only partial]
(cherry picked from commit d4528a28cb5be0c322031f333a6230fa3042931f)
2023-05-05 23:07:06 +00:00
Liujie Xie
61b713df51 ANDROID: vendor_hooks: Add hooks for binder proc transaction
We need pointers to proc and t, the current hooks in binder_proc_transaction
are unable to use.

Bug: 208910215
Change-Id: I730964f965a015e5f5a3e237d9b3bd084b5bd0d0
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit cb7e10d31bca4f247c34d6791d6db048edf7e7a8)
Signed-off-by: Carlos Llamas <cmllamas@google.com>
(cherry picked from commit 8968875ad63f4cda9f263e2a2a930524dd8fdbc7)
2023-05-05 23:07:06 +00:00
xieliujie
40142aa83a ANDROID: vendor_hooks: Add hook for binder
Add hook to support oem's performance feature.

Bug: 186482511

Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: Ib495e80e569cc293eaa98d87a050aee8915eb415
(cherry picked from commit 2337b9185a7e8ebf9748c9fa8eacc0e2b995797b)
(cherry picked from commit c8a2e136151b49d17fd128ccc913723161f9b691)
2023-05-05 23:07:06 +00:00
Liujie Xie
eb74e5b3fe ANDROID: vendor_hooks: Add hooks for mutex and rwsem optimistic spin
These hooks help us do the following things:
a) Record the number of mutex and rwsem optimistic spin.
b) Monitor the time of mutex and rwsem optimistic spin.
c) Make it possible if oems don't want mutex and rwsem to optimistic spin
for a long time.

Bug: 267565260
Change-Id: I2bee30fb17946be85e026213b481aeaeaee2459f
Signed-off-by: Liujie Xie <xieliujie@oppo.com>
(cherry picked from commit d01f7e1269d2651768a0bd91a88f76339a1cb427)
(cherry picked from commit 05b5ff11ad98c5896b352b4c376a84b63684e06c)
2023-05-05 23:07:06 +00:00
xieliujie
86c4152970 ANDROID: vendor_hooks: Add hooks for rwsem and mutex
Add hooks to apply oem's optimization of rwsem and mutex

Bug: 182237112
Signed-off-by: xieliujie <xieliujie@oppo.com>
Change-Id: I6332623732e2d6826b8b61087ca74e55393e0c3d
(cherry picked from commit 0902cc73b793f8b8cc2a80943d85d4ca9b98278b)
Signed-off-by: Carlos Llamas <cmllamas@google.com>
2023-05-05 23:07:06 +00:00
Peifeng Li
6c1c1552e6 ANDROID: vendor_hook: add hooks to protect locking-tsk in cpu scheduler
Providing vendor hooks to record the start time of holding the lock, which
protects rwsem/mutex locking-process from being preemptedfor a short time
in some cases.

- android_vh_record_mutex_lock_starttime
- android_vh_record_rtmutex_lock_starttime
- android_vh_record_rwsem_lock_starttime
- android_vh_record_pcpu_rwsem_starttime

Bug: 241191475

Signed-off-by: Peifeng Li <lipeifeng@oppo.com>
Change-Id: I0e967a1e8b77c32a1ad588acd54028fae2f90c4e
(cherry picked from commit f7294947672eb6b786f3c16b49e71e6a239101ad)
2023-05-05 19:45:34 +00:00
Chris Goldsworthy
342be123fd ANDROID: mm: Create hooks for ZONE_MOVABLE allocs
Create a vendor hook inside of gfp_zone() to modify which allocations
get to enter ZONE_MOVABLE, by zeroing out __GFP_HIGHMEM inside of the
trace hook based on certain conditions.

Separately, create a trace hook in the readahead path to affect the
behavior of the tracehook in gfp_zone().

In 5.15, we had set_skip_swapcache_flags trace-hook in do_swap_page()
but commit ac26e9c7b809 ("ANDROID: cma: allow to use CMA in swap-in path")
added __GFP_CMA explicitly, so the set_skip_swapcache_flags trace hook
is no longer needed.

Note:	To comply with vendor hook guidlines, avoid including types.h in
	trace/hooks/mm.h and use unsigned int for gfp_t.

Bug: 158645321
Change-Id: Idfa6b0b06b1b819d706c847e702bc94ddf7aa55a
Signed-off-by: Chris Goldsworthy <cgoldswo@codeaurora.org>
Signed-off-by: Sukadev Bhattiprolu <quic_sukadev@quicinc.com>
2023-04-26 17:01:52 +00:00
Charan Teja Reddy
88153d9a99 ANDROID: vmscan: Support multiple kswapd threads per node
Page replacement is handled in the Linux Kernel in one of two ways:

1) Asynchronously via kswapd
2) Synchronously, via direct reclaim

At page allocation time the allocating task is immediately given a page
from the zone free list allowing it to go right back to work doing
whatever it was doing; Probably directly or indirectly executing business
logic.

Just prior to satisfying the allocation, free pages is checked to see if
it has reached the zone low watermark and if so, kswapd is awakened.
Kswapd will start scanning pages looking for inactive pages to evict to
make room for new page allocations. The work of kswapd allows tasks to
continue allocating memory from their respective zone free list without
incurring any delay.

When the demand for free pages exceeds the rate that kswapd tasks can
supply them, page allocation works differently. Once the allocating task
finds that the number of free pages is at or below the zone min watermark,
the task will no longer pull pages from the free list. Instead, the task
will run the same CPU-bound routines as kswapd to satisfy its own
allocation by scanning and evicting pages. This is called a direct reclaim.

The time spent performing a direct reclaim can be substantial, often
taking tens to hundreds of milliseconds for small order0 allocations to
half a second or more for order9 huge-page allocations. In fact, kswapd is
not actually required on a linux system. It exists for the sole purpose of
optimizing performance by preventing direct reclaims.

When memory shortfall is sufficient to trigger direct reclaims, they can
occur in any task that is running on the system. A single aggressive
memory allocating task can set the stage for collateral damage to occur in
small tasks that rarely allocate additional memory. Consider the impact of
injecting an additional 100ms of latency when nscd allocates memory to
facilitate caching of a DNS query.

The presence of direct reclaims 10 years ago was a fairly reliable
indicator that too much was being asked of a Linux system. Kswapd was
likely wasting time scanning pages that were ineligible for eviction.
Adding RAM or reducing the working set size would usually make the problem
go away. Since then hardware has evolved to bring a new struggle for
kswapd. Storage speeds have increased by orders of magnitude while CPU
clock speeds stayed the same or even slowed down in exchange for more
cores per package. This presents a throughput problem for a single
threaded kswapd that will get worse with each generation of new hardware.

Test Details

NOTE: The tests below were run with shadow entries disabled. See the
associated patch and cover letter for details

The tests below were designed with the assumption that a kswapd bottleneck
is best demonstrated using filesystem reads. This way, the inactive list
will be full of clean pages, simplifying the analysis and allowing kswapd
to achieve the highest possible steal rate. Maximum steal rates for kswapd
are likely to be the same or lower for any other mix of page types on the
system.

Tests were run on a 2U Oracle X7-2L with 52 Intel Xeon Skylake 2GHz cores,
756GB of RAM and 8 x 3.6 TB NVMe Solid State Disk drives. Each drive has
an XFS file system mounted separately as /d0 through /d7. SSD drives
require multiple concurrent streams to show their potential, so I created
eleven 250GB zero-filled files on each drive so that I could test with
parallel reads.

The test script runs in multiple stages. At each stage, the number of dd
tasks run concurrently is increased by 2. I did not include all of the
test output for brevity.

During each stage dd tasks are launched to read from each drive in a round
robin fashion until the specified number of tasks for the stage has been
reached. Then iostat, vmstat and top are started in the background with 10
second intervals. After five minutes, all of the dd tasks are killed and
the iostat, vmstat and top output is parsed in order to report the
following:

CPU consumption
- sy - aggregate kernel mode CPU consumption from vmstat output. The value
       doesn't tend to fluctuate much so I just grab the highest value.
       Each sample is averaged over 10 seconds
- dd_cpu - for all of the dd tasks averaged across the top samples since
           there is a lot of variation.

Throughput
- in Kbytes
- Command is iostat -x -d 10 -g total

This first test performs reads using O_DIRECT in order to show the maximum
throughput that can be obtained using these drives. It also demonstrates
how rapidly throughput scales as the number of dd tasks are increased.

The dd command for this test looks like this:

Command Used: dd iflag=direct if=/d${i}/$n of=/dev/null bs=4M

Test #1: Direct IO
dd sy dd_cpu throughput
6  0  2.33   14726026.40
10 1  2.95   19954974.80
16 1  2.63   24419689.30
22 1  2.63   25430303.20
28 1  2.91   26026513.20
34 1  2.53   26178618.00
40 1  2.18   26239229.20
46 1  1.91   26250550.40
52 1  1.69   26251845.60
58 1  1.54   26253205.60
64 1  1.43   26253780.80
70 1  1.31   26254154.80
76 1  1.21   26253660.80
82 1  1.12   26254214.80
88 1  1.07   26253770.00
90 1  1.04   26252406.40

Throughput was close to peak with only 22 dd tasks. Very little system CPU
was consumed as expected as the drives DMA directly into the user address
space when using direct IO.

In this next test, the iflag=direct option is removed and we only run the
test until the pgscan_kswapd from /proc/vmstat starts to increment. At
that point metrics are parsed and reported and the pagecache contents are
dropped prior to the next test. Lather, rinse, repeat.

Test #2: standard file system IO, no page replacement
dd sy dd_cpu throughput
6  2  28.78  5134316.40
10 3  31.40  8051218.40
16 5  34.73  11438106.80
22 7  33.65  14140596.40
28 8  31.24  16393455.20
34 10 29.88  18219463.60
40 11 28.33  19644159.60
46 11 25.05  20802497.60
52 13 26.92  22092370.00
58 13 23.29  22884881.20
64 14 23.12  23452248.80
70 15 22.40  23916468.00
76 16 22.06  24328737.20
82 17 20.97  24718693.20
88 16 18.57  25149404.40
90 16 18.31  25245565.60

Each read has to pause after the buffer in kernel space is populated while
those pages are added to the pagecache and copied into the user address
space. For this reason, more parallel streams are required to achieve peak
throughput. The copy operation consumes substantially more CPU than direct
IO as expected.

The next test measures throughput after kswapd starts running. This is the
same test only we wait for kswapd to wake up before we start collecting
metrics. The script actually keeps track of a few things that were not
mentioned earlier. It tracks direct reclaims and page scans by watching
the metrics in /proc/vmstat. CPU consumption for kswapd is tracked the
same way it is tracked for dd.

Since the test is 100% reads, you can assume that the page steal rate for
kswapd and direct reclaims is almost identical to the scan rate.

Test #3: 1 kswapd thread per node
dd sy dd_cpu kswapd0 kswapd1 throughput  dr    pgscan_kswapd pgscan_direct
10 4  26.07  28.56   27.03   7355924.40  0     459316976     0
16 7  34.94  69.33   69.66   10867895.20 0     872661643     0
22 10 36.03  93.99   99.33   13130613.60 489   1037654473    11268334
28 10 30.34  95.90   98.60   14601509.60 671   1182591373    15429142
34 14 34.77  97.50   99.23   16468012.00 10850 1069005644    249839515
40 17 36.32  91.49   97.11   17335987.60 18903 975417728     434467710
46 19 38.40  90.54   91.61   17705394.40 25369 855737040     582427973
52 22 40.88  83.97   83.70   17607680.40 31250 709532935     724282458
58 25 40.89  82.19   80.14   17976905.60 35060 657796473     804117540
64 28 41.77  73.49   75.20   18001910.00 39073 561813658     895289337
70 33 45.51  63.78   64.39   17061897.20 44523 379465571     1020726436
76 36 46.95  57.96   60.32   16964459.60 47717 291299464     1093172384
82 39 47.16  55.43   56.16   16949956.00 49479 247071062     1134163008
88 42 47.41  53.75   47.62   16930911.20 51521 195449924     1180442208
90 43 47.18  51.40   50.59   16864428.00 51618 190758156     1183203901

In the previous test where kswapd was not involved, the system-wide kernel
mode CPU consumption with 90 dd tasks was 16%. In this test CPU consumption
with 90 tasks is at 43%. With 52 cores, and two kswapd tasks (one per NUMA
node), kswapd can only be responsible for a little over 4% of the increase.
The rest is likely caused by 51,618 direct reclaims that scanned 1.2
billion pages over the five minute time period of the test.

Same test, more kswapd tasks:

Test #4: 4 kswapd threads per node
dd sy dd_cpu kswapd0 kswapd1 throughput  dr    pgscan_kswapd pgscan_direct
10 5  27.09  16.65   14.17   7842605.60  0     459105291     0
16 10 37.12  26.02   24.85   11352920.40 15    920527796     358515
22 11 36.94  37.13   35.82   13771869.60 0     1132169011     0
28 13 35.23  48.43   46.86   16089746.00 0     1312902070     0
34 15 33.37  53.02   55.69   18314856.40 0     1476169080     0
40 19 35.90  69.60   64.41   19836126.80 0     1629999149     0
46 22 36.82  88.55   57.20   20740216.40 0     1708478106     0
52 24 34.38  93.76   68.34   21758352.00 0     1794055559     0
58 24 30.51  79.20   82.33   22735594.00 0     1872794397     0
64 26 30.21  97.12   76.73   23302203.60 176   1916593721     4206821
70 33 32.92  92.91   92.87   23776588.00 3575  1817685086     85574159
76 37 31.62  91.20   89.83   24308196.80 4752  1812262569     113981763
82 29 25.53  93.23   92.33   24802791.20 306   2032093122     7350704
88 43 37.12  76.18   77.01   25145694.40 20310 1253204719     487048202
90 42 38.56  73.90   74.57   22516787.60 22774 1193637495     545463615

By increasing the number of kswapd threads, throughput increased by ~50%
while kernel mode CPU utilization decreased or stayed the same, likely due
to a decrease in the number of parallel tasks at any given time doing page
replacement.

Signed-off-by: Buddy Lumpkin <buddy.lumpkin@oracle.com>
Bug: 201263306
Link: https://lore.kernel.org/lkml/1522661062-39745-1-git-send-email-buddy.lumpkin@oracle.com
[charante@codeaurora.org]: Changes made to select number of kswapds through uapi
Signed-off-by: Charan Teja Reddy <charante@codeaurora.org>
[quic_vjitta@quicinc.com]: Changes made to move multiple kswapd threads logic to vendor hooks
Signed-off-by: Vijayanand Jitta <quic_vjitta@quicinc.com>
(cherry picked from commit 0d61a651e4dd3c61d1658cc92e0b0450c8374738)

Change-Id: I8425cab7f40cbeaf65af0ea118c1a9ac7da0930e
[quic_vjitta@quicinc.com]: Resolved minor merge conflicts
Signed-off-by: Vijayanand Jitta <quic_vjitta@quicinc.com>
2023-04-26 17:01:51 +00:00
Kuan-Ying Lee
b28620e7db ANDROID: module: Add vendor hooks
Add vendor hook for module init, so we can get memory type and
use it to do memory type check for architecture
dependent page table setting. To make sure the architecture
dependent tables are created correctly, we need to know when
module parts are initialized and their attributes.

For releasing modules, corresponding tables and attributes should be
destroyed and restored.

These hooks may be invoked in non-atomic context, so it's
necessary to use restricted ones.

Bug: 248994334
Change-Id: Ie9f415c36bca1fb98e021522b627e562d27cdef4
Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>
2023-04-18 19:28:00 +00:00
Kuan-Ying Lee
05b36413b6 ANDROID: kernel: Add restricted vendor hook in creds
Add restricted vendor hook for creds, so we get the creds
information to monitor cred lifetime. During the lifetime,
we store the creds information in a standalone protected
memory and keep track of integrity.

These hooks may be invoked in non-atomic context, so it's
necessary to use restricted ones.

Bug: 248994334
Change-Id: I57fbb759452302fa1ba1e720c76bfe671eab96b5
Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>
2023-04-18 19:28:00 +00:00
Badhri Jagan Sridharan
bbc22694d4 ANDROID: vendor_hooks: Add vendor hook for tcpm logs
Context: https://lore.kernel.org/lkml/CAPTae5LbWVc4Bdiwe69cwwxEGfSYvRv=419dUGR1u8n-WUkYAA@mail.gmail.com/t/

Logs in /sys/kernel/debug/usb/tcpm* are key to debug issues related to
USB charging or data. However, tcpm logbuffer logs do not wraparound once full. Whereas we want it to wrap around so that we capture relevant info in the bugreport when the user collects one. There is sentiment in upstream to get rid of the logbuffer altogether and move to tracing. But trace events are not default enabled in Android, so that implies, even if user can somehow enable the trace event, user would have repro the issue and collect the bugreport. That would cause inconvenience to the end user. The vendor hooks is needed till upstream either allows wrapping around logs (or) Android has a generic way to default enable trace events which can be captured in bugreport.

bypass_log is set to true by the handler if logging into logbuffer is
not needed.

Bug: 189792358
Bug: 271294543
Change-Id: Icacfed2264b6c49b8e803c62f8bd820a146c169a
Signed-off-by: Badhri Jagan Sridharan <badhri@google.com>
(cherry picked from commit bcfe28b62fb4307f7226d74c9718ba4d33224a66)
2023-04-14 19:25:17 +00:00
Kyle Tso
b499bac439 ANDROID: usb: typec: tcpm: Add vendor hook to modify port src caps
Add a vendor hook to modify the port Source Capabilities. If the caps
are changed and the port state is feasible, start a Power Negotiation
AMS to update the current contract to the new one.

Bug: 201006190
Bug: 206108037
Bug: 271294543
Signed-off-by: Kyle Tso <kyletso@google.com>
Change-Id: I8b524242353781cb39b3ba85c6a75bd5a204ebcf
(cherry picked from commit 00e3210a2faa2057fc142367e7e1be886479cf02)
2023-04-14 17:11:19 +00:00
Kyle Tso
c8314fcb76 ANDROID: usb: typec: tcpm: Add vendor hook to store partner source capabilities
Add a hook after receiving the source capabilities from the partner
port. Pass the address of the source capability array so that vendor
code is able to access them.

OOT bug:
Bug: 181629276
Bug: 169215197
Bug: 271294543
Signed-off-by: Kyle Tso <kyletso@google.com>
Change-Id: I11c4a7919c248690e63c3bfbebfa6b8d711175a6
(cherry picked from commit af7214914ab9ead9356652eed35004482265372d)
2023-04-14 17:11:19 +00:00
Badhri Jagan Sridharan
081a88febb ANDROID: usb: typec: tcpm: vendor hook for timer adjustments
linux/usb/pd.h has a bunch of timers for which the Type-C spec defines
a range of values. These values have to be tuned based on the latency
observed in a specific architecture. However, linux opensource sets
them to a specific value without providing a mechanism to set board
specific values. While a generic way is figured out, a vendor hook
is needed in the interim.

For instance, tCCDebounce can have a value between 100msec - 200msec.

Squashed <d98a1df78740eedcc4d3d4c8d44b39b73f81df2d>
"ANDROID: usb: typec: tcpm: Fixup the wait time for SNK_DISCOVERY"

OOT_bug:
Bug: 184308605
Bug: 168245874
Bug: 173252019
Bug: 271294543
Change-Id: I278b34654a7e48990b6ebe25fbe17e3aa4165098
Signed-off-by: Badhri Jagan Sridharan <badhri@google.com>
(cherry picked from commit f694171a837e4dcc03b8e865c973dc3216961233)
2023-04-14 17:11:19 +00:00
Badhri Jagan Sridharan
eeffe02c96 ANDROID: usb: typec: tcpci: Add vendor hook to mask vbus present
This hook is for addressing hardware anomalies where
TCPC_POWER_STATUS_VBUS_PRES bit can return 0 even before falling
below sSinkDisconnect threshold.

Handler has to set bypass to override the value that would otherwise
be returned by this function.
Handler can set vbus or clear vbus to indicate vbus present or absent

Squashed <89f633a9f41259a168473917674dd5cb1bfb991d>
"ANDROID: usb: typec: tcpci: Migrate restricted vendor hook"

OOT_bug:
Bug: 183149028
Bug: 168245874
Bug: 173252019
Bug: 271294543
Signed-off-by: Badhri Jagan Sridharan <badhri@google.com>
Change-Id: I2ca994d49b37bf8600c5913b892fde9acd0dc896
2023-04-14 17:11:19 +00:00
Badhri Jagan Sridharan
b6325c997c ANDROID: usb: typec: tcpci: Add vendor hooks for tcpci interface
Needed for controlling BC1.2 detection logic specific to the chip architecture.
Also, needed to implement additional logic to make debug accessorires
specifically designed for Pixel work. These are outside the purview of Type-C spec.

OOT_bug:
Bug: 169213252
Bug: 168245874
Bug: 173252019
Bug: 271294543
Signed-off-by: Badhri Jagan Sridharan <badhri@google.com>
Change-Id: I36fe75dddd8cd4e2054db01ed4fee7ea08dd8702
(cherry picked from commit 08879ea0d6b55ada45b43c015f1d075629c0c460)
2023-04-14 17:11:19 +00:00
Sachin Gupta
e29773ec20 ANDROID: scsi: ufs: Add hook to influence the UFS clock scaling policy
Add a vendor hook in ufshcd_devfreq_target() to allow vendor modules to
influence the UFS clock scaling policy.

Bug: 277668337

Change-Id: I64b9b77661fffc6cc96fa610985483e6cfc6542b
Signed-off-by: Nitin Rawat <quic_nitirawa@quicinc.com>
Signed-off-by: Sachin Gupta <quic_sachgupt@quicinc.com>
Signed-off-by: Ed Tsai <ed.tsai@mediatek.com>
(cherry picked from commit c8ed7439af7af36412f8ced3979772cf3a529ec4)
2023-04-12 18:38:23 +00:00
Greg Kroah-Hartman
3a7f9f00f0 Revert "ANDROID: cpufreq: Add a restricted vendor hook for freq transition"
This reverts commit d723168991.

The hook it added:
	android_rvh_cpufreq_transition
is not used by anyone, so remove it.  If it is needed in the future it
can be submitted for review again.

Bug: 181889516
Cc: Pavankumar Kondeti <quic_pkondeti@quicinc.com>
Change-Id: I770ef330c0e1e6095b74c0a22845f20b4b71fd70
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-03-31 18:25:45 +00:00
Greg Kroah-Hartman
7f4246b044 Revert "ANDROID: kernel: Add restricted vendor hook in creds"
This reverts commit 1abc68878a.

The hooks added in that commit:
	android_rvh_commit_creds
	android_rvh_exit_creds
	android_rvh_override_creds
	android_rvh_revert_creds
are not used by anyone, so remove them.

If they are needed in the future, they can be resubmitted for review.

Bug: 181639260
Cc: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>
Change-Id: I1b371d6b0be827e39c5163e2ed2134307d81730a
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-03-31 18:25:45 +00:00
Charan Teja Kalla
d13275473e ANDROID: mm/shmem: Add a trace hook to get the shmem folio
Add a trace hook whose callback be used to fill the folio that can be
used for shmem fs. This VH also takes the 'shmem_inode_info' which
can contain the vendor specific data.

Bug: 273448633
Change-Id: Ia48480bba6dba1ee37a3297b69fd61877dae8dc9
Signed-off-by: Charan Teja Kalla <quic_charante@quicinc.com>
2023-03-30 15:35:48 +00:00
Greg Kroah-Hartman
88df355018 Revert "ANDROID: module: Add vendor hooks"
This reverts commit 60e6687899.

The hooks added in it, android_rvh_set_module_core_rw_nx,
android_rvh_set_module_init_rw_nx,
android_rvh_set_module_permit_before_init, and
android_rvh_set_module_permit_after_init, are not used by any vendor
symbol list, so remove them as they are unused.

Bug: 248994334
Cc: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com>
Change-Id: I39d02510916e2a645526f7d3bfaa3e4066901a3e
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2023-03-29 08:05:07 +00:00
Jack Pham
c3007a3231 ANDROID: sound: usb: Add vendor hooks for connect & disconnect
Add vendor hooks to capture when a USB audio interface is probed
and disconnected.  These will allow for supporting offloading of
audio device handling to a co-processor.

The disconnect hook needs to be restricted as the additional
vendor handling may need to perform operations (such as
wait_for_completion) in non-atomic context.

Upstream discussion ongoing
  https://lore.kernel.org/linux-usb/20230308235751.495-1-quic_wcheng@quicinc.com/

Add the previously used Android vendor hooks and exported symbol changes to android14-6.1 until official patches make it upstream.

Bug: 255912822
Change-Id: Idf1da95556814e0ffbdc5f9fb5a25563ebee9be2
Signed-off-by: Jack Pham <quic_jackp@quicinc.com>
Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
2023-03-22 02:29:10 +00:00
Elliot Berman
e0923fa7d5 ANDROID: gic: Add vendor hook for gic-v3 resume
Add vendor hook for gic-v3 resume to allow vendor to know
debug information about interrupts that are pending when
device wakes up.

Bug: 273930524
Change-Id: Id3298715355c8f592f58393db6aa903b18dd812c
Signed-off-by: Elliot Berman <eberman@codeaurora.org>
Signed-off-by: Vamsi Krishna Lanka <quic_vamslank@quicinc.com>
2023-03-20 10:53:38 -07:00
Manaf Meethalavalappu Pallikunhi
a56bbec0f4 ANDROID: thermal: vendor hook to disable thermal cooling stats
Add vendor hook to thermal to allow vendor to selectively disable
thermal cooling device stats feature based on requirement. It helps
vendor to optimize memory footprint due to this feature especially
for low memory devices.

Bug: 218825214
Change-Id: I2ec72505f03575e09229c54765584614b16a3904
Signed-off-by: Manaf Meethalavalappu Pallikunhi <quic_manafm@quicinc.com>
(cherry picked from commit f6e47fd00f24d5e814d316b03974e970dd87879e)
Signed-off-by: Rashid Zafar <quic_rzafar@quicinc.com>
2023-03-15 20:51:32 +00:00
Sangmoon Kim
bb8c223f2f ANDROID: arm64: add vendor hooks for unusal abort cases
Add hooks to gather data of unsual aborts and summarize it with
other information.

Bug: 273189923

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I74eb36b8551ed9a5e6dc87507939a7f4d81c9c18
(cherry picked from commit 102255d1e45607da700f715c5c100427170751a2)
2023-03-13 20:34:25 +00:00
Sangmoon Kim
2d9a325f0c ANDROID: arm64: add vendor hooks for kernel fault cases
Add hooks to gather data of kernel fault and summarize it with
other information.

Bug: 273189923

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I7d6a66837f2e896a413bd8d878f26928669d96e6
(cherry picked from commit 61d7b6c74146bc73ba1f316f8043ac1c0effb0e9)
2023-03-13 20:34:25 +00:00
Sangmoon Kim
c5ea4db533 ANDROID: power: add vendor hooks for try_to_freeze fail
Add hooks to gather data of unfrozen tasks and summarize it
with other information.

Bug: 273189923

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I61da3d253bd9959c6f06e09c9a35c4b242cedafe
(cherry picked from commit 2232e3fc85534a176e7f8bdfe8c56820d10dc111)
2023-03-13 20:34:25 +00:00
Sangmoon Kim
8635a09118 ANDROID: softlockup: add vendor hook for a softlockup task
Add hook to gather data of softlockup and summarize it with
other information.

Bug: 273189923

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I5263bbd573c3fa4b4c981ac26c943721ce09506d
(cherry picked from commit 5cc613a916fdd4285ba5118a0d3063a32c31fbcb)
2023-03-13 20:34:25 +00:00
Sangmoon Kim
e74b4da791 ANDROID: bug: add vendor hook for bug trap
Add hook to gather data of bug trap and summarize it with other
information.

Bug: 273189923

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I1f347c20629786f9bf0b9c50c7f96b50b4360504
(cherry picked from commit ba7e9d1a771d6e5a07bb242a720e6c95cc98fb0f)
2023-03-13 20:34:25 +00:00
Sangmoon Kim
1e6d82d241 ANDROID: Re-apply vendor hooks for information of blocked tasks
This reverts commit 66330b896c (Revert "ANDROID: vendor_hooks:
add waiting information for blocked tasks")

The original patch has been reverted to resolve merge issues
with 5.18-rc1. This patch adds again the vendor hooks for the
original purpose.

Bug: 271799327

Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I86b9b7dd553b7b6a5930ace6280ecd66dc5dc4df
2023-03-10 18:16:02 +00:00
duwenchao
fcc4497c47 ANDROID: vendor_hooks: Add hooks for ufs scheduler
Add hooks to support oem's ufs scheduler feature.

Bug: 183085044
Signed-off-by: duwenchao <duwenchao@xiaomi.com>
Change-Id: I09bf9cba33412c81e4da83292772c0e95cf32022
(cherry picked from commit 00185a421ad41b6e5dbb57162439d5fa33814e5e)
Signed-off-by: Asutosh Das <quic_asutoshd@quicinc.com>
2023-02-21 21:33:00 +00:00
Guru Das Srinagesh
b2487c5b2b ANDROID: gic: Add affinity-setting vendor hooks to gic-v3
Add vendor hooks to allow vendor enhancements to GIC v3, specifically
implementation-defined multi-CPU IRQ target affinity setting.

The gic_v3_set_affinity() vendor hook is introduced as a restricted
vendor hook in order to allow trace hook to be called from irq migration
paths.

This patch includes content from these patches from previous releases:

commit 00c6f53e03 ("ANDROID: gic-v3: Add vendor hook to GIC v3")
commit fdb01338797f ("ANDROID: gic-v3: Update vendor hook to set affinity in GIC v3")
commit d8b068262090 ("ANDROID: gic-v3: Change GIC v3 vendor hook to restricted")
commit fbee76aa8d67 ("ANDROID: gic-v3: Update vendor hook to set affinity in GIC v3")

Bug: 266450121
Change-Id: I457070ba7ae53e012aa7bac846d2bdb11ef6f95d
Signed-off-by: Guru Das Srinagesh <quic_gurus@quicinc.com>
2023-02-01 20:23:30 +00:00
Guru Das Srinagesh
4cd99f6a6f ANDROID: gic: Add affinity-setting vendor hooks to gic
Add vendor hooks to allow vendor enhancements to GIC, specifically
implementation-defined multi-CPU IRQ target affinity setting.

This patch is a refactored version of the android13-5.15 commit
9c25e5d6f583 ("ANDROID: gic: Add vendor hook to GIC").

Bug: 266450121
Change-Id: I4a17ed20f8e868a6479fd53c0230b22652595b01
Signed-off-by: Guru Das Srinagesh <quic_gurus@quicinc.com>
2023-02-01 20:23:30 +00:00
Huang Yiwei
3e4fa5265c ANDROID: hung_task: Add vendor hook for hung task detect
Add vendor hook for hung task detect, so we can decide which
threads need to check, avoiding false alarms. And the NULL
tracehook is used to indicate one check cycle is finished, so
additional checks can be done after one hung task check cycle.

Bug: 188684133
Change-Id: I5d7dfeb071cbfda8121134c38a458202aaa3a8c6
Signed-off-by: Huang Yiwei <quic_hyiwei@quicinc.com>
2023-01-30 11:14:35 +08:00
Sukadev Bhattiprolu
7b49f180f9 ANDROID: iommu: Define vendor hook to limit max alignment
When the IOVA framework applies IOVA alignment it aligns all IOVAs
to the smallest PAGE_SIZE order which is greater than or equal to
the requested IOVA size.

We support use cases that requires large buffers (> 64 MB in size)
to be allocated and mapped in their stage 1 page tables.  However,
with this alignment scheme we find ourselves running out of IOVA
space for 32 bit devices,

Define a vendor hook to allow limiting the alignment value used when
allocating IOVAs.

We addressed this issue in Android13-5.15 with commit 989b762eb159
("FROMLIST: iommu/iova: Add support for IOVA max alignment tuning")

Bug: 190519428
Change-Id: I13032c1b440c050860109aae23f96c0b52782664
Link: https://lore.kernel.org/r/1634148667-409263-1-git-send-email-quic_c_gdjako@quicinc.com/
Signed-off-by: Sukadev Bhattiprolu <quic_sukadev@quicinc.com>
2023-01-20 18:48:47 +00:00
Yun Hsiang
84d818b0a6 ANDROID: sched: add vendor hook for correcting cpu capacity
Current CPU capacity is scaled by frequency
but it will sometime overestimate.
Add vendor hook to adjust cpu capacity.

android_vh_arch_set_freq_scale
	Adjust cpu freq scale for each opp.

Bug: 163935827
Bug: 265389443

Signed-off-by: Yun Hsiang <yun.hsiang@mediatek.com>
Change-Id: I054740a7f60301d5a3b085bfa540d52853f5eec5
(cherry picked from commit e209b3b88d)
2023-01-19 18:06:37 +00:00