android_kernel_msm-6.1_noth.../tools/testing/selftests/resctrl
Ilpo Järvinen 9ea5df7290 selftests/resctrl: Reduce failures due to outliers in MBA/MBM tests
commit ef43c30858754d99373a63dff33280a9969b49bc upstream.

The initial value of 5% chosen for the maximum allowed percentage
difference between resctrl mbm value and IMC mbm value in

commit 06bd03a57f ("selftests/resctrl: Fix MBA/MBM results reporting
       format") was "randomly chosen value" (as admitted by the changelog).

When running tests in our lab across a large number platforms, 5%
difference upper bound for success seems a bit on the low side for the
MBA and MBM tests. Some platforms produce outliers that are slightly
above that, typically 6-7%, which leads MBA/MBM test frequently
failing.

Replace the "randomly chosen value" with a success bound that is based
on those measurements across large number of platforms by relaxing the
MBA/MBM success bound to 8%. The relaxed bound removes the failures due
the frequent outliers.

Fixed commit description style error during merge:
Shuah Khan <skhan@linuxfoundation.org>

Fixes: 06bd03a57f ("selftests/resctrl: Fix MBA/MBM results reporting format")
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-11-28 17:07:13 +00:00
..
.gitignore selftests/resctrl: Create .gitignore to include resctrl_tests 2021-04-02 13:58:42 -06:00
cache.c selftests/resctrl: Close perf value read fd on errors 2023-09-13 09:42:28 +02:00
cat_test.c selftests/resctrl: Allow ->setup() to return errors 2023-05-11 23:03:06 +09:00
cmt_test.c selftests/resctrl: Remove duplicate feature check from CMT test 2023-11-28 17:07:13 +00:00
config selftests/resctrl: Add config dependencies 2021-04-02 13:57:42 -06:00
fill_buf.c selftests/resctrl: Don't leak buffer in fill_cache() 2023-09-13 09:42:28 +02:00
Makefile selftests/resctrl: Move _GNU_SOURCE define into Makefile 2023-11-28 17:07:13 +00:00
mba_test.c selftests/resctrl: Reduce failures due to outliers in MBA/MBM tests 2023-11-28 17:07:13 +00:00
mbm_test.c selftests/resctrl: Reduce failures due to outliers in MBA/MBM tests 2023-11-28 17:07:13 +00:00
README selftests/resctrl: Update README about using kselftest framework to build/run resctrl_tests 2022-04-25 17:11:41 -06:00
resctrl.h selftests/resctrl: Move _GNU_SOURCE define into Makefile 2023-11-28 17:07:13 +00:00
resctrl_tests.c selftests/resctrl: Ensure the benchmark commands fits to its array 2023-11-20 11:52:03 +01:00
resctrl_val.c selftests/resctrl: Allow ->setup() to return errors 2023-05-11 23:03:06 +09:00
resctrlfs.c selftests/resctrl: Extend CPU vendor detection 2022-04-25 16:57:50 -06:00
settings selftests/resctrl: Change the default limited time to 120 seconds 2022-04-25 17:06:53 -06:00

resctrl_tests - resctrl file system test suit

Authors:
	Fenghua Yu <fenghua.yu@intel.com>
	Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>,

resctrl_tests tests various resctrl functionalities and interfaces including
both software and hardware.

Currently it supports Memory Bandwidth Monitoring test and Memory Bandwidth
Allocation test on Intel RDT hardware. More tests will be added in the future.
And the test suit can be extended to cover AMD QoS and ARM MPAM hardware
as well.

resctrl_tests can be run with or without kselftest framework.

WITH KSELFTEST FRAMEWORK
=======================

BUILD
-----

Build executable file "resctrl_tests" from top level directory of the kernel source:
 $ make -C tools/testing/selftests TARGETS=resctrl

RUN
---

Run resctrl_tests as sudo or root since the test needs to mount resctrl file
system and change contents in the file system.
Using kselftest framework will run all supported tests within resctrl_tests:

 $ sudo make -C tools/testing/selftests TARGETS=resctrl run_tests

More details about kselftest framework can be found in
Documentation/dev-tools/kselftest.rst.

WITHOUT KSELFTEST FRAMEWORK
===========================

BUILD
-----

Build executable file "resctrl_tests" from this directory(tools/testing/selftests/resctrl/):
  $ make

RUN
---

Run resctrl_tests as sudo or root since the test needs to mount resctrl file
system and change contents in the file system.
Executing the test without any parameter will run all supported tests:

 $ sudo ./resctrl_tests

OVERVIEW OF EXECUTION
=====================

A test case has four stages:

  - setup: mount resctrl file system, create group, setup schemata, move test
    process pids to tasks, start benchmark.
  - execute: let benchmark run
  - verify: get resctrl data and verify the data with another source, e.g.
    perf event.
  - teardown: umount resctrl and clear temporary files.

ARGUMENTS
=========

Parameter '-h' shows usage information.

usage: resctrl_tests [-h] [-b "benchmark_cmd [options]"] [-t test list] [-n no_of_bits]
        -b benchmark_cmd [options]: run specified benchmark for MBM, MBA and CMT default benchmark is builtin fill_buf
        -t test list: run tests specified in the test list, e.g. -t mbm,mba,cmt,cat
        -n no_of_bits: run cache tests using specified no of bits in cache bit mask
        -p cpu_no: specify CPU number to run the test. 1 is default
        -h: help