linux.git/lib/kunit/executor_test.c, branch v6.6.132

kunit: test: Log the correct filter string in executor_test

2024-03-26T22:19:18+00:00

[ Upstream commit 6f2f793fba78eb4a0d5a34a71bc781118ed923d3 ]

KUnit's executor_test logs the filter string in KUNIT_ASSERT_EQ_MSG(),
but passed a random character from the filter, rather than the whole
string.

This was found by annotating KUNIT_ASSERT_EQ_MSG() to let gcc validate
the format string.

Fixes: 76066f93f1df ("kunit: add tests for filtering attributes")
Signed-off-by: David Gow 
Tested-by: Guenter Roeck 
Reviewed-by: Justin Stitt 
Reviewed-by: Daniel Latypov 
Reviewed-by: Rae Moar 
Signed-off-by: Shuah Khan 
Signed-off-by: Sasha Levin

kunit: test: Fix the possible memory leak in executor_test

2023-11-20T10:59:15+00:00

[ Upstream commit 8040345fdae4cb256c5d981f91ae0f22bea8adcc ]

When CONFIG_KUNIT_ALL_TESTS=y, making CONFIG_DEBUG_KMEMLEAK=y and
CONFIG_DEBUG_KMEMLEAK_AUTO_SCAN=y, the below memory leak is detected.

If kunit_filter_suites() succeeds, not only copy but also filtered_suite
and filtered_suite->test_cases should be freed.

So as Rae suggested, to avoid the suite set never be freed when
KUNIT_ASSERT_EQ() fails and exits after kunit_filter_suites() succeeds,
update kfree_at_end() func to free_suite_set_at_end() to use
kunit_free_suite_set() to free them as kunit_module_exit() and
kunit_run_all_tests() do it. As the second arg got of
free_suite_set_at_end() is a local variable, copy it for free to avoid
wild-memory-access. After applying this patch, the following memory leak
is never detected.

unreferenced object 0xffff8881001de400 (size 1024):
  comm "kunit_try_catch", pid 1396, jiffies 4294720452 (age 932.801s)
  hex dump (first 32 bytes):
    73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00  suite2..........
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc_node_track_caller+0x53/0x150
    [] kmemdup+0x22/0x50
    [] kunit_filter_suites+0x44d/0xcc0
    [] filter_suites_test+0x12f/0x360
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff8881052cd388 (size 192):
  comm "kunit_try_catch", pid 1396, jiffies 4294720452 (age 932.801s)
  hex dump (first 32 bytes):
    a0 85 9e 82 ff ff ff ff 80 cd 7c 84 ff ff ff ff  ..........|.....
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc+0x52/0x150
    [] kunit_filter_suites+0x481/0xcc0
    [] filter_suites_test+0x12f/0x360
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20

unreferenced object 0xffff888100da8400 (size 1024):
  comm "kunit_try_catch", pid 1398, jiffies 4294720454 (age 781.945s)
  hex dump (first 32 bytes):
    73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00  suite2..........
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc_node_track_caller+0x53/0x150
    [] kmemdup+0x22/0x50
    [] kunit_filter_suites+0x44d/0xcc0
    [] filter_suites_test_glob_test+0x12f/0x560
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff888105117878 (size 96):
  comm "kunit_try_catch", pid 1398, jiffies 4294720454 (age 781.945s)
  hex dump (first 32 bytes):
    a0 85 9e 82 ff ff ff ff a0 ac 7c 84 ff ff ff ff  ..........|.....
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc+0x52/0x150
    [] kunit_filter_suites+0x481/0xcc0
    [] filter_suites_test_glob_test+0x12f/0x560
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff888102c31c00 (size 1024):
  comm "kunit_try_catch", pid 1404, jiffies 4294720460 (age 781.948s)
  hex dump (first 32 bytes):
    6e 6f 72 6d 61 6c 5f 73 75 69 74 65 00 00 00 00  normal_suite....
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc_node_track_caller+0x53/0x150
    [] kmemdup+0x22/0x50
    [] kunit_filter_attr_tests+0xf7/0x860
    [] kunit_filter_suites+0x82f/0xcc0
    [] filter_attr_test+0x195/0x5f0
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff8881052cd250 (size 192):
  comm "kunit_try_catch", pid 1404, jiffies 4294720460 (age 781.948s)
  hex dump (first 32 bytes):
    a0 85 9e 82 ff ff ff ff 00 a9 7c 84 ff ff ff ff  ..........|.....
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc+0x52/0x150
    [] kunit_filter_attr_tests+0x1a1/0x860
    [] kunit_filter_suites+0x82f/0xcc0
    [] filter_attr_test+0x195/0x5f0
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff888104f4e400 (size 1024):
  comm "kunit_try_catch", pid 1408, jiffies 4294720464 (age 781.944s)
  hex dump (first 32 bytes):
    73 75 69 74 65 00 00 00 00 00 00 00 00 00 00 00  suite...........
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [] __kmalloc_node_track_caller+0x53/0x150
    [] kmemdup+0x22/0x50
    [] kunit_filter_attr_tests+0xf7/0x860
    [] kunit_filter_suites+0x82f/0xcc0
    [] filter_attr_skip_test+0x133/0x6e0
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20
unreferenced object 0xffff8881052cc620 (size 192):
  comm "kunit_try_catch", pid 1408, jiffies 4294720464 (age 781.944s)
  hex dump (first 32 bytes):
    a0 85 9e 82 ff ff ff ff c0 a8 7c 84 ff ff ff ff  ..........|.....
    00 00 00 00 00 00 00 00 02 00 00 00 02 00 00 00  ................
  backtrace:
    [] __kmalloc+0x52/0x150
    [] kunit_filter_attr_tests+0x1a1/0x860
    [] kunit_filter_suites+0x82f/0xcc0
    [] filter_attr_skip_test+0x133/0x6e0
    [] kunit_generic_run_threadfn_adapter+0x4a/0x90
    [] kthread+0x2b6/0x380
    [] ret_from_fork+0x2d/0x70
    [] ret_from_fork_asm+0x11/0x20

Fixes: e5857d396f35 ("kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suites")
Fixes: 76066f93f1df ("kunit: add tests for filtering attributes")
Signed-off-by: Jinjie Ruan 
Suggested-by: Rae Moar 
Reviewed-by: David Gow 
Suggested-by: David Gow 
Reported-by: kernel test robot 
Closes: https://lore.kernel.org/oe-kbuild-all/202309142251.uJ8saAZv-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202309270433.wGmFRGjd-lkp@intel.com/
Signed-off-by: Shuah Khan 
Signed-off-by: Sasha Levin

kunit: test: Make filter strings in executor_test writable

2023-09-01T15:09:25+00:00

KUnit's attribute filtering feature needs the filter strings passed in
to be writable, as it modifies them in-place during parsing. This works
for the filters passed on the kernel command line, but the string
literals used in the executor tests are at least theoretically read-only
(though they work on x86_64 for some reason). s390 wasn't fooled, and
crashed when these tests were run.

Use a 'char[]' instead, (and make an explicit variable for the current
filter in parse_filter_attr_test), which will store the string in a
writable segment.

Fixes: 76066f93f1df ("kunit: add tests for filtering attributes")
Closes: https://lore.kernel.org/linux-kselftest/55950256-c00a-4d21-a2c0-cf9f0e5b8a9a@roeck-us.net/
Signed-off-by: David Gow 
Tested-by: Guenter Roeck 
Reviewed-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: Report the count of test suites in a module

2023-08-08T19:46:05+00:00

According to KTAP specification[1], results should always start from a
header that provides a TAP protocol version, followed by a test plan with
a count of items to be executed.  That pattern should be followed at each
nesting level.  In the current implementation of the top-most, i.e., test
suite level, those rules apply only for test suites built into the kernel,
executed and reported on boot.  Results submitted to dmesg from kunit test
modules loaded later are missing those top-level headers.

As a consequence, if a kunit test module provides more than one test suite
then, without the top level test plan, external tools that are parsing
dmesg for kunit test output are not able to tell how many test suites
should be expected and whether to continue parsing after complete output
from the first test suite is collected.

Submit the top-level headers also from the kunit test module notifier
initialization callback.

v3: Fix new name of a structure moved to kunit namespace not updated in
    executor_test functions (lkp@intel.com).
v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from
    emitting the headers when called on load of non-test modules.

[1] https://docs.kernel.org/dev-tools/ktap.html#

Signed-off-by: Janusz Krzysztofik 
Cc: Mauro Carvalho Chehab 
Cc: Rae Moar 
Reviewed-by: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: fix uninitialized variables bug in attributes filtering

2023-08-04T19:41:55+00:00

Fix smatch warnings regarding uninitialized variables in the filtering
patch of the new KUnit Attributes feature.

Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")

Reported-by: kernel test robot 
Reported-by: Dan Carpenter 
Closes: https://lore.kernel.org/r/202307270610.s0w4NKEn-lkp@intel.com/

Signed-off-by: Rae Moar 
Reviewed-by: David Gow 
Signed-off-by: Shuah Khan

kunit: add tests for filtering attributes

2023-07-26T19:29:41+00:00

Add four tests to executor_test.c to test behavior of filtering attributes.

- parse_filter_attr_test - to test the parsing of inputted filters

- filter_attr_test - to test the filtering procedure on attributes

- filter_attr_empty_test - to test the behavior when all tests are filtered
  out

- filter_attr_skip_test - to test the configurable filter_action=skip
  option

Reviewed-by: David Gow 
Signed-off-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: Add ability to filter attributes

2023-07-26T19:29:15+00:00

Add filtering of test attributes. Users can filter tests using the
module_param called "filter".

Filters are imputed in the format: 

Example: kunit.filter="speed>slow"

Operations include: >, <, >=, <=, !=, and =. These operations will act the
same for attributes of the same type but may not between types.

Note multiple filters can be inputted by separating them with a comma.
Example: kunit.filter="speed=slow, module!=example"

Since both suites and test cases can have attributes, there may be
conflicts. The process of filtering follows these rules:
- Filtering always operates at a per-test level.
- If a test has an attribute set, then the test's value is filtered on.
- Otherwise, the value falls back to the suite's value.
- If neither are set, the attribute has a global "default" value, which
  is used.

Filtered tests will not be run or show in output. The tests can instead be
skipped using the configurable option "kunit.filter_action=skip".

Note the default settings for running tests remains unfiltered.

Finally, add "filter" methods for the speed and module attributes to parse
and compare attribute values.

Note this filtering functionality will be added to kunit.py in the next
patch.

Reviewed-by: David Gow 
Signed-off-by: Rae Moar 
Signed-off-by: Shuah Khan

kunit: executor_test: Use kunit_add_action()

2023-05-25T14:53:01+00:00

Now we have the kunit_add_action() function, we can use it to implement
kfree_at_end() and free_subsuite_at_end() without the need for extra
helper functions.

Reviewed-by: Benjamin Berg 
Reviewed-by: Maxime Ripard 
Tested-by: Maxime Ripard 
Signed-off-by: David Gow 
Signed-off-by: Shuah Khan

kunit: flatten kunit_suite* to kunit_suite in .kunit_test_suites

2022-07-11T23:13:15+00:00

We currently store kunit suites in the .kunit_test_suites ELF section as
a `struct kunit_suite***` (modulo some `const`s).
For every test file, we store a struct kunit_suite** NULL-terminated array.

This adds quite a bit of complexity to the test filtering code in the
executor.

Instead, let's just make the .kunit_test_suites section contain a single
giant array of struct kunit_suite pointers, which can then be directly
manipulated. This array is not NULL-terminated, and so none of the test
filtering code needs to NULL-terminate anything.

Tested-by: Maíra Canal 
Reviewed-by: Brendan Higgins 
Signed-off-by: Daniel Latypov 
Co-developed-by: David Gow 
Signed-off-by: David Gow 
Signed-off-by: Shuah Khan

kunit: bail out of test filtering logic quicker if OOM

2022-05-12T17:15:08+00:00

When filtering what tests to run (suites and/or cases) via
kunit.filter_glob (e.g. kunit.py run ), we allocate copies of
suites.

These allocations can fail, and we largely don't handle that.
Note: realistically, this probably doesn't matter much.
We're not allocating much memory and this happens early in boot, so if
we can't do that, then there's likely far bigger problems.

This patch makes us immediately bail out from the top-level function
(kunit_filter_suites) with -ENOMEM if any of the underlying kmalloc()
calls return NULL.

Implementation note: we used to return NULL pointers from some functions
to indicate either that all suites/tests were filtered out or there was
an error allocating the new array.

We'll log a short error in this case and not run any tests or print a
TAP header. From a kunit.py user's perspective, they'll get a message
about missing/invalid TAP output and have to dig into the test.log to
see it. Since hitting this error seems so unlikely, it's probably fine
to not invent a way to plumb this error message more visibly.

See also: https://lore.kernel.org/linux-kselftest/20220329103919.2376818-1-lv.ruyi@zte.com.cn/

Signed-off-by: Daniel Latypov 
Reported-by: Zeal Robot 
Reported-by: Lv Ruyi 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan