linux.git/kernel/torture.c, branch v6.1.168

torture: Fix hang during kthread shutdown phase

2023-03-10T08:34:07+00:00

commit d52d3a2bf408ff86f3a79560b5cce80efb340239 upstream.

During rcutorture shutdown, the rcu_torture_cleanup() function calls
torture_cleanup_begin(), which sets the fullstop global variable to
FULLSTOP_RMMOD. This causes the rcutorture threads for readers and
fakewriters to exit all of their "while" loops and start shutting down.

They then call torture_kthread_stopping(), which in turn waits for
kthread_stop() to be called.  However, rcu_torture_cleanup() has
not yet called kthread_stop() on those threads, and before it gets a
chance to do so, multiple instances of torture_kthread_stopping() invoke
schedule_timeout_interruptible(1) in a tight loop.  Tracing confirms that
TIMER_SOFTIRQ can then continuously execute timer callbacks.  If that
TIMER_SOFTIRQ preempts the task executing rcu_torture_cleanup(), that
task might never invoke kthread_stop().

This commit improves this situation by increasing the timeout passed to
schedule_timeout_interruptible() from one jiffy to 1/20th of a second.
This change prevents TIMER_SOFTIRQ from monopolizing its CPU, thus
allowing rcu_torture_cleanup() to carry out the needed kthread_stop()
invocations.  Testing has shown 100 runs of TREE07 passing reliably,
as oppose to the tens-of-percent failure rates seen beforehand.

Cc: Paul McKenney 
Cc: Frederic Weisbecker 
Cc: Zhouyi Zhou 
Cc:  # 6.0.x
Signed-off-by: Joel Fernandes (Google) 
Tested-by: Zhouyi Zhou 
Reviewed-by: Davidlohr Bueso 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Greg Kroah-Hartman

torture: Wake up kthreads after storing task_struct pointer

2022-02-02T01:24:39+00:00

Currently, _torture_create_kthread() uses kthread_run() to create
torture-test kthreads, which means that the resulting task_struct
pointer is stored after the newly created kthread has been marked
runnable.  This in turn can cause spurious failure of checks for
code being run by a particular kthread.  This commit therefore changes
_torture_create_kthread() to use kthread_create(), then to do an explicit
wake_up_process() after the task_struct pointer has been stored.

Reported-by: Frederic Weisbecker 
Reviewed-by: Neeraj Upadhyay 
Reviewed-by: Uladzislau Rezki (Sony) 
Reviewed-by: Frederic Weisbecker 
Signed-off-by: Paul E. McKenney

torture: Distinguish kthread stopping and being asked to stop

2022-02-02T01:24:38+00:00

Right now, if a given kthread (call it "kthread") realizes that it needs
to stop, "Stopping kthread" is written to the console.  When the cleanup
code decides that it is time to stop that kthread, "Stopping kthread
tasks" is written to the console.  These two events might happen in
either order, especially in the case of time-based torture-test shutdown.

But it is hard to distinguish these, especially for those unfamiliar with
the torture tests.  This commit therefore changes the first case from
"Stopping kthread" to "kthread is stopping" to make things more clear.

Signed-off-by: Paul E. McKenney

locktorture,rcutorture,torture: Always log error message

2021-12-08T00:36:17+00:00

Unconditionally log messages corresponding to errors.

Acked-by: Davidlohr Bueso 
Signed-off-by: Li Zhijian 
Signed-off-by: Paul E. McKenney

torture: Replace deprecated CPU-hotplug functions.

2021-08-10T17:48:07+00:00

The functions get_online_cpus() and put_online_cpus() have been
deprecated during the CPU hotplug rework. They map directly to
cpus_read_lock() and cpus_read_unlock().

Replace deprecated CPU-hotplug functions with the official version.
The behavior remains unchanged.

Cc: Davidlohr Bueso 
Cc: "Paul E. McKenney" 
Cc: Josh Triplett 
Signed-off-by: Sebastian Andrzej Siewior 
Signed-off-by: Paul E. McKenney

torture: Replace torture_init_begin string with %s

2021-03-08T22:22:28+00:00

This commit replaces a hard-coded "torture_init_begin" string in
a pr_alert() format with "%s" and __func__.

Signed-off-by: Stephen Zhang 
Signed-off-by: Paul E. McKenney

torture: Maintain torture-specific set of CPUs-online books

2021-01-07T01:17:22+00:00

The TREE01 rcutorture scenario intentionally creates confusion as to the
number of available CPUs by specifying the "maxcpus=8 nr_cpus=43" kernel
boot parameters.  This can disable rcutorture's load shedding, which
currently uses num_online_cpus(), which would count the extra 35 CPUs.
However, the rcutorture guest OS will be provisioned with only 8 CPUs,
which means that rcutorture will present full load even when all but one
of the original 8 CPUs are offline.  This can result in spurious errors
due to extreme overloading of that single remaining CPU.

This commit therefore keeps a separate set of books on the number of
usable online CPUs, so that torture_num_online_cpus() is used for load
shedding instead of num_online_cpus().  Note that initial sizing must
use num_online_cpus() because torture_num_online_cpus() will return
NR_CPUS until shortly after torture_onoff_init() is invoked.

Reported-by: Frederic Weisbecker 
[ paulmck: Apply feedback from kernel test robot. ]
Signed-off-by: Paul E. McKenney

torture: Clean up after torture-test CPU hotplugging

2021-01-07T01:17:22+00:00

This commit puts all CPUs back online at the end of a torture test,
and also unconditionally puts them online at the beginning of the test,
rather than just in the case of built-in tests.  This allows torture tests
to behave in a predictable manner, whether built-in or based on modules.

Signed-off-by: Paul E. McKenney

torture: Throttle VERBOSE_TOROUT_*() output

2021-01-07T01:17:21+00:00

This commit adds kernel boot parameters torture.verbose_sleep_frequency
and torture.verbose_sleep_duration, which allow VERBOSE_TOROUT_*() output
to be throttled with periodic sleeps on large systems.

Signed-off-by: Paul E. McKenney

torture: Make stutter use torture_hrtimeout_*() functions

2021-01-07T01:17:20+00:00

This commit saves a few lines of code by making the stutter_wait()
and torture_stutter() functions use torture_hrtimeout_jiffies() and
torture_hrtimeout_us().

Signed-off-by: Paul E. McKenney