Merge tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace

Pull tracing tools updates from Steven Rostedt:

 - Introduce enum timerlat_tracing_mode

   Now that BPF based sampling has been added to timerlat, add an enum
   to represent which mode timerlat is running in

 - Add action on timelat threshold feature

   A new option, --on-threshold, is added, taking an argument that
   further specifies the action. Actions added in this patch are:

     - trace[,file=<filename>]: Saves tracefs buffer, optionally taking a
             filename
     - signal,num=<sig>,pid=<pid>: Sends signal to process. "parent" might
             be specified instead of number to send signal to parent process
     - shell,command=<command>: Execute shell command

 - Allow resuming tracing in timerlat bpf

   rtla-timerlat BPF program uses a global variable stored in a .bss
   section to store whether tracing has been stopped. Map it to allow it
   to resume tracing after it has been stopped

 - Add continue action to timerlat

   Introduce option to resume tracing after a latency threshold
   overflow. The option is implemented as an action named "continue"

 - Add action on end feature to timerlat

   Implement actions on end next to actions on threshold. A new option,
   --on-end is added, parallel to --on-threshold. Instead of being
   executed whenever a latency threshold is reached, it is executed at
   the end of the measurement

 - Have rtla tests check output with grep

   Add argument to the check command in the test suite that takes a
   regular expression that the output of rtla command is checked
   against. This allows testing for specific information in rtla output
   in addition to checking the return value

 - Add tests for timerlat actions

 - Update the documentation for the new features

* tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
  rtla/tests: Test timerlat -P option using actions
  rtla/tests: Add grep checks for base test cases
  Documentation/rtla: Add actions feature
  rtla/tests: Limit duration to maximum of 10s
  rtla/tests: Add tests for actions
  rtla/tests: Check rtla output with grep
  rtla/timerlat: Add action on end feature
  rtla/timerlat: Add continue action
  rtla/timerlat_bpf: Allow resuming tracing
  rtla/timerlat: Add action on threshold feature
  rtla/timerlat: Introduce enum timerlat_tracing_mode

1 files changed, 64 insertions, 0 deletions

diff --git a/Documentation/tools/rtla/common_timerlat_options.rst b/Documentation/tools/rtla/common_timerlat_options.rst
index 10dc802f8d65..7854368f1827 100644
--- a/Documentation/tools/rtla/common_timerlat_options.rst
+++ b/Documentation/tools/rtla/common_timerlat_options.rst
@@ -55,3 +55,67 @@
         Set timerlat to run without workload, waiting for the user to dispatch a per-cpu
         task that waits for a new period on the tracing/osnoise/per_cpu/cpu$ID/timerlat_fd.
         See linux/tools/rtla/sample/timerlat_load.py for an example of user-load code.
+
+**--on-threshold** *action*
+
+        Defines an action to be executed when tracing is stopped on a latency threshold
+        specified by **-i/--irq** or **-T/--thread**.
+
+        Multiple --on-threshold actions may be specified, and they will be executed in
+        the order they are provided. If any action fails, subsequent actions in the list
+        will not be executed.
+
+        Supported actions are:
+
+        - *trace[,file=<filename>]*
+
+          Saves trace output, optionally taking a filename. Alternative to -t/--trace.
+          Note that nlike -t/--trace, specifying this multiple times will result in
+          the trace being saved multiple times.
+
+        - *signal,num=<sig>,pid=<pid>*
+
+          Sends signal to process. "parent" might be specified in place of pid to target
+          the parent process of rtla.
+
+        - *shell,command=<command>*
+
+          Execute shell command.
+
+        - *continue*
+
+          Continue tracing after actions are executed instead of stopping.
+
+        Example:
+
+        $ rtla timerlat -T 20 --on-threshold trace
+        --on-threshold shell,command="grep ipi_send timerlat_trace.txt"
+        --on-threshold signal,num=2,pid=parent
+
+        This will save a trace with the default filename "timerlat_trace.txt", print its
+        lines that contain the text "ipi_send" on standard output, and send signal 2
+        (SIGINT) to the parent process.
+
+        Performance Considerations:
+
+        For time-sensitive actions, it is recommended to run **rtla timerlat** with BPF
+        support and RT priority. Note that due to implementational limitations, actions
+        might be delayed up to one second after tracing is stopped if BPF mode is not
+        available or disabled.
+
+**--on-end** *action*
+
+        Defines an action to be executed at the end of **rtla timerlat** tracing.
+
+        Multiple --on-end actions can be specified, and they will be executed in the order
+        they are provided. If any action fails, subsequent actions in the list will not be
+        executed.
+
+        See the documentation for **--on-threshold** for the list of supported actions, with
+        the exception that *continue* has no effect.
+
+        Example:
+
+        $ rtla timerlat -d 5s --on-end trace
+
+        This runs rtla timerlat with default options and save trace output at the end.