linux.git/tools/perf/scripts/python/Perf-Trace-Util, branch v6.6.131

perf scripts python: Update audit-libs package name for python3

2023-08-15T19:41:48+00:00

'audit-libs-python' is the package for python2, update it for python3.
On Ubuntu and Fedora, the new package is 'python3-audit'.

Signed-off-by: Wei Li 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Li Bin 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20230815131805.1237491-1-liwei391@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: Support syscall name parsing on arm64

2023-08-15T19:41:48+00:00

In the result of "perf script syscall-counts" on arm64, the syscall
events are not resolved currently. Add "aarch64" to audit uname list to
support name parsing.

* After the patch:

  [root@localhost ~]# perf script syscall-counts sleep 1
  Press control+C to stop and show the summary

  syscall events:

  event                                          count
  ----------------------------------------  -----------
  mmap                                               6
  close                                              5
  mprotect                                           4
  brk                                                3
  newfstatat                                         3
  openat                                             3
  getrandom                                          1
  prlimit64                                          1
  munmap                                             1
  clock_nanosleep                                    1
  set_robust_list                                    1
  set_tid_address                                    1
  exit_group                                         1
  read                                               1
  faccessat                                          1

Signed-off-by: Wei Li 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Li Bin 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Link: https://lore.kernel.org/r/20230815131735.1237221-1-liwei391@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Cope with declarations after statements found in Python.h

2023-08-03T20:01:28+00:00

With -Werror the build was failing on fedora rawhide:

  [perfbuilder@27cfe44d67ed perf-6.5.0-rc2]$ gcc -v
  Using built-in specs.
  COLLECT_GCC=gcc
  COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/13/lto-wrapper
  OFFLOAD_TARGET_NAMES=nvptx-none
  OFFLOAD_TARGET_DEFAULT=1
  Target: x86_64-redhat-linux
  Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-13.2.1-20230728/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
  Thread model: posix
  Supported LTO compression algorithms: zlib zstd
  gcc version 13.2.1 20230728 (Red Hat 13.2.1-1) (GCC)
  [perfbuilder@27cfe44d67ed perf-6.5.0-rc2]$

  In file included from /usr/include/python3.12/Python.h:44,
                   from scripts/python/Perf-Trace-Util/Context.c:14:
  /usr/include/python3.12/object.h: In function 'Py_SIZE':
  /usr/include/python3.12/object.h:217:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    217 |     PyVarObject *var_ob = _PyVarObject_CAST(ob);
        |     ^~~~~~~~~~~
  In file included from /usr/include/python3.12/Python.h:53:
  /usr/include/python3.12/cpython/longintrepr.h: In function '_PyLong_CompactValue':
  /usr/include/python3.12/cpython/longintrepr.h:121:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    121 |     Py_ssize_t sign = 1 - (op->long_value.lv_tag & _PyLong_SIGN_MASK);
        |     ^~~~~~~~~~
  
  In file included from /usr/include/python3.12/Python.h:44,
                   from util/scripting-engines/trace-event-python.c:22:
  /usr/include/python3.12/object.h: In function 'Py_SIZE':
  /usr/include/python3.12/object.h:217:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    217 |     PyVarObject *var_ob = _PyVarObject_CAST(ob);
        |     ^~~~~~~~~~~
    CC      /tmp/build/perf/util/units.o
    CC      /tmp/build/perf/util/time-utils.o
  In file included from /usr/include/python3.12/Python.h:53:
  /usr/include/python3.12/cpython/longintrepr.h: In function '_PyLong_CompactValue':
  /usr/include/python3.12/cpython/longintrepr.h:121:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    121 |     Py_ssize_t sign = 1 - (op->long_value.lv_tag & _PyLong_SIGN_MASK);
        |     ^~~~~~~~~~

So add -Wno-declaration-after-statement to the python scripting CFLAGS.

Reviewed-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Link: https://lore.kernel.org/lkml/ZMpdKeO8gU%2FcWDqH@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf thread: Add accessor functions for thread

2023-06-12T18:57:53+00:00

Using accessors will make it easier to add reference count checking in
later patches.

Committer notes:

thread->nsinfo wasn't wrapped as it is used together with
nsinfo__zput(), where does a trick to set the field with a refcount
being dropped to NULL, and that doesn't work well with using
thread__nsinfo(thread), that loses the &thread->nsinfo pointer.

When refcount checking is added to 'struct thread', later in this
series, nsinfo__zput(RC_CHK_ACCESS(thread)->nsinfo) will be used to
check the thread pointer.

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Ali Saidi 
Cc: Andi Kleen 
Cc: Athira Rajeev 
Cc: Brian Robbins 
Cc: Changbin Du 
Cc: Dmitrii Dolgov <9erthalion6@gmail.com>
Cc: Fangrui Song 
Cc: German Gomez 
Cc: Ingo Molnar 
Cc: Ivan Babrou 
Cc: James Clark 
Cc: Jing Zhang 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: K Prateek Nayak 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Liam Howlett 
Cc: Mark Rutland 
Cc: Miguel Ojeda 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Naveen N. Rao 
Cc: Peter Zijlstra 
Cc: Ravi Bangoria 
Cc: Sean Christopherson 
Cc: Steinar H. Gunderson 
Cc: Suzuki Poulouse 
Cc: Wenyu Liu 
Cc: Will Deacon 
Cc: Yang Jihong 
Cc: Ye Xingchen 
Cc: Yuan Can 
Cc: coresight@lists.linaro.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20230608232823.4027869-4-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf map: Add accessor for dso

2023-04-04T19:41:57+00:00

Later changes will add reference count checking for struct map, with
dso being the most frequently accessed variable. Add an accessor so
that the reference count check is only necessary in one place.

Additional changes:
 - add a dso variable to avoid repeated map__dso calls.
 - in builtin-mem.c dump_raw_samples, code only partially tested for
   dso == NULL. Make the possibility of NULL consistent.
 - in thread.c thread__memcpy fix use of spaces and use tabs.

Committer notes:

Did missing conversions on these files:

   tools/perf/arch/powerpc/util/skip-callchain-idx.c
   tools/perf/arch/powerpc/util/sym-handling.c
   tools/perf/ui/browsers/hists.c
   tools/perf/ui/gtk/annotate.c
   tools/perf/util/cs-etm.c
   tools/perf/util/thread.c
   tools/perf/util/unwind-libunwind-local.c
   tools/perf/util/unwind-libunwind.c

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Alexey Bayduraev 
Cc: Andi Kleen 
Cc: Andrew Morton 
Cc: Andy Shevchenko 
Cc: Darren Hart 
Cc: Davidlohr Bueso 
Cc: Dmitriy Vyukov 
Cc: Eric Dumazet 
Cc: German Gomez 
Cc: Hao Luo 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Miaoqian Lin 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Riccardo Mancini 
Cc: Shunsuke Nakamura 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: Stephen Brennan 
Cc: Steven Rostedt (VMware) 
Cc: Thomas Gleixner 
Cc: Thomas Richter 
Cc: Yury Norov 
Link: https://lore.kernel.org/r/20230320212248.1175731-2-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf maps: Add functions to access maps

2023-04-04T18:45:38+00:00

Introduce functions to access struct maps. These functions reduce the
number of places reference counting is necessary. While tidying APIs do
some small const-ification, in particlar to unwind_libunwind_ops.

Committer notes:

Fixed up tools/perf/util/unwind-libunwind.c:

-               return ops->get_entries(cb, arg, thread, data, max_stack);
+               return ops->get_entries(cb, arg, thread, data, max_stack, best_effort);

Signed-off-by: Ian Rogers 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Alexey Bayduraev 
Cc: Andi Kleen 
Cc: Andrew Morton 
Cc: Andy Shevchenko 
Cc: Darren Hart 
Cc: Davidlohr Bueso 
Cc: Dmitriy Vyukov 
Cc: Eric Dumazet 
Cc: German Gomez 
Cc: Hao Luo 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Leo Yan 
Cc: Madhavan Srinivasan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Miaoqian Lin 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Riccardo Mancini 
Cc: Shunsuke Nakamura 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: Stephen Brennan 
Cc: Steven Rostedt (VMware) 
Cc: Thomas Gleixner 
Cc: Thomas Richter 
Cc: Yury Norov 
Link: https://lore.kernel.org/r/20230320212248.1175731-2-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script: Fix Python support when no libtraceevent

2023-03-15T13:27:07+00:00

Python scripting can be used without libtraceevent. In particular,
scripting for Intel PT does not use tracepoints, and so does not need
libtraceevent support.

Alter the build and employ conditional compilation to allow Python
scripting without libtraceevent.

Example:

 Before:

    $ ldd `which perf` | grep -i python
    $ ldd `which perf` | grep -i libtraceevent
    $ perf record -e intel_pt//u uname
    Linux
    [ perf record: Woken up 1 times to write data ]
    [ perf record: Captured and wrote 0.031 MB perf.data ]
    $ perf script intel-pt-events.py |& head -3
      Error: Couldn't find script `intel-pt-events.py'

     See perf script -l for available scripts.

 After:

    $ ldd `which perf` | grep -i python
            libpython3.10.so.1.0 => /lib/x86_64-linux-gnu/libpython3.10.so.1.0 (0x00007f4bac400000)
    $ ldd `which perf` | grep -i libtraceevent
    $ perf script intel-pt-events.py | head
    Intel PT Branch Trace, Power Events, Event Trace and PTWRITE
         Switch In    8021/8021  [000]     11234.097713404     0/0
           perf-exec  8021/8021  [000]     11234.098041726       psb                        offset: 0x0                0 [unknown] ([unknown])
           perf-exec  8021/8021  [000]     11234.098041726       cbr                         45  freq: 4505 MHz  (161%)                0 [unknown] ([unknown])
               uname  8021/8021  [000]     11234.098082170  branches:uH  tr strt                              0 [unknown] ([unknown]) => 7f3a8b9422b0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
               uname  8021/8021  [000]     11234.098082379  branches:uH  tr end                    7f3a8b9422b0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) => 0 [unknown] ([unknown])
               uname  8021/8021  [000]     11234.098083629  branches:uH  tr strt                              0 [unknown] ([unknown]) => 7f3a8b9422b0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
               uname  8021/8021  [000]     11234.098083629  branches:uH  call                      7f3a8b9422b3 _start+0x3 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) => 7f3a8b943050 _dl_start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
               uname  8021/8021  [000]     11234.098083837  branches:uH  tr end                    7f3a8b943060 _dl_start+0x10 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) => 0 [unknown] ([unknown])  IPC: 0.01 (9/938)
               uname  8021/8021  [000]     11234.098084670  branches:uH  tr strt                              0 [unknown] ([unknown]) => 7f3a8b943060 _dl_start+0x10 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)

Fixes: 378ef0f5d9d7f465 ("perf build: Use libtraceevent from the system")
Signed-off-by: Adrian Hunter 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Link: https://lore.kernel.org/r/20230315084321.14563-1-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf build: Use libtraceevent from the system

2022-12-14T14:16:12+00:00

Remove the LIBTRACEEVENT_DYNAMIC and LIBTRACEFS_DYNAMIC make command
line variables.

If libtraceevent isn't installed or NO_LIBTRACEEVENT=1 is passed to the
build, don't compile in libtraceevent and libtracefs support.

This also disables CONFIG_TRACE that controls "perf trace".

CONFIG_LIBTRACEEVENT is used to control enablement in Build/Makefiles,
HAVE_LIBTRACEEVENT is used in C code.

Without HAVE_LIBTRACEEVENT tracepoints are disabled and as such the
commands kmem, kwork, lock, sched and timechart are removed.  The
majority of commands continue to work including "perf test".

Committer notes:

Fixed up a tools/perf/util/Build reject and added:

  #include 

to tools/perf/util/scripting-engines/trace-event-perl.c.

Committer testing:

  $ rpm -qi libtraceevent-devel
  Name        : libtraceevent-devel
  Version     : 1.5.3
  Release     : 2.fc36
  Architecture: x86_64
  Install Date: Mon 25 Jul 2022 03:20:19 PM -03
  Group       : Unspecified
  Size        : 27728
  License     : LGPLv2+ and GPLv2+
  Signature   : RSA/SHA256, Fri 15 Apr 2022 02:11:58 PM -03, Key ID 999f7cbf38ab71f4
  Source RPM  : libtraceevent-1.5.3-2.fc36.src.rpm
  Build Date  : Fri 15 Apr 2022 10:57:01 AM -03
  Build Host  : buildvm-x86-05.iad2.fedoraproject.org
  Packager    : Fedora Project
  Vendor      : Fedora Project
  URL         : https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/
  Bug URL     : https://bugz.fedoraproject.org/libtraceevent
  Summary     : Development headers of libtraceevent
  Description :
  Development headers of libtraceevent-libs
  $

Default build:

  $ ldd ~/bin/perf | grep tracee
  	libtraceevent.so.1 => /lib64/libtraceevent.so.1 (0x00007f1dcaf8f000)
  $

  # perf trace -e sched:* --max-events 10
       0.000 migration/0/17 sched:sched_migrate_task(comm: "", pid: 1603763 (perf), prio: 120, dest_cpu: 1)
       0.005 migration/0/17 sched:sched_wake_idle_without_ipi(cpu: 1)
       0.011 migration/0/17 sched:sched_switch(prev_comm: "", prev_pid: 17 (migration/0), prev_state: 1, next_comm: "", next_prio: 120)
       1.173 :0/0 sched:sched_wakeup(comm: "", pid: 3138 (gnome-terminal-), prio: 120)
       1.180 :0/0 sched:sched_switch(prev_comm: "", prev_prio: 120, next_comm: "", next_pid: 3138 (gnome-terminal-), next_prio: 120)
       0.156 migration/1/21 sched:sched_migrate_task(comm: "", pid: 1603763 (perf), prio: 120, orig_cpu: 1, dest_cpu: 2)
       0.160 migration/1/21 sched:sched_wake_idle_without_ipi(cpu: 2)
       0.166 migration/1/21 sched:sched_switch(prev_comm: "", prev_pid: 21 (migration/1), prev_state: 1, next_comm: "", next_prio: 120)
       1.183 :0/0 sched:sched_wakeup(comm: "", pid: 1602985 (kworker/u16:0-f), prio: 120, target_cpu: 1)
       1.186 :0/0 sched:sched_switch(prev_comm: "", prev_prio: 120, next_comm: "", next_pid: 1602985 (kworker/u16:0-f), next_prio: 120)
  #

Had to tweak tools/perf/util/setup.py to make sure the python binding
shared object links with libtraceevent if -DHAVE_LIBTRACEEVENT is
present in CFLAGS.

Building with NO_LIBTRACEEVENT=1 uncovered some more build failures:

- Make building of data-convert-bt.c to CONFIG_LIBTRACEEVENT=y

- perf-$(CONFIG_LIBTRACEEVENT) += scripts/

- bpf_kwork.o needs also to be dependent on CONFIG_LIBTRACEEVENT=y

- The python binding needed some fixups and util/trace-event.c can't be
  built and linked with the python binding shared object, so remove it
  in tools/perf/util/setup.py and exclude it from the list of
  dependencies in the python/perf.so Makefile.perf target.

Building without libtraceevent-devel installed uncovered more build
failures:

- The python binding tools/perf/util/python.c was assuming that
  traceevent/parse-events.h was always available, which was the case
  when we defaulted to using the in-kernel tools/lib/traceevent/ files,
  now we need to enclose it under ifdef HAVE_LIBTRACEEVENT, just like
  the other parts of it that deal with tracepoints.

- We have to ifdef the rules in the Build files with
  CONFIG_LIBTRACEEVENT=y to build builtin-trace.c and
  tools/perf/trace/beauty/ as we only ifdef setting CONFIG_TRACE=y when
  setting NO_LIBTRACEEVENT=1 in the make command line, not when we don't
  detect libtraceevent-devel installed in the system. Simplification here
  to avoid these two ways of disabling builtin-trace.c and not having
  CONFIG_TRACE=y when libtraceevent-devel isn't installed is the clean
  way.

From Athira:


tools/perf/arch/powerpc/util/Build
-perf-y += kvm-stat.o
+perf-$(CONFIG_LIBTRACEEVENT) += kvm-stat.o


Then, ditto for arm64 and s390, detected by container cross build tests.

- s/390 uses test__checkevent_tracepoint() that is now only available if
  HAVE_LIBTRACEEVENT is defined, enclose the callsite with ifder HAVE_LIBTRACEEVENT.

Also from Athira:


With this change, I could successfully compile in these environment:
- Without libtraceevent-devel installed
- With libtraceevent-devel installed
- With “make NO_LIBTRACEEVENT=1”


Then, finally rename CONFIG_TRACEEVENT to CONFIG_LIBTRACEEVENT for
consistency with other libraries detected in tools/perf/.

Signed-off-by: Ian Rogers 
Tested-by: Arnaldo Carvalho de Melo 
Tested-by: Athira Rajeev 
Cc: Alexander Shishkin 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Nick Desaulniers 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: bpf@vger.kernel.org
Link: http://lore.kernel.org/lkml/20221205225940.3079667-3-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf scripting python: Add perf_sample_srcline() and perf_sample_srccode()

2021-06-01T13:04:35+00:00

Add perf_sample_srcline() and perf_sample_srccode() to the
perf_trace_context module so that a script can get the srcline or srccode
information.

Signed-off-by: Adrian Hunter 
Cc: Andi Kleen 
Cc: Jiri Olsa 
Link: https://lore.kernel.org/r/20210530192308.7382-11-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf scripting python: Add perf_set_itrace_options()

2021-06-01T13:04:24+00:00

Add perf_set_itrace_options() to the perf_trace_context module so that a
script can set the itrace options for a session if they have not been set
already.

Signed-off-by: Adrian Hunter 
Cc: Andi Kleen 
Cc: Jiri Olsa 
Link: https://lore.kernel.org/r/20210530192308.7382-10-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo