| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Like in 'perf report', we want to hide empty events in the 'perf annotate'
output. This is consistent when the option is set in perf report.
For example, the following command would use 3 events including dummy.
$ perf mem record -a -- perf test -w noploop
$ perf evlist
cpu/mem-loads,ldlat=30/P
cpu/mem-stores/P
dummy:u
Just using perf annotate with --group will show the all 3 events.
$ perf annotate --group --stdio | head
Percent | Source code & Disassembly of ...
--------------------------------------------------------------
: 0 0xe060 <_dl_relocate_object>:
0.00 0.00 0.00 : e060: pushq %rbp
0.00 0.00 0.00 : e061: movq %rsp, %rbp
0.00 0.00 0.00 : e064: pushq %r15
0.00 0.00 0.00 : e066: movq %rdi, %r15
0.00 0.00 0.00 : e069: pushq %r14
0.00 0.00 0.00 : e06b: pushq %r13
0.00 0.00 0.00 : e06d: movl %edx, %r13d
Now with --skip-empty, it'll hide the last dummy event.
$ perf annotate --group --stdio --skip-empty | head
Percent | Source code & Disassembly of ...
------------------------------------------------------
: 0 0xe060 <_dl_relocate_object>:
0.00 0.00 : e060: pushq %rbp
0.00 0.00 : e061: movq %rsp, %rbp
0.00 0.00 : e064: pushq %r15
0.00 0.00 : e066: movq %rdi, %r15
0.00 0.00 : e069: pushq %r14
0.00 0.00 : e06b: pushq %r13
0.00 0.00 : e06d: movl %edx, %r13d
Committer testing:
root@x1:~# perf evlist
cpu_atom/mem-loads,ldlat=30/P
cpu_atom/mem-stores/P
dummy:u
root@x1:~#
Before:
root@x1:~# perf annotate --group --stdio2 do_lookup_x | head -25
Samples: 20 of events 'cpu_atom/mem-loads,ldlat=30/P, cpu_atom/mem-stores/P, dummy:u', 4000 Hz, Event count (approx.): 769079, [percent: local period]
do_lookup_x() /usr/lib64/ld-linux-x86-64.so.2
Percent 0x9900 <do_lookup_x>:
pushq %rbp
movq %rsp,%rbp
pushq %r15
pushq %r14
pushq %r13
pushq %r12
pushq %rbx
subq $0x88,%rsp
movq %rdi,-0x50(%rbp)
movl 8(%r9),%edi
movq 0x10(%rbp),%r12
movq 0x28(%rbp),%r10
movq %rdx,-0x70(%rbp)
movq %rcx,-0x58(%rbp)
movq %rdi,%r11
0.00 5.73 0.00 movq %r8,-0x68(%rbp)
movq (%r9),%r8
movl %esi,%eax
8.30 0.00 0.00 movl 0x30(%rbp),%r9d
movl %esi,%r15d
shrl $6, %eax
movq %r8,%r13
root@x1:~#
After:
root@x1:~# perf annotate --group --skip-empty --stdio2 do_lookup_x | head -25
Samples: 20 of events 'cpu_atom/mem-loads,ldlat=30/P, cpu_atom/mem-stores/P', 4000 Hz, Event count (approx.): 769079, [percent: local period]
do_lookup_x() /usr/lib64/ld-linux-x86-64.so.2
Percent 0x9900 <do_lookup_x>:
pushq %rbp
movq %rsp,%rbp
pushq %r15
pushq %r14
pushq %r13
pushq %r12
pushq %rbx
subq $0x88,%rsp
movq %rdi,-0x50(%rbp)
movl 8(%r9),%edi
movq 0x10(%rbp),%r12
movq 0x28(%rbp),%r10
movq %rdx,-0x70(%rbp)
movq %rcx,-0x58(%rbp)
movq %rdi,%r11
0.00 5.73 movq %r8,-0x68(%rbp)
movq (%r9),%r8
movl %esi,%eax
8.30 0.00 movl 0x30(%rbp),%r9d
movl %esi,%r15d
shrl $6, %eax
movq %r8,%r13
root@x1:~#
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20240803211332.1107222-6-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
The --type-stat option is to be used with --data-type and to print
detailed failure reasons for the data type annotation.
$ perf annotate --data-type --type-stat
Annotate data type stats:
total 294, ok 116 (39.5%), bad 178 (60.5%)
-----------------------------------------------------------
30 : no_sym
40 : no_insn_ops
33 : no_mem_ops
63 : no_var
4 : no_typeinfo
8 : bad_offset
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: linux-toolchains@vger.kernel.org
Cc: linux-trace-devel@vger.kernel.org
Link: https://lore.kernel.org/r/20231213001323.718046-17-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Support data type annotation with new --data-type option. It internally
uses type sort key to collect sample histogram for the type and display
every members like below.
$ perf annotate --data-type
...
Annotate type: 'struct cfs_rq' in [kernel.kallsyms] (13 samples):
============================================================================
samples offset size field
13 0 640 struct cfs_rq {
2 0 16 struct load_weight load {
2 0 8 unsigned long weight;
0 8 4 u32 inv_weight;
};
0 16 8 unsigned long runnable_weight;
0 24 4 unsigned int nr_running;
1 28 4 unsigned int h_nr_running;
...
For simplicity it prints the number of samples per field for now.
But it should be easy to show the overhead percentage instead.
The number at the outer struct is a sum of the numbers of the inner
members. For example, struct cfs_rq got total 13 samples, and 2 came
from the load (struct load_weight) and 1 from h_nr_running. Similarly,
the struct load_weight got total 2 samples and they all came from the
weight field.
I've added two new flags in the symbol_conf for this. The
annotate_data_member is to get the members of the type. This is also
needed for perf report with typeoff sort key. The annotate_data_sample
is to update sample stats for each offset and used only in annotate.
Currently it only support stdio output mode, TUI support can be added
later.
Committer testing:
With the perf.data from the previous csets, a very simple, short
duration one:
# perf annotate --data-type
Annotate type: 'struct list_head' in [kernel.kallsyms] (1 samples):
============================================================================
samples offset size field
1 0 16 struct list_head {
0 0 8 struct list_head* next;
1 8 8 struct list_head* prev;
};
Annotate type: 'char' in [kernel.kallsyms] (1 samples):
============================================================================
samples offset size field
1 0 1 char ;
#
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: linux-toolchains@vger.kernel.org
Cc: linux-trace-devel@vger.kernel.org
Link: https://lore.kernel.org/r/20231213001323.718046-15-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Allow addr2line to be set either on the command line or via the
perfconfig file. This doesn't currently work with llvm-addr2line as
the addr2line code emits two things:
1) the address to decode,
2) a bogus ',' value.
The expectation is the bogus value will generate:
??
??:0
that terminates the addr2line reading. However, the output from
llvm-addr2line is a single line with just the input ',' locking up the
addr2line reading that is expecting a second line.
Signed-off-by: Ian Rogers <irogers@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: German Gomez <german.gomez@arm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Clark <james.clark@arm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sandipan Das <sandipan.das@amd.com>
Cc: Tom Rix <trix@redhat.com>
Cc: llvm@lists.linux.dev
Link: https://lore.kernel.org/r/20230328235543.1082207-2-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Use the global quiet variable everywhere so that all tools hide warnings
in quiet mode and update the documentation to reflect this.
'perf probe' claimed that errors are not printed in quiet mode but I
don't see this so remove it from the docs.
Signed-off-by: James Clark <james.clark@arm.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20221018094137.783081-3-james.clark@arm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Like in 'perf report' and 'perf top', Add this option to limit the
number of functions it displays based on the overhead value in percent.
This affects only stdio and stdio2 output modes. Without this, it
shows very long disassembly lines for every function in the data
file. If users don't want this behavior, they can set a value in
percent to suppress that.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lore.kernel.org/lkml/20220502232015.697243-1-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
The "auxtrace_info" and "auxtrace" functions are not set in "tool" member of
"annotate". As a result, perf annotate does not support parsing itrace data.
Before:
# perf record -e arm_spe_0/branch_filter=1/ -a sleep 1
[ perf record: Woken up 9 times to write data ]
[ perf record: Captured and wrote 20.874 MB perf.data ]
# perf annotate --stdio
Error:
The perf.data data has no samples!
Solution:
1. Add itrace options in help,
2. Set hook functions of "id_index", "auxtrace_info" and "auxtrace" in perf_tool.
After:
# perf record --all-user -e arm_spe_0/branch_filter=1/ ls
Couldn't synthesize bpf events.
perf.data
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.010 MB perf.data ]
# perf annotate --stdio
Percent | Source code & Disassembly of libc-2.28.so for branch-miss (1 samples, percent: local period)
------------------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 0000000000066180 <__getdelim@@GLIBC_2.17>:
0.00 : 66180: stp x29, x30, [sp, #-96]!
0.00 : 66184: cmp x0, #0x0
0.00 : 66188: ccmp x1, #0x0, #0x4, ne // ne = any
0.00 : 6618c: mov x29, sp
0.00 : 66190: stp x24, x25, [sp, #56]
0.00 : 66194: stp x26, x27, [sp, #72]
0.00 : 66198: str x28, [sp, #88]
0.00 : 6619c: b.eq 66450 <__getdelim@@GLIBC_2.17+0x2d0> // b.none
0.00 : 661a0: stp x22, x23, [x29, #40]
0.00 : 661a4: mov x22, x1
0.00 : 661a8: ldr w1, [x3]
0.00 : 661ac: mov w23, w2
0.00 : 661b0: stp x20, x21, [x29, #24]
0.00 : 661b4: mov x20, x3
0.00 : 661b8: mov x21, x0
0.00 : 661bc: tbnz w1, #15, 66360 <__getdelim@@GLIBC_2.17+0x1e0>
0.00 : 661c0: ldr x0, [x3, #136]
0.00 : 661c4: ldr x2, [x0, #8]
0.00 : 661c8: str x19, [x29, #16]
0.00 : 661cc: mrs x19, tpidr_el0
0.00 : 661d0: sub x19, x19, #0x700
0.00 : 661d4: cmp x2, x19
0.00 : 661d8: b.eq 663f0 <__getdelim@@GLIBC_2.17+0x270> // b.none
0.00 : 661dc: mov w1, #0x1 // #1
0.00 : 661e0: ldaxr w2, [x0]
0.00 : 661e4: cmp w2, #0x0
0.00 : 661e8: b.ne 661f4 <__getdelim@@GLIBC_2.17+0x74> // b.any
0.00 : 661ec: stxr w3, w1, [x0]
0.00 : 661f0: cbnz w3, 661e0 <__getdelim@@GLIBC_2.17+0x60>
0.00 : 661f4: b.ne 66448 <__getdelim@@GLIBC_2.17+0x2c8> // b.any
0.00 : 661f8: ldr x0, [x20, #136]
0.00 : 661fc: ldr w1, [x20]
0.00 : 66200: ldr w2, [x0, #4]
0.00 : 66204: str x19, [x0, #8]
0.00 : 66208: add w2, w2, #0x1
0.00 : 6620c: str w2, [x0, #4]
0.00 : 66210: tbnz w1, #5, 66388 <__getdelim@@GLIBC_2.17+0x208>
0.00 : 66214: ldr x19, [x29, #16]
0.00 : 66218: ldr x0, [x21]
0.00 : 6621c: cbz x0, 66228 <__getdelim@@GLIBC_2.17+0xa8>
0.00 : 66220: ldr x0, [x22]
0.00 : 66224: cbnz x0, 6623c <__getdelim@@GLIBC_2.17+0xbc>
0.00 : 66228: mov x0, #0x78 // #120
0.00 : 6622c: str x0, [x22]
0.00 : 66230: bl 20710 <malloc@plt>
0.00 : 66234: str x0, [x21]
0.00 : 66238: cbz x0, 66428 <__getdelim@@GLIBC_2.17+0x2a8>
0.00 : 6623c: ldr x27, [x20, #8]
0.00 : 66240: str x19, [x29, #16]
0.00 : 66244: ldr x19, [x20, #16]
0.00 : 66248: sub x19, x19, x27
0.00 : 6624c: cmp x19, #0x0
0.00 : 66250: b.le 66398 <__getdelim@@GLIBC_2.17+0x218>
0.00 : 66254: mov x25, #0x0 // #0
0.00 : 66258: b 662d8 <__getdelim@@GLIBC_2.17+0x158>
0.00 : 6625c: nop
0.00 : 66260: add x24, x19, x25
0.00 : 66264: ldr x3, [x22]
0.00 : 66268: add x26, x24, #0x1
0.00 : 6626c: ldr x0, [x21]
0.00 : 66270: cmp x3, x26
0.00 : 66274: b.cs 6629c <__getdelim@@GLIBC_2.17+0x11c> // b.hs, b.nlast
0.00 : 66278: lsl x3, x3, #1
0.00 : 6627c: cmp x3, x26
0.00 : 66280: csel x26, x3, x26, cs // cs = hs, nlast
0.00 : 66284: mov x1, x26
0.00 : 66288: bl 206f0 <realloc@plt>
0.00 : 6628c: cbz x0, 66438 <__getdelim@@GLIBC_2.17+0x2b8>
0.00 : 66290: str x0, [x21]
0.00 : 66294: ldr x27, [x20, #8]
0.00 : 66298: str x26, [x22]
0.00 : 6629c: mov x2, x19
0.00 : 662a0: mov x1, x27
0.00 : 662a4: add x0, x0, x25
0.00 : 662a8: bl 87390 <explicit_bzero@@GLIBC_2.25+0x50>
0.00 : 662ac: ldr x0, [x20, #8]
0.00 : 662b0: add x19, x0, x19
0.00 : 662b4: str x19, [x20, #8]
0.00 : 662b8: cbnz x28, 66410 <__getdelim@@GLIBC_2.17+0x290>
0.00 : 662bc: mov x0, x20
0.00 : 662c0: bl 73b80 <__underflow@@GLIBC_2.17>
0.00 : 662c4: cmn w0, #0x1
0.00 : 662c8: b.eq 66410 <__getdelim@@GLIBC_2.17+0x290> // b.none
0.00 : 662cc: ldp x27, x19, [x20, #8]
0.00 : 662d0: mov x25, x24
0.00 : 662d4: sub x19, x19, x27
0.00 : 662d8: mov x2, x19
0.00 : 662dc: mov w1, w23
0.00 : 662e0: mov x0, x27
0.00 : 662e4: bl 807b0 <memchr@@GLIBC_2.17>
0.00 : 662e8: cmp x0, #0x0
0.00 : 662ec: mov x28, x0
0.00 : 662f0: sub x0, x0, x27
0.00 : 662f4: csinc x19, x19, x0, eq // eq = none
0.00 : 662f8: mov x0, #0x7fffffffffffffff // #9223372036854775807
0.00 : 662fc: sub x0, x0, x25
0.00 : 66300: cmp x19, x0
0.00 : 66304: b.lt 66260 <__getdelim@@GLIBC_2.17+0xe0> // b.tstop
0.00 : 66308: adrp x0, 17f000 <sys_sigabbrev@@GLIBC_2.17+0x320>
0.00 : 6630c: ldr x0, [x0, #3624]
0.00 : 66310: mrs x2, tpidr_el0
0.00 : 66314: ldr x19, [x29, #16]
0.00 : 66318: mov w3, #0x4b // #75
0.00 : 6631c: ldr w1, [x20]
0.00 : 66320: mov x24, #0xffffffffffffffff // #-1
0.00 : 66324: str w3, [x2, x0]
0.00 : 66328: tbnz w1, #15, 66340 <__getdelim@@GLIBC_2.17+0x1c0>
0.00 : 6632c: ldr x0, [x20, #136]
0.00 : 66330: ldr w1, [x0, #4]
0.00 : 66334: sub w1, w1, #0x1
0.00 : 66338: str w1, [x0, #4]
0.00 : 6633c: cbz w1, 663b8 <__getdelim@@GLIBC_2.17+0x238>
0.00 : 66340: mov x0, x24
0.00 : 66344: ldr x28, [sp, #88]
0.00 : 66348: ldp x20, x21, [x29, #24]
0.00 : 6634c: ldp x22, x23, [x29, #40]
0.00 : 66350: ldp x24, x25, [sp, #56]
0.00 : 66354: ldp x26, x27, [sp, #72]
0.00 : 66358: ldp x29, x30, [sp], #96
0.00 : 6635c: ret
100.00 : 66360: tbz w1, #5, 66218 <__getdelim@@GLIBC_2.17+0x98>
0.00 : 66364: ldp x20, x21, [x29, #24]
0.00 : 66368: mov x24, #0xffffffffffffffff // #-1
0.00 : 6636c: ldp x22, x23, [x29, #40]
0.00 : 66370: mov x0, x24
0.00 : 66374: ldp x24, x25, [sp, #56]
0.00 : 66378: ldp x26, x27, [sp, #72]
0.00 : 6637c: ldr x28, [sp, #88]
0.00 : 66380: ldp x29, x30, [sp], #96
0.00 : 66384: ret
0.00 : 66388: mov x24, #0xffffffffffffffff // #-1
0.00 : 6638c: ldr x19, [x29, #16]
0.00 : 66390: b 66328 <__getdelim@@GLIBC_2.17+0x1a8>
0.00 : 66394: nop
0.00 : 66398: mov x0, x20
0.00 : 6639c: bl 73b80 <__underflow@@GLIBC_2.17>
0.00 : 663a0: cmn w0, #0x1
0.00 : 663a4: b.eq 66438 <__getdelim@@GLIBC_2.17+0x2b8> // b.none
0.00 : 663a8: ldp x27, x19, [x20, #8]
0.00 : 663ac: sub x19, x19, x27
0.00 : 663b0: b 66254 <__getdelim@@GLIBC_2.17+0xd4>
0.00 : 663b4: nop
0.00 : 663b8: str xzr, [x0, #8]
0.00 : 663bc: ldxr w2, [x0]
0.00 : 663c0: stlxr w3, w1, [x0]
0.00 : 663c4: cbnz w3, 663bc <__getdelim@@GLIBC_2.17+0x23c>
0.00 : 663c8: cmp w2, #0x1
0.00 : 663cc: b.le 66340 <__getdelim@@GLIBC_2.17+0x1c0>
0.00 : 663d0: mov x1, #0x81 // #129
0.00 : 663d4: mov x2, #0x1 // #1
0.00 : 663d8: mov x3, #0x0 // #0
0.00 : 663dc: mov x8, #0x62 // #98
0.00 : 663e0: svc #0x0
0.00 : 663e4: ldp x20, x21, [x29, #24]
0.00 : 663e8: ldp x22, x23, [x29, #40]
0.00 : 663ec: b 66370 <__getdelim@@GLIBC_2.17+0x1f0>
0.00 : 663f0: ldr w2, [x0, #4]
0.00 : 663f4: add w2, w2, #0x1
0.00 : 663f8: str w2, [x0, #4]
0.00 : 663fc: tbz w1, #5, 66214 <__getdelim@@GLIBC_2.17+0x94>
0.00 : 66400: mov x24, #0xffffffffffffffff // #-1
0.00 : 66404: ldr x19, [x29, #16]
0.00 : 66408: b 66330 <__getdelim@@GLIBC_2.17+0x1b0>
0.00 : 6640c: nop
0.00 : 66410: ldr x0, [x21]
0.00 : 66414: strb wzr, [x0, x24]
0.00 : 66418: ldr w1, [x20]
0.00 : 6641c: ldr x19, [x29, #16]
0.00 : 66420: b 66328 <__getdelim@@GLIBC_2.17+0x1a8>
0.00 : 66424: nop
0.00 : 66428: mov x24, #0xffffffffffffffff // #-1
0.00 : 6642c: ldr w1, [x20]
0.00 : 66430: b 66328 <__getdelim@@GLIBC_2.17+0x1a8>
0.00 : 66434: nop
0.00 : 66438: mov x24, #0xffffffffffffffff // #-1
0.00 : 6643c: ldr w1, [x20]
0.00 : 66440: ldr x19, [x29, #16]
0.00 : 66444: b 66328 <__getdelim@@GLIBC_2.17+0x1a8>
0.00 : 66448: bl e3ba0 <pthread_setcanceltype@@GLIBC_2.17+0x30>
0.00 : 6644c: b 661f8 <__getdelim@@GLIBC_2.17+0x78>
0.00 : 66450: adrp x0, 17f000 <sys_sigabbrev@@GLIBC_2.17+0x320>
0.00 : 66454: ldr x0, [x0, #3624]
0.00 : 66458: mrs x1, tpidr_el0
0.00 : 6645c: mov w2, #0x16 // #22
0.00 : 66460: mov x24, #0xffffffffffffffff // #-1
0.00 : 66464: str w2, [x1, x0]
0.00 : 66468: b 66370 <__getdelim@@GLIBC_2.17+0x1f0>
0.00 : 6646c: ldr w1, [x20]
0.00 : 66470: mov x4, x0
0.00 : 66474: tbnz w1, #15, 6648c <__getdelim@@GLIBC_2.17+0x30c>
0.00 : 66478: ldr x0, [x20, #136]
0.00 : 6647c: ldr w1, [x0, #4]
0.00 : 66480: sub w1, w1, #0x1
0.00 : 66484: str w1, [x0, #4]
0.00 : 66488: cbz w1, 66494 <__getdelim@@GLIBC_2.17+0x314>
0.00 : 6648c: mov x0, x4
0.00 : 66490: bl 20e40 <gnu_get_libc_version@@GLIBC_2.17+0x130>
0.00 : 66494: str xzr, [x0, #8]
0.00 : 66498: ldxr w2, [x0]
0.00 : 6649c: stlxr w3, w1, [x0]
0.00 : 664a0: cbnz w3, 66498 <__getdelim@@GLIBC_2.17+0x318>
0.00 : 664a4: cmp w2, #0x1
0.00 : 664a8: b.le 6648c <__getdelim@@GLIBC_2.17+0x30c>
0.00 : 664ac: mov x1, #0x81 // #129
0.00 : 664b0: mov x2, #0x1 // #1
0.00 : 664b4: mov x3, #0x0 // #0
0.00 : 664b8: mov x8, #0x62 // #98
0.00 : 664bc: svc #0x0
0.00 : 664c0: b 6648c <__getdelim@@GLIBC_2.17+0x30c>
Signed-off-by: Yang Jihong <yangjihong1@huawei.com>
Tested-by: Leo Yan <leo.yan@linaro.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lore.kernel.org/lkml/20210615091704.259202-1-yangjihong1@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
'perf annotate' supports --symbol but it's impossible to filter a C++
symbol. With --no-demangle one can filter easily by mangled function
name.
Signed-off-by: Martin Liška <mliska@suse.cz>
Link: http://lore.kernel.org/lkml/c3c7e959-9f7f-18e2-e795-f604275cbac3@suse.cz
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
The objdump utility has useful --prefix / --prefix-strip options to
allow changing source code file names hardcoded into executables' debug
info. Add options to 'perf report', 'perf top' and 'perf annotate',
which are then passed to objdump.
$ mkdir foo
$ echo 'main() { for (;;); }' > foo/foo.c
$ gcc -g foo/foo.c
foo/foo.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
1 | main() { for (;;); }
| ^~~~
$ perf record ./a.out
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.230 MB perf.data (5721 samples) ]
$ mv foo bar
$ perf annotate
<does not show source code>
$ perf annotate --prefix=/home/ak/lsrc/git/bar --prefix-strip=5
<does show source code>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Tested-by: Jiri Olsa <jolsa@redhat.com>
LPU-Reference: 20200107210444.214071-1-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add --percent-type option to set annotation percent type from following
choices:
global-period, local-period, global-hits, local-hits
Examples:
$ perf annotate --percent-type period-local --stdio | head -1
Percent | Source code ... es, percent: local period)
$ perf annotate --percent-type hits-local --stdio | head -1
Percent | Source code ... es, percent: local hits)
$ perf annotate --percent-type hits-global --stdio | head -1
Percent | Source code ... es, percent: global hits)
$ perf annotate --percent-type period-global --stdio | head -1
Percent | Source code ... es, percent: global period)
The local/global keywords set if the percentage is computed in the scope
of the function (local) or the whole data (global).
The period/hits keywords set the base the percentage is computed on -
the samples period or the number of samples (hits).
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/20180804130521.11408-20-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
This is already present in 'perf top', albeit undocumented (will fix),
and is useful to use /proc/kcore instead of vmlinux and then get what is
really in place, not what the kernel starts with, before alternatives,
ftrace .text patching, etc, see the differences:
# perf annotate --stdio2 _raw_spin_lock_irqsave
_raw_spin_lock_irqsave() /lib/modules/4.16.0-rc4/build/vmlinux
Event: anon group { cycles, instructions }
0.00 3.17 → callq __fentry__
0.00 7.94 push %rbx
7.69 36.51 → callq __page_file_index
mov %rax,%rbx
7.69 3.17 → callq *ffffffff82225cd0
xor %eax,%eax
mov $0x1,%edx
80.77 49.21 lock cmpxchg %edx,(%rdi)
test %eax,%eax
↓ jne 2b
3.85 0.00 mov %rbx,%rax
pop %rbx
← retq
2b: mov %eax,%esi
→ callq queued_spin_lock_slowpath
mov %rbx,%rax
pop %rbx
← retq
[root@jouet ~]# perf annotate --ignore-vmlinux --stdio2 _raw_spin_lock_irqsave
_raw_spin_lock_irqsave() /proc/kcore
Event: anon group { cycles, instructions }
0.00 3.17 nop
0.00 7.94 push %rbx
0.00 23.81 pushfq
7.69 12.70 pop %rax
nop
mov %rax,%rbx
7.69 3.17 cli
nop
xor %eax,%eax
mov $0x1,%edx
80.77 49.21 lock cmpxchg %edx,(%rdi)
test %eax,%eax
↓ jne 2b
3.85 0.00 mov %rbx,%rax
pop %rbx
← retq
2b: mov %eax,%esi
→ callq *ffffffff820e96b0
mov %rbx,%rax
pop %rbx
← retq
#
Diff of the output of those commands:
# perf annotate --stdio2 _raw_spin_lock_irqsave > /tmp/vmlinux
# perf annotate --ignore-vmlinux --stdio2 _raw_spin_lock_irqsave > /tmp/kcore
# diff -y /tmp/vmlinux /tmp/kcore
_raw_spin_lock_irqsave() vmlinux | _raw_spin_lock_irqsave() /proc/kcore
Event: anon group { cycles, instructions } Event: anon group { cycles, instructions }
0.00 3.17 → callq __fentry__ | 0.00 3.17 nop
0.00 7.94 push %rbx 0.00 7.94 push %rbx
7.69 36.51 → callq __page_file_index | 0.00 23.81 pushfq
> 7.69 12.70 pop %rax
> nop
mov %rax,%rbx mov %rax,%rbx
7.69 3.17 → callq *ffffffff82225cd0 | 7.69 3.17 cli
> nop
xor %eax,%eax xor %eax,%eax
mov $0x1,%edx mov $0x1,%edx
80.77 49.21 lock cmpxchg %edx,(%rdi) 80.77 49.21 lock cmpxchg %edx,(%rdi)
test %eax,%eax test %eax,%eax
↓ jne 2b ↓ jne 2b
3.85 0.00 mov %rbx,%rax 3.85 0.00 mov %rbx,%rax
pop %rbx pop %rbx
← retq ← retq
2b: mov %eax,%esi 2b: mov %eax,%esi
→ callq queued_spin_lock_slowpath| → callq *ffffffff820e96b0
mov %rbx,%rax mov %rbx,%rax
pop %rbx pop %rbx
← retq ← retq
#
This should be further streamlined by doing both annotations and
allowing the TUI to toggle initial/current, and show the patched
instructions in a slightly different color.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: https://lkml.kernel.org/n/tip-wz8d269hxkcwaczr0r4rhyjg@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
This uses the TUI augmented formatting routines, modulo interactivity.
# perf annotate --ignore-vmlinux --stdio2 _raw_spin_lock_irqsave
_raw_spin_lock_irqsave() /proc/kcore
Event: cycles:ppp
Percent
Disassembly of section load0:
ffffffff9a8734b0 <load0>:
nop
push %rbx
50.00 pushfq
pop %rax
nop
mov %rax,%rbx
cli
nop
xor %eax,%eax
mov $0x1,%edx
50.00 lock cmpxchg %edx,(%rdi)
test %eax,%eax
↓ jne 2b
mov %rbx,%rax
pop %rbx
← retq
2b: mov %eax,%esi
→ callq queued_spin_lock_slowpath
mov %rbx,%rax
pop %rbx
← retq
Tested-by: Jin Yao <yao.jin@linux.intel.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: https://lkml.kernel.org/n/tip-6cte5o8z84mbivbvqlg14uh1@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Some options must require an argument. But input, stdio-color, cpu have
no them. So I added it.
Signed-off-by: Jaecheol Shin <jcgod413@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Link: http://lkml.kernel.org/r/20180207095205.62715-1-jcgod413@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
When the --show-total-period option was introduced we forgot to add an
entry in the man page, fix it.
Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Martin Liška <mliska@suse.cz>
Fixes: 0c4a5bcea460 ("perf annotate: Display total number of samples with --show-total-period")
Link: http://lkml.kernel.org/r/1503046013-5555-1-git-send-email-treeze.taeung@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add --show-nr-samples option to "perf annotate" so that it matches "perf
report".
Committer note:
Note that it can't be used together with --show-total-period, which
seems like a silly limitation, that can be lifted at some point.
Made it bail out if not on --stdio.
Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/1503046008-5511-1-git-send-email-treeze.taeung@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
The -q/--quiet option is to suppress any message. Sometimes users just
want to see the numbers and it can be used for that case.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Suggested-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170217081742.17417-6-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
'perf annotate --stdio' will colorize entries with most hits and
possibly some other aspects of its output, but those colors gets
suppressed if we redirect the output to a non-tty, allow keeping the
colors by adding a new option, --stdio-color, now this use case will
also output escape sequences for colors:
$ perf annotate --stdio-color | more
Based-on-a-patch-by: Peter Zijlstra <peterz@infradead.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-sjrnixani5pg6qez640gaxhf@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1458823940-24583-6-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add --group option to enable event grouping. When enabled, all the
group members information will be shown with the leader so skip
non-leader events.
It only supports --stdio output currently. Later patches will extend
additional features.
$ perf annotate --group --stdio
...
Percent | Source code & Disassembly of libpthread-2.15.so
--------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 000000387dc0aa50 <__pthread_mutex_unlock_usercnt>:
8.08 2.40 5.29 : 387dc0aa50: mov %rdi,%rdx
0.00 0.00 0.00 : 387dc0aa53: mov 0x10(%rdi),%edi
0.00 0.00 0.00 : 387dc0aa56: mov %edi,%eax
0.00 0.80 0.00 : 387dc0aa58: and $0x7f,%eax
3.03 2.40 3.53 : 387dc0aa5b: test $0x7c,%dil
0.00 0.00 0.00 : 387dc0aa5f: jne 387dc0aaa9 <__pthread_mutex_unlock_use
0.00 0.00 0.00 : 387dc0aa61: test %eax,%eax
0.00 0.00 0.00 : 387dc0aa63: jne 387dc0aa85 <__pthread_mutex_unlock_use
0.00 0.00 0.00 : 387dc0aa65: and $0x80,%edi
0.00 0.00 0.00 : 387dc0aa6b: test %esi,%esi
3.03 5.60 7.06 : 387dc0aa6d: movl $0x0,0x8(%rdx)
0.00 0.00 0.59 : 387dc0aa74: je 387dc0aa7a <__pthread_mutex_unlock_use
0.00 0.00 0.00 : 387dc0aa76: subl $0x1,0xc(%rdx)
2.02 5.60 1.18 : 387dc0aa7a: mov %edi,%esi
0.00 0.00 0.00 : 387dc0aa7c: lock decl (%rdx)
83.84 83.20 82.35 : 387dc0aa7f: jne 387dc0aada <_L_unlock_586>
0.00 0.00 0.00 : 387dc0aa81: nop
0.00 0.00 0.00 : 387dc0aa82: xor %eax,%eax
0.00 0.00 0.00 : 387dc0aa84: retq
...
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1362462812-30885-6-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add --skip-missing option for skipping symbols that cannot be used for
annotation. It's the case of kernel symbols that user doesn't have a
vmlinux image file.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1360227734-375-8-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Basic implementation of perf annotate on GTK2. Currently only
shows first symbol. Add a new --gtk option to use it.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung.kim@lge.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1360227734-375-2-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
When analyzing perf data from hosts of other architecture than one of
the local host it's useful to call objdump that is part of a toolchain
for that architecture. Instead of calling regular objdump, call one that
user specified in command line.
Signed-off-by: Maciek Borzecki <maciek.borzecki@gmail.com>
Acked-by: David Ahern <dsahern@gmail.com>
Link: http://lkml.kernel.org/r/1346754750.16299.3.camel@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
The default input file for perf report is not handled the same way as
perf record does it for its output file. This leads to unexpected
behavior of perf report, etc. E.g.:
# perf record -a -e cpu-cycles sleep 2 | perf report | cat
failed to open perf.data: No such file or directory (try 'perf record' first)
While perf record writes to a fifo, perf report expects perf.data to be
read. This patch changes this to accept fifos as input file.
Applies to the following commands:
perf annotate
perf buildid-list
perf evlist
perf kmem
perf lock
perf report
perf sched
perf script
perf timechart
Also fixes char const* -> const char* type declaration for filename
strings.
v2:
* Prevent potential null pointer access to input_name in
builtin-report.c. Needed due to removal of patch "perf report: Setup
browser if stdout is a pipe"
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1323248577-11268-5-git-send-email-robert.richter@amd.com
Signed-off-by: Robert Richter <robert.richter@amd.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Currently the meaning of -C varies by perf command: for perf-top,
perf-stat, perf-record it means cpu list. For perf-report it means comm
list. Then perf-annotate, perf-report and perf-script use -c for cpu
list.
Fix annotate, report and script to use -C for cpu list to be consistent
with top, stat and record. This means report needs to use -c for comm
list which does introduce a backward compatibility change.
v1 -> v2
- update perf-script.txt too
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1321209008-7004-1-git-send-email-dsahern@gmail.com
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
And add the annotation output knobs to all the tools that have
integrated annotation (top, report).
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/n/tip-gnlob67mke6sji2kf4nstp7m@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add -M option to report/annotate to pass directly to objdump. This
allows to use -M intel for intel style disassembler syntax, which is
useful for people who are very used to the Intel syntax.
Link: http://lkml.kernel.org/r/1316122302-24306-2-git-send-email-andi@firstfloor.org
[committer note: Add missing Documentation bits, fixup conflicts with 3e6a2a7]
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Stephane Eranian <eranian@google.com>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
If you have --symfs in perf report, then you also need it for perf
annotate. This allows off-box assembly level analysis of perf.data
samples.
This patch complements:
commit ec5761eab318e50e69fcf8e63e9edaef5949c067
Author: David Ahern <daahern@cisco.com>
Date: Thu Dec 9 13:27:07 2010 -0700
perf symbols: Add symfs option for off-box analysis using specified tree
Acked-by: David Ahern <daahern@cisco.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: David Ahern <daahern@cisco.com>
Link: http://lkml.kernel.org/r/20110729232040.GA21838@quad
Signed-off-by: Stephane Eranian <eranian@google.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
This patch adds two new options to perf annotate:
- --no-asm-raw : Do not display raw instruction encodings
- --no-source : Do not interleave source code with assembly code
We believe those options make the output of annotate more readable.
Systematically displaying source can make it hard to follow code and
especially optimized code.
Raw encodings are not useful in most cases.
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20110517153207.GA9834@quad
Signed-off-by: Stephane Eranian <eranian@google.com>
[committer note: Use the 'no-' option inverting logic]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
Add an option to perf report/annotate/script to specify which
CPUs to operate on. This enables us to take a single system wide
profile and analyse each CPU (or group of CPUs) in isolation.
This was useful when profiling a multiprocess workload where the
bottleneck was on one CPU but this was hidden in the overall
profile. Per process and per thread breakdowns didn't help
because multiple processes were running on each CPU and no
single process consumed an entire CPU.
The patch converts the list of CPUs |
