linux.git/kernel/bpf/trampoline.c, branch v5.15.207

x86/ibt,ftrace: Search for fentry location

2024-07-05T07:14:12+00:00

[ Upstream commit aebfd12521d9c7d0b502cf6d06314cfbcdccfe3b ]

Currently a lot of ftrace code assumes __fentry__ is at sym+0. However
with Intel IBT enabled the first instruction of a function will most
likely be ENDBR.

Change ftrace_location() to not only return the __fentry__ location
when called for the __fentry__ location, but also when called for the
sym+0 location.

Then audit/update all callsites of this function to consistently use
these new semantics.

Suggested-by: Steven Rostedt 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Masami Hiramatsu 
Acked-by: Josh Poimboeuf 
Link: https://lore.kernel.org/r/20220308154318.227581603@infradead.org
Stable-dep-of: e60b613df8b6 ("ftrace: Fix possible use-after-free issue in ftrace_location()")
Signed-off-by: Sasha Levin

bpf: Fix potential array overflow in bpf_trampoline_get_progs()

2022-06-06T06:43:42+00:00

commit a2aa95b71c9bbec793b5c5fa50f0a80d882b3e8d upstream.

The cnt value in the 'cnt >= BPF_MAX_TRAMP_PROGS' check does not
include BPF_TRAMP_MODIFY_RETURN bpf programs, so the number of
the attached BPF_TRAMP_MODIFY_RETURN bpf programs in a trampoline
can exceed BPF_MAX_TRAMP_PROGS.

When this happens, the assignment '*progs++ = aux->prog' in
bpf_trampoline_get_progs() will cause progs array overflow as the
progs field in the bpf_tramp_progs struct can only hold at most
BPF_MAX_TRAMP_PROGS bpf programs.

Fixes: 88fd9e5352fe ("bpf: Refactor trampoline update code")
Signed-off-by: Yuntao Wang 
Link: https://lore.kernel.org/r/20220430130803.210624-1-ytcoode@gmail.com
Signed-off-by: Alexei Starovoitov 
Signed-off-by: Greg Kroah-Hartman

bpf: Fix possible race in inc_misses_counter

2022-03-08T18:12:40+00:00

[ Upstream commit 0e3135d3bfa5dfb658145238d2bc723a8e30c3a3 ]

It seems inc_misses_counter() suffers from same issue fixed in
the commit d979617aa84d ("bpf: Fixes possible race in update_prog_stats()
for 32bit arches"):
As it can run while interrupts are enabled, it could
be re-entered and the u64_stats syncp could be mangled.

Fixes: 9ed9e9ba2337 ("bpf: Count the number of times recursion was prevented")
Signed-off-by: He Fengqing 
Acked-by: John Fastabend 
Link: https://lore.kernel.org/r/20220122102936.1219518-1-hefengqing@huawei.com
Signed-off-by: Alexei Starovoitov 
Signed-off-by: Sasha Levin

bpf: Use u64_stats_t in struct bpf_prog_stats

2022-03-08T18:12:40+00:00

[ Upstream commit 61a0abaee2092eee69e44fe60336aa2f5b578938 ]

Commit 316580b69d0a ("u64_stats: provide u64_stats_t type")
fixed possible load/store tearing on 64bit arches.

For instance the following C code

stats->nsecs += sched_clock() - start;

Could be rightfully implemented like this by a compiler,
confusing concurrent readers a lot:

stats->nsecs += sched_clock();
// arbitrary delay
stats->nsecs -= start;

Signed-off-by: Eric Dumazet 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20211026214133.3114279-4-eric.dumazet@gmail.com
Signed-off-by: Sasha Levin

bpf: Fixes possible race in update_prog_stats() for 32bit arches

2021-11-18T18:16:42+00:00

[ Upstream commit d979617aa84d96acca44c2f5778892b4565e322f ]

It seems update_prog_stats() suffers from same issue fixed
in the prior patch:

As it can run while interrupts are enabled, it could
be re-entered and the u64_stats syncp could be mangled.

Fixes: fec56f5890d9 ("bpf: Introduce BPF trampoline")
Signed-off-by: Eric Dumazet 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20211026214133.3114279-3-eric.dumazet@gmail.com
Signed-off-by: Sasha Levin

bpf: Refactor BPF_PROG_RUN into a function

2021-08-16T22:45:07+00:00

Turn BPF_PROG_RUN into a proper always inlined function. No functional and
performance changes are intended, but it makes it much easier to understand
what's going on with how BPF programs are actually get executed. It's more
obvious what types and callbacks are expected. Also extra () around input
parameters can be dropped, as well as `__` variable prefixes intended to avoid
naming collisions, which makes the code simpler to read and write.

This refactoring also highlighted one extra issue. BPF_PROG_RUN is both
a macro and an enum value (BPF_PROG_RUN == BPF_PROG_TEST_RUN). Turning
BPF_PROG_RUN into a function causes naming conflict compilation error. So
rename BPF_PROG_RUN into lower-case bpf_prog_run(), similar to
bpf_prog_run_xdp(), bpf_prog_run_pin_on_cpu(), etc. All existing callers of
BPF_PROG_RUN, the macro, are switched to bpf_prog_run() explicitly.

Signed-off-by: Andrii Nakryiko 
Signed-off-by: Daniel Borkmann 
Acked-by: Yonghong Song 
Link: https://lore.kernel.org/bpf/20210815070609.987780-2-andrii@kernel.org

bpf: Enable BPF_TRAMP_F_IP_ARG for trampolines with call_get_func_ip

2021-07-16T00:16:06+00:00

Enabling BPF_TRAMP_F_IP_ARG for trampolines that actually need it.

The BPF_TRAMP_F_IP_ARG adds extra 3 instructions to trampoline code
and is used only by programs with bpf_get_func_ip helper, which is
added in following patch and sets call_get_func_ip bit.

This patch ensures that BPF_TRAMP_F_IP_ARG flag is used only for
trampolines that have programs with call_get_func_ip set.

Signed-off-by: Jiri Olsa 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20210714094400.396467-3-jolsa@kernel.org

bpf: Fix spelling mistakes

2021-05-25T04:13:05+00:00

Fix some spelling mistakes in comments:
aother ==> another
Netiher ==> Neither
desribe ==> describe
intializing ==> initializing
funciton ==> function
wont ==> won't and move the word 'the' at the end to the next line
accross ==> across
pathes ==> paths
triggerred ==> triggered
excute ==> execute
ether ==> either
conervative ==> conservative
convetion ==> convention
markes ==> marks
interpeter ==> interpreter

Signed-off-by: Zhen Lei 
Signed-off-by: Andrii Nakryiko 
Link: https://lore.kernel.org/bpf/20210525025659.8898-2-thunder.leizhen@huawei.com

bpf: Allow trampoline re-attach for tracing and lsm programs

2021-04-26T04:09:01+00:00

Currently we don't allow re-attaching of trampolines. Once
it's detached, it can't be re-attach even when the program
is still loaded.

Adding the possibility to re-attach the loaded tracing and
lsm programs.

Fixing missing unlock with proper cleanup goto jump reported
by Julia.

Reported-by: kernel test robot 
Reported-by: Julia Lawall 
Signed-off-by: Jiri Olsa 
Signed-off-by: Alexei Starovoitov 
Acked-by: Toke Høiland-Jørgensen 
Acked-by: Andrii Nakryiko 
Acked-by: KP Singh 
Link: https://lore.kernel.org/bpf/20210414195147.1624932-2-jolsa@kernel.org

bpf: Take module reference for trampoline in module

2021-03-27T02:30:11+00:00

Currently module can be unloaded even if there's a trampoline
register in it. It's easily reproduced by running in parallel:

  # while :; do ./test_progs -t module_attach; done
  # while :; do rmmod bpf_testmod; sleep 0.5; done

Taking the module reference in case the trampoline's ip is
within the module code. Releasing it when the trampoline's
ip is unregistered.

Signed-off-by: Jiri Olsa 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20210326105900.151466-1-jolsa@kernel.org