diff options
| author | Jiayuan Chen <jiayuan.chen@linux.dev> | 2025-04-07 22:21:20 +0800 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2025-06-27 11:07:35 +0100 |
| commit | ed509140af3a1cc67614a50f2c27c93332cbd242 (patch) | |
| tree | cc7df4cbc592dcb0c68ec615fcb1cf1010f24370 /net | |
| parent | f5003f3cbef9d2a0e8e4cc57fcfbd179d70b29ea (diff) | |
| download | linux-ed509140af3a1cc67614a50f2c27c93332cbd242.tar.gz linux-ed509140af3a1cc67614a50f2c27c93332cbd242.tar.bz2 linux-ed509140af3a1cc67614a50f2c27c93332cbd242.zip | |
bpf, sockmap: Fix data lost during EAGAIN retries
[ Upstream commit 7683167196bd727ad5f3c3fc6a9ca70f54520a81 ]
We call skb_bpf_redirect_clear() to clean _sk_redir before handling skb in
backlog, but when sk_psock_handle_skb() return EAGAIN due to sk_rcvbuf
limit, the redirect info in _sk_redir is not recovered.
Fix skb redir loss during EAGAIN retries by restoring _sk_redir
information using skb_bpf_set_redir().
Before this patch:
'''
./bench sockmap -c 2 -p 1 -a --rx-verdict-ingress
Setting up benchmark 'sockmap'...
create socket fd c1:13 p1:14 c2:15 p2:16
Benchmark 'sockmap' started.
Send Speed 1343.172 MB/s, BPF Speed 1343.238 MB/s, Rcv Speed 65.271 MB/s
Send Speed 1352.022 MB/s, BPF Speed 1352.088 MB/s, Rcv Speed 0 MB/s
Send Speed 1354.105 MB/s, BPF Speed 1354.105 MB/s, Rcv Speed 0 MB/s
Send Speed 1355.018 MB/s, BPF Speed 1354.887 MB/s, Rcv Speed 0 MB/s
'''
Due to the high send rate, the RX processing path may frequently hit the
sk_rcvbuf limit. Once triggered, incorrect _sk_redir will cause the flow
to mistakenly enter the "!ingress" path, leading to send failures.
(The Rcv speed depends on tcp_rmem).
After this patch:
'''
./bench sockmap -c 2 -p 1 -a --rx-verdict-ingress
Setting up benchmark 'sockmap'...
create socket fd c1:13 p1:14 c2:15 p2:16
Benchmark 'sockmap' started.
Send Speed 1347.236 MB/s, BPF Speed 1347.367 MB/s, Rcv Speed 65.402 MB/s
Send Speed 1353.320 MB/s, BPF Speed 1353.320 MB/s, Rcv Speed 65.536 MB/s
Send Speed 1353.186 MB/s, BPF Speed 1353.121 MB/s, Rcv Speed 65.536 MB/s
'''
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20250407142234.47591-2-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Diffstat (limited to 'net')
| -rw-r--r-- | net/core/skmsg.c | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/net/core/skmsg.c b/net/core/skmsg.c index e5ba57a5db12..2aa6262f19e8 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -688,7 +688,8 @@ static void sk_psock_backlog(struct work_struct *work) if (ret <= 0) { if (ret == -EAGAIN) { sk_psock_skb_state(psock, state, len, off); - + /* Restore redir info we cleared before */ + skb_bpf_set_redir(skb, psock->sk, ingress); /* Delay slightly to prioritize any * other work that might be here. */ |