linux.git/arch/arm/net, branch v4.19.205

arm, bpf: Fix bugs with ALU64 {RSH, ARSH} BPF_K shift by 0

2020-04-23T08:30:12+00:00

commit bb9562cf5c67813034c96afb50bd21130a504441 upstream.

The current arm BPF JIT does not correctly compile RSH or ARSH when the
immediate shift amount is 0. This causes the "rsh64 by 0 imm" and "arsh64
by 0 imm" BPF selftests to hang the kernel by reaching an instruction
the verifier determines to be unreachable.

The root cause is in how immediate right shifts are encoded on arm.
For LSR and ASR (logical and arithmetic right shift), a bit-pattern
of 00000 in the immediate encodes a shift amount of 32. When the BPF
immediate is 0, the generated code shifts by 32 instead of the expected
behavior (a no-op).

This patch fixes the bugs by adding an additional check if the BPF
immediate is 0. After the change, the above mentioned BPF selftests pass.

Fixes: 39c13c204bb11 ("arm: eBPF JIT compiler")
Co-developed-by: Xi Wang 
Signed-off-by: Xi Wang 
Signed-off-by: Luke Nelson 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20200408181229.10909-1-luke.r.nels@gmail.com
Signed-off-by: Greg Kroah-Hartman

arm, bpf: Fix offset overflow for BPF_MEM BPF_DW

2020-04-23T08:30:11+00:00

commit 4178417cc5359c329790a4a8f4a6604612338cca upstream.

This patch fixes an incorrect check in how immediate memory offsets are
computed for BPF_DW on arm.

For BPF_LDX/ST/STX + BPF_DW, the 32-bit arm JIT breaks down an 8-byte
access into two separate 4-byte accesses using off+0 and off+4. If off
fits in imm12, the JIT emits a ldr/str instruction with the immediate
and avoids the use of a temporary register. While the current check off
<= 0xfff ensures that the first immediate off+0 doesn't overflow imm12,
it's not sufficient for the second immediate off+4, which may cause the
second access of BPF_DW to read/write the wrong address.

This patch fixes the problem by changing the check to
off <= 0xfff - 4 for BPF_DW, ensuring off+4 will never overflow.

A side effect of simplifying the check is that it now allows using
negative immediate offsets in ldr/str. This means that small negative
offsets can also avoid the use of a temporary register.

This patch introduces no new failures in test_verifier or test_bpf.c.

Fixes: c5eae692571d6 ("ARM: net: bpf: improve 64-bit store implementation")
Fixes: ec19e02b343db ("ARM: net: bpf: fix LDX instructions")
Co-developed-by: Xi Wang 
Signed-off-by: Xi Wang 
Signed-off-by: Luke Nelson 
Signed-off-by: Daniel Borkmann 
Link: https://lore.kernel.org/bpf/20200409221752.28448-1-luke.r.nels@gmail.com
Signed-off-by: Greg Kroah-Hartman

ARM: net: bpf: improve 64-bit ALU implementation

2018-07-13T13:26:42+00:00

Improbe the 64-bit ALU implementation from:

  movw    r8, #65532
  movt    r8, #65535
  movw    r9, #65535
  movt    r9, #65535
  ldr     r7, [fp, #-44]
  adds    r7, r7, r8
  str     r7, [fp, #-44]
  ldr     r7, [fp, #-40]
  adc     r7, r7, r9
  str     r7, [fp, #-40]

to:

  movw    r8, #65532
  movt    r8, #65535
  movw    r9, #65535
  movt    r9, #65535
  ldrd    r6, [fp, #-44]
  adds    r6, r6, r8
  adc     r7, r7, r9
  strd    r6, [fp, #-44]

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: improve 64-bit store implementation

2018-07-13T13:26:42+00:00

Improve the 64-bit store implementation from:

  ldr     r6, [fp, #-8]
  str     r8, [r6]
  ldr     r6, [fp, #-8]
  mov     r7, #4
  add     r7, r6, r7
  str     r9, [r7]

to:

  ldr     r6, [fp, #-8]
  str     r8, [r6]
  str     r9, [r6, #4]

We leave the store as two separate STR instructions rather than using
STRD as the store may not be aligned, and STR can handle misalignment.

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: improve 64-bit sign-extended immediate load

2018-07-13T13:26:41+00:00

Improve the 64-bit sign-extended immediate from:

  mov     r6, #1
  str     r6, [fp, #-52]  ; 0xffffffcc
  mov     r6, #0
  str     r6, [fp, #-48]  ; 0xffffffd0

to:

  mov     r6, #1
  mov     r7, #0
  strd    r6, [fp, #-52]  ; 0xffffffcc

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: improve 64-bit load immediate implementation

2018-07-13T13:26:41+00:00

Rather than writing each 32-bit half of the 64-bit immediate value
separately when the register is on the stack:

  movw    r6, #45056      ; 0xb000
  movt    r6, #60979      ; 0xee33
  str     r6, [fp, #-44]  ; 0xffffffd4
  mov     r6, #0
  str     r6, [fp, #-40]  ; 0xffffffd8

arrange to use the double-word store when available instead:

  movw    r6, #45056      ; 0xb000
  movt    r6, #60979      ; 0xee33
  mov     r7, #0
  strd    r6, [fp, #-44]  ; 0xffffffd4

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: use double-word load/stores where available

2018-07-12T18:45:23+00:00

Use double-word load and stores where support for this instruction is
supported by the CPU architecture.

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: always use odd/even register pair

2018-07-12T18:45:23+00:00

Always use an odd/even register pair for our 64-bit registers, so that
we're able to use the double-word load/store instructions in the future.

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: avoid reloading 'array'

2018-07-12T18:45:23+00:00

Rearranging the order of the initial tail call code a little allows is
to avoid reloading the 'array' pointer.

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann

ARM: net: bpf: avoid reloading 'index'

2018-07-12T18:45:23+00:00

Avoid reloading 'index' after we have validated it - it remains in
tmp2[1] up to the point that we begin the code to index the pointer
array, so with a little rearrangement of the registers, we can use
the already loaded value.

Signed-off-by: Russell King 
Signed-off-by: Daniel Borkmann