linux.git/arch/arm64/lib, branch v5.12.5

arm64: kasan: simplify and inline MTE functions

2021-02-26T17:41:03+00:00

This change provides a simpler implementation of mte_get_mem_tag(),
mte_get_random_tag(), and mte_set_mem_tag_range().

Simplifications include removing system_supports_mte() checks as these
functions are onlye called from KASAN runtime that had already checked
system_supports_mte().  Besides that, size and address alignment checks
are removed from mte_set_mem_tag_range(), as KASAN now does those.

This change also moves these functions into the asm/mte-kasan.h header and
implements mte_set_mem_tag_range() via inline assembly to avoid
unnecessary functions calls.

[vincenzo.frascino@arm.com: fix warning in mte_get_random_tag()]
  Link: https://lkml.kernel.org/r/20210211152208.23811-1-vincenzo.frascino@arm.com

Link: https://lkml.kernel.org/r/a26121b294fdf76e369cb7a74351d1c03a908930.1612546384.git.andreyknvl@google.com
Co-developed-by: Vincenzo Frascino 
Signed-off-by: Vincenzo Frascino 
Signed-off-by: Andrey Konovalov 
Reviewed-by: Catalin Marinas 
Cc: Alexander Potapenko 
Cc: Andrey Ryabinin 
Cc: Branislav Rankov 
Cc: Dmitry Vyukov 
Cc: Evgenii Stepanov 
Cc: Kevin Brodsky 
Cc: Marco Elver 
Cc: Peter Collingbourne 
Cc: Will Deacon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: mte: add in-kernel MTE helpers

2020-12-22T20:55:07+00:00

Provide helper functions to manipulate allocation and pointer tags for
kernel addresses.

Low-level helper functions (mte_assign_*, written in assembly) operate tag
values from the [0x0, 0xF] range.  High-level helper functions
(mte_get/set_*) use the [0xF0, 0xFF] range to preserve compatibility with
normal kernel pointers that have 0xFF in their top byte.

MTE_GRANULE_SIZE and related definitions are moved to mte-def.h header
that doesn't have any dependencies and is safe to include into any
low-level header.

Link: https://lkml.kernel.org/r/c31bf759b4411b2d98cdd801eb928e241584fd1f.1606161801.git.andreyknvl@google.com
Signed-off-by: Vincenzo Frascino 
Co-developed-by: Andrey Konovalov 
Signed-off-by: Andrey Konovalov 
Reviewed-by: Catalin Marinas 
Tested-by: Vincenzo Frascino 
Cc: Alexander Potapenko 
Cc: Andrey Ryabinin 
Cc: Branislav Rankov 
Cc: Dmitry Vyukov 
Cc: Evgenii Stepanov 
Cc: Kevin Brodsky 
Cc: Marco Elver 
Cc: Vasily Gorbik 
Cc: Will Deacon 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

arm64: uaccess cleanup macro naming

2020-12-02T19:49:11+00:00

Now the uaccess primitives use LDTR/STTR unconditionally, the
uao_{ldp,stp,user_alternative} asm macros are misnamed, and have a
redundant argument. Let's remove the redundant argument and rename these
to user_{ldp,stp,ldst} respectively to clean this up.

Signed-off-by: Mark Rutland 
Reviewed-by: Robin Murohy 
Cc: Christoph Hellwig 
Cc: James Morse 
Cc: Will Deacon 
Link: https://lore.kernel.org/r/20201202131558.39270-9-mark.rutland@arm.com
Signed-off-by: Catalin Marinas

arm64: uaccess: simplify __copy_user_flushcache()

2020-12-02T19:49:10+00:00

Currently __copy_user_flushcache() open-codes raw_copy_from_user(), and
doesn't use uaccess_mask_ptr() on the user address. Let's have it call
raw_copy_from_user(), which is both a simplification and ensures that
user pointers are masked under speculation.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Robin Murphy 
Cc: Christoph Hellwig 
Cc: Will Deacon 
Link: https://lore.kernel.org/r/20201202131558.39270-6-mark.rutland@arm.com
Signed-off-by: Catalin Marinas

arm64: uaccess: move uao_* alternatives to asm-uaccess.h

2020-11-09T21:49:34+00:00

The uao_* alternative asm macros are only used by the uaccess assembly
routines in arch/arm64/lib/, where they are included indirectly via
asm-uaccess.h. Since they're specific to the uaccess assembly (and will
lose the alternatives in subsequent patches), let's move them into
asm-uaccess.h.

There should be no functional change as a result of this patch.

Signed-off-by: Mark Rutland 
Cc: Catalin Marinas 
Cc: Christoph Hellwig 
Cc: James Morse 
Cc: Will Deacon 
[will: update #include in mte.S to pull in uao asm macros]
Signed-off-by: Will Deacon

arm64: Change .weak to SYM_FUNC_START_WEAK_PI for arch/arm64/lib/mem*.S

2020-10-30T08:32:31+00:00

Commit 39d114ddc682 ("arm64: add KASAN support") added .weak directives to
arch/arm64/lib/mem*.S instead of changing the existing SYM_FUNC_START_PI
macros. This can lead to the assembly snippet `.weak memcpy ... .globl
memcpy` which will produce a STB_WEAK memcpy with GNU as but STB_GLOBAL
memcpy with LLVM's integrated assembler before LLVM 12. LLVM 12 (since
https://reviews.llvm.org/D90108) will error on such an overridden symbol
binding.

Use the appropriate SYM_FUNC_START_WEAK_PI instead.

Fixes: 39d114ddc682 ("arm64: add KASAN support")
Reported-by: Sami Tolvanen 
Signed-off-by: Fangrui Song 
Tested-by: Sami Tolvanen 
Tested-by: Nick Desaulniers 
Reviewed-by: Nick Desaulniers 
Cc: 
Link: https://lore.kernel.org/r/20201029181951.1866093-1-maskray@google.com
Signed-off-by: Will Deacon

arm64: mte: Enable swap of tagged pages

2020-09-04T11:46:07+00:00

When swapping pages out to disk it is necessary to save any tags that
have been set, and restore when swapping back in. Make use of the new
page flag (PG_ARCH_2, locally named PG_mte_tagged) to identify pages
with tags. When swapping out these pages the tags are stored in memory
and later restored when the pages are brought back in. Because shmem can
swap pages back in without restoring the userspace PTE it is also
necessary to add a hook for shmem.

Signed-off-by: Steven Price 
[catalin.marinas@arm.com: move function prototypes to mte.h]
[catalin.marinas@arm.com: drop '_tags' from arch_swap_restore_tags()]
Signed-off-by: Catalin Marinas 
Cc: Andrew Morton 
Cc: Will Deacon

arm64: mte: ptrace: Add PTRACE_{PEEK,POKE}MTETAGS support

2020-09-04T11:46:07+00:00

Add support for bulk setting/getting of the MTE tags in a tracee's
address space at 'addr' in the ptrace() syscall prototype. 'data' points
to a struct iovec in the tracer's address space with iov_base
representing the address of a tracer's buffer of length iov_len. The
tags to be copied to/from the tracer's buffer are stored as one tag per
byte.

On successfully copying at least one tag, ptrace() returns 0 and updates
the tracer's iov_len with the number of tags copied. In case of error,
either -EIO or -EFAULT is returned, trying to follow the ptrace() man
page.

Note that the tag copying functions are not performance critical,
therefore they lack optimisations found in typical memory copy routines.

Signed-off-by: Catalin Marinas 
Cc: Will Deacon 
Cc: Alan Hayward 
Cc: Luis Machado 
Cc: Omair Javaid

arm64: mte: Tags-aware copy_{user_,}highpage() implementations

2020-09-04T11:46:06+00:00

When the Memory Tagging Extension is enabled, the tags need to be
preserved across page copy (e.g. for copy-on-write, page migration).

Introduce MTE-aware copy_{user_,}highpage() functions to copy tags to
the destination if the source page has the PG_mte_tagged flag set.
copy_user_page() does not need to handle tag copying since, with this
patch, it is only called by the DAX code where there is no source page
structure (and no source tags).

Signed-off-by: Vincenzo Frascino 
Co-developed-by: Catalin Marinas 
Signed-off-by: Catalin Marinas 
Cc: Will Deacon

arm64: mte: Clear the tags when a page is mapped in user-space with PROT_MTE

2020-09-04T11:46:06+00:00

Pages allocated by the kernel are not guaranteed to have the tags
zeroed, especially as the kernel does not (yet) use MTE itself. To
ensure the user can still access such pages when mapped into its address
space, clear the tags via set_pte_at(). A new page flag - PG_mte_tagged
(PG_arch_2) - is used to track pages with valid allocation tags.

Since the zero page is mapped as pte_special(), it won't be covered by
the above set_pte_at() mechanism. Clear its tags during early MTE
initialisation.

Co-developed-by: Steven Price 
Signed-off-by: Steven Price 
Signed-off-by: Catalin Marinas 
Cc: Will Deacon