From 46d28947d9876fc0f8f93d3c69813ef6e9852595 Mon Sep 17 00:00:00 2001 From: Thomas Gleixner Date: Wed, 8 Sep 2021 15:29:18 +0200 Subject: x86/extable: Rework the exception table mechanics The exception table entries contain the instruction address, the fixup address and the handler address. All addresses are relative. Storing the handler address has a few downsides: 1) Most handlers need to be exported 2) Handlers can be defined everywhere and there is no overview about the handler types 3) MCE needs to check the handler type to decide whether an in kernel #MC can be recovered. The functionality of the handler itself is not in any way special, but for these checks there need to be separate functions which in the worst case have to be exported. Some of these 'recoverable' exception fixups are pretty obscure and just reuse some other handler to spare code. That obfuscates e.g. the #MC safe copy functions. Cleaning that up would require more handlers and exports Rework the exception fixup mechanics by storing a fixup type number instead of the handler address and invoke the proper handler for each fixup type. Also teach the extable sort to leave the type field alone. This makes most handlers static except for special cases like the MCE MSR fixup and the BPF fixup. This allows to add more types for cleaning up the obscure places without adding more handler code and exports. There is a marginal code size reduction for a production config and it removes _eight_ exported symbols. Signed-off-by: Thomas Gleixner Signed-off-by: Borislav Petkov Acked-by: Alexei Starovoitov Link: https://lkml.kernel.org/r/20210908132525.211958725@linutronix.de --- scripts/sorttable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index f355869c65cd..a9b3324cacf9 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -236,7 +236,7 @@ static void x86_sort_relative_table(char *extab_image, int image_size) w(r(loc) + i, loc); w(r(loc + 1) + i + 4, loc + 1); - w(r(loc + 2) + i + 8, loc + 2); + /* Don't touch the fixup type */ i += sizeof(uint32_t) * 3; } @@ -249,7 +249,7 @@ static void x86_sort_relative_table(char *extab_image, int image_size) w(r(loc) - i, loc); w(r(loc + 1) - (i + 4), loc + 1); - w(r(loc + 2) - (i + 8), loc + 2); + /* Don't touch the fixup type */ i += sizeof(uint32_t) * 3; } -- cgit v1.2.3 From d6e2cc56477538255160ed02fdb11b0da60356cc Mon Sep 17 00:00:00 2001 From: Mark Rutland Date: Tue, 19 Oct 2021 17:02:16 +0100 Subject: arm64: extable: add `type` and `data` fields Subsequent patches will add specialized handlers for fixups, in addition to the simple PC fixup and BPF handlers we have today. In preparation, this patch adds a new `type` field to struct exception_table_entry, and uses this to distinguish the fixup and BPF cases. A `data` field is also added so that subsequent patches can associate data specific to each exception site (e.g. register numbers). Handlers are named ex_handler_*() for consistency, following the exmaple of x86. At the same time, get_ex_fixup() is split out into a helper so that it can be used by other ex_handler_*() functions ins subsequent patches. This patch will increase the size of the exception tables, which will be remedied by subsequent patches removing redundant fixup code. There should be no functional change as a result of this patch. Since each entry is now 12 bytes in size, we must reduce the alignment of each entry from `.align 3` (i.e. 8 bytes) to `.align 2` (i.e. 4 bytes), which is the natrual alignment of the `insn` and `fixup` fields. The current 8-byte alignment is a holdover from when the `insn` and `fixup` fields was 8 bytes, and while not harmful has not been necessary since commit: 6c94f27ac847ff8e ("arm64: switch to relative exception tables") Similarly, RO_EXCEPTION_TABLE_ALIGN is dropped to 4 bytes. Concurrently with this patch, x86's exception table entry format is being updated (similarly to a 12-byte format, with 32-bytes of absolute data). Once both have been merged it should be possible to unify the sorttable logic for the two. Signed-off-by: Mark Rutland Reviewed-by: Ard Biesheuvel Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Catalin Marinas Cc: Daniel Borkmann Cc: James Morse Cc: Jean-Philippe Brucker Cc: Robin Murphy Cc: Will Deacon Link: https://lore.kernel.org/r/20211019160219.5202-11-mark.rutland@arm.com Signed-off-by: Will Deacon --- scripts/sorttable.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 6ee4fa882919..ee95bb47a50d 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -231,6 +231,34 @@ static void sort_relative_table(char *extab_image, int image_size) } } +static void arm64_sort_relative_table(char *extab_image, int image_size) +{ + int i = 0; + + while (i < image_size) { + uint32_t *loc = (uint32_t *)(extab_image + i); + + w(r(loc) + i, loc); + w(r(loc + 1) + i + 4, loc + 1); + /* Don't touch the fixup type or data */ + + i += sizeof(uint32_t) * 3; + } + + qsort(extab_image, image_size / 12, 12, compare_relative_table); + + i = 0; + while (i < image_size) { + uint32_t *loc = (uint32_t *)(extab_image + i); + + w(r(loc) - i, loc); + w(r(loc + 1) - (i + 4), loc + 1); + /* Don't touch the fixup type or data */ + + i += sizeof(uint32_t) * 3; + } +} + static void x86_sort_relative_table(char *extab_image, int image_size) { int i = 0; @@ -343,6 +371,8 @@ static int do_file(char const *const fname, void *addr) custom_sort = s390_sort_relative_table; break; case EM_AARCH64: + custom_sort = arm64_sort_relative_table; + break; case EM_PARISC: case EM_PPC: case EM_PPC64: -- cgit v1.2.3 From e232333be69ee9ccd4281032af0d2416940cb98d Mon Sep 17 00:00:00 2001 From: Mark Rutland Date: Mon, 8 Nov 2021 11:42:20 +0000 Subject: scripts/sorttable: Unify arm64 & x86 sort functions The format of the arm64 and x86 exception table entries is essentially the same as of commits: 46d28947d9876fc0 ("x86/extable: Rework the exception table mechanics") d6e2cc5647753825 ("arm64: extable: add `type` and `data` fields") Both use a 12-byte entry consisting of two 32-bit relative offsets and 32 bits of (absolute) data, and their sort functions are identical aside from commentary, with arm64 saying: /* Don't touch the fixup type or data */ ... and x86 saying: /* Don't touch the fixup type */ Unify the two behind a common sort_relative_table_with_data() function, retaining the arm64 commentary. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Ard Biesheuvel Cc: Borislav Petkov Cc: Catalin Marinas Cc: Thomas Gleixner Cc: Will Deacon Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Palmer Dabbelt --- scripts/sorttable.c | 36 +++--------------------------------- 1 file changed, 3 insertions(+), 33 deletions(-) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index b7c2ad71f9cf..ca9db62bf766 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -231,7 +231,7 @@ static void sort_relative_table(char *extab_image, int image_size) } } -static void arm64_sort_relative_table(char *extab_image, int image_size) +static void sort_relative_table_with_data(char *extab_image, int image_size) { int i = 0; @@ -259,34 +259,6 @@ static void arm64_sort_relative_table(char *extab_image, int image_size) } } -static void x86_sort_relative_table(char *extab_image, int image_size) -{ - int i = 0; - - while (i < image_size) { - uint32_t *loc = (uint32_t *)(extab_image + i); - - w(r(loc) + i, loc); - w(r(loc + 1) + i + 4, loc + 1); - /* Don't touch the fixup type */ - - i += sizeof(uint32_t) * 3; - } - - qsort(extab_image, image_size / 12, 12, compare_relative_table); - - i = 0; - while (i < image_size) { - uint32_t *loc = (uint32_t *)(extab_image + i); - - w(r(loc) - i, loc); - w(r(loc + 1) - (i + 4), loc + 1); - /* Don't touch the fixup type */ - - i += sizeof(uint32_t) * 3; - } -} - static void s390_sort_relative_table(char *extab_image, int image_size) { int i; @@ -364,15 +336,13 @@ static int do_file(char const *const fname, void *addr) switch (r2(&ehdr->e_machine)) { case EM_386: + case EM_AARCH64: case EM_X86_64: - custom_sort = x86_sort_relative_table; + custom_sort = sort_relative_table_with_data; break; case EM_S390: custom_sort = s390_sort_relative_table; break; - case EM_AARCH64: - custom_sort = arm64_sort_relative_table; - break; case EM_PARISC: case EM_PPC: case EM_PPC64: -- cgit v1.2.3 From bb1f85d6046f0db757ac52ed60a5eba5df394819 Mon Sep 17 00:00:00 2001 From: Jisheng Zhang Date: Thu, 18 Nov 2021 19:22:51 +0800 Subject: riscv: switch to relative exception tables Similar as other architectures such as arm64, x86 and so on, use offsets relative to the exception table entry values rather than absolute addresses for both the exception locationand the fixup. However, RISCV label difference will actually produce two relocations, a pair of R_RISCV_ADD32 and R_RISCV_SUB32. Take below simple code for example: $ cat test.S .section .text 1: nop .section __ex_table,"a" .balign 4 .long (1b - .) .previous $ riscv64-linux-gnu-gcc -c test.S $ riscv64-linux-gnu-readelf -r test.o Relocation section '.rela__ex_table' at offset 0x100 contains 2 entries: Offset Info Type Sym. Value Sym. Name + Addend 000000000000 000600000023 R_RISCV_ADD32 0000000000000000 .L1^B1 + 0 000000000000 000500000027 R_RISCV_SUB32 0000000000000000 .L0 + 0 The modpost will complain the R_RISCV_SUB32 relocation, so we need to patch modpost.c to skip this relocation for .rela__ex_table section. After this patch, the __ex_table section size of defconfig vmlinux is reduced from 7072 Bytes to 3536 Bytes. Signed-off-by: Jisheng Zhang Reviewed-by: Kefeng Wang Signed-off-by: Palmer Dabbelt --- scripts/sorttable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index ca9db62bf766..f4a8255036b5 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -346,6 +346,7 @@ static int do_file(char const *const fname, void *addr) case EM_PARISC: case EM_PPC: case EM_PPC64: + case EM_RISCV: custom_sort = sort_relative_table; break; case EM_ARCOMPACT: @@ -353,7 +354,6 @@ static int do_file(char const *const fname, void *addr) case EM_ARM: case EM_MICROBLAZE: case EM_MIPS: - case EM_RISCV: case EM_XTENSA: break; default: -- cgit v1.2.3 From 2bf847db0c7437c28b10fba2981b9a7db4b4e0e2 Mon Sep 17 00:00:00 2001 From: Jisheng Zhang Date: Thu, 18 Nov 2021 19:26:05 +0800 Subject: riscv: extable: add `type` and `data` fields This is a riscv port of commit d6e2cc564775 ("arm64: extable: add `type` and `data` fields"). Signed-off-by: Jisheng Zhang Signed-off-by: Palmer Dabbelt --- scripts/sorttable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index f4a8255036b5..82b162b3941b 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -337,6 +337,7 @@ static int do_file(char const *const fname, void *addr) switch (r2(&ehdr->e_machine)) { case EM_386: case EM_AARCH64: + case EM_RISCV: case EM_X86_64: custom_sort = sort_relative_table_with_data; break; @@ -346,7 +347,6 @@ static int do_file(char const *const fname, void *addr) case EM_PARISC: case EM_PPC: case EM_PPC64: - case EM_RISCV: custom_sort = sort_relative_table; break; case EM_ARCOMPACT: -- cgit v1.2.3 From 72b3942a173c387b27860ba1069636726e208777 Mon Sep 17 00:00:00 2001 From: Yinan Liu Date: Sun, 12 Dec 2021 19:33:58 +0800 Subject: scripts: ftrace - move the sort-processing in ftrace_init When the kernel starts, the initialization of ftrace takes up a portion of the time (approximately 6~8ms) to sort mcount addresses. We can save this time by moving mcount-sorting to compile time. Link: https://lkml.kernel.org/r/20211212113358.34208-2-yinan@linux.alibaba.com Signed-off-by: Yinan Liu Reported-by: kernel test robot Reported-by: kernel test robot Signed-off-by: Steven Rostedt --- scripts/sorttable.c | 2 ++ 1 file changed, 2 insertions(+) (limited to 'scripts/sorttable.c') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index b7c2ad71f9cf..70bdc787ddfb 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -30,6 +30,8 @@ #include #include #include +#include +#include #include #include -- cgit v1.2.3