linux.git/lib/lz4, branch v6.12.80

move asm/unaligned.h to linux/unaligned.h

2024-10-02T21:23:23+00:00

asm/unaligned.h is always an include of asm-generic/unaligned.h;
might as well move that thing to linux/unaligned.h and include
that - there's nothing arch-specific in that header.

auto-generated by the following:

for i in `git grep -l -w asm/unaligned.h`; do
	sed -i -e "s/asm\/unaligned.h/linux\/unaligned.h/" $i
done
for i in `git grep -l -w asm-generic/unaligned.h`; do
	sed -i -e "s/asm-generic\/unaligned.h/linux\/unaligned.h/" $i
done
git mv include/asm-generic/unaligned.h include/linux/unaligned.h
git mv tools/include/asm-generic/unaligned.h tools/include/linux/unaligned.h
sed -i -e "/unaligned.h/d" include/asm-generic/Kbuild
sed -i -e "s/__ASM_GENERIC/__LINUX/" include/linux/unaligned.h tools/include/linux/unaligned.h

lib: lz4hc: export LZ4_resetStreamHC symbol

2024-09-09T23:39:06+00:00

This symbol is needed to enable lz4hc dictionary support.

Link: https://lkml.kernel.org/r/20240902105656.1383858-3-senozhatsky@chromium.org
Signed-off-by: Sergey Senozhatsky 
Cc: Nick Terrell 
Cc: Minchan Kim 
Signed-off-by: Andrew Morton

lib: make LZ4_decompress_safe_forceExtDict() static

2022-07-18T00:31:39+00:00

LZ4_decompress_safe_forceExtDict() is only used in
lib/lz4/lz4_decompress.c, make it static to fix the build warning about
"no previous prototype" [1].

[1] https://lore.kernel.org/lkml/202206260948.akgsho1q-lkp@intel.com/

Link: https://lkml.kernel.org/r/1656298965-8698-1-git-send-email-yangtiezhu@loongson.cn
Signed-off-by: Tiezhu Yang 
Reported-by: kernel test robot 
Signed-off-by: Andrew Morton

lz4: fix LZ4_decompress_safe_partial read out of bound

2022-04-09T00:20:36+00:00

When partialDecoding, it is EOF if we've either filled the output buffer
or can't proceed with reading an offset for following match.

In some extreme corner cases when compressed data is suitably corrupted,
UAF will occur.  As reported by KASAN [1], LZ4_decompress_safe_partial
may lead to read out of bound problem during decoding.  lz4 upstream has
fixed it [2] and this issue has been disscussed here [3] before.

current decompression routine was ported from lz4 v1.8.3, bumping
lib/lz4 to v1.9.+ is certainly a huge work to be done later, so, we'd
better fix it first.

[1] https://lore.kernel.org/all/000000000000830d1205cf7f0477@google.com/
[2] https://github.com/lz4/lz4/commit/c5d6f8a8be3927c0bec91bcc58667a6cfad244ad#
[3] https://lore.kernel.org/all/CC666AE8-4CA4-4951-B6FB-A2EFDE3AC03B@fb.com/

Link: https://lkml.kernel.org/r/20211111105048.2006070-1-guoxuenan@huawei.com
Reported-by: syzbot+63d688f1d899c588fb71@syzkaller.appspotmail.com
Signed-off-by: Guo Xuenan 
Reviewed-by: Nick Terrell 
Acked-by: Gao Xiang 
Cc: Yann Collet 
Cc: Chengyang Fan 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

include/linux/unaligned: replace kernel.h with the necessary inclusions

2022-01-20T06:52:53+00:00

When kernel.h is used in the headers it adds a lot into dependency hell,
especially when there are circular dependencies are involved.

Replace kernel.h inclusion with the list of what is really being used.

The rest of the changes are induced by the above and may not be split.

Link: https://lkml.kernel.org/r/20211209123823.20425-1-andriy.shevchenko@linux.intel.com
Signed-off-by: Andy Shevchenko 
Acked-by: Arend van Spriel 	[brcmfmac]
Acked-by: Kalle Valo 
Cc: Arend van Spriel 
Cc: Franky Lin 
Cc: Hante Meuleman 
Cc: Chi-hsien Lin 
Cc: Wright Feng 
Cc: Chung-hsien Hsu 
Cc: Kalle Valo 
Cc: David S. Miller 
Cc: Jakub Kicinski 
Cc: Heikki Krogerus 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static

2021-07-01T18:06:05+00:00

Declare LZ4_decompress_safe_withPrefix64k as static to fix sparse
warning:

> warning: symbol 'LZ4_decompress_safe_withPrefix64k' was not declared.
> Should it be static?

Link: https://lkml.kernel.org/r/20210511154345.610569-1-thisisrast7@gmail.com
Signed-off-by: Rajat Asthana 
Reviewed-by: Nick Terrell 
Cc: Gao Xiang 
Cc: Joe Perches 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/lz4: explicitly support in-place decompression

2020-12-16T06:46:16+00:00

LZ4 final literal copy could be overlapped when doing
in-place decompression, so it's unsafe to just use memcpy()
on an optimized memcpy approach but memmove() instead.

Upstream LZ4 has updated this years ago [1] (and the impact
is non-sensible [2] plus only a few bytes remain), this commit
just synchronizes LZ4 upstream code to the kernel side as well.

It can be observed as EROFS in-place decompression failure
on specific files when X86_FEATURE_ERMS is unsupported,
memcpy() optimization of commit 59daa706fbec ("x86, mem:
Optimize memcpy by avoiding memory false dependece") will
be enabled then.

Currently most modern x86-CPUs support ERMS, these CPUs just
use "rep movsb" approach so no problem at all. However, it can
still be verified with forcely disabling ERMS feature...

arch/x86/lib/memcpy_64.S:
        ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
-                     "jmp memcpy_erms", X86_FEATURE_ERMS
+                     "jmp memcpy_orig", X86_FEATURE_ERMS

We didn't observe any strange on arm64/arm/x86 platform before
since most memcpy() would behave in an increasing address order
("copy upwards" [3]) and it's the correct order of in-place
decompression but it really needs an update to memmove() for sure
considering it's an undefined behavior according to the standard
and some unique optimization already exists in the kernel.

[1] https://github.com/lz4/lz4/commit/33cb8518ac385835cc17be9a770b27b40cd0e15b
[2] https://github.com/lz4/lz4/pull/717#issuecomment-497818921
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518

Link: https://lkml.kernel.org/r/20201122030749.2698994-1-hsiangkao@redhat.com
Signed-off-by: Gao Xiang 
Reviewed-by: Nick Terrell 
Cc: Yann Collet 
Cc: Miao Xie 
Cc: Chao Yu 
Cc: Li Guifu 
Cc: Guo Xuenan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lz4: fix kernel decompression speed

2020-08-15T02:56:56+00:00

This patch replaces all memcpy() calls with LZ4_memcpy() which calls
__builtin_memcpy() so the compiler can inline it.

LZ4 relies heavily on memcpy() with a constant size being inlined.  In x86
and i386 pre-boot environments memcpy() cannot be inlined because memcpy()
doesn't get defined as __builtin_memcpy().

An equivalent patch has been applied upstream so that the next import
won't lose this change [1].

I've measured the kernel decompression speed using QEMU before and after
this patch for the x86_64 and i386 architectures.  The speed-up is about
10x as shown below.

Code	Arch	Kernel Size	Time	Speed
v5.8	x86_64	11504832 B	148 ms	 79 MB/s
patch	x86_64	11503872 B	 13 ms	885 MB/s
v5.8	i386	 9621216 B	 91 ms	106 MB/s
patch	i386	 9620224 B	 10 ms	962 MB/s

I also measured the time to decompress the initramfs on x86_64, i386, and
arm.  All three show the same decompression speed before and after, as
expected.

[1] https://github.com/lz4/lz4/pull/890

Signed-off-by: Nick Terrell 
Signed-off-by: Andrew Morton 
Cc: Yann Collet 
Cc: Gao Xiang 
Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
Cc: Greg Kroah-Hartman 
Cc: Ingo Molnar 
Cc: Arvind Sankar 
Link: http://lkml.kernel.org/r/20200803194022.2966806-1-nickrterrell@gmail.com
Signed-off-by: Linus Torvalds

lib/lz4/lz4_decompress.c: document deliberate use of `&'

2020-06-11T02:14:17+00:00

This operation was intentional, but tools such as smatch will warn that it
might not have been.

Signed-off-by: Andrew Morton 
Cc: Yann Collet 
Cc: Vasily Averin 
Cc: Gao Xiang 
Link: http://lkml.kernel.org/r/3bf931c6ea0cae3e23f3485801986859851b4f04.camel@perches.com
Signed-off-by: Linus Torvalds

lz4: do not export static symbol

2019-09-20T16:06:26+00:00

Kbuild now complains (rightly) about it.

Signed-off-by: Linus Torvalds