linux.git/arch/arc/lib, branch v6.12.80 Clone of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git ARCv2: memset: don't prefetch for len == 0 which happens a alot 2023-08-17T19:40:26+00:00 Vineet Gupta vgupta@kernel.org 2020-02-07T20:27:02+00:00 1918693ff1891ba57af22dbbf511cf300158a975 This avoids potential "bleeding" when size == 0 as cache line would be dirtied (and possibly fetched from other cores) and due to the same reaons more optimal too. Signed-off-by: Vineet Gupta <vgupta@kernel.org> 
This avoids potential "bleeding" when size == 0 as cache line would be
dirtied (and possibly fetched from other cores) and due to the same
reaons more optimal too.

Signed-off-by: Vineet Gupta <vgupta@kernel.org>
treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19T15:09:55+00:00 Thomas Gleixner tglx@linutronix.de 2019-06-04T08:11:33+00:00 d2912cb15bdda8ba4a5dd73396ad62641af2f520 Based on 2 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation # extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 4122 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Enrico Weigelt <info@metux.net> Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> 
Based on 2 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ARC: memset: fix build with L1_CACHE_SHIFT != 6 2019-04-08T15:41:44+00:00 Eugeniy Paltsev Eugeniy.Paltsev@synopsys.com 2019-04-08T13:04:38+00:00 55c0c4c793b538fb438bcc72481b9dc2f79fe5a9 In case of 'L1_CACHE_SHIFT != 6' we define dummy assembly macroses PREALLOC_INSTR and PREFETCHW_INSTR without arguments. However we pass arguments to them in code which cause build errors. Fix that. Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Cc: <stable@vger.kernel.org> [5.0] Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
In case of 'L1_CACHE_SHIFT != 6' we define dummy assembly macroses
PREALLOC_INSTR and PREFETCHW_INSTR without arguments. However
we pass arguments to them in code which cause build errors.
Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Cc: <stable@vger.kernel.org>    [5.0]
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
ARCv2: Add explcit unaligned access support (and ability to disable too) 2019-02-25T20:10:58+00:00 Eugeniy Paltsev eugeniy.paltsev@synopsys.com 2019-01-30T16:32:41+00:00 76551468833cd5c356b1d9ff4bc9393fcf768a59 As of today we enable unaligned access unconditionally on ARCv2. Do this under a Kconfig option to allow disable it for test, benchmarking etc. Also while at it - Select HAVE_EFFICIENT_UNALIGNED_ACCESS - Although gcc defaults to unaligned access (since GNU 2018.03), add the right toggles for enabling or disabling as appropriate - update bootlog to prints both HW feature status (exists, enabled/disabled) and SW status (used / not used). - wire up the relaxed memcpy for unaligned access Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> [vgupta: squashed patches, handle gcc -mno-unaligned-access quick] 
As of today we enable unaligned access unconditionally on ARCv2.
Do this under a Kconfig option to allow disable it for test, benchmarking
etc. Also while at it

  - Select HAVE_EFFICIENT_UNALIGNED_ACCESS
  - Although gcc defaults to unaligned access (since GNU 2018.03), add the
    right toggles for enabling or disabling as appropriate
  - update bootlog to prints both HW feature status (exists, enabled/disabled)
    and SW status (used / not used).
  - wire up the relaxed memcpy for unaligned access

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
[vgupta: squashed patches, handle gcc -mno-unaligned-access quick]
ARCv2: lib: introduce memcpy optimized for unaligned access 2019-02-25T16:52:16+00:00 Eugeniy Paltsev eugeniy.paltsev@synopsys.com 2019-01-30T16:32:43+00:00 4d1e7918aae59ef504f5170a4f0c7ae82339fcb2 Optimise code to use efficient unaligned memory access which is available on ARCv2. This allows us to really simplify memcpy code and speed up the code one and a half times (in case of unaligned source or destination). Don't wire it up yet ! Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
Optimise code to use efficient unaligned memory access which is
available on ARCv2. This allows us to really simplify memcpy code
and speed up the code one and a half times (in case of unaligned
source or destination).

Don't wire it up yet !

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
ARCv2: lib: memcpy: fix doing prefetchw outside of buffer 2019-02-21T19:03:16+00:00 Eugeniy Paltsev eugeniy.paltsev@synopsys.com 2019-01-30T16:32:40+00:00 f8a15f97664178f27dfbf86a38f780a532cb6df0 ARCv2 optimized memcpy uses PREFETCHW instruction for prefetching the next cache line but doesn't ensure that the line is not past the end of the buffer. PRETECHW changes the line ownership and marks it dirty, which can cause data corruption if this area is used for DMA IO. Fix the issue by avoiding the PREFETCHW. This leads to performance degradation but it is OK as we'll introduce new memcpy implementation optimized for unaligned memory access using. We also cut off all PREFETCH instructions at they are quite useless here: * we call PREFETCH right before LOAD instruction call. * we copy 16 or 32 bytes of data (depending on CONFIG_ARC_HAS_LL64) in a main logical loop. so we call PREFETCH 4 times (or 2 times) for each L1 cache line (in case of 64B L1 cache Line which is default case). Obviously this is not optimal. Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
ARCv2 optimized memcpy uses PREFETCHW instruction for prefetching the
next cache line but doesn't ensure that the line is not past the end of
the buffer. PRETECHW changes the line ownership and marks it dirty,
which can cause data corruption if this area is used for DMA IO.

Fix the issue by avoiding the PREFETCHW. This leads to performance
degradation but it is OK as we'll introduce new memcpy implementation
optimized for unaligned memory access using.

We also cut off all PREFETCH instructions at they are quite useless
here:
 * we call PREFETCH right before LOAD instruction call.
 * we copy 16 or 32 bytes of data (depending on CONFIG_ARC_HAS_LL64)
   in a main logical loop. so we call PREFETCH 4 times (or 2 times)
   for each L1 cache line (in case of 64B L1 cache Line which is
   default case). Obviously this is not optimal.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
ARCv2: lib: memeset: fix doing prefetchw outside of buffer 2019-01-18T00:24:39+00:00 Eugeniy Paltsev Eugeniy.Paltsev@synopsys.com 2019-01-14T15:16:48+00:00 e6a72b7daeeb521753803550f0ed711152bb2555 ARCv2 optimized memset uses PREFETCHW instruction for prefetching the next cache line but doesn't ensure that the line is not past the end of the buffer. PRETECHW changes the line ownership and marks it dirty, which can cause issues in SMP config when next line was already owned by other core. Fix the issue by avoiding the PREFETCHW Some more details: The current code has 3 logical loops (ignroing the unaligned part) (a) Big loop for doing aligned 64 bytes per iteration with PREALLOC (b) Loop for 32 x 2 bytes with PREFETCHW (c) any left over bytes loop (a) was already eliding the last 64 bytes, so PREALLOC was safe. The fix was removing PREFETCW from (b). Another potential issue (applicable to configs with 32 or 128 byte L1 cache line) is that PREALLOC assumes 64 byte cache line and may not do the right thing specially for 32b. While it would be easy to adapt, there are no known configs with those lie sizes, so for now, just compile out PREALLOC in such cases. Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com> Cc: stable@vger.kernel.org #4.4+ Signed-off-by: Vineet Gupta <vgupta@synopsys.com> [vgupta: rewrote changelog, used asm .macro vs. "C" macro] 
ARCv2 optimized memset uses PREFETCHW instruction for prefetching the
next cache line but doesn't ensure that the line is not past the end of
the buffer. PRETECHW changes the line ownership and marks it dirty,
which can cause issues in SMP config when next line was already owned by
other core. Fix the issue by avoiding the PREFETCHW

Some more details:

The current code has 3 logical loops (ignroing the unaligned part)
  (a) Big loop for doing aligned 64 bytes per iteration with PREALLOC
  (b) Loop for 32 x 2 bytes with PREFETCHW
  (c) any left over bytes

loop (a) was already eliding the last 64 bytes, so PREALLOC was
safe. The fix was removing PREFETCW from (b).

Another potential issue (applicable to configs with 32 or 128 byte L1
cache line) is that PREALLOC assumes 64 byte cache line and may not do
the right thing specially for 32b. While it would be easy to adapt,
there are no known configs with those lie sizes, so for now, just
compile out PREALLOC in such cases.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Cc: stable@vger.kernel.org #4.4+
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
[vgupta: rewrote changelog, used asm .macro vs. "C" macro]
ARC: dw2 unwind: enable cfi pseudo ops in string lib 2016-09-30T21:48:22+00:00 Vineet Gupta vgupta@synopsys.com 2016-09-19T23:42:25+00:00 86effd0dc675c36caed7b954d7f4f63b77c353b5 This uses a new set of annoations viz. ENTRY_CFI/END_CFI to enabel cfi ops generation. Note that we didn't change the normal ENTRY/EXIT as we don't actually want unwind info in the trap/exception/interrutp handlers which use these, as unwinder then gets confused (it keeps recursing vs. stopping). Semantically these are leaf routines and unwinding should stop when it hits those routines. Before ------ 28.52% 1.19% 9929 hackbench libuClibc-1.0.17.so [.] __write_nocancel | ---__write_nocancel |--8.95%--EV_Trap | --8.25%--sys_write | |--3.93%--sock_write_iter ... |--2.62%--memset <==== [LEAF entry as no unwind info] ^^^^^^ After ----- 29.46% 1.24% 13622 hackbench libuClibc-1.0.17.so [.] __write_nocancel | ---__write_nocancel |--9.31%--EV_Trap | --8.62%--sys_write | |--4.17%--sock_write_iter ... |--6.19%--sys_write | --6.19%--sock_write_iter | unix_stream_sendmsg | |--1.62%--sock_alloc_send_pskb | |--0.89%--sock_def_readable | |--0.88%--_raw_spin_unlock_irqrestore | |--0.69%--memset | | ^^^^^^ <==== [now in proper callframe] | | | --0.52%--skb_copy_datagram_from_iter Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
This uses a new set of annoations viz. ENTRY_CFI/END_CFI to enabel cfi
ops generation.

Note that we didn't change the normal ENTRY/EXIT as we don't actually
want unwind info in the trap/exception/interrutp handlers which use
these, as unwinder then gets confused (it keeps recursing vs. stopping).
Semantically these are leaf routines and unwinding should stop when it
hits those routines.

Before
------

    28.52%     1.19%          9929  hackbench  libuClibc-1.0.17.so   [.] __write_nocancel
            |
            ---__write_nocancel
               |--8.95%--EV_Trap
               |           --8.25%--sys_write
               |                     |--3.93%--sock_write_iter
     ...
               |--2.62%--memset   <==== [LEAF entry as no unwind info]
                         ^^^^^^

After
-----

    29.46%     1.24%         13622  hackbench  libuClibc-1.0.17.so   [.] __write_nocancel
            |
            ---__write_nocancel
               |--9.31%--EV_Trap
               |           --8.62%--sys_write
               |                     |--4.17%--sock_write_iter
     ...
               |--6.19%--sys_write
               |           --6.19%--sock_write_iter
               |                     unix_stream_sendmsg
               |                     |--1.62%--sock_alloc_send_pskb
               |                     |--0.89%--sock_def_readable
               |                     |--0.88%--_raw_spin_unlock_irqrestore
               |                     |--0.69%--memset
               |                     |         ^^^^^^     <==== [now in proper callframe]
               |                     |
               |                      --0.52%--skb_copy_datagram_from_iter

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
ARCv2: lib: memcpy: use local symbols 2015-11-03T12:03:00+00:00 Vineet Gupta vgupta@synopsys.com 2015-10-29T14:06:03+00:00 ac506b7f2233b35f17172304255e08cabc072aad Otherwise perf profiles don't charge tme to memcpy Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
Otherwise perf profiles don't charge tme to memcpy

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
ARCv2: lib: memset: Don't assume 64-bit load/stores 2015-07-20T14:44:37+00:00 Vineet Gupta vgupta@synopsys.com 2015-07-20T09:05:03+00:00 262137bca7cfbe690f8e904822b68f720998324a There are configurations which may not have LDD/STD Signed-off-by: Claudiu Zissulescu <claziss@synopsys.com> Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Vineet Gupta <vgupta@synopsys.com> 
There are configurations which may not have LDD/STD

Signed-off-by: Claudiu Zissulescu <claziss@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@synopsys.com>