[PATCH] arm64:align function __arch_clear_user
Kai Shen
shenkai8 at huawei.com
Sun Apr 25 03:07:39 BST 2021
On 2021/4/23 23:37, Catalin Marinas wrote:
> On Mon, Apr 19, 2021 at 10:05:16AM +0800, Kai Shen wrote:
>> On 2021/4/14 18:41, Catalin Marinas wrote:
>>> On Wed, Apr 14, 2021 at 05:25:43PM +0800, Kai Shen wrote:
>>>> Performance decreases happen in __arch_clear_user when this
>>>> function is not correctly aligned on HISI-HIP08 arm64 SOC which
>>>> fetches 32 bytes (8 instructions) from icache with a 32-bytes
>>>> aligned end address. As a result, if the hot loop is not 32-bytes
>>>> aligned, it may take more icache fetches which leads to decrease
>>>> in performance.
>>>> Dump of assembler code for function __arch_clear_user:
>>>> 0xffff0000809e3f10 : nop
>>>> 0xffff0000809e3f14 : mov x2, x1
>>>> 0xffff0000809e3f18 : subs x1, x1, #0x8
>>>> 0xffff0000809e3f1c : b.mi 0xffff0000809e3f30 <__arch_clear_user+3
>>>> ----- 0xffff0000809e3f20 : str xzr, [x0],#8
>>>> hot 0xffff0000809e3f24 : nop
>>>> loop 0xffff0000809e3f28 : subs x1, x1, #0x8
>>>> ----- 0xffff0000809e3f2c : b.pl 0xffff0000809e3f20 <__arch_clear_user+1
>>>> The hot loop above takes one icache fetch as the code is in one
>>>> 32-bytes aligned area and the loop takes one more icache fetch
>>>> when it is not aligned like below.
>>>> 0xffff0000809e4178 : str xzr, [x0],#8
>>>> 0xffff0000809e417c : nop
>>>> 0xffff0000809e4180 : subs x1, x1, #0x8
>>>> 0xffff0000809e4184 : b.pl 0xffff0000809e4178 <__arch_clear_user+
>>>> Data collected by perf:
>>>> aligned not aligned
>>>> instructions 57733790 57739065
>>>> L1-dcache-store 14938070 13718242
>>>> L1-dcache-store-misses 349280 349869
>>>> L1-icache-loads 15380895 28500665
>>>> As we can see, L1-icache-loads almost double when the loop is not
>>>> aligned.
>>>> This problem is found in linux 4.19 on HISI-HIP08 arm64 SOC.
>>>> Not sure what the case is on other arm64 SOC, but it should do
>>>> no harm.
>>>> Signed-off-by: Kai Shen <shenkai8 at huawei.com>
>>>
>>> Do you have a real world workload that's affected by this function?
>>>
>>> I'm against adding alignments and nops for specific hardware
>>> implementations. What about lots of other loops that the compiler may
>>> generate or that we wrote in asm?
>>
>> The benchmark we used which suffer performance decrease:
>> https://github.com/redhat-performance/libMicro
>> pread $OPTS -N "pread_z1k" -s 1k -I 300 -f /dev/zero
>> pread $OPTS -N "pread_z10k" -s 10k -I 1000 -f /dev/zero
>> pread $OPTS -N "pread_z100k" -s 100k -I 2000 -f /dev/zero
>
> Is there any real world use-case that would benefit from this
> optimisation? Reading /dev/zero in a loop hardly counts as a practical
> workload.
>
Operations like "dd if=/dev/zero of=/dev/sda1" ?
More information about the linux-arm-kernel
mailing list