Alignment issues in zImage with Linux 4.12, LZ4 and GCC5.3

[Romain Izard]

Hello,

While upgrading the kernel from 4.9 to 4.12 for a custom board with a
Cortex-A5 based CPU, I have encountered a compilation issue that leads to
a data abort during the execution of the LZ4 decompression code in
zImage.

When compiling "lib/lz4/lz4_decompress.c", the output is different in
the full kernel and in the decompression library for zImage. The
disassembly of the LZ4_decompress_fast function looks as follows:

In lib/lz4/lz4_decompress.o:

00000830 <LZ4_decompress_fast>:
     830:    e1a0c00d     mov    ip, sp
     834:    e92ddff0     push    {r4, r5, r6, r7, r8, r9, sl, fp, ip, lr, pc}
     838:    e24cb004     sub    fp, ip, #4
     83c:    e24dd054     sub    sp, sp, #84    ; 0x54
     840:    e300e000     movw    lr, #0
     844:    e340e000     movt    lr, #0
     848:    e1a04002     mov    r4, r2
     84c:    e50b007c     str    r0, [fp, #-124]    ; 0xffffff84
     850:    e1a0c001     mov    ip, r1
     854:    e1a06000     mov    r6, r0
     858:    e24b506c     sub    r5, fp, #108    ; 0x6c
     85c:    e8be000f     ldm    lr!, {r0, r1, r2, r3}
     860:    e3540000     cmp    r4, #0
     864:    e08ca004     add    sl, ip, r4
     868:    e24b404c     sub    r4, fp, #76    ; 0x4c
     86c:    e8a5000f     stmia    r5!, {r0, r1, r2, r3}
     870:    e89e000f     ldm    lr, {r0, r1, r2, r3}
     874:    e28ee010     add    lr, lr, #16
     878:    e885000f     stm    r5, {r0, r1, r2, r3}
     87c:    e8be000f     ldm    lr!, {r0, r1, r2, r3}
     880:    e8a4000f     stmia    r4!, {r0, r1, r2, r3}
     884:    e89e000f     ldm    lr, {r0, r1, r2, r3}
     888:    e884000f     stm    r4, {r0, r1, r2, r3}
     88c:    0a0000b0     beq    b54 <LZ4_decompress_fast+0x324>
     890:    e24ae008     sub    lr, sl, #8
     894:    e24a400c     sub    r4, sl, #12
     898:    e24a9007     sub    r9, sl, #7
     89c:    e1a01006     mov    r1, r6
     8a0:    e24a3005     sub    r3, sl, #5
     8a4:    e50b3070     str    r3, [fp, #-112]    ; 0xffffff90
     8a8:    e5d18000     ldrb    r8, [r1]
     8ac:    e2811001     add    r1, r1, #1
     8b0:    e1a05228     lsr    r5, r8, #4
     8b4:    e355000f     cmp    r5, #15
     8b8:    0a00002f     beq    97c <LZ4_decompress_fast+0x14c>
     8bc:    e08c3005     add    r3, ip, r5
     8c0:    e153000e     cmp    r3, lr
     8c4:    8a000097     bhi    b28 <LZ4_decompress_fast+0x2f8>
     8c8:    e2812008     add    r2, r1, #8
     8cc:    e5126008     ldr    r6, [r2, #-8]
     8d0:    e28cc008     add    ip, ip, #8
     8d4:    e5120004     ldr    r0, [r2, #-4]
     8d8:    e2822008     add    r2, r2, #8
     8dc:    e50c6008     str    r6, [ip, #-8]
     8e0:    e50c0004     str    r0, [ip, #-4]
     8e4:    e153000c     cmp    r3, ip
     8e8:    8afffff7     bhi    8cc <LZ4_decompress_fast+0x9c>

In arch/arm/boot/compressed/decompress.o:

000005fc <LZ4_decompress_fast>:
     5fc:    e1a0c00d     mov    ip, sp
     600:    e92ddff0     push    {r4, r5, r6, r7, r8, r9, sl, fp, ip, lr, pc}
     604:    e24cb004     sub    fp, ip, #4
     608:    e24dd054     sub    sp, sp, #84    ; 0x54
     60c:    e59fc278     ldr    ip, [pc, #632]    ; 88c
<LZ4_decompress_fast+0x290>
     610:    e50b0078     str    r0, [fp, #-120]    ; 0xffffff88
     614:    e1a04001     mov    r4, r1
     618:    e08fc00c     add    ip, pc, ip
     61c:    e1a0a002     mov    sl, r2
     620:    e28c5020     add    r5, ip, #32
     624:    e1a08000     mov    r8, r0
     628:    e3520000     cmp    r2, #0
     62c:    e24b606c     sub    r6, fp, #108    ; 0x6c
     630:    e8bc000f     ldm    ip!, {r0, r1, r2, r3}
     634:    e24be04c     sub    lr, fp, #76    ; 0x4c
     638:    e084a00a     add    sl, r4, sl
     63c:    e8a6000f     stmia    r6!, {r0, r1, r2, r3}
     640:    e89c000f     ldm    ip, {r0, r1, r2, r3}
     644:    e886000f     stm    r6, {r0, r1, r2, r3}
     648:    e8b5000f     ldm    r5!, {r0, r1, r2, r3}
     64c:    e8ae000f     stmia    lr!, {r0, r1, r2, r3}
     650:    e895000f     ldm    r5, {r0, r1, r2, r3}
     654:    e88e000f     stm    lr, {r0, r1, r2, r3}
     658:    0a000083     beq    86c <LZ4_decompress_fast+0x270>
     65c:    e24a3008     sub    r3, sl, #8
     660:    e24a7007     sub    r7, sl, #7
     664:    e5d89000     ldrb    r9, [r8]
     668:    e2888001     add    r8, r8, #1
     66c:    e1a06229     lsr    r6, r9, #4
     670:    e356000f     cmp    r6, #15
     674:    0a00002b     beq    728 <LZ4_decompress_fast+0x12c>
     678:    e0845006     add    r5, r4, r6
     67c:    e1550003     cmp    r5, r3
     680:    8a00006e     bhi    840 <LZ4_decompress_fast+0x244>
     684:    e2882008     add    r2, r8, #8
     688:    e9120003     ldmdb    r2, {r0, r1}
     68c:    e2822008     add    r2, r2, #8
     690:    e8840003     stm    r4, {r0, r1}
     694:    e2844008     add    r4, r4, #8
     698:    e1550004     cmp    r5, r4
     69c:    8afffff9     bhi    688 <LZ4_decompress_fast+0x8c>

In the second case, we can see LDM and STM instructions at addresses
0x688 and 0x690. As r2 does not always contain an aligned address, a
data abort will be generated during decompression.

The compilation options are a little different between both cases:
The library is built with -O3, whereas the zImage decompressor is built
with -O2, -DDISABLE_BRANCH_PROFILING, -fpic, -mno-single-pic-base,
-fno-builtin. All other compilation options are shared in both cases.

For Linux 4.9, the LZ4 decompressor code is completely different, which
explains why the issue appeared when changing kernel versions.

This issue has been detected with a GCC5.3 compiler generated with
crosstool-NG, with cortex-a5 CPU tuning. If I build with the GCC6.3
from my workstation's distribution, and pass '-mcpu=cortex-a5' as a
compilation option for the kernel, I do not see this problem.

Best regards,
-- 
Romain Izard