[PATCH v2] arm64/cpufeature: annotate arm64_use_ng_mappings with ro_after_init to prevent wrong idmap generation

[Yeoreum Yun]

Hi Ryan,

> On 06/05/2025 09:09, Yeoreum Yun wrote:
> > Hi Catalin,
> >
> >> On Sat, May 03, 2025 at 09:23:27PM +0100, Yeoreum Yun wrote:
> >>> Hi Catalin,
> >>>
> >>>> On Sat, May 03, 2025 at 11:16:12AM +0100, Catalin Marinas wrote:
> >>>>> On Fri, 02 May 2025 19:04:12 +0100, Yeoreum Yun wrote:
> >>>>>> create_init_idmap() could be called before .bss section initialization
> >>>>>> which is done in early_map_kernel().
> >>>>>> Therefore, data/test_prot could be set incorrectly by PTE_MAYBE_NG macro.
> >>>>>>
> >>>>>> PTE_MAYBE_NG macro set NG bit according to value of "arm64_use_ng_mappings".
> >>>>>> and this variable places in .bss section.
> >>>>>>
> >>>>>> [...]
> >>>>>
> >>>>> Applied to arm64 (for-next/fixes), with some slight tweaking of the
> >>>>> comment, thanks!
> >>>>>
> >>>>> [1/1] arm64/cpufeature: annotate arm64_use_ng_mappings with ro_after_init to prevent wrong idmap generation
> >>>>>       https://git.kernel.org/arm64/c/12657bcd1835
> >>>>
> >>>> I'm going to drop this for now. The kernel compiled with a clang 19.1.5
> >>>> version I have around (Debian sid) fails to boot, gets stuck early on:
> >>>>
> >>>> $ clang --version
> >>>> Debian clang version 19.1.5 (1)
> >>>> Target: aarch64-unknown-linux-gnu
> >>>> Thread model: posix
> >>>> InstalledDir: /usr/lib/llvm-19/bin
> >>>>
> >>>> I didn't have time to investigate, disassemble etc. I'll have a look
> >>>> next week.
> >>>
> >>> Just for your information.
> >>> When I see the debian package, clang 19.1.5-1 doesn't supply anymore:
> >>>  - https://ftp.debian.org/debian/pool/main/l/llvm-toolchain-19/
> >>>
> >>> and the default version for sid is below:
> >>>
> >>> $ clang-19 --version
> >>> Debian clang version 19.1.7 (3)
> >>> Target: aarch64-unknown-linux-gnu
> >>> Thread model: posix
> >>> InstalledDir: /usr/lib/llvm-19/bin
> >>>
> >>> When I tested with above version with arm64-linux's for-next/fixes
> >>> including this patch. it works well.
> >>
> >> It doesn't seem to be toolchain related. It fails with gcc as well from
> >> Debian stable but you'd need some older CPU (even if emulated, e.g.
> >> qemu). It fails with Cortex-A72 (guest on Raspberry Pi 4) but not
> >> Neoverse-N2. Also changing the annotation from __ro_after_init to
> >> __read_mostly also works.
>
> I think this is likely because __ro_after_init is also "ro before init" - i.e.
> if you try to write to it in the PI code an exception is generated due to it
> being mapped RO. Looks like early_map_kernel() is writiing to it.
>
> I've noticed a similar problem in the past and it would be nice to fix it so
> that PI code maps __ro_after_init RW.
>

Personally, I don't believe this because the create_init_idmap()
maps the the .rodata section with PAGE_KERNEL pgprot
from __initdata_begin to _end.

and at the mark_readonly() the pgprot is changed to PAGE_KERNEL_RO
But, arm64_use_ng_mappings is accessed with write before mark_readonly()
only via smp_cpus_done().

JFYI here is map information:

// mark_readlonly() changes to ro perm below ranges:
ffff800081b30000 g       .rodata	0000000000000000 __start_rodata
ffff800082560000 g       .rodata.text	0000000000000000 __init_begin

// create_init_idmap() maps below range with PAGE_KERNEL.
ffff8000826d0000 g       .altinstructions	0000000000000000 __initdata_begin
ffff800082eb0000 g       .bss	0000000000000000 _end

ffff8000824596d0 g     O .rodata	0000000000000001 arm64_use_ng_mappings

Thanks.


> Thanks,
> Ryan
>
> >
> > Thanks to let me know. But still I've failed to reproduce this
> > on Cortex-a72 and any older cpu on qeum.
> > If you don't mind, would you share your Kconfig?
> >
> >> I haven't debugged it yet but I wonder whether something wants to write
> >> this variable after it was made read-only (well, I couldn't find any by
> >> grep'ing the code, so it needs some step-by-step debugging).
> >>
> > [...]
> >
> > Thanks!
> >
> > --
> > Sincerely,
> > Yeoreum Yun
>

--
Sincerely,
Yeoreum Yun