[PATCH] ARM: head: Remove 1-2 MiB of pointless padding

[Linus Walleij]

On Mon, May 8, 2023 at 4:55 PM Ard Biesheuvel <ardb at kernel.org> wrote:

> Currently, some of the early init code is emitted into .head.text, which
> is allocated as a separate section in the linker script, and if
> CONFIG_STRICT_KERNEL_RWX=y, which is the default, the .text section that
> follows it is aligned to a section mapping boundary, i.e., 1 MiB or 2 MiB,
> depending on whether or not LPAE is enabled.
>
> This results in 1 - 2 MiB of space wasted on padding for no good reason:
> the only part of .head.text that really needs special treatment is the
> first instruction (which is the entrypoint), but everything else is
> essentially inittext and can be emitted as such.
>
> So let's remove everything except the initial instruction from
> .head.text, and convert it into a branch to the actual primary startup
> code, which is moved into the .init.text section. That leaves a
> .head.text section of only 4 bytes, which can be placed at the start of
> the ordinary .text section, without any padding or alignment.
>
> While at it, drop the special Thumb switch sequence, and just use a BL
> branch-and-link instruction for the initial jump so that the linker can
> turn it into an interworking BLX instruction as needed. (No interworking
> variant of the ordinary B branch instruction exists, so the linker would
> need to emit a veneer to effectuate the ARM to Thumb mode switch)
>
> Signed-off-by: Ard Biesheuvel <ardb at kernel.org>

I had some spare cycles so I tested this patch.

I think the thumb switch patch could be send out separately,
I like the simplification!

objdump does show me a big saving in virtual address usage:
everything is stacked from 0xc0008000 and onward, skipping the
big jump to 0xc1000000 at some point in the old layout.

However I get these:

[    4.434936] BUG: pre-text NX section c0000000-c0008000 not aligned to 100000
[    4.442047] BUG: pre-text NX section c0000000-c0008000 not aligned to 100000
[    4.452819] Freeing unused kernel image (initmem) memory: 3072K
[    4.488555] BUG: text/rodata RO section c0008000-c0d00000 not
aligned to 100000
[    4.495910] BUG: text/rodata RO section c0008000-c0d00000 not
aligned to 100000

I guess these are the problems you saw in the follow-up
with section alignment requirements, and you suggest:

> Given that it would only take 1 additional page worth of page tables
> to break up this section, I still don't think the padding is
> justified, and we should just map the start of .text down to pages.

I don't quite follow this solution (mappin the start of .text down
to pages) but I like the sound of it because it sounds like you found
a way to still save all these virtual addresses :)

When I compile the kernel in to vmalloc at 0xf1000000 in some
experimental patches it matters even more, so I'm definitely game
for this.

Yours,
Linus Walleij