Big endian modifications

[Mark Rutland]

Hi,

On Thu, Feb 03, 2022 at 06:48:43PM +0000, Rory Bolt wrote:
> Hello,
> 
> I have been modifying GRUB on Das U-boot to allow it to boot aarch64_be big
> endian kernels on the RockPro64 board. Since the RockPro64 has a PCIe (albeit
> 2.0) slot, it provides a low cost/reasonable performance (vs QEMU) system to
> test our devices, drivers and software stack on a big endian system. I am in
> the process of publishing the required changes to GRUB, however there are two
> minor changes made to the kernel itself.

Kernel patches need to be submitted in a particular format, with a commit
title, commit message, and DCO (Signed-off-by) lines. Please see:

  https://docs.kernel.org/process/submitting-patches.html

  https://docs.kernel.org/process/development-process.html

It's also worthwhile looking at the MAINTAINERS file to Cc relevant people. For
example, Catalin and Will are the maintainers of the arm64 port, and Ard, Marc,
and I have made various changes to this code.

> Arguably the modification to head.S could be moved to grub itself, however it
> is in the defensive spirit of not assuming the boot loader has set up
> everything correctly and is very similar to the startup of NetBSD which
> contains similar code in its startup.

Hmm... big-endian boot *should* already work. The kernel configures its
endianness in init_kernel_el(), and from testing a BE v5.17-rc2 kernel just now
with a FVP Base model and the boot-wrapper (which would leave the CPU in a
little-endian state when jumping to the kernel) things seem to work.

Which kernel version are you testing with, and how *exactly* is the kernel
being entered?

Is it being entered directly by the protocol defined at:

  https://docs.kernel.org/arm64/booting.html

... or some other way (e.g. via the EFI application entry point?).

> The second change to Kconfig allows a EFI/PE/COFF header to be added to a big
> endian kernel (although unfortunately it also adds a useless big endian
> efistub to the kernel too). With the following two changes the modified GRUB
> can boot either big endian or little endian Arm64 images.

This implies that grub is trying to enter the EFI entry point, and a bunch of
things aren't going to work thereafter.

What have you changed on the GRUB side? Because I don't understand why it would
*need* changes, if the PE/COFF header and EFI stub handled being entered in LE,
then switching to BE. That and it should be possible to boot a BE kernel
without neding a PE/COFF header, so something doesn't add up.

> Please consider the following two modifications for inclusion:

As above, we can't take this as-is, but if you've found a regression I'm more
than happy to try to get that fixed. If we *really* need to be able to boot
BE kernels from EFI, there might be things that we can do, but I don't think we
can just remove the CONFIG_EFI dependency on !CPU_BIG_ENDIAN -- the runtime
bits *certainly* won't work, at minimum the stub needs to be built LE, etc.

Also, have you considered running your kernel under KVM, with devices assigned
to it? That might be significantly easier than having a BE kernel that handles
EFI.

Thanks,
Mark.

> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 6a98f1a38c29..40a18d767d15 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -89,6 +89,24 @@
>          *  x24        __primary_switch() .. relocate_kernel()  current RELR displacement
>          */
>  SYM_CODE_START(primary_entry)
> +#ifdef CONFIG_CPU_BIG_ENDIAN
> +        mrs     x21, CurrentEL
> +        lsr     x21, x21, #2
> +        cmp     x21, #0x2
> +        b.lo    1f
> +
> +        mrs     x21, sctlr_el2
> +        orr     x21, x21, #SCTLR_ELx_EE /* set: Big Endian */
> +        msr     sctlr_el2, x21
> +        isb
> +
> +1:
> +        mrs     x21, sctlr_el1
> +        orr     x21, x21, #(SCTLR_ELx_EE | SCTLR_EL1_E0E)       /* set: Big Endian */
> +        msr     sctlr_el1, x21
> +        isb
> +
> +#endif
>         bl      preserve_boot_args
>         bl      init_kernel_el                  // w0=cpu_boot_mode
>         adrp    x23, __PHYS_OFFSET
> 
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index c4207cf9bb17..a9ccbeb75ea7 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1997,7 +1997,7 @@ config EFI_STUB
> 
>  config EFI
>         bool "UEFI runtime support"
> -       depends on OF && !CPU_BIG_ENDIAN
> +       depends on OF
>         depends on KERNEL_MODE_NEON
>         select ARCH_SUPPORTS_ACPI
>         select LIBFDT
> 
> Rory Bolt
> 
> KIOXIA America, Inc. | formerly Toshiba Memory America, Inc.
> 
> 
> 
> 
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel