Big endian modifications

Robin Murphy robin.murphy at arm.com
Thu Feb 3 13:20:00 PST 2022 

On 2022-02-03 20:10, Mark Rutland wrote:
> Hi,
> 
> On Thu, Feb 03, 2022 at 06:48:43PM +0000, Rory Bolt wrote:
>> Hello,
>>
>> I have been modifying GRUB on Das U-boot to allow it to boot aarch64_be big
>> endian kernels on the RockPro64 board. Since the RockPro64 has a PCIe (albeit
>> 2.0) slot, it provides a low cost/reasonable performance (vs QEMU) system to
>> test our devices, drivers and software stack on a big endian system. I am in
>> the process of publishing the required changes to GRUB, however there are two
>> minor changes made to the kernel itself.
> 
> Kernel patches need to be submitted in a particular format, with a commit
> title, commit message, and DCO (Signed-off-by) lines. Please see:
> 
>    https://docs.kernel.org/process/submitting-patches.html
> 
>    https://docs.kernel.org/process/development-process.html
> 
> It's also worthwhile looking at the MAINTAINERS file to Cc relevant people. For
> example, Catalin and Will are the maintainers of the arm64 port, and Ard, Marc,
> and I have made various changes to this code.
> 
>> Arguably the modification to head.S could be moved to grub itself, however it
>> is in the defensive spirit of not assuming the boot loader has set up
>> everything correctly and is very similar to the startup of NetBSD which
>> contains similar code in its startup.
> 
> Hmm... big-endian boot *should* already work. The kernel configures its
> endianness in init_kernel_el(), and from testing a BE v5.17-rc2 kernel just now
> with a FVP Base model and the boot-wrapper (which would leave the CPU in a
> little-endian state when jumping to the kernel) things seem to work.
> 
> Which kernel version are you testing with, and how *exactly* is the kernel
> being entered?
> 
> Is it being entered directly by the protocol defined at:
> 
>    https://docs.kernel.org/arm64/booting.html
> 
> ... or some other way (e.g. via the EFI application entry point?).
> 
>> The second change to Kconfig allows a EFI/PE/COFF header to be added to a big
>> endian kernel (although unfortunately it also adds a useless big endian
>> efistub to the kernel too). With the following two changes the modified GRUB
>> can boot either big endian or little endian Arm64 images.
> 
> This implies that grub is trying to enter the EFI entry point, and a bunch of
> things aren't going to work thereafter.
> 
> What have you changed on the GRUB side? Because I don't understand why it would
> *need* changes, if the PE/COFF header and EFI stub handled being entered in LE,
> then switching to BE. That and it should be possible to boot a BE kernel
> without neding a PE/COFF header, so something doesn't add up.

AFAIK arm64 GRUB only supports calling Linux as an EFI application.

>> Please consider the following two modifications for inclusion:
> 
> As above, we can't take this as-is, but if you've found a regression I'm more
> than happy to try to get that fixed. If we *really* need to be able to boot
> BE kernels from EFI, there might be things that we can do, but I don't think we
> can just remove the CONFIG_EFI dependency on !CPU_BIG_ENDIAN -- the runtime
> bits *certainly* won't work, at minimum the stub needs to be built LE, etc.
> 
> Also, have you considered running your kernel under KVM, with devices assigned
> to it? That might be significantly easier than having a BE kernel that handles
> EFI.

Sadly that's not an option on RockPro64 since there's no suitable IOMMU. 
The number of affordable AArch64 platforms actually capable of device 
assignment is still disappointingly close to none :(

Robin.

> 
> Thanks,
> Mark.
> 
>> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
>> index 6a98f1a38c29..40a18d767d15 100644
>> --- a/arch/arm64/kernel/head.S
>> +++ b/arch/arm64/kernel/head.S
>> @@ -89,6 +89,24 @@
>>           *  x24        __primary_switch() .. relocate_kernel()  current RELR displacement
>>           */
>>   SYM_CODE_START(primary_entry)
>> +#ifdef CONFIG_CPU_BIG_ENDIAN
>> +        mrs     x21, CurrentEL
>> +        lsr     x21, x21, #2
>> +        cmp     x21, #0x2
>> +        b.lo    1f
>> +
>> +        mrs     x21, sctlr_el2
>> +        orr     x21, x21, #SCTLR_ELx_EE /* set: Big Endian */
>> +        msr     sctlr_el2, x21
>> +        isb
>> +
>> +1:
>> +        mrs     x21, sctlr_el1
>> +        orr     x21, x21, #(SCTLR_ELx_EE | SCTLR_EL1_E0E)       /* set: Big Endian */
>> +        msr     sctlr_el1, x21
>> +        isb
>> +
>> +#endif
>>          bl      preserve_boot_args
>>          bl      init_kernel_el                  // w0=cpu_boot_mode
>>          adrp    x23, __PHYS_OFFSET
>>
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index c4207cf9bb17..a9ccbeb75ea7 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -1997,7 +1997,7 @@ config EFI_STUB
>>
>>   config EFI
>>          bool "UEFI runtime support"
>> -       depends on OF && !CPU_BIG_ENDIAN
>> +       depends on OF
>>          depends on KERNEL_MODE_NEON
>>          select ARCH_SUPPORTS_ACPI
>>          select LIBFDT
>>
>> Rory Bolt
>>
>> KIOXIA America, Inc. | formerly Toshiba Memory America, Inc.
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

More information about the linux-arm-kernel mailing list