arm64 regression in kernel 5.12 related to the (n)VHE

[Rafał Miłecki]

Hi,

I just tried upgrading from the old good LTS kernel 5.10 and I
discovered that my bcm4908 boards don't boot anymore with the 5.14-rc5.


The problem is kernel doesn't seem to start booting at all. I see CFE
bootloader messages:

Starting program at 0x0000000000080000
/memory = 0x40000000

and then nothing. Normally the first kernel line should follow like a:
Linux version 5.11.0-rc4 (rmilecki at localhost.localdomain) (aarch64-buildroot-linux-uclibc-gcc.br_real (Buildroot -g91617ed) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #30 SMP Wed Aug 11 14:01:00 CEST 2021


I have zero knowledge of low level arm64 or assembler stuff. I also
don't own any bcm4908 development board or bcm4908 datasheets.

All I could do to help debugging this regression was bisecting. The
first bad commit (I verified it after bisecting process) is:

commit 0c93df9622d4d921bcd0dc83f71fed9e98f5119f
Author: Marc Zyngier <maz at kernel.org>
Date:   Mon Feb 8 09:57:14 2021 +0000

     arm64: Initialise as nVHE before switching to VHE

     As we are aiming to be able to control whether we enable VHE or
     not, let's always drop down to EL1 first, and only then upgrade
     to VHE if at all possible.

     This means that if the kernel is booted at EL2, we always start
     with a nVHE init, drop to EL1 to initialise the the kernel, and
     only then upgrade the kernel EL to EL2 if possible (the process
     is obviously shortened for secondary CPUs).

     The resume path is handled similarly to a secondary CPU boot.

     Signed-off-by: Marc Zyngier <maz at kernel.org>
     Acked-by: David Brazdil <dbrazdil at google.com>
     Acked-by: Catalin Marinas <catalin.marinas at arm.com>
     Link: https://lore.kernel.org/r/20210208095732.3267263-6-maz@kernel.org
     [will: Avoid calling switch_to_vhe twice on kaslr path]
     Signed-off-by: Will Deacon <will at kernel.org>


Could you look at this issue, please? I'm happy to test any patches or
provide any extra info I can obtain using kernel 5.11.


My defconfig for bcm4908 is:

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="${BR_BINARIES_DIR}/rootfs.cpio"
CONFIG_ARCH_BCM4908=y
CONFIG_NR_CPUS=4
CONFIG_CMDLINE="earlycon=bcm63xx_uart,0xff800640"
CONFIG_CMDLINE_FORCE=y
CONFIG_PCI=y
CONFIG_PCIEPORTBUS=y
CONFIG_PCIE_BRCMSTB=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_SERIAL_BCM63XX=y
CONFIG_SERIAL_BCM63XX_CONSOLE=y
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_SLAVE=y
CONFIG_SPI=y
CONFIG_PINCTRL=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_GENERIC_PLATFORM=y
CONFIG_POWER_RESET_SYSCON=y
CONFIG_THERMAL=y
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PLATFORM=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_HCD_PLATFORM=y
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_OHCI_HCD_PLATFORM=y
CONFIG_DMADEVICES=y
CONFIG_RESET_CONTROLLER=y