Intermitted EFI firmware splats on SoftIron

[Ard Biesheuvel]

On Thu, 27 Apr 2023 at 17:42, Mark Brown <broonie at kernel.org> wrote:
>
> Hi,
>
> We've been seeing some intermittent splats from EFI on SoftIron in our
> internal testing at Arm - they're not really reproducible but we figured
> it was worth mentioning them.  The two examples below are from commit
> bc1bb2a49bae915107fd58705ed (just a random commit from Linus' tree):
>

Hi Mark,

Thanks for the report.

> [    9.195519] [Firmware Bug]: Unable to handle write to read-only memory in EFI runtime service

OK, so there are two separate issues here:

#1 - the firmware faults on a write to read-only memory (the line
above, and everything after the 'end trace' below)
#2 - the exception is taken while the firmware was running with
interrupts disabled, and so we return to the caller with interrupts
disabled, which is caught by efi_call_virt_check_flags()



> [    9.204059] ------------[ cut here ]------------
> [    9.208664] WARNING: CPU: 1 PID: 122 at drivers/firmware/efi/runtime-wrappers.c:113 efi_call_virt_check_flags+0x40/0xa4
> [    9.219443] Modules linked in: ip_tables x_tables ipv6
> [    9.224577] CPU: 1 PID: 122 Comm: kworker/u16:14 Tainted: G        W I        6.3.0 #1
> [    9.232483] Hardware name: AMD Overdrive/Supercharger/To be filled by O.E.M., BIOS ROD1002C 04/08/2016
> [    9.241776] Workqueue: efi_rts_wq efi_call_rts
> [    9.246211] pstate: 000000c5 (nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [    9.253163] pc : efi_call_virt_check_flags+0x40/0xa4
> [    9.258118] lr : efi_call_rts+0x1c0/0x4a0
> [    9.262118] sp : ffff80000c163d30
> [    9.265420] x29: ffff80000c163d30 x28: ffff000004010000 x27: ffff80000a367000
> [    9.272547] x26: 0000000000000000 x25: ffff80000a781e48 x24: ffff80000c2a3d88
> [    9.279674] x23: ffff80000c2a3d40 x22: ffff80000c2a3d4c x21: 00000000000000c0
> [    9.286800] x20: ffff800009aa0190 x19: 0000000000000000 x18: 000000000000001c
> [    9.293926] x17: 000000003d29a4bd x16: ffff80000a88c000 x15: 0000000000000003
> [    9.301052] x14: 0000000000000118 x13: 0000000000000000 x12: 0000000000000004
> [    9.308178] x11: 0000000000000000 x10: 0000000000000118 x9 : ffff80000a88bb78
> [    9.315304] x8 : ffff0003fd2cae80 x7 : 0000000000000000 x6 : 0000000000000000
> [    9.322430] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 00000083ffe25248
> [    9.329556] x2 : 0000000000000001 x1 : ffff800009aa0190 x0 : 00000000000000c0
> [    9.336682] Call trace:
> [    9.339116]  efi_call_virt_check_flags+0x40/0xa4
> [    9.343724]  efi_call_rts+0x1c0/0x4a0
> [    9.347377]  process_one_work+0x1d4/0x320
> [    9.351379]  worker_thread+0x6c/0x438
> [    9.355031]  kthread+0x118/0x11c
> [    9.358248]  ret_from_fork+0x10/0x20
> [    9.361814] ---[ end trace 0000000000000000 ]---
>
>     9.371717] efi: [Firmware Bug]: IRQ flags corrupted (0x00000000=>0x000000c0) by EFI set_variable
> [    9.380606] ------------[ cut here ]------------

#2 ends here, #1 below

> [    9.385212] WARNING: CPU: 7 PID: 128 at drivers/firmware/efi/runtime-wrappers.c:341 virt_efi_set_variable+0x194/0x1a4
> [    9.391187] systemd[1]: Created slice system-getty.slice.
> [    9.395814] Modules linked in: ip_tables x_tables ipv6
> [    9.395822] CPU: 7 PID: 128 Comm: kworker/7:1 Tainted: G        W I        6.3.0 #1
> [    9.395827] Hardware name: AMD Overdrive/Supercharger/To be filled by O.E.M., BIOS ROD1002C 04/08/2016
> [    9.395830] Workqueue: events refresh_nv_rng_seed
> [    9.423272] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [    9.423277] pc : virt_efi_set_variable+0x194/0x1a4
> [[0;32m  OK  [0m] Created slice[    9.439692] lr : virt_efi_set_variable+0x178/0x1a4
>  [0;1;39msystem-getty.slice[0m[    9.447337] sp : ffff80000c2a3cf0
> .
> [    9.453415] x29: ffff80000c2a3cf0 x28: 0000000000000000 x27: ffff80000a367000
> [    9.460794] x26: 0000000000000000 x25: ffff80000a802e68 x24: ffff80000a802000
> [    9.461248] systemd[1]: Created slice system-modprobe.slice.
> [    9.467920] x23: ffff800009a9e458 x22: ffff80000c2a3d78 x21: 8000000000000015
> [    9.467926] x20: ffff80000a6cc140 x19: ffff80000a802df0 x18: 0000000000000014
> [    9.480696] x17: 00000000fac1626a x16: 00000000aed5a9c4 x15: 00000000aaaef9db
> [    9.480702] x14: 00000000000001b8 x13: 0000000000000000 x12: 00000000000001b8
> [[0;32m  OK  [0m] Created slice[    9.502065] x11: 0000000000000007 x10: 0000000000000a20 x9 : ffff80000c2a3b60
>  [0;1;39msystem-modprobe.slice[[    9.512053] x8 : ffff000005641900 x7 : ffff0003fd2cae00 x6 : ffff8003f35b9000
> 0m.
> [    9.522036] x5 : 00000000410fd070 x4 : 0000000000000000 x3 : ffff80000a802e50
> [    9.529589] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 8000000000000015
> [    9.536715] Call trace:
> [    9.539149]  virt_efi_set_variable+0x194/0x1a4
> [    9.543583]  refresh_nv_rng_seed+0x78/0xac
> [    9.547669]  process_one_work+0x1d4/0x320
> [    9.551670]  worker_thread+0x6c/0x438
> [    9.555322]  kthread+0x118/0x11c
> [    9.558539]  ret_from_fork+0x10/0x20
> [    9.562104] ---[ end trace 0000000000000000 ]---
>

If this is a new failure, it appears to be caused by the early
non-volatile RNG seed stuff that Jason Donenfeld added recerntly.

I'm not sure if there is anything we might do about this except
disabling EFI runtime services altogether - these boxes and their
firmware are ancient, and it doesn't look like the kernel code is
doing anything wrong so we're not going to work around it. Disabling
EFI runtime services would mean losing access to the RTC, so that may
be problematic for testing, I imagine.