[PATCH v2 29/45] KVM: arm64: GICv3: Set ICH_HCR_EL2.TDIR when interrupts overflow LR capacity

[Marc Zyngier]

On Mon, 17 Nov 2025 08:22:05 +0000,
Fuad Tabba <tabba at google.com> wrote:
> 
> Hi Marc,
> 
> On Fri, 14 Nov 2025 at 17:41, Marc Zyngier <maz at kernel.org> wrote:
> >
> > On Fri, 14 Nov 2025 15:53:33 +0000,
> > Fuad Tabba <tabba at google.com> wrote:
> > >
> > > Hi Marc,
> > >
> > > On Fri, 14 Nov 2025 at 15:02, Marc Zyngier <maz at kernel.org> wrote:
> > > >
> > > > On Fri, 14 Nov 2025 14:20:46 +0000,
> > > > Fuad Tabba <tabba at google.com> wrote:
> > > > >
> > > > > Hi Marc,
> > > > >
> > > > > On Sun, 9 Nov 2025 at 17:17, Marc Zyngier <maz at kernel.org> wrote:
> > > > > >
> > > > > > Now that we are ready to handle deactivation through ICV_DIR_EL1,
> > > > > > set the trap bit if we have active interrupts outside of the LRs.
> > > > > >
> > > > > > Signed-off-by: Marc Zyngier <maz at kernel.org>
> > > > > > ---
> > > > > >  arch/arm64/kvm/vgic/vgic-v3.c | 7 +++++++
> > > > > >  1 file changed, 7 insertions(+)
> > > > > >
> > > > > > diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c
> > > > > > index 1026031f22ff9..26e17ed057f00 100644
> > > > > > --- a/arch/arm64/kvm/vgic/vgic-v3.c
> > > > > > +++ b/arch/arm64/kvm/vgic/vgic-v3.c
> > > > > > @@ -42,6 +42,13 @@ void vgic_v3_configure_hcr(struct kvm_vcpu *vcpu,
> > > > > >                 ICH_HCR_EL2_VGrp0DIE : ICH_HCR_EL2_VGrp0EIE;
> > > > > >         cpuif->vgic_hcr |= (cpuif->vgic_vmcr & ICH_VMCR_ENG1_MASK) ?
> > > > > >                 ICH_HCR_EL2_VGrp1DIE : ICH_HCR_EL2_VGrp1EIE;
> > > > > > +
> > > > > > +       /*
> > > > > > +        * Note that we set the trap irrespective of EOIMode, as that
> > > > > > +        * can change behind our back without any warning...
> > > > > > +        */
> > > > > > +       if (irqs_active_outside_lrs(als))
> > > > > > +               cpuif->vgic_hcr |= ICH_HCR_EL2_TDIR;
> > > > > >  }
> > > > >
> > > > > I just tested these patches as they are on kvmarm/next
> > > > > 2ea7215187c5759fc5d277280e3095b350ca6a50 ("Merge branch
> > > > > 'kvm-arm64/vgic-lr-overflow' into kvmarm/next"), without any
> > > > > additional pKVM patches. I tried running it with pKVM (non-protected)
> > > > > and with just plain nVHE. In both cases, I get a trap to EL2 (0x18)
> > > > > when booting a non-protected guest, which triggers a bug in
> > > > > handle_trap() arch/arm64/kvm/hyp/nvhe/hyp-main.c:706
> > > > >
> > > > > This trap is happening because of setting this particular trap (TDIR).
> > > > > Just removing this trap from vgic_v3_configure_hcr() from the ToT on
> > > > > kvmarm/next boots fine.
> > > >
> > > > This is surprising, as I'm not hitting this on actual HW. Are you
> > > > getting a 0x18 trap? If so, is it coming from the host? Can you
> > > > correlate the PC with what the host is doing?
> > >
> > > I should have given you that earlier, sorry.
> > >
> > > Yes, it's an 0x18 trap from the host (although it happens when I boot
> > > a guest). Here is the relevant part of the backtrace addr2lined and
> > > the full one below.
> > >
> > > handle_percpu_devid_irq+0x90/0x120 (kernel/irq/chip.c:930)
> > > generic_handle_domain_irq+0x40/0x64 (include/linux/irqdesc.h:?)
> > > gic_handle_irq+0x4c/0x110 (include/linux/irqdesc.h:?)
> > > call_on_irq_stack+0x30/0x48 (arch/arm64/kernel/entry.S:893)
> > >
> > > [   28.454804] Code: d65f03c0 92800008 f9000008 17fffffa (d4210000)
> > > [   28.454873] kvm [266]: Hyp Offset: 0xfff1205c3fe00000
> > > [   28.455157] Kernel panic - not syncing: HYP panic:
> > > [   28.455157] PS:204023c9 PC:000e5fa4413e39bc ESR:00000000f2000800
> > > [   28.455157] FAR:ffff800082733d3c HPFAR:0000000000500000 PAR:0000000000000000
> >
> > I expect you have a write to ICC_DIR_EL1 at this address?
> 
> It almost surely must be, but tracking it down hasn't been that easy.
> That said, I think it's ending up in gic_eoimode1_eoi_irq(), which
> calls gic_write_dir() if !gic_arm64_erratum_2941627_needed(d).
> 
> I wonder if your hardware needs that erratum.

No, it doesn't. And this erratum only kicks in if the deactivated
interrupt is an SPI that has been moved to another CPU while being
handled (i.e. never).

It really isn't related to the GIC itself (which the erratum above is
about), but to the CPU interface, and whether TDIR applies to
ICC_DIR_EL1 on top of the virtual variant.

The architecture "deprecates" not trapping, and QEMU abides by this
deprecation. HW that predates the deprecation didn't get the message,
oddly enough... Neither did I, until you reported the crash!

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.