KVM: Nested VGIC emulation leads to infinite IRQ exceptions

[Marc Zyngier]

On Tue, 30 Sep 2025 22:11:54 +0100,
Volodymyr Babchuk <Volodymyr_Babchuk at epam.com> wrote:

[...]

I spent some time to look at this again.
> 
> This is a part of the KVM trace, where you can see that vCPU in question
> tries to perform ERET to Linux in DomU but is being brought back to
> vEL2. In this particular case this is vCPU1 / vvCPU0. I filtered out
> other vCPUs to reduce clutter.
> 
>  qemu-system-aar-41290   [000] d.... 12023.695620: kvm_entry: PC: 0x00000a0000267c80
>  qemu-system-aar-41290   [000] d.... 12023.695620: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0
>  qemu-system-aar-41290   [000] d.... 12023.695621: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0
>  qemu-system-aar-41290   [000] d.... 12023.695621: kvm_timer_emulate: arch_timer_ctx_index: 1 (should_fire: 1)
>  qemu-system-aar-41290   [000] d.... 12023.695621: kvm_timer_emulate: arch_timer_ctx_index: 0 (should_fire: 0)
>  qemu-system-aar-41290   [000] ..... 12023.695621: kvm_exit: TRAP: HSR_EC: 0x001a (ERET), PC: 0x00000a00002674e0

Wants to ERET to EL1

>  qemu-system-aar-41290   [000] ..... 12023.695621: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0
>  qemu-system-aar-41290   [000] d.... 12023.695622: kvm_timer_save_state:    CTL: 0x000000 CVAL:              0x0 arch_timer_ctx_index: 2
>  qemu-system-aar-41290   [000] d.... 12023.695622: kvm_timer_save_state:    CTL: 0x000005 CVAL:   0x426f7d24736c arch_timer_ctx_index: 3

EL2 physical  timer is pending

>  qemu-system-aar-41290   [000] ..... 12023.695622: kvm_nested_eret: elr_el2: 0xffffffc0010ac5a4 spsr_el2: 0x024000c5 (M: EL1h) hcr_el2: 807c663f

Return to EL1, reload the EL1 context

>  qemu-system-aar-41290   [000] ..... 12023.695622: kvm_get_timer_map: VCPU: 1, dv: 1, dp: 0, ev: 2, ep: 3
>  qemu-system-aar-41290   [000] ..... 12023.695622: kvm_timer_update_irq: VCPU: 1, IRQ 27, level 1
>  qemu-system-aar-41290   [000] ..... 12023.695623: vgic_update_irq_pending: VCPU: 1, IRQ 27, level: 1
>  qemu-system-aar-41290   [000] ..... 12023.695623: kvm_timer_update_irq: VCPU: 1, IRQ 30, level 0
>  qemu-system-aar-41290   [000] ..... 12023.695623: vgic_update_irq_pending: VCPU: 1, IRQ 30, level: 0
>  qemu-system-aar-41290   [000] d.... 12023.695623: kvm_timer_restore_state: CTL: 0x000005 CVAL:      0x48aac64bd arch_timer_ctx_index: 1

EL1 virtual timer is pending

>  qemu-system-aar-41290   [000] d.... 12023.695624: kvm_timer_restore_state: CTL: 0x000000 CVAL:              0x0 arch_timer_ctx_index: 0
>  qemu-system-aar-41290   [000] ..... 12023.695624: kvm_timer_emulate: arch_timer_ctx_index: 2 (should_fire: 0)
>  qemu-system-aar-41290   [000] ..... 12023.695624: kvm_timer_emulate: arch_timer_ctx_index: 3 (should_fire: 1)

EL2 physical timer still pending

>  qemu-system-aar-41290   [000] ..... 12023.695626: kvm_get_timer_map: VCPU: 1, dv: 1, dp: 0, ev: 2, ep: 3
>  qemu-system-aar-41290   [000] d.... 12023.695626: kvm_timer_save_state:    CTL: 0x000005 CVAL:      0x48aac64bd arch_timer_ctx_index: 1
>  qemu-system-aar-41290   [000] d.... 12023.695627: kvm_timer_save_state:    CTL: 0x000000 CVAL:              0x0 arch_timer_ctx_index: 0

HW without FEAT_ECV, I presume?

>  qemu-system-aar-41290   [000] ..... 12023.695627: kvm_inject_nested_exception: IRQ: esr_el2 0x0 elr_el2: 0xffffffc0010ac5a4 spsr_el2: 0x024000c5 (M: EL1h) hcr_el2: 807c663f

Take an interrupt from EL1 to EL2, flip the world again.

>  qemu-system-aar-41290   [000] ..... 12023.695627: kvm_get_timer_map: VCPU: 1, dv: 2, dp: 3, ev: 1, ep: 0
>  qemu-system-aar-41290   [000] ..... 12023.695627: kvm_timer_update_irq: VCPU: 1, IRQ 28, level 0
>  qemu-system-aar-41290   [000] ..... 12023.695627: vgic_update_irq_pending: VCPU: 1, IRQ 28, level: 0
>  qemu-system-aar-41290   [000] ..... 12023.695628: kvm_timer_update_irq: VCPU: 1, IRQ 26, level 1
>  qemu-system-aar-41290   [000] ..... 12023.695628: vgic_update_irq_pending: VCPU: 1, IRQ 26, level: 1
>  qemu-system-aar-41290   [000] d.... 12023.695628: kvm_timer_restore_state: CTL: 0x000000 CVAL:              0x0 arch_timer_ctx_index: 2
>  qemu-system-aar-41290   [000] d.... 12023.695628: kvm_timer_restore_state: CTL: 0x000005 CVAL:   0x426f7d24736c arch_timer_ctx_index: 3

Yup, EL2 timer still pending

>  qemu-system-aar-41290   [000] ..... 12023.695629: kvm_timer_emulate: arch_timer_ctx_index: 1 (should_fire: 1)
>  qemu-system-aar-41290   [000] ..... 12023.695629: kvm_timer_emulate: arch_timer_ctx_index: 0 (should_fire: 0)
>  qemu-system-aar-41290   [000] d.... 12023.695632: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0
>  qemu-system-aar-41290   [000] d.... 12023.695632: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0
>  qemu-system-aar-41290   [000] d.... 12023.695633: vgic_update_irq_pending: VCPU: 1, IRQ 25, level: 0
>  qemu-system-aar-41290   [000] d.... 12023.695633: kvm_entry: PC: 0x00000a0000267c80

and we go again.

So the MI doesn't seem to be the cause of this, as empty LRs are not
likely to be the problem.

However, we definitely see timer interrupts firing, EL2 being entered,
and yet, El2 doesn't seem to acknowledge the interrupt. So something
is wrong there, either in Xen on in KVM. You want to instrument what
is happening at this stage (I don't see anything of the like, but my
machines have FEAT_ECV).

	M.

-- 
Without deviation from the norm, progress is not possible.