[PATCH v3 00/40] KVM: arm64: Add GICv5 IRS support
Sascha Bischoff
Sascha.Bischoff at arm.com
Wed Jul 22 07:39:05 PDT 2026
On Tue, 2026-07-21 at 13:40 +0100, Fuad Tabba wrote:
> Hi Sacha,
Hi Fuad,
>
> I have started working through this series and will send per-patch
> comments as I go, starting with the userspace interface since that's
> trickier to change later. I am still coming up to speed on GICv5, so
> please take these with that in mind. A couple of series level things
> first, while v4 is still open.
Thanks a lot for taking the time and for any and all comments.
>
> ...
>
> > Please pay close attention to the save/restore interface and
> > ordering,
> > in particular to make sure that no guest state has been omitted.
>
> This is around 6000 lines of new code, and the only test change is
> the
> nine lines that keep the existing vGICv5 selftest passing now that
> the
> IRS address is mandatory.
I have more tests that I didn't include with this series as they add in
a substantial amount of code, i.e., even more stuff to review. See
below for details.
>
> The ordering part is what worries me most here. Reading the save and
> restore sequences in the documentation, several of the constraints
> exist only as prose, for instance that guest memory must be
> serialised
> after the ISTs have been saved. Nothing in the API expresses them, so
> a VMM that gets one wrong gets no error, and finds out on the
> destination. A save/restore round trip selftest would turn that whole
> class of thing into something mechanical. Would one be feasible for
> the IRS and IST state? Without it I am not sure how much confidence a
> reviewer can really give you on the request above.
I have a set of tests that look at the various "components" that we
have in the GICv5 KVM implementation, including the IST save/restore.
This latter test does also save/restore the IRS MMIO regs as part of
testing the IST save/restore.
This is the output for the set of tests I have for this series.
# ./vgic_v5
Random seed: 0x6b8b4567
Running VGIC_V5 tests.
Test VGICv5 address attrs
Test VGICv5 NR_IRQS attrs
Test VGICv5 IRS_REGS attrs
Test VGICv5 IST attrs
Test VGICv5 userspace PPI attrs
Test VGICv5 CPU sysreg attrs
Test VGICv5 PPIs
Test VGICv5 SPIs
Test VGICv5 LPIs
Test VGICv5 IST save/restore
These do add a further ~2k lines to review across 9 commits, which is
why I'd not included them in the series posting. The git diff --stat
for the local series with these included is currently:
32 files changed, 7749 insertions(+), 194 deletions(-)
>
> ...
>
> > Thanks for taking the time to look at these changes. Apologies for
> > the
> > size of the series, but this is a rather hard set to disentangle
> > and
> > review separately!
>
> The apology rather invites the question, so it may be worth answering
> it up front instead. From the outside, patches 33 to 40, which are
> largely the userspace save/restore support and its documentation,
> look
> separable from getting a GICv5 guest booting with SPIs and LPIs. If
> you think migration support has to land together with the feature
> then
> I would find that convincing, but it would help to say so. Given the
> rework you have already described for v4, this might be a reasonable
> moment to revisit the split.
Noted - I should have been clearer here. The intent was that the
save/restore functionality was bundled together with the IRS, SPI, and
LPI support. With this series it becomes possible to run something real
with KVM on a GICv5 system. Prior to this series, we only had PPI
support which also meant a single VPE, and no ability to run anything
particularly meaningful.
Personally, I'm quite happy to do whatever is best for the reviewers to
make things as easy to review as possible. I can either extend the
series with the set of selftests, or I can drop the save/restore
support for the time being. Happy to hear your thoughts. FWIW, I
believe that Marc preferred them to land together for the reason above.
Thanks again for your input.
Sascha
>
> Cheers,
> /fuad
>
>
>
> >
> > Thanks,
> > Sascha
> >
> > [1]
> > https://lore.kernel.org/all/20260319154937.3619520-1-sascha.bischoff@arm.com/
> > [2] https://developer.arm.com/documentation/aes0070/latest
> > [3] https://developer.arm.com/documentation/108086/latest
> > [4] https://linaro.atlassian.net/wiki/x/CQAF-wY
> > [5]
> > https://lore.kernel.org/all/20260116182606.61856-1-sascha.bischoff@arm.com/
> > [6]
> > https://gitlab.arm.com/linux-arm/linux-sb/-/tree/gicv5_kvm_irs_support_v3
> > [7]
> > https://lore.kernel.org/all/20260427160547.3129448-1-sascha.bischoff@arm.com/
> > [8]
> > https://lore.kernel.org/all/20260521144846.1899475-1-sascha.bischoff@arm.com/
> >
> > Sascha Bischoff (40):
> > irqchip/gic-v5: Allow KVM setup without a maintenance IRQ
> > irqchip/gic-v5: Provide OF IRS config frame attrs to KVM
> > irqchip/gic-v5: Set up gic_kvm_info on ACPI hosts
> > KVM: arm64: gic-v5: Define remaining IRS MMIO registers
> > arm64/sysreg: Add GICv5 GIC VDPEND and VDRCFG encodings
> > arm64/sysreg: Update ICC_CR0_EL1 with LINK and LINK_IDLE fields
> > KVM: arm64: gic-v5: Extract host IRS caps from IRS config frame
> > KVM: arm64: gic-v5: Add VPE doorbell domain
> > KVM: arm64: gic-v5: Create and manage VM and VPE tables
> > KVM: arm64: gic-v5: Introduce guest IST alloc and management
> > KVM: arm64: gic-v5: Implement VMT/vIST IRS MMIO Ops
> > KVM: arm64: gic-v5: Keep GICv5 vCPU limit model-specific
> > KVM: arm64: gic-v5: Implement VPE IRS MMIO Ops
> > KVM: arm64: gic-v5: Set up VMTEs and VPE doorbells
> > KVM: arm64: gic-v5: Add resident/non-resident hyp calls
> > KVM: arm64: gic-v5: Request doorbells when VPEs enter WFI
> > KVM: arm64: gic-v5: Introduce struct vgic_v5_irs and IRS base
> > address
> > KVM: arm64: gic-v5: Add IRS IODEV support to MMIO handlers
> > KVM: arm64: gic-v5: Add KVM_VGIC_V5_ADDR_TYPE_IRS to UAPI
> > KVM: arm64: gic-v5: Add GICv5 IRS IODEV and MMIO emulation
> > KVM: arm64: gic-v5: Initialise per-VM IRS state
> > KVM: arm64: gic-v5: Register the IRS IODEV
> > KVM: arm64: gic-v5: Set IRICHPPIDIS based on IRS enable state
> > KVM: arm64: selftests: Update vGICv5 selftest to set IRS address
> > KVM: arm64: gic-v5: Introduce SPI AP list
> > KVM: arm64: gic-v5: Add GIC VDPEND and GIC VDRCFG hyp calls
> > KVM: arm64: gic-v5: Track SPI state for in-flight SPIs
> > KVM: arm64: gic: Introduce set_pending_state() to irq_op
> > KVM: arm64: gic-v5: Support SPI injection
> > Documentation: KVM: Extend VGICv5 docs for
> > KVM_VGIC_V5_ADDR_TYPE_IRS
> > KVM: arm64: gic-v5: Add GICv5 SPI injection to irqfd
> > KVM: arm64: gic-v5: Mask per-vcpu PPI state in
> > vgic_v5_finalize_ppi_state()
> > KVM: arm64: gic-v5: Add GICv5 EL1 sysreg userspace accessors
> > KVM: arm64: gic-v5: Handle userspace accesses to IRS MMIO region
> > KVM: arm64: gic-v5: Add CoreSight MMIO regs to IRS
> > KVM: arm64: gic-v5: Implement save/restore mechanisms for ISTs
> > Documentation: KVM: Document KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS for
> > VGICv5
> > Documentation: KVM: Add KVM_DEV_ARM_VGIC_GRP_IRS_REGS to VGICv5
> > docs
> > Documentation: KVM: Add docs for KVM_DEV_ARM_VGIC_GRP_IST
> > Documentation: KVM: Add the VGICv5 IRS save/restore sequences
> >
> > .../virt/kvm/devices/arm-vgic-v5.rst | 237 ++-
> > arch/arm64/include/asm/kvm_asm.h | 4 +
> > arch/arm64/include/asm/kvm_hyp.h | 4 +
> > arch/arm64/include/asm/sysreg.h | 18 +-
> > arch/arm64/include/uapi/asm/kvm.h | 7 +
> > arch/arm64/kvm/Makefile | 3 +-
> > arch/arm64/kvm/arm.c | 2 +-
> > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 34 +
> > arch/arm64/kvm/hyp/vgic-v5-sr.c | 55 +
> > arch/arm64/kvm/sys_regs.c | 6 +-
> > arch/arm64/kvm/vgic-sys-reg-v5.c | 519 +++++
> > arch/arm64/kvm/vgic/vgic-init.c | 111 +-
> > arch/arm64/kvm/vgic/vgic-irqfd.c | 17 +-
> > arch/arm64/kvm/vgic/vgic-irs-v5.c | 1138 ++++++++++
> > arch/arm64/kvm/vgic/vgic-kvm-device.c | 252 ++-
> > arch/arm64/kvm/vgic/vgic-mmio.c | 6 +
> > arch/arm64/kvm/vgic/vgic-mmio.h | 2 +
> > arch/arm64/kvm/vgic/vgic-v5-tables.c | 1855
> > +++++++++++++++++
> > arch/arm64/kvm/vgic/vgic-v5-tables.h | 112 +
> > arch/arm64/kvm/vgic/vgic-v5.c | 1220 ++++++++++-
> > arch/arm64/kvm/vgic/vgic.c | 39 +-
> > arch/arm64/kvm/vgic/vgic.h | 21 +-
> > arch/arm64/tools/sysreg | 4 +-
> > drivers/irqchip/irq-gic-v5-irs.c | 19 +-
> > drivers/irqchip/irq-gic-v5.c | 117 +-
> > include/kvm/arm_vgic.h | 165 +-
> > include/linux/irqchip/arm-gic-v5.h | 256 ++-
> > include/linux/irqchip/arm-vgic-info.h | 5 +
> > tools/arch/arm64/include/uapi/asm/kvm.h | 7 +
> > tools/testing/selftests/kvm/arm64/vgic_v5.c | 6 +
> > .../selftests/kvm/include/arm64/gic_v5.h | 3 +
> > 31 files changed, 6093 insertions(+), 151 deletions(-)
> > create mode 100644 arch/arm64/kvm/vgic-sys-reg-v5.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-irs-v5.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-v5-tables.c
> > create mode 100644 arch/arm64/kvm/vgic/vgic-v5-tables.h
> >
> > --
> > 2.34.1
> >
More information about the linux-arm-kernel
mailing list