[PATCH v4 39/48] Documentation: KVM: Add the VGICv5 IRS save/restore sequences

[Sascha Bischoff]

Document the ordering requirements for saving and restoring the GICv5
IRS state.

The IST state is exposed through KVM_DEV_ARM_VGIC_GRP_IST and is copied
to or from userspace-provided buffers. Saving the ISTs therefore does
not update guest memory, so userspace does not need to serialize guest
memory after saving the IST state.

On restore, userspace must create the vCPUs, configure the IRS address,
restore the number of SPIs, and initialize the VGIC before restoring IRS
state. The IRS MMIO registers may be restored in any order, but the
final IRS_IDR* state and the IRS_IST_CFGR/IRS_IST_BASER state that
describes any LPI IST must be restored before KVM_DEV_ARM_VGIC_GRP_IST.
KVM uses that state to validate the userspace IST buffers and allocate
any host LPI IST.

The IRS_*_STATUSR registers are observational state in KVM. Userspace
may save them for validation or debugging, but they are not required as
restore input and do not need to be replayed.

Signed-off-by: Sascha Bischoff <sascha.bischoff at arm.com>
---
 .../virt/kvm/devices/arm-vgic-v5.rst          | 49 +++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
index 318d9033ddb4f..6813051789ac4 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v5.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
@@ -255,3 +255,52 @@ KVM_DEV_ARM_VGIC_GRP_IRS_REGS
                    or tracking pending interrupts
       -ETIMEDOUT   An IRS save/VM operation timed out
       ===========  ============================================================
+
+IRS Save Sequence:
+------------------
+
+The following operations are required when saving the virtual GICv5 IRS:
+
+a) Save the ISTs by issuing KVM_GET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_IST.
+b) Save the IRS MMIO register state by issuing KVM_GET_DEVICE_ATTR on
+   KVM_DEV_ARM_VGIC_GRP_IRS_REGS.
+
+These two steps may be performed in either order. Saving the ISTs writes all
+IST state to userspace-provided buffers and does not update guest memory.
+
+IRS Restore Sequence:
+---------------------
+
+The following ordering must be followed when restoring the virtual GICv5 and
+IRS:
+
+a) Create vCPUs.
+b) Provide the IRS base address by issuing KVM_SET_DEVICE_ATTR on
+   KVM_DEV_ARM_VGIC_GRP_ADDR
+c) Restore the number of SPIs by issuing KVM_SET_DEVICE_ATTR on
+   KVM_DEV_ARM_VGIC_GRP_NR_IRQS.
+d) Initialise the GIC - this sets up the default state and creates the SPI
+   IST - by issuing KVM_SET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_CTRL with
+   KVM_DEV_ARM_VGIC_CTRL_INIT
+e) Restore the IRS MMIO register state by issuing KVM_SET_DEVICE_ATTR on
+   KVM_DEV_ARM_VGIC_GRP_IRS_REGS. The IRS registers may be restored in any
+   order, but any IRS_IDR* state and the IRS_IST_CFGR/IRS_IST_BASER state
+   describing an LPI IST must be restored before restoring the ISTs. KVM uses
+   the restored IRS_IST_CFGR and IRS_IST_BASER state to allocate the LPI IST
+   during the following step.
+f) Restore the ISTs by issuing KVM_SET_DEVICE_ATTR on
+   KVM_DEV_ARM_VGIC_GRP_IST.
+
+The number of SPIs must be restored before VGIC initialization because
+initialization allocates the SPI state and fixes the SPI range exposed by the
+IRS ID registers.
+
+The IRS's MMIO registers must be restored prior to restoring the ISTs as these
+are used to convey the number of LPIs the guest has configured to KVM.
+
+The various ``*_STATUSR`` registers are observational state in the current KVM
+implementation. Userspace may save them for validation or debugging purposes,
+but they are not required as restore input and do not need to be replayed during
+restore.
+
+Then vCPUs can be started.
-- 
2.34.1