[PATCH v3 00/29] KVM: arm64: SMMUv3 driver for pKVM
Mostafa Saleh
smostafa at google.com
Mon Jul 28 10:52:47 PDT 2025
This is v3 of the support for DMA isolation with SMMUv3 on pKVM.
v2: https://lore.kernel.org/kvmarm/20241212180423.1578358-1-smostafa@google.com/
v1: https://lore.kernel.org/kvmarm/20230201125328.2186498-1-jean-philippe@linaro.org/
Main changes (see design section for details):
- This version applies directly to upstream with no out of tree pKVM
dependencies and is not an RFC
- This version adds minimum support for DMA isolation by identity
mapping the SMMU S2 as opposed to the paravirtual interface in the
previous versions.
- This version has much more code re-use as a new shared file (and
structs, macros) for hyp-kernel smmu-v3 code, which makes the
hyp SMMUv3 specific code smaller.
pKVM overview:
=============
The pKVM hypervisor arm64, provides a separation of privileges
between the host and hypervisor parts of KVM, where the hypervisor
is trusted by guests but the host is not [1][2]. The host is initially
trusted during boot, but its privileges are reduced after KVM is
initialized so that, if an adversary later gains access to the large
attack surface of the host, it cannot access guest data.
Currently with pKVM, the host can still instruct DMA-capable devices
like the GPU to access guest and hypervisor memory, which undermines
this isolation. Preventing DMA attacks requires an IOMMU, owned by the
hypervisor.
This series adds a hypervisor driver for the Arm SMMUv3 IOMMU. Since the
hypervisor part of pKVM (called nVHE here) is minimal, moving the whole
host SMMU driver into nVHE isn't really an option. It is too large and
complex and requires infrastructure from all over the kernel. We add a
reduced nVHE driver that deals with populating the SMMU tables and the
command queue, and the host driver still deals with probing and some
Initialization.
Most of SMMUv3 specific source code would still be shared through common source files.
Patches overview
================
The patches are split as follows:
Patches 01-03: Core hypervisor: Add donation for NC, dealing with MMIO,
and arch timer abstraction.
Patches 04-13: SMMUv3 driver split: Split the SMMUv3 driver to 3 parts
(shared kernel, shared hyp, driver specific) and io-pgtable-arm
to 2 parts (shared hyp, driver specific)
Most of these patches are better viewed with --color-moved
Patches 14-17: Hypervisor IOMMU core: two hypercall to enable/disable
devices, IOMMU pagetable management
Patches 18-25: Hypervisor KVM SMMUv3 driver: Setup shadow pgtables, STEs...
Patches 26-29 Kernel KVM SMMUv3 driver: Probe of devices, and populating
info to the hypervisor, IOMMU ops
A development branch is available at:
https://android-kvm.googlesource.com/linux/+log/refs/heads/for-upstream/pkvm-smmu-v3
Design and future work
===============
This patch series is designed to be minimal and include patches only
required for functionality.
The hypervisor SMMUv3 driver will load early before KVM initialises
and registers itself as a KVM IOMMU driver; if it finds SMMUv3 on the platform.
Then after KVM initialises and before it de-privileges it will initialise
the kernel part of the driver, that will probe the SMMUs, initialise some
of the HW and populate a list of SMMUs to the hypervisor part to deal with.
The kernel driver exposes an IOMMU driver to the kernel that only supports
IDENTITY domains.
Then the hypervisor part of the driver will initialise before de-privilege
which then take over the shared structs, cmdq and create a shadow page table
for the SMMU.
In addition to the patches in this series, I am working on part-2 which adds
more features (EVTQ support for debuggability and RPM and possibly some
optimizations such as sharing page tables, split block unmap logic...)
https://android-kvm.googlesource.com/linux/+log/refs/heads/for-upstream/pkvm-smmu-v3-part-2
After that I plan to post 2 RFCs which adds support for translating
domains, one with paravirtual interface(similar to V2) and one with SMMUv3
emulation (similar to[3])
This series is tested on Morello board and Qemu.
[1] https://lore.kernel.org/kvmarm/20220519134204.5379-1-will@kernel.org/
[2] https://www.youtube.com/watch?v=9npebeVFbFw
[3] https://android-kvm.googlesource.com/linux/+log/refs/heads/smostafa/android15-6.6-smmu-nesting-wip-2
Jean-Philippe Brucker (11):
iommu/io-pgtable-arm: Split initialization
iommu/arm-smmu-v3: Move some definitions to a new common file
iommu/arm-smmu-v3: Extract driver-specific bits from probe function
iommu/arm-smmu-v3: Move some functions to arm-smmu-v3-common.c
iommu/arm-smmu-v3: Move firmware probe to arm-smmu-v3-common
iommu/arm-smmu-v3: Move IOMMU registration to arm-smmu-v3-common.c
iommu/arm-smmu-v3-kvm: Add SMMUv3 driver
iommu/arm-smmu-v3-kvm: Initialize registers
iommu/arm-smmu-v3-kvm: Reset the device
iommu/arm-smmu-v3-kvm: Add host driver for pKVM
iommu/arm-smmu-v3-kvm: Pass a list of SMMU devices to the hypervisor
Mostafa Saleh (18):
KVM: arm64: Add a new function to donate memory with prot
KVM: arm64: Donate MMIO to the hypervisor
KVM: arm64: pkvm: Add pkvm_time_get()
iommu/io-pgtable-arm: Split the page table driver
iommu/arm-smmu-v3: Move queue and table allocation to
arm-smmu-v3-common.c
iommu/arm-smmu-v3: Split cmdq code with hyp
iommu/arm-smmu-v3: Move TLB range invalidation into a macro
KVM: arm64: iommu: Introduce IOMMU driver infrastructure
KVM: arm64: iommu: Shadow host stage-2 page table
KVM: arm64: iommu: Add a memory pool
KVM: arm64: iommu: Add enable/disable hypercalls
iommu/arm-smmu-v3-kvm: Setup command queue
iommu/arm-smmu-v3-kvm: Setup stream table
iommu/arm-smmu-v3-kvm: Support io-pgtable
iommu/arm-smmu-v3-kvm: Shadow the CPU stage-2 page table
iommu/arm-smmu-v3-kvm: Add enable/disable device HVCs
iommu/arm-smmu-v3-kvm: Allocate structures and reset device
iommu/arm-smmu-v3-kvm: Add IOMMU ops
arch/arm64/include/asm/kvm_asm.h | 2 +
arch/arm64/include/asm/kvm_host.h | 14 +
arch/arm64/kvm/Makefile | 3 +-
arch/arm64/kvm/arm.c | 8 +-
arch/arm64/kvm/hyp/include/nvhe/iommu.h | 23 +
arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 3 +
arch/arm64/kvm/hyp/include/nvhe/pkvm.h | 2 +
arch/arm64/kvm/hyp/nvhe/Makefile | 10 +-
arch/arm64/kvm/hyp/nvhe/hyp-main.c | 19 +
arch/arm64/kvm/hyp/nvhe/iommu/iommu.c | 129 +++
arch/arm64/kvm/hyp/nvhe/mem_protect.c | 80 +-
arch/arm64/kvm/hyp/nvhe/setup.c | 17 +
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 33 +
arch/arm64/kvm/hyp/pgtable.c | 9 +-
arch/arm64/kvm/iommu.c | 65 ++
arch/arm64/kvm/pkvm.c | 1 +
drivers/iommu/Makefile | 2 +-
drivers/iommu/arm/Kconfig | 9 +
drivers/iommu/arm/arm-smmu-v3/Makefile | 6 +
.../arm/arm-smmu-v3/arm-smmu-v3-common-hyp.c | 114 +++
.../arm/arm-smmu-v3/arm-smmu-v3-common.c | 700 ++++++++++++++
.../arm/arm-smmu-v3/arm-smmu-v3-common.h | 624 ++++++++++++
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c | 393 ++++++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 904 +-----------------
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 532 +----------
.../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 676 +++++++++++++
.../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 51 +
.../arm/arm-smmu-v3/pkvm/io-pgtable-arm.c | 115 +++
drivers/iommu/io-pgtable-arm-common.c | 694 ++++++++++++++
drivers/iommu/io-pgtable-arm.c | 870 +----------------
drivers/iommu/io-pgtable-arm.h | 274 +++++-
31 files changed, 4102 insertions(+), 2280 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/nvhe/iommu.h
create mode 100644 arch/arm64/kvm/hyp/nvhe/iommu/iommu.c
create mode 100644 arch/arm64/kvm/iommu.c
create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common-hyp.c
create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common.c
create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common.h
create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c
create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c
create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h
create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/io-pgtable-arm.c
create mode 100644 drivers/iommu/io-pgtable-arm-common.c
--
2.50.1.552.g942d659e1b-goog
More information about the linux-arm-kernel
mailing list