[RFC PATCH v2 00/15] Arm Core Local Accelerator Driver
Ryan Roberts
ryan.roberts at arm.com
Thu Oct 15 07:25:34 PDT 2026
Hi All,
This is v2 of an RFC to introduce a driver for the Arm Core Local Accelerator
(CLA), a CPU-local interface for programming attached accelerators. While the
interface is agnostic to the accelerator type, the initial (and currently only)
target is a compute engine.
The RFC aims to engage the community and get feedback on some key aspects. This
will help plan our approach for eventual upstreaming. Based on feedback from
Will and Marc in RFCv1, the driver now supports both bare metal and virtualized
environments. I also have commitments to publish the CLA spec, a supporting FVP
and a representaive user space, all to support upstreaming the driver, as per
Will's feedback.
For now you can find documentation included with patch 1, which describes the HW
and driver design. Note that the CLA is not part of the Arm Architecture.
Patch 1 documents the hardware and driver design, and adds a driver skeleton.
Patches 2-5 initialize and probe the device. Patches 6-9 add context management
for switching the device between different users. Patches 10-12 add some (small)
changes to KVM and VFIO, which then enables patches 13-15 to expose the HW for
device assignment into a VM.
Aspects I'm seeking feedback for:
* The general structure of the driver: The current shape addresses the
performance requirements we have for the use cases, and is therefore our
preferred approach. But there are some unusual aspects due to the HW design.
* Driver interaction with architectural support: I've opted to treat the CLA as
a device rather than a CPU extension, so it's implemented as a (mostly)
self-contained driver. It has some unavoidable coupling to the arch code since
it needs to share page tables and ASIDs/VMIDs. All arm64 arch interaction is
contained in cla-mtc.c.
* The location of the driver: although most accelerators will likely be compute
ones, CLA is a generic MMIO interface that could handle any kind of
accelerator. It's currently implemented as a misc driver. accel is another
potential option, but given the current SVA approach, we would not use any of
the services (memory-management or otherwise) that accel provides.
* Virtualization support: since the HW uses the CPU's MMU, we introduced a new
VFIO_CPU_MMU group type, and added the kvm_arch_[get|put]_guest_mmu() API to
KVM/arm64 so that the CLA can share the stage2 pgtables and VMID. The CPU
pinning problem remains unsolved; the vCPU must be pinned to the pCPU that
owns the CLA in order to access it. If the pCPU goes offline or the vCPU is
otherwise unpinned, it would look to the guest like the HW is faulty. This
issue is therefore contained to the guest. I've additionally improved the
robustness of the core driver to deal with this eventuality gracefully.
I consider the core driver to be well tested and robust. The virtualization
portion is prototype and has known issues that would be addressed assuming the
approach gets positive feedback.
Changes since RFCv1 [1]
=======================
- Added KVM guest support through VFIO mdev, including exclusive domain
assignment, stage-2 memory translation and VMID pinning
- Changed the userspace ABI to expose one `/dev/cla/<class-id>` device per
accelerator class; `ARM_CLA_PARAM_DEV_ACCELS` now reports only accelerators
belonging to the opened class (per Arnd)
- Removed `ARM_CLA_UABI_VERSION` and the `ARM_CLA_PARAM_UABI_VERSION` query
(per Arnd)
- Various driver bug fixes and robustness improvements
Source, along with some tests, is available at [2]. Patches based on v7.3-rc5.
[1] https://lore.kernel.org/all/20260717104759.123203-1-ryan.roberts@arm.com/
[2] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/cla-driver-v2-rfc-tests
Thanks,
Ryan
Jean-Philippe Brucker (10):
misc/arm-cla: Add driver skeleton and documentation
misc/arm-cla: Add launch operation helpers
misc/arm-cla: Probe firmware-described devices
misc/arm-cla: Initialize devices on CPU bringup
misc/arm-cla: Accelerator context save and restore
KVM: arm64: Export stage-2 MMU config
vfio: Support VFIO devices that access the CPU MMU
misc/arm-cla: Add stage-2 MTC and VM context switch
misc/arm-cla: Support exclusive context assignment
misc/arm-cla: Add vfio-mdev user interface
Ryan Roberts (4):
dt-bindings: misc: Add Arm Core Local Accelerator
misc/arm-cla: Set up memory translation context
misc/arm-cla: Manage domain contexts
misc/arm-cla: Add userspace interface
Shameer Kolothum (1):
KVM: arm64: Introduce support to pin VMIDs
.../devicetree/bindings/misc/arm,cla.yaml | 77 +++
Documentation/misc-devices/arm-cla.rst | 217 +++++++
Documentation/misc-devices/index.rst | 1 +
MAINTAINERS | 9 +
arch/arm64/include/asm/kvm_host.h | 14 +
arch/arm64/include/asm/kvm_mmu.h | 1 +
arch/arm64/kvm/mmu.c | 102 ++++
arch/arm64/kvm/sys_regs.c | 14 +-
arch/arm64/kvm/vmid.c | 77 ++-
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/arm-cla/Kconfig | 26 +
drivers/misc/arm-cla/Makefile | 15 +
drivers/misc/arm-cla/arm-cla-regs.h | 192 ++++++
drivers/misc/arm-cla/arm-cla.h | 393 ++++++++++++
drivers/misc/arm-cla/cla-ctx.c | 229 +++++++
drivers/misc/arm-cla/cla-init.c | 562 ++++++++++++++++++
drivers/misc/arm-cla/cla-mdev.c | 469 +++++++++++++++
drivers/misc/arm-cla/cla-mtc.c | 335 +++++++++++
drivers/misc/arm-cla/cla-ops.c | 417 +++++++++++++
drivers/misc/arm-cla/cla-regs.c | 370 ++++++++++++
drivers/misc/arm-cla/cla-sched.c | 474 +++++++++++++++
drivers/misc/arm-cla/cla-topology.c | 202 +++++++
drivers/misc/arm-cla/cla-user.c | 541 +++++++++++++++++
drivers/vfio/group.c | 37 +-
drivers/vfio/vfio.h | 5 +
drivers/vfio/vfio_main.c | 16 +-
include/linux/vfio.h | 1 +
include/uapi/linux/arm-cla.h | 198 ++++++
29 files changed, 4969 insertions(+), 27 deletions(-)
create mode 100644 Documentation/devicetree/bindings/misc/arm,cla.yaml
create mode 100644 Documentation/misc-devices/arm-cla.rst
create mode 100644 drivers/misc/arm-cla/Kconfig
create mode 100644 drivers/misc/arm-cla/Makefile
create mode 100644 drivers/misc/arm-cla/arm-cla-regs.h
create mode 100644 drivers/misc/arm-cla/arm-cla.h
create mode 100644 drivers/misc/arm-cla/cla-ctx.c
create mode 100644 drivers/misc/arm-cla/cla-init.c
create mode 100644 drivers/misc/arm-cla/cla-mdev.c
create mode 100644 drivers/misc/arm-cla/cla-mtc.c
create mode 100644 drivers/misc/arm-cla/cla-ops.c
create mode 100644 drivers/misc/arm-cla/cla-regs.c
create mode 100644 drivers/misc/arm-cla/cla-sched.c
create mode 100644 drivers/misc/arm-cla/cla-topology.c
create mode 100644 drivers/misc/arm-cla/cla-user.c
create mode 100644 include/uapi/linux/arm-cla.h
--
2.43.0
More information about the linux-arm-kernel
mailing list