[PATCH RFC v15 01/30] docs: gunyah: Introduce Gunyah Hypervisor
Elliot Berman
quic_eberman at quicinc.com
Fri Dec 15 16:20:42 PST 2023
Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It
does not depend on any lower-privileged OS/kernel code for its core
functionality. This increases its security and can support a smaller
trusted computing based when compared to Type-2 hypervisors.
Add documentation describing the Gunyah hypervisor and the main
components of the Gunyah hypervisor which are of interest to Linux
virtualization development.
Reviewed-by: Bagas Sanjaya <bagasdotme at gmail.com>
Signed-off-by: Elliot Berman <quic_eberman at quicinc.com>
---
Documentation/virt/gunyah/index.rst | 121 ++++++++++++++++++++++++++++
Documentation/virt/gunyah/message-queue.rst | 69 ++++++++++++++++
Documentation/virt/index.rst | 1 +
3 files changed, 191 insertions(+)
diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
new file mode 100644
index 000000000000..fe3f4192b836
--- /dev/null
+++ b/Documentation/virt/gunyah/index.rst
@@ -0,0 +1,121 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Gunyah Hypervisor
+=================
+
+.. toctree::
+ :maxdepth: 1
+
+ message-queue
+
+Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
+a higher CPU privilege level. It does not depend on any lower-privileged
+operating system for its core functionality. This increases its security and can
+support a much smaller trusted computing base than a Type-2 hypervisor.
+
+Gunyah is an open source hypervisor. The source repo is available at
+https://github.com/quic/gunyah-hypervisor.
+
+Gunyah provides these following features.
+
+- Scheduling:
+
+ A scheduler for virtual CPUs (vCPUs) on physical CPUs enables time-sharing
+ of the CPUs. Gunyah supports two models of scheduling:
+
+ 1. "Behind the back" scheduling in which Gunyah hypervisor schedules vCPUS
+ on its own.
+ 2. "Proxy" scheduling in which a delegated VM can donate part of one of its
+ vCPU slice to another VM's vCPU via a hypercall.
+
+- Memory Management:
+
+ APIs handling memory, abstracted as objects, limiting direct use of physical
+ addresses. Memory ownership and usage tracking of all memory under its control.
+ Memory partitioning between VMs is a fundamental security feature.
+
+- Interrupt Virtualization:
+
+ Uses CPU hardware interrupt virtualization capabilities. Interrupts are
+ handled in the hypervisor and routed to the assigned VM.
+
+- Inter-VM Communication:
+
+ There are several different mechanisms provided for communicating between VMs.
+
+- Virtual platform:
+
+ Architectural devices such as interrupt controllers and CPU timers are
+ directly provided by the hypervisor as well as core virtual platform devices
+ and system APIs such as ARM PSCI.
+
+- Device Virtualization:
+
+ Para-virtualization of devices is supported using inter-VM communication.
+
+Architectures supported
+=======================
+AArch64 with a GIC
+
+Resources and Capabilities
+==========================
+
+Services/resources provide by the Gunyah hypervisor are described to a
+virtual machine by capability IDs. For instance, inter-VM communication is
+performed with doorbells and message queues. Gunyah allows access to interact
+with that doorbell via the capability ID. These resources are described in Linux
+as a struct gunyah_resource.
+
+High level management of these resources is performed by the resource manager
+VM. RM informs a guest VM about resources it can access through either the
+device tree or via guest-initiated RPC.
+
+Gunyah tracks all resources and selectively makes them available to some VMs via
+VM-specific capability IDs.
+
+Resource Manager
+================
+
+The resource manager (RM) is a privileged application VM supporting the Gunyah
+Hypervisor. It provides policy enforcement aspects of the virtualization system.
+The resource manager can be treated as an extension of the Hypervisor but is
+separated to its own partition to ensure that the hypervisor layer itself
+remains small and secure and to maintain a separation of policy and mechanism in
+the platform. The resource manager runs at arm64 NS-EL1, similar to other
+virtual machines.
+
+Communication with the resource manager from other virtual machines happens with
+message-queue.rst. Details about the specific messages can be found in
+drivers/virt/gunyah/rsc_mgr.c
+
+::
+
+ +-------+ +--------+ +--------+
+ | RM | | VM_A | | VM_B |
+ +-.-.-.-+ +---.----+ +---.----+
+ | | | |
+ +-.-.-----------.------------.----+
+ | | \==========/ | |
+ | \========================/ |
+ | Gunyah |
+ +---------------------------------+
+
+The source for the resource manager is available at
+https://github.com/quic/gunyah-resource-manager.
+
+The resource manager provides the following features:
+
+- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs
+- VM access control policy, including memory sharing and lending
+- Interrupt routing configuration
+- Forwarding of system-level events (e.g. VM shutdown) to owner VM
+- Resource (capability) discovery
+
+Resource manager presently requires the guest virtual machines uses a device
+tree to describe its configuration. The resource manager applies an overlay to
+the/hypervisor node. This node lets guests know they are running as a Gunyah
+guest VM, how to communicate with resource manager, and basic description and
+capabilities of this VM. See
+Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a
+description of this node.
diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
new file mode 100644
index 000000000000..bd9947240c2a
--- /dev/null
+++ b/Documentation/virt/gunyah/message-queue.rst
@@ -0,0 +1,69 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Message Queues
+==============
+Message queue is a simple low-capacity IPC channel between two virtual machines.
+It is intended for sending small control and configuration messages. Each
+message queue is unidirectional, so a full-duplex IPC channel requires a pair of
+queues.
+
+Messages can be up to 240 bytes in length. Longer messages require a further
+protocol on top of the message queue messages themselves. For instance,
+communication with the resource manager adds a header field for sending longer
+messages via multiple message fragments.
+
+The diagram below shows how message queue works. A typical configuration
+involves 2 message queues. Message queue 1 allows VM_A to send messages to VM_B.
+Message queue 2 allows VM_B to send messages to VM_A.
+
+1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall
+ with the message to inform the hypervisor to add the message to
+ message queue 1's queue. The hypervisor copies memory into the internal
+ message queue representation; the memory doesn't need to be shared between
+ VM_A and VM_B.
+
+2. Gunyah raises the corresponding interrupt for VM_B (Rx vIRQ) when any of
+ these happens:
+
+ a. gunyah_msgq_send() has PUSH flag. This is a typical case.
+ b. Explicility with gunyah_msgq_push command from VM_A.
+ c. Message queue has reached a threshold depth. Typically, this threshold
+ depth is the size of the queue (in other words: when queue is full, Rx
+ vIRQ raised).
+
+3. VM_B calls gunyah_msgq_recv() and Gunyah copies message to requested buffer.
+
+4. Gunyah raises the corresponding interrupt for VM_A (Tx vIRQ) when the message
+ queue falls below a watermark depth. Typically, this is the size of the queue
+ (in other words: when the queue is no longer full, Tx vIRQ raised). Note the
+ watermark depth and the threshold depth for the Rx vIRQ are independent
+ values, although they are both typically the size of the queue.
+ Coincidentally, this signal is conceptually similar to Clear-to-Send.
+
+For VM_B to send a message to VM_A, the process is identical, except that
+hypercalls reference message queue 2's capability ID. Each message queue has its
+own independent vIRQ: two TX message queues will have two vIRQs (and two
+capability IDs).
+
+::
+
+ +---------------+ +-----------------+ +---------------+
+ | VM_A | |Gunyah hypervisor| | VM_B |
+ | | | | | |
+ | | | | | |
+ | | Tx | | | |
+ | |-------->| | Rx vIRQ | |
+ |gunyah_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gunyah_msgq_recv() |
+ | |<------- | | | |
+ | | | | | |
+ | Message Queue | | | | Message Queue |
+ | driver | | | | driver |
+ | | | | | |
+ | | | | | |
+ | | | | Tx | |
+ | | Rx vIRQ | |<--------| |
+ |gunyah_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gunyah_msgq_send() |
+ | | | |-------->| |
+ | | | | | |
+ | | | | | |
+ +---------------+ +-----------------+ +---------------+
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index 7fb55ae08598..15869ee059b3 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -16,6 +16,7 @@ Virtualization Support
coco/sev-guest
coco/tdx-guest
hyperv/index
+ gunyah/index
.. only:: html and subproject
--
2.43.0
More information about the linux-arm-kernel
mailing list