[PATCH v6] Add virtio SCMI device specification
Cristian Marussi
cristian.marussi at arm.com
Mon Feb 15 08:20:44 EST 2021
Hi Peter,
just one remark down below.
On Fri, Feb 12, 2021 at 10:59:20AM +0100, Peter Hilber wrote:
> This patch proposes a new virtio device for the Arm SCMI protocol.
>
> The device provides a simple transport for the Arm SCMI protocol[1]. The
> *S*ystem *C*ontrol and *M*anagement *I*nterface protocol allows speaking
> to system controllers that allow orchestrating things like power
> management, system state management and sensor access. The SCMI protocol
> is used on SoCs where multiple cores and co-processors need access to
> these resources.
>
> The virtio transport allows making use of this protocol in virtualized
> systems.
>
> [1] https://developer.arm.com/docs/den0056/c
>
> Signed-off-by: Peter Hilber <peter.hilber at opensynergy.com>
> ---
>
> Notes:
>
> Since sending out v5 in May 2020, there has been no additional
> feedback w.r.t. the spec. An RfC patch series for the driver is now
> available at [1]. OpenSynergy also has a proprietary implementation
> of the device (without the VIRTIO_SCMI_F_SHARED_MEMORY feature so
> far).
>
> So I would like to request a vote on adding the device in a few weeks.
>
> The PDF output is available at [2].
>
> [1] https://lore.kernel.org/linux-arm-kernel/20201105212116.411422-1-peter.hilber@opensynergy.com/
> [2] https://share.mailbox.org/ajax/share/056076e70571144f50c4ca7571144b319a1d7236dda1cd3b/1/8/MzQ/MzQvMQ
>
> Changes for v6:
>
> - Refer to new SCMI spec version 3.0 (DEN0056C), which has a
> provision for SHM race conditions.
>
> - Resolve conflicts with other added device documentation.
>
> Changes for v5:
>
> - Remove requirements on shared memory added in v4, since an upcoming
> new version of the Arm SCMI spec[1] will provide a generic, more
> powerful way to handle concurrent access to shared memory.
>
> Changes for v4:
>
> - Add more requirements on shared memory regions after feedback from
> Alex Bennée.
>
> Changes for v3:
>
> - Add tentative device ID 32 in device section.
>
> - Remove redundant 'len' fields. The length of the payload fields can
> already be deduced from the generic virtqueue 'len' fields. Therefore,
> remove the redundant device-specific 'len' fields.
>
> - Reword requirement that driver must not put too small buffers into
> eventq.
>
> Changes for v2:
>
> - CC virtio-dev list.
> - Define size of erroneous delayed/not delayed responses.
> - Use correct long name for SCMI.
> - Remove restriction to `embedded' in commit message.
> - Add motivation for conceptual per-message-channels.
> - Device may now just drop notification if no buffers are available.
>
>
> conformance.tex | 34 ++++--
> content.tex | 1 +
> introduction.tex | 3 +
> virtio-scmi.tex | 271 +++++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 303 insertions(+), 6 deletions(-)
> create mode 100644 virtio-scmi.tex
>
[snip]
> +The cmdq is used by the driver to send commands to the device. The
> +device replies with responses (not delayed responses) over the cmdq.
> +
> +The eventq is used by the device to send notifications and delayed
> +responses. The eventq only exists if VIRTIO_SCMI_F_P2A_CHANNELS was
> +negotiated.
> +
> +\subsection{Feature bits}\label{sec:Device Types / SCMI Device / Feature bits}
> +
> +\begin{description}
> +\item[VIRTIO_SCMI_F_P2A_CHANNELS (0)] Device implements some SCMI
> +notifications, or delayed responses.
> +\item[VIRTIO_SCMI_F_SHARED_MEMORY (1)] Device implements any SCMI
> +statistics shared memory region.
> +\end{description}
> +
> +VIRTIO_SCMI_F_P2A_CHANNELS is used to determine the existence of the
> +eventq. The eventq is required for SCMI notifications and delayed
> +responses.
> +
> +VIRTIO_SCMI_F_SHARED_MEMORY is used to determine whether the device
> +provides any SCMI statistics shared memory region. SCMI statistics
> +shared memory regions are defined by some SCMI protocols.
> +
> +The SCMI protocols provide the PROTOCOL_MESSAGE_ATTRIBUTES commands to
> +inquire about the particular SCMI notifications and delayed responses
> +implemented by the device. The SCMI protocols provide additional
> +commands to detect other features implemented by the device.
> +
> +\devicenormative{\subsubsection}{Feature bits}{Device Types / SCMI Device / Feature bits}
> +
> +The device MUST offer VIRTIO_SCMI_F_P2A_CHANNELS if the device can
> +implement at least one SCMI notification, or delayed response.
> +
> +The device MUST offer VIRTIO_SCMI_F_SHARED_MEMORY if the device can
> +implement at least one SCMI statistics shared memory region.
> +
> +\subsection{Device configuration layout}\label{sec:Device Types / SCMI Device / Device configuration layout}
> +
> +There is no configuration data for the device.
> +
I could be missing something here, so I apologize upfront if this has
already been discussed elsewhere, but AFAIU the fact that no device
configuration is exposed here (like a device ID) implies that SCMI devices
are not "identifiable" from within the VirtIO subsystem, so not
distinguishable in other words.
At the same time in the companion RFC driver series it has been already
also ruled out (as of now at least) by the DT maintainer the possibility
to establish a link at the DT level between an SCMI virtio-mmio device
definition and the SCMI related DT descriptions (because, AFAIU, it's not
something needed by any of all the other existent VirtIO devices that are
indeed usually somehow 'identifiable')
These 2 things combined together rule out any future possibility to have
multiple SCMI devices defined on a system (at least not identifiable/usable)
in order to support multiple distinct transport channels against the same
platform (or, less desirable, multiple platforms), and in fact the SCMI virtio
transport RFC driver as of now states upfront that only one device/once channel
configuration is supported.
Even if not the norm probably, I would say that could be useful to leave
open for the future the possibility to redirect some protocols down through
a specific distinct VirtIO channel (other than the main one), at least for
debug/testing purposes if not to support multiple platform residing in
distinct VMs.
Another use case could be separating the transport channel used by some
possibly chatty protocol like Sensors (possibly flooding the system with
lots of notifications) from other SCMI protocols, that could benefit from
having a dedicated quieter virtio-mmio SCMI device channel (with dedicated
IRQs and friends...)
Beside that, also the general fact that this SCMI transport does not support
multiple channels is a bit at odd with all the other SCMI transports
(like mailbox/svc that, it's to be said, are indeed based instead on
shared memory)
So, what I'm saying here is if it's not the case of having some sort of
device_id in the configuration space, so that a platform could expose,
if it wants, multiple identifiable devices and the agents could pick the
device they want and associate with a specific protocol.
(via some DT transport reference to the desired SCMI device_id to use,
similarly to what's possible with shared mems transports)
But, as said, it could have been already discussed and ruled out so feel
free to ignore this and point me at the related pre-existing discussion
threads.
Thanks
Cristian
> +\subsection{Device Initialization}\label{sec:Device Types / SCMI Device / Device Initialization}
> +
> +The
> +\hyperref[sec:General Initialization And Device Operation / Device Initialization]{general
> +requirements on device initialization} apply.
> +
> +\subsection{Device Operation}\label{sec:Device Types / SCMI Device / Device Operation}
> +
> +The SCMI transport used for the device puts each SCMI message into a
> +dedicated virtio buffer. The driver uses the cmdq for transmitting SCMI
> +commands and receiving the corresponding SCMI responses. The device uses
> +the eventq for transmitting SCMI notifications and delayed responses.
> +Each message includes an SCMI protocol header and payload, as defined by
> +the \hyperref[intro:SCMI]{SCMI specification}.
> +
> +\subsubsection{cmdq Operation}\label{sec:Device Types / SCMI Device / Device Operation / cmdq Operation}
> +
> +Each buffer in the cmdq holds a single SCMI command once the buffer has
> +been made available. When the buffer has been marked as used, it
> +contains the SCMI response. An arbitrary number of such SCMI messages
> +can be in transit at the same time. Conceptually, each SCMI message in
> +the cmdq uses its own SCMI A2P (agent to platform) channel.
> +
> +The SCMI response is in the same virtio buffer as the corresponding SCMI
> +command. The response contains the return values which SCMI specifies
> +for each command, whether synchronous or asynchronous. Delayed responses
> +are distinct SCMI messages transmitted over the eventq.
> +
> +Buffers in the cmdq contain both the request and the response. A request
> +has the following layout:
> +
> +\begin{lstlisting}
> +struct virtio_scmi_request {
> + le32 hdr;
> + u8 params[<actual parameters size>];
> +};
> +\end{lstlisting}
> +
> +The virtio_scmi_request fields are interpreted as follows:
> +
> +\begin{description}
> +\item[\field{hdr}] (device-readable) contains the SCMI message header
> +\item[\field{params}] (device-readable) comprises the SCMI message
> +parameters
> +\end{description}
> +
> +A cmdq response has the following layout:
> +
> +\begin{lstlisting}
> +struct virtio_scmi_response {
> + le32 hdr;
> + u8 ret_values[<actual return values size>];
> +};
> +\end{lstlisting}
> +
> +The virtio_scmi_response fields are interpreted as follows:
> +
> +\begin{description}
> +\item[\field{hdr}] (device-writable) contains the SCMI message header
> +\item[\field{ret_values}] (device-writable) comprises the SCMI message
> +return values
> +\end{description}
> +
> +If VIRTIO_SCMI_F_P2A_CHANNELS was not negotiated, the device responds to
> +SCMI commands as if no SCMI notifications or delayed responses were
> +implemented.
> +
> +\devicenormative{\paragraph}{cmdq Operation}{Device Types / SCMI Device / Device Operation / cmdq Operation}
> +
> +The device MAY process available commands out of order and in parallel.
> +
> +The device MUST process all available commands eventually, even in the
> +case of bursts of multiple command messages.
> +
> +If the \field{status} field in the \field{virtio_scmi_response}
> +\field{ret_values} has a value other than SUCCESS, the device MUST set
> +the size of \field{ret_values} to the size of the \field{status} field.
> +
> +If the driver requests an SCMI notification or a delayed response and
> +there are currently NOT enough available buffers in the eventq, the
> +device SHOULD still return SCMI status code SUCCESS.
> +
> +If VIRTIO_SCMI_F_P2A_CHANNELS was not negotiated, the device MUST deny
> +any request for an SCMI notification or a delayed response by returning
> +SCMI status code NOT_SUPPORTED.
> +
> +If VIRTIO_SCMI_F_P2A_CHANNELS was not negotiated, the device MUST NOT
> +indicate in the PROTOCOL_MESSAGE_ATTRIBUTES return values that any SCMI
> +notification, or delayed response, is implemented.
> +
> +\drivernormative{\paragraph}{cmdq Operation}{Device Types / SCMI Device / Device Operation / cmdq Operation}
> +
> +Before sending a command, the driver MUST wait for responses to all
> +commands whose completion the driver considers prerequisites to
> +executing the command.
> +
> +With every command message, the driver MUST provide enough
> +device-writable memory to enable the device to return corresponding
> +return values.
> +
> +If VIRTIO_SCMI_F_P2A_CHANNELS was not negotiated, the driver MUST NOT
> +request any SCMI notification, nor any delayed response.
> +
> +\subsubsection{Setting Up eventq Buffers}
> +
> +The driver has to populate the eventq before the device can use it.
> +
> +\drivernormative{\paragraph}{Setting Up eventq Buffers}{Device Types / SCMI Device / Device Operation / Setting Up eventq Buffers}
> +
> +If VIRTIO_SCMI_F_P2A_CHANNELS was negotiated, the driver SHOULD populate
> +the eventq with buffers.
> +
> +The driver MUST NOT put device-readable descriptors into the eventq.
> +
> +The driver MUST NOT put into the eventq any buffer which is smaller than
> +the largest SCMI P2A (platform to agent) message which the driver will
> +request.
> +
> +\subsubsection{eventq Operation}
> +
> +Each buffer in the eventq holds (once the buffer is marked as used)
> +either a single SCMI notification, or a single SCMI delayed response. An
> +arbitrary number of such SCMI messages can be in transit at the same
> +time. Conceptually, each SCMI message transmitted over the eventq uses
> +its own SCMI P2A (platform to agent) channel. Buffers in the eventq have
> +the following layout:
> +
> +\begin{lstlisting}
> +struct virtio_scmi_event_msg {
> + /* start of device-writable data */
> + le32 hdr;
> + u8 payload[<actual payload size>];
> +};
> +\end{lstlisting}
> +
> +\begin{description}
> +\item[\field{hdr}] (device-writable) contains the SCMI message header
> +\item[\field{payload}] (device-writable) comprises the SCMI message
> +payload
> +\end{description}
> +
> +\devicenormative{\paragraph}{eventq Operation}{Device Types / SCMI Device / Device Operation / eventq Operation}
> +
> +If the device intends to send a notification and there are no available
> +buffers in the eventq, the device MAY drop the notification, or send a
> +corresponding notification later, once enough buffers become available.
> +
> +The device MAY send the notification later if the events which cause the
> +notification take place in quick succession.
> +
> +If the device sends the notification later, the device MAY send the
> +notification with updated data, unless the specific SCMI protocol
> +disallows this.
> +
> +If the device intends to send a notification and there are available
> +buffers, but one of the buffers is too small to fit the notification,
> +the device MAY omit the notification.
> +
> +If the device intends to send a delayed response and there are no
> +available buffers in the eventq, the device MUST send the corresponding
> +delayed response once enough buffers become available.
> +
> +If the \field{status} field in a delayed response \field{payload} has a
> +value other than SUCCESS, the device MUST set the size of
> +\field{payload} to the size of the \field{status} field.
> +
> +\subsubsection{Shared Memory Operation}
> +
> +Various SCMI protocols define statistics shared memory regions (for
> +statistics and sensor values).
> +
> +\devicenormative{\paragraph}{Shared Memory Operation}{Device Types / SCMI Device / Device Operation / Shared Memory Operation}
> +
> +If VIRTIO_SCMI_F_SHARED_MEMORY was negotiated, the device MAY implement
> +an SCMI statistics shared memory region using a virtio shared memory
> +region.
> +
> +If the device implements a shared memory region, the device MUST assign
> +the corresponding shmid as per the following table:
> +
> +\begin{tabular}{|l|l|}
> +\hline
> +SCMI statistics shared memory region & Virtio shmid \\
> +\hline \hline
> +Reserved (invalid) & 0 \\
> +\hline
> +Power state statistics shared memory region & 1 \\
> +\hline
> +Performance domain statistics shared memory region & 2 \\
> +\hline
> +Sensor Values Shared Memory & 3 \\
> +\hline
> +Reserved for future use & 4 to 0x7F \\
> +\hline
> +Vendor-specific statistics shared memory regions & 0x80 to 0xFF \\
> +\hline
> +Reserved for future use & 0x100 and greater \\
> +\hline
> +\end{tabular}
>
> base-commit: f5fd3fca7e4006108b3f9ba91c0b76c5eb6c0726
> --
> 2.25.1
>
>
More information about the linux-arm-kernel
mailing list