[PATCH 0/3] MPXY interdomain bridge driver

[Pawandeep Oza]

From: Oza Pawandeep <pawandeep.oza at oss.qualcomm.com>

This series introduces two new capabilities to OpenSBI:

  1. A domain registration notifier mechanism
  2. RPMI REQUEST_FORWARD Service Group definations
  3. An FDT-configured MPXY bridge driver for inter-domain message passing

--- Motivation ---

Modern SBI deployments increasingly require cooperation between multiple
domains sharing the same physical hart — for example a trusted bare-metal
domain and an untrusted domain running on a single
core. While OpenSBI's domain context switching infrastructure provides the
mechanism for hart multiplexing between domains, there was no existing
facility for domains to exchange messages across domain boundaries.

This series addresses that gap by implementing a bridge over the MPXY
(Message Proxy) extension that uses domain context switching as its
transport.

--- Patch 1: Domain Registration Notifier ---

The first patch introduces a lightweight notifier infrastructure in
sbi_domain.c. Subsystems can register a callback via
sbi_domain_register_notifier() which is invoked from sbi_domain_register()
whenever a new domain is registered. This allows subsystems to observe
domain registration events and perform domain-specific setup at
registration time.

The MPXY bridge driver uses this to bind its channels to the named source
and destination domains as they appear during domain registration, without
requiring polling or deferred initialization.

--- Patch 2: RPMI REQUEST_FORWARD Service Group Definitions ---

Adds message protocol definitions for the RPMI REQUEST_FORWARD service
group (SERVICEGROUP_ID: 0x000D) to rpmi_msgprot.h. These definitions
implement the wire format for forwarded RPMI request messages between
domains as specified in the RPMI specification.

New definitions:

  - enum rpmi_request_forward_service_id: service IDs:
      REQFWD_ENABLE_NOTIFICATION       (0x01)
      REQFWD_RETRIEVE_CURRENT_MESSAGE  (0x02)
      REQFWD_COMPLETE_CURRENT_MESSAGE  (0x03)

  - struct rpmi_request_forward_resp: response for SERVICE_ID 0x02
    carrying status, remaining bytes, returned bytes and a flexible
    array member for inline message data.

  - struct rpmi_request_complete_resp: response for SERVICE_ID 0x03
    carrying status and the number of pending forwarded messages.

--- Patch 3: FDT-based MPXY Bridge Driver ---

The second patch implements the bridge driver itself. It is probed from
the FDT via the compatible string "opensbi,mpxy-bridge" and configured
with four properties:

  - opensbi,source-domain-name
  - opensbi,destination-domain-name
  - opensbi,source-channel-id
  - opensbi,destination-channel-id

The bridge registers two MPXY channels — one for the source domain
(sender) and one for the destination domain (receiver). It implements
the RPMI REQUEST_FORWARD (SERVICEGROUP_ID: 0x000D) service group:

  - SERVICE_ID 0x02: REQFWD_RETRIEVE_CURRENT_MESSAGE
  - SERVICE_ID 0x03: REQFWD_COMPLETE_CURRENT_MESSAGE

Message delivery uses per-hart slots so that each hart manages its own
message state independently. Domain context switching is performed via
sbi_domain_context_enter() and sbi_domain_context_exit() to cooperatively
yield execution between the source and destination domains on the shared
hart.

The driver validates at bind time that the source and destination domains
share identical possible_harts masks, enforcing the requirement that
cooperative context switching is only possible when both domains can run
on the same physical hart.

--- Device Tree Example ---

The following DT snippet illustrates a two-domain configuration with a
single shared hart (cpu1). The trusted-domain runs a bare-metal application
at 0x80100000 in S-mode. The untrusted-domain runs Linux with access to
the rest of memory. A bridge node connects them:

  opensbi-domains {
      compatible = "opensbi,domain,config";

      tmem: tmem {
          compatible = "opensbi,domain,memregion";
          base = <0x0 0x80100000>;
          order = <20>;
      };

      tuart: tuart {
          compatible = "opensbi,domain,memregion";
          base = <0x0 0x10011000>;
          order = <12>;
          mmio;
          devices = <&uart1>;
      };

      allmem: allmem {
          compatible = "opensbi,domain,memregion";
          base = <0x0 0x0>;
          order = <64>;
      };

      tdomain: trusted-domain {
          compatible = "opensbi,domain,instance";
          possible-harts = <&cpu1>;
          regions = <&tmem 0x3f>, <&tuart 0x3f>;
          boot-hart = <&cpu1>;
          next-arg1 = <0x0 0x0>;
          next-addr = <0x0 0x80100000>;
          next-mode = <0x1>;
          system-reset-allowed;
      };

      udomain: untrusted-domain {
          compatible = "opensbi,domain,instance";
          possible-harts = <&cpu1>;
          regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x3f>;
      };

      bridge at 0 {
          compatible = "opensbi,mpxy-bridge";
          opensbi,source-domain-name = "untrusted-domain";
          opensbi,destination-domain-name = "trusted-domain";
          opensbi,source-channel-id = <1>;
          opensbi,destination-channel-id = <2>;
      };
  };

--- Testing ---

The series was tested on QEMU SiFive Unleashed (sifive_u) with the
above device tree configuration. The test setup consists of:

  - hart1 shared between trusted-domain and untrusted-domain
  - trusted-domain: bare-metal application loaded at 0x80100000
    booting in S-mode via the domain's next-addr
  - untrusted-domain: Linux kernel with a kernel thread sending
    periodic MPXY messages via FID#5 (SEND_MSG_WITH_RESPONSE)
    and FID#6 (SEND_MSG_NO_RESPONSE) on MPXY channel 0x1
  - A separate bare-metal telnet console application polling
    MPXY channel 0x2 using REQFWD_RETRIEVE_CURRENT_MESSAGE
    and REQFWD_COMPLETE_CURRENT_MESSAGE

The Linux kernel thread sends a test payload ("MPXY-HELLO seq=N") every
30 seconds alternating between FID#5 (expecting a response) and FID#6
(fire-and-forget). The bare-metal application retrieves the forwarded
message via the REQFWD service group and echoes it back as the
COMPLETE response. The Linux driver verifies receipt of the echoed
response.

Both FID#5 and FID#6 paths were exercised and verified end-to-end
across the domain boundary on a single shared hart without requiring
any dedicated inter-processor communication hardware.

Oza Pawandeep (3):
  lib: sbi: domain: add domain registration notifier infrastructure
  sbi_utils: mailbox: add RPMI REQUEST_FORWARD service group definitions
  sbi_utils: mpxy: add FDT-based MPXY bridge driver

 include/sbi/sbi_domain.h                 |  10 +
 include/sbi_utils/mailbox/rpmi_msgprot.h |  20 +
 lib/sbi/sbi_domain.c                     |  59 ++
 lib/utils/mpxy/fdt_mpxy_bridge.c         | 676 +++++++++++++++++++++++
 lib/utils/mpxy/objects.mk                |   3 +
 5 files changed, 768 insertions(+)
 create mode 100644 lib/utils/mpxy/fdt_mpxy_bridge.c

-- 
2.43.0