[PATCH v2 0/2] Fix domain assignment TOCTOU races

[Huamao Wu]

The HART-to-domain pointer and the assigned-HART bitmap are updated by
separate statements: the old domain's bit is cleared, the pointer is
rewritten, then the new domain's bit is set. While a HART moves from
domain A to domain B it is in no bitmap at all, and for part of that
window the pointer already says B, so a reader that looks at both gets
an answer that depends on which of the two it read.

Patch 1 moves the whole update into sbi_update_hartindex_to_domain(),
serialized by a new domain_hart_assign_lock, so a reader holding that lock
cannot observe the pointer and the bitmap disagree. Patch 2 holds the
same lock across the ownership check and the STOPPED -> START_PENDING
transition in sbi_hsm_hart_start(), closing the matching window between
validating a start request and accepting it. Mechanics are in the commit
messages.

The diff this series follows was written against b3f5892d and the series
is on d28bc6f8; the only adaptations are that the lock is exported for
sbi_hsm.c, and that the open-coded bitmap update in sbi_domain_register()
is folded into the updater instead of clearing the bit outside the lock.

Scope: after patch 2 every reader that takes domain_hart_assign_lock sees
the two agree, but a HART is still briefly absent from every bitmap, so
readers of assigned_harts_lock alone -- sbi_domain_get_assigned_hartmask()
callers and the sbi_system_suspend() scan -- can still miss a HART under
reassignment. Fixing that means changing when the set happens, not where
the lock sits. sbi_sse_inject_from_ecall() also reads the pointer and the
bitmap separately, and closing it would widen the lock into an SSE
deliverability rule. Both are outside the scope of this series.

Changes since v1:
- Reorganized per review: patch 1 is limited to
  sbi_update_hartindex_to_domain() and its callers, patch 2 only
  serializes HSM hart_start.
- The API names introduced by v1 are dropped; existing names are kept.
- Problem statement and fix unchanged.

These patches can also be found in the domain-assignment-race-fix-v2
branch at: https://github.com/kasperis7/opensbi/tree/domain-assignment-race-fix-v2

Huamao Wu (2):
  lib: sbi_domain: Make HART to domain updates atomic
  lib: sbi_hsm: Serialize HART start against domain reassignment

 include/sbi/sbi_domain.h     | 16 ++++++++++++++--
 lib/sbi/sbi_domain.c         | 37 +++++++++++++++++++++++++++---------
 lib/sbi/sbi_domain_context.c |  8 --------
 lib/sbi/sbi_hsm.c            | 35 +++++++++++++++++++++++++++-------
 4 files changed, 70 insertions(+), 26 deletions(-)

-- 
2.43.0