[PATCH v4 09/13] arm64: debug: split single stepping exception entry

[Will Deacon]

On Fri, Jun 20, 2025 at 10:12:03PM +0100, Ada Couprie Diaz wrote:
> Currently all debug exceptions share common entry code and are routed
> to `do_debug_exception()`, which calls dynamically-registered
> handlers for each specific debug exception. This is unfortunate as
> different debug exceptions have different entry handling requirements,
> and it would be better to handle these distinct requirements earlier.
> 
> The single stepping exception has the most constraints : it can be
> exploited to train branch predictors and it needs special handling at EL1
> for the Cortex-A76 erratum #1463225. We need to conserve all those
> mitigations.
> However, it does not write an address at FAR_EL1, as only hardware
> watchpoints do so.
> 
> The single-step handler does its own signaling if it needs to and only
> returns 0, so we can call it directly from `entry-common.c`.
> 
> Split the single stepping exception entry, adjust the function signature,
> keep the security mitigation and erratum handling.
> Further, as the EL0 and EL1 code paths are cleanly separated, we can split
> `do_softstep()` into `do_el0_softstep()` and `do_el1_softstep()` and
> call them directly from the relevant entry paths.
> We can also remove `NOKPROBE_SYMBOL` for the EL0 path, as it cannot
> lead to a kprobe recursion.
> 
> Move the call to `arm64_apply_bp_hardening()` to `entry-common.c` so that
> we can do it as early as possible, and only for the exceptions coming
> from EL0, where it is needed.
> This is safe to do as it is `noinstr`, as are all the functions it
> may call. `el0_ia()` and `el0_pc()` already call it this way.
> 
> When taking a soft-step exception from EL0, most of the single stepping
> handling is safely preemptible : the only possible handler is
> `uprobe_single_step_handler()`. It only operates on task-local data and
> properly checks its validity, then raises a Thread Information Flag,
> processed before returning to userspace in `do_notify_resume()`, which
> is already preemptible.
> However, the soft-step handler first calls `reinstall_suspended_bps()`
> to check if there is any hardware breakpoint or watchpoint pending
> or already stepped through.
> This cannot be preempted as it manipulates the hardware breakpoint and
> watchpoint registers.
> 
> Move the call to `try_step_suspended_breakpoints()` to `entry-common.c`
> and adjust the relevant comments.
> We can now safely unmask interrupts before handling the step itself,
> fixing a PREEMPT_RT issue where the handler could call a sleeping function
> with preemption disabled.
> 
> Signed-off-by: Ada Couprie Diaz <ada.coupriediaz at arm.com>
> Closes: https://lore.kernel.org/linux-arm-kernel/Z6YW_Kx4S2tmj2BP@uudg.org/
> Tested-by: Luis Claudio R. Goncalves <lgoncalv at redhat.com>
> ---
>  arch/arm64/include/asm/exception.h |  2 +
>  arch/arm64/kernel/debug-monitors.c | 77 +++++++++++-------------------
>  arch/arm64/kernel/entry-common.c   | 43 +++++++++++++++++
>  arch/arm64/kernel/hw_breakpoint.c  |  2 +-
>  4 files changed, 75 insertions(+), 49 deletions(-)

Reviewed-by: Will Deacon <will at kernel.org>

Will