[PATCH] riscv: uprobes: Avoid redundant icache flushes for the XOL slot

[Davidlohr Bueso]

ping?

On Fri, 16 Oct 2026, Davidlohr Bueso wrote:

>arch_uprobe_copy_ixol() unconditionally calls flush_icache_range(),
>which has no range form on riscv and is a flush_icache_all(). It
>runs once per single-step, as the generic code allocates and frees
>an out-of-line slot around each one.
>
>The allocator hands out the lowest free slot and the free path leaves
>the slot contents alone, so a hot probe keeps getting the same slot
>back holding the instruction it already wrote there. Rewriting it
>costs a broadcast for nothing.
>
>Return early when the slot already holds this instruction and its
>breakpoint, similar to commit bdf94836c22a ("arm64: uprobes: Optimize
>cache flushes for xol slot"). arm64 can compare against the source,
>but riscv appends an ebreak after the opcode, so compare the
>instruction and that breakpoint word instead.
>
>Measured with the selftests/bpf bench on a SiFive P550, with one
>producer pinned to a hart:
>
>                            before       after
>  trig-uprobe-nop           0.075 M/s    0.514 M/s    6.9x
>  trig-uprobe-push          0.079 M/s    0.425 M/s    5.4x
>  trig-uprobe-ret           0.075 M/s    0.514 M/s    6.9x
>  trig-uprobe-multi-nop     0.079 M/s    0.548 M/s    6.9x
>  trig-uprobe-multi-push    0.078 M/s    0.545 M/s    7.0x
>  trig-uprobe-multi-ret     0.081 M/s    0.549 M/s    6.8x
>
>The uretprobe variants gain 4.8x to 6.0x.
>
>Signed-off-by: Davidlohr Bueso <dave at stgolabs.net>
>---
> arch/riscv/kernel/probes/uprobes.c | 19 +++++++++++++++----
> 1 file changed, 15 insertions(+), 4 deletions(-)
>
>diff --git a/arch/riscv/kernel/probes/uprobes.c b/arch/riscv/kernel/probes/uprobes.c
>index eb177d0ce8ab..44e566f6b75b 100644
>--- a/arch/riscv/kernel/probes/uprobes.c
>+++ b/arch/riscv/kernel/probes/uprobes.c
>@@ -168,15 +168,26 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr,
> 	void *kaddr = kmap_local_page(page);
> 	void *dst = kaddr + (vaddr & ~PAGE_MASK);
> 	unsigned long start = (unsigned long)dst;
>+	unsigned long insn_len;
>+
>+	insn_len = vaddr ? GET_INSN_LENGTH(*(probe_opcode_t *)src) : len;
>+
>+	/*
>+	 * Slots are reused without being cleared, so a hot probe usually gets
>+	 * one back already holding this instruction and its breakpoint. Skip
>+	 * the rewrite and the flush then.
>+	 */
>+	if (!memcmp(dst, src, insn_len) &&
>+	    (!vaddr || *(uprobe_opcode_t *)(dst + insn_len) == __BUG_INSN_32))
>+		goto done;
>
> 	memcpy(dst, src, len);
>
> 	/* Add ebreak behind opcode to simulate singlestep */
>-	if (vaddr) {
>-		dst += GET_INSN_LENGTH(*(probe_opcode_t *)src);
>-		*(uprobe_opcode_t *)dst = __BUG_INSN_32;
>-	}
>+	if (vaddr)
>+		*(uprobe_opcode_t *)(dst + insn_len) = __BUG_INSN_32;
>
> 	flush_icache_range(start, start + len);
>+done:
> 	kunmap_local(kaddr);
> }
>-- 
>2.39.5
>