[PATCH v2] bpf, arm64: Fix racy plt target update in bpf_arch_text_poke()

[Xu Kuohai]

On 10/2/2026 8:12 PM, Matthew Wood wrote:
> When a long-jump trampoline is attached to or detached from a bpf prog,
> bpf_arch_text_poke() updates the plt target at the end of the prog by
> temporarily making the page writable:
> 
> 	set_memory_rw(page);
> 	WRITE_ONCE(plt->target, plt_target);
> 	set_memory_ro(page);
> 
> Since commit 1dad391daef1 ("bpf, arm64: use bpf_prog_pack for memory
> management"), bpf progs are allocated from the bpf prog pack, so
> unrelated progs and their plts commonly share a page. The sequence
> above is not serialized against other pokers, it runs before text_mutex
> is taken and callers only hold the lock of the trampoline being
> updated. Two CPUs attaching to or detaching from different target progs
> in the same page can therefore race:
> 
> 	CPU A                           CPU B
> 	set_memory_rw(page)
> 	                                set_memory_rw(page)
> 	                                WRITE_ONCE(plt_B->target, ...)
> 	                                set_memory_ro(page)
> 	WRITE_ONCE(plt_A->target, ...)  <- permission fault
> 
> This was hit on an arm64 server with 64K pages while attaching fentry
> programs to bpf progs:
> 
>    Unable to handle kernel write to read-only memory at virtual address ffff80008f46d5f8
>    ESR = 0x000000009600004f
>    FSC = 0x0f: level 3 permission fault
>    pte=00c00200ea5a0783
>    Internal error: Oops: 000000009600004f [#1]  SMP
>    pc : bpf_arch_text_poke+0x214/0x238
>    lr : bpf_arch_text_poke+0x200/0x238
>    Call trace:
>     bpf_arch_text_poke+0x214/0x238 (P)
>     __bpf_trampoline_link_prog+0x1c8/0x470
>     bpf_trampoline_link_prog+0x64/0x90
>     bpf_tracing_prog_attach+0x318/0x4a8
>     bpf_raw_tp_link_attach+0x104/0x258
>     bpf_raw_tracepoint_open+0x6c/0x90
>     __sys_bpf+0x134c/0x3e10
> 
> Rather than serializing the permission changes, stop changing page
> permissions altogether and write the plt target with
> aarch64_insn_write_literal_u64(). It writes through the text patching
> fixmap under patch_lock, the same way the rest of the prog pack is
> written, and performs a single-copy atomic 64-bit store. The plt target
> is naturally aligned (see build_plt()), so CPUs concurrently executing
> the plt still observe either the old or the new target. This is the
> same helper ftrace and static calls use to update 64-bit literals that
> are loaded concurrently.
> 
> Fixes: 1dad391daef1 ("bpf, arm64: use bpf_prog_pack for memory management")
> Signed-off-by: Matthew Wood <thepacketgeek at gmail.com>
> ---
> Changes in v2:
> - Replace page permissions change with call to aarch64_insn_write_literal_u64
> - Link to v1: https://lore.kernel.org/linux-arm-kernel/CADvopvZPO1gDMDhDOPcArjRJ0cPpSUZ5suMg-1kwRi+_-Xeitw@mail.gmail.com/
> ---
>   arch/arm64/net/bpf_jit_comp.c | 12 +++++++-----
>   1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index 475e70653454..e353403cc1c3 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -3339,13 +3339,15 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
>   		plt_target = (u64)&dummy_tramp;
> 
>   	if (plt_target) {
> -		/* non-zero plt_target indicates we're patching a bpf prog,
> -		 * which is read only.
> +		/*
> +		 * non-zero plt_target indicates we're patching a bpf prog,
> +		 * which is read only. The prog shares its page in the bpf
> +		 * prog pack with other progs, so write the aligned target
> +		 * through the text patching fixmap without flipping the
> +		 * page permissions to avoid racing with concurrent pokers.
>   		 */
> -		if (set_memory_rw(PAGE_MASK & ((uintptr_t)&plt->target), 1))
> +		if (aarch64_insn_write_literal_u64(&plt->target, plt_target))
>   			return -EFAULT;
> -		WRITE_ONCE(plt->target, plt_target);
> -		set_memory_ro(PAGE_MASK & ((uintptr_t)&plt->target), 1);
>   		/* since plt target points to either the new trampoline
>   		 * or dummy_tramp, even if another CPU reads the old plt
>   		 * target value before fetching the bl instruction to plt,


I think it's better to combine the two nested if statements into one, and merge
the two comments into a single comment above the new if statement.

Otherwise lgtm.

> --
> 2.53.0-Meta