[PATCH v2] arm64/irqflags: __always_inline the arch_local_irq_*() helpers

[Breno Leitao]

On Mon, Apr 27, 2026 at 02:08:59PM +0100, Mark Rutland wrote:
> On Mon, Apr 27, 2026 at 01:26:18PM +0100, Catalin Marinas wrote:
> > On Tue, Apr 21, 2026 at 08:58:57AM -0700, Breno Leitao wrote:
> > > Force-inline all of the arch_local_irq_*() wrappers so they cannot be
> > > emitted out-of-line:
> > > 
> > >   - arch_local_irq_enable()
> > >   - arch_local_irq_disable()
> > >   - arch_local_save_flags()
> > >   - arch_irqs_disabled_flags()
> > >   - arch_irqs_disabled()
> > >   - arch_local_irq_save()
> > >   - arch_local_irq_restore()
> > 
> > I'll queue this, thanks!
> > 
> > I think we should also do local_daif_{mask,restore,inherit} as they seem
> > to be called from noinstr locations in entry-common.c.
> 
> I agree we probably should mark those as __always_inline, but I beleive
> they're safe as-is.

Returning to this topic, I've also observed arch_counter_get_cntpct()
appearing in backtraces during lock spinning, which obscures the actual
code location and makes debugging less straightforward.

This scenario is common at Meta when a remote function call targets a
stuck CPU, resulting in an infinite loop, and dying at arch_counter_get_cntpc().

Is it worth making arch_counter_get_cntpc() always inline as well?


 [     T18] Sending NMI from CPU 1 to CPUs 18:
 [     C18] NMI backtrace for cpu 18
 [     C18] CPU: 18 UID: 0 PID: 7467 Comm: dynoKernelMon Kdump: loaded Not tainted 6.16.1-0_fbk3_0_gd6c130b80483 #1 NONE
 [     C18] pstate: 63401009 (nZCv daif +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
 [     C18] pc : arch_counter_get_cntpct+0x14/0x18
 [     C18] lr : ktime_get_mono_fast_ns+0x5c/0x1a8
 [     C18] sp : ffff8000d43afa30
 [     C18] x29: ffff8000d43afa30 x28: 00000000000f4240 x27: 0000000000000000
 [     C18] x26: 0000028ac8fe4369 x25: 0000000000001388 x24: 000000000054fb54
 [     C18] x23: ffff800082fbd000 x22: 0000028f22c480dd x21: ffff800081b60000
 [     C18] x20: ffff800080c71058 x19: ffff800082fb8848 x18: 0000000000000018
 [     C18] x17: 0000000000000058 x16: 00000000ffffffff x15: 00000000ffff0000
 [     C18] x14: 0000000000000002 x13: 0000000000000003 x12: 000000000000ffd8
 [     C18] x11: ffff8000d43affd8 x10: 0000000000000038 x9 : 0000000000000000
 [     C18] x8 : ffff8000d43afa30 x7 : ff7fff7f7f7f7f7f x6 : 000000000000000a
 [     C18] x5 : ffff8000832772ba x4 : 0000000000000000 x3 : 0000000000000000
 [     C18] x2 : 0000000000000000 x1 : 000000000014b4c0 x0 : 000002a87573b7a0
 [     C18] Call trace:
 [     C18]  arch_counter_get_cntpct+0x14/0x18 (P)
 [     C18]  smp_call_function_single+0x2d0/0xd90
 [     C18]  event_function_call+0x80/0x2e0
 [     C18]  _perf_event_disable+0x5c/0xf0
 [     C18]  perf_ioctl+0x170/0x890
 [     C18]  __arm64_sys_ioctl+0xbb0/0xd28
 [     C18]  invoke_syscall+0x4c/0xd0
 [     C18]  do_el0_svc+0x80/0xb8
 [     C18]  el0_svc+0x30/0xf0
 [     C18]  el0t_64_sync_handler+0x70/0x100
 [     C18]  el0t_64_sync+0x17c/0x180


Thanks,
--breno