[PATCH v2] rcu: Use an intermediate irq_work to start process_srcu()

Mon Jul 6 04:32:39 PDT 2026

On 6/24/26 10:55, Emil Renner Berthing wrote:
> Quoting Samuel Holland (2026-06-23 17:53:19)
>> On 2026-06-23 4:34 AM, Emil Renner Berthing wrote:
>>> Quoting Boqun Feng (2026-06-22 17:38:10)
>>>> On Sat, Jun 20, 2026 at 05:25:25AM -0400, Emil Renner Berthing wrote:
>>>>> Quoting Boqun Feng (2026-06-18 17:30:10)
>>>>>> On Thu, Jun 18, 2026 at 08:11:36AM -0500, Emil Renner Berthing wrote:
>>>>>>> Quoting Boqun Feng (2026-03-20 23:29:16)
>>>>>>>> Since commit c27cea4416a3 ("rcu: Re-implement RCU Tasks Trace in terms
>>>>>>>> of SRCU-fast") we switched to SRCU in BPF. However as BPF instrument can
>>>>>>>> happen basically everywhere (including where a scheduler lock is held),
>>>>>>>> call_srcu() now needs to avoid acquiring scheduler lock because
>>>>>>>> otherwise it could cause deadlock [1]. Fix this by following what the
>>>>>>>> previous RCU Tasks Trace did: using an irq_work to delay the queuing of
>>>>>>>> the work to start process_srcu().
>>>>>>>>
>>>>>>>> [boqun: Apply Joel's feedback]
>>>>>>>> [boqun: Apply Andrea's test feedback]
>>>>>>>>
>>>>>>>> Reported-by: Andrea Righi <arighi at nvidia.com>
>>>>>>>> Closes: https://lore.kernel.org/all/abjzvz_tL_siV17s@gpd4/
>>>>>>>> Fixes: commit c27cea4416a3 ("rcu: Re-implement RCU Tasks Trace in terms of SRCU-fast")
>>>>>>>> Link: https://lore.kernel.org/rcu/3c4c5a29-24ea-492d-aeee-e0d9605b4183@nvidia.com/ [1]
>>>>>>>> Suggested-by: Zqiang <qiang.zhang at linux.dev>
>>>>>>>> Tested-by: Andrea Righi <arighi at nvidia.com>
>>>>>>>> Signed-off-by: Boqun Feng <boqun at kernel.org>
>>>>>>>
>>>>>>> Thanks for the patch, which is now upstream as
>>>>>>> 7c405fb3279b ("rcu: Use an intermediate irq_work to start process_srcu()")
>>>>>>>
>>>>>>> Unfortunately it seems to halt booting on the Allwinner D1 single-core RISC-V
>>>>>>> SoC both on the 7.0 and 7.1 kernels:
>>>>>>>
>>>>>>
>>>>>> By "single-core", is it only one CPU in that system?
>>>>>>
>>>>>>> https://esmil.dk/d1-7.0.txt
>>>>>>> https://esmil.dk/d1-7.1.txt
>>>>>>>
>>>>>
>>>>> Yes, the SoC only has one CPU core capable of running Linux at least.
>>>>>
>>>>>> From the callstack,
>>>>>>
>>>>>> [  243.002818] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>>>>> [  243.010681] task:kworker/0:0     state:D stack:0     pid:9     tgid:9     ppid:2      task_flags:0x4208060 flags:0x00000000
>>>>>> [  243.021858] Workqueue: device_link_wq device_link_release_fn
>>>>>> [  243.027587] Call Trace:
>>>>>> [  243.030079] [<ffffffff80745b66>] __schedule+0x232/0x530
>>>>>> [  243.035365] [<ffffffff80745e82>] schedule+0x1e/0x94
>>>>>> [  243.040298] [<ffffffff8074aa7a>] schedule_timeout+0x6e/0xa8
>>>>>> [  243.045929] [<ffffffff8004d6a0>] wakeup_preempt+0x8c/0x98
>>>>>> [  243.051380] [<ffffffff80746aee>] wait_for_completion+0x3a/0xb0
>>>>>> [  243.057267] [<ffffffff80087226>] __synchronize_srcu.part.0+0x4e/0x60
>>>>>> [  243.063712] [<ffffffff80084414>] rcu_tasks_get_gp_data+0xc/0x10
>>>>>> [  243.069699] [<ffffffff80490898>] device_link_release_fn+0x14/0x80
>>>>>> [  243.075850] [<ffffffff8003dbd0>] process_one_work+0xf8/0x1c8
>>>>>> [  243.081583] [<ffffffff8003e282>] worker_thread+0x11a/0x254
>>>>>> [  243.087127] [<ffffffff8003e164>] rescuer_thread+0x3a0/0x3a4
>>>>>> [  243.092755] [<ffffffff8003e164>] rescuer_thread+0x3a0/0x3a4
>>>>>> [  243.098381] [<ffffffff8004499e>] kthread+0xbe/0xe4
>>>>>> [  243.103231] [<ffffffff8001349e>] ret_from_fork_kernel+0x6/0xd0
>>>>>> [  243.109127] [<ffffffff800517a8>] schedule_tail+0x8/0xac
>>>>>> [  243.114410] [<ffffffff800448dc>] kthreads_online_cpu+0x0/0x4
>>>>>> [  243.120121] [<ffffffff8074be0a>] ret_from_fork_kernel_asm+0x12/0x18
>>>>>>
>>>>>> I will guess for some reasons the irq_work was missed or the IPI never
>>>>>> happened. Is it possible for you to enable trace points ipi_send_cpu and
>>>>>> ipi_entry to get more information?
>>>>>
>>>>> I have to admit I've not used tracepoints before, but it seems like only
>>>>> ipi_send_cpu and ipi_send_cpumask is available on RISC-V.
>>>>> I've now added trace_event=ipi:* and tp_printk to the command line, and this
>>>>
>>>> Thank you! Yeah, this is what I meant ;-)
>>>>
>>>>> is what I get:
>>>>> https://esmil.dk/d1-7.0-2.txt
>>>>>
>>>>
>>>>
>>>> [    0.071132] rcu:     Max phase no-delay instances is 1000.
>>>> [    0.077837] ipi_send_cpu: cpu=0 callsite=irq_work_queue+0x22/0x4c callback=srcu_irq_work+0x0/0x58
>>>>
>>>> Is this the last line you can see from your (serial?) console, or we are
>>>> missing a few more lines? Maybe it'll should the same hung task
>>>> messages?
>>>
>>> Yes, sorry for not being clear about that. This is the exact same binary booted
>>> without the two extra command line parametrs:
>>> https://esmil.dk/d1-7.0-3.txt
>>>
>>> But with the two parameters added the board reboots before getting to the hung
>>> task messages. The bootloader enables the watchdog, so I think what happens is
>>> that the kernel halts earlier with the tracepoints enabled, and doesn't get to
>>> load the watchdog driver to keep the machine from rebooting.
>>>
>>> The same happens with just trace_event=ipi:* added, just without the last
>>> message above.
>>>
>>>>
>>>> The ipi_send_cpu message means RCU was able to send an IPI, now if there
>>>> is no ipi_handler between the send and the hung task, that would mean
>>>> the IPI was missed.
>>
>> Assuming that your OpenSBI version is exactly v1.3 per the boot log, and not
>> "v1.3 + patches", then you would indeed not be receiving IPIs. You need at least
>> this commit: https://github.com/riscv-software-src/opensbi/commit/446fa65eb579
>> (included in v1.4) or alternatively to add the CLINT to the M-mode devicetree.
> 
> So then we're in a situation similar to [1], where upgrading the kernel from
> 6.19 to 7.0 requires you to update the firmware first.

Which normally should never happen to ensure users feel save to update
the kernel. But I wonder if that is relevant, as Indrek Kruusa reported
problems with that board, too -- and there is a fix that fixes it:
https://lore.kernel.org/linux-riscv/20260624220434.4183732-1-felixonmars@archlinux.org/

Does that maybe fix your problem, too (even if it was bisected to a
different commit, it might be related somehow, that's why I'm asking)?

Ciao, Thorsten

> To me and from the thread
> that seems like a really obvious regression. So the question is, how can we make
> the current kernel work like < 7.0 kernels worked?
> 
> [1]: https://lore.kernel.org/all/87jzgjnh9z.ffs@tglx/#t