Hard lock-up on RK3328 with no kernel console output
Craig McQueen
craig at mcqueen.au
Wed Jul 8 20:15:12 PDT 2026
On 25-June-2026 I wrote:
> I have a Rockchip RK3328 SOM based embedded Linux system. I'm using Yocto scarthgap with kernel v6.6.123.
>
> I'm seeing complete system lock-ups occur roughly every 1 to 72 hours, with no diagnostic information shown in the kernel serial console output. It is abrupt and looks like it's a total shut-down. But it does get rebooted by the RK3328 HW watchdog. By reading the CRU_GLB_CNT_TH register after reboot, I confirm that the watchdog triggered the reset. If I don't enable the RK3328 HW watchdog, it stays locked-up indefinitely until an external reset/power-cycle.
>
> I am struggling to diagnose/debug this issue because I have been unable to get any kernel log output on the serial console. Normally I would expect to see a kernel message about a hard lock-up, or kernel oops, etc. I have enabled (among other things):
>
> CONFIG_HARDLOCKUP_DETECTOR=y
> CONFIG_HARDLOCKUP_DETECTOR_BUDDY=y
> CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y
>
> I have also tried using the kernel PSTORE, PSTORE_RAM, PSTORE_CONSOLE feature, in conjunction with U-Boot `pstore` command. After a lock-up and reboot, I was able to read the kernel dmesg log with U-Boot `pstore display` command. But it also didn't show any lock-up diagnostics.
>
> In this abrupt lock-up scenario, I would appreciate any advice on:
>
> * How to get any diagnostics out of the system (eg any other kernel config changes to try).
> * What sort of known fault mechanisms could lead to such a hard lock-up.
>
> I may try using a SWD/JTAG debugger, once I can get a J-Link debugger.
Once I set up the debugger, I needed to set kernel command parameter `cpuidle.off=1` for the debugger to be able to reliably access the processor cores. But once I did that, the lock-up problem disappeared.
So it looks as though my lock-up bug is related to CPU idle management.
--
Craig McQueen
More information about the Linux-rockchip
mailing list