[PATCH v3 12/12] arm64: Add HOTPLUG_PARALLEL support for secondary CPUs

[Jinjie Ruan]

Support for parallel secondary CPU bringup is already utilized by x86,
MIPS and RISC-V. This patch brings this capability to the arm64
architecture.

To fully enable HOTPLUG_PARALLEL, this patch implements an arm64-specific
arch_cpuhp_init_parallel_bringup() handler.

In parallel bringup, early `set_cpu_present(cpu, 0)` inside
cpu_die_early() removes the secondary CPU prematurely, causing the primary
CPU's second-stage cpuhp_bringup_mask() sweep to skip it and drop
failure logs.

Remove this early unregistration from the secondary CPU, deferring the
set_cpu_present(cpu, 0) call to the primary CPU's cleanup path to ensure
robust parallel boot timeout detection.

Tested natively with ATF on QEMU arm64 virt machine with 64 cores
and also tested with KVM arm64 guest with 128 vCPUs.

Tested-by: Michael Kelley <mhklinux at outlook.com>
Signed-off-by: Jinjie Ruan <ruanjinjie at huawei.com>
---
 arch/arm64/Kconfig      |  2 +-
 arch/arm64/kernel/smp.c | 12 ++++++++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 24496e9967a8..a9d8030e7492 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -231,7 +231,7 @@ config ARM64
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_GENERIC_VDSO
-	select HOTPLUG_SPLIT_STARTUP if SMP
+	select HOTPLUG_PARALLEL if SMP
 	select HOTPLUG_SMT if HOTPLUG_CPU
 	select IRQ_DOMAIN
 	select IRQ_FORCED_THREADING
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 98ddbe50081d..a973b2d3bab1 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -93,6 +93,15 @@ static inline int op_cpu_kill(unsigned int cpu)
 }
 #endif
 
+extern const struct cpu_operations cpu_psci_ops;
+
+/* Establish whether parallel bringup can be supported. */
+bool __init arch_cpuhp_init_parallel_bringup(void)
+{
+	const struct cpu_operations *ops = get_secondary_cpu_ops();
+
+	return ops == &cpu_psci_ops;
+}
 
 /*
  * Boot a secondary CPU, and assign it the specified idle task.
@@ -137,6 +146,7 @@ void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu, bool is_alive)
 	 * We failed to synchronise with the CPU, so check if it left us
 	 * any breadcrumbs.
 	 */
+	set_cpu_present(cpu, 0);
 	cpu_boot_data[cpu].task = NULL;
 	status = READ_ONCE(cpu_boot_data[cpu].status);
 	if (status == CPU_MMU_OFF)
@@ -416,8 +426,6 @@ void __noreturn cpu_die_early(void)
 
 	pr_crit("CPU%d: will not boot\n", cpu);
 
-	/* Mark this CPU absent */
-	set_cpu_present(cpu, 0);
 	rcutree_report_cpu_dead();
 
 	if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
-- 
2.34.1