[PATCH v4 2/2] Drivers: hv: Run VMBus kexec cleanup from syscore shutdown

Shradha Gupta shradhagupta at linux.microsoft.com
Wed Oct 28 23:03:30 PDT 2026


On Mon, Oct 26, 2026 at 03:52:27PM +0530, Mukesh Pilaniya wrote:
> 
> 
> On 09/10/26 3:52 pm, Shradha Gupta wrote:
> > ARM64 Hyper-V guests do not run the VMBus kexec cleanup before
> > secondary CPUs are stopped. Consequently, the old VMBus connection
> > and SynIC state remain active when the kexec kernel starts.
> > 
> > Run the VMBus kexec handler from the existing VMBus syscore shutdown
> > callback. syscore_shutdown() runs after device shutdown and before
> > machine shutdown, while all CPUs needed by the SynIC CPU hotplug
> > teardown are still available. Guard the callback with
> > kexec_is_in_progress() so normal restart, halt, and poweroff paths are
> > unchanged.
> > 
> > Use the same path on x86 and remove the architecture-specific handler
> > pointer and the weak setup/remove interfaces. VMBus teardown now
> > precedes hv_stimer_global_cleanup() on x86.
> > 
> > The new x86 ordering relies on commit be0cfab740e5 ("clocksource:
> > hyper-v: Remove support for stimer interrupts in message mode"), which
> > removed the synthetic timer's dependency on VMBus message delivery.
> > Older kernels that retain message-mode stimer interrupts must preserve
> > the existing x86 ordering and therefore require an ARM64-specific
> > backport.
> > 
> > Document the VMBus cleanup path and its position in shutdown ordering.
> > 
> > Fixes: 9bbb888824e3 ("arm64: hyperv: Initialize hypervisor on boot")
> > Link: https://lore.kernel.org/all/20260814093133.3191250-1-shradhagupta@linux.microsoft.com/
> > Suggested-by: Catalin Marinas <catalin.marinas at arm.com>
> > Suggested-by: Michael Kelley <mhklinux at outlook.com>
> > Signed-off-by: Shradha Gupta <shradhagupta at linux.microsoft.com>
> > Reviewed-by: Michael Kelley <mhklinux at outlook.com>
> > Reviewed-by: Bradley Morgan <brads at mainlining.org>
> > Reviewed-by: Naman Jain <namjain at linux.microsoft.com>
> > ---
> >  Documentation/virt/hyperv/overview.rst | 12 +++++++-----
> >  arch/x86/kernel/cpu/mshyperv.c         | 27 ++++++---------------------
> >  drivers/hv/hv_common.c                 | 10 ----------
> >  drivers/hv/vmbus_drv.c                 | 22 ++++++++++++++--------
> >  include/asm-generic/mshyperv.h         |  2 --
> >  5 files changed, 27 insertions(+), 46 deletions(-)
> > 
> > diff --git a/Documentation/virt/hyperv/overview.rst b/Documentation/virt/hyperv/overview.rst
> > index 77408a89d1a4..34a53ba7a122 100644
> > --- a/Documentation/virt/hyperv/overview.rst
> > +++ b/Documentation/virt/hyperv/overview.rst
> > @@ -114,11 +114,13 @@ a shared page or remove an overlay page after the new kernel is
> >  using the page for a different purpose, corrupting the new kernel.
> >  Hyper-V does not provide a single "set everything" operation to
> >  guest VMs, so Linux code must individually revoke all sharing before
> > -doing kexec.   See hv_kexec_handler() and hv_crash_handler().  But
> > -the crash/panic path still has holes in cleanup because some shared
> > -pages are set using per-CPU synthetic registers and there's no
> > -mechanism to revoke the shared pages for CPUs other than the CPU
> > -running the panic path.
> > +doing kexec.  For a normal kexec reboot, the VMBus driver calls
> > +hv_kexec_handler() from its syscore shutdown callback, after device
> > +shutdown and before the architecture-specific machine shutdown.  The
> > +crash/panic path uses hv_crash_handler(), but still has holes in
> > +cleanup because some shared pages are set using per-CPU synthetic
> > +registers and there is no mechanism to revoke the shared pages for
> > +CPUs other than the CPU running the panic path.
> >  
> >  CPU Management
> >  --------------
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index b4af7c0a70ac..fec1c37dafbf 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -148,7 +148,6 @@ EXPORT_SYMBOL_GPL(hv_set_msr);
> >  static void (*mshv_handler)(void);
> >  static void (*vmbus_handler)(void);
> >  static void (*hv_stimer0_handler)(void);
> > -static void (*hv_kexec_handler)(void);
> >  static void (*hv_crash_handler)(struct pt_regs *regs);
> >  
> >  DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback)
> > @@ -215,16 +214,6 @@ void hv_remove_stimer0_handler(void)
> >  	hv_stimer0_handler = NULL;
> >  }
> >  
> > -void hv_setup_kexec_handler(void (*handler)(void))
> > -{
> > -	hv_kexec_handler = handler;
> > -}
> > -
> > -void hv_remove_kexec_handler(void)
> > -{
> > -	hv_kexec_handler = NULL;
> > -}
> > -
> >  void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs))
> >  {
> >  	hv_crash_handler = handler;
> > @@ -240,17 +229,13 @@ static void hv_machine_shutdown(void)
> >  {
> >  	if (kexec_in_progress) {
> >  		hv_stimer_global_cleanup();
> > -
> > -		if (hv_kexec_handler)
> > -			hv_kexec_handler();
> > -	}
> > -
> > -	/*
> > -	 * Call hv_cpu_die() on all the CPUs, otherwise later the hypervisor
> > -	 * corrupts the old VP Assist Pages and can crash the kexec kernel.
> > -	 */
> > -	if (kexec_in_progress)
> > +		/*
> > +		 * Call hv_cpu_die() on all the CPUs, otherwise later the
> > +		 * hypervisor corrupts the old VP Assist Pages and can crash
> > +		 * the kexec kernel.
> > +		 */
> >  		cpuhp_remove_state(CPUHP_AP_HYPERV_ONLINE);
> > +	}
> >  
> After this patch, the x86 kexec path ends up with two
> cpuhp_remove_state() calls, one inside hv_kexec_handler() (now fired
> from syscore shutdown) on hyperv_cpuhp_online, and one in
> hv_machine_shutdown() on CPUHP_AP_HYPERV_ONLINE.
> 
> Could you confirm these refer to distinct CPUHP slots? If they resolve
> to the same slot, we'd be calling cpuhp_remove_state() twice on it,
> which would be a bug.
>

Hi Mukesh,

Yes, these refer to distinct CPUHP states.

hyperv_cpuhp_online holds the dynamic state allocated in vmbus_drv.c,
while CPUHP_AP_HYPERV_ONLINE is a seperate fixed state registered
in hv_init.c

Both removals already existed in the x86 kexec path. Previously,
hv_machine_shutdown() called hv_kexec_handler() removing the
VMBus/SynIC state, and then removed CPUHP_AP_HYPERV_ONLINE.


This patch moves the VMBus/SynIC cleanup into the common syscore
shutdown callback for both ARM64 and x86. The removal of the fixed
x86 Hyper-V state remains in hv_machine_shutdown(). Neither state
is removed twice.

Thanks,
Shradha
 
> >  	/* The function calls stop_other_cpus(). */
> >  	native_machine_shutdown();
> > diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> > index 31256cb22b39..bb290a5eaf1d 100644
> > --- a/drivers/hv/hv_common.c
> > +++ b/drivers/hv/hv_common.c
> > @@ -681,16 +681,6 @@ void __weak hv_setup_mshv_handler(void (*handler)(void))
> >  }
> >  EXPORT_SYMBOL_GPL(hv_setup_mshv_handler);
> >  
> > -void __weak hv_setup_kexec_handler(void (*handler)(void))
> > -{
> > -}
> > -EXPORT_SYMBOL_GPL(hv_setup_kexec_handler);
> > -
> > -void __weak hv_remove_kexec_handler(void)
> > -{
> > -}
> > -EXPORT_SYMBOL_GPL(hv_remove_kexec_handler);
> > -
> >  void __weak hv_setup_crash_handler(void (*handler)(struct pt_regs *regs))
> >  {
> >  }
> > diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> > index 5ebdbe24b5a1..3532ea779d34 100644
> > --- a/drivers/hv/vmbus_drv.c
> > +++ b/drivers/hv/vmbus_drv.c
> > @@ -33,6 +33,7 @@
> >  #include <linux/sysfb.h>
> >  #include <linux/efi.h>
> >  #include <linux/kernel.h>
> > +#include <linux/kexec.h>
> >  #include <linux/syscore_ops.h>
> >  #include <linux/dma-map-ops.h>
> >  #include <linux/pci.h>
> > @@ -2904,6 +2905,12 @@ static void hv_kexec_handler(void)
> >  	cpuhp_remove_state(hyperv_cpuhp_online);
> >  };
> >  
> > +static void hv_vmbus_syscore_shutdown(void *data)
> > +{
> > +	if (kexec_is_in_progress())
> > +		hv_kexec_handler();
> > +}
> > +
> >  static void hv_crash_handler(struct pt_regs *regs)
> >  {
> >  	int cpu;
> > @@ -2958,14 +2965,15 @@ static void hv_synic_resume(void *data)
> >  	 */
> >  }
> >  
> > -/* The callbacks run only on CPU0, with irqs_disabled. */
> > -static const struct syscore_ops hv_synic_syscore_ops = {
> > +/* The suspend and resume callbacks run only on CPU0, with irqs_disabled. */
> > +static const struct syscore_ops hv_vmbus_syscore_ops = {
> >  	.suspend = hv_synic_suspend,
> >  	.resume = hv_synic_resume,
> > +	.shutdown = hv_vmbus_syscore_shutdown,
> >  };
> >  
> > -static struct syscore hv_synic_syscore = {
> > -	.ops = &hv_synic_syscore_ops,
> > +static struct syscore hv_vmbus_syscore = {
> > +	.ops = &hv_vmbus_syscore_ops,
> >  };
> >  
> >  static int __init hv_acpi_init(void)
> > @@ -3014,10 +3022,9 @@ static int __init hv_acpi_init(void)
> >  	if (ret)
> >  		goto cleanup;
> >  
> > -	hv_setup_kexec_handler(hv_kexec_handler);
> >  	hv_setup_crash_handler(hv_crash_handler);
> >  
> > -	register_syscore(&hv_synic_syscore);
> > +	register_syscore(&hv_vmbus_syscore);
> >  
> >  	return 0;
> >  
> > @@ -3039,9 +3046,8 @@ static void __exit vmbus_exit(void)
> >  		 */
> >  		return;
> >  
> > -	unregister_syscore(&hv_synic_syscore);
> > +	unregister_syscore(&hv_vmbus_syscore);
> >  
> > -	hv_remove_kexec_handler();
> >  	hv_remove_crash_handler();
> >  	vmbus_connection.conn_state = DISCONNECTED;
> >  	hv_stimer_global_cleanup();
> > diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
> > index bf601d67cecb..98fe5c0f54ea 100644
> > --- a/include/asm-generic/mshyperv.h
> > +++ b/include/asm-generic/mshyperv.h
> > @@ -183,8 +183,6 @@ void hv_remove_vmbus_handler(void);
> >  void hv_setup_stimer0_handler(void (*handler)(void));
> >  void hv_remove_stimer0_handler(void);
> >  
> > -void hv_setup_kexec_handler(void (*handler)(void));
> > -void hv_remove_kexec_handler(void);
> >  void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs));
> >  void hv_remove_crash_handler(void);
> >  void hv_setup_mshv_handler(void (*handler)(void));
> 
> -- 
> Regards,
> Mukesh Pilaniya
> 
> 



More information about the kexec mailing list