[PATCH v6 03/12] PCI: liveupdate: Track incoming preserved PCI devices
David Matlack
dmatlack at google.com
Mon Jun 8 13:57:45 PDT 2026
On 2026-06-06 10:08 AM, Pranjal Shrivastava wrote:
> On Fri, May 22, 2026 at 08:24:01PM +0000, David Matlack wrote:
> > During PCI enumeration, the previous kernel might have passed state about
> > devices that were preserved across kexec. The PCI core needs to fetch
> > this state to identify which devices are "incoming" and require special
> > handling.
> >
> > Add pci_liveupdate_setup_device() which is called during device setup
> > to fetch the serialized state (struct pci_ser) from the Live Update
> > Orchestrator. The first time this happens, pci_flb_retrieve() will run
> > and convert the array of pci_dev_ser structs into an xarray so that it
> > can be looked up efficiently.
> >
> > If a device is found in the xarray, the PCI core stores a pointer to its
> > state in dev->liveupdate_incoming and holds a reference to the incoming
> > FLB until pci_liveupdate_finish() is called by the driver.
> >
> > This ensures proper lifecycle management for incoming preserved devices
> > and allows the PCI core and drivers to apply specific Live Update
> > logic to them in subsequent commits.
> >
> > Drivers can check if a device is an incoming preserved device (e.g.
> > during probe) by calling pci_liveupdate_is_incoming().
> >
> > CONFIG_64BIT is now required to enable CONFIG_PCI_LIVEUPDATE so that the
> > domain and bdf can be guaranteed to fit in an unsigned long and be used
> > as the xarray key.
> >
> > Signed-off-by: David Matlack <dmatlack at google.com>
> > ---
> > MAINTAINERS | 1 +
> > drivers/pci/Kconfig | 2 +-
> > drivers/pci/liveupdate.c | 230 ++++++++++++++++++++++++++++++++-
> > drivers/pci/liveupdate.h | 5 +
> > drivers/pci/probe.c | 3 +
> > include/linux/pci_liveupdate.h | 13 ++
> > 6 files changed, 251 insertions(+), 3 deletions(-)
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 6c618830cf61..0e262c0ceb43 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -20537,6 +20537,7 @@ L: linux-pci at vger.kernel.org
> > S: Maintained
> > T: git git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux.git
> > F: drivers/pci/liveupdate.c
> > +F: drivers/pci/liveupdate.h
> > F: include/linux/kho/abi/pci.h
> > F: include/linux/pci_liveupdate.h
> >
> > diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> > index 10c9b65aa242..e68ae5c172d4 100644
> > --- a/drivers/pci/Kconfig
> > +++ b/drivers/pci/Kconfig
> > @@ -330,7 +330,7 @@ config VGA_ARB_MAX_GPUS
> >
> > config PCI_LIVEUPDATE
> > bool "PCI Live Update Support"
> > - depends on PCI && LIVEUPDATE
> > + depends on PCI && LIVEUPDATE && 64BIT
>
> I see that the static assertions in Patch 1 work because of the 64BIT
> enforcement here. In that case, should we have the assertions check u64?
The static asserts have nothing to do with the 64BIT enforcement here.
The static asserts just verify that the array elements in struct pci_ser
are naturally aligned (unsigned long) so they can be accessed
efficiently. The requirement here for CONFIG_64BIT is for the xarray
key.
Theoretically if we got the xarray to work with 32-bit architectures
then we could drop the CONFIG_64BIT requirement here.
>
> > help
> > Enable PCI core support for preserving PCI devices across Live
> > Update. This, in combination with support in a device's driver,
> >
>
> [...]
>
> > static int pci_flb_retrieve(struct liveupdate_flb_op_args *args)
> > {
> > - args->obj = phys_to_virt(args->data);
> > + struct pci_ser *ser = phys_to_virt(args->data);
> > + struct pci_flb_incoming *incoming;
> > + int ret = -ENOMEM;
> > + u32 i;
> > +
> > + incoming = kmalloc_obj(*incoming);
> > + if (!incoming)
> > + goto err_restore_free;
> > +
> > + incoming->ser = ser;
> > + xa_init(&incoming->xa);
> > +
> > + for (i = 0; i < incoming->ser->max_nr_devices; i++) {
> > + struct pci_dev_ser *dev_ser = &incoming->ser->devices[i];
> > + unsigned long key;
> > +
> > + if (!dev_ser->refcount)
> > + continue;
> > +
> > + key = pci_ser_xa_key(dev_ser->domain, dev_ser->bdf);
> > + ret = xa_insert(&incoming->xa, key, dev_ser, GFP_KERNEL);
> > + if (ret)
> > + goto err_xa_destroy;
> > + }
> > +
> > + args->obj = incoming;
> > return 0;
> > +
> > +err_xa_destroy:
> > + xa_destroy(&incoming->xa);
> > + kfree(incoming);
> > +err_restore_free:
> > + kho_restore_free(ser);
>
> I tend to partly agree with Sashiko[1] here.. it raises a policy-hole.
> We may need a policy here, the options I have in mind are:
>
> 1. Retrieve shall ONLY be tried once, if it fails (like -ENOMEM in the
> xArray alloc), it's a liveupdate failure. We can't retry liveupdate.
>
> 2. Retrying retrieve is allowed.
>
> The only downside with option 1 is, the user may want flexibility due to
> certain subsystems OR may choose NOT to use the proposed LUOd and instead
> have its own user-space component which might try funny things or have a
> different use-case.
>
> In such a situation, the system may have transiently run out of memory
> during the kexec transition (for e.g. a subsystem uses GFP_ATOMIC to
> allocate memory and temporarily runs out of the atomic pool). [Note we
> removed it in IOMMU v1 [2] but subsystems may have a use-case for it]
>
> If the kernel frees the KHO page on the first failure, it removes any
> chance of recovery. :/
>
> Thus, it might make sense to let the user decide if it wants to fail the
> liveupdate or retry again based on the failure type / source?
The plan is to have LUO enforce that retrieve() is only called once:
https://lore.kernel.org/kexec/20260528174140.1921129-3-dmatlack@google.com/
Supporting retry gets complicated since there's many different places
where retrieve() could have failed.
>
> [...]
>
> The changes LGTM, except for policy-based, kho_restore_free discussion.
>
> Reviewed-by: Pranjal Shrivastava <praan at google.com>
>
> Thanks,
> Praan
>
> [1] https://lore.kernel.org/all/20260522211333.D56A21F000E9@smtp.kernel.org/
> [2] https://lore.kernel.org/all/20260203220948.2176157-2-skhawaja@google.com/
More information about the kexec
mailing list