[PATCH 2/9] kho: Make bootmem reservations visible to fleet monitoring

Chris Down chris at chrisdown.name
Tue Nov 10 23:53:01 PST 2026


We want to use live update at Meta to upgrade hypervisors whose VMs
can't be migrated away first. But this poses a problem: whether we can
enable KHO on a class of host depends on how much memory it leaves
available for unmovable allocations. In an early test, a 64 GiB host
with hugetlb_cma=6G set aside about 30 GiB of bootmem, which the running
kernel can only use for movable allocations.

The reservation varies with the kernel, memory size, NUMA layout and
command line. Our fleet collector needs the actual size from each host
so we can decide where to enable KHO and catch a kernel release that
takes those hosts over their bootmem budget.

Today the collector would have to sum kho/out/bootmem_len in debugfs,
which isn't a userspace ABI. In /proc/meminfo, free bootmem contributes
to CmaFree, but the reservation doesn't contribute to CmaTotal. The cma
count in /proc/zoneinfo includes bootmem after a cold boot but not after
a KHO kexec. On an 8 GiB QEMU guest booted with
kho_bootmem=32M,1536M,512M:

                CmaTotal  CmaFree     zoneinfo cma (all zones)
    cold boot   0 kB      2127872 kB  2129920 kB
    KHO kexec   0 kB       881336 kB  0 kB

Even if bootmem becomes a CMA area, CmaTotal will still include other
areas such as hugetlb CMA, so inferring the bootmem size is still
error-prone.

Let's report the total bootmem reservation in bytes at
/sys/kernel/kexec/kho_bootmem_size.

Signed-off-by: Chris Down <chris at chrisdown.name>
---
 .../ABI/testing/sysfs-kernel-kexec-kdump          |  8 ++++++++
 Documentation/admin-guide/mm/kho.rst              |  9 +++++++++
 include/linux/kexec_handover.h                    |  1 +
 kernel/kexec_core.c                               | 13 +++++++++++++
 kernel/liveupdate/kexec_handover.c                | 15 +++++++++++++++
 5 files changed, 46 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-kernel-kexec-kdump b/Documentation/ABI/testing/sysfs-kernel-kexec-kdump
index f59051b5d96d..80941a7d8a7c 100644
--- a/Documentation/ABI/testing/sysfs-kernel-kexec-kdump
+++ b/Documentation/ABI/testing/sysfs-kernel-kexec-kdump
@@ -59,3 +59,11 @@ Description:	read only
 		to the crash (kdump) kernel. It lists the start and end physical
 		addresses of CMA regions assigned for crashkernel use.
 User:		kdump service
+
+What:		/sys/kernel/kexec/kho_bootmem_size
+Date:		Nov 2026
+Contact:	kexec at lists.infradead.org
+Description:	read only
+		Total size of the KHO bootmem regions in bytes. Returns 0
+		when KHO is disabled.
+User:		system monitoring
diff --git a/Documentation/admin-guide/mm/kho.rst b/Documentation/admin-guide/mm/kho.rst
index f9d2cc5999ee..c329d720bf25 100644
--- a/Documentation/admin-guide/mm/kho.rst
+++ b/Documentation/admin-guide/mm/kho.rst
@@ -81,6 +81,15 @@ At scale, correlating crashes to the previous kernel version enables
 faster root cause analysis when issues only occur in specific kernel
 transition scenarios.
 
+sysfs Interfaces
+================
+
+``/sys/kernel/kexec/kho_bootmem_size``
+    Total size of the KHO bootmem regions in bytes, or 0 when KHO is
+    disabled.
+
+See Documentation/ABI/testing/sysfs-kernel-kexec-kdump for the sysfs ABI.
+
 debugfs Interfaces
 ==================
 
diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h
index ef2b188c8ed0..736d51e79fd2 100644
--- a/include/linux/kexec_handover.h
+++ b/include/linux/kexec_handover.h
@@ -40,6 +40,7 @@ void kho_populate(phys_addr_t fdt_phys, u64 fdt_len, phys_addr_t bootmem_phys,
 		  u64 bootmem_len);
 
 bool kho_bootmem_overlap(phys_addr_t phys, size_t size);
+phys_addr_t kho_bootmem_size(void);
 
 static inline enum migratetype kho_bootmem_migratetype(unsigned long pfn,
 						       enum migratetype mt)
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index f4a31565c447..ed105e6351eb 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -13,6 +13,7 @@
 #include <linux/slab.h>
 #include <linux/fs.h>
 #include <linux/kexec.h>
+#include <linux/kexec_handover.h>
 #include <linux/mutex.h>
 #include <linux/list.h>
 #include <linux/liveupdate.h>
@@ -1324,6 +1325,15 @@ static struct kobj_attribute crash_elfcorehdr_size_attr = __ATTR_RO(crash_elfcor
 #endif /* CONFIG_CRASH_HOTPLUG */
 #endif /* CONFIG_CRASH_DUMP */
 
+#ifdef CONFIG_KEXEC_HANDOVER
+static ssize_t kho_bootmem_size_show(struct kobject *kobj,
+				     struct kobj_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%llu\n", (unsigned long long)kho_bootmem_size());
+}
+static struct kobj_attribute kho_bootmem_size_attr = __ATTR_RO(kho_bootmem_size);
+#endif
+
 static struct attribute *kexec_attrs[] = {
 	&loaded_attr.attr,
 #ifdef CONFIG_CRASH_DUMP
@@ -1335,6 +1345,9 @@ static struct attribute *kexec_attrs[] = {
 #ifdef CONFIG_CRASH_HOTPLUG
 	&crash_elfcorehdr_size_attr.attr,
 #endif
+#endif
+#ifdef CONFIG_KEXEC_HANDOVER
+	&kho_bootmem_size_attr.attr,
 #endif
 	NULL
 };
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 3046b89e5db4..21d77e29d2bb 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -942,6 +942,21 @@ static void __init kho_reserve_bootmem(void)
 	kho_enable = false;
 }
 
+phys_addr_t kho_bootmem_size(void)
+{
+	phys_addr_t size = 0;
+	unsigned int i;
+
+	/* kho_init() can disable KHO without clearing kho_bootmem_cnt. */
+	if (!kho_enable)
+		return 0;
+
+	for (i = 0; i < kho_bootmem_cnt; i++)
+		size += kho_bootmem[i].size;
+
+	return size;
+}
+
 /*
  * Look for free blocks of 1G. This is a heuristic chosen to work efficiently
  * with large systems with hundreds of gigabytes of memory. It will work poorly
-- 
2.54.0




More information about the kexec mailing list