[PATCH 12/12] mm/hugetlb: make bootmem allocation work with KHO

[Mike Rapoport]

Hi Pratyush,

On Wed, Apr 29, 2026 at 03:39:14PM +0200, Pratyush Yadav wrote:
> From: "Pratyush Yadav (Google)" <pratyush at kernel.org>
> 
> Gigantic page allocation is somewhat broken currently when KHO is used.
> 
> Firstly, they break KHO scratch size accounting. RSRV_KERN is used to
> track how much memory is reserved for use by the kernel. Since
> alloc_bootmem() calls the memblock_alloc*() APIs, the hugepages
> allocated also get marked as RSRV_KERN.

The intended semantics of RSRV_KERN was to distinguish memory reserved
because it's in use by firmware from the memory reserved or allocated by
the kernel. It does not have to be directly used by the kernel, some of
PG_Reserved memory is mapped to user-space and it's not only hugetlb. There
are zero pages, VDSO and maybe something else, I don't remember.

> Allocations marked RSRV_KERN are used by KHO to calculate how much
> scratch space it should reserve to make sure the next kernel has enough
> memory to boot when it is in scratch-only phase. Counting hugepages in

The size of RSRV_KERN allocations is also used in
memblock_estimated_nr_free_pages() that's currently used by
set_max_threads() to set several rlimits and by s390 to allocate colored
zero pages. Excluding hugetlb from that will skew the calculations there.

> that blows up scratch size, and can lead to the scratch allocation
> failing, making KHO unusable. This will show up when huge pages make up
> more than 50% of the system, which is a fairly common use case.
> 
> Secondly, while not supported right now, huge pages are user memory and
> can be preserved via KHO. The scratch spaces should not have any
> preserved memory. Allocating hugepages from scratch (on a KHO boot) can
> lead to them being un-preservable.
> 
> Introduce memblock_alloc_nid_user(). This does two things: first, it
> instructs __memblock_alloc_range_nid() to not use scratch areas to
> fulfill allocation. If KHO is in scratch-only mode, allocations will
> only be made from extended scratch areas. Second, it removes RSRV_KERN
> from the allocation to make sure it doesn't mess up scratch size
> accounting.
> 
> To reduce duplication, introduce __memblock_alloc_range_nid() which does
> exactly what memblock_alloc_range_nid() used to do, but takes the flags
> from its caller. Then make memblock_alloc_range_nid() a wrapper to it.
> This lets memblock_alloc_nid_user() re-use most of the logic without
> causing churn to update all callers of memblock_alloc_range_nid() and
> adding yet another argument to it.

That's neat :)

But I'm not too fond of the memblock_alloc_nid_user() as a concept. That
early at boot everything is still kernel, even though hugetlb pages might
become user afterwards if they are actually consumed.
Another thing, is that adding such global API to memblock could be abused
and suddenly some early code will clear RSRV_KERN for a random pieces of
memory.

If we'd still need a special memblock function for gigantic pages
allocation I'd rather make it explicit that it's for hugetlb and keep it in
mm/internal.h

I was thinking about possible alternative solutions and here's what I came
up with

1. SCRATCH_EXT nicely increases the memory pool available to memblock, but
   the decision which memory can be used for which allocations becomes more
   complex. It should make sure that SCRATCH is not used for hugetlb, but
   OTOH it's preferred for other early allocations. With implicit
   interaction between choose_memblock_flags() and should_skip_region()
   this seems to me quite a headache.
   memblock_reserved_clear_kern() should be changed to something else to
   keep set_max_threads() working. And, IMHO, memblock_alloc_nid_user()
   should be turned into memblock_alloc_gigantic_hugetlb() and only exposed
   to MM code. It's even possible that it will duplicate some of
   memblock_alloc_range_nid) rather than use it because hugetlb is always a
   special case.

2. Split memblock_reserve() part from kho_mem_retrieve() to run before
   hugetlb allocations. With this we won't need new types and APIs, we can
   ensure that hugetlb allocations do not use SCRATCH by reserving scratch
   areas before hugetlb allocations and releasing them afterwards.
   Obviously this is the slowest option as it will slow down all memblock
   allocations from the point we memblock_reserve() preserved memory. Still
   realistically I wouldn't expect large impact on performance because the
   heaviest part there is reserving of the preserved memory that has to
   happen anyway. Also, I don't thing that a system that uses a lot of
   gigantic pages will have a lot of preserved chunks scattered around.

3. Move the complexity into hugetlb and make it preserve all the gigantic
   pages with KHO. This means, though, that we won't be able to increase
   the number of gigantic pages after the first boot (although decreasing
   it seems easy) and that we need to let scratch auto scaling understand
   what were the "normal" memblock allocations and what were the
   allocations of the gigantic pages.

4. Invert SCRATCH_EXT logic and instead of freeing large chunks around the
   preserved memory to SCRATCH_EXT, reserve memory surrounding the
   preserved areas and release scratch_only before hugetlb allocations.
   We'd still need to somehow prevent hugetlb allocation spilling into
   scratch and there's a nasty piece of releasing the memory around the
   preserved chunks. On the bright side, I think it's feasible to defer the
   release of those regions and free them when we are already
   multithreaded. This probably the most involved alternative but it also
   could help with the bottleneck of kho_mem_retrieve() creating too many
   memblock.reserved regions.

Thoughts?

> Signed-off-by: Pratyush Yadav (Google) <pratyush at kernel.org>
> ---
> 
> Notes:
>     Checkpatch complains here about the alignment of arguments of
>     memblock_alloc_range_nid() with open parentheses. That can be ignored
>     since the code already was mis-aligned, and for good reason.
> 
>  include/linux/memblock.h |   4 ++
>  mm/hugetlb.c             |  19 ++----
>  mm/memblock.c            | 138 ++++++++++++++++++++++++++++++---------
>  3 files changed, 116 insertions(+), 45 deletions(-)

-- 
Sincerely yours,
Mike.