[PATCH v7 6/7] mm/vmalloc: map contiguous pages in batches for vmap() if possible

[David Hildenbrand (Arm)]

On 7/15/26 14:08, Wen Jiang wrote:
> From: "Barry Song (Xiaomi)" <baohua at kernel.org>
> 
> In many cases, the pages passed to vmap() may include high-order
> pages. For example, the systemheap often allocates pages in descending
> order: order 8, then 4, then 0. Currently, vmap() iterates over every
> page individually—even pages inside a high-order block are handled
> one by one.
> 
> This patch detects physically contiguous pages (regardless of whether
> they are compound or non-compound) by scanning with
> num_pages_contiguous(), and maps them as a single contiguous block
> whenever possible. The mapping order is determined by taking the
> minimum of the contiguous page count and the pfn alignment, allowing
> graceful degradation when pfn alignment is less than the contiguous
> range.
> 
> Pages with the same page_shift are coalesced and mapped via
> vmap_pages_range_noflush_walk() to avoid page table rewalk.
> 
> As users typically allocate memory in descending orders (e.g.
> 8 → 4 → 0), once an order-0 page is encountered, we stop scanning
> for contiguous pages since subsequent pages are likely order-0 as well.
> 
> Signed-off-by: Barry Song (Xiaomi) <baohua at kernel.org>
> Co-developed-by: Dev Jain <dev.jain at arm.com>
> Signed-off-by: Dev Jain <dev.jain at arm.com>
> Signed-off-by: Wen Jiang <jiangwen6 at xiaomi.com>
> Tested-by: Xueyuan Chen <xueyuan.chen21 at gmail.com>
> Tested-by: Leo Yan <leo.yan at arm.com>
> Reviewed-by: Dev Jain <dev.jain at arm.com>
> ---
>  mm/vmalloc.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 80 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index a1e025120e9df..92f9cd5e9def5 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -3557,6 +3557,84 @@ static inline unsigned int vm_shift(pgprot_t prot, unsigned long size)
>  	return arch_vmap_pte_supported_shift(size);
>  }
> 
> +static inline int get_vmap_batch_order(struct page **pages,
> +		pgprot_t prot, unsigned int max_steps, unsigned int idx)

Why pass in pages, idx when you can really just pass pages+idx?

And just call it "nr_pages" instead of "max_steps".

> +{
> +	unsigned long pfn;
> +	unsigned int nr_contig;
> +	int order;
> +
> +	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
> +		return 0;
> +
> +	nr_contig = num_pages_contiguous(&pages[idx], max_steps);
> +	if (nr_contig < 2)
> +		return 0;
> +
> +	order = ilog2(nr_contig);
> +	pfn = page_to_pfn(pages[idx]);
> +
> +	/* Limit order by pfn alignment */
> +	if (pfn > 0)
> +		order = min_t(int, order, __ffs(pfn));

Wouldn't it make sense to determine that before you call num_pages_contiguous?
Because then, you can just scan to that maximum instead of the given nr_pages.

> +
> +	if (vm_shift(prot, PAGE_SIZE << order) == PAGE_SHIFT)
> +		return 0;
> +
> +	return order;
> +}
> +
> +static int vmap_pages_range_batched(unsigned long addr, unsigned long end,
> +		pgprot_t prot, struct page **pages)
> +{
> +	unsigned int count = (end - addr) >> PAGE_SHIFT;

"nr_pages" ? Also, can be const.

> +	unsigned int prev_shift = 0, idx = 0;
> +	unsigned long map_addr = addr, batch_end = addr;
> +	int err;
> +
> +	err = kmsan_vmap_pages_range_noflush(addr, end, prot, pages,
> +					     PAGE_SHIFT, GFP_KERNEL);

Is the indentation on the second parameter line off?

> +	if (err)
> +		goto out;
> +
> +	for (unsigned int i = 0; i < count; ) {
> +		unsigned int shift = PAGE_SHIFT +
> +			get_vmap_batch_order(pages, prot, count - i, i);

Having two indices, i and idx, is just confusing.

The whole function is a bit overly complicated. Does it really buy us much to
batch over vmap_pages_range_noflush_walk() calling with the same shift?

> +
> +		if (!i)
> +			prev_shift = shift;
> +
> +		if (shift != prev_shift) {
> +			err = vmap_pages_range_noflush_walk(map_addr, batch_end,
> +					prot, pages + idx, prev_shift);
> +			if (err)
> +				goto out;
> +			prev_shift = shift;
> +			map_addr = batch_end;
> +			idx = i;
> +		}
> +
> +		/*
> +		 * Once small pages are encountered, the remaining pages
> +		 * are likely small as well.

"small pages" is odd. Maybe

"Once we fail to batch pages, we expect to fail batching for all remaining
pages, so just give up."

> +		 */
> +		if (shift == PAGE_SHIFT)
> +			break;
> +
> +		batch_end += 1UL << shift;
> +		i += 1U << (shift - PAGE_SHIFT);
> +	}
> +
> +	/* Remaining */
> +	if (map_addr < end)
> +		err = vmap_pages_range_noflush_walk(map_addr, end,
> +				prot, pages + idx, prev_shift);
> +
> +out:
> +	flush_cache_vmap(addr, end);
> +	return err;
> +}
> +
>  /**
>   * vmap - map an array of pages into virtually contiguous space
>   * @pages: array of page pointers
> @@ -3600,8 +3678,8 @@ void *vmap(struct page **pages, unsigned int count,
>  		return NULL;
> 
>  	addr = (unsigned long)area->addr;
> -	if (vmap_pages_range(addr, addr + size, pgprot_nx(prot),
> -				pages, PAGE_SHIFT) < 0) {
> +	if (vmap_pages_range_batched(addr, addr + size, pgprot_nx(prot),
> +				pages) < 0) {

Nit: single line would make that nicer to read.

-- 
Cheers,

David