[PATCH] arm64: Warn the user when a small VA_BITS value wastes memory

[Mark Rutland]

On Tue, Dec 15, 2020 at 03:29:18PM +0000, Marc Zyngier wrote:
> The memblock code ignores any memory that doesn't fit in the
> linear mapping. In order to preserve the distance between two physical
> memory locations and their mappings in the linear map, any hole between
> two memory regions occupies the same space in the linear map.
> 
> On most systems, this is hardly a problem (the memory banks are close
> together, and VA_BITS represents a large space compared to the available
> memory *and* the potential gaps).
> 
> On NUMA systems, things are quite different: the gaps between the
> memory nodes can be pretty large compared to the memory size itself,
> and the range from memblock_start_of_DRAM() to memblock_end_of_DRAM()
> can exceed the space described by VA_BITS.
> 
> Unfortunately, we're not very good at making this obvious to the user,
> and on a D05 system (two sockets and 4 nodes with 64GB each)
> accidentally configured with 39bit VA, we display something like this:
> 
> [    0.000000] NUMA: NODE_DATA [mem 0x1ffbffe100-0x1ffbffffff]
> [    0.000000] NUMA: NODE_DATA [mem 0x2febfc1100-0x2febfc2fff]
> [    0.000000] NUMA: Initmem setup node 2 [<memory-less node>]
> [    0.000000] NUMA: NODE_DATA [mem 0x2febfbf200-0x2febfc10ff]
> [    0.000000] NUMA: NODE_DATA(2) on node 1
> [    0.000000] NUMA: Initmem setup node 3 [<memory-less node>]
> [    0.000000] NUMA: NODE_DATA [mem 0x2febfbd300-0x2febfbf1ff]
> [    0.000000] NUMA: NODE_DATA(3) on node 1
> 
> which isn't very explicit, and doesn't tell the user why 128GB
> have suddently disappeared.
> 
> Let's add a warning message telling the user that memory has been
> truncated, and offer a potential solution (bumping VA_BITS up).
> 
> Signed-off-by: Marc Zyngier <maz at kernel.org>
> ---
>  arch/arm64/mm/init.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 095540667f0f..cd5da059de85 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -283,6 +283,9 @@ void __init arm64_memblock_init(void)
>  	memstart_addr = round_down(memblock_start_of_DRAM(),
>  				   ARM64_MEMSTART_ALIGN);
>  
> +	if ((memblock_end_of_DRAM() - memstart_addr) > linear_region_size)
> +		pr_warn("Memory doesn't fit in the linear mapping, VA_BITS too small\n");


Hmm... we might want to restructure this code a bit, since we have a few
checks that are /almost/ the same, and we could probably place all the
memory clipping under this new condition. That said, after having a go,
I couldn't find a neat solution.

FWIW, for this as-is:

Acked-by: Mark Rutland <mark.rutland at arm.com>

Mark.