[PATCH] arm64: Make ARCH_DMA_MINALIGN configurable

[Catalin Marinas]

On Mon, May 17, 2021 at 09:43:32AM +0200, Vincent Whitchurch wrote:
> ARCH_DMA_MINALIGN is hardcoded to 128, but this wastes memory if the
> kernel is only intended to be run on platforms with cache line sizes of
> 64 bytes.
> 
> Make this configurable (hidden under CONFIG_EXPERT).  Setting this to 64
> bytes reduces the slab memory usage of my Cortex-A53-based system by
> ~6%, measured right after startup.

I agree that we waste some memory since the kmalloc caches start from
128 but I don't think a config option is the right.

An option would be to try not to rely on the hard-coded
ARCH_DMA_MINALIGN when the slab caches are created but use
cache_line_size(). It's a bit tricky as the cache_line_size() returned
value may be tweaked by DT or PPTT after the boot caches have been
created (see commit 7b8c87b297a7).

Another option I recall discussing with Arnd about two years ago was to
start with the default 128 at boot but add the smaller slab caches
later, once we have more information. This can be just another 64 byte
cache or even go all the way down to 8 byte if all the devices are
cache coherent.

-- 
Catalin