[PATCH] arm64: Make ARCH_DMA_MINALIGN configurable

[Ard Biesheuvel]

On Mon, 17 May 2021 at 13:06, Catalin Marinas <catalin.marinas at arm.com> wrote:
>
> On Mon, May 17, 2021 at 09:43:32AM +0200, Vincent Whitchurch wrote:
> > ARCH_DMA_MINALIGN is hardcoded to 128, but this wastes memory if the
> > kernel is only intended to be run on platforms with cache line sizes of
> > 64 bytes.
> >
> > Make this configurable (hidden under CONFIG_EXPERT).  Setting this to 64
> > bytes reduces the slab memory usage of my Cortex-A53-based system by
> > ~6%, measured right after startup.
>
> I agree that we waste some memory since the kmalloc caches start from
> 128 but I don't think a config option is the right.
>
> An option would be to try not to rely on the hard-coded
> ARCH_DMA_MINALIGN when the slab caches are created but use
> cache_line_size(). It's a bit tricky as the cache_line_size() returned
> value may be tweaked by DT or PPTT after the boot caches have been
> created (see commit 7b8c87b297a7).
>
> Another option I recall discussing with Arnd about two years ago was to
> start with the default 128 at boot but add the smaller slab caches
> later, once we have more information. This can be just another 64 byte
> cache or even go all the way down to 8 byte if all the devices are
> cache coherent.
>

ARCH_SLAB_MINALIGN is also used to statically align (members of)
struct types, so doing this at runtime is going to have limited
effect.

If a) ThunderX is the only platform we care about (do we?) that has
128 byte cachelines, and b) DMA is cache coherent on such platforms,
couldn't we separate ARCH_SLAB_MINALIGN from ARCH_DMA_MINALIGN? I.e.,
set the first to 64 and keep the second at 128?