[PATCH v2 2/2] arm64: mm: reserve per-numa CMA after numa_init

[Song Bao Hua (Barry Song)]

> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy at arm.com]
> Sent: Thursday, June 25, 2020 11:16 PM
> To: Song Bao Hua (Barry Song) <song.bao.hua at hisilicon.com>; hch at lst.de;
> m.szyprowski at samsung.com; will at kernel.org;
> ganapatrao.kulkarni at cavium.com; catalin.marinas at arm.com
> Cc: iommu at lists.linux-foundation.org; Linuxarm <linuxarm at huawei.com>;
> linux-arm-kernel at lists.infradead.org; linux-kernel at vger.kernel.org; Nicolas
> Saenz Julienne <nsaenzjulienne at suse.de>; Steve Capper
> <steve.capper at arm.com>; Andrew Morton <akpm at linux-foundation.org>;
> Mike Rapoport <rppt at linux.ibm.com>
> Subject: Re: [PATCH v2 2/2] arm64: mm: reserve per-numa CMA after
> numa_init
> 
> On 2020-06-25 08:43, Barry Song wrote:
> > Right now, smmu is using dma_alloc_coherent() to get memory to save
> queues
> > and tables. Typically, on ARM64 server, there is a default CMA located at
> > node0, which could be far away from node2, node3 etc.
> > with this patch, smmu will get memory from local numa node to save
> command
> > queues and page tables. that means dma_unmap latency will be shrunk
> much.
> > Meanwhile, when iommu.passthrough is on, device drivers which call dma_
> > alloc_coherent() will also get local memory and avoid the travel between
> > numa nodes.
> >
> > Cc: Christoph Hellwig <hch at lst.de>
> > Cc: Marek Szyprowski <m.szyprowski at samsung.com>
> > Cc: Will Deacon <will at kernel.org>
> > Cc: Robin Murphy <robin.murphy at arm.com>
> > Cc: Ganapatrao Kulkarni <ganapatrao.kulkarni at cavium.com>
> > Cc: Catalin Marinas <catalin.marinas at arm.com>
> > Cc: Nicolas Saenz Julienne <nsaenzjulienne at suse.de>
> > Cc: Steve Capper <steve.capper at arm.com>
> > Cc: Andrew Morton <akpm at linux-foundation.org>
> > Cc: Mike Rapoport <rppt at linux.ibm.com>
> > Signed-off-by: Barry Song <song.bao.hua at hisilicon.com>
> > ---
> >   arch/arm64/mm/init.c | 2 ++
> >   1 file changed, 2 insertions(+)
> >
> > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > index 1e93cfc7c47a..07d4d1fe7983 100644
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -420,6 +420,8 @@ void __init bootmem_init(void)
> >
> >   	arm64_numa_init();
> >
> > +	dma_pernuma_cma_reserve();
> > +
> 
> It might be worth putting this after the hugetlb_cma_reserve() call for
> clarity, since the comment below applies equally to this call too.

Yep, it looks even better though dma_pernuma_cma_reserve() is self-documenting by name.

> 
> Robin.
> 
> >   	/*
> >   	 * must be done after arm64_numa_init() which calls numa_init() to
> >   	 * initialize node_online_map that gets used in hugetlb_cma_reserve()
> >
Thanks
Barry