[PATCH 4/4] arm64: gicv3: its: Increase FORCE_MAX_ZONEORDER for Cavium ThunderX

[Will Deacon]

On Tue, May 12, 2015 at 05:20:49PM +0100, Robert Richter wrote:
> On 12.05.15 13:30:57, Will Deacon wrote:
> > On Mon, May 11, 2015 at 10:14:38AM +0100, Robert Richter wrote:
> > > On 05.05.15 11:53:29, Will Deacon wrote:
> > > > On Sun, May 03, 2015 at 09:49:32PM +0100, Robert Richter wrote:
> > > > > From: Radha Mohan Chintakuntla <rchintakuntla at cavium.com>
> > > > > 
> > > > > In case of ARCH_THUNDER, there is a need to allocate the GICv3 ITS table
> > > > > which is bigger than the allowed max order. So we are forcing it only in
> > > > > case of 4KB page size.
> > > > 
> > > > Does this problem disappear if the ITS driver uses dma_alloc_coherent
> > > > instead? That would also allow us to remove the __flush_dcache_area abuse
> > > > from the driver.
> > > 
> > > __get_free_pages() is also used internally in dma_alloc_coherent().
> > > 
> > > There is another case if the device brings dma mem with it. I am not
> > > sure if it would be possible to assign some phys memory via devicetree
> > > to the interrupt controller and then assign that range for its table
> > > allocation.
> > > 
> > > Another option would be to allocate a hugepage. This would require
> > > setting up hugepages during boottime. I need to figure out whether
> > > that could work.
> 
> For allocation of 16MB cont. phys mem of a defconfig kernel (4KB
> default pagesize) I see this different approaches:

16MB sounds like an awful lot. Is this because you have tonnes of MSIs or
a sparse DeviceID space or both?

>  * set FORCE_MAX_ZONEORDER to 13 as default,
> 
>  * set FORCE_MAX_ZONEORDER to 13 if ARM_GIC_V3 is set,
> 
>  * set FORCE_MAX_ZONEORDER to 13 if ARCH_THUNDER is set (this patch),

I'm not hugely fond of these suggestions, as there's still no guarantee
that such a huge allocation is going to succeed and we end up bumping
MAX_ORDER for all platforms in defconfig if we enable THUNDER there.

>  * use hugepages if enabled (defconfig has the following options
>    enable: CGROUP_HUGETLB, TRANSPARENT_HUGEPAGE, HUGETLBFS, this might
>    work with current default kernel without changing defconfig
>    options),

I don't think hugepages help with DMA.

>  * use devicetree to reserve mem for gicv3 (need to check ACPI).

Using a carveout like this might be the best bet. I assume the memory used
by the ITS can never be reclaimed by the syste (and therefore there's no
issue with wastage)?

> Do you see any direction?

Dunno, does CMA also require the MAX_ORDER bump?

Will