dma-pool fixes

[Amit Pundir]

On Wed, 29 Jul 2020 at 17:52, Amit Pundir <amit.pundir at linaro.org> wrote:
>
> On Wed, 29 Jul 2020 at 16:15, Nicolas Saenz Julienne
> <nsaenzjulienne at suse.de> wrote:
> >
> > On Tue, 2020-07-28 at 17:30 +0200, Christoph Hellwig wrote:
> > > On Tue, Jul 28, 2020 at 06:18:41PM +0530, Amit Pundir wrote:
> > > > > Oh well, this leaves me confused again.  It looks like your setup
> > > > > really needs a CMA in zone normal for the dma or dma32 pool.
> > > >
> > > > Anything I should look up in the downstream kernel/dts?
> > >
> > > I don't have a good idea right now.  Nicolas, can you think of something
> > > else?
> >
> > To summarise, the device is:
> >  - Using the dma-direct code path.
> >  - Requesting ZONE_DMA memory to then fail when provided memory falls in
> >    ZONE_DMA. Actually, the only acceptable memory comes from CMA, which is
> >    located topmost of the 4GB boundary.
> >
> > My wild guess is that we may be abusing an iommu identity mapping setup by
> > firmware.
> >

Hi Nicolas, Christoph,

Just out of curiosity, I'm wondering if we can restore the earlier
behaviour and make DMA atomic allocation configured thru platform
specific device tree instead?

Or if you can allow a more hackish approach to restore the earlier
logic, using of_machine_is_compatible() just for my device for the
time being. Meanwhile I'm checking with other developers running the
mainline kernel on sdm845 phones like OnePlus 6/6T, if they see this
issue too.

Regards,
Amit Pundir

> > That said, what would be helpful to me is to find out the troublesome device.
> > Amit, could you try adding this patch along with Christoph's modified series
> > (so the board boots). Ultimately DMA atomic allocations are not that common, so
> > we should get only a few hits:
>
> Hi, still not hitting dma_alloc_from_pool().
>
> I hit the following direct alloc path only once, at starting:
>
> dma_alloc_coherent ()
> -> dma_alloc_attrs()
>    -> dma_is_direct() -> dma_direct_alloc()
>       -> dma_direct_alloc_pages()
>          -> dma_should_alloc_from_pool() #returns FALSE from here
>
> After that I'm hitting following iommu dma alloc path all the time:
>
> dma_alloc_coherent()
> -> dma_alloc_attrs()
>    -> (ops->alloc) -> iommu_dma_alloc()
>       -> iommu_dma_alloc_remap() #always returns from here
>
> So dma_alloc_from_pool() is not getting called at all in either of the
> above cases.
>
> >
> > diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
> > index 83fda1039493..de93fce6d5d2 100644
> > --- a/kernel/dma/pool.c
> > +++ b/kernel/dma/pool.c
> > @@ -276,8 +276,11 @@ struct page *dma_alloc_from_pool(struct device *dev, size_t size,
> >         while ((pool = dma_guess_pool(pool, gfp))) {
> >                 page = __dma_alloc_from_pool(dev, size, pool, cpu_addr,
> >                                              phys_addr_ok);
> > -               if (page)
> > +               if (page) {
> > +                       dev_err(dev, "%s: phys addr 0x%llx, size %lu, dev->coherent_dma_mask 0x%llx, dev->bus_dma_limit 0x%llx\n",
> > +                               __func__, (phys_addr_t)*cpu_addr, size, dev->coherent_dma_mask, dev->bus_dma_limit);
> >                         return page;
> > +               }
> >         }
> >
> >         WARN(1, "Failed to get suitable pool for %s\n", dev_name(dev));
> >
> >