[PATCH v2 0/2] iommu: Allow passing custom allocators to pgtable drivers

[Boris Brezillon]

Hi Jason,

On Fri, 10 Nov 2023 11:14:28 -0400
Jason Gunthorpe <jgg at ziepe.ca> wrote:

> On Fri, Nov 10, 2023 at 10:43:50AM +0100, Boris Brezillon wrote:
> > Hello,
> > 
> > This patchset is an attempt at making page table allocation
> > customizable. This is useful to some GPU drivers for various reasons:
> > 
> > - speed-up upcoming page table allocations by managing a pool of free
> >   pages
> > - batch page table allocation instead of allocating one page at a time
> > - pre-reserve pages for page tables needed for map/unmap operations and
> >   return the unused page tables to some pool  
> 
> Why would these topics be unique to GPU drivers as a user?

They are not, it's just that the primary use case for this extension
was a GPU driver, and that other GPU driver maintainers have expressed
interest in this solution too. Gaurav has a different use case that has
nothing to do with caching MMU page table allocations.

> 
> Shouldn't improving the allocator in the io page table be done
> generically?

While most of it could be made generic, the pre-reservation is a bit
special for VM_BIND: we need to pre-reserve page tables without knowing
the state of the page table tree (over-reservation), because page table
updates are executed asynchronously (the state of the VM when we
prepare the request might differ from its state when we execute it). We
also need to make sure no other pre-reservation requests steal pages
from the pool of pages we reserved for requests that were not executed
yet.

I'm not saying this is impossible to implement, but it sounds too
specific for a generic io-pgtable cache.

> 
> > A real example of how such custom allocators can be used is available
> > here[1]. v2 of the Panthor driver is approaching submission, and I

That's actually v3, not v2.

> > figured I'd try to upstream the dependencies separately, which is
> > why I submit this series now, even though the user of this new API
> > will come afterwards. If you'd prefer to have those patches submitted
> > along with the Panthor driver, let me know.  
> 
> Patches like this should come with users,

Fine, I can submit those two patches as part of the panthor patch
series if you prefer.

> but I think you should
> refocus this effort to improving the io pagetable itself not allowing
> a GPU driver to replace its insides.

These 2 patches don't come out of the blue. This has been discussed
with Robin who suggested to go for this simple solution where
allocations are deferred to the io-pgtable user. Turns out Gaurav
came up with a different use case where these custom page table
allocators would be useful.

Regards,

Boris