[PATCH 5/7] nvme-pci: use a better encoding for small prp pool allocations

[Caleb Sander Mateos]

On Tue, May 13, 2025 at 8:06 AM Keith Busch <kbusch at kernel.org> wrote:
>
> On Tue, May 13, 2025 at 09:00:21AM +0200, Christoph Hellwig wrote:
> > +static inline struct dma_pool *nvme_dma_pool(struct nvme_queue *nvmeq,
> > +             struct nvme_iod *iod)
> > +{
> > +     if (iod->flags & IOD_SMALL_DESCRIPTOR)
> > +             return nvmeq->descriptor_pools.small;
> > +     return nvmeq->descriptor_pools.large;
> > +}
>
> ...
>
> >               if (i == NVME_CTRL_PAGE_SIZE >> 3) {
> >                       __le64 *old_prp_list = prp_list;
> > -                     prp_list = dma_pool_alloc(pool, GFP_ATOMIC, &prp_dma);
> > +
> > +                     prp_list = dma_pool_alloc(nvme_dma_pool(nvmeq, iod),
> > +                                     GFP_ATOMIC, &prp_dma);
>
> You could assume nvmeq->descriptor_pools.large here. We'd never use the
> small pool if we've enough elements to chain lists, and it would skip
> the extra condition branch on each loop.

Right, and the existing code already assumes this branch is taken only
in the large case since iod->descriptors[0] is always set before the
loop but nr_descriptors is only initialized to 1 in the large case.

The existing approach of storing the struct dma_pool * in a local
variable seems fine to me and would avoid another pointer dereference
here.

Best,
Caleb

>
> The same optimization is done in the free side already, too.