[PATCH] nvme-pci: add adaptive interrupt polling

[changfengnan]

> From: "Anuj Gupta"<anuj1072538 at gmail.com>
> Date:  Tue, Aug 25, 2026, 10:59
> Subject:  Re: [PATCH] nvme-pci: add adaptive interrupt polling
> To: "Fengnan Chang"<changfengnan at bytedance.com>
> Cc: "Keith Busch"<kbusch at kernel.org>, "Jens Axboe"<axboe at kernel.dk>, "Christoph Hellwig"<hch at lst.de>, "Sagi Grimberg"<sagi at grimberg.me>, <linux-nvme at lists.infradead.org>, <linux-kernel at vger.kernel.org>, <jun1.zeng at intel.com>, "Guzebing"<guzebing at bytedance.com>, <anuj20.g at samsung.com>
> > +/*
> > + * Drain CQEs from IRQ_POLL_SOFTIRQ and compare completion progress with the
> > + * IRQ baseline.  Re-arm while within the allowed lag; leave poll mode on lag
> > + * or teardown, and start another window only after a faster full window.
> > + */
> > +static int nvme_adaptive_irq_poll(struct irq_poll *iop, int budget)
> > +{
> > +        struct nvme_adaptive_poll *adaptive = container_of(iop,
> > +                                        struct nvme_adaptive_poll, iopoll);
> > +        struct nvme_queue *nvmeq = adaptive->nvmeq;
> > +        unsigned int completions, limit;
> > +        unsigned long flags;
> > +        u64 deadline, elapsed, now;
> > +        DEFINE_IO_COMP_BATCH(iob);
> > +
> > +        spin_lock_irqsave(&nvmeq->cq_poll_lock, flags);
> > +        if (unlikely(!test_bit(NVMEQ_ADAPTIVE_POLLING, &nvmeq->flags))) {
> > +                completions = 0;
> > +                irq_poll_complete(iop);
> > +                goto out;
> > +        }
> > +        if (!test_bit(NVMEQ_ENABLED, &nvmeq->flags)) {
> > +                completions = 0;
> > +                irq_poll_complete(iop);
> > +                nvme_adaptive_poll_end(nvmeq, false);
> > +                goto out;
> > +        }
> > +
> > +        limit = min_t(unsigned int,
> > +                      budget,
> > +                      NVME_ADAPTIVE_EPISODE_CQES - adaptive->completions);
> > +        completions = nvme_poll_cq_bounded(nvmeq, &iob, limit);
> > +        adaptive->completions += completions;
> > +
> > +        if (completions >= budget &&
> > +            adaptive->completions < NVME_ADAPTIVE_EPISODE_CQES)
> > +                goto out;
> > +        irq_poll_complete(iop);
> 
> If the last drain reaches NVME_ADAPTIVE_EPISODE_CQES with
> completions == budget, this calls irq_poll_complete() but still returns
> budget. irq_poll_softirq() will then call list_move_tail() on an entry
> that has already been removed.

Thanks, you are right, I also noticed this issue during testing.
In next version, irq_poll_complete() is called only when
completions < budget, so a full-budget return leaves the iopoll state untouched.

> Should the full-budget case always goto out and let the next invocation
> complete and evaluate the episode?
>