[RFC PATCH 2/3] PCI/MSI: Implement irq_mask_nowait() for MSI-x

[Luigi Rizzo]

On Wed, Oct 14, 2026 at 11:10 PM Thomas Gleixner <tglx at kernel.org> wrote:
>
> On Wed, Oct 14 2026 at 12:31, Bjorn Helgaas wrote:
> >> Skip the read-back that flushes the MSI-x mask write. Parent domains
> >> that mask on behalf of PCI/MSI continue using their own irq_mask().
> >>
> >> Signed-off-by: Keith Busch <kbusch at kernel.org>
> >
> > I assume Thomas will review/ack the pci/msi/ parts.
>
> Yes. I actually suggested that "nowait" mechanism to Keith at LPC.

The topic also came up around last December. The readback wastes 1000-1500ns
under load. I hacked a conditional in pci_msix_mask()
https://lore.kernel.org/all/20251220193120.3339162-1-lrizzo@google.com/
and you suggested a callback irq_mask_dev calling pci_irq_mask_msix_lazy()
https://lore.kernel.org/all/87jyxqiuta.ffs@tglx/

As usual naming is the hard part...

The mechanism is definitely useful (with moderation the mask is a frequent
operation, so saving the extra 1-2us per call is measurable).
I was a bit worried about adding yet another callback, but there are
already 30 or so,
hence one or two more does not make much of a difference.

On passing, on ARM this will not help unless we partially revert the
optimization
that skips PCIe if the interrupt has MSI_FLAG_PCI_MSI_MASK_PARENT
https://lore.kernel.org/all/20250517103011.2573288-1-maz@kernel.org/

> Though I hate the name of that callback because irq_mask() does not
> really wait. It guarantees that the mask store has reached the device
> which means that after that point no further interrupts will be sent by
> the device (assuming sane hardware). That means it's a synchronous
> operation which involves flushing the store, so maybe irq_mask_noflush()
> or irq_mask_async() might be more accurate and descriptive.
>
> > Cc'd Luigi since I know he's interested in interrupt moderation in
> > general.
>
> Correct. I'm curious whether the interrupt frequency based moderation
> mechanism Luigi is proposing does solve the NVME issues as well or can
> be combined with some device specific hint mechanism.

I really think we need both. It depends on what the issue is:

- for hardware level problems (backpressure on PCIe from the interrupt
dispatch pipeline)
  we need to actually disable interrupts at the PCIe sources, and
address the problem at
  a global SoC level, not on the single device (NICs, NVME, VFs used by VMs...).
  I honestly believe something global like GSIM is the best answer,
because the alternative
  of having all drivers implement moderation and coordinate the delays
would be hard to manage.

- for pure CPU overload due to e.g. excessive interrupts from a single source,
  GSIM can help a bit (e.g. preventing livelock, or cutting some
10-20% interrupt overhead),
  but driver specific solutions have a lot more visibility on what is happening
   and may possibly do a better job. As an example,
   Fengnan's solution not only moderates (or does timed polling, GSIM
has that too),
   but can also dynamically switch completion processing between
hardirq and softirq.

   https://lore.kernel.org/linux-nvme/20260818033846.53790-1-changfengnan@bytedance.com/

cheers
luigi

>
> That said, Luigi's approach should benefit from the "nowait" mechanism
> as well.
>
> Luigi?
>
> Thanks,
>
>         tglx