SMMUv3 interrupt handling via custom logic

[Will Deacon]

Hey Stefano, [+Marc]

On Fri, Jun 27, 2025 at 12:20:37PM -0700, Stefano Stabellini wrote:
> On Fri, 27 Jun 2025, Robin Murphy wrote:
> > On 2025-06-27 8:19 am, Michal Simek wrote:
> > > We are using smmu-v3 in our SOC and I would like to ask you for
> > > recommendation how to handle our interrupt cases.
> > > 
> > > here is description which we are using
> > > 
> > > smmu: iommu at ec000000 {
> > >      compatible = "arm,smmu-v3";
> > >      reg = <...>;
> > >      #iommu-cells = <1>;
> > >      interrupt-names = "combined";
> > >      interrupts = <0 169 4>;
> > > };
> > > 
> > > but it is missing one important detail which just arise that actually there
> > > is additional HW logic which deals with SMMU interrupts separately.
> > > There is a secure part (global, cmd, event - gerror, cmdq-sync, eventq in
> > > DT)
> > > and non secure part (pri, global, cmd, event - priq, gerror, cmdq-sync,
> > > eventq in DT).
> > > Based on my information all these interrupts should be acked once handled to
> > > be able to get another one.
> > > The driver itself is able to handle them separately but we didn't create any
> > > solution to reach custom HW to do it.
> > > 
> > > I looked at f935448acf46 ("iommu/arm-smmu-v3: Add workaround for Cavium
> > > ThunderX2 erratum #126") which introduced combined IRQs but it looks like
> > > that there is no need for additional ACK of that IRQs.
> > 
> > Per the architecture, SMMU interrupts are logically edge-triggered so there is
> > nothing to clear at the SMMU end (the "interrupt status" is implicit in
> > whatever condition caused an interrupt to be sent, e.g. the event queue
> > becoming non-empty, SMMU_GERROR becoming different from SMMU_GERRORN, etc.)
> > 
> > If this is an Arm SMMU IP (MMU-600/700/S3) then the physical interrupt outputs
> > are most definitely rising-edge. If somone's stuck some interrupt combiner in
> > between those and the main interrupt controller, then yes, that interrupt
> > combiner really should have its own driver.
> > 
> > > The HW logic itself is handling secure and non secure settings for SMMU
> > > that's why would be the best to avoid directly mapping it in Linux.
> > > 
> > > One way to go is to create secondary interrupt controller driver
> > > a) ioremap one with notice about secure part because we are using SMMU only
> > > with NS world
> > > b) firmware based to tunnel accesses via SMCs and allow only access to
> > > limited amount of registers
> > > 
> > > The second way is likely create any hooks in the driver to be able to
> > > provide additional SOC specific hooks.
> > 
> > If this thing is munging *all* the SMMU interrupt outputs as I suspect, then
> > the big problem with that idea is that "the driver" is at least two separate
> > drivers (SMMU and PMU), 3 if it has RAS and you ever want to entertain the
> > idea of kernel-first handling.
> 
> Yeah... I tend to favor simple solutions when possible and the secondary
> interrupt controller driver approach is looking increasingly complex.

I think that's probably the right way to go, though. Moving the
configuration to firmware just means you now have two problems instead
of one.

It looks like MTK may have done something similar:

https://lore.kernel.org/r/20250616025628.25454-7-xueqi.zhang@mediatek.com

so if changes are needed to irqchip to handle the limited capabilities
of the extra logic (as per Marc's comments in the thread above), it
would be good to make sure that's reusable. What I really _don't_ want
is a half-baked, custom interrupt handling mechanism in the IOMMU code.

> In addition to what Robin mentioned, I understand that this email is
> directed to linux-arm-kernel and the LKML, so the focus is naturally to
> solve the problem for Linux. However, let me point out that this issue
> also affects Xen, all hypervisors, and proprietary operating systems.
> The "big problem" is even bigger :-(  Complexity will multiply very
> quickly.

Sounds like you need to feed that back to the geniuses who designed this
hardware :). If you want your hardware to work well with existing
software, it's generally best to build it to spec.

Will