Purpose of irqbalance

[Neil Horman]

On Wed, Sep 14, 2016 at 08:26:48PM +0530, Sriram V wrote:
> Hi Neil,
> 
> smp_affinity for all the irqs are set to 0xf. However, i still see
> that the irqs are routed only to cpu0. Is there any way of finding out
> why is this the case? What could be the problem?
> 

Its not a problem, I just explained to you how this works.  If you set an smp
affinity mask for an irq to more than a single bit (like 0xf), then you make all
of those cpus elligible to recieve the interrupt.  That in no way means that the
hardware will balance the irq to all the cpus.  The typical configuraion in
hardware is to send the interrupt to the lowest numbered cpu that is currently
not processing an interrupt (in your case cpu0).  This maximizes cache hit rates
and increases performance.  There is nothing wrong with this, its working as
designed.  If you want to spread irqs to multiple cpus, you need to assign a
single bit mask to each irq, which is what irqbalance typically does.

> I think that if the affinity is set like 0x1, 0x2, 0x4... it will
> work.. Wanted to find out the reason why cpu0 alone is processing all
> the interrupts.
> 
See above.

> Also, from your explanation - irq balance uses only the smp_affinity
> sysfs entry for the irq balancing. Is this correct?
> 
Its in proc, but yes, thats the only user space interface to assign irq affinity
in linux.

Neil

> Regards,
> Sriram
> 
> On Wed, Sep 14, 2016 at 8:06 PM, Neil Horman <nhorman at tuxdriver.com> wrote:
> > On Wed, Sep 14, 2016 at 06:52:23PM +0530, Sriram V wrote:
> >> Hi,
> >>
> >> Why is it that setting the smp_affinity alone is not sufficient for
> >> irq to be routed to other cpus?
> >> I tried doing that, but still the irq is only routed to cpu0.
> >>
> >> In otherwords, why do we need a app to make the routing? how does
> >> irqbalance app does that?
> >>
> >> Regards,
> >> Sriram
> >>
> > Thats really two separate questions:
> >
> > 1) How does the irq smp_affinity setting affect irq routing
> >
> > 2) Why do we use irqbalance to handle affinity setting
> >
> > The answer to (1) is that the smp_affinity mask provides a user defined set of
> > elligible cpus that can handle the corresponding irq.  For instance, setting irq
> > 10's smp_affinity to 3 (0x11) means that cpu 0 or cpu1 is allowed to handle irq
> > 10's interrupt.  The cpu which is actually selected is done by the hardware, and
> > usually amounts to the lowest numbered cpu within the elligible set that isn't
> > already handling an interrupt (which may explain, depending on how you set
> > smp_affinity) why cpu0 is still handling the interrupt your interested in.
> >
> > The answer to (2) is convienience.  Manually setting smp affinity for all irqs
> > is a tedious process as well as an ongoing one (consider hotplug or sriov use
> > cases).  Irq balance can make adjustments to smp affinity to best balance irq
> > workload according to different policies as irqs are allocated and freed.
> >
> > Hope that helps
> > Neil
> >
> >> _______________________________________________
> >> irqbalance mailing list
> >> irqbalance at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/irqbalance
> >>
>