[PATCH] PCI: uniphier: Serialize INTx masking/unmasking

[Kunihiko Hayashi]

Hi Marc,

On 2021/08/25 18:07, Marc Zyngier wrote:
> On Wed, 25 Aug 2021 01:01:08 +0100,
> Kunihiko Hayashi <hayashi.kunihiko at socionext.com> wrote:
>>
>> Hi Marc,
>>
>> On 2021/08/24 1:57, Marc Zyngier wrote:
>>> On Mon, 23 Aug 2021 16:09:27 +0100,
>>> Pali Rohár <pali at kernel.org> wrote:
>>>>
>>>> + Marc (who originally reported this issue)
>>>>
>>>> On Monday 23 August 2021 20:18:20 Kunihiko Hayashi wrote:
>>>>> The condition register PCI_RCV_INTX is used in irq_mask(), irq_unmask()
>>>>> and irq_ack() callbacks. Accesses to register can occur at the same time
>>>>> without a lock.
>>>>> Add a lock into each callback to prevent the issue.
>>>>>
>>>>> Fixes: 7e6d5cd88a6f ("PCI: uniphier: Add UniPhier PCIe host controller support")
>>>>> Suggested-by: Pali Rohár <pali at kernel.org>
>>>>> Signed-off-by: Kunihiko Hayashi <hayashi.kunihiko at socionext.com>
>>>>
>>>> Acked-by: Pali Rohár <pali at kernel.org>
>>>>
>>>>> ---
>>>>>    drivers/pci/controller/dwc/pcie-uniphier.c | 15 +++++++++++++++
>>>>>    1 file changed, 15 insertions(+)
>>>>>
>>>>> The previous patch is as follows:
>>>>> https://lore.kernel.org/linux-pci/1629370566-29984-1-git-send-email-hayashi.kunihiko@socionext.com/
>>>>>
>>>>> Changes in the previous patch:
>>>>> - Change the subject and commit message
>>>>>
>>>>> diff --git a/drivers/pci/controller/dwc/pcie-uniphier.c b/drivers/pci/controller/dwc/pcie-uniphier.c
>>>>> index ebe43e9..5075714 100644
>>>>> --- a/drivers/pci/controller/dwc/pcie-uniphier.c
>>>>> +++ b/drivers/pci/controller/dwc/pcie-uniphier.c
>>>>> @@ -186,12 +186,17 @@ static void uniphier_pcie_irq_ack(struct irq_data *d)
>>>>>    	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
>>>>>    	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
>>>>>    	struct uniphier_pcie_priv *priv = to_uniphier_pcie(pci);
>>>>> +	unsigned long flags;
>>>>>    	u32 val;
>>>>>    +	raw_spin_lock_irqsave(&pp->lock, flags);
>>>>> +
>>>>>    	val = readl(priv->base + PCL_RCV_INTX);
>>>>>    	val &= ~PCL_RCV_INTX_ALL_STATUS;
>>>>>    	val |= BIT(irqd_to_hwirq(d) + PCL_RCV_INTX_STATUS_SHIFT);
>>>>>    	writel(val, priv->base + PCL_RCV_INTX);
>>>>> +
>>>>> +	raw_spin_unlock_irqrestore(&pp->lock, flags);
>>>>>    }
>>>>>      static void uniphier_pcie_irq_mask(struct irq_data *d)
>>>>> @@ -199,12 +204,17 @@ static void uniphier_pcie_irq_mask(struct irq_data *d)
>>>>>    	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
>>>>>    	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
>>>>>    	struct uniphier_pcie_priv *priv = to_uniphier_pcie(pci);
>>>>> +	unsigned long flags;
>>>>>    	u32 val;
>>>>>    +	raw_spin_lock_irqsave(&pp->lock, flags);
>>>>> +
>>>>>    	val = readl(priv->base + PCL_RCV_INTX);
>>>>>    	val &= ~PCL_RCV_INTX_ALL_MASK;
>>>>>    	val |= BIT(irqd_to_hwirq(d) + PCL_RCV_INTX_MASK_SHIFT);
>>>
>>> This looks extremely suspicious. You clear all the INTX mask bits, and
>>> only set the one you need. How about the pre-existing bits?
>>
>> Thanks for pointing out. No need to clear all INTX mask bits.
>> The pre-existing bits should be preserved.
>>
>>>
>>>>>    	writel(val, priv->base + PCL_RCV_INTX);
>>>>> +
>>>>> +	raw_spin_unlock_irqrestore(&pp->lock, flags);
>>>>>    }
>>>>>      static void uniphier_pcie_irq_unmask(struct irq_data *d)
>>>>> @@ -212,12 +222,17 @@ static void uniphier_pcie_irq_unmask(struct irq_data *d)
>>>>>    	struct pcie_port *pp = irq_data_get_irq_chip_data(d);
>>>>>    	struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
>>>>>    	struct uniphier_pcie_priv *priv = to_uniphier_pcie(pci);
>>>>> +	unsigned long flags;
>>>>>    	u32 val;
>>>>>    +	raw_spin_lock_irqsave(&pp->lock, flags);
>>>>> +
>>>>>    	val = readl(priv->base + PCL_RCV_INTX);
>>>>>    	val &= ~PCL_RCV_INTX_ALL_MASK;
>>>>>    	val &= ~BIT(irqd_to_hwirq(d) + PCL_RCV_INTX_MASK_SHIFT);
>>>
>>> And by the same token, this second line is totally useless.
>>>
>>> I think masking/unmasking is broken in this driver, locking or not.
>>
>> Yes, this second line should be removed, too.
> 
> You mean the *first* line, right? The one clearing all the INTx
> bits. If you remove the second line, you won't fix anything.
This is ambiguous. I mean that I will remove the following line:

     	val &= ~PCL_RCV_INTX_ALL_MASK;

So the fixed unmasking code is as follows.

     	val = readl(priv->base + PCL_RCV_INTX);
     	val &= ~BIT(irqd_to_hwirq(d) + PCL_RCV_INTX_STATUS_SHIFT);
     	writel(val, priv->base + PCL_RCV_INTX);

Thank you,

---
Best Regards
Kunihiko Hayashi