[PATCH v5 2/2] PCI: quirks: Fix ThunderX2 dma alias handling

[Jayachandran C]

On Cavium ThunderX2 arm64 SoCs (called Broadcom Vulcan earlier), the
PCI topology is slightly unusual.  For a multi-node system, it looks
like:

    00:00.0 [PCI] bridge to [bus 01-1e]
    01:0a.0 [PCI-PCIe bridge, type 8] bridge to [bus 02-04]
    02:00.0 [PCIe root port, type 4] bridge to [bus 03-04] (XLATE_ROOT)
    03:00.0 PCIe Endpoint

pci_for_each_dma_alias() assumes IOMMU translation is done at the root
of the PCI hierarchy.  It generates 03:00.0, 01:0a.0, and 00:00.0 as
DMA aliases for 03:00.0 because buses 01 and 00 are non-PCIe buses that
doesn't carry the Requester ID.

Because the ThunderX2 IOMMU is at 02:00.0, the Requester IDs 01:0a.0
or 00:00.0 are never valid for the endpoint.  This quirk stops alias
generation at the XLATE_ROOT bridge so we won't generate 01:0a.0 or
00:00.0

The current IOMMU code only maps the last alias (this is a separate bug
in itself).  Prior to this quirk, we only created IOMMU mappings for the
invalid Requester ID 00:00:0, which never matched any DMA transactions.

With this quirk, we create IOMMU mappings for a valid Requester ID,
which fixes devices with no aliases but leaves devices with aliases
still broken.

The last alias for the endpoint is also used by the ARM GICv3 MSI-X
code.  Without this quirk, the GIC Interrupt Translation Tables are
setup with the invalid Requester ID, and the MSI-X generated by the
device fails to be translated and routed.

Signed-off-by: Jayachandran C <jnair at caviumnetworks.com>
Reviewed-by: Robin Murphy <robin.murphy at arm.com>
Acked-by: David Daney <david.daney at cavium.com>
---
 drivers/pci/quirks.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 6736836..564a84a 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3958,6 +3958,20 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias);
 
 /*
+ * The IOMMU and interrupt controller on Broadcom Vulcan/Cavium ThunderX2 are
+ * associated not at the root bus, but at a bridge below. This quirk flag
+ * will ensure that the aliases are identified correctly.
+ */
+static void quirk_bridge_cavm_thrx2_pcie_root(struct pci_dev *pdev)
+{
+	pdev->dev_flags |= PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT;
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000,
+				quirk_bridge_cavm_thrx2_pcie_root);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084,
+				quirk_bridge_cavm_thrx2_pcie_root);
+
+/*
  * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero)
  * class code.  Fix it.
  */
-- 
2.7.4