[PATCH] arm64: Add architecture support for PCI
arnd at arndb.de
Tue Feb 4 04:44:52 EST 2014
On Monday 03 February 2014 16:31:37 Jason Gunthorpe wrote:
> Specifying 'use EHCI, AHCI, etc' - which are all PCI based standards
> without clearly specifying exactly how PCI is suppose to work is
> completely bonkers.
> What is needed is a spec that says:
> 1) Here is how you generate config TLPs. A MMIO region that
> conforms to the already specified x86 ECAM would
> be perfect
> 2) Here is a dword by dword break down of the entire config space in
> a SOC. Here is where a on-board AHCI controller must show up in
> config space. Here is how an external PCI-E port must show
> up. Etc. Most of this is already specified, but it clearly needs
> to be layed out explicitly for ARM SOCs to actually follow it.
> 3) Here is how you specify the aperture(s) associated with PCI BAR's
> and bridge windows in config space. And yes: The CONFIG SPACE
> BARS MUST WORK.
> 4) Here is how MSI works, these are the values you put in the
> address/data and here is how you collect the interrupt.
> 5) Here is how Legacy INTx must be mapped into the GIC.
> This is what x86 does, and they have been doing it well for 10
> years. If you want to play in the server game you have to properly
> implement PCI.
I'm pretty sure the authors of the SBSA actually thought that was
what they wrote, by referring to external specifications (pci-3.0,
ehci, ahci, ...). However, it seems they were either foolish enough
to believe that hardware designers would follow these specs, or
they were intentionally misled and got talked into putting ambiguous
terminology in because there were already hardware designs that
are not exactly in line with the spirit of the SBSA but can be
argued to be conforming to the text for a lax interpretation.
I think EHCI is a much better example than PCI here, because the
spec has exactly one line to say about it, where it spends a whole
chapter on PCI.
Here is how a sane person would read SBSA to create a compliant
I have to use EHCI version 1.1 to provide USB-2.0 support. EHCI
is a PCI device, so I'll put it behind a PCIe port that complies
to the PCIe section of the SBSA. Since EHCI by itself only provides
high-speed USB, and USB-2.0 mandates I provide low-speed and
full-speed as well, I have to add a USB hub device. It would have
been easier to just use OHCI for these, but SBSA says I can't.
Now I want to integrate the EHCI into my SoC and not waste one
of my precious PCIe root ports, so I have to create another PCI
domain with its own ECAM compliant config space to put it into.
Fortunately SBSA lets me add an arbitrary number of PCI domains,
as long as they are all strictly compliant. To software it will
look exactly as if it was on an external card, I just have to
ensure the boot loader correctly sets up the clocks and the phy
before an SBSA compliant OS gets loaded, all runtime power
management will get handled through the EHCI-1.1 energy-efficiency
Here is how a crazy person would read the same sentence in the SBSA:
I have an IP block that implements the EHCI register set, that
should be good enough. It's not a fast device, so I can put it
on a non-coherent bus. Since the SoC will be used for networking,
I'll put the registers into big-endian configuration to make it
easier for the OS to access. I'm not allowed to have USB-1.1
according to SBSA, so I can get away without a hub or an extra
OHCI. I can't support MSI because it's not a PCI device, and
the GIC is pretty full, so I'll just connect the IRQ line to
the GPIO controller. In order to do better power management,
I'll design a fancy PHY that the device driver will manage
for implementing autosuspend. I should also give the OS
fine-grained control over the clocks, but it will have to share
the clock domain with the other devices on the same edge of the
chip. The EHCI device is not part of PCI, which measn I don't
have to use the standard SMMU. However, my EHCI implementation
can only do 32-bit DMA, and I'll have to design my own IOMMU
to let it access the entire memory range. USB-OTG is a great
feature and we already paid for having this in our EHCI
implementation, better make sure it comes up in endpoint mode
after waking up from power saving.
More information about the linux-arm-kernel