[PATCH v7 01/10] PCI: host: rcar: Add Renesas R-Car PCIe driver

[Phil Edworthy]

Hi Bjorn,

On 01 May 2014 18:31, Bjorn wrote:
> [+cc Srikanth, Michal, Grant because we're having the same discussion
> about root bus numbers in the Xilinx host bridge driver]
> 
> On Thu, May 1, 2014 at 3:50 AM, Phil Edworthy
> <phil.edworthy at renesas.com> wrote:
> 
> > When would the PCI core change the root bus number to something other
> than set in sys->busnr?
> 
> The PCI core never changes the root bus number of a host bridge, but
> it's important for the core to know exactly what bus numbers are
> behind each host bridge so we can correctly assign bus numbers, MMIO,
> and I/O port resources.
> 
> The reason is that when we enumerate a PCI-to-PCI bridge, we have to
> know whether bus numbers are available for the secondary bus.  For
> example, assume two host bridges configured (or wired) as follows:
> 
>   A leads to [bus 00-1f], with aperture [mem 0x80000000-0x8fffffff]
>   B leads to [bus 20-ff], with aperture [mem 0x90000000-0x9fffffff]
> 
> Further, assume a PCI-to-PCI bridge at 1f:00.0.  If the host bridge
> driver doesn't tell us the [bus 00-1f] range for host bridge A, the
> PCI core will assume [bus 00-ff].  When the core enumerates the
> 1f:00.0 bridge, it will assign bus number 20 to its secondary bus.
> Now it enumerates devices on bus 20.  The core expects this to be done
> via host bridge A, but enumeration uses ID-routed config transactions,
> so they are claimed by host bridge B instead.  If we find a device
> 20:00.0, the core thinks it is below A, but it is really below B.
> 
> Now we assign resources to 20:00.0, and since we think it's below A,
> we assign them from the [mem 0x80000000-0x8fffffff] aperture.  This
> obviously doesn't work, because when the driver performs MMIO
> accesses, which are address-routed, they are claimed by host bridge A,
> not B, so the 20:00.0 device never sees them.
Thanks for the clear explanation.

> If your host bridge really claims ALL bus numbers (00-ff), that's
> fine, and you can tell the PCI core that.  But if you have several
> such bridges, they should be placed in separate domains.
That makes sense.

In the case of the R-Car PCIe host bridge, as far as I can tell, the hardware doesn’t care about bus numbers at all, with the exception that the driver uses one bus number to target the bridge itself. We can use as many or as few bus numbers for the secondary bus as you like.

So, the R-Car PCIe driver should implement bus ranges, but should it also use separate PCI domains for multiple controllers?

> This is probably more detail than you wanted, and maybe you don't ever
> expect to have multiple host bridges or complicated hierarchies of
> devices.  But the PCI core has to deal with these issues on other
> systems, so it's important to get these details right.

Thanks
Phil