[PATCH v2 22/27] arm: mvebu: add PCIe Device Tree informations for Armada XP
Jason Gunthorpe
jgunthorpe at obsidianresearch.com
Wed Feb 6 20:05:02 EST 2013
On Wed, Feb 06, 2013 at 10:41:14PM +0000, Arnd Bergmann wrote:
> This would mean that rather than putting the mapped physical address
> (0xc0000000, 0xc1000000, ...) in here, you would actually have 64-bit
> address as the destination as well, in whatever format the
> address map hardware uses, I assume using a numbered 32 bit
> address space for each object that can be remapped.
Do you mean like:
mbus_matrix {
/* At this level addresses are
<32 bit MBUS target ID> <32 bit target address offset>
target_id is an internal MBUS specification
*/
#address-cells = <2>;
#size-cells = <1>;
pex0 {
ranges = <
// CPU window bridged to PCI MMIO
0x82000000 0x00000000 0x00000000 0x<target_id> 0x0 0x0 0x8000000
// CPU IO window bridged to PCI IO
0x81000000 0x00000000 0x00000000 0x<target_id> 0x0 0x0 0xa0000
...>;
}
nand {
reg = <0x<target_id> 0 0x200>;
};
internal_regs {
ranges = <0 0x<target_id> 0 0x10000>;
timer {
reg = <0x20300 0x20>;
};
};
}
?
I think the big issue would go back to how to pool all the link
decoders together in a way that fits OF and Linux's PCI core will
understand?
How does of_translate_address work in a world like this?
> This would also let you do the PCI memory address assignment for
> each port separately, starting at bus address 0, followed by
> finding a location in the CPU address space and passing
> the start as the sys->mem_offset argument to
> pci_add_resource_offset.
That goes back to the original problem - the goal is to have only one
pci_sys_data, not one for every link.
The host driver would have to request a large region of physical
address space and still dole it out on a link by link basis. Not sure
how to model that in DT??
In any event, changing how all the dynamic windows are configured in
DT is a big job (there was another thread about this) it seems
orthogonal to the PCI host driver..
> > + pcie at 0,0 {
> > + device_type = "pciex";
> > + reg = <0x0800 0 0xd0040000 0 0x2000>;
> > + #address-cells = <3>;
> > + #size-cells = <2>;
> > + marvell,pcie-port = <0>;
> > + marvell,pcie-lane = <0>;
> > + interrupts = <1>;
> > + clocks = <&gateclk 5>;
> > + status = "disabled";
> > + };
>
> I think you are missing a "ranges" property here, at least an empty
> one, which is required by the standard but not currently enforced
> in the code.
Maybe.. according to the standard the ranges in this stanza should
reflect the bridge configuration, but that isn't known when the DT is
written. An empty ranges means identity and that isn't really right
either.
Also, what should 'reg' be so that the PCI core binds the OF nodes
properly? The standard says reg should have the configuration space
address of the bridge, and I noticed Thierry was using something that
almost looked like a config space address in his driver..
But that seems overly tricky.. When using the link stanzas, shouldn't
this scheme be more like this:
// The PCI-E host bridge
pex at 0 {
device_type = "pciex"; // <<-- Important!!
ranges = <
// Driver internal control registers in MMIO space
0x82000000 0x10000000 0xd0040000 0xd0040000 0x0 0x8000000
// CPU 0xe0000000 -> e8000000 bridged to PCI MMIO
0x82000000 0x00000000 0xe0000000 0xe0000000 0x0 0x8000000
// CPU IO bus 0x0000000 -> 0xa0000 bridged to PCI IO
0x81000000 0x00000000 0x00000000 0x00000000 0x0 0xa0000
// CPU 0xe8000000 -> f0000000 bridged to PCI MMIO, prefetchable
0xC2000000 0x00000000 0xe8000000 0xe8000000 0x0 0x8000000
>;
//...
// PCI-PCI bridge to Physical link 0
pcie at 0,0 {
device_type = "pciex";
// The configuration space address of the PCI-PCI bridge required by OF
reg = <0x80 0 0 0 0>; // Bus 0, Dev 0x10, Fn 0
/* The 'bar' on the PCI-PCI bridge, maps to internal control
registers, required by the driver. */
assigned-addresses = <0x82000000 0x10000000 0xd0040000 0 0x2000>;
// ..
}
}
Thierry, what do you think?
I struggled with this particular area of the OF system recently and
Grant Likely was very helpful - the above is based on that
discussion..
======
Thomas, here is one possible alternate idea, not sure on the merit,
just including it because it is close to what I've got in my DT right
now..:
// The entire multi-lane controller and PCI root bus
pex at 0 {
device_type = "pciex";
/* This section configures the Linux PCI host driver. Each line is a
physical PCI-Link. (Erorrs included :) */
compatible = "marvell,armada-370-xp-pcie";
regs = <0xd0040000 0x00002000 // port 0.0
0xd0042000 0x00002000 // port 2.0
0xd0044000 0x00002000 // port 0.1
0xd0048000 0x00002000 // port 0.2
0xd004C000 0x00002000>; // port 0.3
io-cpu-window = <0xc0000000 0xa0000>;
interrupts = <58 59 60 61 99>;
clocks = <&gateclk 5
&gateclk 6
&gateclk 7
&gateclk 8
&gateclk 26>
marvell,port-lane <0 0
0 1
0 2
0 3
2 0>;
/* Below here configures the aggregate PCI bus the PEX0 stanza
describes */
#address-cells = <3>;
#size-cells = <2>;
// CPU resources allocated to this PCI host bridge
bus-range = <0x00 0xff>;
ranges = <
// CPU 0xe0000000 -> e8000000 bridged to PCI MMIO
0x82000000 0x00000000 0xe0000000 0xe0000000 0x0 0x8000000
// CPU IO bus 0x0000000 -> 0xa0000 bridged to PCI IO
0x81000000 0x00000000 0x00000000 0x00000000 0x0 0xa0000
// CPU 0xe8000000 -> f0000000 bridged to PCI MMIO, prefetchable
0xC2000000 0x00000000 0xe8000000 0xe8000000 0x0 0x8000000
>;
/* Any PCI devices associated with this bus go here, relative to
the above ranges. As example: */
link at 0 {
device_type = "pciex";
// Root port at bus 0, device 0x10, function 0
reg = <0x00000080 0 0 0 0>;
ep {
device_type = "pciex";
// End port on bus 1, device 0, function 0
reg = <0x00000100 0 0 0 0>;
}
}
}
Thoughts:
- 'regs' in the main stanza, keep to CPU addresses instead of
confusing translated fake ranges address
- each regs line is matches to an interrupt, clock and port-lane
line to describe a link. The above describes 5 links.
- The CPU physical address window to use for the IO space
is set via io-cpu-window, not much choice here, the PCI
format ranges must be 0 based.
- It is not necessary to have per-root port stanzas at all
with the above.
- There is only one PCI bus stanza, the top level.
Jason
More information about the linux-arm-kernel
mailing list