orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)
Dieter Kiermaier
dk-arm-linux at gmx.de
Wed Nov 11 11:50:35 EST 2009
Am Mittwoch 11 November 2009 16:21:34 schrieb Lennert Buytenhek:
> On Wed, Nov 11, 2009 at 06:29:22AM -0800, Dieter Kiermaier wrote:
>
> > > > > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > > > > address 0xf1020100 to enable bus scanning.
> > > > > I don't know what this really does but it helped to get my kernel up and running.
> > > >
> >
> > I've tested again:
> >
> > with 2.6.22.18 marvell stock kernel my fpga behing the pcie->pci bridge works as expected.
> > Here is the output of lscpi:
> >
> > sh-3.2# uname -a
> > Linux DB88FXX81 2.6.22.18 #1 Wed Apr 22 20:31:28 IST 2009 armv5tejl GNU/Linux
> > sh-3.2#
> >
> >
> > sh-3.2# lspci -vv
> > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > Latency: 0, Cache Line Size: 32 bytes
> > Bus: primary=00, secondary=01, subordinate=01, sec-latency=240
> > I/O behind bridge: 00001000-00004fff
> > Memory behind bridge: e8000000-ebffffff
> > Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> > BridgeCtl: Parity+ SERR- NoISA+ VGA- MAbort+ >Reset- FastB2B-
> > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> > Capabilities: [40] Power Management version 2
> > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> > Bridge: PM- B3+
> > Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
> > DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> > ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> > DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> > RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
> > MaxPayload 128 bytes, MaxReadReq 512 bytes
> > DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
> > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
> > ClockPM- Surprise- LLActRep- BwNot-
> > LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
> > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> > Capabilities: [100] #11ab
> > Capabilities: [221] #f0e8
> > Capabilities: [f1e] #2a0
> > Capabilities: [e80] #187
> >
> > 01:08.0 Class ff00: Device 1731:0101 (rev 10)
> > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> > Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> > Region 0: Memory at e8000000 (32-bit, non-prefetchable) [size=64M]
> > Region 1: I/O ports at <ignored>
> > Kernel driver in use: ArtistaNET-III frame buffer driver
> >
> >
> > If I switch to mainline linux kernel (there is no difference between openrd, vanilla, orion.git kernels!) my problems start:
> > I have to patch openrd_init() as follows:
> >
> > diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > index 77617c7..670312b 100644
> > --- a/arch/arm/mach-kirkwood/openrd_base-setup.c
> > +++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > @@ -14,6 +14,7 @@
> > #include <linux/mtd/partitions.h>
> > #include <linux/ata_platform.h>
> > #include <linux/mv643xx_eth.h>
> > +#include <asm/io.h>
> > #include <asm/mach-types.h>
> > #include <asm/mach/arch.h>
> > #include <mach/kirkwood.h>
> > @@ -76,9 +77,21 @@ static void __init openrd_base_init(void)
> >
> > static int __init openrd_base_pci_init(void)
> > {
> > + u32 cpu_config_reg;
> > + void __iomem *base;
> > +
> > + base = ioremap(0xf1020100, 4);
> > + if (base)
> > + {
> > + cpu_config_reg = readl(base);
> > + cpu_config_reg &= ~(1 << 2);
> > + writel(cpu_config_reg, base);
> > + printk("register 0x20100: %x\n", readl(base));
> > + }
> > + iounmap(base);
> > if (machine_is_openrd_base())
> > kirkwood_pcie_init();
> > -
> > +
> > return 0;
> > }
> > subsys_initcall(openrd_base_pci_init);
> > ----
> > to allow my system to boot up
>
> What's likely happening is that your boot loader either enables or
> does not disable this bit, and the 2.6.22.18 kernel disables the bit,
> while the upstream kernel leaves the bit untouched.
>
> What this bit does is to decide whether or not aborts on the PCI
> interface are translated into processor aborts. It's not really
> necessary to have this enabled, as the transaction will return
> 0xffffffff to the CPU anyway, which is then handled appropriately
> as well.
>
> What uboot version are you using? The uboot versions I have on my
> Kirkwood boards all jump to the OS with this bit already cleared.
> Perhaps we should clear it explicitly from Linux.
>
>
> > and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> > I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.
>
> The device shows up on the bus, the bridge primary/secondary bus numbers
> look good, and the secondary memory address range on the bridge looks
> properly programmed. It all looks good to me.
>
> What do you mean by 'not reachable'? I'm guessing that you're trying
> to access the memory BAR on the 01:08.0 device directly from userland
> by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
> and only getting 0xffffffff back because you don't have an actual kernel
> driver for this FPGA board and thus you're not calling pci_enable_device()
> on your device, causing MEM/IO decoding not to have been enabled on
> the device as seems to be the case in your dump?
>
I've tested once again (I really forgotten to load my kernel module at the last mail, sorry again!).
Here you can see the result (with a non working driver!);
sh-3.2# uname -a
Linux DB88FXX81 2.6.32-rc6-00014-g92bdfb9-dirty #3 PREEMPT Wed Nov 11 15:10:49 CET 2009 armv5tel GNU/Linux
sh-3.2# cat /proc/iomem
00000000-0fffffff : System RAM
00026000-00464fff : Kernel text
0048a000-00579c7b : Kernel data
10000000-1fffffff : System RAM
e0000000-e7ffffff : PCIe Memory Space
e0000000-e3ffffff : PCI Bus 0000:01
e0000000-e3ffffff : 0000:01:08.0
e0000000-e00fffff : mv_video MMIO
f1010300-f101030f : rtc-mv
f1010300-f101030f : rtc-mv
f1011000-f101101f : mv64xxx_i2c.0
f1011000-f101101f : mv64xxx_i2c adapter
f1012000-f10120ff : serial8250.0
f1012000-f101201f : serial
f1030000-f103ffff : regs
f1050000-f1050fff : orion-ehci.0
f1050000-f1050fff : ehci_hcd
f1060800-f10608ff : xor 0 low
f1060900-f10609ff : xor 1 low
f1060a00-f1060aff : xor 0 high
f1060b00-f1060bff : xor 1 high
f1072000-f1073fff : ge00 base
f1080000-f1084fff : sata base
f1090000-f10903ff : mvsdio
f1090000-f10903ff : mvsdio
f3000000-f30003ff : orion_nand
f4000000-f40007ff : sram
sh-3.2# lspci -vv
00:00.0 Class 0580: Device 11ab:6281 (rev 02)
Subsystem: Device 11ab:11ab
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 9
Region 0: Memory at <ignored> (64-bit, prefetchable)
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [60] Express (v1) Root Port (Slot-), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 unlimited
ClockPM+ Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
Capabilities: [100] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
00:01.0 Class 0604: Device 11ab:2211 (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR+ <PERR- INTx-
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 00001000-00004fff
Memory behind bridge: e0000000-e3ffffff
Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort+ >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+
Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
Capabilities: [100] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
01:08.0 Class ff00: Device 1731:0101 (rev 10)
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Region 0: Memory at e0000000 (32-bit, non-prefetchable) [size=64M]
Region 1: I/O ports at <unassigned> [disabled]
Kernel driver in use: ArtistaNET-III frame buffer driver
sh-3.2#
sh-3.2# lsmod
mv_video 3616 0 - Live 0xbf000000
sh-3.2#
Again, many thanks for helping!
Dieter
> For what it's worth, I have various mv78xx0 and Kirkwood boards with
> 88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
> the devices behind those bridges do too.
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
More information about the linux-arm-kernel
mailing list