Intel I350 mini-PCIe card (igb) on Mirabox (mvebu / Armada 370)

[Willy Tarreau]

Hi Neil,

On Fri, Apr 04, 2014 at 02:19:44PM +0100, Neil Greatorex wrote:
> With this patch, I can get one port on the card working. With both ports 
> enabled, I still get an OOPS, so some further work is needed.

Concerning this point, here's an update on my side. I found where the crash
happens, but I don't exactly understand why, I suspect that an area is not
correctly mapped :

root at xpgp:~# insmod /tmp/igb.ko 
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
igb: Copyright (c) 2007-2013 Intel Corporation.
PCI: enabling device 0000:00:09.0 (0140 -> 0143)
PCI: enabling device 0000:02:00.0 (0140 -> 0142)
request_region(pdev=edb21000, 00000049)
hw_addr = pci_iomap(pdev=edb21000, 0, 0) = f0300000
mem_start=e0000000 mem_end=e007ffff
hw_addr = f0300000
hw_addr=f0300000
igb 0000:02:00.0: added PHC on eth4
igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:02:00.0: eth4: (PCIe:5.0Gb/s:Width x1) 00:30:18:a6:6c:6a
igb 0000:02:00.0: eth4: PBA No: FFFFFF-0FF
igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
PCI: enabling device 0000:02:00.1 (0140 -> 0142)
request_region(pdev=ed99d800, 00000049)
hw_addr = pci_iomap(pdev=ed99d800, 0, 0) = f0400000
mem_start=e0100000 mem_end=e017ffff
hw_addr = f0400000
hw_addr=f0400000
Unhandled fault: external abort on non-linefetch (0x1008) at 0xf0400018
Internal error: : 1008 [#1] SMP THUMB2

in e1000_82575.o:igb_get_invariants_82575() :
    123c:       f884 33b4       strb.w  r3, [r4, #948]
    1240:       f884 33b9       strb.w  r3, [r4, #953]
    1244:       f8c4 6300       str.w   r6, [r4, #768]
    1248:       6863            ldr     r3, [r4, #4]
=>  124a:       f8d3 8018       ldr.w   r8, [r3, #24]

in e1000_82575.c:igb_get_invariants_82575() :
        hw->phy.media_type = e1000_media_type_copper;
        dev_spec->sgmii_active = false;
        dev_spec->module_plugged = false;

here=>  ctrl_ext = rd32(E1000_CTRL_EXT);

according to e1000_hw.h:

  #define E1000_CTRL_EXT 0x00018      /* Extended Device Control - RW */

according to e1000_regs.h:

  #define rd32(reg) (readl(hw->hw_addr + reg))

===> ctrl_ext = readl((hw->hw_addr = 0xf0400000) + (reg = 0x18))

As you can see, the sequence is exactly the same for both ports. The
first one has no problem performing the readl(), but the second one
cannot. Both of them got the memory address returned by a call to
pci_iomap(dev, 0, 0). I could verify that the pci_resource_len() for
both is 524288 (0x80000).

The last "hwaddr=f0400000" is printed just before calling rd32() and
shows that it was still OK there. Since the resource flags are 0x40200,
we only have IORESOURCE_MEM so pci_iomap() calls ioremap_nocache().

Thus I suspect something is not behaving correctly in the code which
configures the emulated bridge and/or the memory areas, resulting in
the second port not being correctly mapped, thus causing the crash.

But that's the deepest I can go unfortunately, I got lost after that.

Regards,
Willy