Intel I350 mini-PCIe card (igb) on Mirabox (mvebu / Armada 370)

[Jason Gunthorpe]

On Fri, Mar 28, 2014 at 01:03:55AM +0000, Neil Greatorex wrote:

> With the help you've given, I think I've identified what the problem
> is. It is basically what Thomas suggested the problem was (that he'd
> seen with the Armada 38x). The call to mvebu_pcie_set_local_dev_nr
> causes the PCIe link to reset. In my tests with this Intel card, it
> takes ~25ms for the link to go down, and then another ~13ms for it
> to come back up.

Wow, fascinating - not only does the link go down, but it takes 25ms,
and it seemed based on the patch I sent you, that the first config op
triggers this reset immediately..

If I have time I'll look in the spec, maybe the devnumber is encoded
in low level flow control packets or something that makes sense of
this..

What does your kernel boot log look like? Does the USB PEX link behave
the same?? Presumably it happens faster?

A different option would be to force the link down, then change the
dev number, then bring it back up. At least that way the process is
under the control of the driver and we are not waiting for the
mysterious 25 ms. Or maybe change the dev number, then trigger a
link reset...

> I have put together a patch below that I would be interested to get
> your opinions on and also for you to test against any cards you may
> have. I only have the one mini PCIe card so it's not a great test!

Unfortunately my systems here never boot with the PEX up so I cannot
easily verify..

One suggestion about the patch I have is to move everything you added
into the mvebu_pcie_set_local_dev_nr function and bail if the
devnumber is already correct. That way the bootloader could set it
properly and avoid this.

Regards,
Jason