Fwd: Question abount VSOL/CarlitoxxPro SFP Patch on Marvell Armada (MCBIN DS)
Giammarco lynx
stich86 at gmail.com
Tue Mar 8 02:08:34 PST 2022
Hi Russell,
i've tried to change quirks on the sfp-bus.c, but without success:
root at mcbin:~# dmesg | grep eth3
[ 3.035613] mvpp2 f4000000.ethernet eth3: Using firmware node mac
address 00:51:82:11:22:03
[ 12.115130] sfp sfp-eth3: Host maximum power 2.0W
[ 12.119867] sfp sfp-eth3: tx disable 1 -> 1
[ 12.119873] sfp sfp-eth3: SM: enter empty:detached:down event insert
[ 12.119877] sfp sfp-eth3: SM: exit probe:detached:down
[ 12.120155] sfp sfp-eth3: SM: enter probe:detached:down event dev_attach
[ 12.120159] sfp sfp-eth3: SM: exit probe:down:down
[ 12.420121] sfp sfp-eth3: SM: enter probe:down:down event timeout
[ 12.427901] sfp sfp-eth3: Detected broken RTL8672/RTL9601C emulated EEPROM
[ 12.434811] sfp sfp-eth3: Switching to reading EEPROM to one byte at a time
[ 12.482001] sfp sfp-eth3: module OEM V2801F
rev 1.0 sn 202101195032 dc 210119
[ 12.491796] mvpp2 f4000000.ethernet eth3: requesting link mode
inband/2500base-x with support 0000000,00000200,00008440
[ 12.491799] sfp sfp-eth3: SM: exit present:down:down
[ 12.520104] sfp sfp-eth3: skipping hwmon device registration due to
broken EEPROM
[ 12.527618] sfp sfp-eth3: diagnostic EEPROM area cannot be read
atomically to guarantee data coherency
[ 34.635760] mvpp2 f4000000.ethernet eth3: configuring for
inband/2500base-x link mode
[ 34.643644] mvpp2 f4000000.ethernet eth3: major config 2500base-x
[ 34.649768] mvpp2 f4000000.ethernet eth3: phylink_mac_config:
mode=inband/2500base-x/Unknown/Unknown adv=0000000,00000000,00008440
pause=04 link=0 an=1
[ 34.663452] sfp sfp-eth3: SM: enter present:down:down event dev_up
[ 34.669658] sfp sfp-eth3: tx disable 1 -> 0
[ 34.673893] sfp sfp-eth3: SM: exit present:up:wait
[ 34.740110] sfp sfp-eth3: SM: enter present:up:wait event timeout
[ 34.746236] sfp sfp-eth3: SM: exit present:up:link_up
[ 57.613403] sfp sfp-eth3: los 0 -> 1
[ 57.616998] sfp sfp-eth3: SM: enter present:up:link_up event los_high
[ 57.623474] sfp sfp-eth3: SM: exit present:up:link_up
[ 57.628555] sfp sfp-eth3: mod-def0 1 -> 0
[ 57.632587] sfp sfp-eth3: tx-fault 0 -> 1
[ 57.636615] sfp sfp-eth3: SM: enter present:up:link_up event remove
[ 57.642912] sfp sfp-eth3: module removed
[ 57.647018] sfp sfp-eth3: tx disable 0 -> 1
[ 57.651236] sfp sfp-eth3: SM: exit empty:up:down
[ 57.655872] sfp sfp-eth3: SM: enter empty:up:down event tx_fault
[ 57.661940] sfp sfp-eth3: SM: exit empty:up:down
[ 59.404709] sfp sfp-eth3: tx-fault 1 -> 0
[ 59.408737] sfp sfp-eth3: SM: enter empty:up:down event tx_clear
[ 59.414775] sfp sfp-eth3: SM: exit empty:up:down
[ 59.419417] sfp sfp-eth3: mod-def0 0 -> 1
[ 59.423445] sfp sfp-eth3: tx-fault 0 -> 1
[ 59.427471] sfp sfp-eth3: SM: enter empty:up:down event insert
[ 59.433333] sfp sfp-eth3: SM: exit probe:up:down
[ 59.437968] sfp sfp-eth3: SM: enter probe:up:down event tx_fault
[ 59.444003] sfp sfp-eth3: SM: exit probe:up:down
[ 59.740106] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 61.836667] sfp sfp-eth3: SM: exit probe:up:down
[ 61.940105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 61.946161] sfp sfp-eth3: SM: exit probe:up:down
[ 62.050106] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.056159] sfp sfp-eth3: SM: exit probe:up:down
[ 62.160105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.166158] sfp sfp-eth3: SM: exit probe:up:down
[ 62.270105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.276159] sfp sfp-eth3: SM: exit probe:up:down
[ 62.380105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.386159] sfp sfp-eth3: SM: exit probe:up:down
[ 62.490105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.496160] sfp sfp-eth3: SM: exit probe:up:down
[ 62.600105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.606160] sfp sfp-eth3: SM: exit probe:up:down
[ 62.710105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.716159] sfp sfp-eth3: SM: exit probe:up:down
[ 62.820105] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 62.826159] sfp sfp-eth3: please wait, module slow to respond
[ 62.831965] sfp sfp-eth3: SM: exit probe:up:down
[ 67.910109] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 67.916168] sfp sfp-eth3: SM: exit probe:up:down
[ 72.950108] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 72.956167] sfp sfp-eth3: SM: exit probe:up:down
[ 77.990111] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 77.996175] sfp sfp-eth3: SM: exit probe:up:down
[ 81.427524] sfp sfp-eth3: los 1 -> 0
[ 81.431121] sfp sfp-eth3: tx-fault 1 -> 0
[ 81.435148] sfp sfp-eth3: SM: enter probe:up:down event tx_clear
[ 81.441185] sfp sfp-eth3: SM: exit probe:up:down
[ 81.445820] sfp sfp-eth3: SM: enter probe:up:down event los_low
[ 81.451767] sfp sfp-eth3: SM: exit probe:up:down
[ 83.030114] sfp sfp-eth3: SM: enter probe:up:down event timeout
[ 83.043617] sfp sfp-eth3: Detected broken RTL8672/RTL9601C emulated EEPROM
[ 83.050525] sfp sfp-eth3: Switching to reading EEPROM to one byte at a time
[ 83.097713] sfp sfp-eth3: module OEM V2801F
rev 1.0 sn 202101195032 dc 210119
[ 83.107507] mvpp2 f4000000.ethernet eth3: requesting link mode
inband/2500base-x with support 0000000,00000200,00008440
[ 83.118375] sfp sfp-eth3: tx disable 1 -> 0
[ 83.122621] sfp sfp-eth3: SM: exit present:up:wait
[ 83.127435] sfp sfp-eth3: skipping hwmon device registration due to
broken EEPROM
[ 83.134954] sfp sfp-eth3: diagnostic EEPROM area cannot be read
atomically to guarantee data coherency
[ 83.180105] sfp sfp-eth3: SM: enter present:up:wait event timeout
[ 83.186242] sfp sfp-eth3: SM: exit present:up:link_up
root at mcbin:~#
Please note that this stick should supporto HiSGMII (2500base-x), but
currently the firmware on-top seems stuck so link only at 1000base-x
At the moment we don't have any other solution than put interface UP
and then un-plug/plug module again?
Thanks,
Giammarco
Il giorno mar 8 mar 2022 alle ore 00:01 Russell King (Oracle)
<linux at armlinux.org.uk> ha scritto:
>
> On Mon, Mar 07, 2022 at 09:12:45PM +0100, Giammarco lynx wrote:
> > Il giorno lun 7 mar 2022 alle ore 21:09 Giammarco lynx
> > <stich86 at gmail.com> ha scritto:
> > > Then after "ifconfig eth3 up" i've removed and reinserted the SFP,
> > > this is the log:
> > >
> > > [ 56.127227] sfp sfp-eth3: module removed
> > > [ 56.131175] sfp sfp-eth3: tx disable 0 -> 1
>
> TX_DISABLE asserted.
>
> > > [ 56.135432] sfp sfp-eth3: SM: exit empty:up:down
> > > [ 56.140067] sfp sfp-eth3: SM: enter empty:up:down event tx_fault
> > > [ 56.146103] sfp sfp-eth3: SM: exit empty:up:down
> > > [ 59.350731] sfp sfp-eth3: mod-def0 0 -> 1
>
> Module inserted, and starts booting...
>
> > > [ 59.354757] sfp sfp-eth3: SM: enter empty:up:down event insert
> > > [ 59.360619] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 59.680123] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 59.686185] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 59.800122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 59.806179] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 59.920122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 59.926179] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.040122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.046178] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.160123] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.166178] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.270122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.276177] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.380122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.386178] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.500122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.506177] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.620121] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.626176] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 60.740122] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 60.746177] sfp sfp-eth3: please wait, module slow to respond
> > > [ 60.751982] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 65.840198] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 65.846262] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 70.870128] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 70.876192] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 75.920133] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 75.926196] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 80.896801] mvpp2 f4000000.ethernet eth3: mac link up
>
> Link established with the host MAC at 1G, probably as part of the
> module detecting what speed the host is operating at.
>
> > > [ 80.960124] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 80.966183] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 81.330385] sfp sfp-eth3: los 1 -> 0
> > > [ 81.333978] sfp sfp-eth3: tx-fault 1 -> 0
>
> Some of the modules IOs change state.
>
> > > [ 81.338004] sfp sfp-eth3: SM: enter probe:up:down event tx_clear
> > > [ 81.344044] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 81.348680] sfp sfp-eth3: SM: enter probe:up:down event los_low
> > > [ 81.354629] sfp sfp-eth3: SM: exit probe:up:down
> > > [ 86.000154] sfp sfp-eth3: SM: enter probe:up:down event timeout
> > > [ 86.013661] sfp sfp-eth3: Detected broken RTL8672/RTL9601C emulated EEPROM
>
> Module emulated EEPROM becomes readable about 27 seconds after
> insertion Firmware probably almost finished booting.
>
> > > [ 86.020569] sfp sfp-eth3: Switching to reading EEPROM to one byte at a time
> > > [ 86.067758] sfp sfp-eth3: module OEM V2801F
> > > rev 1.0 sn 202101195032 dc 210119
> > > [ 86.077555] mvpp2 f4000000.ethernet eth3: requesting link mode
> > > inband/1000base-x with support 0000000,00000200,00000440
> > > [ 86.088390] sfp sfp-eth3: tx disable 1 -> 0
>
> We've now detected what the module is, and we configure for it, which
> the EEPROM indicates it supports 1200Mbaud, which is 1G speed.
>
> > > [ 86.092636] sfp sfp-eth3: SM: exit present:up:wait
> > > [ 86.097448] sfp sfp-eth3: skipping hwmon device registration due to
> > > broken EEPROM
> > > [ 86.104965] sfp sfp-eth3: diagnostic EEPROM area cannot be read
> > > atomically to guarantee data coherency
> > > [ 86.160128] sfp sfp-eth3: SM: enter present:up:wait event timeout
> > > [ 86.166252] sfp sfp-eth3: SM: exit present:up:link_up
>
> sfp.c determines link up...
>
> > > [ 86.171404] mvpp2 f4000000.ethernet eth3: Link is Up - 1Gbps/Full -
> > > flow control off
> > > [ 86.179193] IPv6: ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
>
> Network device now indicates that link has been established at 1G
> speed.
>
> > > In that way it goes up. So how can we avoid this behaviour?
>
> Very difficult to avoid it. My guess is that when you power up the
> system with the module inserted, it sees the interface operating at
> 2.5G speed and locks itself to 2.5G speed.
>
> Eventually, our kernel boots, and sfp reads the EEPROM. The EEPROM
> indicates that it supports 1G speed, so we switch the host interface
> to 1000base-X, and that causes link to be lost (although I don't see
> any sign of that happening in your debug.)
>
> That means that the module has decided to use 2.5G speed, but its
> reporting it supports 1G speed in the EEPROM... so its doing something
> different. No surprises that the link fails to come up.
>
> We could add an entry to the sfp_quirks[] table in sfp-bus.c for this
> module using the sfp_quirk_2500basex function to also indicate it
> supports 2.5G speed:
>
> }, {
> // VSOL/CarlitoxxPro SFP can also work at 2.5G speed
> .part = "V2801F",
> .modes = sfp_quirk_2500basex,
>
> However, we are always going to be stuck in the situation that this is
> going to be unreliable. Whatever speed the network interface is in when
> this module is inserted, it's going to lock to that speed. If we then
> change it later, we'll never link.
>
> A switch will likely only ever use one speed on its SFP port, meaning
> it will always present 1000base-X, or it will always present 2500base-X,
> so it will always work with a switch, but not a more capable MAC
> interface that can switch between these at will.
>
> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
More information about the linux-arm-kernel
mailing list