Lynx 10G SerDes Driver on my kernel

[Tanjeff Moos]

On 12/3/25 19:00, Vladimir Oltean wrote:
> On Wed, Dec 03, 2025 at 05:25:46PM +0100, Tanjeff Moos wrote:
>>> Each SerDes protocol requires a different clock net frequency:
>>> - LANE_MODE_1000BASEX_SGMII requires PLLnCR0_FRATE_5G
>>> - LANE_MODE_2500BASEX requires PLLnCR0_FRATE_3_125G
>>> - LANE_MODE_10GBASER requires PLLnCR0_FRATE_5_15625G
>>> https://urldefense.com/v3/__https://github.com/nxp-qoriq/linux/blob/lf-6.12.34-2.1.0/drivers/phy/freescale/phy-fsl-lynx-10g.c*L2273-L2298__;Iw!!I9LPvj3b!C2MhsRe43DSJeXGT0xpGpqZ8TOnWlFtW1VY2DPka6a9H7t5QWgCMneu8mRvQGeR1ZcaSQidiREblHfFkXcKclJkqX_1ePg$ 
>>
>> According to LS1046ARM [1] SGMII requires 1.25 Gbps. I don't understand
>> where the 5 GHz comes from. I believe I am missing some knowledge here.
> 
> The clock net is an internal frequency provided by the PLL for lanes.
> It is given by PLLnCR0[FRATE_SEL]. I said that the lane can further
> divide or multiply it using a power of 2 factor.
> 
> Indeed SGMII wants a 1.25 Gbps baud rate, and it gets that by dividing
> the 5 GHz clock net provided by the PLL (PLLnCR0_FRATE_5G) by 4 (it uses
> RAT_SEL_QUARTER for LNaGCR0_TRAT_SEL and LNaGCR0_RRAT_SEL).
> 
> Because we _know_ that lanes can divide the 5 GHz clock net frequency by
> 4 to get 1.25, we say that a PLL which uses PLLnCR0_FRATE_5G is good for
> SGMII.
> 
>>> The problem is that 2 PLLs can't provide 3 distinct clock net frequencies.
>>> So you can either support 1G and 2.5G, or 1G and 10G, or 2.5G and 10G.
1G and 10G is sufficient. 1G (SGMII) also supports 100M. 10G (XFI) also supports
2.5G and 5G via rate limiting.

>>
>> According to table 31-4 "Valid SerDes RCW Encodings and Reference Clocks"
>> (page 1919):
>> - SGMII can be used with PLL=100 MHz or PLL=125 MHZ
>> - 2.5G SGMII can be used with PLL=125 MHz or PLL=156.25 MHZ
>> - XFI can be used with PLL=156.25 MHZ
>> My clocks receive the following frequencies (on all devices, "fast" and
>> "slow"):
>> - PLL 1: 125 MHz
>> - PLL 2: 156.25 MHz
>> By choosing the right PLL I can use SGMII, 2.5G SGMII or XFI, which I
>> actually do. It is possible to freely assign PLL 1 or PLL 2 to any lane
>> using LNxGCR0[RPLL_LES].
> 
> Oof, this is hard to explain, let me try. The short summary is: you can
> use the 125 MHz PLL refclk for both SGMII and 2.5G SGMII, but not at the
> same time.
> 
> The clock net frequency is derived from the PLL reference clock, but
> they are not one and the same thing. Their relationship is:
> - From one reference clock you can provide one of two clock net
>   frequencies.
> - One clock net frequency can be provided from different reference clock
>   frequencies.
> 
> The entity that loads the RCW configures the SerDes PLLs in this way.
> 
> It knows the SerDes protocols per lane, and their respective mappings to
> a PLL or the other, through RCW[SRDS_PRTCL_S1].
> 
> 1. It looks at PLLF. Based on which SerDes protocols are mapped to it, it
> programs the clock net frequency (PLLnCR0[FRATE_SEL]) at the right value
> for those protocols. It is a *fact* that SGMII requires an FRATE_SEL of
> 5 GHz, 2500Base-X of 3.125 GHz, etc etc.
> 
> 2. Then, the RCW loader asks itself - how do I obtain this PLLnCR0[FRATE_SEL]?
> 
> As you say: SGMII (aka the 5 GHz clock net frequency) can be obtained
> from 100 MHz or 125 MHz reference clock. This information sits in the
> RCW[SRDS_PLL_REF_CLK_SEL_S1] field. If 1, it will try to obtain the
> clock net from the higher frequency for that protocol. If 0, from the
> lower reference clock frequency.
> 
> So applied to the particular case of SGMII from step 1, you are telling
> it in the RCW to generate 5 GHz clock net from the higher reference
> clock, aka 125 MHz. This translates into a write to PLLnCR0[RFCLK_SEL]
> of 001b (aka 125 MHz).
> 
> This is all to say - when you use SGMII from the 125 MHz refclk, you
> have to use RCW[SRDS_PLL_REF_CLK_SEL_S1]=1, and when you use 2500Base-X
> from the same refclk, you have to set RCW[SRDS_PLL_REF_CLK_SEL_S1]=0.
> 
> What happens under the hood is that these 2 settings result in different
> clock net frequencies produced by that PLL, incompatible with each other.
> You can either produce one, or the other, at any given time.
Wow, this was an elaborate explanation which pushed my understanding one level
up. Thank you for investing your time!

For clarity, I like to re-state my PLL configuration:

- "slow" devices:
  - RCW[SRDS_PRTCL_S1] = 0x2233
  - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
  - PLL 1:     5 GHz (derived from external 125 MHz)
  - PLL 2: 3.125 GHz (derived from external 156.25 MHz)

- "fast" devices:
  - RCW[SRDS_PRTCL_S1] = 0x1133
  - RCW[SRDS_PLL_REF_CLK_SEL_S1] = 0b11
  - PLL 1:       5 GHz (derived from external 125 MHz)
  - PLL 2: 5.15625 GHz (derived from external 156.25 MHz)

My patch never reconfigures the PLLs or their clock net frequencies. It 
assigns lanes to PLL 1 (for SGMII) or PLL 2 (for 2.5SGMII). This is only
done on "slow" devices. The "fast" devices always use XFI (PLL 2).

> 
> In Linux we don't change the PLL refclk and clock net selection that was
> made by the RCW. Doing so would mean powering down all lanes that use
> that PLL, and unbinding and rebinding the Ethernet driver (because the
> supported_interfaces mask will change, and the phylink instances need to
> be destroyed and re-created to pick up that change). It is a very
> invasive operation.
The lynx-10g driver seems to alter PLL assignment in lynx_10g_lane_set_pll().
That means that a lane could be operated in SGMII or 2.5SGMII mode by assigning
the correct PLL (given that the PLLs are properly pre-configured via RCW).

But it also means that in theory I could configure PLL1 = 5 GHz and PLL2 = 5.15625 GHz
and then switch between SGMII (100M/1G) and XFI (2.5G / 5G / 10G via rate limiting).
This way all media speeds would be possible.

However, switching to another PLL isn't enough, I also have to switch the lane's protocol.
This worked for SGMII and 2.5SGMII, but I never managed to switch between SGMII to XFI.

> 
>>> You are just requesting to support the 10G, 5G and 2.5G media speeds via
>>> LANE_MODE_10GBASER (with pause-based rate adaptation in the Ethernet PHY),
>>> and the 1G and 100M media speeds via LANE_MODE_1000BASEX_SGMII.
>>
>> Yes, that is my minimum target. I hoped that the "fast" device can support
>> all speeds, but it's not mandatory.
> 
> What do you mean "can support all speeds", do you mean the media side
> speeds?
Yes, indeed.

> By switching from 10GBase-R to SGMII it would support them all,
> wouldn't it?
I don't think so. SGMII does only support 100M and 1G. Tell me if I'm wrong.

> Above I'm just re-explaining why you can't support 3 SerDes
> protocols which require different clock net frequencies when architecturally
> we have just 2 PLLs.