Multi-PHYs and multiple-ports bonding support

[Russell King (Oracle)]

On Tue, Oct 18, 2022 at 10:02:05AM +0200, Maxime Chevallier wrote:
> Hello Russell,
> 
> On Mon, 17 Oct 2022 10:24:49 +0100
> "Russell King (Oracle)" <linux at armlinux.org.uk> wrote:
> 
> > On Mon, Oct 17, 2022 at 10:51:00AM +0200, Maxime Chevallier wrote:
> > > 2) Changes in Phylink
> > > 
> > > This might be the tricky part, as we need to track several ports,
> > > possibly connected to different PHYs, to get their state. For now, I
> > > haven't prototyped any of this yet.  
> > 
> > The problem is _way_ larger than phylink. It's a fundamental
> > throughout the net layer that there is one-PHY to one-MAC
> > relationship. Phylink just adopts this because it is the established
> > norm, and trying to fix it is rather rediculous without a use case.
> > 
> > See code such as the ethtool code, where the MAC and associated layers
> > are# entirely bypassed with all the PHY-accessing ethtool commands and
> > the commands are passed directly to phylib for the PHY registered
> > against the netdev.
> > 
> > We do have use cases though - consider a setup such as the mcbin with
> > the 3310 in SGMII mode on the fibre link and a copper PHY plugged in
> > with its own PHY - a stacked PHY situation (we don't support this
> > right now.) Which PHY should the MII ioctls, ethtool, and possibly the
> > PTP timestamp code be accessing with a copper SFP module plugged in?
> > 
> > This needs to be solved for your multi-PHY case, because you need to
> > deal with programming e.g. the link advertisement in both PHYs, not
> > just one - and with the above model, you have no choice which PHY gets
> > the call, it's always going to be the one registered with the netdev.
> > 
> > The point I'm making is that you're suggesting this is a phylink
> > issue, but it isn't, it's a generic networking layering bypass issue.
> > If the net code always forwarded the ethtool etc stuff to the MAC and
> > let the MAC make appropriate decisions about how these were handled,
> > then we would have a properly layered approach where each layer can
> > decide how a particular interface is implemented - to cope with
> > situations such as the one you describe.
> 
> I agree with all you say, and indeed this problem is a good opportunity
> IMO to consider the other use-cases like the one you mention and come
> up with a nice solution.

However, this isn't really "other use-cases" that I'm talking about
above, but a problem that needs solving for your case.

> When you mention that ethtool bypasses the MAC layer and talks to
> phylib, since phylink has the overall view of the link, and abstracts
> the phy away from the MAC, I would think this is a good place to
> manage this tree of PHYs/ports, but on the other hand that's adding
> quite a lot of complexity to phylink.

phylink doesn't abstract the PHY from the networking layer. What we
have are these call paths through the layers:

net --> mac --> phylink --> phy
 |                           ^
 `---------------------------'
      (bypass call path)

That bypass call path will be a problem as soon as you start talking
about having more than one PHY for a MAC.

Yes, changing phylink fixes some of the issues, but doesn't get away
from the fundamental issue that both the MAC and phylink are bypassed
for certain paths.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!