[PATCH 0/4] PCI: replace dublicated MRRS limit quirks

[Bjorn Helgaas]

On Wed, Jul 07, 2021 at 06:43:13PM +0200, Neil Armstrong wrote:
> On 07/07/2021 17:54, Bjorn Helgaas wrote:
> > On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
> >> In their Designware PCIe controller driver, amlogic sets the
> >> Max_Payload_Size & Max_Read_Request_Size to 256:
> >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
> >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
> >> in their root port PCIe Express Device Control Register.
> >>
> >> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
> >> Max_Read_Request_Size are used to decompose into AXI burst, but it
> >> seems the Max_Payload_Size & Max_Read_Request_Size are set by
> >> default to 512 but the internal Max_Payload_Size_Supported is set to
> >> 256, thus changing these values to 256 at runtime to match and
> >> optimize bandwidth.
> >>
> >> It's said, "Reducing Outbound Decomposition" :
> >>  - "Ensure that your application master does not generate bursts of
> >>    size greater than or equal to Max_Payload_Size"
> >>
> >>  - "Program your PCIe system with a larger value of Max_Payload_Size
> >>    without exceeding Max_Payload_Size_Supported"
> >>
> >>  - "Program your PCIe system with a larger value of Max_Read_Request
> >>    without exceeding Max_Payload_Size_Supported:
> >>
> >> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
> >> Outbound Decomposition which decreases PCIe link and degrades the
> >> AXI bus by doubling the bursts, leading to this fix to avoid
> >> overflowing the AXI bus.
> >>
> >> So it seems to be still needed, I assume this *should* be handled in
> >> the core somehow to propagate these settings to child endpoints to
> >> match the root port Max_Payload_Size & Max_Read_Request sizes.
> >>
> >> Maybe by adding a core function to set these values instead of using
> >> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
> >> and set a state on the root port to propagate the value ?
> > 
> > I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
> > context.  The above *seems* to say that MPS/MRRS settings affect AXI
> > bus usage.
> 
> It does when the TLPs are directed to the RC.

That's a defect in the RC.

> > The MPS and MRRS registers are defined to affect traffic on *PCIe*.  If
> > a platform uses MPS and MRRS values to optimize transfers on non-PCIe
> > links, that's a problem because the PCI core code that manages MPS and
> > MRRS has no knowledge of those non-PCIe parts of the system.
> 
> Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect
> transfers on the internal SoC/Processor/Chip internal bus/fabric.
> 
> > You might be able to deal with this in Synopsys-specific code somehow,
> > but it's going to be a bit of a hassle because I don't want it to make
> > maintenance of the generic MPS/MRRS code harder.
> 
> I understand, but this is why these quirks are currently implemented in the
> controller driver and only applies when the controller has been probed
> and to each endpoint detected on this particular controller.
> 
> So we may continue having separate quirks for each controller if the core
> isn't the right place to handle MPS/MRRS.

The PCI core is the correct place to handle MPS/MRRS because their
behavior is defined by the PCIe spec.

Quirks are the way to work around this defect in the Synopsys PCIe IP.

Bjorn