pcie-brcmstb: failed to enter low-power link state (L23)

[Jonathan Bell]

On Mon, 30 Dec 2024 at 20:42, Stefan Wahren <wahrenst at gmx.net> wrote:
>
> Hi Florian,
>
> Am 30.12.24 um 19:47 schrieb Florian Fainelli:
> > Hi Stefan,
> >
> > On 12/27/24 04:44, Stefan Wahren wrote:
> >> Hi,
> >> I did some experiments with s2idle on Raspberry Pi 4 (8 GB RAM,
> >> arm64/defconfig) and noticed that pcie-brcmstb fails to enter L23 state
> >> during suspend. I don't have any clue about PCI or this specific IP.
> >>
> >> Here is my current working branch [1], because s2idle on BCM2711
> >> currently requires some quirks.
> >>
> >> In order to help, I just added a simple dump of possible relevant
> >> registers at the begin of brcm_pcie_enter_l23():
> >>
> >> [   46.822440] PM: suspend entry (s2idle)
> >> [   47.050887] Filesystems sync: 0.228 seconds
> >> [   47.152159] Freezing user space processes
> >> [   47.154443] Freezing user space processes completed (elapsed 0.002
> >> seconds)
> >> [   47.154472] OOM killer disabled.
> >> [   47.154475] Freezing remaining freezable tasks
> >> [   47.155742] Freezing remaining freezable tasks completed (elapsed
> >> 0.001 seconds)
> >> [   48.797565] PM: suspend of devices complete after 1641.374 msecs
> >> [   48.797580] PM: start suspend of devices complete after 1641.829
> >> msecs
> >> [   48.798059] PM: late suspend of devices complete after 0.473 msecs
> >> [   48.810510] brcm-pcie fd500000.pcie:
> >> PCIE_RC_CFG_PRIV1_LINK_CAPABILITY: 00315e12
> >> [   48.810519] brcm-pcie fd500000.pcie: PCIE_RC_CFG_PRIV1_ROOT_CAP:
> >> 0000000f
> >> [   48.810523] brcm-pcie fd500000.pcie: PCIE_MISC_HARD_PCIE_HARD_DEBUG:
> >> 00200000
> >> [   48.810528] brcm-pcie fd500000.pcie: PCIE_MISC_MISC_CTRL: 88003480
> >> [   48.810532] brcm-pcie fd500000.pcie: PCIE_MISC_PCIE_CTRL: 00000000
> >> [   48.810535] brcm-pcie fd500000.pcie: PCIE_MISC_PCIE_STATUS: 000000b0
> >> [   48.846632] brcm-pcie fd500000.pcie: failed to enter low-power
> >> link state
> >> [   48.846640] xhci_hcd 0000:01:00.0: Possible wake-up device;
> >> regulators will not be disabled
> >> [   48.846725] PM: noirq suspend of devices complete after 48.628 msecs
> >> [   48.846730] PM: suspend-to-idle
> >> [   53.672017] PM: Triggering wakeup from IRQ 25
> >> [   53.672032] PM: resume from suspend-to-idle
> >>
> >> Does BCM2711 support L23 mode?
> >
> > AFAICT it does, but the end-point might not, and this is not
> > considered a fatal error, in fact we should probably demote this message.
> Agree, this match my observations with the CM4, which didn't have any
> PCI end point connected and didn't show this error.
>
> In case L23 state is optional, i agree this shouldn't be an error
> message. Maybe the comments for this functions should be extended.
>
> I reported this issue, because i thought this is the root cause why the
> XHCI controller isn't available after resume. Seems to be a red hering.
>
> > Would you have the lspci -vvv of the XHCI controller, too?
>
> Sure
>
> pi at raspberrypi:~$ lspci -vvv
> 00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge
> (rev 10) (prog-if 00 [Normal decode])
>          Device tree node:
> /sys/firmware/devicetree/base/scb/pcie at 7d500000/pci at 0,0
>          Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Latency: 0
>          Interrupt: pin A routed to IRQ 39
>          Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
>          I/O behind bridge: 00000000-00000fff [size=4K]
>          Memory behind bridge: f8000000-f80fffff [size=1M]
>          Prefetchable memory behind bridge:
> 00000000fff00000-00000000000fffff [disabled]
>          Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- <SERR- <PERR-
>          BridgeCtl: Parity- SERR- NoISA- VGA- VGA16- MAbort- >Reset-
> FastB2B-
>                  PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>          Capabilities: <access denied>
>          Kernel driver in use: pcieport
>
> 01:00.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host
> Controller (rev 01) (prog-if 30 [XHCI])
>          Subsystem: VIA Technologies, Inc. VL805 USB 3.0 Host Controller
>          Device tree node:
> /sys/firmware/devicetree/base/scb/pcie at 7d500000/pci at 0,0/usb at 0,0
>          Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B- DisINTx+
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Latency: 0, Cache Line Size: 64 bytes
>          Interrupt: pin A routed to IRQ 41
>          Region 0: Memory at 600000000 (64-bit, non-prefetchable) [size=4K]
>          Capabilities: <access denied>
>          Kernel driver in use: xhci_hcd
>
L23 entry for a PCIe endpoint is optional. Using PCI PM signalling to
enter D3hot, then turning power off, is sufficient for functionality.
It may be the case that the VL805 ignores link L2 PME handshakes, and
that's a valid thing to do.
I'd suggest testing CM4 with a network adapter that supports wake-on-LAN.

RP1 does support L23 entry and L2 exit through in-band signalling, but
the BCM2712 Videocore firmware probably won't correctly maintain or
sequence RP1's reset pin through platform suspend.

Regards
Jonathan