[FS#263] WRT3200ACM WAN connection failure in under 24 hours

Sat Feb 25 15:51:47 PST 2017

For full information about what has changed in this task, visit the URL below and click the History tab.

FS#263 - WRT3200ACM WAN connection failure in under 24 hours
User who did this: Baptiste Jonglez (bjonglez)
Task details edited:
-------
__Symptom__

WAN connectivity is lost until the device is rebooted; failure aligns with stack trace (attached).  I've observed three failures of this same type, and all have occurred at around 18 hours of device up time.  Work-around (yet to be tested) is to schedule a periodic device reboot every 12 hours.

__Device problem occurs on__

  * Linksys WRT3200ACM (version 1)
  * LEDE configuration is near-stock, with the only modifications being PPPoE configuration and WiFi setup/enable.  Issue does not appear to be similar to FS#227 (PPPoE).

__Software versions of LEDE release, packages, etc.__

Release: 

    [0.000000] Linux version 4.4.28 (buildbot at builds) (gcc version 5.4.0 (LEDE GCC 5.4.0 r2062) ) #0 SMP Mon Oct 31 16:13:37 2016
    [0.000000] CPU: ARMv7 Processor [414fc091] revision 1 (ARMv7), cr=10c5387d
    [0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
    [0.000000] Machine model: Linksys WRT3200ACM

Packages:
    Default packages + luci-ssl

Possibly helpful info (80211 related output in log):

    pci 0000:00:01.0: enabling device (0140 -> 0142)
    ieee80211 phy0: priv->iobase0 = e0e00000
    ieee80211 phy0: priv->iobase1 = e1080000
    ieee80211 phy0: priv->pcmd_buf = de248000  priv->pphys_cmd_buf = 1e248000
    ieee80211 phy0: fw download start
    ieee80211 phy0: FwSize = 207660 downloaded Size = 207660 curr_iteration 65522
    ieee80211 phy0: fw download complete
    ieee80211 phy0: pcmd = de248000
    ieee80211 phy0: firmware version: 0x7080004
    ieee80211 phy0: firmware region code: 10
    ieee80211 phy0: 2G disabled, 5G enabled
    ieee80211 phy0: 4 TX antennas, 4 RX antennas
    pci 0000:00:02.0: enabling device (0140 -> 0142)
    ieee80211 phy1: priv->iobase0 = e1200000
    ieee80211 phy1: priv->iobase1 = e1480000
    ieee80211 phy1: priv->pcmd_buf = dda20000  priv->pphys_cmd_buf = 1da20000
    ieee80211 phy1: fw download start
    ieee80211 phy1: FwSize = 207660 downloaded Size = 207660 curr_iteration 65527
    ieee80211 phy1: fw download complete
    ieee80211 phy1: pcmd = dda20000
    ieee80211 phy1: firmware version: 0x7080004
    ieee80211 phy1: firmware region code: 10
    ieee80211 phy1: 2G enabled, 5G disabled
    ieee80211 phy1: 4 TX antennas, 4 RX antennas

__Steps to reproduce__

Device was idle before the stack trace, but logs show some DHCP-related activity within 1 minute prior to the fault.

__Stack Trace__
  Wed Nov  2 17:10:41 2016 kern.err kernel: [74541.045379] ieee80211 phy1: create ba result error 1
  Wed Nov  2 17:10:41 2016 kern.err kernel: [74541.059396] ieee80211 phy1: ampdu operation error code: -22
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.085673] ------------[ cut here ]------------
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.090364] WARNING: CPU: 0 PID: 5305 at compat-wireless-2016-10-08/net/mac80211/agg-tx.c:398 ___ieee80211_stop_tx_ba_session+0x1e8/0x1f8 [mac80211]()
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.103914] Modules linked in: pppoe ppp_async iptable_nat pppox ppp_generic nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_id xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_CT slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_raw iptable_mangle iptable_fWed Nov  2 17:10:42 2016 kern.warn kernel: [74542.162765] CPU: 0 PID: 5305 Comm: kworker/u4:2 Not tainted 4.4.28 #0
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.169230] Hardware name: Marvell Armada 380/385 (Device Tree)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.175192] Workqueue: phy1 ieee80211_iface_work [mac80211]
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.180804] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.188584] [] (show_stack) from [] (dump_stack+0x8c/0xa0)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.195838] [] (dump_stack) from [] (warn_slowpath_common+0x94/0xb0)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.203962] [] (warn_slowpath_common) from [] (warn_slowpath_null+0x1c/0x24)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.212799] [] (warn_slowpath_null) from [] (___ieee80211_stop_tx_ba_session+0x1e8/0x1f8 [mac80211])
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.223749] [] (___ieee80211_stop_tx_ba_session [mac80211]) from [] (__ieee80211_stop_tx_ba_session+0x2c/0x40 [mac80211])
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.236527] [] (__ieee80211_stop_tx_ba_session [mac80211]) from [] (ieee80211_iface_work+0x1c0/0x610 [mac80211])
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.248510] [] (ieee80211_iface_work [mac80211]) from [] (process_one_work+0x228/0x3bc)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.258292] [] (process_one_work) from [] (worker_thread+0x310/0x504)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.266505] [] (worker_thread) from [] (kthread+0xf0/0xf8)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.273758] [] (kthread) from [] (ret_from_fork+0x14/0x3c)
  Wed Nov  2 17:10:42 2016 kern.warn kernel: [74542.281021] ---[ end trace 09d229b863156d80 ]---
-------
Status: Unconfirmed -> Waiting on reporter

More information can be found at the following URL:
https://bugs.lede-project.org/index.php?do=details&task_id=263