Performance degradation over time

Tue Mar 8 18:01:57 PST 2016

So with the most recent firmware I am experiencing different performance issues. First, the latest firmware after coming up does significantly less than it used to. For 2.4 GHz where before it would do ~50Mbps it now can only average ~30Mbps (the rate is sporadic though anywhere from 10Mbps to 40Mbps) and this is about 3 inches away from my antennas on an unused channel. Secondly sometimes I have seen it drop down to < .1 Mbps. I grabbed the fw_stats. One thing that seems drastically different between my 2.4 Ghz and my 5 Ghz is that 2.4 has extremely high error counts where my 5 GHz is < 400. Could this be a symptom of misconfiguration or more firmware issues?

             ath10k PDEV stats
             =================

           Channel noise floor        -82
              Channel TX power         46
                TX frame count  340181623
                RX frame count 1520199208
                RX clear count 1951746151
                   Cycle count 3237081850
               PHY error count      71162
                 RTS bad count      16735
                RTS good count      27724
                 FCS bad count    1104584
               No beacon count          0
                 MIB int count         25

          ath10k PDEV TX stats
             =================

            HTT cookies queued    1071875
             HTT cookies disp.    1071875
                   MSDU queued    1119755
                   MPDU queued    1119543
                 MSDUs dropped          0
                  Local enqued      47880
                   Local freed      47880
                     HW queued     311696
                  PPDUs reaped     311696
                 Num underruns          3
                 PPDUs cleaned          1
                  MPDUs requed     491558
             Excessive retries      57767
                       HW rate         67
            Sched self tiggers     141832
     Dropped due to SW retries         39
       Illegal rate phy errors          0
        Pdev continuous xretry          0
                    TX timeout          2
                   PDEV resets          5
                  PHY underrun          0
  MPDU is more than txop limit          0

          ath10k PDEV RX stats
             =================

         Mid PPDU route change        256
       Tot. number of statuses    1508637
        Extra frags on rings 0          0
        Extra frags on rings 1        430
        Extra frags on rings 2        548
        Extra frags on rings 3          0
        MSDUs delivered to HTT    1508637
        MPDUs delivered to HTT    1508637
      MSDUs delivered to stack     137106
      MPDUs delivered to stack     137106
               Oversized AMSUs          0
                    PHY errors    4009165
              PHY errors drops      66554
   MPDU errors (FCS, MIC, ENC)     229322

             ath10k VDEV stats (0)
             =================

             ath10k PEER stats (3)
             =================

              Peer MAC address 44:c3:06:00:01:c8
                     Peer RSSI 0
                  Peer TX rate 0
                  Peer RX rate 0

              Peer MAC address 20:c9:d0:43:34:17
                     Peer RSSI 32
                  Peer TX rate 52000
                  Peer RX rate 52000

              Peer MAC address 00:00:00:00:70:3e
                     Peer RSSI 25939
                  Peer TX rate 19
                  Peer RX rate 2000

--  
Matt Keeler

On March 8, 2016 at 11:38:59, Matthew Keeler (mjkeeler7 at gmail.com(mailto:mjkeeler7 at gmail.com)) wrote:

>  
> I am using firmware version 10.2.4.70.9-2 (from the linux-firmware arch package). So it looks to be a little out of date. I will try the latest firmware and see if it fixes things.
>  
> Thanks for the tips about the simulate firmware crash and the fw stats.  
>  
> --
> Matt Keeler
>  
> On March 8, 2016 at 11:32:58, Michal Kazior (michal.kazior at tieto.com(mailto:michal.kazior at tieto.com)) wrote:
>  
> > On 8 March 2016 at 15:09, Matthew Keeler wrote:
> > >
> > > I have an arch linux box running kernel version 4.4 and within it two Airetos AEX-QCA9880-NX cards in it. I have hostapd configured to use one for 2.4 GHz bgn and the other for 5 GHz n/ac. After a fresh boot of the box I can get about 50Mbps over 2.4 GHz using iperf and 350-400MBps over 5GHz. After a couple of days or so the performance of my 2.4GHz card is dropping to about 5Mbps. Has anyone else had similar issues with this kind of performance degradation or does anyone know a good place to start to try and figure why this could be happening.
> > >
> > > A couple things I have checked: nothing in dmesg, nothing abnormal in hostapd logs, restarting hostapd doesn’t seem to help, only rebooting helps.
> >
> > Hi,
> >
> > You didn't mention firmware version that you're using:
> >
> > dmesg | grep ath10k.*firmware
> >
> > From the looks of it it seems like a firmware (probably rate control
> > module) issue and you can't do much in driver about that. At best you
> > could force firmware reboot via debugfs to workaround the problem:
> >
> > echo hw-restart > /sys/kernel/debug/ieee80211/phy0/ath10k/simulate_fw_crash
> >
> > FWIW I suggest you to try out latest 10.2.4 if you didn't do that already, e.g.
> >
> > https://github.com/kvalo/ath10k-firmware/blob/master/QCA988X/10.2.4/firmware-5.bin_10.2.4.70.22-2
> >
> > I guess you could also try getting fw stats after fresh boot and then
> > after the problem is observed and compare:
> >
> > cat /sys/kernel/debug/ieee80211/phy0/ath10k/fw_stats
> >
> >
> > Michał
>  