help troubleshooting low throughput
Tim Harvey
tharvey at gateworks.com
Thu May 22 11:37:42 PDT 2014
On Thu, May 22, 2014 at 3:08 AM, Michal Kazior <michal.kazior at tieto.com> wrote:
> On 22 May 2014 11:46, Tim Harvey <tharvey at gateworks.com> wrote:
>> On Thu, May 22, 2014 at 2:39 AM, Tim Harvey <tharvey at gateworks.com> wrote:
>>> Greetings,
>>>
>>> I could use some help troubleshooting a low throughput issue. I'm
>>> currently using the following:
>>> - UNEX DAXA-O1 11ac/n/a 3x3 MIMO qca988x hw2.0
>>> http://www.unex.com.tw/product/daxa-o1
>>> - 80MHz channel w/o local interference
>>> - ath10k git 0dbbb028a7c461777bf4a0d53780e539e6f40e14 (May 16)
>>> - up-to-date git of hostapd/wpa_supplicant/iw
>>> - fw 10.1.467.2-1 api 2 htt 2.1
>>> - infrastructure mode using
>>> http://wireless.kernel.org/en/users/Drivers/ath10k/configuration#Full_hostapd_configuration
>
> Did you just copy&paste the example config file (and updated
> interface=) or did you do something extra?
Hi Michal,
I disabled bridge mode, DFS, wpa, wps and added
'vht_oper_centr_freq_seg0_idx=42' which appears to be something new
that is required or hostapd bails out:
### hostapd configuration file
ctrl_interface=/var/run/hostapd
interface=wlan0
driver=nl80211
#bridge=br-lan
### IEEE 802.11
ssid=timtest
hw_mode=a
channel=36
max_num_sta=128
auth_algs=1
disassoc_low_ack=1
### DFS
#ieee80211h=1
#ieee80211d=1
#country_code=FR
ieee80211h=0
ieee80211d=0
### IEEE 802.11n
ieee80211n=1
ht_capab=[HT40+][LDPC][SHORT-GI-20][SHORT-GI-40][TX-STBC][RX-STBC1][DSSS_CCK-40]
### IEEE 802.11ac
ieee80211ac=1
vht_oper_chwidth=1
vht_oper_centr_freq_seg0_idx=42
vht_capab=[MAX-MPDU-11454][RXLDPC][SHORT-GI-80][TX-STBC-2BY1][RX-STBC-1][MAX-A-MPDU-LEN-EXP7][RX-ANTENNA-PATTERN][TX-ANTENNA-PATTERN]
### WPA/IEEE 802.11i
#wpa=2
#wpa_key_mgmt=WPA-PSK
#wpa_passphrase=12345678
#wpa_pairwise=CCMP
wpa=0
### Wi-Fi Protected Setup (WPS)
#wps_state=2
#ap_setup_locked=0
#wps_pin_requests=/var/run/hostapd_wps_pin_requests
#device_name=QCA Access Point
#manufacturer=Qualcomm Atheros
#device_type=6-0050F204-1
#config_methods=virtual_push_button physical_push_button label keypad
virtual_display
#pbc_in_m1=1
#ap_pin=12345670
#upnp_iface=br-lan
#eap_server=1
### hostapd event logger configuration
logger_syslog=127
logger_syslog_level=2
logger_stdout=127
logger_stdout_level=2
### WMM
wmm_enabled=1
uapsd_advertisement_enabled=1
wmm_ac_bk_cwmin=4
wmm_ac_bk_cwmax=10
wmm_ac_bk_aifs=7
wmm_ac_bk_txop_limit=0
wmm_ac_bk_acm=0
wmm_ac_be_aifs=3
wmm_ac_be_cwmin=4
wmm_ac_be_cwmax=10
wmm_ac_be_txop_limit=0
wmm_ac_be_acm=0
wmm_ac_vi_aifs=2
wmm_ac_vi_cwmin=3
wmm_ac_vi_cwmax=4
wmm_ac_vi_txop_limit=94
wmm_ac_vi_acm=0
wmm_ac_vo_aifs=2
wmm_ac_vo_cwmin=2
wmm_ac_vo_cwmax=3
wmm_ac_vo_txop_limit=47
wmm_ac_vo_acm=0
### TX queue parameters
tx_queue_data3_aifs=7
tx_queue_data3_cwmin=15
tx_queue_data3_cwmax=1023
tx_queue_data3_burst=0
tx_queue_data2_aifs=3
tx_queue_data2_cwmin=15
tx_queue_data2_cwmax=63
tx_queue_data2_burst=0
tx_queue_data1_aifs=1
tx_queue_data1_cwmin=7
tx_queue_data1_cwmax=15
tx_queue_data1_burst=3.0
tx_queue_data0_aifs=1
tx_queue_data0_cwmin=3
tx_queue_data0_cwmax=7
tx_queue_data0_burst=1.5
>
> Typically the most important option is vht_capab MAX-A-MPDU-LEN-EXPx
> (x=0 or 7 depending on hostapd version). Without this aggregation is
> crippled and it yields pretty poor throughput.
good to know - got that.
hostapd is git 4e0a94b7dc76db58cddbbcfe0be0bfef547f6dd0 Fri May 16
built with following config:
CONFIG_DRIVER_HOSTAP=y
CONFIG_DRIVER_WIRED=y
CONFIG_DRIVER_NL80211=y
CONFIG_LIBNL32=y
CONFIG_IAPP=y
CONFIG_RSN_PREAUTH=y
CONFIG_PEERKEY=y
CONFIG_IEEE80211W=y
CONFIG_EAP=y
CONFIG_EAP_MD5=y
CONFIG_EAP_TLS=y
CONFIG_EAP_MSCHAPV2=y
CONFIG_EAP_PEAP=y
CONFIG_EAP_GTC=y
CONFIG_EAP_TTLS=y
CONFIG_WPS=y
CONFIG_PKCS12=y
CONFIG_RADIUS_SERVER=y
CONFIG_IPV6=y
CONFIG_DRIVER_RADIUS_ACL=y
CONFIG_IEEE80211N=y
CONFIG_IEEE80211AC=y
CONFIG_ACS=y
root at ap-99:~# hostapd -v
hostapd v2.2-devel
User space daemon for IEEE 802.11 AP management,
IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator
Copyright (c) 2002-2014, Jouni Malinen <j at w1.fi> and contributors
>
> What about the client? What is it? Is it connected in VHT80 mode?
identical to the ap, but running wpa_supplicant from same git rev as
hostapd with:
network={
scan_ssid=1
ssid="timtest"
key_mgmt=NONE
}
Its showing 80Mhz MCS 5 (between 5 and 8)
root at sta-97:~# iw wlan0 station dump
Station 60:02:b4:9d:99:7f (on wlan0)
inactive time: 590 ms
rx bytes: 160004
rx packets: 1824
tx bytes: 9832
tx packets: 87
tx retries: 0
tx failed: 0
signal: -53 dBm
signal avg: -52 dBm
tx bitrate: 6.0 MBit/s
rx bitrate: 975.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 3
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
ap is showing 80MHz width between MCS 5 and MCS 8:
root at ap-99:~# iw wlan0 station dump
Station 60:02:b4:9d:99:62 (on wlan0)
inactive time: 0 ms
rx bytes: 275591916
rx packets: 182178
tx bytes: 4394890
tx packets: 50807
tx retries: 0
tx failed: 0
signal: -60 dBm
signal avg: -60 dBm
tx bitrate: 6.0 MBit/s
rx bitrate: 702.0 MBit/s VHT-MCS 5 80MHz VHT-NSS 3
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
>
>
>>> I'm using iperf for throughput tests and getting no more than 220mbps
>>> best case, typically more like 120mbps. The rx bitrate bounces around
>>> MCS 5 to 8 and shows 3 spatial streams so I would be expecting a much
>>> higher throughput. The cards are in boards with a quad-core ARM 1GHz
>>> Cortex-A9 CPU and there is no indication the system is bottle-necked.
>>> There are no other kernel modules loaded other thank
>>> ath10k_pci/ath10k_core/ath and debugging is disabled.
>
> Currently ath10k doesn't really scale much with number of CPUs. There
> are basically two tasklets that could split the work just a little
> bit, but this requires interrupt spreading. From what I know some ARM
> chips can't do that so ath10k ends up using only single CPU all the
> time. 1GHz of an A9 should still be enough to get you 500mbps+ though.
interesting. I see [ath10k_wq] in ps, what is the other task? ath10k
will just register 1 interrupt for PCI, how would you spread that if
only 1 ath10k device is in the system?
I would agree that a 1GHz CortexA9 should be able to do well. The top
application shows that only CPU0 is being utilized and never more than
25% or so (softirq mostly) and mostly idle. So I don't think this is
any sort of CPU bottleneck.
>
> Did you run TCP and/or UDP tests? What direction did you test
> (station->ap / ap->station)?
both - the best throughput I see is appx 220mbps TCP and 260mbps UDP
and this is consistent in both directions.
>
>
>>> Using tcpdump/wireshark to inspect the radiotap headers I only see
>>> packets with 'Antenna: 0' - Does this field indicate what transmitter
>>> the pkt was received on and indicate I'm only receiving from a single
>>> transmitter?
>
> Antenna callbacks aren't implemented in master branch yet. You can
> check ath-next-test for that or cherry picks Ben's patches.
ok - I will do that. Does this field indicate which antenna the frame
was sent out or received from?
>
> You can verify max number of spatial streams with iw by looking at `HT
> TX/RX MCS rate indexes supported:` and/or VHT TX/RX MCS sets.
>
These look fine (see below) but is there a way to actually prove (ie
by radiotap) that frames come in/out of multiple antennas?
root at ap-99:~# iw phy0 info
Wiphy phy0
max # scan SSIDs: 16
max scan IEs length: 199 bytes
Retry short limit: 7
Retry long limit: 4
Coverage class: 0 (up to 0m)
Device supports AP-side u-APSD.
Supported Ciphers:
* WEP40 (00-0f-ac:1)
* WEP104 (00-0f-ac:5)
* TKIP (00-0f-ac:2)
* CCMP (00-0f-ac:4)
* CMAC (00-0f-ac:6)
Available Antennas: TX 0 RX 0
Supported interface modes:
* IBSS
* managed
* AP
* AP/VLAN
* monitor
* P2P-client
* P2P-GO
Band 2:
Capabilities: 0x19e3
RX LDPC
HT20/HT40
Static SM Power Save
RX HT20 SGI
RX HT40 SGI
TX STBC
RX STBC 1-stream
Max AMSDU length: 7935 bytes
DSSS/CCK HT40
Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
Minimum RX AMPDU time spacing: 8 usec (0x06)
HT TX/RX MCS rate indexes supported: 0-23
VHT Capabilities (0x338001b2):
Max MPDU length: 11454
Supported Channel Width: neither 160 nor 80+80
RX LDPC
short GI (80 MHz)
TX STBC
RX antenna pattern consistency
TX antenna pattern consistency
VHT RX MCS set:
1 streams: MCS 0-9
2 streams: MCS 0-9
3 streams: MCS 0-9
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT RX highest supported: 0 Mbps
VHT TX MCS set:
1 streams: MCS 0-9
2 streams: MCS 0-9
3 streams: MCS 0-9
4 streams: not supported
5 streams: not supported
6 streams: not supported
7 streams: not supported
8 streams: not supported
VHT TX highest supported: 0 Mbps
Bitrates (non-HT):
* 6.0 Mbps
* 9.0 Mbps
* 12.0 Mbps
* 18.0 Mbps
* 24.0 Mbps
* 36.0 Mbps
* 48.0 Mbps
* 54.0 Mbps
Frequencies:
Frequencies:
* 5180 MHz [36] (17.0 dBm)
* 5200 MHz [40] (17.0 dBm)
* 5220 MHz [44] (17.0 dBm)
* 5240 MHz [48] (17.0 dBm)
* 5260 MHz [52] (23.0 dBm) (no IR, radar detection)
DFS state: usable (for 4294712 sec)
* 5280 MHz [56] (23.0 dBm) (no IR, radar detection)
DFS state: usable (for 4294712 sec)
* 5300 MHz [60] (23.0 dBm) (no IR, radar detection)
DFS state: usable (for 4294712 sec)
* 5320 MHz [64] (23.0 dBm) (no IR, radar detection)
DFS state: usable (for 4294712 sec)
* 5500 MHz [100] (disabled)
* 5520 MHz [104] (disabled)
* 5540 MHz [108] (disabled)
* 5560 MHz [112] (disabled)
* 5580 MHz [116] (disabled)
* 5600 MHz [120] (disabled)
* 5620 MHz [124] (disabled)
* 5640 MHz [128] (disabled)
* 5660 MHz [132] (disabled)
* 5680 MHz [136] (disabled)
* 5700 MHz [140] (disabled)
* 5745 MHz [149] (30.0 dBm)
* 5765 MHz [153] (30.0 dBm)
* 5785 MHz [157] (30.0 dBm)
* 5805 MHz [161] (30.0 dBm)
* 5825 MHz [165] (30.0 dBm)
Supported commands:
* new_interface
* set_interface
* new_key
* start_ap
* new_station
* new_mpath
* set_mesh_config
* set_bss
* authenticate
* associate
* deauthenticate
* disassociate
* join_ibss
* join_mesh
* remain_on_channel
* set_tx_bitrate_mask
* frame
* frame_wait_cancel
* set_wiphy_netns
* set_channel
* set_wds_peer
* probe_client
* set_noack_map
* register_beacons
* start_p2p_device
* set_mcast_rate
* Unknown command (104)
* connect
* disconnect
Supported TX frame types:
* IBSS: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80
0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* managed: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* AP: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x80
0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* AP/VLAN: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* mesh point: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* P2P-client: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* P2P-GO: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
* P2P-device: 0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70
0x80 0x90 0xa0 0xb0 0xc0 0xd0 0xe0 0xf0
Supported RX frame types:
* IBSS: 0x40 0xb0 0xc0 0xd0
* managed: 0x40 0xd0
* AP: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
* AP/VLAN: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
* mesh point: 0xb0 0xc0 0xd0
* P2P-client: 0x40 0xd0
* P2P-GO: 0x00 0x20 0x40 0xa0 0xb0 0xc0 0xd0
* P2P-device: 0x40 0xd0
software interface modes (can always be added):
* AP/VLAN
* monitor
valid interface combinations:
* #{ AP } <= 8,
total <= 8, #channels <= 1, STA/AP BI must match,
radar detect widths: { 20 MHz (no HT), 20 MHz, 40 MHz, 80 MHz }
HT Capability overrides:
* MCS: ff ff ff ff ff ff ff ff ff ff
* maximum A-MSDU length
* supported channel width
* short GI for 40 MHz
* max A-MPDU length exponent
* min MPDU start spacing
Device supports TX status socket option.
Device supports HT-IBSS.
Device supports scan flush.
>
>>> I've noticed 'iw phy0 info' shows 'Available Antennas: TX 0 RX 0' and
>>> any attempt to set the antenna mask with 'iw phy set antenna' fails
>>> with an 'Operation not supported (-95)' - does this indicate an issue
>>> with ath10k, iw, or perhaps the card's EEPROM?
>
> It would be more helpful if you could provide `iw list` output instead
> next time.
right - I've put it above this time.
>
>
>>> I've also noticed that 'iw wlan0 station dump' statistics show in the
>>> rx bitrate stat that sometimes SGI is flagged and sometimes its not -
>>> should I expect this to change like this? I was under the impression
>>> that was a static configuration.
>
> I think this ath10k's hw rate control is free to pick SGI/LGI
> (assuming target station supports SGI at all).
I always assumed a user would want to force one way or another - does
the rate control engine try to optimize by using short guard-band
interval (if supported) and then back-off to long if it detects lots
of retries?
>
>
> [...]
>
>> I neglected to mention that on whichever end is running the iperf
>> server (receiving) I do see periodic 'failed to pop...' warnings, a
>> pair of them a couple times a minute. I'm not quite clear if this is a
>> hardware issue or something else:
>>
>> [ 3763.131280] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3769.081093] ath10k: failed to pop chained msdus, dropping
>> [ 3769.086575] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3781.092383] ath10k: failed to pop chained msdus, dropping
>> [ 3781.097869] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3784.367225] ath10k: failed to pop chained msdus, dropping
>> [ 3784.372735] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3809.501484] ath10k: failed to pop chained msdus, dropping
>> [ 3809.506993] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3821.723404] ath10k: failed to pop chained msdus, dropping
>> [ 3821.728914] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3832.067136] ath10k: failed to pop chained msdus, dropping
>> [ 3832.072690] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3833.632859] ath10k: failed to pop chained msdus, dropping
>> [ 3833.638341] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3837.940414] ath10k: failed to pop chained msdus, dropping
>> [ 3837.945982] ath10k: failed to pop amsdu from htt rx ring -22
>> [ 3844.100514] ath10k: failed to pop chained msdus, dropping
>
> You don't have to worry about these messages.
>
> This was recently introduced by one of my patches to further analyze a
> bug that Avery was seeing (kernel panic due to skb_push). I plan on
> making a patch to clean this up.
>
> Anyway, thanks for letting know :-)
ok - I saw the patch but from the commit message I through it was
indicating these were failures. Thanks for the explanation.
>
>
> Michał
Thanks for helping!
Tim
More information about the ath10k
mailing list