Infinite loop in pmksa_cache_get_okc?

[michael-dev]

Hi,

this is with the hostapd a5ed45586c63ffd8f9d2b44e27c251d7bacbeaf4 + 
OpenWRT patches on P1020WLAN (PowerPC Dualcore), compiled with -O0 -g 
gdb.
There are two APs, which share LAN but no SSID. After some time of 
operation (minutes to weeks), suddenly hostapd hangs consuming 100% CPU, 
producing no more output and no station can (re-)connect.
GDB Backtrace shows that hostapd is within pmksa_cache_get_okc when 
breaking and still in there (with same function args) some time later 
again. This could be an infinite loop in pmksa_cache_get_okc, though 
that function is probably not the cause but if looping is in 
pmksa_cache_get_okc it looks more like a corrupt data structure.

I'm wondering off the _pthread_cleanup_pop_restore in the backtrace - 
is it right or is this backtrace broken?
Has anybody seen such errors, too?

Output of hostapd in gdb (last lines):

345750848.257285: wlan0: CTRL-EVENT-EAP-STARTED xx:xx:xx:01:64:c9
1345750848.257788: wlan0: CTRL-EVENT-EAP-PROPOSED-METHOD vendor=0 
method=1
1345750873.789353: wlan1_1: STA xx:xx:xx:cf:f0:8b WPA: group key 
handshake completed (RSN)
1345750899.017465: wlan0_1: STA xx:xx:xx:da:d6:a7 WPA: group key 
handshake completed (RSN)
1345750929.429790: wlan0: AP-STA-DISCONNECTED xx:xx:xx:01:64:c9
1345750930.173182: wlan0_1: STA xx:xx:xx:01:64:c9 RADIUS: stopped 
accounting session 503354BC-0000003A
1345750930.714156: wlan0_1: STA xx:xx:xx:da:d6:a7 WPA: group key 
handshake completed (RSN)
1345750930.894667: wlan0_1: STA xx:xx:xx:01:64:c9 IEEE 802.11: 
authenticated

Program received signal SIGINT, Interrupt.
0x100194b4 in pmksa_cache_get_okc (pmksa=<value optimized out>, 
aa=0x1030acb4 "", spa=0x10314468 "??\001d?", pmkid=0x10317064 
"\221W\1777???\031\036#G$d??\030-\032\f\020\031?") at 
../src/ap/pmksa_cache_auth.c:71
71                      pos = pos->hnext;
(gdb) l
66                                              pos->hnext;
67                              }
68                              break;
69                      }
70                      prev = pos;
71                      pos = pos->hnext;
72              }
73
74              pos = pmksa->pmksa;
75              prev = NULL;
(gdb) bt
#0  0x100194b4 in pmksa_cache_get_okc (pmksa=<value optimized out>, 
aa=0x1030acb4 "", spa=0x10314468 "??\001d?", pmkid=0x10317064 
"\221W\1777???\031\036#G$d??\030-\032\f\020\031?") at 
../src/ap/pmksa_cache_auth.c:71
#1  0x481898ac in _pthread_cleanup_pop_restore () from 
/lib/libpthread.so.0
#2  0x10017264 in wpa_auth_okc_iter (a=<value optimized out>, 
ctx=<value optimized out>) at ../src/ap/wpa_auth_ie.c:393
#3  0x10008d48 in wpa_auth_iface_iter (iface=0x10317064, 
ctx=0x10314468) at ../src/ap/wpa_auth_glue.c:269
#4  0x10007a5c in hostapd_for_each_interface (interfaces=0xbfb19258, 
cb=0x1030acb4, ctx=0xbfb18858) at ../src/ap/hostapd.c:166
#5  0x10008dc0 in hostapd_wpa_auth_for_each_auth (ctx=<value optimized 
out>, cb=<value optimized out>, cb_ctx=<value optimized out>) at 
../src/ap/wpa_auth_glue.c:269
#6  0x10013250 in wpa_auth_for_each_auth (wpa_auth=<value optimized 
out>, cb=<value optimized out>, cb_ctx=<value optimized out>) at 
../src/ap/wpa_auth.c:2494
#7  0x10018020 in wpa_validate_wpa_ie (wpa_auth=0x1030abf0, 
sm=0x10314460, wpa_ie=0x1031704c "0&\001", wpa_ie_len=40, mdie=<value 
optimized out>, mdie_len=<value optimized out>) at 
../src/ap/wpa_auth_ie.c:131
#8  0x100664cc in handle_assoc (hapd=0x10305f28, mgmt=0x10317018, 
len=172, reassoc=1) at ../src/ap/ieee802_11.c:50
#9  0x10067360 in ieee802_11_mgmt (hapd=0x10305f28, buf=0x10317018 " ", 
len=172, fi=0xbfb18b68) at ../src/ap/ieee802_11.c:50
#10 0x100098f8 in hostapd_mgmt_rx (hapd=<value optimized out>, 
rx_mgmt=0xbfb18cb8) at ../src/ap/drv_callbacks.c:340
#11 0x1000a218 in hostapd_wpa_event (ctx=0x10305940, 
event=EVENT_RX_MGMT, data=0xbfb18cb8) at ../src/ap/drv_callbacks.c:340
#12 0x10032498 in mlme_event (drv=0x103073e8, cmd=<value optimized 
out>, frame=<value optimized out>, freq=<value optimized out>, 
ack=<value optimized out>, cookie=<value optimized out>, sig=0x1031700c)
     at ../src/drivers/driver_nl80211.c:7634
#13 0x100325ec in process_bss_event (msg=<value optimized out>, 
arg=0x10307928) at ../src/drivers/driver_nl80211.c:7634
#14 0x48041ce8 in nl_cb_call (cb=0x1030aa90, type=0, msg=0x1030db80) at 
../include/netlink-local.h:126
#15 0x480435d8 in recvmsgs (sk=0x10306908, cb=0x1030aa90) at nl.c:227
#16 0x4804376c in nl_recvmsgs (sk=0x10306908, cb=0x1030aa90) at 
nl.c:227
#17 0x1002a408 in wpa_driver_nl80211_event_receive (sock=<value 
optimized out>, eloop_ctx=<value optimized out>, handle=0x10306908) at 
../src/drivers/driver_nl80211.c:7634
#18 0x1001ad7c in eloop_sock_table_dispatch (table=0x1009b410, 
fds=0x10317b18) at ../src/utils/eloop.c:372
#19 0x1001b704 in eloop_run () at ../src/utils/eloop.c:372
#20 0x100023d8 in main (argc=<value optimized out>, argv=0xbfb195b4)
(gdb) s
Program received signal SIGINT, Interrupt.
0x100194b4 in pmksa_cache_get_okc (pmksa=<value optimized out>, 
aa=0x1030acb4 "", spa=0x10314468 "??\001d?", pmkid=0x10317064 
"\221W\1777???\031\036#G$d??\030-\032\f\020\031?") at 
../src/ap/pmksa_cache_auth.c:71
71                      pos = pos->hnext;
(gdb) c
Continuing.
Program received signal SIGINT, Interrupt.
0x100194b4 in pmksa_cache_get_okc (pmksa=<value optimized out>, 
aa=0x1030acb4 "", spa=0x10314468 "??\001d?", pmkid=0x10317064 
"\221W\1777???\031\036#G$d??\030-\032\f\020\031?") at 
../src/ap/pmksa_cache_auth.c:71
71                      pos = pos->hnext;
(gdb) Quit

Regards,
  M. Braun