crash in wpa_supplicant

Mon Mar 31 10:00:58 PDT 2014

On Mon, 2014-03-31 at 17:48 +0300, Alexandru Costache wrote:
> To this point it looks to me that, at least in the version that I'm dealing
> with, after an interface has been de-initialised, there may still be some
> _scan, _sched_scan and _delayed_sched_scan timeouts left registered on the
> event loop. Removing all of them in wpa_supplicant_deinit_iface(), right
> before os_free(wpa_s) seems to get the job done.

In recent git master, _scan, _delayed_sched_scan, and _sched_scan all
get cleaned up from wpa_supplicant_cleanup() which is called from
pa_supplicant_deinit_iface().  That should remove them all.  However,
maybe some code called after that *re*-schedules them, which would be a
bug.  If you could put some wpa_dbg() calls after the
wpa_supplicant_cancel_sched_scan() call in wpa_supplicant_cleanup(), and
then at any callsite that schedules that timeout, we could see if that's
the case.

However, a quick audit shows that these timeouts may not be properly
removed, which are likely bugs:

dbus_common.c::process_timeout() (should remove from
wpas_dbus_deinit_common())

hs20_supplicant.c::hs20_continue_icon_fetch() (should remove from ??)

interworking.c::interworking_continue_anqp() (should remove from ??)

p2p_supplicant.c::wpas_p2p_psk_failure_removal() (should remove from
wpas_p2p_deinit()

wps_supplicant.c::wpas_wps_pbc_overlap_cb() (should remove from
wpas_wps_deinit())

Care to submit some patches for these issues?

Dan

> Could this approach affect the application in a way that you are aware of?
> Or is there maybe a better way to avoid these crashes?
> 
> Thanks,
> Alexandru
> 
> 
> On Fri, Mar 28, 2014 at 11:12 AM, Alexandru Costache <
> alexandru.costache.100 at gmail.com> wrote:
> 
> > Hi,
> >
> > I was running a stress test with connman 1.19 and wpa_supplicant 2.0, more
> > exactly a script which enables and disables tethering in a loop
> >
> > for i in {1..100};
> > do
> > connmanctl tether wifi on test12 testTest123!
> > sleep 1;
> > connmanctl tether wifi off
> > sleep 1;
> > done
> >
> > Sooner or later a race occurs and wpa_supplicant crashes, at different
> > points. It looks like when wpa_supplicant_scan() or
> > wpa_supplicant_delayed_sched_scan_timeout() are scheduled to run, that
> > wpa_supplicant data provided is no longer valid.
> >
> > Usually backtraces are similar to the ones below:
> >
> > 0xb7738424 in __kernel_vsyscall ()
> > (gdb) bt
> > #0  0xb7738424 in __kernel_vsyscall ()
> > #1  0xb7394cc1 in raise () from /lib/libc.so.6
> > #2  0xb73980ee in abort () from /lib/libc.so.6
> > #3  0xb74fcc95 in _dbus_abort () from /usr/lib/libdbus-1.so.3
> > #4  0xb74f1e46 in _dbus_warn_check_failed () from /usr/lib/libdbus-1.so.3
> > #5  0xb74e052e in dbus_message_unref () from /usr/lib/libdbus-1.so.3
> > #6  0xb74d295b in _dbus_connection_unlock () from /usr/lib/libdbus-1.so.3
> > #7  0xb74d7f41 in  () from /usr/lib/libdbus-1.so.3
> > #8  0x080c4aea in wpa_dbus_mark_property_changed (iface=0xb74c4660,
> > path=0x82c3318 "\210G,\b\360'+\b\020", interface=0x8118940
> > "fi.w1.wpa_supplicant1.Interface",
> >     property=0x8118690 "ApScan") at dbus/dbus_new_helpers.c:882
> > #9  0x080c6c35 in wpas_dbus_signal_prop_changed (wpa_s=0x82b9430,
> > property=WPAS_DBUS_PROP_AP_SCAN) at dbus/dbus_new.c:1763
> > #10 0x0805299a in wpas_notify_ap_scan_changed (wpa_s=0x82b9430) at
> > notify.c:114
> > #11 0x080de495 in wpa_supplicant_scan (eloop_ctx=0x82b9430,
> > timeout_ctx=0x0) at scan.c:620
> > #12 0x080579d3 in eloop_run () at ../src/utils/eloop.c:783
> > #13 0x080d486e in wpa_supplicant_run (global=0x82ae630) at
> > wpa_supplicant.c:3381
> > #14 0x080dff86 in main (argc=4, argv=0xbfbc42d4) at main.c:307
> >
> >     Invalid path
> >
> >
> > =============================================
> >
> >
> > #0  0xb74e07f0 in __memcpy_ssse3 () from /lib/libc.so.6
> > #1  0x080dee33 in wpa_supplicant_build_filter_ssids (eloop_ctx=0x988d1f0,
> > timeout_ctx=0x0) at /usr/include/bits/string3.h:52
> > #2  wpa_supplicant_scan (eloop_ctx=0x988d1f0, timeout_ctx=0x0) at
> > scan.c:829
> > #3  0x080579d3 in eloop_run () at ../src/utils/eloop.c:783
> > #4  0x080d490e in wpa_supplicant_run (global=0x9877630) at
> > wpa_supplicant.c:3381
> > #5  0x080e0026 in main (argc=3, argv=0xbf826c54) at main.c:307
> >
> >  ssid->ssid = 0x80
> >
> >
> > =============================================
> >
> >
> > (gdb) bt
> > #0  wpa_supplicant_req_sched_scan (wpa_s=0x90c87d0) at scan.c:1011
> > #1  0x080de2eb in wpa_supplicant_delayed_sched_scan_timeout
> > (eloop_ctx=0x90c87d0, timeout_ctx=0x0) at scan.c:246
> > #2  0x080579d3 in eloop_run () at ../src/utils/eloop.c:783
> > #3  0x080d490e in wpa_supplicant_run (global=0x90b6630) at
> > wpa_supplicant.c:3381
> > #4  0x080e0026 in main (argc=3, argv=0xbf96e414) at main.c:307
> >
> > (gdb) p wpa_s->max_sched_scan_ssids
> > $1 = -1217658744
> >
> >
> > ==============================================
> >
> > (gdb) bt
> > #0  0x080dd803 in wpa_supplicant_enabled_networks (wpa_s=0x9b95178) at
> > scan.c:96
> > #1  0x080de3f8 in wpa_supplicant_scan (eloop_ctx=0x9b95178,
> > timeout_ctx=0x0) at scan.c:607
> > #2  0x080579d3 in eloop_run () at ../src/utils/eloop.c:783
> > #3  0x080d490e in wpa_supplicant_run (global=0x9b7e630) at
> > wpa_supplicant.c:3381
> > #4  0x080e0026 in main (argc=3, argv=0xbffae3e4) at main.c:307
> >
> > (gdb) p wpa_s->conf
> > $1 = (struct wpa_config *) 0x0
> >
> >
> > ==============================================
> >
> > (gdb) bt
> > #0  wpa_drv_set_operstate (wpa_s=0x869e378, state=WPA_DISCONNECTED) at
> > driver_i.h:214
> > #1  wpa_supplicant_set_state (wpa_s=0x869e378, state=WPA_DISCONNECTED) at
> > wpa_supplicant.c:680
> > #2  0x080de630 in wpa_supplicant_scan (eloop_ctx=0x869e378,
> > timeout_ctx=0x0) at scan.c:602
> > #3  0x080579d3 in eloop_run () at ../src/utils/eloop.c:783
> > #4  0x080d490e in wpa_supplicant_run (global=0x8693630) at
> > wpa_supplicant.c:3381
> > #5  0x080e0026 in main (argc=3, argv=0xbfc615b4) at main.c:307
> >
> > ==============================================
> > I'm not very familiar with what happens here so I don't know how the fix
> > should look like. Maybe this has already been fixed in a later version?
> > Please let me know if there's other info that I should provide, like
> > detailed debug logs.
> >
> > Thank you,
> > Alexandru
> >
> _______________________________________________
> HostAP mailing list
> HostAP at lists.shmoo.com
> http://lists.shmoo.com/mailman/listinfo/hostap