[PATCH v2] hwsim tests: Add test cases for AP channel switch announcement

[Jouni Malinen]

On Fri, Jan 10, 2014 at 04:26:33PM +0200, Luca Coelho wrote:
> On Sat, 2013-12-14 at 21:55 -0800, Jouni Malinen wrote:
> > All of these test cases fail with older versions of
> > mac80211/cfg80211/hwsim (e.g., with the version I'm running on my build
> > server). As such, I cannot apply this yet.
> 
> I see.  So we should keep all the tests backward-compatible? Seems like
> a lot of extra work, but makes sense since the kernel version you're
> using to run the tests is independent of the hostap version...

Depends on what you mean with backward-compatible.. There is no need to
be backwards compatible with older wpa_supplicant/hostapd versions, but
there is need for the full setup to work with number of different kernel
versions. This does not need to include everything, but in general, it
would be nice to be able to at minimum report SKIP instead of FAIL if
the kernel does not support something. In general, I think something
went wrong with the kernel interface design or hostapd/wpa_supplicant
implementation if that cannot be done easily since
hostapd/wpa_supplicant themselves will obviously need to work with older
kernel versions and it is useful helpful to know what functionality is
available.

At minimum, there will be a hard blocker on the new test cases not
failing on my test server since I try not to commit anything that breaks
those tests. I may be convinced to update the Backports version there
when proposing a new test case, but in general, there better be a good
reason for this and why the test case could not be made to report SKIP
instead.

> > Would there be any way for hostapd to detect whether CSA functionality
> > is properly supported (I sure hope so..) and then make these test case
> > report SKIP rather than FAIL?
> 
> It can be done, but AFAICT there's currently no way to get this
> information via the control interface.  To do this, I we have to export
> the capa.flags information via the control API.

That has already been done.. However, the current wireless-testing.git
snapshot is still disabling CSA and if I re-enable it in cfg80211 and
use the latest mac80211_hwsim contribution to claim support for it, some
of these test cases fail due to kernel issues:

START ap_csa_1_switch 1/6
PASS ap_csa_1_switch 4.358744 2014-01-14 15:02:09.299603
START ap_csa_1_switch_count_0 2/6
FAIL ap_csa_1_switch_count_0 3.481473 2014-01-14 15:02:12.795904
START ap_csa_1_switch_count_1 3/6
PASS ap_csa_1_switch_count_1 3.312774 2014-01-14 15:02:16.222982
START ap_csa_2_switches 4/6
PASS ap_csa_2_switches 5.610565 2014-01-14 15:02:21.848812
START ap_csa_2_switches_count_0 5/6
FAIL ap_csa_2_switches_count_0 3.788668 2014-01-14 15:02:25.652828
START ap_csa_2_switches_count_1 6/6
PASS ap_csa_2_switches_count_1 3.759073 2014-01-14 15:02:29.531521
failed tests: ap_csa_1_switch_count_0 ap_csa_2_switches_count_0

With both failures due to this one:
WARNING: CPU: 0 PID: 0 at net/mac80211/tx.c:2443 ieee80211_update_csa+0x221/0x2a0()


There needs to be a reasonable expectation of being able to run these
tests with wireless-testing.git successfully before it is justifiable to
add the test cases into hostap.git.
 
-- 
Jouni Malinen                                            PGP id EFC895FA