getting users involved in on-device testing

[Bruno Randolf]

Hi,

On 05/05/16 11:05, John Crispin wrote:
>> We could also try to integrate this with some automated testing.

Some automated way to do testing and submit the results would definetly
be good and preferable to a form-based webpage. Of course basic stuff
like "boots", "wifi worked", "leds worked" require manual checking, but
others like iperf or ssl benchmarks should be standardized and done the
same way on all machines, to make the results comparable.

If I imagine that I'd volunteer to test on say 5 different devices, for
every image release, I'd definitely like to have a script that would do
all the standard tests and submit the results, optimally thru a package
that I can install on the device itself. Maybe even some external setup
like IP of iperf server could be standardized or configurable. Then on
the device to be tested I'd image running something like this:

root at LEDE-device:~# lede-test
Does the system LED blink? [y/n]
Does LED xyz blink? [y/n]
Press Button X... [button detected]
Ping ethernet host IP 192.168.0.221 ... [OK]
Ping Wifi host IP 192.168.1.221 ... [OK]
Iperf test... [result]
SSL benchmark [result]
Additional comments: ...
Submit results to Server? [y/n]

As the test runs on the device it should be written either in C or ash
so we can also have it on smaller devices.

bruno