getting users involved in on-device testing

[John Crispin]

On 05/05/2016 13:17, Bruno Randolf wrote:
> Hi,
> 
> On 05/05/16 11:05, John Crispin wrote:
>>> We could also try to integrate this with some automated testing.
> 
> Some automated way to do testing and submit the results would definetly
> be good and preferable to a form-based webpage. Of course basic stuff
> like "boots", "wifi worked", "leds worked" require manual checking, but
> others like iperf or ssl benchmarks should be standardized and done the
> same way on all machines, to make the results comparable.
> 
> If I imagine that I'd volunteer to test on say 5 different devices, for
> every image release, I'd definitely like to have a script that would do
> all the standard tests and submit the results, optimally thru a package
> that I can install on the device itself. Maybe even some external setup
> like IP of iperf server could be standardized or configurable. Then on
> the device to be tested I'd image running something like this:
> 
> root at LEDE-device:~# lede-test
> Does the system LED blink? [y/n]
> Does LED xyz blink? [y/n]
> Press Button X... [button detected]
> Ping ethernet host IP 192.168.0.221 ... [OK]
> Ping Wifi host IP 192.168.1.221 ... [OK]
> Iperf test... [result]
> SSL benchmark [result]
> Additional comments: ...
> Submit results to Server? [y/n]

Please enter OTP to start upload of results [...]

love the idea, i'll add it to my proposal.

	John

> 



> As the test runs on the device it should be written either in C or ash
> so we can also have it on smaller devices.
> 
> bruno
>