getting users involved in on-device testing

[John Crispin]

Hi,

It would be nice if there was a process in place that would allow
community members to easily be able to test images on devices and report
the results some place. after some time thinking of different ways to do
this I came up with one possible solution and wanted to know what others
think about this.

we setup a file similar to how the MAINTAINERS file in the kernel.
anyone can add his name/email and boards he would like to be a tester
of. every time an image is built for said board, an email is generated
and sent out to the according persons. this email would contains an otp.
with this credential you could then log into some web frontend and get a
simple mask along the lines of

* wifi worked
* ethernet worked
* leds worked
* buttons worked
* iperf
* ssl benchmark
...

this could all be done in a rather trivial manner and i dont think the
work behind such a setup is really that huge. using otps would for
example eliminate the need for user credential management. results could
simply be stored in files on the backend and then harvested later by a
secondary script etc etc...

next it would be possible to generate static content based on aggregated
data that will show a traffic light style support status for various
boards, listing when it was last tested, with what revision and what the
test results were.

would something like this make sense ?

	John