[PATCH 0/2] Add mesh test cases

[Jouni Malinen]

On Tue, Mar 28, 2017 at 10:30:08AM +0900, Masashi Honma wrote:
> These two patches add mesh test cases.
> The first test wmediumd_path_movement has 86% success rate(succeeded 86
> of 100 tests), beacause it works based on probability.

Are you saying this test case is expected to fail 14% of the times? That
is not really acceptable for hwsim test cases. These are supposed to
pass 100% of the time and every FAIL report results in extra work when
figuring out if there were regressions somewhere.

If a test case is expected to fail every now and then even if nothing is
wrong in the implementation, that test case should not be reporting
FAIL. It might be possible to run the test multiple times and report
success on the first case the test passes. Or alternatively, not report
FAIL at all if the goal is to just add coverage for some area.

-- 
Jouni Malinen                                            PGP id EFC895FA