[PATCH blktests v4 00/17] support test case repeat by different conditions

[Shinichiro Kawasaki]

On May 04, 2024 / 17:14, Shin'ichiro Kawasaki wrote:
> In the recent discussion for nvme test group [1], two pain points were mentioned
> regarding the test case runs.
> 
> 1) Several test cases in nvme test group do exactly the same test except the
>    NVME transport backend set up condition difference (device vs. file). This
>    results in duplicate test script codes. It is desired to unify the test cases
>    and run them repeatedly with the different conditions.
> 
> 2) NVME transport types can be specified with nvme_trtype parameter so that the
>    same tests can be run for various transport types. However, some test cases
>    do not depend on the transport types. They are repeated in multiple runs for
>    the various transport types under the exact same conditions. It is desired to
>    repeat the test cases only when such repetition is required.
> 
> [1] https://lore.kernel.org/linux-block/w2eaegjopbah5qbjsvpnrwln2t5dr7mv3v4n2e63m5tjqiochm@uonrjm2i2g72/
> 
> One idea to address these pain points is to add the test repeat feature to the
> nvme test group. However, Daniel questioned if the feature could be implemented
> in the blktests framework. Actually, a similar feature has already been
> implemented to repeat some test cases for non-zoned block devices and zoned
> block devices. However, this feature is implemented only for the zoned and non-
> zoned device conditions. It can not fulfill the desires for nvme test group.
> 
> This series proposes to generalize the feature in the blktests framework to
> repeat the test cases with different conditions. Introduce a new function
> set_conditions() that each test case can define and instruct the framework to
> repeat the test case. This series applies this feature to nvme test group so
> that the test cases can be repeated for NVME transport types and backend types
> in the ideal way. For this purpose, this series introduces new config parameters
> NVMET_TRTYPES and NVMET_BLKDEV_TYPES. Taking this chance, it renames other
> lowercase config parameters nvme_img_size, nvme_num_iter and use_rxe to
> uppercase to follow the guide for environment variables.
[...]

Thank you for the discussions and the comments on the series. This series has
got applied.