[PATCH v2 02/18] selftest/firmware: Add selftest timeout in settings

[Scott Branden]

works.

On 2020-07-22 12:30 p.m., Kees Cook wrote:
> The firmware tests would always time out for me. Add a correct timeout,
> including details on how the value was reached. Additionally allow the
> test harness to skip comments in settings files and report how long a
> given timeout was.
>
> Signed-off-by: Kees Cook <keescook at chromium.org>
Acked-by: Scott Branden <scott.branden at broadcom.com>
> ---
>   tools/testing/selftests/firmware/settings   | 8 ++++++++
>   tools/testing/selftests/kselftest/runner.sh | 6 +++++-
>   2 files changed, 13 insertions(+), 1 deletion(-)
>   create mode 100644 tools/testing/selftests/firmware/settings
>
> diff --git a/tools/testing/selftests/firmware/settings b/tools/testing/selftests/firmware/settings
> new file mode 100644
> index 000000000000..085e664ee093
> --- /dev/null
> +++ b/tools/testing/selftests/firmware/settings
> @@ -0,0 +1,8 @@
> +# The async firmware timeout is set to 1 second (but ends up being effectively
> +# 2 seconds). There are 3 test configs, each done with and without firmware
> +# present, each with 2 "nowait" functions tested 5 times. Expected time for a
> +# normal execution should be 2 * 3 * 2 * 2 * 5 = 120 seconds for those alone.
> +# Additionally, fw_fallback may take 5 seconds for internal timeouts in each
> +# of the 3 configs, so at least another 15 seconds are needed. Add another
> +# 10 seconds for each testing config: 120 + 15 + 30
> +timeout=165
> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> index 676b3a8b114d..cd5ddf979f15 100644
> --- a/tools/testing/selftests/kselftest/runner.sh
> +++ b/tools/testing/selftests/kselftest/runner.sh
> @@ -53,6 +53,10 @@ run_one()
>   	settings="$BASE_DIR/$DIR/settings"
>   	if [ -r "$settings" ] ; then
>   		while read line ; do
> +			# Skip comments.
> +			if echo "$line" | grep -q '^#'; then
> +				continue
> +			fi
>   			field=$(echo "$line" | cut -d= -f1)
>   			value=$(echo "$line" | cut -d= -f2-)
>   			eval "kselftest_$field"="$value"
> @@ -80,7 +84,7 @@ run_one()
>   			echo "not ok $test_num $TEST_HDR_MSG # SKIP"
>   		elif [ $rc -eq $timeout_rc ]; then \
>   			echo "#"
> -			echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"
> +			echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT $kselftest_timeout seconds"
>   		else
>   			echo "not ok $test_num $TEST_HDR_MSG # exit=$rc"
>   		fi)