[PATCH v2 12/13] test: self: run selftests as part of the pytest suite

[Rouven Czerwinski]

Hi Ahamd,

On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
> We don't want to enable self tests in the normal configs as they may,
> in future, bloat size needlessly. Enable it instead in the base.cfg
> fragment and add a test that verifies the selftest command
> runs without errors.
> 
> Selftests can be run on startup for CONFIG_SHELL_NONE systems. This is
> not implemented here. For such systems the test will be skipped
> as CONFIG_CMD_SELFTEST won't be defined. To manually skip with
> emulate.pl, add --no-kconfig-base.
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum at pengutronix.de>
> ---
>  test/kconfig/base.cfg      | 4 ++++
>  test/py/test_bselftests.py | 8 ++++++++
>  2 files changed, 12 insertions(+)
>  create mode 100644 test/py/test_bselftests.py
> 
> diff --git a/test/kconfig/base.cfg b/test/kconfig/base.cfg
> index e69de29bb2d1..6a9f68349816 100644
> --- a/test/kconfig/base.cfg
> +++ b/test/kconfig/base.cfg
> @@ -0,0 +1,4 @@
> +CONFIG_TEST=y
> +CONFIG_SELFTEST=y
> +CONFIG_CMD_SELFTEST=y
> +CONFIG_SELFTEST_ENABLE_ALL=y
> diff --git a/test/py/test_bselftests.py b/test/py/test_bselftests.py
> new file mode 100644
> index 000000000000..48e9d38c12fc
> --- /dev/null
> +++ b/test/py/test_bselftests.py
> @@ -0,0 +1,8 @@
> +import pytest
> +from .helper import *
> +
> +def test_bselftest(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_SELFTEST")
> +
> +    _, _, returncode = barebox.run('selftest')
> +    assert returncode == 0

This will give you the default timeout of 30 seconds for the command
run. As long as you are confident the selftests are going to stay under
this limit this is fine, however you'll probably want to make this
explicit :-)

Also you probably want something akin to this:

    stdout, stderr, returncode = barebox.run('selftest')
    assert returncode == 0, "selftest failed:\n {}\n".format("".join(stdout))

Which will output the selftest stdout if the selftest fails.

- rcz