[PATCH v4 00/11] kunit: create a centralized executor to dispatch all KUnit tests
davidgow at google.com
Wed Jun 24 21:47:00 EDT 2020
Glad this is back out there: a couple of minor nitpicks below:
On Thu, Jun 25, 2020 at 4:58 AM Brendan Higgins
<brendanhiggins at google.com> wrote:
> ## TL;DR
> This patchset adds a centralized executor to dispatch tests rather than
> relying on late_initcall to schedule each test suite separately along
> with a couple of new features that depend on it.
> Also, sorry for the extreme delay in getting this out. Part of the delay
> came from finding that there were actually several architectures that
> the previous revision of this patchset didn't work on, so I went through
> and attempted to test this patchset on every architecture - more on that
> ## What am I trying to do?
> Conceptually, I am trying to provide a mechanism by which test suites
> can be grouped together so that they can be reasoned about collectively.
> The last two of three patches in this series add features which depend
> on this:
> PATCH 8/11 Prints out a test plan right before KUnit tests are run;
> this is valuable because it makes it possible for a test
> harness to detect whether the number of tests run matches the
> number of tests expected to be run, ensuring that no tests
> silently failed. The test plan includes a count of tests that
> will run. With the centralized executor, the tests are
> located in a single data structure and thus can be counted.
This appears to actually be patch 9/11.
> PATCH 9/11 Add a new kernel command-line option which allows the user to
> specify that the kernel poweroff, halt, or reboot after
> completing all KUnit tests; this is very handy for running
> KUnit tests on UML or a VM so that the UML/VM process exits
> cleanly immediately after running all tests without needing a
> special initramfs. The centralized executor provides a
> definitive point when all tests have completed and the
> poweroff, halt, or reboot could occur.
This seems to have been merged into the above patch (9/11).
> In addition, by dispatching tests from a single location, we can
> guarantee that all KUnit tests run after late_init is complete, which
> was a concern during the initial KUnit patchset review (this has not
> been a problem in practice, but resolving with certainty is nevertheless
> Other use cases for this exist, but the above features should provide an
> idea of the value that this could provide.
> ## Changes since last revision:
> - On the last revision I got some messages from 0day that showed that
> this patchset didn't work on several architectures, one issue that
> this patchset addresses is that we were aligning both memory segments
> as well as structures in the segments to specific byte boundaries
> which was incorrect.
> - The issue mentioned above also caused me to test on additional
> architectures which revealed that some architectures other than UML
> do not use the default init linker section macro that most
> architectures use. There are now several new patches (2, 3, 4, and
> - Fixed a formatting consistency issue in the kernel params
> documentation patch (9/9).
> - Add a brief blurb on how and when the kunit_test_suite macro works.
> ## Remaining work to be done:
> The only architecture for which I was able to get a compiler, but was
> apparently unable to get KUnit into a section that the executor to see
> was m68k - not sure why.
> Alan Maguire (1):
> kunit: test: create a single centralized executor for all tests
> Brendan Higgins (10):
> vmlinux.lds.h: add linker section for KUnit test suites
> arch: arm64: add linker section for KUnit test suites
> arch: microblaze: add linker section for KUnit test suites
> arch: powerpc: add linker section for KUnit test suites
> arch: um: add linker section for KUnit test suites
> arch: xtensa: add linker section for KUnit test suites
> init: main: add KUnit to kernel init
> kunit: test: add test plan to KUnit TAP format
> Documentation: Add kunit_shutdown to kernel-parameters.txt
> Documentation: kunit: add a brief blurb about kunit_test_suite
> .../admin-guide/kernel-parameters.txt | 8 ++
> Documentation/dev-tools/kunit/usage.rst | 5 ++
> arch/arm64/kernel/vmlinux.lds.S | 3 +
> arch/microblaze/kernel/vmlinux.lds.S | 4 +
> arch/powerpc/kernel/vmlinux.lds.S | 4 +
> arch/um/include/asm/common.lds.S | 4 +
> arch/xtensa/kernel/vmlinux.lds.S | 4 +
> include/asm-generic/vmlinux.lds.h | 8 ++
> include/kunit/test.h | 73 ++++++++++++-----
> init/main.c | 4 +
> lib/kunit/Makefile | 3 +-
> lib/kunit/executor.c | 63 +++++++++++++++
> lib/kunit/test.c | 13 +--
> tools/testing/kunit/kunit_kernel.py | 2 +-
> tools/testing/kunit/kunit_parser.py | 74 +++++++++++++++---
> .../test_is_test_passed-all_passed.log | Bin 1562 -> 1567 bytes
> .../test_data/test_is_test_passed-crash.log | Bin 3016 -> 3021 bytes
> .../test_data/test_is_test_passed-failure.log | Bin 1700 -> 1705 bytes
> 18 files changed, 226 insertions(+), 46 deletions(-)
> create mode 100644 lib/kunit/executor.c
> base-commit: 4333a9b0b67bb4e8bcd91bdd80da80b0ec151162
> prerequisite-patch-id: 2d4b5aa9fa8ada9ae04c8584b47c299a822b9455
> prerequisite-patch-id: 582b6d9d28ce4b71628890ec832df6522ca68de0
> These patches are available for download with dependencies here:
>  https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
>  https://patchwork.kernel.org/patch/11383635/
More information about the linux-arm-kernel