Reusable Github Actions and containers [Was: Re: [PATCH uci 2/2] CI: Add github action]

[Petr Štetiar]

Hauke Mehrtens <hauke at hauke-m.de> [2023-03-09 00:18:10]:

Hi,

thanks for taking care, LGTM for a start.

I'll just provide my past experience, something to consider as we're likely
going to bump into those in the long term, so ideally take them into the
account in the long term.

> clang 14 generates debug information in DWARF 5 format, but valgrind
> 19.0 does not support that. Install valgrind 20.0 from experimental
> which supports it.

IMO we should aim for reproducible results, thus we should likely provide tools
containers with a known versions inside, so anyone is able to get same results
using the source code Git hash and tool container version.

Your current approach is highly dynamic, so if you're lucky enough, you might
get a green CI light in the PR branch as the pipeline was run for example 7
days ago, so you're going to merge it into the upstream branch, but then it's
going to fail as meanwhile Debian has bumped several packages and you're going
to get a CI pipeline failure.

IMO there should be a tools container Git repository, where we could build,
test and provide versioned containers, for example:

 ghcr.io/openwrt/ci-tools/native-testing/clang14:20230305
 ghcr.io/openwrt/ci-tools/native-testing/clang15:20230305
 ghcr.io/openwrt/ci-tools/native-testing/gcc-8:20230305
 ghcr.io/openwrt/ci-tools/native-testing/gcc-11:20230305
 ghcr.io/openwrt/ci-tools/native-testing/gcc-12:20230305

We want to test with with multiple compiler versions as those tested changes
might be backported into stable branches, we should even consider to have
stable branches in every project so we could CI test them there as well, folks
would simply create a backport PR in the respective project.

> +++ b/.github/workflows/test.yml
> @@ -0,0 +1,83 @@
> +name: 'Run tests'

We've like 30+ C projects which we should likely cover in the end, thus
considering possible additional 2 stable branches in each, it's around 60+ of
similar workflow files duplicated in various repositories.

I would consider this future maintenance overhead (imagine just a simple
change or a fix being propagated into 60+ repositories/branches), so I would
recommend to create a shareable Github Action instead:

 uses: openwrt/gh-actions-ci-native
 env:
   CI_NATIVE_TOOLCHAIN: clang14

 uses: openwrt/gh-actions-ci-sdk
 env:
   CI_TARGET_SDK_RELEASE: master
   CI_TARGET_SDK: ath79-generic
   CI_TARGET_BUILD_DEPENDS: uci

You can take a look at the ucode project for a very dated, but still working
GitHub example, some bits are even present in uci repsitory in the .gitlab-ci.yml
file.

> +      - name: Checkout libubox
> +        uses: actions/checkout at v3
> +        with:
> +          repository: openwrt/libubox
> +          path: libubox

This looks like another source of unreliability, similar to the toolchain
versions above. For the start, I would simply include all those dependencies
in the native toolchain container, thus we would need to have separate
containers for each branch:

 ghcr.io/openwrt/ci-tools/native-testing/clang14_snapshot:20230305
 ghcr.io/openwrt/ci-tools/native-testing/clang14_openwrt-22.03:20230305
 ghcr.io/openwrt/ci-tools/native-testing/clang14_openwrt-21.02:20230305

 ghcr.io/openwrt/ci-tools/native-testing/distro_snapshot:20230305 (distro default gcc12)
 ghcr.io/openwrt/ci-tools/native-testing/distro_21.02:20230305 (gcc8)
 ghcr.io/openwrt/ci-tools/native-testing/distro_22.03:20230305 (gcc11)

So in the end, ideally, interested developer can have the same CI
failure/result locally and debug/fix it faster:

 $ git clone https://github.com/openwrt/uci && cd uci
 $ wget -q https://github.com/openwrt/some-project/raw/master/Makefile -O Makefile.ci
 $ make ci-prepare -f Makefile.ci
 $ podman run --rm --tty --interactive --volume $(pwd):/home/build/openwrt \
        ghcr.io/openwrt/ci-tools/native-testing/clang14_snapshot:20230305 \
	make ci-native-build -f Makefile.ci

Have a nice weekend.

Cheers,

Petr