Monocultures are bad [Was: Re: [PATCH] build: Extract trusted-firmware-a.mk]

[Andre Heider]

On 12/10/2020 20:45, Tomasz Maciej Nowak wrote:
> Hi.
> 
> W dniu 12.10.2020 o 16:32, Andre Heider pisze:
>> On 12/10/2020 12:21, Petr Štetiar wrote:
>>> Hauke Mehrtens <hauke at hauke-m.de> [2020-10-11 20:41:21]:
>>>
>>> Hi,
>>>
>>>    (I've expressed this already in my initial review[1] which was not addressed
>>>     and pushed anyway.)
>>>
>>>> +TFA_MAKE_FLAGS += \
>>>> +        CROSS_CM3=$(STAGING_DIR_IMAGE)/$(LINARO_NAME)-$(LINARO_RELEASE).$(LINARO_VERSION)/bin/arm-linux-gnueabi- \
>>>
>>> AFAIU this is Linux only toolchain, so we should either stop pretending, that
>>> we support usage of FreeBSD/macOS as the build host OS or fix this for good.
>>
>> As with other boards, we need a secondary cross compiler for a cortex m3 cpu here. Yes, currently a linux binary toolchain is downloaded. It's not optimal, and I'm not a fan of this solution either. But that's already used, this patch doesn't change that fact.
> 
> But It'll force other targets to comply with that or workaround it.
> 
>>
>> Building a firmware for these boards is a total clusterfuck. So one goal of all the espressobin PRs has been for openwrt to provide binary firmware files. With that in mind I see two options:
>>
>> 1) Error out on !Linux build hosts. Maybe that's good enough, because we mainly want it to work on buildbot
>> 2) build a cm3 cross compiler
>>
>> I'd go for 2), but the main question is if that's acceptable, because every buildslave would need to build it. We just need a bare metal toolchain though, so just binutils and gcc, no libc, g++ and whatnot. That's actually not as much as it sounds. And if we had that, the sunxi and rockchip atf package could use it too instead of downloading binaries (yay).
> 
> That's also what I suggested on the same PR [2]. But building another toolchain consumes resources on buildbot and I think people managing buildbots won't like that.
> 
>>
>>> BTW I'm wondering why this can't be handled in the same way
>>> arm-trusted-firmware-rockchip is?
>>
>> For sunxi and rockchip, you start with an atf build, and then pass the filename of the resulting binary to the u-boot build system.
> 
> U-Boot build system cares little for ATF. It doesn't need it to build itself.

It doesn't, but for those platforms u-boot's binman is used to assemble 
a firmware in a digestible form. Which is why it's easy to ship atf 
binaries there.

> 
>>
>> For A3700, it's the other way around. You cannot ship atf binaries (yuck), because atf is the last element in the build chain.
> 
> You can change that. The resulting bootloader is actually an FIP image[3] packed with fiptool, which is part of ATF build system. You can build each binary (wtmi, atf, u-boot) individually and later combine them with fiptool (never done that personally). As I understand it, because fiptool is part of ATF, the build process was tailored so that You can build ready-to-flash bootloader in one go.
> What could be done to split the process:
> a) modify ATF Makefile to skip packaging the FIP image and do it when requested from provided components,
> b) build the wtmi binaries with https://github.com/atf-builds (I don't know if that'll be accepted, since that's not exactly ATF and it needs additional dependencies) or create another entity to do that,
> c) download binaries and create the FIP image within ATF package.
> 
> or
> 
> a) modify ATF Makefile to skip packaging the FIP image and build ATF with wtmi binaries with https://github.com/atf-builds,
> b) download the binaries within ATF package,
> c) add fiptool to tools,
> d) combine binaries with fiptool in whichever OpenWrt buildroot place You want.

Worth mentioning is turrix mox, which has its own wtmi and firmware 
creation:
https://gitlab.nic.cz/turris/mox-boot-builder/-/tree/master

> I don't have time to tackle that ATM and it's waaaay on the bottom on my ToDo list with very low priority, so whoever has motivation and time please do.

The whole build process with all its dependencies is a mess, there're 
even more ways to clean that up. But in the end anything that doesn't 
make it to atf's or u-boot's upstream is more or less unmaintainable. 
And that's probably not easy, since it implies testing on Marvell's end.

> 
>>
>>>
>>> 1. https://github.com/openwrt/openwrt/pull/2521#pullrequestreview-309946728
> 
> 2. https://github.com/openwrt/openwrt/pull/2521#issuecomment-549079784
> 3. https://trustedfirmware-a.readthedocs.io/en/latest/plat/marvell/armada/build.html#build-output
> 
> Regards
>