OpenWrt Next Generation Ideas

[Felix Fietkau]

On 31.03.23 14:52, Arınç ÜNAL wrote:
> On 31.03.2023 14:33, Daniel Golle wrote:
>> Hi!
>> 
>> On Fri, Mar 31, 2023 at 12:44:12PM +0300, Arınç ÜNAL wrote:
>>> Hi all,
>>>
>>> These are the ideas I've been thinking about for the future of OpenWrt for a
>>> while. It looks complete enough to share it with all of you.
>>>
>>> I'm willing to put a great deal of effort to get as much out-of-tree patches
>>> on mainline Linux as possible.
>>>
>>> You can make a comment on Notion or discuss it here, I'm wondering if the
>>> ideas are feasible and how well it would benefit the people maintaining
>>> OpenWrt.
>>>
>>> https://arinc9.notion.site/OpenWrt-next-gen-ideas-6db745e7584b4823950291c96f2326bb
>> 
>> I will comment here, I don't have an account on Notion and it seems
>> to be required to be able to comment there.
>> 
>>> defconfig for each device instead of config for each (sub)target.
>> 
>> Given that we support thousands of devices this will not only increase
>> the time needed to build a release or snapshot by several magnitudes,
>> but also make debugging **much** harder. As of now, all devices of a
>> subtarget are using the same kernel, hence e.g. symbol offsets in a
>> kernel stack dump match for all of them. To reproduce or investigate
>> a problem it's hence enough to have similar hardware, not necessarily
>> the exact same board. As we are already lacking testers and maintainers
>> for the relatively small amount of targets/subtargets, have a build for
>> each board would make things much worse...
>> 
>> Per-device builds would also be an invitation to downstream users to
>> introduce device-specific (kernel-)hacks. If you want that, better go
>> for OpenEmbedded.
>> 
>> We can modularize things more or even have more sub-targets if it's
>> really needed to save space.
>> 
>> The disadvantages outweight the advantages imho when it comes to having
>> a complete kernel build for each device.
> 
> Hmm, what about we enable the bare minimum of kernel options for a
> target, which is already how it is, then select the rest as kernel
> modules (like on the makefile of a target for each device) on the
> defconfig for each device? So, in the end, it wouldn't be any different
> than selecting a kernel module package from the OpenWrt SDK which, I
> believe, does not change the symbol offsets in the kernel stack.
> 
> My reason for pushing for the use defconfigs is that anyone can build
> the Linux kernel for their device, without needing OpenWrt. So the work
> for adding support for a device would benefit far more people.
There are a lot of options in the OpenWrt menuconfig (including kmod 
package selections) which *do* affect the kernel compilation in a major 
way. They can change struct sizes, enabled features, affect compiled-in 
code inside functions, etc.

For maintenance, I strongly believe that switching from the current 
system to maintaining full kernel config files is a huge step backwards, 
because maintaining individual config files makes them so much easier to 
accidentally go out of sync with each other.

If you want to make it easier to build per-device kernels outside of
OpenWrt, I'd recommend adding a build system feature to export target 
defconfig files.

>>> Either submit all out-of-tree patches on OpenWrt to Linux or get rid
>>> of them and find a better solution for what the unacceptable patch
>>> does.
>> 
>> This would of course be great, but especially for legacy devices it may
>> not be possible in many cases. Think of all the devices stuck on
>> swconfig, just to name one example... Think of all the completely
>> broken vendor bootloaders which require hacks (mangeling kernel cmdline
>> and such) and cannot easily be replaced...
> 
> Those can stay until eventually the support for them will be dropped on
> newer OpenWrt versions. I believe there are a lot of out-of-tree patches
> that are not for old devices and can be on mainline Linux.
I believe that we will always need to carry some patches even for newer 
platforms which will never be accepted upstream, and probably have no 
place there.
One example is the NMBM NAND mapping code on MTK platforms. It was 
written by MTK long after UBI had already established itself as a much 
better solution. And yet, we still have to deal with it because we don't 
want to make it a hard requirement to reflash U-Boot on every single 
newer NAND based MTK device.
We also need to carry some hacks to Kconfig files to support our kmod 
selection and our out-of-tree mac80211/cfg80211 builds.

- Felix