OpenWrt Next Generation Ideas

[Arınç ÜNAL]

On 31.03.2023 14:33, Daniel Golle wrote:
> Hi!
> 
> On Fri, Mar 31, 2023 at 12:44:12PM +0300, Arınç ÜNAL wrote:
>> Hi all,
>>
>> These are the ideas I've been thinking about for the future of OpenWrt for a
>> while. It looks complete enough to share it with all of you.
>>
>> I'm willing to put a great deal of effort to get as much out-of-tree patches
>> on mainline Linux as possible.
>>
>> You can make a comment on Notion or discuss it here, I'm wondering if the
>> ideas are feasible and how well it would benefit the people maintaining
>> OpenWrt.
>>
>> https://arinc9.notion.site/OpenWrt-next-gen-ideas-6db745e7584b4823950291c96f2326bb
> 
> I will comment here, I don't have an account on Notion and it seems
> to be required to be able to comment there.
> 
>> defconfig for each device instead of config for each (sub)target.
> 
> Given that we support thousands of devices this will not only increase
> the time needed to build a release or snapshot by several magnitudes,
> but also make debugging **much** harder. As of now, all devices of a
> subtarget are using the same kernel, hence e.g. symbol offsets in a
> kernel stack dump match for all of them. To reproduce or investigate
> a problem it's hence enough to have similar hardware, not necessarily
> the exact same board. As we are already lacking testers and maintainers
> for the relatively small amount of targets/subtargets, have a build for
> each board would make things much worse...
> 
> Per-device builds would also be an invitation to downstream users to
> introduce device-specific (kernel-)hacks. If you want that, better go
> for OpenEmbedded.
> 
> We can modularize things more or even have more sub-targets if it's
> really needed to save space.
> 
> The disadvantages outweight the advantages imho when it comes to having
> a complete kernel build for each device.

Hmm, what about we enable the bare minimum of kernel options for a 
target, which is already how it is, then select the rest as kernel 
modules (like on the makefile of a target for each device) on the 
defconfig for each device? So, in the end, it wouldn't be any different 
than selecting a kernel module package from the OpenWrt SDK which, I 
believe, does not change the symbol offsets in the kernel stack.

My reason for pushing for the use defconfigs is that anyone can build 
the Linux kernel for their device, without needing OpenWrt. So the work 
for adding support for a device would benefit far more people.

> 
>> Some Modest Virtualization Observations
> 
> How is this related? Virtualization (with OpenWrt being the guest)
> matters on x86 which is usually not that space-constraint.
> And maybe armvirt. If space is a problem for older x86 boards, let's
> disable guest support in x86/legacy.

It was a broad example without any explanation. What caught my attention 
there is the configuration of the kernel that causes problems, if I 
understand correctly.

> 
>> Contribute defconfigs and all the devicetrees on OpenWrt to Linux.
> 
> For devicetrees this would of course be desirable, but also implies a
> lot of work and discussions. If you are up to get it started (ie. setup
> a tree to collect cleaned-up and ready to submit dts), I think it would
> be worth the effort, at least for more recent targets/SoCs.

I've been meaning to do this for the mt7621 SoC devices for months. The 
main roadblock is that some drivers are out-of-tree, like the NAND flash 
so it makes no sense to have them defined on the devicetree. Getting the 
out-of-tree patches on mainline Linux is another step so it'll happen 
eventually.

I'll get this started with my Linux fork on GitHub.

> 
> Regarding defconfigs I don't think we need an individual defconfig for
> each board. The problem here is also that OpenWrt currently has a layered
> approach (generic->target->subtarget) approach while Linux itself has
> a flat approach, and using that would result in a lot of duplication,
> which would in turn make keeping all those defconfigs up-to-date quite
> a lot of work...

Not really, once you make a defconfig for a device, the only case for it 
to change in the future is if a kernel option was renamed, which is very 
rare. Anyway, even in that case, there are known ways to update them by 
bulk.

https://lore.kernel.org/kernel-janitors/20220929090645.1389-1-lukas.bulwahn@gmail.com/

> 
>> Either submit all out-of-tree patches on OpenWrt to Linux or get rid
>> of them and find a better solution for what the unacceptable patch
>> does.
> 
> This would of course be great, but especially for legacy devices it may
> not be possible in many cases. Think of all the devices stuck on
> swconfig, just to name one example... Think of all the completely
> broken vendor bootloaders which require hacks (mangeling kernel cmdline
> and such) and cannot easily be replaced...

Those can stay until eventually the support for them will be dropped on 
newer OpenWrt versions. I believe there are a lot of out-of-tree patches 
that are not for old devices and can be on mainline Linux.

> 
>> Bugfix backporting should happen only after it's accepted to Linux.
>> The patch must be identical to the commit on Linux.
> 
> The wording here might be a bit too strict to support our existing
> mess, but I generally agree. So I'd say 'should' instead of 'must', but
> otherwise agree.
> 
>> Feature backporting should be done only if it's thoroughly tested.
> 
> ... and testing often happens in the OpenWrt tree. So it's a bit of
> a chicken-egg problem, as often developers don't even have all the
> different hardware needed for testing. But generally I agree.
> A way to ease testing *before* pushing to openwrt.git or posting to
> upstream mailing lists would be to have snapshot builds also for
> developers' staging trees.
> 
>> Kernel Solution
>> Make a mode menu.
>> Filesystem only.
> 
> So which kernel headers should be used to build e.g. libc and netlink
> users?
> In a way it is also currently possible to build generic images for
> most architectures using the armvirt, malta and x86 targets. Of course,
> also in this case a kernel is being built.
> 
>> Make a kernel selection menu where the user can choose to feed the
>> kernel directory of their own or use the longterm one defined on the
>> OpenWrt SDK. Add this as a suboption to the full image mode.
> 
> What about CONFIG_EXTERNAL_KERNEL_TREE and friends...?

I've never tried this but looks like it may need a bit of extra options. 
No patches must be applied, kernel module packages must not be compiled, 
etc.

Arınç