image size on 20.xx builds -- what about a new SMALL target

[Rich Brown]

I'm cc'ing the OpenWrt-Adm list on this as well, because I smell a policy decision looming. Some thoughts:

1) In my opinion, we should not aim for a third (Tiny/Small/Normal) build for 20.xx. We don't need any more hurdles to overcome before we ship an RC1 version.

2) I had not understood how "close to the wire" we are with 8MByte machines. (That is, vendors reserving 1.5MBytes+ for their own purposes, leaving us with a max of 6.5Mbytes...)

3) Instead of creating an entire new "Small" build, with its attendant new directories, enormous load on the build infrastructure, etc., I wonder if we should spend the engineering time improving the chef.libremesh.org ImageBuilder facility. This has the following good attributes:
	- People can build an image of whatever size they want, to their own specifications
	- It gives us information about the popularity of various devices, and their common configurations
	- We would no longer need to guess what features to put into the "default" stable release
	- I also envision an enhancement to chef.libremesh.org that lets you preview the size of already-built images that contain the packages you have specified. This dramatically helps with caching, and makes the user experience vastly better as well. Someone can either wait a long time while their image builds, or they can just download one that's pre-built, potentially with a few extra packages.

Thoughts? Thanks.

Rich

> On Aug 3, 2020, at 8:40 AM, Henrique de Moraes Holschuh <henrique at nic.br> wrote:
> 
> On 02/08/2020 19:11, Daniel Golle wrote:
>> On Sun, Aug 02, 2020 at 11:49:54PM +0200, Bjoern Franke wrote:
>>> Hi,
>>> 
>>>> 
>>>> Your Filesystem has a size of 5.81M. MT7620 has a kernel size of around 2M, which exceeds the available space on flash.
>>> 
>>> On 19.07.3, the listed lineup of packages has 5.54M and it fits:
>>> 
>>> Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
>>> 	compressed data, compressed metadata, compressed fragments,
>>> 	no xattrs, compressed ids
>>> 	duplicates are removed
>>> Filesystem size 5674.78 Kbytes (5.54 Mbytes)
>>> 	35.35% of uncompressed filesystem size (16051.56 Kbytes)
>>> Inode table size 13736 bytes (13.41 Kbytes)
>>> 	22.46% of uncompressed inode table size (61170 bytes)
>>> Directory table size 17662 bytes (17.25 Kbytes)
>>> 	44.53% of uncompressed directory table size (39661 bytes)
>>> 
>>> 
>>> 
>>>> You can try to remove other packages (such as ppp) from the image. Stripping LuCI should also save a lot of space, given you only
>>>> need it for the initial installation.
>>>> 
>>> 
>>> Yes, but I was wondering why the image gets 300Kb bigger on 20.xx.
>> Apart from the usual kernel growth we have also enabled a bunch of
>> previously disabled kernel features on targets which are not marked
>> as SMALL_FLASH[1]. MT7620 is a mixed bag in that regard and it would
>> make sense to split it similar to ath79 into a 'tiny' and a 'generic'
>> variant. The 'mt7620-generic' variant could then also ship with NAND
>> support and support for Xiami miRouter 3. The 'mt7620-tiny' variant
>> would have SMALL_FLASH set and hence come without the features
>> enabled in [1] which should give us ~ 200kB.
>> [1]: https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=fcb41decf6c622482b20af45a77e62db8d95046e
> 
> This sort of commit can easily be a problem even for 8MiB FLASH devices.
> 
> (Some?) vendors are adding a lot of cruft we can't simply jettison from FLASH, e.g. the dual-bootloaders and "http recovery" stuff on TP-Link Archer C60v3 and later, and I understand, several other of their models.
> 
> That effectively reduces usable FLASH enough to matter on 8MiB devices. It is already a problem on OpenWrt 19.07 -- for anything that needs FLASH-filling monsters like OpenSSL and ca-certificates, at least.
> 
> IMO, if things keep going that way (and there is no reason to believe they would not -- or should not, for that matter), we would truly benefit from a SMALL variant soon (not just TINY).  And by soon, I do mean OpenWrt 20.x if at all possible :-(
> 
> Note that I AM NOT speaking against changes that end up increasing core openwrt size.  I am just stating that such changes show the need of one or two knobs for anything that is not possible to build as a "module" kernel-side or as an optional package *and* isn't something you'd strictly need on smaller devices.
> 
> There's TINY, but it removes too much -- being the single true/false knob that exists ATM -- and it seems to be really set up for devices with something like 3-5MiB usable FLASH.  It is also too useful to drop/repurpose, so I am *not* suggesting that.
> 
> A new SMALL could target 5.5MiB to ~6.5MiB *usable* FLASH, IMHO.  One would look at everything that is !TINY to see if it should also be !SMALL, and review for !SMALL what already landed that increases the static kernel size and always-installed package size.
> 
> For not-yet merged changes, you'd ask for !TINY and perhaps !SMALL if they increase in any relevant way the static kernel size, or always-installed package size.  Note that for TINY, 10KiB is already relevant. For !SMALL I am not sure, but I'd say 50KiB as a first try. And exceptional cases would only need a sufficiently good justification in the commit log (as in, enough to convince the core team).
> 
> -- 
> Henrique de Moraes Holschuh
> 
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel