[LEDE-DEV] Removing package overlay functionality

[Philip Prindeville]

> On Apr 16, 2017, at 9:51 PM, Yousong Zhou <yszhou4tech at gmail.com> wrote:
> 
> On 17 April 2017 at 10:38, Philip Prindeville
> <philipp at redfish-solutions.com> wrote:
>> commit b044bd5921e9644c9df9655bef10cee0af730724
>> Author: Felix Fietkau <nbd at nbd.name>
>> Date:   Mon Apr 3 12:36:35 2017 +0200
>> 
>>    build: remove package makefile overlay functionality
>> 
>>    Recent attempts to use it have shown that it does not work properly
>>    except for a few undocumented cases. It's better to remove this now to
>>    avoid having more people fall into the same trap
>> 
>>    Signed-off-by: Felix Fietkau <nbd at nbd.name>
>> 
>> 
>> I was using the overlay functionality, having been steered toward it after my suggestion of adding pre- and post-install hooks that could be added by the developer to customize the build process was nixed (for instance, if you want to run a trivial sed script on a config file which is generated from a .in file and therefore hard to patch).
>> 
>> Yousong steered me toward using the Build/IncludeOverlay mechanism (see the thread “Makefile Question” around 13 Feb 2017).
>> 
>> Now I abruptly found the overlay mechanism has been deleted, and I don’t recall seeing any discussion on whether to do this or not, just the following post script after the fact:
>> 
>>> I agree. I've pushed a commit to my staging tree that removes package
>>> overlay support.
>> 
>> If it’s hard to use, it should be fixed, not removed.  If it can’t be fixed, then it should be better documents.
>> 
>> Telling someone to use “method X” and then 2 months later deleting it after they’ve spent the time trying to figure out how to use it isn’t cool.
>> 
>> My impression was that LEDE was going to be a little more welcoming to a broader developer community.
>> 
>> Yanking the carpet out from under people doesn’t strike me as particularly welcoming, nor much of a reward for someone bothering to learn about some of the arcane aspects of the system (why bother, if your knowledge could be rendered obsolete in the time it takes someone else to snap their fingers?).
>> 
>> I think we can and should do better than this.
>> 
>> -Philip
>> 
> 
> Hi, Philip
> 
> Sorry for the earlier kind of misguiding advices and your loss of time
> ;)  Like said earlier about host uci support, it will be hard decision
> to drop a feature and it's certainly not cool.  But I also think this
> change itself is a right one.
> 
> Apart from the already known shortcomings, this buildoverlay feature
> seems to me encourage writing temporary hacking code that's bound to
> fail at later point of time.  The other minor thing is that overlay
> makefile and package to be overlaid are connected together only by a
> directory name which is very fragile now that we can have that name
> shared among packages from different feeds.
> 
> There are other cleaner ways to do what buildovery could do.  Editing
> directly Makefile of package to be overlaid is the quick and obvious
> way.  Maintaining your own feeds of packages is another.
> 
> Regards,
>                yousong


I considered editing the Makefile for the package, but then I have to update the patch every time the package is updated in LEDE/OpenWRT.

That’s work, and maintaining my own feeds is even more work, since I don’t get the benefit of automatically picking up fixes from LEDE or OpenWRT (but instead have to do merges/rebases periodically).

What about having a file like $(TOPDIR)/customize.mk which we could -include…  and it could have things like:

define package/base-files/pre-hook
    …
endef

define package/base-files/post-hook
    …
endef


and we could check for such variables being defined, and if they are, call them with macro expansion.

If you want to customize /etc/banner for instance, to brand it to your build, there should be a trivial way to do this.  Editing Makefiles and maintaining package feeds are definitely not trivial.

And besides, the justification for killing the Build/IncludeOverlay was that "is very likely hard to maintain later”…  yet we’ve replaced one mechanism which was “hard to maintain” for another one which is every bit as much of an effort to maintain, and very likely an ongoing effort.  At least the Build/IncludeOverlay stuff was nicely compartmentalized so it didn’t need any ongoing maintenance effort.

-Philip