[LEDE-DEV] Release 17.01.0 binary packages have changed and SDK inconsistency

[James Feeney]

Hi Daniel

> Well, this was actually fixed by having a specific snapshot of all
> feeds referenced in the SDK. Which is exactly the cause of all the
> confusion now...
> ...
>>> ... It would mean that every build was repeatable.
>
> Every build of what exactly?
> Do you realise that *none* of the packages from any of the feeds are
> actually included in the released target binaries?

Realize?  No - I'm still learning how this build process works in LEDE.  My
impression is that you are distinguishing between "packages" and some other type
of thing, which the lede-project.org site seems to implicitly, and generically,
refer to as just "firmware" - even though a "package" is also a type of
"firmware", and this "firmware" is also a type of "package".  I will infer that
you mean to distinguish between the "kernel and base filesystem" package from
all other "packages".

You also seem to be implying that all the various "kernel and base filesystem"
package builds are now precisely repeatable - even though this fact does not
seem to be documented anywhere.

> Strictly speaking the package feeds are simply not a part of the
> project making the release.

With respect to the lingo, "package feeds" and "project making the release", I
have no understanding of the meaning of that statement.  Perhaps you could
define precisely "package feeds" and "project making the release"?

> From my understanding the original issue which was brought up by Pau
> is the result of a lack of documentation how to properly use the SDK
> and ImageBuilder. It doesn't have much too do with that past trauma
> of yours, though obviously resembled it closely enough to trigger some
> painful memories and write an email using lots of quotation marks and
> emphasis to make us *see* "something".

Ha!  Quite!

> Release builds (ie. building the buildroot from source) are entirely
> reproducible, thanks to the long and tideous efforts of the
> reproducible builds team, eliminating timestamps and other obstacles.

Ok - I don't actually know what is meant by "buildroot", but my impression is
that you mean the kernel and base filesystem, yes?  And so then, any package
labeled in similar form to "17.01.0" will be uniquely reproducible at any time
in the future, yes?

> I fail to see the issue which would even remotely justifies the degree
> of inflated drama.

Simply a combination of my uninformed state of mind, where I am left to "fill in
the blanks" with the information presented in the past, specifically that "you
don't really have a repeatable build."  Since that seems to have changed, it
would be good to advertise that fact in public, which is to say, to document
that fact on the lede-project.org website.

> I fail to see how this is unprofessional or even
> inconsistent or anything like that. Maybe there have been
> misunderstandings caused by wrong expectations caused by a lack of
> documentation.

Perhaps, depending upon what is meant by "wrong expectations".  *My*
expectations of the LEDE project are simply *my* expectations, no matter what
the state of the LEDE project might be, documented or not.

> I also fail to see how git submodules could make things
> any better. Imho using git submodules, especially for meta-stuff is
> a very bad practise, many distributions decided against that and for
> good reason, see e.g. some input from puppetlinux [1]. Money quote:
> "All of this smells of an antipattern. Git submodules sound great
> initially, but the farther down this road that you go, the more work
> you're going to generate trying to work around the limitations of git
> submodules."
> Maybe subtrees can be better. Having an explicite manifest, like
> feeds.conf in the SDK, does the trick. I agree that it's a bit
> hidden and that practise has not been very well documented (yet).

I cannot speak to that specifically.  If, as you suggest, there is a solution
already in place for LEDE "controlling" the source code from the package source
repositories, then this is no longer of any importance.  But I am not quite sure
that I have heard that that is so.  Are all of the "package" builds versioned
and repeatable?

> Anyway. Back to the real issue here:
> Imho you shouldn't use the SDK to build core packages or even any
> package provided in a binary feed belonging to that release and then
> use that version (which you have just built locally) in the
> ImageBuilder. Yes, you may need to build it in order to build your
> local packages to satisfy build dependencies. But when it comes to use
> the ImageBuilder, always use the upstream binary feeds and only include
> your hand full of local packages in a seperate feed.
> If you really need to modify packages already included in an upstream
> binary feed, rename them and use the (now working) PROVIDES variable to
> satisfy dependencies (or make sure your modifications go upstream!!!)

The specifics of that process would be outside my purview.  As long as the
"package" builds are versioned and repeatable, then this is only a matter of how
the tools are being used.  Then again, I don't know what is true here.  If, on
the other hand, the "package" builds are not versioned and repeatable, I'd say
that there is a serious systemic problem with the LEDE build system.

Please clarify?  As LEDE is developing new project management protocols,
everyone "outside" is on the learning curve.

To be clear on my end, I expect every "component" of the LEDE project to be:
1) instructively documented
2) explicitly named
3) distinctly versioned
4) repeatably buildable


Thanks
James