[LEDE-DEV] Release Preparation Questions

[David Lang]

Feed is a concept created by OpenWRT within the last couple of years. They used 
to have all the source for everything in one big repository, but they weren't 
keeping up with the updates for everything well.

So they looked at the things that they were actually maintaining and kept them 
in the OpenWRT repository, and all the other things that they weren't 
maintaining, just using the code as provided by upstream were broken out into 
multiple separate sources, each of which is not maintained by OpenWRT, but 
rather by the upstream developers.

These other repositories are the 'feeds'.

LEDE maintains this structure and uses the same feeds as OpenWRT

David Lang

On Sun, 20 Nov 2016, James Feeney wrote:

> Date: Sun, 20 Nov 2016 18:11:16 -0700
> From: James Feeney <james at nurealm.net>
> To: lede-dev at lists.infradead.org
> Subject: Re: [LEDE-DEV] Release Preparation Questions
> 
> Ok - thanks for taking the time to explain this versioning process.  So now, I
> should take the time to "feed it back" and see if I understand.  Ha!
>
>> https://git.lede-project.org/source.git
> 404 Not Found
>
> Hmm.  Try Google...
>
> https://git.lede-project.org/?p=source.git
> description	LEDE Source Repository
>
> Ok.
>
> $ whois lede-project.org
> Admin Name: Jo-Philipp Wich
> Admin Country: DE
>
> That helps orient things.
>
>> it does not require twenty different variables but it has to deal
>> with (currently) five independent repositories:
>> - feeds.conf.default: git://git.lede-project.org/feed/packages.git
>
> Hmm - try Google again:
>
> https://git.lede-project.org/
>
> Ah ha.
>
> feed/packages.git	Mirror of packages feed		Git	119 min ago
> feed/routing.git	Mirror of routing feed		Git	2 weeks ago
> feed/telephony.git	Mirror of telephony feed	Git	4 months ago
> ...
> project/luci.git	Lua Configuration Interface...	Git	28 hours ago
>
> Ok.  I don't know what I'm seeing yet, but ok.
>
>> ... described in feeds.conf.default within the base repo.
>
> Hmm - still drawing a blank...
>
> source.git	LEDE Source Repository	Git	102 min ago
>
> https://git.lede-project.org/?p=source.git;a=tree
> https://git.lede-project.org/?p=source.git;a=blob;f=README
>
>   9 Run "./scripts/feeds update -a" to get all the latest package definitions
>  10 defined in feeds.conf / feeds.conf.default respectively
>  11 and "./scripts/feeds install -a" to install symlinks of all of them into
>  12 package/feeds/.
>
> ... getting closer...
>
> https://git.lede-project.org/?p=source.git;a=blob;f=feeds.conf.default
>
> Ah ha!
>
>   1 src-git packages https://git.lede-project.org/feed/packages.git
>   2 src-git luci https://git.lede-project.org/project/luci.git
>   3 src-git routing https://git.lede-project.org/feed/routing.git
>   4 src-git telephony https://git.lede-project.org/feed/telephony.git
>   5 #src-git targets https://github.com/openwrt/targets.git
>   6 #src-git management https://github.com/openwrt-management/packages.git
>   7 #src-git oldpackages http://git.openwrt.org/packages.git
>   8 #src-link custom /usr/src/openwrt/custom-feed
>
> Mmm-hmm - I seem to have gone in a circle.  So, the term "feed" is a unique
> concept, referring to three "gitish" directories including the name "feed", and
> one "gitish" directory *not* including the name "feed".  Hmm...  Well, that
> seems like an inconsistent concept.  So, a "feed" is whatever git "directory" a
> file "feeds.conf.default" says it is.  That is not a traditional hierarchy, and
> it is a mistake for me to try to think of LEDE as a kind of file system
> hierarchy.  It is more like a kind of linked-list, with a "root" located in a
> file, which is located in a git directory "projects/source.git/tree", which is
> best discovered by already knowing its location.
>
> This is an organizational model which I have not encountered before now.  It is
> certainly not the most "discoverable" model which I could imagine, but ok.
>
> And then, there is a top-level git directory name, "source.git", where the term
> "source" is not used in the usual manner, to mean literally the directory
> holding the source code, but instead, seems to refer to that directory which
> contains the file which enumerates the elements of the term "feed".
>
> And also, there is the idea of "base repository", where the "base" is being
> defined as the "source", and the "source" is not really the "base" in the
> traditional sense, but instead is located under a traditional "base" URL,
> "https://git.lede-project.org/".
>
> Again, this is a nontraditional use of the term "base", which seems to be used
> interchangeably with the term "source" - but ok.
>
> Lots of specialized, nontraditional, terminology, - "base", "source", "feed" -
> but ok.
>
> Now, revisiting the concept of "version number":
>
> 1) "base" repository - https://git.lede-project.org/?p=source.git
> 2) "feeds" - described in feeds.conf.default within the base repo.
>   1 src-git packages https://git.lede-project.org/feed/packages.git
>   2 src-git luci https://git.lede-project.org/project/luci.git
>   3 src-git routing https://git.lede-project.org/feed/routing.git
>   4 src-git telephony https://git.lede-project.org/feed/telephony.git
> 3) release branch number - an input variable, major.minor version, usually
> year.month - or instead, the number of the build being produced from a branch.
> There were two different definitions offered.  I'm guessing that you meant the
> "build number" after the "major.minor" version number.
> 4) git tags - exactly match the source state; "commit hash".
> 5) builds - the binary artifacts produced, branch release builds.
> 6) master - master branch of the "base" repository.
> 7) release branch - used for *all* builds within a major.minor version; "source
> branch".
> 8) subrelease - Any release within a base version; patch levels; security updates.
> 9) nickname - Symbolic name given to an entire release branch; all 16.11.*
> releases will carry the same nickname.
>
> Hmm - it is still not clear to me how the term "branch" will apply here,
> because: 1) the hierarchy of "base.feeds" is referencing four *other* git
> repositories, each of which has its *own* "head" branch name, each of which - I
> assume - may be changed or selected independently; and 2) the "feeds" link list
> does *not* have any version information within it, itself.
>
> It would be possible, I suppose, that each of these linked repositories could be
> *required* to use the *same* versioned naming scheme.  The implication would be
> that a "build" version would mean a build of the collection of all components of
> the "feed" list with identical version names.  But that is not the way it is
> now.  Instead, looking into the feed components, I see some "tags" and some
> "heads", but nothing that is common to all the components, except that they all
> have a "head" named "master".  There are many "master branches".  Perhaps the
> git cognoscenti have some way of synchronizing all of these master branches such
> that a "year.month.build-number" can be uniquely associated with some
> instantaneous version of several feed components?
>
> Please explain - I'm missing some piece of the puzzle - probably because I do
> not know enough about using git.
>
>> If a builtin package or the kernel itself gain security fixes,
>> the images need to get rebuilt since people cannot update those
>> components with opkg.
>
> This is what is so unusual for me about LEDE/OpenWRT.  It is not a simple
> program package, where every component is versioned in lock-step.  But neither
> is it a traditional distribution, in which all of the components can be
> versioned and installed independently, especially as in a rolling distribution.
> And it is not a "glacial" distribution, like Debian, where the versioning
> measures time in years.  Perhaps it is more similar to Cyanogenmod, which has
> dated "nightly" and "snapshot" builds.  The term "release" is inappropriate in a
> fast moving environment like that.  The term "build" seems better suited.
>
> Cyanogenmod uses the term "manifest" instead of "feed", and uses "repo", a
> specialized "repository management tool that we built on top of Git."  You are
> familiar?  In a sense, the top level of the build system is the "manifest" file
> "default.xml" with its own web page, https://github.com/CyanogenMod/android.
> Hmm - LEDE has a program "[source.git] / scripts / feeds", which serves the same
> function as "repo", yes?  Maybe I begin to understand.  And then, the term
> "feed" would actually mean one of several git source code repositories
> enumerated in the LEDE "manifest".  And "feeds" means a LEDE git repository
> management and build utility.
>
> I think that, essentially, I must understand how a LEDE build version number can
> be related to an instantaneous version of four independent git repositories.
>
> Would it be useful to version the manifest, and have the versioned manifest
> enumerate specific versions of the manifest components?  I assume that each
> component could specify an instantaneous snapshot version of each master branch
> of that component?  Perhaps git tags could be included inside the manifest, and
> no one would have to memorize or recognize or refer to these, because the
> manifest itself would have a human readable version number, as in
> "year-month-build".
>
>> Did you ever build the complete LEDE or OpenWrt from scratch?
>
> Ha!  NO! That's the issue for me.  The linux kernel - ok, I'm familiar with
> that.  LEDE/OpenWRT - I'm still trying to wrap my head around it!
>
> The terminology would make more sense to me if "base" referred to a "base
> package" which include the linux kernel and required utilities, unique to each
> hardware system, even though it were a "package" that could not be installed by
> opkg.  As I understand, the kernel and the root file system have to be installed
> together, as a matched set, though I cannot say why.  The "version number" would
> imply the version of this LEDE "base package".  The LEDE "source" would refer to
> just the git repository "https://git.lede-project.org/", and the git directory
> "source.git" would instead be named "build.git". The file "feeds.conf.default"
> could be called "manifest.default", which would be a concept shared with
> Cyanogenmod.
>
> _______________________________________________
> Lede-dev mailing list
> Lede-dev at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev
>