[OpenWrt-Devel] openwrt/packages: [RFC] Regarding preferences re: switch to codeload

[Rosen Penev]

On Mon, Aug 13, 2018 at 2:45 AM Jo-Philipp Wich <jo at mein.io> wrote:
>
> Hi,
>
> personally I'm opposed to the entire code load thing.
>
> First of all I was unable to reproduce the tarballs offered by Github.
>
> Github seems to use an extended tar (pax) format while we pack our SCM
> clones using the more traditional ustar format, however even using `tar
> -cp -H pax --numeric-owner --owner=0 --group=0 --sort=name --mtime ...`
> seems to yield a different tar stream compared to whatever is offered by
> Github;
>
>  - The order of the entries in the archive also seems to deviate from
>    that of `tar --sort=name`, it looks as if Github archives are sorted
>    using the "C" collate while GNU tar uses something else.
>
>  - The PAX header format seems to be different, Github uses a global PAX
>    header while GNU tar produces per-member headers
>
>  - There seem to be proprietary tags inside Github tar (comment=<sha1>)
>    which are not present in the GNU equivalent
>
> Furthermore I dislike the idea of tailoring download mechanisms around a
> specific proprietary service.
>
> If the allegations about hash changes for unknown reasons are correct,
> then this raises a huge red flag for me
I can only imagine this to be the case if git history gets changed.
I'm sure it's automated.
> and I see no reason to not
> assume that codeload tarballs will eventually change as well, become
> rate limited, redirected, discontinued or changed in other arbitrary ways.
Well, there are people who believe Microsoft will cripple GitHub.
>
> So TLDR; I prefer a locally reproducible, cached tarball of a given SCM
> clone over an opaque Github offer.
It's not reproducible in all cases. See
https://github.com/openwrt/openwrt/pull/815

We could not produce the same tarball with the same hash. I was using
Ubuntu 16.04 FWIW.
>
>
> My 2cents,
Mine are that it's better to use a tarball that is not locally
generated. I understand that ultimately the buildbots generate the
tarballs but then you potentially have a situation where a package
bump has the wrong hash since the buildbot and someone's local
environment can differ enough to matter.

My understanding is that OpenWrt can be built on Linux and some BSDs
like macOS or FreeBSD.

The initial motivation for moving packages to codeload was to adjust
uscan's reporting of version numbers as git revisions when there are
git tags available. eg: using

PKG_SOURCE_VERSION:=1ee4c4eac2f1f771ecc57d4f7ce298739bbc8a82 vs.
PKG_SOURCE_VERSION:=v$(PKG_VERSION)

I also don't see why both codeload and buildbot generated tarballs
can't coexist.
> Jo
>
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel