[OpenWrt-Devel] [RFD] sysupgrade and configuration management

[Luna Jernberg]

Having problems installing luci at all on latest NIGHTLY on a Linksys WRT
1200 mvebu <https://downloads.openwrt.org/snapshots/targets/mvebu/> /
cortexa9 <https://downloads.openwrt.org/snapshots/targets/mvebu/cortexa9/>
/ The peopel on irc told me to send an email:

05:18 < luna> There is problems installing luci in the latest NIGHTLY
releases
05:18 < luna> root at RainbowDashv2:~# opkg install luci
05:18 < luna> Unknown package 'luci'.
05:18 < luna> Collected errors:
05:18 < luna>  * opkg_install_cmd: Cannot install package luci.
05:19 < DonkeyHotei> did you do "opkg update" ?
05:19 < Ziginox> luna: did you-
05:19 < Ziginox> yeah that
05:20 < luna> yes
05:20 < DonkeyHotei> is the luci line perhaps commented out in
/etc/opkg/distfeeds.conf?
05:21 < luna> nope
05:21 < DonkeyHotei> hmm
05:21 < DonkeyHotei> which target is this?
05:22 < luna>  mvebu / cortexa9 /
05:22 < luna> a Linksys WRT 1200




On Fri, Aug 24, 2018 at 1:23 PM, Paul Oranje <por at oranjevos.nl> wrote:

> Hyjacking on what luizluca at gmail.com wrote on 18 aug. 2018 at 01:49,
> with subject [OpenWrt-Devel] [PATCH v3 0/4] base-files: add new backup
> options
>
> > Today, every file listed in /etc/sysupgrade.conf, /etc/config,
> > marked as changed conffile and others will be in backup. Some of
> > these files from previous OpenWrt version might break something.
> > /etc/profile is a good example of what should not be in backup if
> > unchanged. Also, any conffile that has a new required parameter
> > might break after restore.
> >
> > When the user changes a file, it is expected that he/she knows that
> > he/she is doing. The problem is when OpenWrt replaces a file with one
> > from a previous version that the user might not even know that it exists.
> >
> > The new '-u' option asks backup to (oportunisticly) skip any file that
> > is equals to /rom. If a system does not have /rom, it simply does
> > nothing. /rom does not need to actually be squashfs. If the user is using
> > ext4, he/she could simply copy files into /rom before the first change
> > is made. IMHO, it should even be the default behavior.
> >
> > A backup might also miss important files for the user. The user must
> > insert every single file needed in /etc/sysupgrade.conf in order to get
> > it into a backup. However, it is easy to simply miss one. '-c' option
> > does try to save everything, but limited to /etc, probably to skip code
> > files. Saving /overlay also works but only when restoring to an
> > identical OpenWrt version. The new '-o' is equivalent to saving
> > /overlay, but it skips any files that came from a package, except those
> > marked as a changed config file, sysupgrade.conf or /lib/upgrade/keep.d.
> > It does work with '-u', skipping files touched but reverted to the
> > original state.
> >
> > After the user seeded a new OpenWrt with a backup generated with '-o'
> > and '-u', the next natural step is to reinstall all previously installed
> > packages. '-k' adds a file into the backup containing the list of
> > installed packages and also its origin (rom or overlay). It is a one-line
> > script to reinstall them all.
> >
> > My normal upgrade procedure is:
> >
> > # sysupgrade -o -k -u openwrt-new-version.img
> > # <auto reboot>
> > # opkg update
> > # grep "\toverlay" /etc/backup/installed_packages.txt | cut -f1 | xargs
> -r opkg install
> > # rm /etc/backup/installed_packages.txt
> > # reboot
> >
> > Those options could be used by Luci, exposed to user during an upgrade.
> > The (re)installation step could even become automatic (on demand) or
> > offered to the user when Luci detects /etc/backup/installed_packages.txt
> > presence.
> >
>
> Very nice, but having read the mail thread around Luiz' his patch, I want
> to ask for a discussion, but outside of that patch.
>
> AFAICT from the discussion it follows that sysupgrade, opkg and UCI should
> be kept (as much as possible) orthogonal. From the thread: opkg cannot
> always be assumed to be available, external config management may need all
> configuration files, not just those that changed and not just diffs, config
> files may need changes, etc.
>
> The backup functionality in sysupgrade serves a common case of keeping
> config files over upgrades where that's needed because the rootfs/overlay
> is re-created. Cases where sysupgrade does not destroy the (overlay)
> filesystem contents also exist and obviously config files aren't lost than.
>
> After/during a sysupgrade a normal step is the restoration of the previous
> configuration, i.e. of config files *and* extra previously installed
> packages - currently sysupgrade does not handle re-install of extra user
> installed packages. This functionality could also serve cases where a
> certain configuration is to be put on a new or other device. Where the
> filesystem its contents is not lost during sysupgrade, this would just
> amount to an opkg upgrade of all outdated packages after sysupgrade has
> upgraded the kernel and possibly the rootfs.
>
> In the more common sysupgrade case though (overlay contents are lost),
> some means is needed to preserve configuration data in order to be able to
> restore the configuration. Where this data is preserved during sysupgrade
> depends completely on the device specific sysupgrade mechanism; *what* data
> is saved and *how* it is re-applied is preferably opaque to, and
> independent of, sysupgrade.
>
> Also, during upgrades new versions of packages may require updates to the
> config files. Preserving customisations of the config files cannot just be
> handled with back-up/restore and may require applying a diff or user
> intervention. How to deal with that can really only be determined by the
> package maintainer and so must be handled by the package itself. Anyhow,
> this clearly is in the domain of opkg and applies to upgrades outside of
> sysupgrade as well.
>
> In short, factoring out the configuration management functionality (CM)
> seems a good idea. The sysupgrade subsystem handles with upgrading kernel
> and rootfs and may invoke CM functions to preserve the user configuration.
> CM can could also take care of user installed packages. CM calls opkg to
> install and/or upgrade additional packages. Truly general back-up (e.g. as
> a defense against data loss) is outside the scope of sysupgrade.
>
> Quite a few efforts have already been made toward some kind of CM, notably
> by offering this as an external service. The task to factor out generic CM
> functionality that can be called upon in different contexts is probably
> quite complicated and comments on that idea are welcome.
>
> What would be a practical API, if any, for CM ?
> What (minimum) configuration data would be needed by CM, i.e. what's
> needed besides config files to store user install selections ?
> What's needed to enhance opkg so that it can handle in a general way
> updates to config file while preserving user settings ?
> Compatibility issues ?
>
> Paul
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/openwrt-devel/attachments/20180828/5409c028/attachment.htm>
-------------- next part --------------
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel