[OpenWrt-Devel] [RFD] sysupgrade and configuration management

Tue Aug 28 14:49:54 EDT 2018

Todays sysupgrade fixed my problems now :)

On Tue, Aug 28, 2018 at 5:25 AM, Luna Jernberg <droidbittin at gmail.com>
wrote:

> Having problems installing luci at all on latest NIGHTLY on a Linksys WRT
> 1200 mvebu <https://downloads.openwrt.org/snapshots/targets/mvebu/> /
> cortexa9 <https://downloads.openwrt.org/snapshots/targets/mvebu/cortexa9/>
> / The peopel on irc told me to send an email:
>
> 05:18 < luna> There is problems installing luci in the latest NIGHTLY
> releases
> 05:18 < luna> root at RainbowDashv2:~# opkg install luci
> 05:18 < luna> Unknown package 'luci'.
> 05:18 < luna> Collected errors:
> 05:18 < luna>  * opkg_install_cmd: Cannot install package luci.
> 05:19 < DonkeyHotei> did you do "opkg update" ?
> 05:19 < Ziginox> luna: did you-
> 05:19 < Ziginox> yeah that
> 05:20 < luna> yes
> 05:20 < DonkeyHotei> is the luci line perhaps commented out in
> /etc/opkg/distfeeds.conf?
> 05:21 < luna> nope
> 05:21 < DonkeyHotei> hmm
> 05:21 < DonkeyHotei> which target is this?
> 05:22 < luna>  mvebu / cortexa9 /
> 05:22 < luna> a Linksys WRT 1200
>
>
>
>
> On Fri, Aug 24, 2018 at 1:23 PM, Paul Oranje <por at oranjevos.nl> wrote:
>
>> Hyjacking on what luizluca at gmail.com wrote on 18 aug. 2018 at 01:49,
>> with subject [OpenWrt-Devel] [PATCH v3 0/4] base-files: add new backup
>> options
>>
>> > Today, every file listed in /etc/sysupgrade.conf, /etc/config,
>> > marked as changed conffile and others will be in backup. Some of
>> > these files from previous OpenWrt version might break something.
>> > /etc/profile is a good example of what should not be in backup if
>> > unchanged. Also, any conffile that has a new required parameter
>> > might break after restore.
>> >
>> > When the user changes a file, it is expected that he/she knows that
>> > he/she is doing. The problem is when OpenWrt replaces a file with one
>> > from a previous version that the user might not even know that it
>> exists.
>> >
>> > The new '-u' option asks backup to (oportunisticly) skip any file that
>> > is equals to /rom. If a system does not have /rom, it simply does
>> > nothing. /rom does not need to actually be squashfs. If the user is
>> using
>> > ext4, he/she could simply copy files into /rom before the first change
>> > is made. IMHO, it should even be the default behavior.
>> >
>> > A backup might also miss important files for the user. The user must
>> > insert every single file needed in /etc/sysupgrade.conf in order to get
>> > it into a backup. However, it is easy to simply miss one. '-c' option
>> > does try to save everything, but limited to /etc, probably to skip code
>> > files. Saving /overlay also works but only when restoring to an
>> > identical OpenWrt version. The new '-o' is equivalent to saving
>> > /overlay, but it skips any files that came from a package, except those
>> > marked as a changed config file, sysupgrade.conf or /lib/upgrade/keep.d.
>> > It does work with '-u', skipping files touched but reverted to the
>> > original state.
>> >
>> > After the user seeded a new OpenWrt with a backup generated with '-o'
>> > and '-u', the next natural step is to reinstall all previously installed
>> > packages. '-k' adds a file into the backup containing the list of
>> > installed packages and also its origin (rom or overlay). It is a
>> one-line
>> > script to reinstall them all.
>> >
>> > My normal upgrade procedure is:
>> >
>> > # sysupgrade -o -k -u openwrt-new-version.img
>> > # <auto reboot>
>> > # opkg update
>> > # grep "\toverlay" /etc/backup/installed_packages.txt | cut -f1 |
>> xargs -r opkg install
>> > # rm /etc/backup/installed_packages.txt
>> > # reboot
>> >
>> > Those options could be used by Luci, exposed to user during an upgrade.
>> > The (re)installation step could even become automatic (on demand) or
>> > offered to the user when Luci detects /etc/backup/installed_packages
>> .txt
>> > presence.
>> >
>>
>> Very nice, but having read the mail thread around Luiz' his patch, I want
>> to ask for a discussion, but outside of that patch.
>>
>> AFAICT from the discussion it follows that sysupgrade, opkg and UCI
>> should be kept (as much as possible) orthogonal. From the thread: opkg
>> cannot always be assumed to be available, external config management may
>> need all configuration files, not just those that changed and not just
>> diffs, config files may need changes, etc.
>>
>> The backup functionality in sysupgrade serves a common case of keeping
>> config files over upgrades where that's needed because the rootfs/overlay
>> is re-created. Cases where sysupgrade does not destroy the (overlay)
>> filesystem contents also exist and obviously config files aren't lost than.
>>
>> After/during a sysupgrade a normal step is the restoration of the
>> previous configuration, i.e. of config files *and* extra previously
>> installed packages - currently sysupgrade does not handle re-install of
>> extra user installed packages. This functionality could also serve cases
>> where a certain configuration is to be put on a new or other device. Where
>> the filesystem its contents is not lost during sysupgrade, this would just
>> amount to an opkg upgrade of all outdated packages after sysupgrade has
>> upgraded the kernel and possibly the rootfs.
>>
>> In the more common sysupgrade case though (overlay contents are lost),
>> some means is needed to preserve configuration data in order to be able to
>> restore the configuration. Where this data is preserved during sysupgrade
>> depends completely on the device specific sysupgrade mechanism; *what* data
>> is saved and *how* it is re-applied is preferably opaque to, and
>> independent of, sysupgrade.
>>
>> Also, during upgrades new versions of packages may require updates to the
>> config files. Preserving customisations of the config files cannot just be
>> handled with back-up/restore and may require applying a diff or user
>> intervention. How to deal with that can really only be determined by the
>> package maintainer and so must be handled by the package itself. Anyhow,
>> this clearly is in the domain of opkg and applies to upgrades outside of
>> sysupgrade as well.
>>
>> In short, factoring out the configuration management functionality (CM)
>> seems a good idea. The sysupgrade subsystem handles with upgrading kernel
>> and rootfs and may invoke CM functions to preserve the user configuration.
>> CM can could also take care of user installed packages. CM calls opkg to
>> install and/or upgrade additional packages. Truly general back-up (e.g. as
>> a defense against data loss) is outside the scope of sysupgrade.
>>
>> Quite a few efforts have already been made toward some kind of CM,
>> notably by offering this as an external service. The task to factor out
>> generic CM  functionality that can be called upon in different contexts is
>> probably quite complicated and comments on that idea are welcome.
>>
>> What would be a practical API, if any, for CM ?
>> What (minimum) configuration data would be needed by CM, i.e. what's
>> needed besides config files to store user install selections ?
>> What's needed to enhance opkg so that it can handle in a general way
>> updates to config file while preserving user settings ?
>> Compatibility issues ?
>>
>> Paul
>> _______________________________________________
>> openwrt-devel mailing list
>> openwrt-devel at lists.openwrt.org
>> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/openwrt-devel/attachments/20180828/3ab6818f/attachment.htm>
-------------- next part --------------
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel