[OpenWrt-Devel] [RFD] sysupgrade and configuration management

[Luiz Angelo Daros de Luca]

Hi Paul,

> Very nice, but having read the mail thread around Luiz' his patch, I want to ask for a discussion, but outside of that patch.
>
> AFAICT from the discussion it follows that sysupgrade, opkg and UCI should be kept (as much as possible) orthogonal. From the thread: opkg cannot always be assumed to be available, external config management may need all configuration files, not just those that changed and not just diffs, config files may need changes, etc.
>
> The backup functionality in sysupgrade serves a common case of keeping config files over upgrades where that's needed because the rootfs/overlay is re-created. Cases where sysupgrade does not destroy the (overlay) filesystem contents also exist and obviously config files aren't lost than.

Is there any case where rootfs/overlay is not destroyed? Even for x86,
sysupgrade uses dd.

> After/during a sysupgrade a normal step is the restoration of the previous configuration, i.e. of config files *and* extra previously installed packages - currently sysupgrade does not handle re-install of extra user installed packages. This functionality could also serve cases where a certain configuration is to be put on a new or other device. Where the filesystem its contents is not lost during sysupgrade, this would just amount to an opkg upgrade of all outdated packages after sysupgrade has upgraded the kernel and possibly the rootfs.
>
> In the more common sysupgrade case though (overlay contents are lost), some means is needed to preserve configuration data in order to be able to restore the configuration. Where this data is preserved during sysupgrade depends completely on the device specific sysupgrade mechanism; *what* data is saved and *how* it is re-applied is preferably opaque to, and independent of, sysupgrade.

IMHO, backup should be a different script. It's strange to ask
sysupgrade for a backup (and worse to restore one). It might be just
legacy calling for a refactoring.

> Also, during upgrades new versions of packages may require updates to the config files. Preserving customisations of the config files cannot just be handled with back-up/restore and may require applying a diff or user intervention. How to deal with that can really only be determined by the package maintainer and so must be handled by the package itself. Anyhow, this clearly is in the domain of opkg and applies to upgrades outside of sysupgrade as well.

RPM, as opkg, simply saves the files and let user solve it manually.
Doing it automagically will require too much resource from maintainers
and the device that we might not have.
I guess we have what we can get. CM might replace the "package
installation and configuration" part of the backup but it is not a
backup. You still might need to save some data.
For me, backup and CM are different beasts.

> In short, factoring out the configuration management functionality (CM) seems a good idea. The sysupgrade subsystem handles with upgrading kernel and rootfs and may invoke CM functions to preserve the user configuration. CM can could also take care of user installed packages. CM calls opkg to install and/or upgrade additional packages. Truly general back-up (e.g. as a defense against data loss) is outside the scope of sysupgrade.
>
> Quite a few efforts have already been made toward some kind of CM, notably by offering this as an external service. The task to factor out generic CM  functionality that can be called upon in different contexts is probably quite complicated and comments on that idea are welcome.
>
> What would be a practical API, if any, for CM ?
> What (minimum) configuration data would be needed by CM, i.e. what's needed besides config files to store user install selections ?
> What's needed to enhance opkg so that it can handle in a general way updates to config file while preserving user settings ?
> Compatibility issues ?
>
> Paul

I just opened a ticket for improving opkg
(https://bugs.openwrt.org/index.php?do=details&task_id=1821).

Another great improvement would be to require that every single file
in /etc/config should be uci generated. Today, some files are manually
edited. So, when (l)uci touches them, every single line is changed,
making diff simply useless. Would it be possible to enforce that rule
in OpenWrt? OpenWrt-packages could do that with travis (uci import
conf1/uci export conf1 to conf2/fail if conf1 and conf2 differ). It
will, although, prohibit any comments in config files from packages.

My patch and these suggestions try to reduce the burden of an upgrade.
Instead of implementing CM, it might be more useful to have tools that
help with the config merging process, like a vimdiff-like app for Luci
(remember, some users does not know how to use CLI).

Regards,

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel