[LEDE-DEV] Better backups to make upgrade easier

[Stefan Hellermann]

2017-06-19 5:49 GMT+02:00 Luiz Angelo Daros de Luca <luizluca at gmail.com>:
>
> Hello,
>
> Upgrade process with LEDE/OpenWrt installations aren't as smooth as a
> user would expect. I know that PC have a lot more resources but their
> upgrade process is normally very straightforward. They can upgrade
> thousands of packages with little to no intervention.
>
> On LEDE, it is normally not feasible to simply backup all overlay for
> a upgrade. It works for a disaster recovery but, in an upgrade, extra
> installed packages might break with the new runtime env.
>
> The standard way to upgrade and keep configuration is using the system
> generated config backup. It works with simple installation but they
> will fail whenever you need any extra package. It is also designed to
> ignore all files but the ones listed as config files or in
> sysupgrade.conf. So, a user must doublecheck the list of files in
> backup before every upgrade in order to not miss an extra script file,
> a ssh key, etc.
>
> At the same time, backup saves unneeded stuff. For example, every
> backup includes /etc/profile, /etc/protocol. Do we really need all
> those files, even when they are still untouched? opkg already keeps
> hash of all of them for "opkg list-changed-conffiles". Someone that
> upgrade from CC simply misses all changes to those file, which
> disables, i.e., the use of /etc/profile.d/. (BTW, /etc/profile.d/
> files are not listed as conf file for backup). Who knows what else
> will break in the future.

I send a patch to correct this in 2013, but it was NAKed as it breaks
machines without opkg. Maybe we can make it depend on
CONFIG_PACKAGE_opkg.

https://lists.openwrt.org/pipermail/openwrt-devel/2013-February/018742.html

>
>
> The user must take note on every extra package was installed on the
> system before an upgrade in order to know what is needed to be
> reinstalled post upgrade. There is not an easy way to list every
> package I asked to be installed. On a overlay system, I can deduce it
> by looking at /overlay/upper/usr/lib/opkg/info/*.control.
> /usr/lib/opkg/status was my first try but it does not keep the
> installation order and new packages appear in the middle of the list.
> It is also interesting to ignore those packages installed as
> dependency (Auto-Installed: yes at /usr/lib/opkg/status; not shown by
> opkg).
>
> In my upgrades, I created a new type of backup composed of two parts:
> 1) save the list of packages I asked to be installed (not
> auto-installed, not built-in)
> 2) save all files in overlay, excluding those from packages but
> including changed config files.
>
> This is not ideal, as it does not track changed non-config files from
> packages (should it? maybe warn about it), but it works flawlessly
> with 99% of my setups and saves me a lot of time.
>
> With those two files, I can flash a new version already providing the
> file 2) with the option "-f". After the installation, I simply batch
> install all packages from 1). If any package from 1) is needed to
> reach the target system, I create a custom image adding those files
> and flash it, again with the 2) config file.
>
> This is my backup script for reference
> https://drive.google.com/open?id=0B54xzz44RpW6dk1WRTFXX1kwdU0
> 1) is backup-*-installed_packages_by_user.txt and 2)
> backup-*-overlay_diff.tgz. It also generates extra backups options.
>
> Would it be interesting to:
> a) stop saving not changed config files in current backup (this is
> already breaking things)
> b) create an "expanded backup", based on 2). Add it to luci.
> c) make luci accept a config file during the flash process (as sysupgrade does)
> d) save the list of installed packages (all, after flashed, user
> installed or all of them) in a different file or inside every backup
> e) there is already a GSoC for the installation of the extra packages
> (required for unattended upgrade). However, it would also be possible
> to inform during the flashing that the system must try to download and
> install those extra packages (included in the backup file) as soon as
> internet is up. This does not need to work for all scenarios as it is
> not an unattended nor automatic upgrade.
>
> I can create those patches but, at first, I would like to discuss if
> the idea would be accepted.
>
> Regards,
>
> ---
>      Luiz Angelo Daros de Luca
>             luizluca at gmail.com
>
> _______________________________________________
> Lede-dev mailing list
> Lede-dev at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/lede-dev