[LEDE-DEV] [PATCH] base-files: sysupgrade. Added the ability to use URL as source
Сергеев Сергей
adron at yapic.net
Wed Aug 24 21:58:16 PDT 2016
Hello. See below:
>
> On 24/08/2016 11:12, Сергеев Сергей wrote:
>> Hello. Ok. Leave all as is. But...(comments below):
>>> On 13/08/2016 13:04, adron at yapic.net wrote:
>>>> From: Sergey Sergeev <adron at yapic.net>
>>>>
>>>> scp uls(like this
>>>> scp://adron@192.168.88.6:lede/lede-ar71xx-mikrotik-NAND-512b-squashfs-sysupgrade.bin)
>>>>
>>>> is supported too. And you also can specify custom ssh
>>>> port(...8.88.6:22110:lede...)
>>>>
>>>> Signed-off-by: Sergey Sergeev <adron at yapic.net>
>>>> ---
>>>> package/base-files/files/lib/upgrade/common.sh | 26
>>>> +++++++++++++++++++++-----
>>>> package/base-files/files/sbin/sysupgrade | 15 +++++++++++++++
>>>> 2 files changed, 36 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/package/base-files/files/lib/upgrade/common.sh
>>>> b/package/base-files/files/lib/upgrade/common.sh
>>>> index ea03f77..080bea5 100644
>>>> --- a/package/base-files/files/lib/upgrade/common.sh
>>>> +++ b/package/base-files/files/lib/upgrade/common.sh
>>>> @@ -180,20 +180,36 @@ get_image() { # <source> [ <command> ]
>>>> local from="$1"
>>>> local conc="$2"
>>>> local cmd
>>>> + local need_quotes=1
>>>> case "$from" in
>>>> - http://*|ftp://*) cmd="wget -O- -q";;
>>>> - *) cmd="cat";;
>>>> + scp://*)
>>>> + cmd="ssh"
>>>> + # extract custom ssh port(if specified)
>>>> + local custom_port=`echo $from | sed -ne
>>>> 's!.\+:\([0-9]\+\):.\+!\1!p'`
>>>> + [ -n "$custom_port" ] && {
>>>> + cmd="$cmd -p $custom_port"
>>>> + # remove custom port from url
>>>> + from=`echo $from | sed -ne 's!:\([0-9]\+\):!:!p'`
>>>> + }
>>>> + need_quotes=0 # here we add quotes manually
>>>> + from=`echo $from | sed -ne 's!scp://!\"!p' | sed -ne
>>>> 's!:\(.\+\)!\" \"cat \1\"!p'`
>>>> + ;;
>>>> + http://* | https://* | ftp://*) cmd="wget -O-" ;;
>>>> + *) cmd="cat" ;;
>>>> esac
>>>> +
>>>> + # if needed => add quotes to $from var
>>>> + [ $need_quotes -eq 1 ] && from=\"$from\"
>>>> +
>>>> if [ -z "$conc" ]; then
>>>> - local magic="$(eval $cmd \"$from\" 2>/dev/null | dd bs=2
>>>> count=1 2>/dev/null | hexdump -n 2 -e '1/1 "%02x"')"
>>>> + local magic="$(eval $cmd $from | dd bs=2 count=1 2>/dev/null
>>>> | hexdump -n 2 -e '1/1 "%02x"')"
>>>> case "$magic" in
>>>> 1f8b) conc="zcat";;
>>>> 425a) conc="bzcat";;
>>>> esac
>>>> fi
>>>> -
>>>> - eval "$cmd \"$from\" 2>/dev/null ${conc:+| $conc}"
>>>> + eval "$cmd $from ${conc:+| $conc}"
>>>> }
>>>> get_magic_word() {
>>>> diff --git a/package/base-files/files/sbin/sysupgrade
>>>> b/package/base-files/files/sbin/sysupgrade
>>>> index 2f441f8..fc1df22 100755
>>>> --- a/package/base-files/files/sbin/sysupgrade
>>>> +++ b/package/base-files/files/sbin/sysupgrade
>>>> @@ -137,6 +137,21 @@ include /lib/upgrade
>>>> [ "$1" = "nand" ] && nand_upgrade_stage2 $@
>>>> +get_if_URL(){
>>>> + local url="$1"
>>>> + local url_repl_file="/tmp/sysupgrade-URL.bin"
>>>> +
>>>> + case "$url" in
>>>> + http://* | https://* | ftp://* | scp://*)
>>>> + get_image "$url" "cat" > $url_repl_file
>>>> + ARGV=${ARGV/"$url"/"$url_repl_file"}
>>>> + ;;
>>>> + esac
>>>> +}
>>>> +
>>>> +# get image file from URL if specified
>>>> +get_if_URL "$ARGV"
>>>> +
>>> this get_if_url mus go away. the way it should work is
>>> 1) derive local file basename from url
>>> 2) if not there yet, download
>>> 3) if already downloaded, cat it
>>>
>>> that way only get_image needs to be changed and the functionality is
>>> self contained. get_image should transparently download the image and if
>>> it already did that cat it
>>>
>>> John
>> Here's how it happens(without get_if_UR) with TP-link devices(wget
>> verbose mode is on):
>> root at tl-wr841-v10:~# sysupgrade
>> http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgra
>>
>> de.bin
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Saving config files...
>> killall: watchdog: no process killed
>> Sending TERM to remaining processes ... dnsmasq ubusd askfirst logd rpcd
>> netifd uhttpd sync_helper ntpd
>> Sending KILL to remaining processes ... askfirst
>> Switching to ramdisk...
>> Performing system upgrade...
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> Unlocking firmware ...
>>
>> - 100% |*******************************| 3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>>
>> Appending jffs2 data from /tmp/sysupgrade.tgz to firmwar
>> Upgrade completed
>> Rebooting system...
>>
>> As you can see in this case, the same firmware file is downloaded 8
>> times in a row!
>> If we have a slow or unstable connection - this may be a problem.
>> And in my opinion: it is logically incorrect.
>>
> hence step 2/3. if get_image already has the file it should cat it
> instead of downloading it again.
Ok. Can we use:
cache_file_name=$(echo "$url" | md5sum | sed -e 's;\s.\+;;')
for cache file names instead of extracting the filename from url?
>
>> P.S: In case of not TP-Link devices we even can'tupgrade the firmware
>> using url(see target/linux/ar71xx/base-files/lib/upgrade/platform.sh,
>> function platform_do_upgrade_combined that not use cat_image at all and
>> try to do dd for raw url!)
> well that would need fixing then wont it !
>
> John
>
>>>
>>>> do_save_conffiles() {
>>>> local conf_tar="${1:-$CONF_TAR}"
>>>>
More information about the Lede-dev
mailing list