[LEDE-DEV] [PATCH] base-files: sysupgrade. Added the ability to use URL as source

Сергеев Сергей adron at yapic.net
Wed Aug 24 21:58:16 PDT 2016


Hello. See below:
>
> On 24/08/2016 11:12, Сергеев Сергей wrote:
>> Hello. Ok. Leave all as is. But...(comments below):
>>> On 13/08/2016 13:04, adron at yapic.net wrote:
>>>> From: Sergey Sergeev <adron at yapic.net>
>>>>
>>>>     scp uls(like this
>>>> scp://adron@192.168.88.6:lede/lede-ar71xx-mikrotik-NAND-512b-squashfs-sysupgrade.bin)
>>>>
>>>>       is supported too. And you also can specify custom ssh
>>>> port(...8.88.6:22110:lede...)
>>>>
>>>> Signed-off-by: Sergey Sergeev <adron at yapic.net>
>>>> ---
>>>>    package/base-files/files/lib/upgrade/common.sh | 26
>>>> +++++++++++++++++++++-----
>>>>    package/base-files/files/sbin/sysupgrade       | 15 +++++++++++++++
>>>>    2 files changed, 36 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/package/base-files/files/lib/upgrade/common.sh
>>>> b/package/base-files/files/lib/upgrade/common.sh
>>>> index ea03f77..080bea5 100644
>>>> --- a/package/base-files/files/lib/upgrade/common.sh
>>>> +++ b/package/base-files/files/lib/upgrade/common.sh
>>>> @@ -180,20 +180,36 @@ get_image() { # <source> [ <command> ]
>>>>        local from="$1"
>>>>        local conc="$2"
>>>>        local cmd
>>>> +    local need_quotes=1
>>>>          case "$from" in
>>>> -        http://*|ftp://*) cmd="wget -O- -q";;
>>>> -        *) cmd="cat";;
>>>> +        scp://*)
>>>> +            cmd="ssh"
>>>> +            # extract custom ssh port(if specified)
>>>> +            local custom_port=`echo $from | sed -ne
>>>> 's!.\+:\([0-9]\+\):.\+!\1!p'`
>>>> +            [ -n "$custom_port" ] && {
>>>> +                cmd="$cmd -p $custom_port"
>>>> +                # remove custom port from url
>>>> +                from=`echo $from | sed -ne 's!:\([0-9]\+\):!:!p'`
>>>> +            }
>>>> +            need_quotes=0 # here we add quotes manually
>>>> +            from=`echo $from | sed -ne 's!scp://!\"!p' | sed -ne
>>>> 's!:\(.\+\)!\" \"cat \1\"!p'`
>>>> +        ;;
>>>> +        http://* | https://* | ftp://*) cmd="wget -O-" ;;
>>>> +        *) cmd="cat" ;;
>>>>        esac
>>>> +
>>>> +    # if needed => add quotes to $from var
>>>> +    [ $need_quotes -eq 1 ] && from=\"$from\"
>>>> +
>>>>        if [ -z "$conc" ]; then
>>>> -        local magic="$(eval $cmd \"$from\" 2>/dev/null | dd bs=2
>>>> count=1 2>/dev/null | hexdump -n 2 -e '1/1 "%02x"')"
>>>> +        local magic="$(eval $cmd $from | dd bs=2 count=1 2>/dev/null
>>>> | hexdump -n 2 -e '1/1 "%02x"')"
>>>>            case "$magic" in
>>>>                1f8b) conc="zcat";;
>>>>                425a) conc="bzcat";;
>>>>            esac
>>>>        fi
>>>> -
>>>> -    eval "$cmd \"$from\" 2>/dev/null ${conc:+| $conc}"
>>>> +    eval "$cmd $from ${conc:+| $conc}"
>>>>    }
>>>>      get_magic_word() {
>>>> diff --git a/package/base-files/files/sbin/sysupgrade
>>>> b/package/base-files/files/sbin/sysupgrade
>>>> index 2f441f8..fc1df22 100755
>>>> --- a/package/base-files/files/sbin/sysupgrade
>>>> +++ b/package/base-files/files/sbin/sysupgrade
>>>> @@ -137,6 +137,21 @@ include /lib/upgrade
>>>>      [ "$1" = "nand" ] && nand_upgrade_stage2 $@
>>>>    +get_if_URL(){
>>>> +    local url="$1"
>>>> +    local url_repl_file="/tmp/sysupgrade-URL.bin"
>>>> +
>>>> +    case "$url" in
>>>> +        http://* | https://* | ftp://* | scp://*)
>>>> +        get_image "$url" "cat" > $url_repl_file
>>>> +        ARGV=${ARGV/"$url"/"$url_repl_file"}
>>>> +        ;;
>>>> +    esac
>>>> +}
>>>> +
>>>> +# get image file from URL if specified
>>>> +get_if_URL "$ARGV"
>>>> +
>>> this get_if_url mus go away. the way it should work is
>>> 1) derive local file basename from url
>>> 2) if not there yet, download
>>> 3) if already downloaded, cat it
>>>
>>> that way only get_image needs to be changed and the functionality is
>>> self contained. get_image should transparently download the image and if
>>> it already did that cat it
>>>
>>>      John
>> Here's how it happens(without get_if_UR)  with TP-link devices(wget
>> verbose mode is on):
>> root at tl-wr841-v10:~# sysupgrade
>> http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgra
>>
>> de.bin
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Saving config files...
>> killall: watchdog: no process killed
>> Sending TERM to remaining processes ... dnsmasq ubusd askfirst logd rpcd
>> netifd uhttpd sync_helper ntpd
>> Sending KILL to remaining processes ... askfirst
>> Switching to ramdisk...
>> Performing system upgrade...
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>> Downloading
>> 'http://192.168.88.6/lede/tplink/lede-ar71xx-generic-tl-wr841-v10-squashfs-sysupgrade.bin'
>>
>> Connecting to 192.168.88.6:80
>> Writing to stdout
>> Unlocking firmware ...
>>
>> -                    100% |*******************************|  3392k
>> 0:00:00 ETA
>> Download completed (3473412 bytes)
>>
>> Appending jffs2 data from /tmp/sysupgrade.tgz to firmwar
>> Upgrade completed
>> Rebooting system...
>>
>> As you can see in this case, the same firmware file is downloaded 8
>> times in a row!
>> If we have a slow or unstable connection - this may be a problem.
>> And in my opinion: it is logically incorrect.
>>
> hence step 2/3. if get_image already has the file it should cat it
> instead of downloading it again.
Ok. Can we use:
cache_file_name=$(echo "$url" | md5sum | sed -e 's;\s.\+;;')
for cache file names instead of extracting the filename from url?
>
>> P.S: In case of not TP-Link devices we even can'tupgrade the firmware
>> using url(see target/linux/ar71xx/base-files/lib/upgrade/platform.sh,
>> function platform_do_upgrade_combined that not use cat_image at all and
>> try to do dd for raw url!)
> well that would need fixing then wont it !
>
> 	John
>
>>>
>>>>    do_save_conffiles() {
>>>>        local conf_tar="${1:-$CONF_TAR}"
>>>>   




More information about the Lede-dev mailing list