[LEDE-DEV] jshn.sh: json_load replaces "-" with "_"

[Paul Spooren]

On 10/30/17 3:28 AM, Matthias Schiffer wrote:
> On 10/29/2017 09:21 PM, Paul Spooren wrote:
>> Hi all,
>>
>> I'm currently trying to use jshn.sh but it changes the data and makes it
>> unusable.
>>
>> Below an example:
>>
>> root at LEDE:~# json_string="{ \"foo-bar\": 10 }"
>> root at LEDE:~# echo $json_string
>> { "foo-bar": 10 }
>> root at LEDE:~# json_init
>> root at LEDE:~# json_load "$json_string"
>> root at LEDE:~# json_dump
>> { "foo_bar": 10 }
>>
>> Expected would be the same output as input. Is this a bug or a feature?
>>
>> Context:
>>
>> Running snapshot build. I'm trying to use rpcd-mod-packagelist [1] but
>> as e.g. "kmod-r8169" becomes "kmod_r8169" it's useless.
>>
>> Best,
>>
>> Paul Spooren
>>
>> [1]:
>> https://github.com/openwrt/packages/tree/master/utils/rpcd-mod-packagelist
> This is definitely a bug in jshn.sh (or possibly a "known limitation" to
> keep the code simple, as shell variable names can't contain dashes). It
> should be possible to fix this in jshn with some clever escaping of keys -
> at least for such "simple" special characters (without looking into it in
> detail, I assume other unusual characters that aren't allowed in shell
> variable names are also an issue; supporting the full range of unicode
> might need more work, or entail a prohibitive performance penalty.)
>
> It might actually be much easier to fix the concrete issue by rewriting
> rpcd-mod-packagelist in C (as I suggested in [2]).
>
> Matthias
>
>
> [2] https://github.com/openwrt/packages/pull/5037
>
>
As requested [1] here the detailed use case:

TL;DR: packages are parsed via rpcd-mod-packagelist, additional info is
attached and send as a json post to an "image on demand" server.

I started to create an build server [2] that builds firmware images
based on delivered parameters. A possible use case is an simple "online
imagebuilder" [3] or as a simple way to upgrade you existing system to a
new release without the need to reinstall any package.

ubus and rpcd is needed for the luci frontend
luci-app-attendedsysupgrade [4]. Recently I created a CLI version with
the same functionality [5]. To send a valid request I need a packagelist
from rpcd-mod-packagelist and some information from $(ubus call system
board). To combine both to a json request I tried the following:

json_init
json_load "$(ubus call packagelist list)"
json_add_string "target" $target
json_add_string "subtarget" $subtarget

This results in a valid structure but all packagenames are renamed
(rpcd-mod-packagelist becomes rpcd_mod_packagelist). The error happens
as json_load runs jshn(.c) -r $input, jshn replaces all "-" with "_".
This seemed strange to me as jshn.sh can handle json_add_string
"foo-bar" "foo" (as used in rpcd-mod-packagelist).

A possible solution could be to add the code of rpcd-mod-packagelist to
the attendedsysupgrade.sh to avoid json_load.

Paul

[1]: https://github.com/openwrt/packages/pull/5037#issuecomment-340416041
[2]: https://github.com/aparcar/gsoc17-attended-sysupgrade
[3]: https://ledeupdate.planetexpress.cc/imagebuilder
[4]:
https://github.com/openwrt/luci/tree/master/applications/luci-app-attendedsysupgrade
[5]: https://github.com/aparcar/packages/tree/attendedsysupgrade