Downloading news videos with Web PVR Quick URL has stopped working

[Vangelis forthnet]

On Thu Oct 29 14:35:39 GMT 2015, Paul Phillips wrote:

> I used to be able to download news videos
> using Web PVR Quick URL. Now I can't.
> (snip)
> INFO: Trying pid: http://www.bbc.co.uk/news/health-34665151 using type: tv
> (snip)
> WARNING: The 'default' programme version could not be determined
> WARNING: No programme versions found
> (snip)
> Recording complete
>
> Any ideas what might be wrong?

 Hello Paul :-)
 I'm not a Web PVR user, but I do dinstinctly remember
using the CLI not so long ago for fetching BBC News videos
(they have the added benefit of not being geo-blocked,
at least the majority of them...).

However, trying now (GiP 2.94):

get_iplayer --url="http://www.bbc.com/news/health-34665151" --modes=best -w  
--file-prefix="Study reveals psychological effects of skin 
conditions" --tag-podcast-tv --force

results in the same download failure you reported...

 At first glance, it appears to be a change on the beeb's end...
Incidentally (though OT), get_flash_videos also used
to be able to fetch such BBC News clips, but now returns a
Error: Couldn't find BBC XML playlist URL in 
http://www.bbc.com/news/health-34665151 at get_flash_videos line 8914.

In times of trouble, I always try the "manual" route for downloading:

If you start the playback in a browser and then
right-click on the embedded player, you can get
the version PID (vpid) string for the clip:
1500kbps | RTMP (akamai) | p036hyhk | 704x396
=> vpid=p036hyhk
(you can also find this info by scavenging
Page Source for "vpid"; it's in line 302, see:
vpid":"p036hyhk).
With vpid in hand, you can try any of the following
mediaselector URLs to get access to streams:

http://www.bbc.co.uk/mediaselector/4/mtis/stream/p036hyhk -> streams
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/journalism-pc/vpid/p036hyhk - 
 > streams
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/journalism-http-tablet/vpid/p036hyhk - 
 > streams

If you are familiar with rtmpdump commands,
then you should be able to compose yourself
a command to fetch from the akamai CDN
(see attachment).
Of special note is the Direct Download link
provided by the 3rd URL, under
supplier="mf_akamai_world_plain":
http://vod-pro-ww-live.edgesuite.net/mps_h264_hi/public/news/health/1203000/1203704_h264_1500k.mp4?__gda__=1446260596_56cd91ac301a78a92eee8b42ee969eda
(appended token has limited lifespan...)

Unfortunately, GiP can't be fed a vpid string -
I and others have requested this feature in the
past, which could prove handy in cases like this
where things break, but the maintainer is
adamant in not implementing it... :-(

Instead, one must feed it a PID string or,
in this case, a "playlist" URL, which GiP
would then parse to retrieve
the all-important vpid string...

Formerly, I would start playback in Firefox
and then, via its web console, sniff traffic for
URLs containing "playlist"; now this method
reveals nothing; after being puzzled initially,
I closely inspected the whole traffic involved
and after a while I realised the sought after
playlist URL was hidden inside a new guise;
if I filter for "34665151" (found in the clip's URL),
I get the following URL:
http://www.bbc.co.uk/news/video_and_audio/api/meta/health-34665151/group/see-alsos?callback=cpItems
which is a JSON file containing all the info I need!
"href":"http:\/\/playlists.bbc.co.uk\/news\/health-34665151A\/playlist.sxml" 
=>
http://playlists.bbc.co.uk/news/health-34665151A/playlist.sxml
This is a URL/PID that GiP can now process:

get_iplayer --url="http://playlists.bbc.co.uk/news/health-34665151A/playlist.sxml" 
 -i | FindStr "modes:"
=>
modes: default: 
flashhigh1,flashlow1,flashvhigh1,hlshigh1,hlsstd1,hlsstd2,hlsvhigh1

Notice the absence of the "direct download" httpmodes,
which GiP 2.94 does not support...
Finally,

get_iplayer --url="http://playlists.bbc.co.uk/news/health-34665151A/playlist.sxml" 
 --modes=best -w --file-prefix="Study reveals psychological effects of skin 
conditions"

will fetch the flashvhigh1 tvmode!

 I believe dinkypumpkin is currently focused on
GiP providing support for the main BBC iPlayer
TV/Radio sites; BBC Sports/News/Archives are
low in his priorities list.
I am under the impression he does read the list traffic,
however if you Paul (or anyone reading this with a
forum registration) feel the urge to alert him about
BBC News being "broken", please do so at:

https://squarepenguin.co.uk/forums/

Kind regards (and Happy Halloween),
Vangelis. 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: RTMPdump_cmd.txt
URL: <http://lists.infradead.org/pipermail/get_iplayer/attachments/20151031/33628bff/attachment.txt>