Inconsistencies

Sat Aug 29 13:26:50 PDT 2015

On Sat Aug 29 15:41:55 BST 2015, Alan Milewczyk wrote:

>> Ah! Now I've not tried hls. That's worth an experiment.
>> Can I just substitute  --modes=hlsvhigh for tvmode=best ?
>
> Yes, I was originally alerted to this option by Vangelis when I was
> having problems with looooong programmes, such as Wimbledon.

Hello Jim & Alan - only now came to read this thread;
some clarifications/corrections:

@Alan:
If you go back to our original exchange,
you'll discover that I NEVER SUGGESTED
--modes=hlsvhigh;
this value is equivalent to flashvhigh in terms of
video file resolution and will only produce
832x468p video files; what I did infact suggest was
--modes=hlshd
which will fetch HD files (1280x720p) via HTTP
- when the programme has been made available in
HD, that is - and can be used as an alternative to
--modes=flashhd or --modes=best for >4GiB
video files over RTMP (to circumvent the well
known/discussed in this list issue that the RTMP
protocol (hence rtmpdump) has with  >4GiB files).
So NO,
--modes=hlsvhigh & --tvmode=best
ARE NOT INTERCHANGEABLE in many levels...

> Also, if you specify the following command line:
> get_iplayer "programme name" --option
> you will get a list of the various options available.
> I forgot to say that I check this switch before
> I go down the hslvhigh route.
> At least the output alerts you to an HD version being available
> when you are initially only able to download the SD option.

 With respect Alan, you must have meant to type
another command to the one posted;
"--option" is not even a valid switch...
To inquire the available modes prior to downloading,
you should use the --info (-i) switch!
On windows, I type:

get_iplayer --pid=<pid> -i | FindStr modes

(on Linux:
get_iplayer --pid=<pid> -i | grep modes )

and in the CPW GiP informs me of the available
tvmodes for a given PID, along with estimates of their file sizes...

@Jim:

For TV files, GiP 2.94 relies upon rtmpdump to
fetch flash modes (over RTMP) and upon ffmpeg to fetch
hls modes (over HTTP).
I can assure you with a high degree of certainty that
GiP 2.94 tries its best to download HD files IF THEY
HAVE BEEN MADE AVAILABLE by the beeb for
a specific PID; so the --modes=best approach you take
is correct; do bare in mind though that, as stated once again
in this thread, HD files (especially if they correspond to live
events) DO TAKE LONGER to be produced by the encoder
chain the beeb uses; so do not jump right up to fetch a programme
immediately after transmission; --modes=best may produce
the flashvhigh flavour, but the next day used with --force
may fetch flashhd...

 Since the beginning of July, the BBC has made available
on iPlayer for video-on-demand some new types of streams,
produced by their partner "Unified Streaming"
http://www.unified-streaming.com/
delivered via AdobeHDS, AppleHLS and (more recently)
MPEG-DASH.

1. AFAIK, the MPEG-DASH streams can't yet be downloaded
(chime in if anyone knows of a MPEG-DASH dumping script/tool).
FYI, the specs of the audio/video streams of those are:

available audio streams:
1. 48kbps HE-AACv1 (SR=24kHz)
2. 96kbps HE-AACv1 (SR=24kHz)

Available video streams:
1. 384x216p, 281kbps, 25FPS
2. 512x288p, 437kbps, 25FPS
3. 704x396p, 827kbps, 25FPS
4. 960x540p, 1604kbps, 25FPS
5. 960x540p, 2812kbps, 50FPS
6. 1280x720p, 5070kbps, 50FPS

Each elementary audio stream is combined
adaptively with a video stream, depending
on network conditions..

2. The USP (Unified Streaming Platform) AdobeHDS streams
offer the following variants:

1) Video: resolution 1280x720p, FR 50fps, BR 5070kbps - Audio: HE-AACv1 
96kbps - overall BR 5166 - average filesize 2.13GB/h
2) Video: resolution 960x540p, FR 50fps, BR 2812kbps - Audio: HE-AACv1 
96kbps - overall BR 2908 - average filesize 1.21GB/h
3) Video: resolution 960x540p, FR 25fps, BR 1604kbps - Audio: HE-AACv1 
96kbps - overall BR 1700 - average filesize 724MB/h
4) Video: resolution 704x396p, FR 25fps, BR 827kbps - Audio: HE-AACv1 
96kbps - overall BR 923 - average filesize 393MB/h
5) Video: resolution 512x288p, FR 25fps, BR 437kbps - Audio: HE-AACv1 
96kbps - overall BR 533
6) Video: resolution 384x216p, FR 25fps, BR 281kbps - Audio: HE-AACv1 
96kbps - overall BR 377

 While it is possible to download those streams currently
(not with GiP), I will refrain from discussing this further,
at least for now - after all, the beeb can encrypt all the USP streams
in the near future with the simple turning on of a switch...

3. The USP AppleHLS streams will be supported in GiP 2.95
(already supported in the develop branch); since those streams
are targeted for mobile devices, no HD resolutions are offered
currently; the best quality variant for TV files has the following specs:

Video: resolution 960x540p, FR 25fps, BR 1604kbps - Audio: HE-AACv1 96kbps - 
overall BR 1700 - average filesize 724MB/h

So this is slightly better than the current flash(hls)vhigh tvmode
in terms of the video stream...
You will notice Jim that even with these new types of streams,
destined to substitute sometime later the existing RTMP/HLS
ones, the audio quality has been left the same (96kbps),
if not worsened, since the audio streams now employ the SBR
profile 
(https://en.wikipedia.org/wiki/High-Efficiency_Advanced_Audio_Coding).

Now, with regards to what you called "Inconsistencies" in the
fashion the BBC provides HD versions for video files, from
my experience I can say the following:

PID strings which begin with "b0xxxxxx" 99.9% of times
correspond to (full) broadcast episodes/shows - these do
come into HD versions (as a rule of thumb; exceptions may
apply...).
E.g. Prom 32, pid=b065gyln; show's page is:
http://www.bbc.co.uk/programmes/b065gyln
where you can see it was broadcast on telly
(BBC Four, 14/08/2015, 19:30 BST)
get_iplayer --pid=b065gyln -i | grep modes
will alert you of the HD variants (flashhd, hlshd).

PID strings that begin with "p0xxxxxx" can also correspond
sometimes to full episodes broadcast on TV, but usually
the are reserved for Red Button/iPlayer-only material.
If the BBC treats the video material as an excerpt of a
broadcast episode, then they label it as "clip" and this
is usually not made available in HD; e.g.
Prom 32, pid=p030dys6; show's page is:
http://www.bbc.co.uk/programmes/p030dys6
where you can see "This clip is from" :
http://www.bbc.co.uk/programmes/b064ncm2

get_iplayer --pid=p030dys6 -i | grep modes

will let you know that the highest qualities are
flashvhigh, hlsvhigh (832x468).

On the other hand,
Prom 32, pid=p02wwc3g; show's page is:
http://www.bbc.co.uk/programmes/p02wwc3g,
IS NOT TREATED BY THE BBC AS A CLIP,
rather like a standalone iPlayer-only episode
(notice the lack of  the "This clip is from"
reference on the show's page) and these are
usually made available as HD;

get_iplayer --pid=p02wwc3g -i | grep modes

will support my claim (flashhd, hlshd present).

I haven't tested them all myself, but I bet you
two pence that all the individual PIDs you mention in
http://lists.infradead.org/pipermail/get_iplayer/2015-August/008162.html
can fall under the same logic:
If it's a "clip", then 832x468 is the highest you can get...

I hope this is an interesting "before-bed" read for you...
Apologies to all others that got bored by reading this to the end...

Many kind regards to all,
Vangelis. 