Inconsistencies

[Jim web]

In article <3A4ECFBEE5214BA59C0111B454BE93E2 at vasonote>, Vangelis
forthnet

Snipped, thanks for the clarifications!

>  With respect Alan, you must have meant to type another command to the
> one posted; "--option" is not even a valid switch... To inquire the
> available modes prior to downloading, you should use the --info (-i)
> switch! On windows, I type:

> get_iplayer --pid=<pid> -i | FindStr modes

> (on Linux: get_iplayer --pid=<pid> -i | grep modes )

> and in the CPW GiP informs me of the available tvmodes for a given PID,
> along with estimates of their file sizes...

OK, thanks, I'll try using that as it means I can check in advance as well
as checking afterward with ffprobe.


> @Jim:

> For TV files, GiP 2.94 relies upon rtmpdump to fetch flash modes (over
> RTMP) and upon ffmpeg to fetch hls modes (over HTTP). I can assure you
> with a high degree of certainty that GiP 2.94 tries its best to download
> HD files IF THEY HAVE BEEN MADE AVAILABLE by the beeb for a specific
> PID; so the --modes=best approach you take is correct; do bare in mind
> though that, as stated once again in this thread, HD files (especially
> if they correspond to live events) DO TAKE LONGER to be produced by the
> encoder chain the beeb uses; so do not jump right up to fetch a
> programme immediately after transmission; --modes=best may produce the
> flashvhigh flavour, but the next day used with --force may fetch
> flashhd...

Yes, I understand that and try to allow for it.

My usual practice is to do a fetch the next day. During each day I write a
simple text file listing the pid's of the items I want. Then feed this to
gip about 7:30am the next morning. (This is to put the fetching outside the
time period that is 'metered' by my DSP.)

This nearly always gives me HD. If I doesn't I retry a few days or a week
or two later.

So for the Proms 'inconsistencies' I was trying over a number of days
before I raised the matter here. I've gone on trying and, erm, the
inconsistencies are consistent. ;-> 

So far I've only used the 'best' mode though. not the other suggestions.


>  Since the beginning of July, the BBC has made available on iPlayer for
> video-on-demand some new types of streams, produced by their partner
> "Unified Streaming" http://www.unified-streaming.com/ delivered via
> AdobeHDS, AppleHLS and (more recently) MPEG-DASH.

That's interesting. I was going to ask someone something anyway, so I'll
also ask about the start of MPEG-DASH. So far I've only been aware of tests
like the surround sound ones. (Which TBH don't interest me much as I'm old
fashioned and would rather have the bandwidth devoted to better stereo.)


> 1. AFAIK, the MPEG-DASH streams can't yet be downloaded (chime in if
> anyone knows of a MPEG-DASH dumping script/tool). FYI, the specs of the
> audio/video streams of those are:

> available audio streams: 1. 48kbps HE-AACv1 (SR=24kHz) 2. 96kbps
> HE-AACv1 (SR=24kHz)

Do you mean an audio sample rate of 24k? That seems alarmingly low. The
usual rate for all the BBC output/feeds (pace VHF/FM chain) is 48k. Is the
system to rely entirely on spectral 'extrapolation' (i.e. SBR) above about
10kHz? I can see that would be OK for speech or industrial-grade pop with
'autotuned-into-android singers, etc. But isn't ideal for other types of
music, etc.



> 3. The USP AppleHLS streams will be supported in GiP 2.95 (already
> supported in the develop branch); since those streams are targeted for
> mobile devices, no HD resolutions are offered currently; the best
> quality variant for TV files has the following specs:

> Video: resolution 960x540p, FR 25fps, BR 1604kbps - Audio: HE-AACv1
> 96kbps - overall BR 1700 - average filesize 724MB/h

> So this is slightly better than the current flash(hls)vhigh tvmode in
> terms of the video stream... You will notice Jim that even with these
> new types of streams, destined to substitute sometime later the existing
> RTMP/HLS ones, the audio quality has been left the same (96kbps), if not
> worsened, since the audio streams now employ the SBR profile
> (https://en.wikipedia.org/wiki/High-Efficiency_Advanced_Audio_Coding).

The difficulty here is that how the encoder and its settings apply the SBR,
etc, is critical when the bitrates are crap. I confess I'm amazed that the
HDTV currently sounds as decent as it does given the low rate. But I'm
getting the impression that as the UK's broadband, etc, improves, the BBC
are limiting the bitrates. Presumably to keep down the loading (and hence
cost). If you're right about the 24k I'll ask around about this and
who/how/when it was decided.

Presimably TV people are obsessed with video quality and regard audio as an
annoying relative.


> Now, with regards to what you called "Inconsistencies" in the fashion
> the BBC provides HD versions for video files, from my experience I can
> say the following:

> PID strings which begin with "b0xxxxxx" 99.9% of times correspond to
> (full) broadcast episodes/shows - these do come into HD versions (as a
> rule of thumb; exceptions may apply...). E.g. Prom 32, pid=b065gyln;
> show's page is: http://www.bbc.co.uk/programmes/b065gyln where you can
> see it was broadcast on telly (BBC Four, 14/08/2015, 19:30 BST)
> get_iplayer --pid=b065gyln -i | grep modes will alert you of the HD
> variants (flashhd, hlshd).

> PID strings that begin with "p0xxxxxx" can also correspond sometimes to
> full episodes broadcast on TV, but usually the are reserved for Red
> Button/iPlayer-only material. If the BBC treats the video material as an
> excerpt of a broadcast episode, then they label it as "clip" and this is
> usually not made available in HD; e.g. Prom 32, pid=p030dys6; show's
> page is: http://www.bbc.co.uk/programmes/p030dys6 where you can see
> "This clip is from" : http://www.bbc.co.uk/programmes/b064ncm2

> get_iplayer --pid=p030dys6 -i | grep modes

> will let you know that the highest qualities are flashvhigh, hlsvhigh
> (832x468).


Is there a specific list for linking mode 'names' like flashvhigh to
a specific resolution? When I use the above command I get a list
of the names, but no list of their resolutions.



> On the other hand, Prom 32, pid=p02wwc3g; show's page is:
> http://www.bbc.co.uk/programmes/p02wwc3g, IS NOT TREATED BY THE BBC AS A
> CLIP, rather like a standalone iPlayer-only episode (notice the lack of 
> the "This clip is from" reference on the show's page) and these are
> usually made available as HD;

> get_iplayer --pid=p02wwc3g -i | grep modes

> will support my claim (flashhd, hlshd present).

> I haven't tested them all myself, but I bet you two pence that all the
> individual PIDs you mention in
> http://lists.infradead.org/pipermail/get_iplayer/2015-August/008162.html
> can fall under the same logic: If it's a "clip", then 832x468 is the
> highest you can get...

Thanks for the above. I suspect you are correct wrt to the 'clip'
treatment. The puzzle, though, is why some items are regarded that way when
others in the same Prom are not! At first I wondered if the distinction was
between items that were broadcast over-air and those not. But no, that
doesn't fit. It may be that different people selecting item from a concert
simply make different assumptions. So differences in opinion rather than
any rule.


It occurs to me to wonder if the p0/b0 variations are again sometimes
down to operators having different working methods. I've often seen
p0 programmes which clearly aren't a 'clip' from a program. 


It occurs to me now that FWIW in the past I've had someone at the BBC
send me source files so I can check them against what I 'received' via the
broadcasts or iplayer. The aim being to check to find if I get what
the BBC expect me to get as to validate the chain. However more
than once they have sent me an 'entire evening' file for radio as this
was easier for them than just sending a target program itself that was
on that evening. So maybe some operators start from different input or
working assumptions and there is no rule checked and enforced.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html