Stream corrupt error and then rtmpdump goes nuts

[Neill Mitchell]

Yes, I think we understand this pretty well now :) It also explains why
adding the --funroll-loops compiler optimisation option improves matters.
The initial give away is that it always failed within the first few
hundred KB of the download. Initialisation overheads was enough to tip
it over.

I've got 100% reliability now after upping rtmpdump's working buffer and
optimising the loops.

Cheers
Neill.

On 10/01/13 22:07,get_iplayer-request at lists.infradead.org  wrote:
> ------------------------------ Message: 2 Date: Thu, 10 Jan 2013
> 20:55:07 +0000 From: dinkypumpkin<dinkypumpkin at gmail.com>  To:
>get_iplayer at lists.infradead.org  Subject: Re: Stream corrupt error and
> then rtmpdump goes nuts Message-ID:<50EF2AAB.2000109 at gmail.com>  
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed On
> 10/01/2013 16:11, Neill Mitchell wrote:
>> Yes, I see exactly the same problems. I've upped the TCP buffers to 12MB
>> and it's still unreliable. When you think about it, if the program can't
>> keep up then it doesn't matter how big you make the low level TCP
>> buffers, rtmpdump's own buffer will overflow. The TCP buffers are there
>> to optimise the naturally bursty nature of normal data transfers. But
>> these streams are a sustained download of hundreds of megs.
> If it's breaking at exactly the same point for different downloads, then
> rtmpdump must be the problem.  I've seen network parameter fiddling help
> my own applications on underpowered clients, but that obviously doesn't
> apply here.
>
>> I think this is down to purely down to loop overhead processing speed.
>> When rtmpdump has 1KB buffers it simply can't loop round fast enough to
>> keep up with the data pouring in at 80MB/sec. With 4KB buffers the loop
>> runs at a quarter of the frequency and it manages.
> I had another look at the code and I think I understand this now.  The
> buffer you're resizing doesn't have anything directly to do with reading
> packets off the wire, so apologies for the wild goose chase.  A lot of
> the media packets will be >64K, so by pre-allocating the working buffer
> to 256K, you're saving a bunch of memory allocations and buffer copies,
> which would speed up the processing a lot in relative terms.
>