Stream corrupt error and then rtmpdump goes nuts

[dinkypumpkin]

On 10/01/2013 16:11, Neill Mitchell wrote:
> Yes, I see exactly the same problems. I've upped the TCP buffers to 12MB
> and it's still unreliable. When you think about it, if the program can't
> keep up then it doesn't matter how big you make the low level TCP
> buffers, rtmpdump's own buffer will overflow. The TCP buffers are there
> to optimise the naturally bursty nature of normal data transfers. But
> these streams are a sustained download of hundreds of megs.

If it's breaking at exactly the same point for different downloads, then 
rtmpdump must be the problem.  I've seen network parameter fiddling help 
my own applications on underpowered clients, but that obviously doesn't 
apply here.

> I think this is down to purely down to loop overhead processing speed.
> When rtmpdump has 1KB buffers it simply can't loop round fast enough to
> keep up with the data pouring in at 80MB/sec. With 4KB buffers the loop
> runs at a quarter of the frequency and it manages.

I had another look at the code and I think I understand this now.  The 
buffer you're resizing doesn't have anything directly to do with reading 
packets off the wire, so apologies for the wild goose chase.  A lot of 
the media packets will be >64K, so by pre-allocating the working buffer 
to 256K, you're saving a bunch of memory allocations and buffer copies, 
which would speed up the processing a lot in relative terms.