[BUG,REGRESSION?] 3.11.6+,3.12: GbE iface rate drops to few KB/s

[Willy Tarreau]

On Tue, Nov 19, 2013 at 04:43:48PM -0800, Eric Dumazet wrote:
> On Wed, 2013-11-20 at 01:35 +0100, Willy Tarreau wrote:
> > On Tue, Nov 19, 2013 at 04:08:49PM -0800, Eric Dumazet wrote:
> > > On Wed, 2013-11-20 at 00:53 +0100, Arnaud Ebalard wrote:
> > > 
> > > > Anyway, I think if the thread keeps going on improving mvneta, I'll do
> > > > all additional tests from RAM and will stop polluting netdev w/ possible
> > > > sata/disk/fs issues.
> > > 
> > > ;)
> > > 
> > > Alternative would be to use netperf or iperf to not use disk at all
> > > and focus on TCP/network issues only.
> > 
> > Yes, that's for the same reason that I continue to use inject/httpterm
> > for such purposes :
> >   - httpterm uses tee()+splice() to send pre-built pages without copying ;
> >   - inject uses recv(MSG_TRUNC) to ack everything without copying.
> > 
> > Both of them are really interesting to test the hardware's capabilities
> > and to push components in the middle to their limits without causing too
> > much burden to the end points.
> > 
> > I don't know if either netperf or iperf can make use of this now, and
> > I've been used to my tools, but I should take a look again.
> 
> netperf -t TCP_SENDFILE  does the zero copy at sender.
> 
> And more generally -V option does copy avoidance
> 
> (Use splice(sockfd -> nullfd) at receiver if I remember well)

OK thanks for the info.

> Anyway, we should do the normal copy, because it might demonstrate
> scheduling problems.

Yes, especially in this case, though I got the issue with GSO only,
so it might vary as well.

> If you want to test raw speed, you could use pktgen ;)

Except I'm mostly focused on HTTP as you know. And for generating higher
packet rates than pktgen, I have an absolutely ugly patch that I'm a
bit ashamed of for mvneta to multiply the number of emitted descriptors
for a given skb by skb->sk->sk_mark (which I set using setsockopt), this
allows me to generate up to 1.488 Mpps on a USB-powered system, not that
bad in my opinion :-)

Willy