[bug report] nvme sends invalid command capsule over rdma transport for 5KiB write when target supports MSDBD > 1

[Walker, Benjamin]

> From: Christoph Hellwig <hch at infradead.org>
 
> On Wed, May 26, 2021 at 05:49:41PM +0300, Max Gurtovoy wrote:
> > > We do need for_each_sg here indeed, but you also need to keep
> > > incrementing sge for each loop iteration.  I think we can also drop
> > > the scat local variable with just a single users and all the
> > > renaming while we're at it.
> >
> > Is the above fixing the issue ?
> >
> > Seems like code refactoring to me, right ?
> 
> It fixes support for chained SGLs when using inline segments.  Not sure if it fixes
> the original bug report, but the current code is broken.

I'll get this re-tested shortly, but I've spent some time on getting this to reproduce
and it is really easy to make it happen. Simply start the SPDK target,
create an RDMA transport (RoCEv2 is fine) with in-capsule data size of 8K and
add any kind of disk as a namespace (malloc works). Then connect to it from
the kernel initiator, create and mount a filesystem on it (I used xfs), generate
a large file and then write to it in 5k blocks using dd with the odirect flag. It
hits immediately. The command claims it is a write of 10 blocks, but the single
SGL element describes in-capsule data with length 4096.

It does not happen when:
1) The transport is TCP
2) The in-capsule data size is 4K

I'll apply the patch and re-test now.

Thanks,
Ben