[PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO

[Clay Mayers]

> From: Keith Busch <kbusch at kernel.org>
> Sent: Monday, February 8, 2021 7:13 PM
> To: Clay Mayers <Clay.Mayers at kioxia.com>
> Cc: linux-nvme at lists.infradead.org; Jens Axboe <axboe at fb.com>; Chaitanya
> Kulkarni <Chaitanya.Kulkarni at wdc.com>; Christoph Hellwig <hch at lst.de>;
> Sagi Grimberg <sagi at grimberg.me>
> Subject: Re: [PATCH V2 0/2] nvme: Support for fused
> NVME_IOCTL_SUBMIT_IO
> 
> On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> > Is there any other feedback on V2?
> >
> > My main concern I have about my implementation is how fused requests
> > are tunneled through the mq request layer.  The 1st request is marked
> > as started but it won't be in the device until the 2nd command is
> > queued.  As Keith pointed out, a device reset can split the two so
> > care must be taken to correctly handle this case.  Despite this, I
> > thought this was a better approach than modifying mq requests to be
> > fused.  Especially given Christoph's concern of cost vs value.  This is the
> lightest touch I could come up with.
> >
> > Further consideration of this patch may need a more compelling use case.
> > I've worked on a proprietary storage systems that relied on fused
> > NVMeOF support so it seems compelling to me.  There's a comment in
> > target/core.c that there is "no support for fused commands yet"
> > implying it's been considered.  Is pci only support for fused too soon
> > or too little?  What would make it more compelling?
> 
> The complications it introduces to the IO path and error handling for an
> archaic feature has me on the "Nak" side. NVMeOF was introduced well after
> the spec define Reservations, and the kernel has supported that capability
> for many years. I'm not aware of any other use case for fused commands, so
> it appears to be dead weight in the spec.

Thanks for sharing your perspective.  I can see the point that reservations are
good enough for many.  For VmWare, moving from reservations to cmp/write
on SCSI led to an increase in performance (couldn't find an actual # but heard
30% anecdotally). That's a specialized use case (storage for VMs) on a much
slower medium that may not translate to more general storage systems or
much faster storage.