[RFC] Multi level NVMe stack

[Ming Lin]

On Fri, 2015-07-31 at 10:04 -0700, Ming Lin wrote:
> On Thu, Jul 30, 2015 at 7:52 AM, Christoph Hellwig <hch at infradead.org> wrote:
> > On Wed, Jul 29, 2015 at 04:55:18PM -0700, Ming Lin wrote:
> >> I'm working on a virtio-nvme driver.
> >> NVMe over Fabrics initiator will probably be available some time.
> >>
> >> I wonder does it make sense to have a multi level NVMe stack(similar as
> >> SCSI). So we can re-use the NVMe core code.
> >
> > Hi Ming,
> >
> > I've put some work in for abstraction, see the work to submit the
> > internal NVMe command through struct request.
> >
> > Once we actually get other transports we can split main data structures,
> > and move the parts sitting above that abstraction out into a separate
> > module.
> >
> > From both looking at the I/O code I'd prefer not to add any abstractions
> > in the I/O code for.
> >
> > I'd suggest you prototype the code without big refactoring first,
> > and then compare your implementation against the NVMe virtualization
> > extensions from Google and check if it's worth it:
> >
> > http://lists.infradead.org/pipermail/linux-nvme/2014-July/001076.html
> 
> I'll prototype the code.

So here is the super draft prototype code.
It's basically modified from linux/nvme-core.c and qemu/nvme.c.

http://minggr.noip.me/cgit/cgit.cgi/linux/log/?h=virtio-nvme/prototype
http://minggr.noip.me/cgit/cgit.cgi/qemu/log/?h=virtio-nvme/prototype

I haven't tried Google extensions yet.
A quick comparison with qemu-nvme:

		qemu-nvme	qemu-virtio-nvme
4k read: 	95.7MB/s	105.3MB/s	+10%
4k write:	90.2MB/s	96.3MB/s	+6.7%

I'll post the patches after I make it in good shape and do more
optimizations.