[RFC 00/12] io_uring dmabuf read/write support

[Pavel Begunkov]

Disclaimer: haven't been tested well enough yet and needs more beating

For past couple of months David Wei, Vishal Verma and other folks
have been mentioning that it'd be great to have dmabuf support for
read/write and other operations in io_uring. The topic is not new,
it has been discussed many times in different contexts including
networking. The last relevant attempt was premapped dma tags by
Keith [1], and this patch set took a lot from it.

This series implements it for read/write io_uring requests. The uAPI
looks similar to normal registered buffers, the user will need to
register a dmabuf in io_uring first and then use it as any other
registered buffer. On registration the user also specifies a file
to map the dmabuf for.

// register
io_uring_update_buffers(ring, { dma_buf_fd, target_file_fd });
// use
reg_buf_idx = 0;
io_uring_prep_read_fixed(sqe, target_file_fd, buffer_offset,
                         buffer_size, file_offset, reg_buf_idx);

It's an RFC to discuss the overall direction. The series misses
parts like bio splitting and nvme sgl support, and otherwise
there are some rough edges and probably problems, which will
need more testing and attention.

[1] https://lore.kernel.org/io-uring/20220805162444.3985535-1-kbusch@fb.com/

simple liburing based example:
git: https://github.com/isilence/liburing.git dmabuf-rw
link: https://github.com/isilence/liburing/tree/dmabuf-rw

kernel branch:
git: https://github.com/isilence/linux.git dmabuf-rw-v1

Pavel Begunkov (12):
  file: add callback returning dev for dma operations
  iov_iter: introduce iter type for pre-registered dma
  block: move around bio flagging helpers
  block: introduce dmavec bio type
  block: implement ->get_dma_device callback
  nvme-pci: add support for user passed dma vectors
  io_uring/rsrc: extended reg buffer registration
  io_uring: add basic dmabuf helpers
  io_uring/rsrc: add imu flags
  io_uring/rsrc: add dmabuf-backed buffer registeration
  io_uring/rsrc: implement dmabuf regbuf import
  io_uring/rw: enable dma registered buffers

 block/bdev.c                  |  11 ++
 block/bio.c                   |  21 ++++
 block/blk-merge.c             |  32 +++++
 block/blk.h                   |   2 +-
 block/fops.c                  |   3 +
 drivers/nvme/host/pci.c       | 158 +++++++++++++++++++++++
 include/linux/bio.h           |  59 ++++++---
 include/linux/blk-mq.h        |   2 +
 include/linux/blk_types.h     |   6 +-
 include/linux/blkdev.h        |   2 +
 include/linux/fs.h            |   2 +
 include/linux/uio.h           |  14 +++
 include/uapi/linux/io_uring.h |  13 +-
 io_uring/Makefile             |   1 +
 io_uring/dmabuf.c             |  60 +++++++++
 io_uring/dmabuf.h             |  34 +++++
 io_uring/rsrc.c               | 230 ++++++++++++++++++++++++++++++----
 io_uring/rsrc.h               |  23 +++-
 io_uring/rw.c                 |   7 +-
 lib/iov_iter.c                |  70 ++++++++++-
 20 files changed, 701 insertions(+), 49 deletions(-)
 create mode 100644 io_uring/dmabuf.c
 create mode 100644 io_uring/dmabuf.h

-- 
2.49.0