[PATCH 0/6] io_uring/nvme: support fixed buffer for metadata

[Caleb Sander Mateos]

Hi, any feedback on this series?

On a 4 KB io_uring NVMe passthru read workload with metadata, we
measure 14% of the CPU time spent under iov_iter_extract_pages()
(called from bio_integrity_map_user()) and bio_integrity_unmap_user().
Using io_uring registered buffers for the metadata avoids this
significant overhead.

Thanks,
Caleb

On Wed, Sep 9, 2026 at 3:29 PM Caleb Sander Mateos
<csander at purestorage.com> wrote:
>
> io_uring NVMe passthrough supports using a "fixed" (registered) buffer
> for data, but not metadata. On high-IOPS workloads, the pinning and
> unpinning overhead for the metadata pages is significant and could be
> avoided if fixed metadata buffers were supported.
>
> This patch series adds the necessary plumbing to allow NVMe passthrough
> commands to use fixed buffers for their metadata. The metadata and data
> fixed buffer indices can be specified (or omitted) independently.
> Supporting separate fixed buffers is important as metadata buffers are
> often stored in separate memory from the corresponding data buffers. For
> ublk zero-copy I/Os, sharing a buffer index would be impossible as the
> data buffer is a kernel registered buffer while the metadata buffer is a
> userspace registered buffer.
>
> My main question is which layer the metadata fixed buffer should belong
> to: core io_uring, io_uring_cmd, or NVMe passthrough? In this initial
> implementation, the io_uring_cmd layer stores the metadata buffer node
> and the NVMe passthrough layer defines the UAPI.
>
> The main argument for moving the implementation to a more generic layer
> would be to reuse it for other io_uring request types. For example,
> IORING_RW_ATTR_FLAG_PI could also benefit from a fixed metadata buffer
> option, though this series doesn't implement it yet.
>
> On the other hand, core io_uring_sqe and io_kiocb space is very limited,
> so it may be undesirable to dedicate it for a somewhat niche use case.
> The metadata buffer node storage could be pushed to the NVMe passthrough
> layer after my in-flight series [1] to reclaim nvme_uring_cmd_pdu space.
> However, managing request-scoped resources from a ->uring_cmd()
> implementation is a pain, as the same request can call ->uring_cmd()
> multiple times and may or may not complete when ->uring_cmd() returns,
> depending on the ->uring_cmd() return value. io_req_uring_cleanup(), in
> contrast, provides a single cleanup path for all uring_cmds.
>
> [1]: https://lore.kernel.org/io-uring/20260909155848.2069290-1-csander@purestorage.com/T/
>
> Caleb Sander Mateos (6):
>   bio-integrity: remove dead bio_integrity_copy_user() error path
>   nvme/ioctl: remove struct nvme_uring_data
>   blk-integrity: pass iov_iter to blk_rq_integrity_map_user()
>   nvme/ioctl: pass iov_iter to nvme_map_user_request()
>   io_uring/cmd: support fixed buffer for metadata
>   nvme/ioctl: support fixed buffer for metadata
>
>  block/bio-integrity.c           | 51 ++++++++++--------
>  block/blk-integrity.c           |  7 +--
>  drivers/nvme/host/ioctl.c       | 91 +++++++++++++++++++--------------
>  include/linux/bio-integrity.h   |  1 +
>  include/linux/blk-integrity.h   |  6 +--
>  include/linux/io_uring/cmd.h    | 12 ++++-
>  include/uapi/linux/nvme_ioctl.h |  5 +-
>  io_uring/rsrc.c                 | 28 ++++------
>  io_uring/rsrc.h                 | 17 ++++++
>  io_uring/uring_cmd.c            | 30 ++++++++++-
>  10 files changed, 160 insertions(+), 88 deletions(-)
>
> --
> 2.55.0
>