[PATCH v1 0/2] update RDMA controllers queue depth
Max Gurtovoy
mgurtovoy at nvidia.com
Tue Sep 21 12:04:43 PDT 2021
Hi all,
This series is solving the issue that was introduced by Mark Ruijter
while testing SPDK initiators on Vmware-7.x while connecting to Linux
RDMA target running on NVIDIA's ConnectX-6 Mellanox Technologies
adapter. During connection establishment, the NVMf target controller
expose a 1024 queue depth capability but wasn't able to satisfy this
depth in reality. The reason for that is that the NVMf driver didn't
take the underlying HW capabilities into consideration. For now, limit
the RDMA queue depth to a value of 128 (that is the default and works
for all the RDMA controllers probably). For that, introduce a new
controller operation to return the possible queue size for a given HW.
Other transport will continue with thier old behaviour.
In the future, in order to increase this size, we'll need to create a
special RDMA API to calculate a possible queue depth for ULPs. As we
know, the RDMA IO operations sometimes are built from multiple WRs (such
as memory registrations and invalidations) that the ULP driver should
take this into consideration during device discovery and queue depth
calculations.
Max Gurtovoy (2):
nvmet: add get_queue_size op for controllers
nvmet-rdma: implement get_queue_size controller op
drivers/nvme/target/core.c | 8 +++++---
drivers/nvme/target/nvmet.h | 1 +
drivers/nvme/target/rdma.c | 8 ++++++++
3 files changed, 14 insertions(+), 3 deletions(-)
--
2.18.1
More information about the Linux-nvme
mailing list