SPDK initiators (Vmware 7.x) can not connect to nvmet-rdma.

[Mark Ruijter]

The device is a Mellanox ConnectX-6 controller.
Vmware can connect to an SPDK target started on the exact same Ubuntu target system.

4: enp129s0f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4096 qdisc mq state UP group default qlen 25000
    link/ether b8:ce:f6:92:b7:b6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.34/24 brd 192.168.100.255 scope global enp129s0f0np0
       valid_lft forever preferred_lft forever

81:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
81:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]

--
# cd /sys/kernel/config/nvmet/ports/
# ls
11290  11291  21290  21291
# cd 11290
# cat *
ipv4
192.168.100.34
not specified
4420
rdma
--

On 01/09/2021, 16:58, "Sagi Grimberg" <sagi at grimberg.me> wrote:


    > Hi Sagi,
    > 
    > I am using VMware 7.x as initiator with RDMA.
    > The target system is running Ubuntu 20.04.3 LTS with kernel 5.11.22+.
    > 
    > The device that is exported is an LVM volume, however I also tested with a file backed loop device.
    > Connecting with SPDK seems to be the problem and as reported on the SPDK mailing-list it can be used to reproduce the issue when VMWare is not available.
    > ./perf -q 64 -P 1 -s 4096 -w read -t 300 -c 0x1 -o 4096 -r 'trtype:RDMA adrfam:IPv4 traddr:169.254.85.8 trsvcid:4420'
    > 
    > This seems to produce a similar result:
    > nvme connect --transport=rdma --queue-size=1024 --nqn=testnqn_1 --traddr=169.254.85.8 --trsvcid=4420
    > 
    > I hope this helps,

    I meant which rdma device are you using? that device is failing the
    qp creation.