consultation about NVME_SGL_FMT_INVALIDATE not set when using nvme-rdma t10-pi

[Chaitanya Kulkarni]

Hi Ruozhu,

On 10/27/21 3:49 AM, Max Gurtovoy wrote:
> Hi Ruozhu,
> 
> The performance tests we did for all the IO sizes showed better 
> performance with local invalidation in most of the cases (mainly for 
> large IOs, the remote invalidation task is harder for the HW).
> 
> Maybe there is 1 or 2 cases, such as small IO latency, that improves 
> with remote-invalidation but for now we decided to disable it.
> 
> I guess we can add a capability for the device that will indicate it in 
> the future.
> 
> What is the difference you see ? and what is the scenario ?
> 
> Thanks,
> 
> -Max.
> 
> On 10/27/2021 11:50 AM, liruozhu wrote:
>> Hi Max,
>>
>> I was testing T10-PI feature of nvme-rdma recently, and found that 
>> when using T10-PI, nvme-rdma driver did not set 
>> NVME_SGL_FMT_INVALIDATE flag in the sgl type field, so the host 
>> software needs to do invalid rkey by itself. I tried to add the flag 
>> to test it. Turns out T10-PI feature is still working, and I get 
>> better IO latency with it enable.

Please share your patch (informal or formal) with the performance 
numbers and the workload you have used that shows better I/O latency.
It will make easier for everyone to understand the benefits...

>> I read the original patches in the mailing list, and found that patch 
>> v1 set this flag. But it was silent dropped on patch v5 without any 
>> comments. Is it for any special considerations to delete it?
>>
>> Thanks,
>> Ruozhu
>>
>