[Lsf-pc] [LSF/MM/BPF ATTEND][LSF/MM/BPF TOPIC] Meta/Integrity/PI improvements

[Kanchan Joshi]

On 4/2/2024 4:15 PM, Dongyang Li wrote:
> Martin, Kanchan,
>>
>> Kanchan,
>>
>>> - Generic user interface that user-space can use to exchange meta.
>>> A
>>> new io_uring opcode IORING_OP_READ/WRITE_META - seems feasible for
>>> direct IO.
>>
>> Yep. I'm interested in this too. Reviving this effort is near the top
>> of
>> my todo list so I'm happy to collaborate.
> If we are going to have a interface to exchange meta/integrity to user-
> space, we could also have a interface in kernel to do the same?

Not sure if I follow.
Currently when blk-integrity allocates/attaches the meta buffer, it 
decides what to put in it and how to go about integrity 
generation/verification.
When user-space is sending the meta buffer, it will decide what to 
put/verify. Passed meta buffer will be used directly, and blk-integrity 
will only facilitate that without doing any in-kernel 
generation/verification.

> It would be useful for some network filesystem/block device drivers
> like nbd/drbd/NVMe-oF to use blk-integrity as network checksum, and the
> same checksum covers the I/O on the server as well.
> 
> The integrity can be generated on the client and send over network,
> on server blk-integrity can just offload to storage.
> Verify follows the same principle: on server blk-integrity gets
> the PI from storage using the interface, and send over network,
> on client we can do the usual verify.
> 
> In the past we tried to achieve this, there's patch to add optional
> generate/verify functions and they take priority over the ones from the
> integrity profile, and the optional generate/verify functions does the
> meta/PI exchange, but that didn't get traction. It would be much better
> if we can have an bio interface for this.

Any link to the patches?
I am not sure what this bio interface is for. Does this mean 
verify/generate functions to be specified for each bio?
Now also in-kernel users can add the meta buffer to the bio. It is up to 
the bio owner to implement any custom processing on this meta buffer.