nvmf question - synchronization between target/initiator regarding partitions

[Hannes Reinecke]

On 08/07/2017 03:42 PM, Guilherme G. Piccoli wrote:
> We observed that it's possible to perform partition operations in both
> nvmf target and initiator block devices, like creating and deleting
> partitions.
> 
> But there is no sync mechanism between target and initiator regarding
> the partitions operations. After creating a partition on initiator, for
> example, we don't see it on target side. We could format it like ext4 on
> initiator, and still target cannot see it. It's possible to trigger a
> BLKRRPART ioctl on target, which end up calling revalidate_disk() on
> nvme driver and then partitions are perceived.
> 
> So, question: is this behavior expected/acceptable? Is it completely up
> to userspace to deal with the synchronization between host/target?
> I think answer might be yes since partitions are a higher level of
> abstraction than nvmf (which is purely block device aware).
> 
Yes, this is to be expected.
After all, one could argue that no partition information should ever be
visible on the target seeing that the device is exported.
As the initiator has exclusive access, the target has no business
looking at the contents; in fact, this might lead to unexpected
corruptions if things like udev jump in on the target side and try to
'correct' things with journal replay etc.

> But if kernel could/should jump in, we possibly could try to issue a
> revalidate_disk on target whenever this operation is performed on
> initiator side (and vice-versa). I confess I couldn't find such sync
> idea in NVMe over fabrics spec, though.
> Also, reading NVMe spec 1.3, we do have the optional feature
> "reservations", but seems it doesn't mention target (only multiple hosts).
> 
See above.
Never try this :-)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)