[PATCH v7 1/1] nvmet: support reservation feature

[Sagi Grimberg]

On 13/03/2024 13:56, Guixin Liu wrote:
>
> 在 2024/3/13 17:54, Sagi Grimberg 写道:
>>
>>
>> On 13/03/2024 5:42, Guixin Liu wrote:
>>>
>>> 在 2024/3/13 05:31, Sagi Grimberg 写道:
>>>> So I understand from you that there is out-of-band mechanism for 
>>>> fence guarantee?
>>>>
>>>> In any event, Christoph reminded me that preempt and abort is 
>>>> mandatory, so we cannot
>>>> get away without doing it.
>>>>
>>> Agree.
>>>> I suggest to introduce a per-ns-per-controller percpuref, with an 
>>>> xarray that is rcu protected.
>>>
>>> I will implement it in v9, and I also think that we should a 
>>> time-limited wait.
>>
>> If it is time-limited, and you didn't manage to wait, so you need to 
>> fail the preempt and abort
>> action (without DNR set so the reservation holder can retry if it 
>> chooses to).
>>
> It can wait.
>>>
>>>>
>>>> btw, I also think that reservations on nsid=0xffffffff should also 
>>>> work. IIRC there were at least
>>>> two use-cases that would have benefited from reservations on all 
>>>> subsystem namespaces.
>>>> It can also be a lot more lightweight to implement that (outside of 
>>>> conflicting reservations checks). 
>>>
>>> You mean one host can reserve all of namespaces by sending one 
>>> reservation command?
>>
>> Yes. The use-case is to have the reservation in the subsystem level.
>> The spec seems to imply that this is a valid reservation request.
>>
>>>
>>> Well, Could you please tell me which two use-cases?
>>
>> Don't remember for sure, but IIRC I've seen people run oracle rac 
>> with all namespaces in the
>> nvmet subsystem is exposed to DB hosts, and there were many namespaces.
>>
>>>
>>> I think this will make the target code too complicated, because the 
>>> reservation situation on each
>>>
>>> namespace may different, we should handle it one-by-one, and if 
>>> failed in middle, we should revert
>>>
>>> the handled namespaces.
>>
>> Not sure why you'd assume that, this would be a subsystem-wide 
>> reservation, which should simply
>> cross-check individual namespace reservations for conflicts. Sure it 
>> adds complication, but I think implementing
>> it on every individual namespace is a naive implementation.
>
> Well, this is strange, a few years ago, when I tested SCSI 
> reservations with Oracle RAC,
>
> I did not observe any "all LUN" reservation commands being issued.
>
> I also researched LIO and SPDK, and found that neither supports "all 
> LUN/NS" reservation commands.
>
> When a host issued a reservation command without carrying a specific 
> LUN or NS ID, both returned an error.

Didn't say this is actually in use, but a use-case that would benefit.