[PATCH 15/18] nvmet-tcp: enable TLS handshake upcall

[Chuck Lever III]

> On Mar 22, 2023, at 12:43 PM, Hannes Reinecke <hare at suse.de> wrote:
> 
> On 3/22/23 16:42, Sagi Grimberg wrote:
>>>>> The 'data_ready' call might happen at any time after the 'accept' call and us calling into userspace.
>>>>> In particular we have this flow of control:
>>>>> 
>>>>> 1. Kernel: accept()
>>>>> 2. Kernel: handshake request
>>>>> 3. Userspace: read data from socket
>>>>> 4. Userspace: tls handshake
>>>>> 5. Kernel: handshake complete
>>>>> 
>>>>> If the 'data_ready' event occurs between 1. and 3. userspace wouldn't know that something has happened, and will be sitting there waiting for data which is already present.
>>>> 
>>>> Umm, doesn't userspace read from the socket once we trigger the upcall?
>>>> it should. But I still don't understand what is the difference between
>>>> us waiking up userspace, from the default sock doing the same?
>>>> 
>>> No, it doesn't (or, rather, can't).
>>> After processing 'accept()' (from the kernel code) data might already be present (after all, why would we get an 'accept' call otherwise?).
>>> But the daemon has not been started up (yet); that's only done in
>>> step 3). But 'data_ready' has already been called, so by the time userland is able to do a 'read()' on the socket it won't be seeing anything.
>> Not sure I understand. if data exists, userspace will read from the
>> socket and get data, whenever that is. >
> That's what I thought, too.
> But then the userspace daemon just sat there doing nothing.

I haven't been following this discussion in detail, but
if the kernel disables the normal TCP data_ready callback,
then user space won't get any data. That's why SunRPC's
data_ready calls the previous sk_data_ready and then shunts
its own data_ready callback during handshakes. Without that
call to the old sk_data_ready, the user space endpoint won't
see any received data.

As long as the previous sk_data_ready callback is invoked
properly during the handshake, the received data is queued
and the user endpoint will behave exactly as you want.

--
Chuck Lever