[PATCH v7 0/3] FDP and per-io hints

[Javier Gonzalez]

> -----Original Message-----
> From: Christoph Hellwig <hch at lst.de>
> Sent: Monday, October 14, 2024 9:47 AM
> To: Javier Gonzalez <javier.gonz at samsung.com>
> Cc: Christoph Hellwig <hch at lst.de>; Jens Axboe <axboe at kernel.dk>; Keith Busch
> <kbusch at kernel.org>; Martin K. Petersen <martin.petersen at oracle.com>; Kanchan
> Joshi <joshi.k at samsung.com>; hare at suse.de; sagi at grimberg.me;
> brauner at kernel.org; viro at zeniv.linux.org.uk; jack at suse.cz; jaegeuk at kernel.org;
> bcrl at kvack.org; dhowells at redhat.com; bvanassche at acm.org;
> asml.silence at gmail.com; linux-nvme at lists.infradead.org; linux-
> fsdevel at vger.kernel.org; io-uring at vger.kernel.org; linux-block at vger.kernel.org;
> linux-aio at kvack.org; gost.dev at samsung.com; vishak.g at samsung.com
> Subject: Re: [PATCH v7 0/3] FDP and per-io hints
> 
> On Mon, Oct 14, 2024 at 07:02:11AM +0000, Javier Gonzalez wrote:
> > > And exactly that is the problem.  For file systems we can't support
> > > that sanely.  So IFF you absolutely want the per-I/O hints we need
> > > an opt in by the file operations.  I've said that at least twice
> > > in this discussion before, but as everyone likes to have political
> > > discussions instead of technical ones no one replied to that.
> >
> > Is it a way forward to add this in a new spin of the series - keeping the
> > temperature mapping on the NVMe side?
> 
> What do you gain from that?  NVMe does not understand data temperatures,
> so why make up that claim?  

I expressed this a couple of times in this thread. It is no problem to map temperatures
to a protocol that does not understand the semantics. It is not perfect, and in time
I agree that we would benefit from exposing the raw hints without semantics from
Block layer upwards.

But the temperatures are already there. We are not adding anything new. And thanks
to this, we can enable existing users with _minimal_ changes to user-space. As pointed
on the XFS zoned discussions, this is very similar to what you guys are doing (exactly to
re-use what it is already there), and it works. On top of this,  applications that already
understand temperatures (e.g., RocksDB) will be able to leverage FDP SSDs without changes.

> Especially as it directly undermindes any file system work to actually make use of it.

I do not think it does. If a FS wants to use the temperatures, then they would be able to leverage
FDP besides SCSI. And if we come up with a better interface later on, we can make the changes then.
I really do not see the issue. If we were adding a temperature abstraction now, I would agree with
You that we would need to cover the use-case you mention for FSs from the beginning, but this
Is already here. Seems like a fair compromise to support current users.


> 
> > If not, what would be acceptable for a first version, before getting into adding
> > a new interface to expose agnostic hints?
> 
> Just iterate on Kanchan's series for a block layer (and possible user level,
> but that's less urgent) interface for stream separation?

We can work on this later, but it is not that easy. Look at the mail I forwarded 
Form Christian. We now can store hints in the same structure as the temperatures. 
I believe that we would be able to support 128 hints. If we are serious about supporting 
hints as a general interface, 128 hints is not nearly enough. Moreover:

  - How do we convince VFS folks to give us more space for hints at this point?
  - How do we make agnostic hints and temperature co-exist? Do we use a bit to select 
     and create a protocol? This seems over-complicated
  - When we are exposing general hints to the block layer, how do we support N:N 
     FS:Block_Devices? and DMs? This is not obvious, and it goes well beyond what we
     need today, but we probably need to think about it.
 
And these are just questions that we know. You see that this is a big lift (which we 
can work together on!), but will delay current users by months. And honestly, I do
not feel comfortable doing this alone; we need application and  FS folks like you to 
help guide this so that it is really usable.

I say it again: Your idea about properly supporting hints in FS is good, and I think we should
work on it. But I do not believe that the current patches (with added things like opt in file ops) 
get in the way of this.  So if the disagreement is if these patches are harmful, let's talk about 
this explicitly and try to agree; we do not need to go over the need for a new hint interface. 
We already agree on this.