[LSF/MM/BPF TOPIC] NVMe Controller Data Queue (CDQ)

[Chaitanya Kulkarni]

On 2/18/26 04:04, Joel Granados wrote:
> Hello all.
>
> I would like to propose Controller Data Queue and how it fits into the NVMe
> driver.
>
> I sent out an RFC back in July [1] describing the first version of a Controller
> Data Queue implementation in the kernel. Technical feedback at that point
> included:
>    1. CDQs are useful **only** for tracking LBAs during live migration (LM)
>    2. LM needed to be in a "proper" state for CDQs to be useful in the linux
>       kernel
>    3. CDQs should be managed like any other queue and not through an ioctl
>
> Since then, there have been six additional versions [2] that included things
> like exploring different ways of allocating memory, interactions with IOMMU, use
> of the tail pointer trigger and how to involve (or not) user space. I have been
> reluctant to send out a second RFC as the implementations are still heavy on the
> user space side.
>
> # Discussion
>
> Here are some points that I think would be interesting:
> 1. Should there be special considerations for the fact that CDQs don't really
>     have a submission queue? Specifically: How does the current NVMe queue
>     implementation need to change in order to include the fact that there are no
>     submissions for CDQs? Do they need to change at all?
>
> 2. Should the memory backing up the CDQ be segregated from the one used for
>     "regular" queues? This is relevant as CDQs might live longer and be larger
>     (proportional to the device being LM'ed) than "regular" queues.
>
> 3. Live Migration is the encompassing subject and it would be interesting to
>     note if we can already define requirements for the CDQ based on existing LM
>     POCs (if they exist) or LM implementation ideas.
>
> This proposal came out shorter than I expected, but I believe its a good
> opportunity to address the question of how LM/CDQ will look like in the
> Linux Kernel.
>
> Best Regards
>
> Joel
>
> [1] https://lore.kerne.org/20250714-jag-cdq-v1-0-01e027d256d5@kernel.org
> [2] https://github.com/SamsungDS/linux/tree/sam-cdq-v{2,3,4,5,6,7}
>

What is your usecase for this ?

-ck