[PATCH RFC] nvme-ioctl: propagate PRP1 from ioctl to admin cmd

[顾泽兵]

> The system setup where this patch has been used is as follows:
> - P2P PCIe capable CPU (currently also IOMMU disabled)
> - patched Linux in-Kernel NVMe driver for local PCIe NVMe SSDs
> - FPGA accelerator implementing NVMe IO queue memory and IO queue handling,
>   exposed via PCIe BAR
> - vfio-pcie Kernel driver plus vfio userspace FPGA driver / application
> - The userspace application creates new NVMe IO queues at the SSD using the
>   patched admin ioctl and points them towards the FPGA BAR. It then informs
>   the FPGA about the SSD BAR address and IO queue ID. From then on the FPGA
>   can access the SSD storage entirely without software interaction.

Hi David,

I would like to ask for your insight on one point about the FPGA
queue-offload setup described in the RFC. This is not about the PRP1
ioctl change itself; I am personally interested in FPGA/NVMe datapath
offload and would like to better understand how your setup handled this.

For the I/O queues handled by the FPGA, how does the FPGA learn that the
SSD has posted new CQEs?

Did your implementation disable interrupts for those CQs and let the
FPGA poll the CQ phase tag, or did you use MSI/MSI-X with the
corresponding NVMe MSI-X vector targeting an FPGA BAR event register
instead of the host interrupt controller?

I also wonder how the I/O work was submitted to the FPGA in this model.
Does the CPU still provide the FPGA with per-I/O information such as the
data buffer address and the NVMe namespace/LBA range, while the FPGA
then builds and submits the NVMe commands? Or is the FPGA able to derive
most of that by itself after the initial queue setup?

Thanks,
Guzebing