[PATCH 1/2] nvme: support fused nvme requests

[Clay Mayers]

Thanks for the very fast feedback.  I will certainly reduce the memory
hit for non-fused cmds and look into dealing with a controller reset.

On Tue, Jan 5, 2021 at 3:52 PM Keith Busch <kbusch at kernel.org> wrote:
>
> On Tue, Jan 05, 2021 at 02:49:38PM -0800, klayph at gmail.com wrote:
> > @@ -157,6 +157,8 @@ struct nvme_request {
> >       u8                      flags;
> >       u16                     status;
> >       struct nvme_ctrl        *ctrl;
> > +     struct nvme_request     *nrq2;  /* Points to second fused request */
> > +     struct nvme_command     cmnd;   /* Saved fused first cmnd */
>
> This struct is getting a bit large, and we allocate 1000's of them. An
> extra 72B (3x its current size) for something that likely never gets
> used (and can't be used for fabrics) seems a bit much.
>
> > @@ -918,7 +942,13 @@ static blk_status_t nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
> >       }
> >
> >       blk_mq_start_request(req);
> > -     nvme_submit_cmd(nvmeq, &cmnd, bd->last);
> > +
> > +     if (cmnd.common.flags & NVME_CMD_FUSE_FIRST)
> > +             memcpy(&nvme_req(req)->nrq2->cmnd, &cmnd, sizeof(cmnd));
> > +     else if (cmnd.common.flags & NVME_CMD_FUSE_SECOND)
> > +             nvme_submit_cmd2(nvmeq, &nvme_req(req)->cmnd, &cmnd, bd->last);
> > +     else
> > +             nvme_submit_cmd(nvmeq, &cmnd, bd->last);
>
> There's an unlikely scenario where this breaks down, but it'd look
> something like this: the FUSE_FIRST is submitted to the driver, then
> before the FUSE_SECOND reaches the driver, the controller is reset.
> Because you're only using this through the passthrough interface where
> the FAILFAST flag is set, the FUSE_FIRST command is ended in failure,
> which releases the command ID for reuse. Once the controller is
> restarted, the FUSE_SECOND is dispatched to the driver using an invalid
> command ID stored in its 'cmnd' field.