[PATCH] NVMe: Use CMB for the SQ if available

[Sam Bradshaw (sbradshaw)]

> @@ -376,7 +394,12 @@ static int __nvme_submit_cmd(struct nvme_queue
> *nvmeq, struct nvme_command *cmd)  {
>  	u16 tail = nvmeq->sq_tail;
> 
> -	memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
> +	if (nvmeq->cmb_mapped)
> +		memcpy_toio(&nvmeq->sq_cmds[tail], cmd,
> +				sizeof(*cmd));
> +	else
> +		memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
> +
>  	if (++tail == nvmeq->q_depth)
>  		tail = 0;
>  	writel(tail, nvmeq->q_db);

I think a store fence is necessary between memcpy_toio() and the doorbell ring.
This applies elsewhere in the patch as well.

For example, we've seen rare cases where Haswells do not emit the whole SQE out 
of the write combine buffers before the doorbell write traverses PCIe.  Other 
architectures may have a similar need.