Make NVME shutdown two-pass - Version 3

[Jeremy Allison]

This is version 3 of a patchset originally written by
Tanjore Suresh <tansuresh at google.com> to make shutdown
of nvme devices two-pass.

Changes from version 2:

1). As requested by Christoph Hellwig, don't add shutdown_pre()
shutdown_post() methods, only add a shutdown_wait() method
to prevent clutter in the interfaces. Two pass shutdown
is only done if both shutdown() and shutdown_wait() methods
are defined.

2). As requested by Christoph Hellwig, split out the addition
of the 'enum shutdown_type' into a separate patch to make the
subsequent changes clearer.

3). Use 'two-pass' instead of async in the interface
documentation. This isn't truly async as there's no
notification on completion.

-------------------------------------------------------------
Currently the Linux nvme driver shutdown code steps
through each connected drive, sets the NVME_CC_SHN_NORMAL
(normal shutdown) flag and then polls the given drive
waiting for the response NVME_CSTS_SHST_CMPLT flag
(shutdown complete).

Each drive is taking around 13 seconds to respond to this.

The customer has 20+ drives on the box so this time adds
up on shutdown when the nvme driver is being shut down.

This patchset changes shutdown to proceed in parallel,
so the NVME_CC_SHN_NORMAL (normal shutdown) flag is
sent to all drives first, and then it polls waiting
for the NVME_CSTS_SHST_CMPLT flag (shutdown complete)
for all drives.

In the specific customer case it reduces the NVME
shutdown time from over 300 seconds to around 15
seconds.
-------------------------------------------------------------

Thanks for your consideration,

Jeremy Allison.
CIQ / Samba Team.