[PATCH blktests] nvme: test log page offsets

[Keith Busch]

On Wed, Feb 07, 2024 at 04:32:02AM +0000, Shinichiro Kawasaki wrote:
> Thanks for the explanations. The issue sounds nasty. IIUC, the motivation to
> add this test case is to catch the device issue early and to not lose
> developer's precious time to debug it again.
> 
> Having said that, this test case will be the first test case to test *devices*,
> and it will extend the role of blktests. So far, all blktests test cases are
> intended to test kernel/driver *code*.
> 
> With this background, I have two questions in my mind:
> 
> Q1) What is the expected action to take when blktests users observe failure of
>     the test case? Report to linux-block or linux-nvme? Or report to the
>     device manufacturer?
> 
> Q2) When a new nvme driver patch needs check before submit, is this test case
>     worth running?
> 
> I guess the answer of Q1 is 'Report to the device manufacturer' and the answer
> of Q2 is 'No'. It would be the better to clarify these points with some more
> descriptions in the comment block of the test case.
> 
> Another idea is to separate the test case from the nvme group to a new group
> called "nvmedev" or "devcompat", which is dedicated to check that the storage
> devices work good with Linux storage stack. It will make it easier for the
> blktests users to understand the action to take, and which test group to run.
> What do you think?

Splitting out device specific sanity tests sounds fine to me. There are
various suites for interop and spec compliance, but I don't know of any
good open source repos providing that.

And instead of end-users running the tests, I imagine vendors could
include a test run ahead of a product release or drive qualification.
Though to be honest, I think this patch is a bit silly to start off that
kind of ambitious project. I'll check with other potential parties to
see if there's interest in collaborating on something more complete.
I'm not sure how well existing device tests could fit in the blktests
framework, but maybe it could work.