From jack at mudshark.org  Thu Nov 10 13:34:22 2011
From: jack at mudshark.org (Jack Cummings)
Date: Thu, 10 Nov 2011 10:34:22 -0800
Subject: NVMe prototyping hardware?
Message-ID: <20111110183422.GD54976@ice9.mudshark.org>


Hello, 

does anyone know of any NVMe prototyping hardware to develop software against?

--Jack 

--
Jack (John) Cummings                           http://mudshark.org/
PGP fingerprint: A927 992E 197B 0A86 27BC  964D A1DA CBAC CC48 B110
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://merlin.infradead.org/pipermail/linux-nvme/attachments/20111110/09c5de2b/attachment.sig>

From Kwok.Kong at idt.com  Wed Nov 23 13:17:19 2011
From: Kwok.Kong at idt.com (Kong, Kwok)
Date: Wed, 23 Nov 2011 10:17:19 -0800
Subject: submission queue and completion queue allocation in NVMe
Message-ID: <5AC4D6A45304E048A30F105EB302760902CBB066@CORPEXCH1.na.ads.idt.com>

Matthew,

In the current linux driver, you send a Get Feature with Feature ID = 7
(Number of Queues) to allocate submission and completion queues from the
controller.

My understanding is that you should use Set Feature with Feature ID = 7
to allocate submission and completion queues from the controller.  The
Get Feature with ID = 7 returns the number of queues that have been
allocated.

If you send a Get Feature with ID =7 without first sending a Set Feature
with ID =7, the Get feature should return 0 in Dword 0 as no queues have
been allocated.

Do you agree ?

Thanks

-kwok 


From Kwok.Kong at idt.com  Wed Nov 23 13:53:37 2011
From: Kwok.Kong at idt.com (Kong, Kwok)
Date: Wed, 23 Nov 2011 10:53:37 -0800
Subject: LBA Range handling 
Message-ID: <5AC4D6A45304E048A30F105EB302760902CBB0AF@CORPEXCH1.na.ads.idt.com>

Matthew,

One of the purpose of the BA Range Type is to allow a driver not to
"export" a LBA range that the driver does not understand. i.e. if a SSD
is used in a RAID system previously,  the LBA range is configured to be
"RAID".  When a non-RAID driver is loaded by mistake, the non-RAID
driver should not export this LBA Range to the OS and hence the RAID
data cannot be overwritten unexpectedly.

When a SSD is brand new, the LBA range type is going to be "Reserved" as
it has never been used.

In the current linux driver, only the hidden attributes is checked.
Should more checking be done ?

In the Windows driver, we (IDT, Intel and Sandforace) have decided to
use the following algorithms to handle the LBA Range:

- Retrieve LBA Range entries via Get Features (ID#3) for each one of the
namespaces;
- Exam the first LBA Range entry of a given namespace;

if (NLB == Namespace Size)
{
    if (Type == 00b)
    {
	  // This is a brand new disk for this driver	
        Issue a Set Features (ID#3) command:
		- Type = 1 (Filesystem)
		- Attributes = 1 (can be overwritten, visible)
		- Starting LBA = 0
		- Number of Logical Blocks = size of the Name Space
        Export NameSpace to the OS
    }
    else if (Type == 01b)
    {
        If (Attributes_bit_1 == 0)
            Do not "export" NS to the OS
        else
        {
            if (Attributes_bit_0 == 0)
               "export" NS to the OS as "Read Only"
            else
               "export" NS to the OS
        }
    }
    else
    {
        Do not "export" the NS to the OS;
    }
}
else
{
    Do not "export" the NS to the OS;
}


Do you see any problem with this algorithms ?

If not, then should the linux driver match the Windows driver algorithm
?

Thanks

-Kwok


From willy at linux.intel.com  Wed Nov 23 15:50:52 2011
From: willy at linux.intel.com (Matthew Wilcox)
Date: Wed, 23 Nov 2011 15:50:52 -0500
Subject: submission queue and completion queue allocation in NVMe
In-Reply-To: <5AC4D6A45304E048A30F105EB302760902CBB066@CORPEXCH1.na.ads.idt.com>
References: <5AC4D6A45304E048A30F105EB302760902CBB066@CORPEXCH1.na.ads.idt.com>
Message-ID: <20111123205052.GA14291@linux.intel.com>

On Wed, Nov 23, 2011 at 10:17:19AM -0800, Kong, Kwok wrote:
> Matthew,
> 
> In the current linux driver, you send a Get Feature with Feature ID = 7
> (Number of Queues) to allocate submission and completion queues from the
> controller.
> 
> My understanding is that you should use Set Feature with Feature ID = 7
> to allocate submission and completion queues from the controller.  The
> Get Feature with ID = 7 returns the number of queues that have been
> allocated.
> 
> If you send a Get Feature with ID =7 without first sending a Set Feature
> with ID =7, the Get feature should return 0 in Dword 0 as no queues have
> been allocated.
> 
> Do you agree ?

That wasn't my understanding ... but 7.6.1 says I was incorrect.

If I just change the call to be nvme_admin_set_features instead of
get_features, that's sufficient, right?  I don't need to issue a Set
Features and then a Get Features to find out what was set.


From Kwok.Kong at idt.com  Wed Nov 23 16:21:25 2011
From: Kwok.Kong at idt.com (Kong, Kwok)
Date: Wed, 23 Nov 2011 13:21:25 -0800
Subject: submission queue and completion queue allocation in NVMe
In-Reply-To: <20111123205052.GA14291@linux.intel.com>
References: <5AC4D6A45304E048A30F105EB302760902CBB066@CORPEXCH1.na.ads.idt.com>
 <20111123205052.GA14291@linux.intel.com>
Message-ID: <5AC4D6A45304E048A30F105EB302760902CBB193@CORPEXCH1.na.ads.idt.com>

Agreed. All you need to do is to change the call to be
nvme_admin_set_features instead of get feature.
Set features returns the number of queues that have been allocated. You
don't need to call get feature to find out what was set.

-Kwok


-----Original Message-----
From: Matthew Wilcox [mailto:willy at linux.intel.com] 
Sent: Wednesday, November 23, 2011 12:51 PM
To: Kong, Kwok
Cc: linux-nvme at lists.infradead.org
Subject: Re: submission queue and completion queue allocation in NVMe

On Wed, Nov 23, 2011 at 10:17:19AM -0800, Kong, Kwok wrote:
> Matthew,
> 
> In the current linux driver, you send a Get Feature with Feature ID = 
> 7 (Number of Queues) to allocate submission and completion queues from

> the controller.
> 
> My understanding is that you should use Set Feature with Feature ID = 
> 7 to allocate submission and completion queues from the controller.  
> The Get Feature with ID = 7 returns the number of queues that have 
> been allocated.
> 
> If you send a Get Feature with ID =7 without first sending a Set 
> Feature with ID =7, the Get feature should return 0 in Dword 0 as no 
> queues have been allocated.
> 
> Do you agree ?

That wasn't my understanding ... but 7.6.1 says I was incorrect.

If I just change the call to be nvme_admin_set_features instead of
get_features, that's sufficient, right?  I don't need to issue a Set
Features and then a Get Features to find out what was set.


From willy at linux.intel.com  Wed Nov 23 16:34:47 2011
From: willy at linux.intel.com (Matthew Wilcox)
Date: Wed, 23 Nov 2011 16:34:47 -0500
Subject: LBA Range handling
In-Reply-To: <5AC4D6A45304E048A30F105EB302760902CBB0AF@CORPEXCH1.na.ads.idt.com>
References: <5AC4D6A45304E048A30F105EB302760902CBB0AF@CORPEXCH1.na.ads.idt.com>
Message-ID: <20111123213447.GB14291@linux.intel.com>

On Wed, Nov 23, 2011 at 10:53:37AM -0800, Kong, Kwok wrote:
> One of the purpose of the BA Range Type is to allow a driver not to
> "export" a LBA range that the driver does not understand. i.e. if a SSD
> is used in a RAID system previously,  the LBA range is configured to be
> "RAID".  When a non-RAID driver is loaded by mistake, the non-RAID
> driver should not export this LBA Range to the OS and hence the RAID
> data cannot be overwritten unexpectedly.
> 
> When a SSD is brand new, the LBA range type is going to be "Reserved" as
> it has never been used.

umm ... it is?  We should probably get the committee to change that
type 0 from Reserved (which means you can't use it) to 'Unspecified'
or something ...

> In the current linux driver, only the hidden attributes is checked.
> Should more checking be done ?
> 
> In the Windows driver, we (IDT, Intel and Sandforace) have decided to
> use the following algorithms to handle the LBA Range:
> 
> - Retrieve LBA Range entries via Get Features (ID#3) for each one of the
> namespaces;
> - Exam the first LBA Range entry of a given namespace;
> 
> if (NLB == Namespace Size)
> {
>     if (Type == 00b)
>     {
> 	  // This is a brand new disk for this driver	
>         Issue a Set Features (ID#3) command:
> 		- Type = 1 (Filesystem)
> 		- Attributes = 1 (can be overwritten, visible)
> 		- Starting LBA = 0
> 		- Number of Logical Blocks = size of the Name Space
>         Export NameSpace to the OS
>     }
>     else if (Type == 01b)
>     {
>         If (Attributes_bit_1 == 0)
>             Do not "export" NS to the OS
>         else
>         {
>             if (Attributes_bit_0 == 0)
>                "export" NS to the OS as "Read Only"
>             else
>                "export" NS to the OS
>         }
>     }
>     else
>     {
>         Do not "export" the NS to the OS;
>     }
> }
> else
> {
>     Do not "export" the NS to the OS;
> }
> 
> 
> Do you see any problem with this algorithms ?

Yes.  If the type of the LBA range is 'page cache', for example, then
it needs to be exported to the OS in order for the OS to use it.  Also,
I think you have the sense of the hidden bit wrong.  The only change
I'd make to my current algorithm is that we should probably obey the
read-only bit.

Also, I don't know how to handle multiple LBA range types yet ... they
ought to be handled through the Linux partition mechanism, but it doesn't
have any concept of devices providing their own partitioning scheme yet.

> If not, then should the linux driver match the Windows driver algorithm
> ?
> 
> Thanks
> 
> -Kwok
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://merlin.infradead.org/mailman/listinfo/linux-nvme


From Kwok.Kong at idt.com  Wed Nov 23 16:47:20 2011
From: Kwok.Kong at idt.com (Kong, Kwok)
Date: Wed, 23 Nov 2011 13:47:20 -0800
Subject: LBA Range handling
In-Reply-To: <20111123213447.GB14291@linux.intel.com>
References: <5AC4D6A45304E048A30F105EB302760902CBB0AF@CORPEXCH1.na.ads.idt.com>
 <20111123213447.GB14291@linux.intel.com>
Message-ID: <5AC4D6A45304E048A30F105EB302760902CBB1D2@CORPEXCH1.na.ads.idt.com>

>> When a SSD is brand new, the LBA range type is going to be "Reserved"

>> as it has never been used.
>
>umm ... it is?  We should probably get the committee to change that
type 0 from Reserved (which means you can't use >it) to 'Unspecified'
>or something ...

I agree.  


>> Do you see any problem with this algorithms ?
>
>Yes.  If the type of the LBA range is 'page cache', for example, then
it needs to be exported to the OS in order for >the OS to use it.  Also,
I think you have the sense of the hidden bit wrong.  The only change I'd
make to my current >algorithm is that we should probably obey the
read-only bit.
>
>Also, I don't know how to handle multiple LBA range types yet ... they
ought to be handled through the Linux partition >mechanism, but it
doesn't have any concept of devices providing their own partitioning
scheme yet.

Actually, I am not sure how do use the LBA range.  Who should set the
LBA Range ? Should it be the driver or the application ?  What should
the driver behave if the type is 'page cache', 'RAID', 'Reserved' or
'FILESYSTEM' ?

How are we supposed to use the LBA range ?

Thanks

-Kwok




From willy at linux.intel.com  Thu Nov 24 13:06:31 2011
From: willy at linux.intel.com (Matthew Wilcox)
Date: Thu, 24 Nov 2011 13:06:31 -0500
Subject: LBA Range handling
In-Reply-To: <5AC4D6A45304E048A30F105EB302760902CBB1D2@CORPEXCH1.na.ads.idt.com>
References: <5AC4D6A45304E048A30F105EB302760902CBB0AF@CORPEXCH1.na.ads.idt.com>
 <20111123213447.GB14291@linux.intel.com>
 <5AC4D6A45304E048A30F105EB302760902CBB1D2@CORPEXCH1.na.ads.idt.com>
Message-ID: <20111124180631.GD14291@linux.intel.com>

On Wed, Nov 23, 2011 at 01:47:20PM -0800, Kong, Kwok wrote:
> Actually, I am not sure how do use the LBA range.  Who should set the
> LBA Range ? Should it be the driver or the application ?  What should
> the driver behave if the type is 'page cache', 'RAID', 'Reserved' or
> 'FILESYSTEM' ?
> 
> How are we supposed to use the LBA range ?

My understanding is that there will be a user-space application (like
fdisk) that will write the LBA Range Type attribute.  The 'type' is
mostly for the benefit of this application and shouldn't be interpreted
by the driver.


