Skip to content

ken-yossy/nvmetool-win

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

nvmetool-win: Sample program of accessing NVMe device using Windows' inbox NVMe driver

build lint

Abstract

This software demonstrates that issuing some NVMe commands from userland to NVMe device using Windows inbox NVMe driver (stornvme.sys)[1].

Specification that this software refered to is NVMe 1.3d[2] and NVMe 1.4b[6].

Some fields introduced in NVMe Base Specification revision 2.0[10] are supported.

For StorNVMe on Windows 10 version 1903 and later, Supported NVMe Admin and NVM command sets and associated opcodes are shown in [8], and supported features are shown in [9].

Note for this software

This software is a SAMPLE and DEMONSTRATION program to show how to access NVMe drive with Windows' inbox NVMe driver.

It is NOT intended to provide versatile tool with such functions like accepting arbitrary value for parameters, file input / output, non-interactive mode, support for vendor specific commands.

If you need such functions, you can create your own tool based on this software.

Thank you.

Command matrix

Table 1. Command matrix (Admin Command Set)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

Opcode Command name Status Note
00h Delete I/O Submission Queue 1
01h Create I/O Submission Queue 1
02h Get Log Page See Table 4
04h Delete I/O Completion Queue 1
05h Create I/O Completion Queue 1
06h Identify See Table 3
08h Abort 1
09h Set Features See Table 5
0Ah Get Features See Table 5
0Ch Asynchronous Event Request 1
0Dh Namespace Management 2
10h Firmware Commit
11h Firmware Image Download
14h Device Self-test x
15h Namespace Attachment 2
18h Keep Alive Not listed in [8]
19h Directive Send 1
1Ah Directive Receive 1
1Ch Virtualization Management Not supported
1Dh NVMe-MI Send 2
1Eh NVMe-MI Receive 2
20h Capacity Management from 2.0
24h Lockdown from 2.0
7Ch Doorbell Buffer Config Not supported
80h Format NVM x Only for data drive. Need to support Format NVM command and cryptographic erase is supported along Format NVM (bit 2 of the FNA field in Identify Controller data structure)
81h Security Send
82h Security Receive x Getting TCG Level 0 Discovery Data for example in this tool
84h Sanitize 2
86h Get LBA Status Not supported
C0h -- FFh Vendor Specific Sample (Template) code exists

Note 1: probably cannot issue this command because only "Internal Driver Use" is written in [8]

Note 2: can issue this command only in WinPE mode according to [8]

Note 3: To use sample (template) code for issuing Vendor Specific Commands, you need to edit the codes as you want to do

Table 2. Command matrix (NVM Command Set)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

Opcode Command name Status Note
00h Flush x
01h Write x
02h Read x
04h Write Uncorrectable 1
05h Compare Can issue only in WinPE mode [8]
08h Write Zeroes 1
09h Dataset Management x Only Deallocate is supported
0Ch Verify 1
0Dh Reservation Register 1
0Eh Reservation Report 1
11h Reservation Acquire 1
15h Reservation Release 1
19h Copy from 2.0
80h -- FFh Vendor Specific Sample (Template) code exists

Note 1: these command are stated as "Not supported" in [8]

Note 2: To use sample (template) code for issuing Vendor Specific Commands, you need to edit the codes as you want to do

Controller or Namespace Structure (CNS) for Identify command

CNS values from 10h to 13h are seemed to be able to issue only in WinPE mode, because these values are mandatory for controllers that support the Namespace Management capability in NVMe spec., but Namespace Management command can only be issued in WinPE mode according to [8].

CNS values 14h and 15h are seemed to be unable to issue, because these values are mandatory for controllers that support Virtualization Enhancements in NVMe spec., but Virtualization Management command is not supported according to [8].

Table 3. Controller or Namespace Structure (CNS)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

CNS Description Status Note
00h Namespace x NSID = 1
01h Controller x CNTID = 0
02h Active Namespace ID list we can't specify NSID... and seems no data is returned...
03h Namespace Identification Descriptor list x
04h NVM Set list
05h I/O Command Set specific Identify Namespace data structure x NSID = 1
06h I/O Command Set specific Identify Controller data structure CNTID = 0
07h Active Namespace ID list associated with the specified I/O Command Set
08h I/O Command Set Independent Identify Namespace data structure NSID = 1
09h -- 0Fh Reserved
10h Allocated Namespace ID list Probably can issue only in WinPE mode [8]
11h Specified allocated namespace Probably can issue only in WinPE mode [8]
12h List of controllers attached to specified namespace Probably can issue only in WinPE mode [8]
13h List of controllers exist in the NVM subsystem Probably can issue only in WinPE mode [8]
14h Primary Controller Capabilities Probably cannot issue [8]
15h Secondary controller list Probably cannot issue [8]
16h Namespace Granularity List
17h UUID List
18h Domain List
19h Endurance Group List
1Ah I/O Command Set specific Allocated Namespace ID list
1Bh I/O Command Set specific Identify Namespace data structure
1Ch I/O Command Set data structure
1Dh -- 1Fh Reserved Spec says CNS "18h to 1Fh" are reserved, I think it's typo.
20h -- FFh Reserved

Log Identifier (LID) for Get Log Page command

Table 4. Log Identifier (LID)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

LID Description Status Note
00h Supported Log Pages -- error code = 1117 (The request could not be performed because of an I/O device error.)
01h Error Information x Number of entries to be retrieved from devices depends on the value "ELPE" in Identify Controller data
02h SMART / Health Information x CNTID = 0, NSID = 0xFFFFFFFh
03h Firmware Slot Information x
04h Changed Namespace List
05h Command Supported and Effects x
06h Device Self-test x
07h Telemetry Host-initiated x Getting header only. see also Note 1.
08h Telemetry Controller-initiated x Getting header only.
09h Endurance Group Information x
0Ah Predictable Latency Per NVM Set
0Bh Predictable Latency Event Aggregate
0Ch Asymmetric Namespace Access
0Dh Persistent Event Log
0Eh LBA Status Information
0Fh Endurance Group Event Aggregate -- error code = 1117 (The request could not be performed because of an I/O device error.)
10h Media Unit Status
11h Supported Capacity Configuration List
12h Feature Identifiers Supported and Effects
13h NVMe-MI Commands Supported and Effects
14h Command and Feature Lockdown
15h Boot Partition
16h Rotational Media Information
17h -- 6Fh Reserved
70h Discovery For NVMe over Fabrics (NVMeoF)
71h -- 7Fh Reserved for NVMeoF
80h Reservation Notification Probably cannot issue [8]
81h Sanitize Status Probably can issue only in WinPE mode [8]
82h -- BFh Reserved
C0h -- FFh Vendor Specific

Note 1: If you want to set Create Telemetry Host-Initiated Data to 1, you need Windows 10 version 2004 (build 10041) or later. And to do this, this tool uses IOCTL_STORAGE_GET_DEVICE_INTERNAL_LOG described in [9]. But I don't know products (SSDs) that can respond this type of request (command)...

Feature Identifier (FID) for Get/Set Features command

According to [8], only Host Controlled Thermal Management is supported via Set Features command with IOCTL_STORAGE_SET_PROPERTY.

Table 5. Feature Identifier (FID)

Legend for "Status" column: "x" => can issue, "--" => cannot issue, " "(blank) => not confirmed

FID Description Status (Get/Set) Note
01h Arbitration x /
02h Power Management x /
03h LBA Range Type -- / error code = 1117 (The request could not be performed because of an I/O device error.)
04h Temperature Threashold x /
05h Error Recovery x /
06h Volatile Write Cache x / -- error code = 1117 (The request could not be performed because of an I/O device error.)
07h Number of Queues -- / error code = 1117 (The request could not be performed because of an I/O device error.)
08h Interrupt Coalescing x /
09h Interrupt Vector Configuration /
0Ah Write Atomicity Normal x /
0Bh Asynchronous Event Configuration x /
0Ch Autonomous Power State Transition -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Dh Host Memory Buffer -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Eh Timestamp -- / error code = 1117 (The request could not be performed because of an I/O device error.)
0Fh Keep Alive Timer /
10h Host Controlled Thermal Management x / x "SV = 1" is not supported
11h Non-operational Power State Config x /
12h Read Recovery Level Config /
13h Predictable Latency Mode Config /
14h Predictable Latency Mode Window /
15h LBA Status Information Report Interval /
16h Host Behavior Support /
17h Sanitize Config /
18h Endurance Group Event Configuration /
19h I/O Command Set Profile /
1Ah Spinup Control /
1Bh -- 77h Reserved /
78h -- 7Ch Refer to the NVMe-MI spec. /
7Dh Enhanced Controller Metadata /
7Eh Controller Metadata /
7Fh Namespace Metadata /
80h Software Progress Marker x /
81h Host Identifier /
82h Reservation Notification Mask / 1
83h Reservation Persistence / 1
84h Namespace Write Protect Config /
85h -- BFh Reserved /
C0h -- FFh Vendor specific /

Note 1: FID 82h and 83h are seemed to be unable to issue, because commands related to Reservation are not supported according to [8].

Note for some commands

Read command

Read command is issued via SCSI Pass-through[3].

Issue Read command with Starting Logical Block Address (SLBA) = 0 and Number of Logical Blocks (NLB) = 1.

Write command

Write command is issued via SCSI Pass-through[3].

Issue Write command with SLBA = 0, NLB = 1, and data is "DEADBEEF".

Dataset Management command

Dataset Management command is issued via SCSI Pass-through[3] with IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES [4][5] (only Deallocate bit is set).

LBA region to be deallocated is SLBA = 0 and NLB = 1 (Sector 0).

An NVMe drive that supports Dataset Management command is also needed.

Supporting Dataset Management command can be checked by bit 2 of "Optional NVM Command Support (ONCS)" field in Identify Controller Data Structure.

If the bit is set to 1, the controller support Dataset Management command.

For further information, see "5.15.3 Identify Controller data structure (CNS 01h)" in the NVMe specification[2].

An NVMe drive that return fixed data pattern for reading deallocated logical block is recommended for checking the result of deallocation.

It can be checked by bit 2:0 of "Deallocate Logical Block Features (DLFEAT)" field in Identify Namespace Data Structure.

If it is 001b, the drive returns all bytes cleared to 00h for reading deallocated logical block.

If it is 010b, the drive returns all bytes set to FFh for reading deallocated logical block.

For further information, see "5.15.2 Identify Namespace data structure (CNS 00h)" in the NVMe specification[2].

Caution

Issuing "Write", "Dataset Management (Deallocate)", and "Format NVM" command to devices under management by file systems (e.g. FAT, NTFS) will corrupt your system.

These commands should be issued with highest care.

"Read" command does not have such a side effect... :-)

Environment

Confirmed on the following software environment:

  • Tested operating system and device driver (latest)
    • Windows 11 Pro 64bit (Version 22H2, Build 22621.963)
      • stornvme.sys (version 10.0.22621.755, WinBuild 160101.0800)
  • Developing environment
    • Microsoft Visual Studio Community 2022 (Version 17.4.3)
      • Windows SDK 10.0.22621.0
      • Windows Driver Kit 10.0.22605.1

Limitations

Only tested with the NVMe drive directly attached to PC via PCIe.

It is ok to access to M.2 drives and M.2 drives that attached to M.2-PCIe converter Add-In-Card (AIC).

Also, it may be ok with U.2 drives.

But it may not work over protocol translations such as usb-nvme.

To build and run program

  1. Double click NVMeTool.sln to start Visual Studio
  2. Build program with Build -> Build Solution (or Control+Shift+B)
  3. Run program (F5)

Note that, when you run (debug) in Visual Studio, confirm that the command line argument is set to appropriate drive ID for your system.

You can find the settings of command line arguments from Debug menu -> NVMeTool Debug Properties -> Configuration Properties -> Debugging -> Command Arguments.

Future Works

  • Test commands, LIDs, FIDs, and CNSs in the NVMe specification and listed in [8]
  • Test in WinPE mode

License

This software is released under the MIT License and Microsoft Public License (MS-PL), see LICENSE.

Reference

[1] Microsoft, "Working with NVMe drives", https://docs.microsoft.com/en-us/windows/desktop/fileio/working-with-nvme-devices, Retrieved May 14, 2019

[2] NVM Express, "NVM Express[TM] Base Specification", Revision 1.3d, March 20, 2019

[3] Microsoft, "SCSI Pass-Through Interface Tool", https://github.com/microsoft/Windows-driver-samples/tree/master/storage/tools/spti, Retrieved May 14, 2019.

[4] Mark E. Russinovich, et al., "Windows Internals, Part 2", 6th Edition, October 2012, Microsoft Press, ISBN: 978-0735665873

[5] Microsoft, "IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES IOCTL", https://docs.microsoft.com/ja-jp/windows/desktop/api/winioctl/ni-winioctl-ioctl_storage_manage_data_set_attributes, Retrieved May 14, 2019

[6] NVM Express, "NVM Express[TM] Base Specification", Revision 1.4b, Sept. 21, 2020

[7] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/nvme-features-supported-by-stornvme, Retrieved June 25, 2020

[8] Microsoft, "StorNVMe Command Set Support", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-command-set-support, Retrieved June 25, 2020

[9] Microsoft, "NVMe Features Supported by StorNVMe", https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-feature-support, Retrieved June 29, 2020

[10] NVM Express, "NVM Express[TM] Base Specification", Revision 2.0, June 2, 2021

About

Communicate with NVMe SSD using Windows' inbox device driver

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages