pynvme 1.0

test NVMe devices in Python.

pynvme

test NVMe devices in Python. [https://github.com/cranechu/pynvme]

• Install
• VSCode
• Tutorial
• Files
• Fixtures
• Debug
• Classes
  • Pcie
  • Controller
  • Namespace
  • Qpair
  • Buffer
  • IOWorker
  • Subsystem

The pynvme is a python extension module. Users can operate NVMe SSD intuitively in Python scripts. It is designed for NVMe SSD testing with performance considered. Integrated with third-party tools, vscode and pytest, pynvme provides a convenient and professional solution to test NVMe devices.

The pynvme wraps SPDK NVMe driver in a Python extension, with abstracted classes, e.g. Controller, Namespace, Qpair, Buffer, and IOWorker. With pynvme, we can send any NVMe command in Python scripts, with a list of other capabilites:

1. access PCI configuration space
2. access NVMe registers in BAR space
3. send any NVMe admin/IO commands
4. callback functions are supported
5. MSIx interrupt is supported
6. transparent checksum verification
7. IOWorker generates high-performance IO
8. integrated with pytest
9. integrated with VSCode

Before moving forward, check and backup your data in the NVMe SSD to be tested. It is always recommended to attach just one piece of NVMe SSD in your system to avoid mistakes.

Install

Users can install and use pynvme in commodity computers.

System Requirement

1. CPU: x86_64
2. OS: Linux, recommend Fedora 29, Ubuntu is also tested
3. Memory: 8GB or larger, 2MB hugepage size.
4. SATA: install OS and pynvme in a SATA drive.
5. NVMe: NVMe SSD is the device to be tested. Backup your data!
6. Python3. Python2 is not supported.
7. sudo privilege is required.
8. RAID mode in BIOS (Intel® RST) should be disabled.

Source Code

Fetch pynvme source code from GitHub, and simply run install.sh to build pynvme. install.sh generates the pynvme python package nvme.cpython-37m-x86_64-linux-gnu.so.

git clone https://github.com/cranechu/pynvme
cd pynvme
./install.sh

Now, it is ready to start vscode. Or, you can continue to refer to detailed installation instructions below.

Prerequisites

First, to fetch all required dependencies source code and packages.

git submodule update --init --recursive
sudo dnf install python3-pip -y # Ubuntu: sudo apt-get install python3-pip

Build

Compile the SPDK, and then pynvme.

cd spdk; ./configure --without-isal; cd ..   # configurate SPDK
make spdk                                    # compile SPDK
make                                         # compile pynvme

Now, you can find the generated binary file like: nvme.cpython-37m-x86_64-linux-gnu.so

Test

Setup SPDK runtime environment to remove kernel NVMe driver and enable SPDK NVMe driver. Now, we can run tests!

# backup your data in NVMe SSD before testing
make setup
make test
make test TESTS=scripts
make test TESTS=scripts/demo_test.py
make test TESTS=scripts/utility_test.py::test_download_firmware

By default, it runs tests in driver_test.py. However, these are tests of pynvme itself, instead of SSD drives. Your DUT drive may fail in some test cases. Please add your tests in scripts directory. Test logs are saved in file test.log.

After test, you may wish to bring kernel NVMe driver back like this:

make reset

User can find pynvme documents in README.md, or use help() in python:

sudo python3 -c "import nvme; help(nvme)"  # press q to quit

VSCode

The pynvme works with VSCode! And pytest too!

1. First of all, install vscode here: https://code.visualstudio.com/

2. Root user is not recommended in vscode, so just use your ordinary non-root user. It is required to configurate the user account to run sudo without a password.

sudo visudo

3. In order to monitor qpairs status and cmdlog along the progress of testing, user can install vscode extension pynvme-console. The extension provides DUT status and cmdlogs in VSCode UI.

code --install-extension pynvme-console-1.0.0.vsix

4. Before start vscode, modify .vscode/settings.json with the correct pcie address (bus:device.function, which can be found by lspci shell command) of your DUT device.

lspci
# 01:00.0 Non-Volatile memory controller: Lite-On Technology Corporation Device 2300 (rev 01)

5. Then in pynvme folder, we can start vscode to edit, debug and run scripts:

make setup; code .  # make sure to enable SPDK nvme driver before starting vscode

6. Now, it is ready to create, debug and run test scripts on your NVMe SSD. Import following packages in new test script files.

import pytest
import logging

import nvme as d    # import pynvme's python package

VSCode is convenient and powerful, but it consumes a lot of resources. So, for formal performance test, it is recommended to run tests in command line, by make test.

Tutorial

After installation, pynvme generates the binary extension which can be import-ed in python scripts. Example:

import nvme as d

nvme0 = d.Controller(b"01:00.0")  # initialize NVMe controller with its PCIe BDF address
id_buf = d.Buffer(4096)  # allocate the buffer
nvme0.identify(id_buf, nsid=0xffffffff, cns=1)  # read namespace identify data into buffer
nvme0.waitdone()  # nvme commands are executed asynchronously, so we have to wait the completion before access the id_buf.
print(id_buf.dump())   # print the whole buffer

In order to write test scripts more efficently, pynvme provides pytest fixtures. We can write more in intuitive test scripts. Example

import pytest
import nvme as d

def test_dump_namespace_identify_data(nvme0):
    id_buf = d.Buffer()
    nvme0.identify(id_buf, nsid=0xffff_ffff, cns=1).waitdone()
    print(id_buf.dump())

The pytest can collect and execute these test scripts in both command line and IDE (e.g. VSCode). Example:

sudo python3 -m pytest test_file_name.py::test_function_name --pciaddr=BB:DD.FF  # find the BDF address by lspci

By default, pytest captures all outputs, and only test results are printed. By adding the option "-s" in the above command line, pytest will also print scripts and pynvme's messages.Please refer to pytest documents for more instructions.

To make the simplisity a step further, pynvme provides more python facilities. If the optional type hint is given to the fixtures, VSCode can give you more help. Example:

import pytest
import nvme as d

def test_namespace_identify_size(nvme0n1: d.Namespace):
    assert nvme0n1.id_data(7, 0) != 0

Callback functions are supported. If available, the callback function is called when the command completes. Example:

import pytest
import nvme as d

def test_hello_world(nvme0, nvme0n1:d.Namespace):
    read_buf = d.Buffer(512)
    data_buf = d.Buffer(512)
    data_buf[10:21] = b'hello world'
    qpair = d.Qpair(nvme0, 16)  # create IO SQ/CQ pair, with 16 queue-depth
    assert read_buf[10:21] != b'hello world'

    def write_cb(cdw0, status1):  # command callback function
        nvme0n1.read(qpair, read_buf, 0, 1)
    nvme0n1.write(qpair, data_buf, 0, 1, cb=write_cb)
    qpair.waitdone(2)
    assert read_buf[10:21] == b'hello world'

The pynvme can send any kinds of commands, even invalid one. Example:

import pytest

def test_invalid_io_command_0xff(nvme0n1):
    q = d.Qpair(nvme0, 8)
    with pytest.warns(UserWarning, match="ERROR status: 00/01"):
        nvme0n1.send_cmd(0xff, q, nsid=1).waitdone()

The performance is low to send read write IO one by one in python, so pynvme provides IOWorker. IOWorker sends IO in a separated process, so we can send other admin commands simultaneously. Example:

import time
import pytest
from pytemperature import k2c

def test_ioworker_with_temperature(nvme0, nvme0n1):
    smart_log = d.Buffer(512, "smart log page")
    with nvme0n1.ioworker(io_size=8, lba_align=16,
                          lba_random=True, qdepth=16,
                          read_percentage=0, time=30):
        # run ioworker for 30 seconds, while monitoring temperature for 40 seconds
        for i in range(40):
            nvme0.getlogpage(0x02, smart_log, 512).waitdone()
            ktemp = smart_log.data(2, 1)
            logging.info("temperature: %0.2f degreeC" % k2c(ktemp))
            time.sleep(1)

For more examples of pynvme test scripts, please refer to driver_test.py, demo_test.py, and a presentation.

Features

Pynvme writes and reads data in buffer to NVMe device LBA space. In order to verify the data integrity, it injects LBA address and version information into the write data buffer, and check with them after read completion. Furthermore, Pynvme computes and verifies CRC32 of each LBA on the fly. Both data buffer and LBA CRC32 are stored in host memory, so ECC memory are recommended if you are considering serious tests.

Buffer should be allocated for data commands, and held till that command is completed because the buffer is being used by NVMe device. Users need to pay more attention on the life scope of the buffer in Python test scripts.

NVMe commands are all asynchronous. Test scripts can sync through waitdone() method to make sure the command is completed. The method waitdone() polls command Completion Queues. When the optional callback function is provided in a command in python scripts, the callback function is called when that command is completed in waitdone(). The command timeout limit of pynvme is 5 seconds.

Pynvme driver provides two arguments to python callback functions: cdw0 of the Completion Queue Entry, and the status. The argument status includes both Phase Tag and Status Field.

Pynvme traces recent thousands of commands in the cmdlog, as well as the completion entries. The cmdlog traces each qpair's commands and status. Pynvme supports up to 16 qpairs (including the admin qpair of the controller). Users can list cmdlog of each qpair to find the commands issued in different command queues.

The cost is high and inconvenient to send each read and write command in Python scripts. Pynvme provides the low-cost IOWorker to send IOs in different processes. IOWorker takes full use of multi-core to not only send read/write IO in high speed, but also verify the correctness of data on the fly. User can get IOWorker's test statistics through its close() method. Here is an example of reading 4K data randomly with the IOWorker.

Example:

    >>> r = nvme0n1.ioworker(io_size = 8, lba_align = 8,
                             lba_random = True, qdepth = 16,
                             read_percentage = 100, time = 10).start().close()
    >>> print(r.io_count_read)
    >>> print(r.mseconds)
    >>> print("IOPS: %dK/s\n", r.io_count_read/r.mseconds)

The controller is not responsible for checking the LBA of a Read or Write command to ensure any type of ordering between commands (NVMe spec 1.3c, 6.3). It means conflicted read write operations on NVMe devices cannot predict the final data result, and thus hard to verify data correctness. Similarly, after writing of multiple IOWorkers in the same LBA region, the subsequent read does not know the latest data content. As a mitigation solution, we suggest to separate read and write operations to different IOWorkers and different LBA regions in test scripts, so it can be avoid to read and write same LBA at simultaneously. For those read and write operations on same LBA region, scripts have to complete one before submitting the other. Test scripts can disable or enable inline verification of read by function config(). By default, it is disabled.

Qpair instance is created based on Controller instance. So, user creates qpair after the controller. In the other side, user should free qpair before the controller. But without explicit code, Python may not do the job in right order. One of the mitigation solution is pytest fixture scope. User can define Controller fixture as session scope and Qpair as function. In the situation, qpair is always deleted before the controller. Admin qpair is managed by controller, so users do not need to create the admin qpair.

Files

Here is a brief introduction on source code files.

files	notes
spdk	pynvme is built on SPDK
driver_wrap.pyx	pynvme uses cython to bind python and C. All python classes are defined here.
cdriver.pxd	interface between python and C
driver.h	interface of C
driver.c	the core part of pynvme, which extends SPDK for test purpose
setup.py	cython configuration for compile
Makefile	it is a part of SPDK makefiles
driver_test.py	pytest cases for pynvme test. Users can develop more test cases for their NVMe devices.
conftest.py	predefined pytest fixtures. Find more details below.
pytest.ini	pytest runtime configuration
install.sh	build pynvme for the first time

Fixtures

Pynvme uses pytest to test it self. Users can also use pytest as the test framework to test their NVMe devices. Pytest's fixture is a powerful way to create and free resources in the test.

fixture	scope	notes
pciaddr	session	PCIe BDF address of the DUT, pass in by argument --pciaddr
pcie	session	the object of the PCIe device.
nvme0	session	the object of NVMe controller
nvme0n1	session	the object of first Namespace of the controller
verify	function	declare this fixture in test cases where data crc is to be verified.

Debug

1. assert: it is recommended to compile SPDK with --enable-debug.
2. log: users can change log levels for driver and scripts. All logs are captured/hidden by pytest in default. Please use argument "-s" to print logs in test time.
   1. driver: spdk_log_set_print_level in driver.c, for SPDK related logs
   2. scripts: log_cli_level in pytest.ini, for python/pytest scripts
3. gdb: when driver crashes or misbehaviours, use can collect debug information through gdb.
   1. core dump: sudo coredumpctl debug
   2. generate core dump in dead loop: CTRL-\
   3. test within gdb: sudo gdb --args python3 -m pytest --color=yes --pciaddr=01:00.0 "driver_test.py::test_create_device"

If you meet any issue, or have any suggestions, please report them to Issues. They are warmly welcome.

Classes

Buffer

Buffer(self, /, *args, **kwargs)

Buffer class allocated in DPDK memzone,so can be used by DMA. Data in buffer is clear to 0 in initialization.

Attributes

• size (int): the size (in bytes) of the buffer. Default: 4096

• name (str): the name of the buffer. Default: 'buffer'

• Examples:

    >>> b = Buffer(1024, 'example')
    >>> b[0] = 0x5a
    >>> b[1:3] = [1, 2]
    >>> b[4:] = [10, 11, 12, 13]
    >>> b.dump(16)
    example
    00000000  5a 01 02 00 0a 0b 0c 0d  00 00 00 00 00 00 00 00   Z...............
    >>> b[:8:2]
    b'Z\x02\n\x0c'
    >>> b.data(2) == 2
    True
    >>> b[2] == 2
    True
    >>> b.data(2, 0) == 0x02015a
    True
    >>> len(b)
    1024
    >>> b
    <buffer name: example>
    >>> b[8:] = b'xyc'
    example
    00000000  5a 01 02 00 0a 0b 0c 0d  78 79 63 00 00 00 00 00   Z.......xyc.....
    >>> b.set_dsm_range(1, 0x1234567887654321, 0xabcdef12)
    >>> b.dump(64)
    buffer
    00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
    00000010  00 00 00 00 12 ef cd ab  21 43 65 87 78 56 34 12  ........!Ce.xV4.
    00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
    00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................

data

Buffer.data(self, byte_end, byte_begin, type)

get field in the buffer. Little endian for integers.

Attributes

• byte_end (int): the end byte number of this field, which is specified in NVMe spec. Included.
• byte_begin (int): the begin byte number of this field, which is specified in NVMe spec. It can be omitted if begin is the same as end when the field has only 1 byte. Included. Default: None, means only get 1 byte defined in byte_end
• type (type): the type of the field. It should be int or str. Default: int, convert to integer python object

Returns

(int or str): the data in the specified field

dump

Buffer.dump(self, size)

get the buffer content

Attributes

• size: the size of the buffer to print,. Default: None, means to print the whole buffer

set_dsm_range

Buffer.set_dsm_range(self, index, lba, lba_count)

set dsm ranges in the buffer, for dsm/deallocation (a.ka trim) commands

Attributes

• index (int): the index of the dsm range to set
• lba (int): the start lba of the range
• lba_count (int): the lba count of the range

config

config(verify, fua_read=False, fua_write=False)

config driver global setting

Attributes

• verify (bool): enable inline checksum verification of read
• fua_read (bool): enable FUA of read. Default: False
• fua_write (bool): enable FUA of write. Default: False

Returns

None

Controller

Controller(self, /, *args, **kwargs)

Controller class. Prefer to use fixture "nvme0" in test scripts.

Attributes

• addr (bytes): the bus/device/function address of the DUT, for example:

• b'01:00.0' (PCIe BDF address); b'127.0.0.1' (TCP IP address).

• Example:

    >>> n = Controller(b'01:00.0')
    >>> hex(n[0])     # CAP register
    '0x28030fff'
    >>> hex(n[0x1c])  # CSTS register
    '0x1'
    >>> n.id_data(23, 4, str)
    'TW0546VPLOH007A6003Y'
    >>> n.supports(0x18)
    False
    >>> n.supports(0x80)
    True
    >>> id_buf = Buffer()
    >>> n.identify().waitdone()
    >>> id_buf.dump(64)
    buffer
    00000000  a4 14 4b 1b 54 57 30 35  34 36 56 50 4c 4f 48 30  ..K.TW0546VPLOH0
    00000010  30 37 41 36 30 30 33 59  43 41 33 2d 38 44 32 35  07A6003YCA3-8D25
    00000020  36 2d 51 31 31 20 4e 56  4d 65 20 4c 49 54 45 4f  6-Q11 NVMe LITEO
    00000030  4e 20 32 35 36 47 42 20  20 20 20 20 20 20 20 20   N 256GB
    >>> n.cmdlog(2)
    driver.c:1451:log_cmd_dump: *NOTICE*: dump qpair 0, latest tail in cmdlog: 1
    driver.c:1462:log_cmd_dump: *NOTICE*: index 0, 2018-10-14 14:52:25.533708
    nvme_qpair.c: 118:nvme_admin_qpair_print_command: *NOTICE*: IDENTIFY (06) sqid:0 cid:0 nsid:1 cdw10:00000001 cdw11:00000000
    driver.c:1469:log_cmd_dump: *NOTICE*: index 0, 2018-10-14 14:52:25.534030
    nvme_qpair.c: 306:nvme_qpair_print_completion: *NOTICE*: SUCCESS (00/00) sqid:0 cid:95 cdw0:0 sqhd:0142 p:1 m:0 dnr:0
    driver.c:1462:log_cmd_dump: *NOTICE*: index 1, 1970-01-01 07:30:00.000000
    nvme_qpair.c: 118:nvme_admin_qpair_print_command: *NOTICE*: DELETE IO SQ (00) sqid:0 cid:0 nsid:0 cdw10:00000000 cdw11:00000000
    driver.c:1469:log_cmd_dump: *NOTICE*: index 1, 1970-01-01 07:30:00.000000
    nvme_qpair.c: 306:nvme_qpair_print_completion: *NOTICE*: SUCCESS (00/00) sqid:0 cid:0 cdw0:0 sqhd:0000 p:0 m:0 dnr:0

abort

Controller.abort(self, cid, sqid, cb)

abort admin commands

Attributes

• cid (int): command id of the command to be aborted
• sqid (int): sq id of the command to be aborted. Default: 0, to abort the admin command
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

aer

Controller.aer(self, cb)

asynchorous event request admin command.

Not suggested to use this command in scripts because driver manages to send and monitor aer commands. Scripts should register an aer callback function if it wants to handle aer, and use the fixture aer.

Attributes

• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

cmdlog

Controller.cmdlog(self, count)

print recent commands and their completions.

Attributes

• count (int): the number of commands to print. Default: 0, to print the whole cmdlog

cmdname

Controller.cmdname(self, opcode)

get the name of the admin command

Attributes

• opcode (int): the opcode of the admin command

Returns

(str): the command name

downfw

Controller.downfw(self, filename, slot, action)

firmware download utility: by 4K, and activate in next reset

Attributes

• filename (str): the pathname of the firmware binary file to download
• slot (int): firmware slot field in the command. Default: 0, decided by device
• cb (function): callback function called at completion. Default: None

Returns

dst

Controller.dst(self, stc, nsid, cb)

device self test (DST) admin command

Attributes

• stc (int): selftest code (stc) field in the command
• nsid (int): nsid field in the command. Default: 0xffffffff
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

format

Controller.format(self, lbaf, ses, nsid, cb)

format admin command

Attributes

• lbaf (int): lbaf (lba format) field in the command. Default: 0
• ses (int): ses field in the command. Default: 0, no secure erase
• nsid (int): nsid field in the command. Default: 1
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

fw_commit

Controller.fw_commit(self, slot, action, cb)

firmware commit admin command

Attributes

• slot (int): firmware slot field in the command
• action (int): action field in the command
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

fw_download

Controller.fw_download(self, buf, offset, size, cb)

firmware download admin command

Attributes

• buf (Buffer): the buffer to hold the firmware data
• offset (int): offset field in the command
• size (int): size field in the command. Default: None, means the size of the buffer
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

getfeatures

Controller.getfeatures(self, fid, cdw11, cdw12, cdw13, cdw14, cdw15, sel, buf, cb)

getfeatures admin command

Attributes

• fid (int): feature id
• cdw11 (int): cdw11 in the command. Default: 0
• sel (int): sel field in the command. Default: 0
• buf (Buffer): the buffer to hold the feature data. Default: None
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

getlogpage

Controller.getlogpage(self, lid, buf, size, offset, nsid, cb)

getlogpage admin command

Attributes

• lid (int): Log Page Identifier
• buf (Buffer): buffer to hold the log page
• size (int): size (in byte) of data to get from the log page,. Default: None, means the size is the same of the buffer
• offset (int): the location within a log page
• nsid (int): nsid field in the command. Default: 0xffffffff
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

id_data

Controller.id_data(self, byte_end, byte_begin, type, nsid, cns)

get field in controller identify data

Attributes

• byte_end (int): the end byte number of this field, which is specified in NVMe spec. Included.
• byte_begin (int): the begin byte number of this field, which is specified in NVMe spec. It can be omitted if begin is the same as end when the field has only 1 byte. Included. Default: None, means only get 1 byte defined in byte_end
• type (type): the type of the field. It should be int or str. Default: int, convert to integer python object

Returns

(int or str): the data in the specified field

identify

Controller.identify(self, buf, nsid, cns, cb)

identify admin command

Attributes

• buf (Buffer): the buffer to hold the identify data
• nsid (int): nsid field in the command. Default: 0
• cns (int): cns field in the command. Default: 1
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

mdts

max data transfer size

register_aer_cb

Controller.register_aer_cb(self, func)

register aer callback to driver.

It is recommended to use fixture aer(func) in pytest scripts. When aer is triggered, the python callback function will be called. It is unregistered by aer fixture when test finish.

Attributes

• func (function): callback function called at aer completion

reset

Controller.reset(self)

controller reset: cc.en 1 => 0 => 1

Notices

Test scripts should delete all io qpairs before reset!

sanitize

Controller.sanitize(self, option, pattern, cb)

sanitize admin command

Attributes

• option (int): sanitize option field in the command
• pattern (int): pattern field in the command for overwrite method. Default: 0x5aa5a55a
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

send_cmd

Controller.send_cmd(self, opcode, buf, nsid, cdw10, cdw11, cdw12, cdw13, cdw14, cdw15, cb)

send generic admin commands.

This is a generic method. Scripts can use this method to send all kinds of commands, like Vendor Specific commands, and even not existed commands.

Attributes

• opcode (int): operate code of the command
• buf (Buffer): buffer of the command. Default: None
• nsid (int): nsid field of the command. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

setfeatures

Controller.setfeatures(self, fid, cdw11, cdw12, cdw13, cdw14, cdw15, sv, buf, cb)

setfeatures admin command

Attributes

• fid (int): feature id
• cdw11 (int): cdw11 in the command. Default: 0
• sv (int): sv field in the command. Default: 0
• buf (Buffer): the buffer to hold the feature data. Default: None
• cb (function): callback function called at completion. Default: None

Returns

self (Controller)

supports

Controller.supports(self, opcode)

check if the admin command is supported

Attributes

• opcode (int): the opcode of the admin command

Returns

(bool): if the command is supported

waitdone

Controller.waitdone(self, expected)

sync until expected commands completion

Attributes

• expected (int): expected commands to complete. Default: 1

Notices

Do not call this function in commands callback functions.

DotDict

DotDict(self, *args, **kwargs)

utility class to access dict members by . operation

Namespace

Namespace(self, /, *args, **kwargs)

Namespace class. Prefer to use fixture "nvme0n1" in test scripts.

Attributes

• nvme (Controller): controller where to create the queue
• nsid (int): nsid of the namespace

capacity

bytes of namespace capacity

close

Namespace.close(self)

close namespace to release it resources in host memory.

Notice: Release resources explictly, del is not garentee to call dealloc. Fixture nvme0n1 uses this function, and prefer to use fixture in scripts, instead of calling this function directly.

cmdname

Namespace.cmdname(self, opcode)

get the name of the IO command

Attributes

• opcode (int): the opcode of the IO command

Returns

(str): the command name

compare

Namespace.compare(self, qpair, buf, lba, lba_count, io_flags, cb)

compare IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• buf (Buffer): the data buffer of the command, meta data is not supported.
• lba (int): the starting lba address, 64 bits
• lba_count (int): the lba count of this command, 16 bits. Default: 1
• io_flags (int): io flags defined in NVMe specification, 16 bits. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the command fails

Notices

buf cannot be released before the command completes.

dsm

Namespace.dsm(self, qpair, buf, range_count, attribute, cb)

data-set management IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• buf (Buffer): the buffer of the lba ranges. Use buffer.set_dsm_range to prepare the buffer.
• range_count (int): the count of lba ranges in the buffer
• attribute (int): attribute field of the command. Default: 0x4, as deallocation/trim
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the command fails

Notices

buf cannot be released before the command completes.

flush

Namespace.flush(self, qpair, cb)

flush IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the command fails

format

Namespace.format(self, data_size, meta_size, ses)

change the format of this namespace

Attributes

• data_size (int): data size. Default: 512
• meta_size (int): meta data size. Default: 0
• ses (int): ses field in the command. Default: 0, no secure erase

Returns

(int or None): the lba format has the specified data size and meta data size

Notices

this facility not only sends format admin command, but also updates driver to activate new format immediately

get_lba_format

Namespace.get_lba_format(self, data_size, meta_size)

find the lba format by its data size and meta data size

Attributes

• data_size (int): data size. Default: 512
• meta_size (int): meta data size. Default: 0

Returns

(int or None): the lba format has the specified data size and meta data size

id_data

Namespace.id_data(self, byte_end, byte_begin, type)

get field in namespace identify data

Attributes

• byte_end (int): the end byte number of this field, which is specified in NVMe spec. Included.
• byte_begin (int): the begin byte number of this field, which is specified in NVMe spec. It can be omitted if begin is the same as end when the field has only 1 byte. Included. Default: None, means only get 1 byte defined in byte_end
• type (type): the type of the field. It should be int or str. Default: int, convert to integer python object

Returns

(int or str): the data in the specified field

ioworker

Namespace.ioworker(self, io_size, lba_align, lba_random, read_percentage, time, qdepth, region_start, region_end, iops, io_count, lba_start, qprio, output_io_per_second, output_percentile_latency)

workers sending different read/write IO on different CPU cores.

User defines IO characteristics in parameters, and then the ioworker executes without user intervesion, until the test is completed. IOWorker returns some statistic data at last.

User can start multiple IOWorkers, and they will be binded to different CPU cores. Each IOWorker creates its own Qpair, so active IOWorker counts is limited by maximum IO queues that DUT can provide.

Each ioworker can run upto 24 hours.

Attributes

• io_size (short): IO size, unit is LBA
• lba_align (short): IO alignment, unit is LBA
• lba_random (bool): True if sending IO with random starting LBA
• read_percentage (int): sending read/write mixed IO, 0 means write only, 100 means read only
• time (int): specified maximum time of the IOWorker in seconds, up to 24*3600. Default:0, means no limit
• qdepth (int): queue depth of the Qpair created by the IOWorker, up to 1024. Default: 64
• region_start (long): sending IO in the specified LBA region, start. Default: 0
• region_end (long): sending IO in the specified LBA region, end but not include. Default: 0xffff_ffff_ffff_ffff
• iops (int): specified maximum IOPS. IOWorker throttles the sending IO speed. Default: 0, means no limit
• io_count (long): specified maximum IO counts to send. Default: 0, means no limit
• lba_start (long): the LBA address of the first command. Default: 0, means start from region_start
• qprio (int): SQ priority. Default: 0, as Round Robin arbitration
• output_io_per_second (list): list to hold the output data of io_per_second. Default: None, not to collect the data
• output_percentile_latency (dict): dict of io counter on different percentile latency. Dict key is the percentage, and the value is the latency in ms. Default: None, not to collect the data

Returns

ioworker object

nsid

id of the namespace

read

Namespace.read(self, qpair, buf, lba, lba_count, io_flags, cb)

read IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• buf (Buffer): the data buffer of the command, meta data is not supported.
• lba (int): the starting lba address, 64 bits
• lba_count (int): the lba count of this command, 16 bits. Default: 1
• io_flags (int): io flags defined in NVMe specification, 16 bits. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the read command fails

Notices

buf cannot be released before the command completes.

send_cmd

Namespace.send_cmd(self, opcode, qpair, buf, nsid, cdw10, cdw11, cdw12, cdw13, cdw14, cdw15, cb)

send generic IO commands.

This is a generic method. Scripts can use this method to send all kinds of commands, like Vendor Specific commands, and even not existed commands.

Attributes

• opcode (int): operate code of the command
• qpair (Qpair): qpair used to send this command
• buf (Buffer): buffer of the command. Default: None
• nsid (int): nsid field of the command. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

supports

Namespace.supports(self, opcode)

check if the IO command is supported

Attributes

• opcode (int): the opcode of the IO command

Returns

(bool): if the command is supported

write

Namespace.write(self, qpair, buf, lba, lba_count, io_flags, cb)

write IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• buf (Buffer): the data buffer of the write command, meta data is not supported.
• lba (int): the starting lba address, 64 bits
• lba_count (int): the lba count of this command, 16 bits
• io_flags (int): io flags defined in NVMe specification, 16 bits. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the write command fails

Notices

buf cannot be released before the command completes.

write_uncorrectable

Namespace.write_uncorrectable(self, qpair, lba, lba_count, cb)

write uncorrectable IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• lba (int): the starting lba address, 64 bits
• lba_count (int): the lba count of this command, 16 bits. Default: 1
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the command fails

write_zeroes

Namespace.write_zeroes(self, qpair, lba, lba_count, io_flags, cb)

write zeroes IO command

Attributes

• qpair (Qpair): use the qpair to send this command
• lba (int): the starting lba address, 64 bits
• lba_count (int): the lba count of this command, 16 bits. Default: 1
• io_flags (int): io flags defined in NVMe specification, 16 bits. Default: 0
• cb (function): callback function called at completion. Default: None

Returns

qpair (Qpair): the qpair used to send this command, for ease of chained call

Raises

• SystemError: the command fails

Pcie

Pcie(self, /, *args, **kwargs)

Pcie class. Prefer to use fixture "pcie" in test scripts

Attributes

• nvme (Controller): the nvme controller object of that subsystem

cap_offset

Pcie.cap_offset(self, cap_id)

get the offset of a capability

Attributes

• cap_id (int): capability id

Returns

(int): the offset of the register or None if the capability is not existed

register

Pcie.register(self, offset, byte_count)

access registers in pcie config space, and get its integer value.

Attributes

• offset (int): the offset (in bytes) of the register in the config space
• byte_count (int): the size (in bytes) of the register

Returns

(int): the value of the register

reset

Pcie.reset(self)

reset this pcie device

Qpair

Qpair(self, /, *args, **kwargs)

Qpair class. IO SQ and CQ are combinded as qpairs.

Attributes

• nvme (Controller): controller where to create the queue
• depth (int): SQ/CQ queue depth
• prio (int): when Weighted Round Robin is enabled, specify SQ priority here

cmdlog

Qpair.cmdlog(self, count)

print recent IO commands and their completions in this qpair.

Attributes

• count (int): the number of commands to print. Default: 0, to print the whole cmdlog

waitdone

Qpair.waitdone(self, expected)

sync until expected commands completion

Attributes

• expected (int): expected commands to complete. Default: 1

Notices

Do not call this function in commands callback functions.

Subsystem

Subsystem(self, /, *args, **kwargs)

Subsystem class. Prefer to use fixture "subsystem" in test scripts.

Attributes

• nvme (Controller): the nvme controller object of that subsystem

power_cycle

Subsystem.power_cycle(self, sec)

power off and on in seconds

Attributes

• sec (int): the seconds between power off and power on

reset

Subsystem.reset(self)

reset the nvme subsystem through register nssr.nssrc

shutdown_notify

Subsystem.shutdown_notify(self, abrupt)

notify nvme subsystem a shutdown event through register cc.chn

Attributes

• abrupt (bool): it will be an abrupt shutdown (return immediately) or clean shutdown (wait shutdown completely)