diff options
Diffstat (limited to 'tests/README')
| -rw-r--r-- | tests/README | 98 |
1 files changed, 98 insertions, 0 deletions
diff --git a/tests/README b/tests/README new file mode 100644 index 0000000..a5fa708 --- /dev/null +++ b/tests/README @@ -0,0 +1,98 @@ +nvmetests +========= + + This contains a NVMe tests framework. The purpose of this framework + to use nvme cli and test various supported commands and scenarios for + NVMe device. + + In current implementation this framework uses nvme-cli to + interact with underlying controller/namespace. + + Note these tests expect to run against real hardware and will + read and write data to /dev/nvme0! + + DO NOT RUN THEM IF YOU DO NOT KNOW WHAT YOU ARE DOING! + + You have been warned. + +1. Common Package Dependencies +------------------------------ + + 1. Python(>= 3.3) + 2. nose2 (Installation guide http://nose2.readthedocs.io/) + 3. flake8 (https://pypi.python.org/pypi/flake8) + 4. mypy (https://pypi.org/project/mypy/) + 5. autopep8 (https://pypi.org/project/autopep8/) + 6. isort (https://pypi.org/project/isort/) + + Python package management system pip can be used to install most of the + listed packages(https://pip.pypa.io/en/stable/installing/) :- + $ pip install nose2 flake8 mypy autopep8 isort + +2. Overview +----------- + + This framework follows simple class hierarchy. Each test file contains + one test. Each test is direct subclass or indirect subclass of TestNVMe + class which represents one testcase. To write a new testcase one can copy + existing template "nvme_simple_template_test.py" and start adding new + testcase specific functionality. For detailed information please look into + section 3. + + For more information about tests, class hierarchy and code please refer :- + + 1. Documentation :- html/ + 2. Class Index :- html/index.html + 3. Class Hierarchy :- html/class-tree.html + + For each testcase it will create log directory mentioned in + configuration file. This directory will be used for a temporary files + and storing execution logs of each testcases. Current implementation stores + stdout and stderr for each testcase under log directory, e.g. :- + + $ tree nvmetests/ + nvmetests/ + ├── TestNVMeAttachDetachNSCmd + │ ├── stderr.log + │ └── stdout.log + ├── TestNVMeFlushCmd + │ ├── stderr.log + │ └── stdout.log + └── TestNVMeFormatCmd + ├── stderr.log + └── stdout.log + . + . + . + +3. Walk-Through Example for writing a new testcase +-------------------------------------------------- + 1. Copy simple test template file from current directory + with appropriate name, replace "simple_template" with testcase name + in new file name. Update config.json if necessary. + 2. Write a testcase main function, make sure its name is starting with + test_*. + 3. Based on the requirement one can inherit TestNVMe or TestNVMeIO + class. + 4. Write test precondition code into setUp. Make sure you are calling + super class setUp. + 5. Write test post condition code into tearDown. Make sure you are calling + super class tearDown. + 6. Before writing a new function have a look into TestNVMe to see if it + can be reused. + 7. Once testcase is ready make sure :- + a. Run flake8, mypy, autopep8 and isort on the testcase and fix + errors/warnings. + - Example "$ ninja -C .build lint-python" will run flake8 and + mypy on all the python files in current directory. + - Example "$ ninja -C .build format-python" will run autopep8 and + isort on all the python files in the current directory. + +4. Running testcases with framework +----------------------------------- + 1. Running single testcase (in the source tree) with nose2 :- + $ nose2 --verbose --start-dir tests nvme_writezeros_test + $ nose2 --verbose --start-dir tests nvme_read_write_test + + 2. Running all the testcases (in the build root directory) with ninja :- + $ ninja test -C .build |