summaryrefslogtreecommitdiffstats
path: root/tests/README
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--tests/README98
1 files changed, 98 insertions, 0 deletions
diff --git a/tests/README b/tests/README
new file mode 100644
index 0000000..a5fa708
--- /dev/null
+++ b/tests/README
@@ -0,0 +1,98 @@
+nvmetests
+=========
+
+ This contains a NVMe tests framework. The purpose of this framework
+ to use nvme cli and test various supported commands and scenarios for
+ NVMe device.
+
+ In current implementation this framework uses nvme-cli to
+ interact with underlying controller/namespace.
+
+ Note these tests expect to run against real hardware and will
+ read and write data to /dev/nvme0!
+
+ DO NOT RUN THEM IF YOU DO NOT KNOW WHAT YOU ARE DOING!
+
+ You have been warned.
+
+1. Common Package Dependencies
+------------------------------
+
+ 1. Python(>= 3.3)
+ 2. nose2 (Installation guide http://nose2.readthedocs.io/)
+ 3. flake8 (https://pypi.python.org/pypi/flake8)
+ 4. mypy (https://pypi.org/project/mypy/)
+ 5. autopep8 (https://pypi.org/project/autopep8/)
+ 6. isort (https://pypi.org/project/isort/)
+
+ Python package management system pip can be used to install most of the
+ listed packages(https://pip.pypa.io/en/stable/installing/) :-
+ $ pip install nose2 flake8 mypy autopep8 isort
+
+2. Overview
+-----------
+
+ This framework follows simple class hierarchy. Each test file contains
+ one test. Each test is direct subclass or indirect subclass of TestNVMe
+ class which represents one testcase. To write a new testcase one can copy
+ existing template "nvme_simple_template_test.py" and start adding new
+ testcase specific functionality. For detailed information please look into
+ section 3.
+
+ For more information about tests, class hierarchy and code please refer :-
+
+ 1. Documentation :- html/
+ 2. Class Index :- html/index.html
+ 3. Class Hierarchy :- html/class-tree.html
+
+ For each testcase it will create log directory mentioned in
+ configuration file. This directory will be used for a temporary files
+ and storing execution logs of each testcases. Current implementation stores
+ stdout and stderr for each testcase under log directory, e.g. :-
+
+ $ tree nvmetests/
+ nvmetests/
+ ├── TestNVMeAttachDetachNSCmd
+ │   ├── stderr.log
+ │   └── stdout.log
+ ├── TestNVMeFlushCmd
+ │   ├── stderr.log
+ │   └── stdout.log
+ └── TestNVMeFormatCmd
+ ├── stderr.log
+ └── stdout.log
+ .
+ .
+ .
+
+3. Walk-Through Example for writing a new testcase
+--------------------------------------------------
+ 1. Copy simple test template file from current directory
+ with appropriate name, replace "simple_template" with testcase name
+ in new file name. Update config.json if necessary.
+ 2. Write a testcase main function, make sure its name is starting with
+ test_*.
+ 3. Based on the requirement one can inherit TestNVMe or TestNVMeIO
+ class.
+ 4. Write test precondition code into setUp. Make sure you are calling
+ super class setUp.
+ 5. Write test post condition code into tearDown. Make sure you are calling
+ super class tearDown.
+ 6. Before writing a new function have a look into TestNVMe to see if it
+ can be reused.
+ 7. Once testcase is ready make sure :-
+ a. Run flake8, mypy, autopep8 and isort on the testcase and fix
+ errors/warnings.
+ - Example "$ ninja -C .build lint-python" will run flake8 and
+ mypy on all the python files in current directory.
+ - Example "$ ninja -C .build format-python" will run autopep8 and
+ isort on all the python files in the current directory.
+
+4. Running testcases with framework
+-----------------------------------
+ 1. Running single testcase (in the source tree) with nose2 :-
+ $ nose2 --verbose --start-dir tests nvme_writezeros_test
+ $ nose2 --verbose --start-dir tests nvme_read_write_test
+
+ 2. Running all the testcases (in the build root directory) with ninja :-
+ $ ninja test -C .build