blob: 14b24e5080667e75d84daa51de4654d378dc1804 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
nvmetests
=========
This contains NVMe unit tests framework. The purpose of this framework
to use nvme cli and test various supported commands and scenarios for
NVMe device.
In current implementation this framework uses nvme cli to
interact with underlying controller/namespace.
1. Common Package Dependencies
------------------------------
1. Python(>= 3.3)
2. nose2 (Installation guide http://nose2.readthedocs.io/)
3. flake8 (https://pypi.python.org/pypi/flake8)
4. mypy (https://pypi.org/project/mypy/)
5. autopep8 (https://pypi.org/project/autopep8/)
6. isort (https://pypi.org/project/isort/)
Python package management system pip can be used to install most of the
listed packages(https://pip.pypa.io/en/stable/installing/) :-
$ pip install nose2 flake8 mypy autopep8 isort
2. Overview
-----------
This framework follows simple class hierarchy. Each test file contains
one test. Each test is direct subclass or indirect subclass of TestNVMe
class which represents one testcase. To write a new testcase one can copy
existing template "nvme_simple_template_test.py" and start adding new
testcase specific functionality. For detailed information please look into
section 3.
For more information about tests, class hierarchy and code please refer :-
1. Documentation :- html/
2. Class Index :- html/index.html
3. Class Hierarchy :- html/class-tree.html
For each testcase it will create log directory mentioned in
configuration file. This directory will be used for a temporary files
and storing execution logs of each testcases. Current implementation stores
stdout and stderr for each testcase under log directory, e.g. :-
$ tree nvmetests/
nvmetests/
├── TestNVMeAttachDetachNSCmd
│ ├── stderr.log
│ └── stdout.log
├── TestNVMeFlushCmd
│ ├── stderr.log
│ └── stdout.log
└── TestNVMeFormatCmd
├── stderr.log
└── stdout.log
.
.
.
3. Walk-Through Example for writing a new testcase
--------------------------------------------------
1. Copy simple test template file from current directory
with appropriate name, replace "simple_template" with testcase name
in new file name. Update config.json if necessary.
2. Write a testcase main function, make sure its name is starting with
test_*.
3. Based on the requirement one can inherit TestNVMe or TestNVMeIO
class.
4. Write test precondition code into setUp. Make sure you are calling
super class setUp.
5. Write test post condition code into tearDown. Make sure you are calling
super class tearDown.
6. Before writing a new function have a look into TestNVMe to see if it
can be reused.
7. Once testcase is ready make sure :-
a. Run flake8, mypy, autopep8 and isort on the testcase and fix
errors/warnings.
- Example "$ ninja -C .build lint-python" will run flake8 and
mypy on all the python files in current directory.
- Example "$ ninja -C .build format-python" will run autopep8 and
isort on all the python files in the current directory.
4. Running testcases with framework
-----------------------------------
1. Running single testcase (in the source tree) with nose2 :-
$ nose2 --verbose --start-dir tests nvme_writezeros_test
$ nose2 --verbose --start-dir tests nvme_read_write_test
2. Running all the testcases (in the build root directory) with ninja :-
$ ninja test -C .build
|
