101 lines
3.5 KiB
Markdown
101 lines
3.5 KiB
Markdown
# Mn Python tests
|
||
|
||
_Marionette_ is the codename of a [remote protocol] built in to
|
||
Firefox as well as the name of a functional test framework for
|
||
automating user interface tests.
|
||
|
||
The in-tree test framework supports tests written in Python, using
|
||
Python’s [unittest] library. Test cases are written as a subclass
|
||
of `MarionetteTestCase`, with child tests belonging to instance
|
||
methods that have a name starting with `test_`.
|
||
|
||
You can additionally define [`setUp`] and [`tearDown`] instance
|
||
methods to execute code before and after child tests, and
|
||
[`setUpClass`]/[`tearDownClass`] for the parent test. When you use
|
||
these, it is important to remember calling the `MarionetteTestCase`
|
||
superclass’ own [`setUp`]/[`tearDown`] methods since they handle
|
||
setup/cleanup of the session.
|
||
|
||
The test structure is illustrated here:
|
||
|
||
```python
|
||
from marionette_harness import MarionetteTestCase
|
||
|
||
class TestSomething(MarionetteTestCase):
|
||
def setUp(self):
|
||
# code to execute before any tests are run
|
||
MarionetteTestCase.setUp(self)
|
||
|
||
def test_foo(self):
|
||
# run test for 'foo'
|
||
|
||
def test_bar(self):
|
||
# run test for 'bar'
|
||
|
||
def tearDown(self):
|
||
# code to execute after all tests are run
|
||
MarionetteTestCase.tearDown(self)
|
||
```
|
||
|
||
[remote protocol]: Protocol.md
|
||
[unittest]: https://docs.python.org/3/library/unittest.html
|
||
[`setUp`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp
|
||
[`setUpClass`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass
|
||
[`tearDown`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown
|
||
[`tearDownClass`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDownClass
|
||
|
||
## Test assertions
|
||
|
||
Assertions are provided courtesy of [unittest]. For example:
|
||
|
||
```python
|
||
from marionette_harness import MarionetteTestCase
|
||
|
||
class TestSomething(MarionetteTestCase):
|
||
def test_foo(self):
|
||
self.assertEqual(9, 3 * 3, '3 x 3 should be 9')
|
||
self.assertTrue(type(2) == int, '2 should be an integer')
|
||
```
|
||
|
||
## The API
|
||
|
||
The full API documentation is found [here], but the key objects are:
|
||
|
||
* `MarionetteTestCase`: a subclass for `unittest.TestCase`
|
||
used as a base class for all tests to run.
|
||
|
||
* {class}`Marionette <marionette_driver.marionette.Marionette>`: client that speaks to Firefox
|
||
|
||
[here]: /python/marionette_driver.rst
|
||
|
||
## Registering Test Manifests
|
||
|
||
To run Marionette Python tests locally via `mach` or as part of the `Mn` tests jobs
|
||
in CI they need to be registered. This happens by adding a manifest file to the tree,
|
||
which includes a reference to the test files and expectations for results.
|
||
|
||
Such a manifest file can look like the following and is stored with the extension `.toml`:
|
||
|
||
```ini
|
||
[DEFAULT]
|
||
|
||
["test_expected_fail.py"]
|
||
expected = "fail"
|
||
|
||
["test_not_on_windows.py"]
|
||
skip-if = ["os == 'win'"]
|
||
```
|
||
|
||
The registration of such a manifest file is done in two different ways:
|
||
|
||
1. To run the tests locally via `./mach test` or `./mach marionette-test` the
|
||
created Marionette manifest file needs to be referenced in the folder's related
|
||
`moz.build` file by adding it to the `MARIONETTE_MANIFESTS` variable like:
|
||
|
||
MARIONETTE_MANIFESTS += ["test/marionette/manifest.toml"]
|
||
|
||
2. To run the tests in CI the manifest file also needs to be included in the
|
||
Marionette's own [master manifest file]. This ensures that the test packaging step
|
||
will find the tests and include them as well in the test package.
|
||
|
||
[master manifest file]: https://searchfox.org/mozilla-central/source/testing/marionette/harness/marionette_harness/tests/unit-tests.toml
|