diff options
Diffstat (limited to 'docs/gtest/index.rst')
| -rw-r--r-- | docs/gtest/index.rst | 311 |
1 files changed, 311 insertions, 0 deletions
diff --git a/docs/gtest/index.rst b/docs/gtest/index.rst new file mode 100644 index 0000000000..a685f8ea8a --- /dev/null +++ b/docs/gtest/index.rst @@ -0,0 +1,311 @@ +GTest +===== + +GTest (googletest) is Google's framework for writing C++ tests on a +variety of platforms (Linux, Mac OS X, Windows, ...). +Based on the xUnit architecture, it supports automatic test +discovery, a rich set of assertions, user-defined assertions, death +tests, fatal and non-fatal failures, value- and type-parameterized +tests, various options for running the tests, and XML test report +generation. + +Integration +----------- + +GTest is run as a standard test task on Win/Mac/Linux and Android, under +treeherder symbol 'GTest'. + + +Running tests +------------- + +The Firefox build process will build GTest on supported platforms as +long as you don't disable tests in your mozconfig. However xul-gtest will +only be built when tests are required to save an expensive second +linking process. + +To run the unit tests use 'mach gtest' when invoking Gecko. + +Running selected tests +~~~~~~~~~~~~~~~~~~~~~~ + +Tests can be selected using mach. You can also use environment variables +support by GTest. See `Running Test Programs: Running a Subset of the +Tests <https://github.com/google/googletest/blob/master/docs/advanced.md#running-a-subset-of-the-tests>`__ +for more details. + +:: + + mach gtest Moz2D.* + + +Configuring GTest +~~~~~~~~~~~~~~~~~ + +GTest can be controlled from other environment variables. See `Running +Test Programs: Advanced +Options <https://github.com/google/googletest/blob/master/docs/advanced.md#running-test-programs-advanced-options>`__ +for more details. + + +Debugging a GTest Unit Test +--------------------------- + +To debug a gtest, pass --debug to the normal command. + +.. code-block:: shell + + ./mach gtest --debug [ Prefix.Test ] + +If that doesn't work, you can try running the firefox binary under the +debugger with the MOZ_RUN_GTEST environment variable set to 1. + +.. code-block:: shell + + MOZ_RUN_GTEST=1 ./mach run --debug [--debugger gdb] + +.. warning:: + + Don't forget to build + run 'mach gtest' to relink when using + MOZ_RUN_GTEST since it's not part of a top level build. + +Note that this will load an alternate libxul - the one which has the +test code built in, which resides in a gtest/subdirectory of your +objdir. This gtest-enabled libxul is not built as part of the regular +build, so you must ensure that it is built before running the above +command. A simple way to do this is to just run "mach gtest" which will +rebuild libxul and run the tests. You can also extract the commands +needed to just rebuild that libxul `from +mach <https://hg.mozilla.org/mozilla-central/file/3673d2c688b4/python/mozbuild/mozbuild/mach_commands.py#l486>`__ +and run those directly. Finally, note that you may have to run through +the tests once for gdb to load all the relevant libraries and for +breakpoint symbols to resolve properly. + +Note that you can debug a subset of the tests (including a single test) +by using the GTEST_FILTER environment variable: + +.. code-block:: shell + + GTEST_FILTER='AsyncPanZoom*' MOZ_RUN_GTEST=1 ./mach run --debug [--debugger gdb] + + +Debugging with Xcode +~~~~~~~~~~~~~~~~~~~~ + +See :ref:`Debugging On macOS` for initial +setup. You'll likely want to create a separate scheme for running GTest +("Product" > "Scheme" > "New Scheme…"). In addition to GTEST_FILTER, Set +the following environment variables: + +:: + + MOZ_XRE_DIR=/path-to-object-directory/obj-ff-dbg/dist/bin + MOZ_RUN_GTEST=True + +and under the "Options" tab for the scheme, set the working directory +to: + +:: + + ☑️ Use custom working directory: /path-to-object-directory/obj-ff-dbg/_tests/gtest + + +Debugging with Visual Studio Code +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add a configuration like this to your launch.json file (you can edit it +via Run / Open Configurations): + +:: + + { + "name": "(gdb) Launch gtest", + "type": "cppdbg", + "request": "launch", + "program": "${workspaceFolder}/obj-x86_64-pc-linux-gnu/dist/bin/firefox", + "args": [], + "stopAtEntry": false, + "cwd": "${workspaceFolder}/obj-x86_64-pc-linux-gnu/_tests/gtest", + "environment": [{"name": "MOZ_RUN_GTEST", "value": "True"}, + {"name": "GTEST_FILTER", "value": "AsyncPanZoom*"}], + "externalConsole": false, + "MIMode": "gdb", + "setupCommands": [ + { + "description": "Enable pretty-printing for gdb", + "text": "-enable-pretty-printing", + "ignoreFailures": true + } + ] + }, + + +Writing a GTest Unit Test +------------------------- + +Most of the `GTest +documentation <https://github.com/google/googletest/blob/master/googletest/README.md>`__ +will apply here. The `GTest +primer <https://github.com/google/googletest/blob/master/docs/primer.md>`__ +is a recommended read. + +.. warning:: + + GTest will run tests in parallel. Don't add unit tests that are not + threadsafe, such as tests that require focus or use specific sockets. + +.. warning:: + + GTest will run without initializing mozilla services. Initialize and + tear down any dependencies you have in your test fixtures. Avoid + writing integration tests and focus on testing individual units. + +See https://hg.mozilla.org/mozilla-central/rev/ed612eec41a44867a for an +example of how to add a simple test. + +If you're converting an existing C++ unit test to a GTest, `this +commit <https://hg.mozilla.org/mozilla-central/rev/40740cddc131>`__ may +serve as a useful reference. + + +Setting prefs for a test +~~~~~~~~~~~~~~~~~~~~~~~~ + +If tests cover functionality that is disabled by default, you'll have to +change the relevant preferences either in the individual test: + +:: + + bool oldPref = Preferences::GetBool(prefKey); + Preferences::SetBool(prefKey, true); + … // test code + Preferences::SetBool(prefKey, oldPref); + +or, if it applies more broadly, the change can be applied to the whole +fixture (see `the GTest +docs <https://github.com/google/googletest/blob/master/googletest/README.md>`__, +or +`AutoInitializeImageLib <https://searchfox.org/mozilla-central/search?q=AutoInitializeImageLib%3A%3AAutoInitializeImageLib&path=>`__ +as an example). + + +Adding a test to the build system +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Find a gtest directory appropriate for the module. If none exist create +a directory using the following convention: '<submodule>/tests/gtest'. +Create a moz.build file (in the newly created directory) with a module +declaration, replacing gfxtest with a unique name, and set +UNIFIED_SOURCES to contain all of the test file names. + +What we're doing here is creating a list of source files that will be +compiled and linked only against the gtest version of libxul. This will +let these source files call internal xul symbols without making them +part of the binary we ship to users. + +.. code-block:: + + # -*- Mode: python; c-basic-offset: 4; indent-tabs-mode: nil; tab-width: 40 -*- + # vim: set filetype=python: + # This Source Code Form is subject to the terms of the Mozilla Public + # License, v. 2.0. If a copy of the MPL was not distributed with this + # file, you can obtain one at https://mozilla.org/MPL/2.0/. + + Library('gfxtest') + + UNIFIED_SOURCES = [ + <ListTestFiles>, + ] + + FINAL_LIBRARY = 'xul-gtest' + +Update '<submodule>/moz.build' in the parent directory to build your new +subdirectory in: + +.. code-block:: python + + TEST_DIRS += [ + "gtest", + ] + +When adding tests to an existing moz.build file (it has FINAL_LIBRARY = +'xul-gtest'), add the following. That's it--there is no test manifest +required. Your tests will be automatically registered using a static +constructor. + +.. code-block:: python + + UNIFIED_SOURCES = [ + 'TestFoo.cpp', + ] + +Notes +~~~~~ + +The include file for the class you are testing may not need to be +globally exported, but it does need to be made available to the unit +test you are writing. In that case, add something like this to the +Makefile.in inside of the testing directory. + +.. code-block:: python + + LOCAL_INCLUDES += [ + '/gfx/2d', + '/gfx/2d/unittest', + '/gfx/layers', + ] + +Gtests currently run from the test package under the **GTest** symbol on +`Treeherder <https://treeherder.mozilla.org/>`__ if you want to verify +that your test is working. Formerly they were run under the **B** +symbol, during \`make check`. + + +MozGTestBench +------------- + +A Mozilla GTest Microbench is just a GTest that reports the test +duration to perfherder. It's an easy way to add low level performance +test. Keep in mind that there's a non-zero cost to monitoring +performance test so use them sparingly. You can still perform test +assertions. + + +Writing a Microbench GTest +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use 'MOZ_GTEST_BENCH' instead of 'TEST' to time the execution of your +test. Example: + +.. code-block:: cpp + + #include "gtest/MozGTestBench.h" // For MOZ_GTEST_BENCH + + ... + + MOZ_GTEST_BENCH(GfxBench, TEST_NAME, []{ + // Test to time the execution + }); + +Make sure this file is registered with the file system using the +instructions above. If everything worked correctly you should see this +in the GTest log for your corresponding test: + +.. code-block:: js + + PERFHERDER_DATA: {"framework": {"name": "platform_microbench"}, "suites": [{"name": "GfxBench", "subtests": [{"name": "CompositorSimpleTree", "value": 252674, "lowerIsBetter": true}]}]} + + +Sheriffing policy +~~~~~~~~~~~~~~~~~ + +Microbench tests measure the speed of a very specific operation. A +regression in a micro-benchmark may not lead to a user visible +regression and should not be treated as strictly as a Talos regression. +Large changes in microbench scores will also be expected when the code +is directly modified and should be accepted if the developer intended to +change that code. Micro-benchmarks however provide a framework for +adding performance tests for platform code and regression tests for +performance fixes. They will catch unintended regressions in code and +when correlated with a Talos regression might indicate the source of the +regression. |