Adding upstream version 124.0.1.upstream/124.0.1

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 240 insertions, 0 deletions

diff --git a/toolkit/components/glean/docs/dev/testing.md b/toolkit/components/glean/docs/dev/testing.md
new file mode 100644
index 0000000000..fe6c3e0fe7
--- /dev/null
+++ b/toolkit/components/glean/docs/dev/testing.md
@@ -0,0 +1,240 @@
+# Testing
+
+```{admonition} This documentation is about testing FOG itself
+This document contains information about how FOG tests itself,
+how to add new tests, how and what to log, and stuff like that.
+If you're interested in learning how to test instrumentation you added,
+you'll want to read
+[the instrumetnation testing docs](../user/instrumentation_tests) instead.
+```
+
+Given the multiple API languages, processes, and dependencies,
+testing FOG is a matter of choosing the right tool for the situation.
+
+## One Big Command
+
+To run all the things, here's the tl;dr:
+
+`./mach build && ./mach lint -W -w -o --fix
+&& ./mach rusttests && ./mach gtest "FOG*"
+&& python3 ./mach python-test toolkit/components/glean/tests/pytest
+&& ./mach test toolkit/components/glean/tests/xpcshell
+&& ./mach telemetry-tests-client toolkit/components/telemetry/tests/marionette/tests/client/test_fog* --gecko-log "-"
+&& ./mach test toolkit/components/glean/tests/browser
+`
+
+## Logging
+
+An often-overlooked first line of testing is "what do the logs say?".
+To turn on logging for FOG, use any of the following:
+* Run Firefox with `RUST_LOG="fog_control,fog,glean_core"`.
+    * On some platforms this will use terminal colours to indicate log level.
+* Run Firefox with `MOZ_LOG="timestamp,sync,glean::*:5,fog::*:5,fog_control::*:5,glean_core::*:5"`.
+* Set the following prefs:
+    * `logging.config.timestamp` to `true`
+    * `logging.config.sync` to `true`
+    * `logging.fog_control::*` to `5`
+    * `logging.fog::*` to `5`
+    * `logging.glean::*` to `5`
+    * `logging.glean_core::*` to `5`
+    * `logging.config.clear_on_startup` to `false` (or all these prefs will be cleared on startup)
+
+For more information on logging in Gecko, see the
+[Gecko Logging docs](/xpcom/logging).
+
+User-destined logs (of the "You did something wrong" variety) might output to the
+[Browser Console](/devtools-user/browser_console/index)
+if they originate in JS land. Open via
+<kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>J</kbd>, or
+<kbd>Cmd</kbd>+<kbd>Shift</kbd>+<kbd>J</kbd>.
+
+```{admonition} Note
+At present, Rust logging in non-main processes just doesn't work.
+```
+
+### What to log, and to where?
+
+FOG covers a lot a ground (languages, layers):
+where you are determines what logging you have available.
+
+Here are some common situtations for logging:
+
+#### JS to C++
+
+If your logging is aimed at the user using the JS API
+(e.g. because the type provided isn't convertable to the necessary C++ type)
+then use the Browser Console via
+[FOG's Common's `LogToBrowserConsole`](https://searchfox.org/mozilla-central/rev/d107bc8aeadcc816ba85cb21c1a6a1aac1d4ef9f/toolkit/components/glean/bindings/private/Common.cpp#19).
+
+#### C++
+
+If you are in C++ and didn't come from JS, use `MOZ_LOG` with module `fog`.
+
+#### Rust
+
+Use the logging macros from `log`, e.g. `log::info!` or `log::error!`.
+Remember that, no matter the log level, `log::debug!` and `log::trace!`
+[will not appear in non-debug builds](/testing-rust-code/index.md#gecko-logging)
+
+If you are logging due to a situation caused by and fixable by a developer using the API,
+use `log::error!(...)`. Otherwise, use a quieter level.
+
+## `about:glean`
+
+`about:glean` is a page in a running Firefox that allows you to
+[debug the Glean SDK](https://mozilla.github.io/glean/book/user/debugging/index.html)
+in Firefox Desktop.
+It does this through the displayed user interface (just follow the instructions).
+
+## Linting
+
+To keep in accordance with Mozilla's various and several Coding Styles,
+we rely on `mach lint`.
+
+To lint the code in the "usual" way, automatically fixing where possible, run:
+`./mach lint -W -w -o --fix`
+
+This should keep you from checking in code that will automatically be backed out.
+
+## Rust
+
+Not all of our Rust code can be tested in a single fashion, unfortunately.
+
+### Using `rusttests` (Treeherder symbol `Br` (a build task))
+
+If the crate you're testing has no Gecko symbols you can write standard
+[Rust tests](https://doc.rust-lang.org/book/ch11-01-writing-tests.html).
+
+This supports both unit tests
+(inline in the file under test) and integration tests
+(in the `tests/` folder in the crate root).
+Metric type tests are currently written as unit tests inline in the file,
+as they require access to the metric ID, which should only be exposed in tests.
+
+To run FOG's `rusttests` suite use `mach rusttests`
+
+If the crate uses only a few Gecko symbols, they may use the
+`with_gecko` feature to conditionally use them.
+This allows the crate to test its non-Gecko-adjacent code using Rust tests.
+(You will need to cover the Gecko-adjacent code via another means.)
+
+**Note:** Some FOG rusttests panic on purpose. They print stack traces to stdout.
+If the rusttests fail and you see a stack trace,
+double-check it isn't from a purposefully-panicking test.
+
+**Note:** If a test fails, it is very likely they'll poison the test lock.
+This will cause all subsequent tests that attempt to take the test lock
+(which is all of them)
+to also fail due to `PoisonError`s. They can be safely ignored.
+
+### Using `gtest` (Treeherder symbol `GTest` (a build task))
+
+Because Gecko symbols aren't built for the
+`rusttests` build,
+any test that is written for code that uses Gecko symbols should be written as a
+[`gtest`](https://github.com/google/googletest)
+in `toolkit/components/glean/tests/gtest/`.
+You can write the actual test code in Rust.
+It needs to be accompanied by a C++ GTest that calls a C FFI-exported Rust function.
+See [Testing & Debugging Rust Code](/testing-rust-code/index.md) for more.
+See [`toolkit/components/glean/tests/gtest/TestFog.cpp`](https://searchfox.org/mozilla-central/source/toolkit/components/glean/tests/gtest/TestFog.cpp)
+and [`toolkit/components/glean/tests/gtest/test.rs`](https://searchfox.org/mozilla-central/source/toolkit/components/glean/tests/gtest/test.rs)
+for an example.
+
+By necessity these can only be integration tests against the compiled crate.
+
+**Note:** When adding a new test file, don't forget to add it to
+`toolkit/components/glean/tests/gtest/moz.build` and use the
+`FOG` prefix in your test names
+(e.g. `TEST(FOG, YourTestName) { ... }`).
+
+To run FOG's Rust `gtest` suite use `mach gtest FOG.*`
+
+## Python (Treeherder symbol `py3(fp)` aka `source-test-python-fog`)
+
+The [Glean Parser](https://github.com/mozilla/glean_parser/)
+has been augmented to generate FOG-specific APIs for Glean metrics.
+This augmentation is tested by running:
+
+`mach test toolkit/components/glean/tests/pytest`
+
+These tests require Python 3+.
+If your default Python is Python 2, you may need to instead run:
+
+`python3 mach python-test toolkit/components/glean/tests/pytest`
+
+These tests check the code generator output against known good file contents.
+If you change the code generator the files will need an update.
+Run the test suite with the `UPDATE_EXPECT` environment variable set to do that automatically:
+
+`UPDATE_EXPECT=1 mach test toolkit/components/glean/tests/pytest`
+
+## C++ (Treeherder symbol `GTest` (a build task))
+
+To test the C++ parts of FOG's implementation
+(like metric types)
+you should use `gtest`.
+FOG's `gtest` tests are in
+[`gtest/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/gtest/).
+
+You can either add a test case to an existing file or add a new file.
+If you add a new file, remember to add it to the
+[`moz.build`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/gtest/moz.build))
+or the test runner won't be able to find it.
+
+All tests should start with `FOG` so that all tests are run with
+`./mach gtest FOG*`.
+
+## JS (Treeherder symbol `X(Xn)` for some number `n`)
+
+To test the JS parts of FOG's implementation
+(like metric types)
+you should use `xpcshell`.
+FOG's `xpcshell` tests are in
+[`xpcshell/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/xpcshell).
+
+You can either add a test case to an existing file or add a new file.
+If you add a new file, remember to add it to the
+[`xpcshell.ini`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/xpcshell/xpcshell.ini)
+or the test runner will not be able to find it.
+
+To run FOG's JS tests, run:
+`./mach test toolkit/components/glean/tests/xpcshell`
+
+## Non-content-process multiprocess (Browser Chrome Mochitests with Treeherder symbol `M(bcN)` for some number `N`)
+
+To test e.g. the GPU process support you need a full Firefox browser:
+xpcshell doesn't have the flexibility.
+To test that and have access to privileged JS (i.e. `Glean` and `FOG` APIs),
+we use browser-chrome-flavoured mochitests you can find in
+[`browser/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/browser).
+
+If you need to add a new test file, remember to add it to the
+[`browser.ini`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/glean/tests/browser/browser.ini)
+manifest, or the test runner will not be able to find it.
+
+To run FOG's browser chrome tests, run:
+`./mach test toolkit/components/glean/tests/browser`
+
+## Integration (Marionette, borrowing `telemetry-tests-client` Treeherder symbol `tt(c)`)
+
+To test pings (See [bug 1681742](https://bugzilla.mozilla.org/show_bug.cgi?id=1681742))
+or anything that requires one or more full browsers running,
+we use the `telemetry-tests-client` suite in
+[`toolkit/components/telemetry/tests/marionette/tests/client/`](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/tests/marionette/tests/client/).
+
+For more information on this suite, look to
+[Firefox Telemetry's Test Documentation](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/internals/tests.html#integration-tests-telemetry-tests-client-and-telemetry-integration-tests)
+and
+[Marionette's Documentation](/testing/marionette/Testing.md).
+
+To run these integration tests, run:
+`./mach telemetry-tests-client toolkit/components/telemetry/tests/marionette/tests/client/`
+
+To capture the Firefox under test's logs, use the `--gecko-log` parameter.
+For example, to echo to stdout:
+`./mach telemetry-tests-client toolkit/components/telemetry/tests/marionette/tests/client/test_fog* --gecko-log "-"`
+
+**Note:** Running the `tt(c)` suite in this way ignored skip directives in the manifest.
+This means that you might run tests that are not expected to succeed on your platform.
+Check `toolkit/components/telemetry/tests/marionette/tests/client/manifest.ini` for details.