From 6bf0a5cb5034a7e684dcc3500e841785237ce2dd Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 7 Apr 2024 19:32:43 +0200 Subject: Adding upstream version 1:115.7.0. Signed-off-by: Daniel Baumann --- .../android/docs/geckoview/contributor/junit.rst | 373 +++++++++++++++++++++ 1 file changed, 373 insertions(+) create mode 100644 mobile/android/docs/geckoview/contributor/junit.rst (limited to 'mobile/android/docs/geckoview/contributor/junit.rst') diff --git a/mobile/android/docs/geckoview/contributor/junit.rst b/mobile/android/docs/geckoview/contributor/junit.rst new file mode 100644 index 0000000000..bf8cfd4615 --- /dev/null +++ b/mobile/android/docs/geckoview/contributor/junit.rst @@ -0,0 +1,373 @@ +.. -*- Mode: rst; fill-column: 80; -*- + +==================== +Junit Test Framework +==================== + +GeckoView has `a lot +`_ +of `custom +`_ +code that is used to run junit tests. This document is an overview of what this +code does and how it works. + +.. contents:: Table of Contents + :depth: 2 + :local: + +Introduction +============ + +`GeckoView `_ is an Android Library that can be used to +embed Gecko, the Web Engine behind Firefox, in applications. It is the +foundation for Firefox on Android, and it is intended to be used to build Web +Browsers, but can also be used to build other types of apps that need to +display Web content. + +GeckoView itself has no UI elements besides the Web View and uses Java +interfaces called "delegates" to let embedders (i.e. apps that use GeckoView) +implement UI behavior. + +For example, when a Web page's JavaScript code calls ``alert('Hello')`` the +embedder will receive a call to the `onAlertPrompt +`_ +method of the `PromptDelegate +`_ +interface with all the information needed to display the prompt. + +As most delegate methods deal with UI elements, GeckoView will execute them on +the UI thread for the embedder's convenience. + +GeckoResult +----------- + +One thing that is important to understand for what follows is `GeckoResult +`_. +``GeckoResult`` is a promise-like object that is used throughout the GeckoView +API, it allows embedders to asynchronously respond to delegate calls and +GeckoView to return results asynchronously. This is especially important for +GeckoView as it never provides synchronous access to Gecko as a design +principle. + +For example, when installing a WebExtension in GeckoView, the resulting +`WebExtension +`_ +object is returned in a ``GeckoResult``, which is completed when the extension +is fully installed: + +.. code:: java + + public GeckoResult install(...) + +To simplify memory safety, ``GeckoResult`` will always `execute callbacks +`_ +in the same thread where it was created, turning asynchronous code into +single-threaded javascript-style code. This is currently `implemented +`_ +using the Android Looper for the thread, which restricts ``GeckoResult`` to +threads that have a looper, like the Android UI thread. + +Testing overview +---------------- + +Given that GeckoView is effectively a translation layer between Gecko and the +embedder, it's mostly tested through integration tests. The vast majority of +the GeckoView tests are of the form: + +- Load simple test web page +- Interact with the web page through a privileged JavaScript test API +- Verify that the right delegates are called with the right inputs + +and most of the test framework is built around making sure that these +interactions are easy to write and verify. + +Tests in GeckoView can be run using the ``mach`` interface, which is used by +most Gecko tests. E.g. to run the `loadUnknownHost +`_ +test in ``NavigationDelegateTest`` you would type on your terminal: + +.. code:: shell + + ./mach geckoview-junit org.mozilla.geckoview.test.NavigationDelegateTest#loadUnknownHost + +Another way to run GeckoView tests is through the `Android Studio IDE +`_. By running tests this way, however, +some parts of the test framework are not initialized, and thus some tests +behave differently or fail, as will be explained later. + +Testing envelope +---------------- + +Being a library, GeckoView has a natural, stable, testing envelope, namely the +GeckoView API. The vast majority of GeckoView tests only use +publicly-accessible APIs to verify the behavior of the API. + +Whenever the API is not enough to properly test behavior, the testing framework +offers targeted "privileged" testing APIs. + +Using a restricted, stable testing envelope has proven over the years to be an +effective way of writing consistent tests that don't break upon refactoring. + +Testing Environment +------------------- + +When run through ``mach``, the GeckoView junit tests run in a similar +environment as mochitests (a type of Web regression tests used in Gecko). They +have access to the mochitest web server at `example.com`, and inherit most of +the testing prefs and profile. + +Note the environment will not be the same as mochitests when the test is run +through Android Studio, the prefs will be inherited from the default GeckoView +prefs (i.e. the same prefs that would be enabled in a consumer's build of +GeckoView) and the mochitest web server will not be available. + +Tests account for this using the `isAutomation +`_ +check, which essentially checks whether the test is running under ``mach`` or +via Android Studio. + +Unlike most other junit tests in the wild, GeckoView tests run in the UI +thread. This is done so that the GeckoResult objects are created on the right +thread. Without this, every test would most likely include a lot of blocks that +run code in the UI thread, adding significant boilerplate. + +Running tests on the UI thread is achieved by registering a custom ``TestRule`` +called `GeckoSessionTestRule +`_, +which, among other things, `overrides the evaluate +`_ +method and wraps everything into a ``instrumentation.runOnMainSync`` call. + +Verifying delegates +=================== + +As mentioned earlier, verifying that a delegate call happens is one of the most +common assertions that a GeckoView test makes. To facilitate that, +``GeckoSessionTestRule`` offers several ``delegate*`` utilities like: + +.. code:: java + + sessionRule.delegateUntilTestEnd(...) + sessionRule.delegateDuringNextWait(...) + sessionRule.waitUntilCalled(...) + sessionRule.forCallbacksDuringWait(...) + +These all take an arbitrary delegate object (which may include multiple +delegate implementations) and handle installing and cleaning up the delegate as +needed. + +Another set of facilities that ``GeckoSessionTestRule`` offers allow tests to +synchronously ``wait*`` for events, e.g. + +.. code:: java + + sessionRule.waitForJS(...) + sessionRule.waitForResult(...) + sessionRule.waitForPageStop(...) + +These facilities work together with the ``delegate*`` facilities by marking the +``NextWait`` or the ``DuringWait`` events. + +As an example, a test could load a page using ``session.loadUri``, wait until +the page has finished loading using ``waitForPageStop`` and then verify that +the expected delegate was called using ``forCallbacksDuringWait``. + +Note that the ``DuringWait`` here always refers to the last time a ``wait*`` +method was called and finished executing. + +The next sections will go into how this works and how it's implemented. + +Tracking delegate calls +----------------------- + +One thing you might have noticed in the above section is that +``forCallbacksDuringWait`` moves "backward" in time by replaying the delegates +called that happened while the wait was being executed. +``GeckoSessionTestRule`` achieves this by `injecting a proxy object +`_ +into every delegate, and `proxying every call +`_ +to the current delegate according to the ``delegate`` test calls. + +The proxy delegate `is built +`_ +using the Java reflection's ``Proxy.newProxyInstance`` method and receives `a +callback +`_ +every time a method on the delegate is being executed. + +``GeckoSessionTestRule`` maintains a list of `"default" delegates +`_ +used in GeckoView, and will `use reflection +`_ +to match the object passed into the ``delegate*`` calls to the proxy delegates. + +For example, when calling + +.. code:: java + + sessionRule.delegateUntilTestEnd(object : NavigationDelegate, ProgressDelegate {}) + +``GeckoSessionTestRule`` will know to redirect all ``NavigationDelegate`` and +``ProgressDelegate`` calls to the object passed in ``delegateUntilTestEnd``. + +Replaying delegate calls +------------------------ + +Some delegate methods require output data to be passed in by the embedder, and +this requires extra care when going "backward in time" by replaying the +delegate's call. + +For example, whenever a page loads, GeckoView will call +``GeckoResult onLoadRequest(...)`` to know if the load can +continue or not. When replaying delegates, however, we don't know what the +value of ``onLoadRequest`` will be (or if the test is going to install a +delegate for it, either!). + +What ``GeckoSessionTestRule`` does, instead, is to `return the default value +`_ +for the delegate method, and ignore the replayed delegate method return value. +This can be a little confusing for test writers, for example this code `will +not` stop the page from loading: + +.. code:: java + + session.loadUri("https://www.mozilla.org") + sessionRule.waitForPageStop() + sessionRule.forCallbacksDuringWait(object : NavigationDelegate { + override fun onLoadRequest(session: GeckoSession, request: LoadRequest) : + GeckoResult? { + // this value is ignored + return GeckoResult.deny() + } + }) + +as the page has already loaded by the time the ``forCallbacksDuringWait`` call is +executed. + +Tracking Waits +-------------- + +To track when a ``wait`` occurs and to know when to replay delegate calls, +``GeckoSessionTestRule`` `stores +`_ +the list of delegate calls in a ``List`` object, where +``CallRecord`` is a class that has enough information to replay a delegate +call. The test rule will track the `start and end index +`_ +of the last wait's delegate calls and `replay it +`_ +when ``forCallbacksDuringWait`` is called. + +To wait until a delegate call happens, the test rule will first `examine +`_ +the already executed delegate calls using the call record list described above. +If none of the calls match, then it will `wait for new calls +`_ +to happen, using ``UiThreadUtils.waitForCondition``. + +``waitForCondition`` is also used to implement other type of ``wait*`` methods +like ``waitForResult``, which waits until a ``GeckoResult`` is executed. + +``waitForCondition`` runs on the UI thread, and it synchronously waits for an +event to occur. The events it waits for normally execute on the UI thread as +well, so it `injects itself +`_ +in the Android event loop, checking for the condition after every event has +executed. If no more events remain in the queue, `it posts a delayed 100ms +`_ +task to avoid clogging the event loop. + +Executing Javascript +==================== + +As you might have noticed from an earlier section, the test rule allows tests +to run arbitrary JavaScript code using ``waitForJS``. The GeckoView API, +however, doesn't offer such an API. + +The way ``waitForJS`` and ``evaluateJS`` are implemented will be the focus of +this section. + +How embedders run javascript +---------------------------- + +The only supported way of accessing a web page for embedders is to `write a +built-in WebExtension +`_ +and install it. This was done intentionally to avoid having to rewrite a lot of +the Web-Content-related APIs that the WebExtension API offers. + +GeckoView extends the WebExtension API to allow embedders to communicate to the +extension by `overloading +`_ +the native messaging API (which is not normally implemented on mobile). +Embedders can register themselves as a `native app +`_ +and the built-in extension will be able to `exchange messages +`_ +and `open ports +`_ +with the embedder. + +This is still a controversial topic among smaller embedders, especially solo +developers, and we have discussed internally the possibility to expose a +simpler API to run one-off javascript snippets, similar to what Chromium's +WebView offers, but nothing has been developed so far. + +The test runner extension +------------------------- + +To run arbitrary javascript in GeckoView, the test runner installs a `support +extension +`_. + +The test framework then `establishes +`_ +a port for the background script, used to run code in the main process, and a +port for every window, to be able to run javascript on test web pages. + +When ``evaluateJS`` is called, the test framework will send `a message +`_ +to the extension which then `calls eval +`_ +on it and returns the `JSON`-stringified version of the result `back +`_ +to the test framework. + +The test framework also supports promises with `evaluatePromiseJS +`_. +It works similarly to ``evaluateJS`` but instead of returning the stringified +value, it `sets +`_ +the return value of the ``eval`` call into the ``this`` object, keyed by a +randomly-generated UUID. + +.. code:: java + + this[uuid] = eval(...) + +``evaluatePromiseJS`` then returns an ``ExtensionPromise`` Java object which +has a ``getValue`` method on it, which will essentially execute `await +this[uuid] +`_ +to get the value from the promise when needed. + +Beyond executing javascript +--------------------------- + +A natural way of breaking the boundaries of the GeckoView API is to run a +so-called "experiment extension". Experiment extensions have access to the full +Gecko front-end, which is written in JavaScript, and don't have limits on what +they can do. Experiment extensions are essentially what old add-ons used to be +in Firefox, very powerful and very dangerous. + +The test runner uses experiments to offer `privileged APIs +`_ +to tests like ``setPref`` or ``getLinkColor`` (which is not normally available +to websites for privacy concerns). + +Each privileged API is exposed as an `ordinary Java API +`_ +and the test framework doesn't offer a way to run arbitrary chrome code to +discourage developers from relying too much on implementation-dependent +privileged code. -- cgit v1.2.3