diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-15 03:35:49 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-05-15 03:35:49 +0000 |
| commit | d8bbc7858622b6d9c278469aab701ca0b609cddf (patch) | |
| tree | eff41dc61d9f714852212739e6b3738b82a2af87 /mobile/android/fenix/docs | |
| parent | Releasing progress-linux version 125.0.3-1~progress7.99u1. (diff) | |
| download | firefox-d8bbc7858622b6d9c278469aab701ca0b609cddf.tar.xz firefox-d8bbc7858622b6d9c278469aab701ca0b609cddf.zip | |
Merging upstream version 126.0.
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'mobile/android/fenix/docs')
39 files changed, 2169 insertions, 0 deletions
diff --git a/mobile/android/fenix/docs/Acronym-dictionary.md b/mobile/android/fenix/docs/Acronym-dictionary.md new file mode 100644 index 0000000000..1b5c0f2d3f --- /dev/null +++ b/mobile/android/fenix/docs/Acronym-dictionary.md @@ -0,0 +1,19 @@ +# Acronym Dictionary + +Explanations of common acronyms that are seen in the Fenix project boards and issues. + +**AC**: "[Android Components](https://github.com/mozilla-mobile/android-components)", the Mozilla team responsible for componentization. + +**AS**: "[Application Services](https://github.com/mozilla/application-services)", the Mozilla team responsible for sync and storage (logins, bookmarks, etc). + +**CFR**: "Contextual feature recommendation", i.e. popup windows. + +**CTA**: "Call to action", a marketing/product term for things that pop up and ask for user interaction. + +**ETP**: "Enhanced tracking protection". The shield icon. + +**PWA**: "[Progressive web application](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps)", Fenix can act as a host for these applications. + +**FCM**: "Firebase Cloud Messaging", is the Firebase service used to deliver WebPush messaging to web apps and Firefox Send Tab, to Android devices that have [Google Play Services](https://en.wikipedia.org/wiki/Google_Play_Services). + +**PTR**: "[Pull-to-refresh](https://developer.android.com/develop/ui/views/touch-and-input/swipe)", is the feature to refresh the website the user is currently on when pulling down from the top of the page. diff --git a/mobile/android/fenix/docs/Addressing-a-performance-regression.md b/mobile/android/fenix/docs/Addressing-a-performance-regression.md new file mode 100644 index 0000000000..27a391df03 --- /dev/null +++ b/mobile/android/fenix/docs/Addressing-a-performance-regression.md @@ -0,0 +1,106 @@ +# Addressing a performance regression + +To keep Fenix in a good shape, the performance team weekly runs multiple performance tests to identify regressions. The results are kept and maintained [here](https://earthangel-b40313e5.influxcloud.net/d/s3AT6t7nk/fenix-startup-nightly-via-backfill-py?orgId=1). + +The main tests are: + +* MAIN first frame: +Simulates a COLD MAIN (app icon launch) launch to report FullyDrawn, e.g when the user sees the app fully drawn after launching it. + +* COLD VIEW Nav start: +An app link launch to load a page. It measures until "navigation starts" which is an internal Gecko event that indicates we're starting to load a page. + +### What to do after a performance regression is reported. + +Weekly after, the performance team runs the tests, if there is any regression, they will open a ticket, providing the dates between when was the last non-regressing and regressing version (an [example](https://github.com/mozilla-mobile/fenix/issues/25253) ticket). **These dates are important** for us to discover which commit introduced the regression. As we would like to identify which commit is the offending one, we need to bitset the commit range from non-regressing to regressing version. Fortunately for us, the performance team has a tool that can help us with that it’s called [backfil](https://github.com/mozilla-mobile/perf-tools/blob/main/backfill.py). + +The tool can take a commit range start/end, build all the APKs, run the performance tests and provide the same data that it’s plotted [here](https://earthangel-b40313e5.influxcloud.net/d/s3AT6t7nk/fenix-startup-nightly-via-backfill-py?orgId=1). With it we can identify the offending commit. + +### Install the required dependencies. + +* Pull the [perf-tools](https://github.com/mozilla-mobile/perf-tools) repository, and on it follow the [configuration instructions](https://github.com/mozilla-mobile/perf-tools#configuration). +* Make you can run `adb` on the terminal, as the performance tools we use it through the scripts. + +### Finding the regressing commit. + +Now that you have all the dependencies installed, we will need to find the commit hash, where there the regression was reported and when the regression was not present, as [backfill](https://github.com/mozilla-mobile/perf-tools/blob/main/backfill.py), need them as parameters. + +On the reported [ticket](https://github.com/mozilla-mobile/fenix/issues/25253), the performance team gave us the date when the regression was introduced (5/10), and when it wasn't present (5/9). + + + +We can find each commit hash date by downloading the APKs, from Task Cluster and going to the about page. + +For example: + +**For 05/10**. +[https://firefox-ci-tc.services.mozilla.com/tasks/index/mobile.v2.fenix.nightly.2022.05.10.latest/armeabi-v7a](https://firefox-ci-tc.services.mozilla.com/tasks/index/mobile.v2.fenix.nightly.2022.06.06.latest/armeabi-v7a) + +<img width="503" alt="image" src="https://user-images.githubusercontent.com/773158/174894842-3341aff6-185f-4402-9c30-3cd5c56dcd3d.png"> + +From it, we can find [2f7f5988f](https://github.com/mozilla-mobile/fenix/commit/2f7f5988fccad2cf2043eed4b6849b32a4c76048) when the regression was spotted. + +**For 05/09**. +[https://firefox-ci-tc.services.mozilla.com/tasks/index/mobile.v2.fenix.nightly.2022.05.10.latest/armeabi-v7a](https://firefox-ci-tc.services.mozilla.com/tasks/index/mobile.v2.fenix.nightly.2022.06.06.latest/armeabi-v7a) + +<img width="499" alt="image" src="https://user-images.githubusercontent.com/773158/174894896-a3a66219-ba0e-4174-b4fa-842e049e8d6d.png"> + +When the regression was not present [98455c01e](https://github.com/mozilla-mobile/fenix/commit/98455c01eeba7c63775f18817cd079f5d08b4513) + +Using the commits we can construct this range: +https://github.com/mozilla-mobile/fenix/compare/98455c01eeba7c63775f18817cd079f5d08b4513...2f7f5988fccad2cf2043eed4b6849b32a4c76048 +<img width="856" alt="commit range" src="https://user-images.githubusercontent.com/773158/175076734-9c585df2-7c5d-4b17-9135-9447643fb5d0.png"> + + + +With it we can see each commit that could introduced the regression. + +### Using backfill.py + +With the info that we found above, execute `backfill.py` + +``` +perf-tools-main % python3 backfill.py --tests cold_main_first_frame --startcommit 98455c01eeba7c63775f18817cd079f5d08b4513 --endcommit 2f7f5988fccad2cf2043eed4b6849b32a4c76048 --git_remote_name https://github.com/mozilla-mobile/fenix.git --repository_to_test_path ../fenix fenix nightly armeabi-v7a commitsRange +``` + +Where: +* **cold_main_first_frame**: it's the test we would like to run, we could also pass `cold_view_nav_start` depending on the regression type. +* **--startcommit**: it's the commit before the regression. +* **--endcommit** it's the commit where the regression appears. +* **--repository_to_test_path** is the path where your local Fenix repository is. + + +**Note**: Make sure your repository includes all the tokens (Sentry, Nimbus, … etc) that we include in our release builds, as not adding them could affect the test results, as we want the APKs to be the same [experience as normal users will have](https://wiki.mozilla.org/Performance/Fenix#How_to_measure_what_users_experience). Part of this is making sure you have [autosignReleaseWithDebugKey in your local.properties](https://github.com/mozilla-mobile/fenix#automatically-sign-release-builds). + + +🕐 Be patient, as we will have to build an APK for each possible commit in the range and for this range there [are 19 commits](https://github.com/mozilla-mobile/fenix/compare/98455c01eeba7c63775f18817cd079f5d08b4513...2f7f5988fccad2cf2043eed4b6849b32a4c76048) then we will build 19 APKs, and run the performance test for each one. + + +As the script progress, we will start to see some activity on the `perf-tools` **directory**, as each APKs will go there with the format `apk_commit_<HASH>.apk` + + + +After all the APKs are built, the script will continue with the testing phase, it will run the tests per each commit/APKs, and create a directory named `backfill_output` where it will create two **.txt files per commit** `apk_commit_<HASH>-cold_main_first_frame-analysis.txt` and `apk_commit_<HASH>-cold_main_first_frame-durations.txt` + + + +These files are the output of the script: + +* **Cold_main_first_frame-analysis.txt**: Will contain key information about the test results like max,mean,median, and min. + +* **Cold_main_first_frame-durations.txt**: Will contain the raw information of each repetition of the test. + + +With these files, we can identify which commit, introduced the regression by checking file by file which results are closer to the ones reported one the regression ticket. + +After we found the regressing commit, we just have to update the ticket, posting our finding and tagging the person that introduced to research how to optimize the patch. Normally if the regression is significant we will ask to undo the commit until the patch is optimized. + +## Extra tips: +* In case you would like to run the same test that are run via `backfil` for an specific APK, you can find more information [here](https://wiki.mozilla.org/Performance/Fenix/Performance_reviews#Measuring_cold_start_up_duration). +* If you would like to graph the results you can use `python3 analyze_durations.py --graph results.txt`. +*Just keep in mind, the results provide from the Performance team are from running the tests on a Moto G 5. Running on a more powerful device could cause the results to diverge. +* In case, you need to start recording a profile in startup https://profiler.firefox.com/docs/#/./guide-startup-shutdown?id=firefox-for-android. + + +### Identifying the source of the regression. +Our main task, when looking for a performance regression is just to identify the faulty commit, but if we would like to figure out what is the exact cause, we will need to take [profile](https://wiki.mozilla.org/Performance/Fenix/Performance_reviews#Profile) from regressing version, and the version before to try to identify what could be causing the issue, checking the code path of the regressing commit could give us some hints to where to look. diff --git a/mobile/android/fenix/docs/Architecture-Decisions.md b/mobile/android/fenix/docs/Architecture-Decisions.md new file mode 100644 index 0000000000..10c11dc2d6 --- /dev/null +++ b/mobile/android/fenix/docs/Architecture-Decisions.md @@ -0,0 +1,94 @@ +# Architecture Decisions + +For an overview of our current architecture, please see [this document](https://github.com/mozilla-mobile/fenix/blob/master/docs/architecture-overview.md) + +--- + +These are some of the major architecture decisions we've made so far in Fenix. [Why?](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions) + +--- + +## Overview + +Several apps have suffered from insufficient attention to appropriate Android application architecture. This leads to spaghetti code, God objects, and leaky abstractions that make testing and maintaining our apps significantly more costly and challenging. + +--- + +## Goals + +Our architecture should: + +* Have classes which aim to fulfill a single responsibility, thereby separating concerns. +* Improve the ability of developers to write effective automated unit and UI tests. +* Make the bulk of application code more readable, easing onboarding of contributors. +* Improve the ability to debug application errors with useful stack traces. +* Be amenable to code reuse when desirable. +* Handle A/B experimentation without vast code branches or confusion. + +--- + +## Special Concerns + +As a browser, Fenix will need to interface with the Android Components and the GeckoView rendering engine. This means any architecture we choose must not require that all components of the app are implemented similarly. Also, our application state will need to synchronize with the engine’s state as the two will not always be in perfect sync due to hidden internals. + +--- + +## Component Architecture + +### Context + +A/B testing of layout and UX is a fundamental necessity of Fenix. We have a lot of hypotheses needing validation and actual usage data about what might make a browser better to acquire and retain more users. A lot of questionable decisions have been made in older mobile browsers that do not seem ideal for mobile devices, but we need data to tell us if our assumptions are correct. + +### Decision + +We did a review of modern app architectures used by companies throughout the tech industry and came across the Netflix "componentization" architecture. Netflix had a special desire to A/B test a lot of different layouts for their app's user interface and built their architecture for this express purpose. + +Netflix's architecture moves all UX-related code away from activities and fragments. Instead, fragments subscribe to components through RxJava and components inflate themselves. + +The actual components' inflated views are dropped into ConstraintLayouts and are tied together by applying programmatic ConstraintSets. ConstraintLayouts and ConstraintSets also tie nicely into the new and ultra-powerful MotionLayouts to empower rich animations. + +Here are some videos of Juliano Moraes of Netflix describing their architecture: + +[DroidCon NYC part 1 video](https://www.youtube.com/watch?v=dS9gho9Rxn4) + +[DroidCon SF part 2 video](https://www.youtube.com/watch?v=1cWwfh_5ZQs) + +Here's a [sample repository](https://github.com/julianomoraes/componentizationArch) demonstrating in a very simple form how it functions. + +The goal of software architecture is to minimize the cost of change. This decision is possibly the biggest factor for reducing the cost of changes to Fenix. It also plays well with the Android Components project, which provides so many of the components that will make up this project. + +### Consequences + +We will package the UI into components which are reusable and can be remixed for A/B layout tests. + +We will keep all UI/UX code out of activities and fragments to obey the [Single Responsibility Principle](https://blog.cleancoder.com/uncle-bob/2014/05/08/SingleReponsibilityPrinciple.html). These classes will exist to handle OS business logic and to bind components. + +One downside of this is the extra cost of getting contributors to avoid making UI/UX changes directly from activities and fragments. Another is programmatic layouts don't play well with previewing layouts. We'll all need to learn how to use ConstraintLayouts and ConstraintSets effectively. + +--- + +## Localized, MVI (Model-View-Intent) Unidirectional Architecture + +### Context + +Race conditions are the bane of Android apps everywhere. They often happen because multiple systems cannot agree about state. + +### Decision + +The best solution is to have all state changes flow in a functional, reactive manner out from a single source of truth with careful thread locking as required. This solution will be familiar to anyone who has worked with Redux or Flux. + +Within each component, we'd like state changes to occur in a cycle. The user interface's contents are rendered by presenting an initial ViewState to the screen. When the user interacts with the app, a Change object is passed to a state reducer. The reducer function copies the current, immutable state with the requested changes and passes it to the Model. The View subscribes to these state changes and updates itself reactively. + +In this manner, the UX of the app only flows in a single direction from a source of truth. Because all changes happen in a serialized RxKotlin Observable, they will be applied in the order they happen. + +However, unlike some MVI architectures, we will not focus on keeping a global ViewState that encompasses all state. There are times when we want OS calls, NDK calls, and third party component calls to be the source of truth. Rather than trying to make a single ViewState that contains all state, we'll be able to observe a merged observable of all actions and state changes. This will be invaluable for debugging the app. + +Here's a [state diagram of the MVI architecture](https://staltz.com/img/mvi-unidir-ui-arch.jpg). + +### Consequences + +We will experiment with writing new components using MVI unidirectional principles. We will need to take care when passing data between components that we do not override a more authoritative source of truth. + +Because all changes can be represented by a single, merged and serialized Observable or Flowable, we should be able to use this for debugging. All ViewStates, Changes, and Actions/Intents will be easily loggable to observe the causes of state issues. + +--- diff --git a/mobile/android/fenix/docs/Crash-Monitoring.md b/mobile/android/fenix/docs/Crash-Monitoring.md new file mode 100644 index 0000000000..c40f3fcbe5 --- /dev/null +++ b/mobile/android/fenix/docs/Crash-Monitoring.md @@ -0,0 +1,46 @@ +# Crash Monitoring + +## Important +* The main goal here is not to file an issue for every single distinct crash report, but to find regressions of new problems that need to be addressed. +* Once you're familiar with the process this should not take more than 10 mins in the morning. +* Quick links that you should check everyday [Sentry Query](https://sentry.io/organizations/mozilla/issues/?groupStatsPeriod=auto&page=0&project=6295546&query=is%3Aunresolved+level%3Afatal+firstSeen%3A-1w&sort=freq&statsPeriod=14d) and [Socorro Query](https://crash-stats.mozilla.org/topcrashers/?product=Fenix&days=3&_range_type=build&process_type=any) + +## Things to note before starting: +* Since we are focused on Java crashes, Sentry currently is the better choice. +* Ignore `level:info` issues (blue labels in Sentry). These issues are informational only. + +## What to do when monitoring crashes: +* Look at Sentry Fenix-nightly overview. Go though trending issues. [Sentry Dashboard](https://sentry.io/organizations/mozilla/projects/fenix-nightly/?project=6295546). +* Look at Sentry custom search [Sentry Query](https://sentry.io/organizations/mozilla/issues/?groupStatsPeriod=auto&page=0&project=6295546&query=is%3Aunresolved+level%3Afatal+firstSeen%3A-1w&sort=freq&statsPeriod=14d). +* Sign up for https://groups.google.com/a/mozilla.org/g/stability to get a daily email on the stability of Fenix. + +## How to determine a crash requires actions: +* Crashes that have level either Fatal or Error. +* Crashes that are occurring on latest Fenix and A-C versions. +* Crashes that are spiking. +* Crashes that are new. +* Crashes that happen repeatedly and often. +* Crashes that are happening to multiple users. + +## When a crash report requires actions: +* Is this a crash due to a recent change? If so, contact the developer. + * The histogram on the right side can help determine this along with checking the Firefox-Beta and Firefox Sentry products. +* Triage the crash to determine if the issue is real and requires a Bugzilla issue to track it. + * When filing an issue add a link to it as a comment in the Sentry crash for the products (nightly, beta, release) where the crash appears. +* Notify the relevant teams on Slack/Matrix that there's a new crash in Nightly that needs urgent attention, e.g. **#synced-client-integrations** for all things involving application services (A-S), **#nimbus-rust-sdk** for Nimbus, and **[GeckoView on Matrix](https://chat.mozilla.org/#/room/#geckoview:mozilla.org)**. + +## What can you do to help when not monitoring crashes +* If you recently landed a new module / change that is significant, contact the crash monitor so they are aware of it. + +## Crash monitoring with Socorro +* Look at [Top Crashers for Fenix Nightly](https://crash-stats.mozilla.org/topcrashers/?product=Fenix&days=3&_range_type=build&process_type=any) for reports on Nightly builds. + * This will return zero results if GV build ID is greater than 3 days old. Change to the 7 day view and ask #releaseduty-mobile in Slack about the GV upgrade task being broken. +* Use Sentry and Bugzilla to determine if the crash has already been reported. If a Bugzilla bug has been filed for a crash, a link to the bug should be listed in Socorro's "Bugzilla IDs" column. +* If the crash is new and the volume is high, then consider filing an issue using the crash-stats Bugzilla tab from a crash ID. + +## How to file bugs in Bugzilla from Socorro +* Go to the crash report. +* "Reports" tab. +* Select a report. +* Select the "Bugzilla" tab. +* Just above the header "Related Bugs", you will see a "Create a bug" option. Select the product that matches the crash report. diff --git a/mobile/android/fenix/docs/Creating-a-release-branch.md b/mobile/android/fenix/docs/Creating-a-release-branch.md new file mode 100644 index 0000000000..16b5da7cd7 --- /dev/null +++ b/mobile/android/fenix/docs/Creating-a-release-branch.md @@ -0,0 +1,3 @@ +# Creating a release branch + +Please refer to https://mozac.org/contributing/release-checklist for creating a Beta release branch for the firefox-android monorepo. diff --git a/mobile/android/fenix/docs/Data-Practices.md b/mobile/android/fenix/docs/Data-Practices.md new file mode 100644 index 0000000000..78d71777ad --- /dev/null +++ b/mobile/android/fenix/docs/Data-Practices.md @@ -0,0 +1,25 @@ +# Data Practices + +This document outlines how data is collected in Firefox Preview. + +## Telemetry + +When a user has "Telemetry" enabled under Data Choices in the browser settings, Firefox Preview sends a "core" ping and an "event" ping to Mozilla's telemetry service. "core" ping using the same format documented on firefox-source-docs.mozilla.org. + +[Here](https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md) is a list of Event Pings, Metrics Pings, and Activation Pings. + +**User can disable telemetry by turning the Telemetry toggle off under Data Choices.** + + +*** +## Adjust + +See [here](https://github.com/mozilla-mobile/fenix/wiki/Adjust-Usage) for details on Adjust usage in Firefox Preview. + +*** + +## Sentry + +Sentry collects a stack trace for each crash in Fenix. + +If the user has "Telemetry" enabled under Data Choices in the browser settings, then Sentry collects breadcrumbs containing the name of each Android Fragment in the app. This helps an engineer diagnose the cause of the crash by seeing the internal names of screens visited before the crash occurred, i.e. Browser, Search, Home, etc. No information is stored about any arguments passed to any Fragments. diff --git a/mobile/android/fenix/docs/Development-Test-Plan.md b/mobile/android/fenix/docs/Development-Test-Plan.md new file mode 100644 index 0000000000..4c94c709a3 --- /dev/null +++ b/mobile/android/fenix/docs/Development-Test-Plan.md @@ -0,0 +1,25 @@ +# Development Test Plan + +It's important to know where we require tests and where they don't add significant value. This document intends to lay out that distinction. + +In general, it's not necessary to test that the Android framework does what it says it does. If a code path sets the value of a TextView, one does not need to write an Espresso or Robolectric test to check that the TextView was set. This is why the Android framework team at Google has tests. Espresso tests are expensive in terms of runtime and should be used to ensure that all units of our software are correctly integrated together as a whole. Even Robolectric imposes a noticeable cost on test runtime, but it has proper uses. + +## Components + +Our app architecture is tied together by components that render UI. Generally, it's smart to write a set of unit tests for every UI component, but to mock the view. This can be accomplished by creating a Spy for the component. One may observe inputs and outputs of the component under test using RxKotlin TestObservers. + +When testing Components, it's important to test any significant logic. If a reducer has more logic than mere Kotlin copy() operators, it should be tested. + +## UIViews + +When testing UIViews, we don't need to verify TextView set calls are made, but we should verify any branching logic that exists. If the ViewState is not basically ready to display, those transformations probably belong in the component and should be tested there. If the UIView class is simply a straight conduit from the ViewState directly to the UI, then there is no need for unit tests. + +## Fragments + +Fragments can get complicated. In theory, there should not be much View logic if the architecture is correctly integrated. One can mock out any components and the extension methods which perform layout and then use Robolectric to verify significant logic. + +Generally, there shouldn't be large, branching blocks directly inside Android lifecycle callbacks. These should be broken out into methods, which can be tested on their own. + +## Activities + +Activities should exist only as entry points to the app in order to display fragments. The logic should be limited and should be testable as Robolectric tests with TestNavigators for testing navigation. diff --git a/mobile/android/fenix/docs/Fennec-Migration.md b/mobile/android/fenix/docs/Fennec-Migration.md new file mode 100644 index 0000000000..3b8e488f22 --- /dev/null +++ b/mobile/android/fenix/docs/Fennec-Migration.md @@ -0,0 +1,136 @@ +# Fennec Migration + +Project board: https://github.com/orgs/mozilla-mobile/projects/40 + +## 📱 Testing + +⚠️ **Warning**: Replacing a _Fennec_ (Firefox for Android) installation with _Fenix_ (Firefox Preview) can (and at the time of writing this definitely **will**) result in **DATA LOSS**. Do not replace an installation of Fennec (Firefox for Android) that contains data you do not want to risk losing (e.g. open tabs, history, bookmarks, top sites, ..). + +## Release + +The following links point to the latest *Fenix* (Firefox Preview) builds (Nightly; from `main`) that are setup to **replace** a *Fennec* (Firefox for Android) release version (`org.mozilla.firefox`). + +* [ARM64/Aarch64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-production.latest/artifacts/public/build/arm64-v8a/geckoBeta/target.apk) +* [ARM devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-production.latest/artifacts/public/build/armeabi-v7a/geckoBeta/target.apk) +* [x86_64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-production.latest/artifacts/public/build/x86_64/geckoBeta/target.apk) +* [x86 devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-production.latest/artifacts/public/build/x86/geckoBeta/target.apk) + +## Beta + +The following links point to the latest *Fenix* (Firefox Preview) builds (Nightly; from `main`) that are setup to **replace** a *Fennec Beta* (Firefox for Android - Beta) release version (`org.mozilla.firefox.beta`). + +* [ARM64/Aarch64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-beta.latest/artifacts/public/build/arm64-v8a/geckoBeta/target.apk) +* [ARM devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-beta.latest/artifacts/public/build/armeabi-v7a/geckoBeta/target.apk) +* [x86_64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-beta.latest/artifacts/public/build/x86_64/geckoBeta/target.apk) +* [x86 devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/project.mobile.fenix.v2.fennec-beta.latest/artifacts/public/build/x86/geckoBeta/target.apk) + +## Nightly + +The following links point to the latest *Fenix* (Firefox Preview) builds (Nightly; from `main`) that are setup to **replace** a *Fennec Nightly* (Firefox for Android - Nightly) release version (`org.mozilla.fennec_aurora`). + +* [ARM64/Aarch64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/mobile.v2.fenix.nightly.latest.arm64-v8a/artifacts/public/build/arm64-v8a/target.apk) +* [ARM devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/mobile.v2.fenix.nightly.latest.armeabi-v7a/artifacts/public/build/armeabi-v7a/target.apk) +* [x86_64 devices (64 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/mobile.v2.fenix.nightly.latest.x86_64/artifacts/public/build/x86_64/target.apk) +* [x86 devices (32 bit; Android 5+)](https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/mobile.v2.fenix.nightly.latest.x86/artifacts/public/build/x86/target.apk) + +## 📝 Changelog + +The data migration work is tracked on the following project board: +https://github.com/orgs/mozilla-mobile/projects/40 + +* **2019-09-05** - The first [migration builds](https://tools.taskcluster.net/index/project.mobile.fenix.v2.fennec-production/latest) are available now. A Firefox for Android (release) installation can be replaced with them. No actual migration code is in those builds yet. The replaced build is a "clean" Fenix installation. +* **2019-10-22** - First iteration of migration code to migrate history, bookmarks and open tabs landed in builds. +* **2019-11-02** - Firefox Account users remain logged in after migrating to Fenix. + +## 💻 Development + +When working on migration code it is helpful to have a local Fennec build and a local Fenix build that can replace the Fennec build. The following manual setup is needed to achieve that. + +In the example commands below, we assume you are replacing a **Fennec Nightly** build with a **Fenix Nightly** build. + +## Fennec + +Download the latest version of Fennec: + + - [ARM64/Aarch64 devices (64 bit; Android 5+)](https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-esr68-android-aarch64/) + - [ARM devices (32 bit; Android 5+)](https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-esr68-android-api-16/) + - [x86_64 devices (64 bit; Android 5+)](https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-esr68-android-x86_64/) + - [x86 devices (32 bit; Android 5+)](https://archive.mozilla.org/pub/mobile/nightly/latest-mozilla-esr68-android-x86/) + +Strip out the original signature: + +``` +zip --delete fennec.apk "META-INF/*" +``` + +Re-sign the APK with your own debug key (that will also be used later for Fenix): + +``` +jarsigner -verbose -keystore ~/.android/debug.keystore -storepass android -keypass android fennec.apk androiddebugkey +``` + +You can now install this APk that is ready to be used for migration: + +``` +adb install fennec.apk +``` + +## Fenix + +In the `app/build.gradle`, add the following line in the correct scope to sign your app with the same debug key used on the Fennec APK: + +```groovy +android { + buildTypes { + fennecNightly { + signingConfig signingConfigs.debug + } + } +} +``` + +Follow the build instructions in the [README](https://github.com/mozilla-mobile/fenix/blob/main/README.md) to get a Fenix build setup. + +Now select the `geckoNightlyFennecNightly` build variant in Android Studio and deploy it. This build should have replaced your Fennec build now. + +## Sample browser + +When working on migration code that lives in the [Android Components repository](https://github.com/mozilla-mobile/android-components) it can be helpful to replace a local Fennec build with the sample browser (instead of Fenix). The following setup is needed for that. + +Add the sharedUserId to the AndroidManifest.xml of sample browser: + +```XML +<manifest xmlns:android="http://schemas.android.com/apk/res/android" + xmlns:tools="http://schemas.android.com/tools" + android:sharedUserId="org.mozilla.fennec_$USERNAME.sharedID" + [..] +``` + +Modify the application id in `build.gradle` of the `samples-browser` module and use a versionCode that is higher than your Fennec build (`2100000000` is the highest allowed version code and therefore should always work). + +```Groovy + defaultConfig { + applicationId "org.mozilla.fennec_$USERNAME" + [..] + versionCode 2100000000 +``` + + + +Click on "Sync Project with Gradle Files" and deploy the sample browser. This build should have replaced your Fennec build now. + +## Emulator snapshots + +When testing migration code the following process has to be repeated multiple times: + +* (1) Uninstall an already existing Fennec/Fenix installation +* (2) Install Fennec +* (3) Use Fennec to create the necessary data for testing the migration +* (4) Install Fenix +* (5) Debug / Test + +Steps (1) to (3) can be quite time consuming. Emulator snapshots can help with that: + +* Launch an emulator and perform steps 1 to 3. You may need to modify your Fennec build to create an X86 build for your emulator (target `i686-linux-android`). +* Click on the three dot menu in the emulator toolbar and select "Snapshots". Press the "Take Snapshot" button. If needed give you snapshot a descriptive name in case you will need to have multiple "test snapshots". +* With the "Play" button you can always reset your emulator to that state and repeat the migration process. diff --git a/mobile/android/fenix/docs/Firebase-Cloud-Messaging-for-WebPush.md b/mobile/android/fenix/docs/Firebase-Cloud-Messaging-for-WebPush.md new file mode 100644 index 0000000000..b3cd18e47d --- /dev/null +++ b/mobile/android/fenix/docs/Firebase-Cloud-Messaging-for-WebPush.md @@ -0,0 +1,46 @@ +# Firebase Cloud Messaging for WebPush + +## Testing +If you want to test WebPush support for features like Send Tab or WebPush for web apps, then you need to follow the instructions below to enable this for your debug application: + +1. Download the XML credential file from the [Fenix team's Google Drive](https://drive.google.com/file/d/1DwgmqJTSKOY9vMuW2OmRNDXpIY69thTI/view?usp=drive_link). +2. Place the file in the directory `fenix/app/src/debug/res/values/`. + * This will be compiled into the `fenixDebug` variant. +3. Run the app and verify you receive log messages in `adb logcat` like the ones below: + ``` + Received a new registration token from push service. + Got new Firebase token. + ``` + +## Components and code +The important components for push to work are the `feature-push` (specifically look at the [README](https://searchfox.org/mozilla-mobile/source/firefox-android/android-components/components/feature/push/README.md)) and `lib-push-firebase`. + +For GeckoView, look at [`WebPushEngineIntegration.kt`](https://searchfox.org/mozilla-mobile/source/firefox-android/fenix/app/src/main/java/org/mozilla/fenix/push/WebPushEngineIntegration.kt) that connects the `AutoPushFeature` from `feature-push` to the `Engine`. + +For Firefox Sync, it's the [`feature-accounts-push`](https://searchfox.org/mozilla-mobile/source/firefox-android/android-components/components/feature/accounts-push). + +## Firefox Sync & Send Tab +If you are testing WebPush support with Firefox Sync, you should also expect to see this log shortly after signing into a Firefox Sync account: +``` +Created a new subscription: [context data] +``` + +## FAQ + +**Q. Why are credentials stored in Google Drive?** + * Firebase services require a credential file, typically named `google-services.json`, which are retrieved from the Firebase console for all the applications in that project. Our requirements are somewhat unique, so we generate our own XML file from `google-services.json` and use that. + According to [Firebase engineers](https://groups.google.com/forum/#!msg/firebase-talk/bamCgTDajkw/uVEJXjtiBwAJ), it should be safe to commit our `google-services.json` but we avoid doing that as there are other forks of Fenix that must not use these credentials. + For example, the Google I/O app commits [their release version](https://github.com/google/iosched/blob/b428d2be4bb96bd423e47cb709c906ce5d02150f/mobile/google-services.json) of the file to Github as well. + +**Q. What are the special requirements which mean we can't directly use `google-services.json`?** + + * Following the 'Getting Started' Firebase instructions to initialize the service in Fenix would not work as we explicitly chose not to use the `com.google.gms.google-services` Gradle plugin as the Android service would be initialized with a [`ContentProvider`](https://firebase.blog/posts/2016/12/how-does-firebase-initialize-on-android) eagerly which reduces our control on how the application can start up in a performant way since the Browser startup sequence is quite unique. + +**Q. Where do we get the `google-services.json` file from?** + * Ask your friendly Release Management teammate if they can access the Cloud Messaging console at `console.firebase.google.com`. + +**Q. How do I generate the XML file from `google-services.json`?** + + * The easiest way is to use [this clever web app](https://dandar3.github.io/android/google-services-json-to-xml.html) that does this for you. + * The more official and tedious way, would be to do this manually by following [the instructions on the google services plugin](https://developers.google.com/android/guides/google-services-plugin#adding_the_json_file) site. + * Note that the `google-services.json` file may have the credentials for multiple applications, so ensure you are copying the instructions for the correct project. diff --git a/mobile/android/fenix/docs/Firefox-for-Android-Team-Processes.md b/mobile/android/fenix/docs/Firefox-for-Android-Team-Processes.md new file mode 100644 index 0000000000..c28320118a --- /dev/null +++ b/mobile/android/fenix/docs/Firefox-for-Android-Team-Processes.md @@ -0,0 +1,115 @@ +# Firefox for Android Team Processes + +## Communication channels +For internal Firefox Android communication start off in #mobile-android-team. + +Dedicated channels on Matrix work as well: +* [#Fenix](https://matrix.to/#/#fenix:mozilla.org) - Fenix development discussions. +* [#android-components](https://matrix.to/#/#android-components:mozilla.org) - shared core components for our mobile browsers/projects. +* [#geckoview](https://matrix.to/#/#geckoview:mozilla.org) - working on the Android layer of Gecko that interfaces with Android Components and Fenix. + +## Design Feasibility +Who: Design lead, designer, engineering lead, engineer, product lead, product manager +Purpose: meet any time we start design for a new feature (esp large ones) to discuss: +* Overall concept and user goals +* Where the feature fits within the Fenix system +* Any existing technical constraints or dependencies (on Android OS, or other Firefox Mobile teams like GV or A-C) +* Alignment on user stories + +Design Handoff +Who: designer, engineer, product manager, QA lead +Purpose: before engineering sprint for a feature starts to discuss: +* Overall purpose of feature and how it relates to or interacts with existing features +* Discuss expectations for functionality +* Discuss visual and interactive design elements +* Negotiate scope and discuss changes if tasks are too large for a single sprint + +## Triage +* The team triages Bugzilla issues asynchronously. This involves reviewing all new issues (bugs, crashes, and feature requests) to determine whether they should be addressed in MVP or added to backlog for future sprints. + +### Process +* Add appropriate labels for [features](https://github.com/mozilla-mobile/fenix/labels?q=Feature) +* Add to appropriate Projects ([1](https://github.com/orgs/mozilla-mobile/projects), [2](https://github.com/mozilla-mobile/fenix/projects)) +* Note any urgent issues and tag the Product Team when needed + +## Sprint Planning +### Sprint pre-planning: +* Build sprint plan based on what is ready (UX/dependencies), priority, and complexity. + +### During Sprint Planning: +* Assign stories/bugs to sprint up to capacity, and resolve any open questions. +* Size stories as needed. Most sizing is happening asynchronously. +* Go over UX Sprint Plan + +## Backlog Grooming +* During Backlog Grooming, review and all user stories assigned to Backlog that have not been sized. +* ... + +## During Sprint +* Engineers will label stories as “in progress” as they work begin work on them, and assign story to themselves. +* IF a user story has a UX component that needs review, when it is ready for review, engineer will add a screenshot/gif and @mention the designer in the user story. Add the [needs:ux-review](https://github.com/mozilla-mobile/fenix/labels/needs%3AUX-review) label to the PR. +* When a PR has been merged in, the Merger verify that Milestone this code will ship in to the issue. (Careful around soft code freeze! The release may have already been cut, which affects the current milestone.) +* Engineers will label stories with [eng:qa:needed](https://github.com/mozilla-mobile/fenix/labels/eng%3Aqa%3Aneeded) when the ticket is ready to be tested. +* QA will label the story as [eng:qa:verified](https://github.com/mozilla-mobile/fenix/labels/eng%3Aqa%3Averified) and close the ticket (which will move it to the “Done” column). +* If a ticket becomes blocked and will miss the sprint, engineers will label it “waiting” and notify the Product Team with the reason. This issue will be moved back to the appropriate backlog. +* Engineers will remove the “waiting” label and re-apply the appropriate label (“in progress,” “QA needed”). + +## UX Review +IF a user story has a UX component that needs review, when it is ready for review: +* Consider hopping on a call to do a ‘desk check’ with the Designer* +* Engineer will add a gif/screenshot/apk (as applicable to the issue) +* Engineer will @mention the designer in the user story and ping the Designer on Slack, and add the `needs:UX-feedback` label + + UX designer reviews the component (consider hopping on a call to work through minor changes). Communication is key to here between designer and engineer in how they want to go through edits. + + We will also go through UX review during Sprint Demos. + +## Copy Review +IF a user story has a String that needs a review, when it is ready for review: +* Raise an issue in Fenix as feature request. +* Add the needs:strings label. +* Add the screenshot where the string will be applied. +* Add detail explanation of the use case. +* Comment asking the UX designer to review. + +## Engineering review +Use tags on open PRs to show which part of the process it is on. Some notable ones: +1. [needs:review](https://github.com/mozilla-mobile/fenix/labels/needs%3Areview) - PR that needs a review. Anyone should jump on any reviews with this label and help out with reviews. Thanks in advance. +2. [pr:needs-ac](https://github.com/mozilla-mobile/fenix/labels/needs%3Aac) - PR that is waiting for a AC bump. Typically, we use “waiting”, but this provides us with a bit more context. +3. [pr:approved](https://github.com/mozilla-mobile/fenix/labels/pr%3Aapproved) - PR that has been approved. This one is a bit easier to parse visually compare to the Approved in the GitHub summary +4. [pr:waiting-for-authors](https://github.com/mozilla-mobile/fenix/labels/pr%3Awaiting-for-authors) - PR that has been approved and awaiting any changes before they can land. Usually a PR might be approved, but has not been landed because it is waiting for followup changes. + +## QA +* Engineers will label stories as [eng:qa:needed](https://github.com/mozilla-mobile/fenix/labels/eng%3Aqa%3Aneeded) when the ticket is ready to be tested (which will move the ticket to the ‘Ready for QA’ column’). +* QA will review the ticket and determine whether it can be manually tested. If no QA is needed, QA will close the ticket and move it to the ‘Done’ column. + +IF a defect is found: +* a comment is added to the story @mentioning the engineer +* the defect is linked as a dependency for the story +* The QA Needed label is removed. +* The related story is set to ‘In Progress’. +* When the defect has been fixed, the Engineer will add the ‘QA Needed’ label to the story again. +* When all critical defects have been fixed, QA will label the story as “QA verified” and close the ticket (which will move it to the “Done” column). + +## Testing Performance Impact of Code Changes +- The Performance team wrote this up for reference: https://wiki.mozilla.org/Performance/Fenix/Performance_reviews + + +## Accessibility +* During design reviews, we should ask ourselves if custom UI is necessary to accomplish the desired behavior. Oftentimes, out of the box android UI has accessibility and localization built in, whereas with custom pieces we need to build it ourselves. +* For new features that have UI elements that are unique to Fenix or have user interaction, test them personally with TalkBack (just as you would test to make sure the feature works before submitting a patch) +* Before submitting a patch, run the Accessibility Scanner over the screens affected to ensure no new issues are introduced (and post a screenshot of your results) +* QA should test tickets that have “eng:qa:needed” with accessibility services (like TalkBack) enabled and only mark them as “eng:qa:verified” if these services work well with the new feature +* As mentioned in the ticket, QA team will be adding accessibility checks to UI tests over time +* Remember: the earlier we catch issues in the process (closer to dev coding or UX designing) the less work it is for everyone involved! So please be diligent about going through these checks before submitting a patch. + +## Templates +- We've created templates for new issues with instruction on how to fill them out based on their request/nature [here](https://github.com/mozilla-mobile/fenix/issues/new/choose) + +More details in [link to Process Doc & Flowchart](https://docs.google.com/document/d/1w_6G4uCfQjyBh0ilQZKz3G-0IvBhzExlg80kaJrBA3c/) + + +## Adding Locales +The completion rate of different locales can be seen on [Pontoon](https://pontoon.mozilla.org/projects/android-l10n/). +While the project is in Preview stage, all locales will be the same between Firefox Preview and Firefox Preview Nightly. +Before we do our next large, milestone release, we'll update the Release locales to match the ones approved by L10N to have reached sufficient localization completion. diff --git a/mobile/android/fenix/docs/Guide-to-merging-contributor-PRs.md b/mobile/android/fenix/docs/Guide-to-merging-contributor-PRs.md new file mode 100644 index 0000000000..5af5caed72 --- /dev/null +++ b/mobile/android/fenix/docs/Guide-to-merging-contributor-PRs.md @@ -0,0 +1,79 @@ +# Guide to merging contributor PRs for Firefox Android team members + +Contributor PRs will run only a specific suite of CI tests (excluding UI tests) in order to protect secrets. Use the following steps when reviewing and merging a contributor PR. + +## Process for landing contributor PR +_Note: these instructions use https://cli.github.com/_ + +1. Fetch upstream changes and locally check out the contributor's branch onto your fork. +```sh +git fetch --all +gh pr checkout <PR number> + +# Example: +gh pr checkout 1234 # for https://github.com/mozilla-mobile/firefox-android/pull/1234 +``` +2. Build and run contributor's changes locally to verify that it works correctly. + +3. Review the code to make sure everything is clean. + +4. ~~Once a Firefox Android team member has reviewed the PR and deemed it safe, comment the following to start UI tests. + ```bors try```~~ + `Bors` public instance is now offline, as they announced in [their newsletter](https://bugzilla.mozilla.org/show_bug.cgi?id=1850420). Please refer to section "Process for running UI tests on a contributor PR" above. + + +5. Once the UI tests have all completed and passed, approve the PR and add `approved` and `needs landing` label the contributor's PR. + +6. Monitor the merging process to make sure it lands as expected. + +## Process for running UI tests on a contributor PR + +1. Make sure you have already checked out the contributor's branch onto your fork, following the previous session's first step. +```sh +git fetch --all +gh pr checkout <PR number> + +# Example: +gh pr checkout 1234 # for https://github.com/mozilla-mobile/firefox-android/pull/1234 +``` + +3. Rename your local branch's to any name +``` +git branch -m <new branch name> + +# Example: If the contributor's branch is named `eliserichards:my-fun-branch1` +git branch -m ci-for-my-fun-branch1 +``` + +3. Push branch to your fork using any branch name: +```sh +git push origin <pr-branch-name> + +# Example: If the contributor's branch is named `eliserichards:my-fun-branch1` +git push origin ci-for-my-fun-branch1 +``` + +4. Create a PR from _your fork's_ copy of the branch e.g. https://github.com/mozilla-mobile/firefox-android/compare/main...eliserichards:my-fun-branch1 + +* Please note in the PR description which PR you are running CI for. Example: https://github.com/mozilla-mobile/firefox-android/pull/4577 + +*** + +**Once you create this PR, the CI for both the original and the duplicate PRs will run. When everything is green, you can merge either of them.** + +5. To land the original: +* i. Make sure that contributor's branch hasn't diverged from yours (they must have the same SHA). +* ii. The change has to be on the top of the main branch when it is first in line in the merge queue. +* iii. It requires the needs-landing label. + +**NB**: Adding `needs-landing` label while failing to ensure the same SHA will block the mergify queue and will require manual intervention: mergify will trigger CI for the original PR again and wait for it to finish, but CI won’t run all the checks because there is no PR with the same SHA any more that backs it up. If that happens, talk to the release team. + + +## Process for updating contributor PR (if contributor needs help or is unresponsive) + +```sh +git remote add <Contributor remote name> <Contributor repository> +git checkout <Contributor remote name>/<Contributor branch name> +git rebase upstream/main (or any other actions you want to fixup in their PR) +git push <Contributor remote name> HEAD:<Contributor branch name> -f +``` diff --git a/mobile/android/fenix/docs/Home.md b/mobile/android/fenix/docs/Home.md new file mode 100644 index 0000000000..557d331561 --- /dev/null +++ b/mobile/android/fenix/docs/Home.md @@ -0,0 +1,31 @@ +# Firefox for Android (Fenix Project) + +Firefox for Android is Mozilla's new browser for Android devices. Powered by GeckoView and built on Android Components, Fenix is a pilot for early adopters, developers, and anyone who wants to help make a better, more private Firefox for Android. + +Firefox for Android is the first step in building a better mobile browser: one that’s faster, more secure and more independent than any mobile Firefox browser before it. It will combine the speed, privacy, control, and easy-to-use features you have come to expect from Firefox. + +*Don't see what you're looking for? Check out our shared docs: https://github.com/mozilla-mobile/shared-docs.* + +## User support + +* Support articles: https://support.mozilla.org/en-US/products/mobile +* Support forum: https://support.mozilla.org/en-US/questions/mobile + +## Download + +* Google Play: [Release](https://play.google.com/store/apps/details?id=org.mozilla.firefox) +* Google Play: [Beta](https://play.google.com/store/apps/details?id=org.mozilla.firefox_beta) +* Google Play: [Nightly](https://play.google.com/store/apps/details?id=org.mozilla.fenix) +* Release APKs: https://github.com/mozilla-mobile/fenix/releases +* Nightly APKs: https://firefox-ci-tc.services.mozilla.com/tasks/index/mobile.v2.fenix.nightly.latest + +## Contribute + +* [Contributing (Writing code, translating the app, testing the app)](https://github.com/mozilla-mobile/shared-docs/blob/master/android/CONTRIBUTING.md) +* [List of good first issues for new contributors](https://github.com/mozilla-mobile/fenix/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22+) +* [Mozilla Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/) + +## Communication +* External: [#fenix](https://chat.mozilla.org/#/room/#fenix:mozilla.org) on [Element](https://wiki.mozilla.org/Matrix) +* External: [Firefox on Android discourse](https://discourse.mozilla.org/c/firefox-android/468) +* Internal: Find us on slack #fenix-team diff --git a/mobile/android/fenix/docs/Implementing-Telemetry.md b/mobile/android/fenix/docs/Implementing-Telemetry.md new file mode 100644 index 0000000000..a2dfff6571 --- /dev/null +++ b/mobile/android/fenix/docs/Implementing-Telemetry.md @@ -0,0 +1,46 @@ +# Implementing Telemetry + +## Things to note before implementation: +* Understand that telemetry is important, it is not just a checkmark for feature completion. +* The consumer of the telemetry is the data science team. +* When in doubt, please follow the example implementation, documentation and data review format linked below. +* Avoid using SharedPreferences. +* Write unit tests. + +## Procedure to follow before implementing the telemetry: +1. Contact Product team to understand the feature that we are adding telemetry to. +2. Contact the Data Science team to get the full requirements. This includes: +* The categories that the Data Science team expects data from? +* What are the telemetries the Data Science team expects in each category? +* What type of data for each telemetry? +3. Work with the Data Science team to raise/lower expectations. Refine the requirements until every telemetry is clearly specified. This includes: +* Inform the Data Science team which telemetry is not achievable. (if exists) +* Inform the Data Science team other possible telemetry that they might not know about. +* Inform the Data Science team what might not make sense to collection. (Ex: B always happens when A happens) +* Help the Data Science team collect the best telemetry data possible. +4. Consult with the Glean team if there’s any questions. (Ex: What type of data to use) + +## Procedure to follow when implementing a Glean telemetry event +* A full example of adding an event with keys can be found [here](https://github.com/mozilla-mobile/android-components/pull/10837) (Android Components), [here](https://github.com/mozilla-mobile/fenix/pull/20909) (Fenix) and [here](https://github.com/mozilla/glean-annotations/pull/77) (Glean Annotation) +1. Create an event in [metrics.yaml](https://github.com/mozilla-mobile/fenix/blob/master/app/metrics.yaml) and do a project rebuild to generate the event +2. To add feature tags see steps [here](https://github.com/mozilla-mobile/fenix/wiki/Metric-Feature-Tags). +5. Send the event from the proper place in the code with the appropriate generated method (e.g. `GeneratedClassMetrics.generatedEvent.record()`) +6. Create pull requests +7. Submit a data review ([example here](https://github.com/mozilla-mobile/fenix/pull/20909#issuecomment-902119039)). There's also a [command-line tool for generating Data Review Requests](https://chuttenblog.wordpress.com/2021/09/07/this-week-in-glean-data-reviews-are-important-glean-parser-makes-them-easy/) +8. Update the [metrics.yaml](https://github.com/mozilla-mobile/fenix/blob/master/app/metrics.yaml) with the data review +9. For startup metrics, make sure to [manually test it](https://github.com/mozilla-mobile/fenix/wiki/Test-telemetry-pings) + +## Review +See example [here](https://github.com/mozilla-mobile/fenix/pull/20909) +* Add a developer that understands telemetry to review your change. +* Add a developer from the Glean team as reviewer if needed. +* Data review format [here](https://github.com/mozilla/data-review/blob/main/request.md). ([example here](https://github.com/mozilla-mobile/fenix/pull/20909#issuecomment-902119039)) + +## After Merge +1. Make a note to revisit your telemetry changes when it makes it to beta/release. +* for events, go to Glean dictionary and find the event you want to verify. Click on the Looker link on the bottom of the page to confirm that the event is being reported. (For example, for credit_cards.modified, the Glean dictionary link is https://dictionary.telemetry.mozilla.org/apps/fenix/metrics/credit_cards_modified. On the bottom click on the "credit_cards.modified" link next to Looker to see event count) +* for metrics, create a query (ex: https://sql.telemetry.mozilla.org/queries/82373) to confirm that metric is being reported. +2. Work with the data science team to make sure that they are seeing data that meet their requirements. + +## Renewing Expiring Telemetry +See steps [here](https://github.com/mozilla-mobile/fenix/wiki/Creating-a-release-branch#renew-telemetry) diff --git a/mobile/android/fenix/docs/Implementing-a-Nimbus-Experiment.md b/mobile/android/fenix/docs/Implementing-a-Nimbus-Experiment.md new file mode 100644 index 0000000000..274206db28 --- /dev/null +++ b/mobile/android/fenix/docs/Implementing-a-Nimbus-Experiment.md @@ -0,0 +1,14 @@ +# Implementing a Nimbus Experiment + +Follow instructions in https://experimenter.info/mobile-feature-api. Example implementation [here](https://github.com/mozilla-mobile/fenix/pull/23996) + +Nimbus FML https://experimenter.info/fml-spec/ + + +There are some clarification on how to test your Nimbus implementation: +1. Add `nimbus.remote-settings.url=https://settings-cdn.stage.mozaws.net` to local.properties. +2. After building Fenix, make sure you turn on `Secret Settings` -> `Use Nimbus Preview Collections`. +3. The experiment in https://stage.experimenter.nonprod.dataops.mozgcp.net/nimbus/ does not have to be live for the test. In preview is sufficient. +4. Example of a test is [here](https://stage.experimenter.nonprod.dataops.mozgcp.net/nimbus/unified-search-test) +5. Make sure you archive the test after you're done with it. +6. In your PR, make sure to submit the change for .experimenter.yaml as well. diff --git a/mobile/android/fenix/docs/List-of-fenix-threads.md b/mobile/android/fenix/docs/List-of-fenix-threads.md new file mode 100644 index 0000000000..110eed4bdb --- /dev/null +++ b/mobile/android/fenix/docs/List-of-fenix-threads.md @@ -0,0 +1,53 @@ +# List of Fenix Threads + +To profile background threads using the Firefox Profiler, you need to specify their names. It uses a case-insensitive substring match, e.g. specifying `default` will match all threads in the kotlin default dispatcher which have a name like, `DefaultDispatcher-worker-*`. This document is a list of the threads in fenix (via `ThreadGroup.list()` as of Mar 2022) to make using this functionality easier: +``` +AutoSave-thread-1 +BrowserIcons-thread-1 +BrowserIcons-thread-2 +BrowserIcons-thread-3 +BrowserStore-thread-1 +ConnectivityThread +DefaultDispatcher-worker-1 +DefaultDispatcher-worker-2 +DefaultDispatcher-worker-3 +DefaultDispatcher-worker-4 +DefaultDispatcher-worker-5 +DefaultDispatcher-worker-6 +DefaultDispatcher-worker-7 +DefaultDispatcher-worker-8 +FinalizerDaemon +FinalizerWatchdogDaemon +FxaAccountManager-thread-1 +Gecko +GeckoInputConnection +GleanAPIPool +HeapTaskDaemon +HistoryMetadataService-thread-1 +LeakCanary-Heap-Dump +NimbusDbScope-thread-1 +NimbusFetchScope-thread-1 +PlacesStorageWriteScope-thread-1 +ReferenceQueueDaemon +ThumbnailStorage-thread-1 +ThumbnailStorage-thread-2 +ThumbnailStorage-thread-3 +WM.task-1 +WM.task-2 +WM.task-3 +WM.task-4 +androidx.work-1 +androidx.work-2 +arch_disk_io_0 +arch_disk_io_1 +arch_disk_io_2 +arch_disk_io_3 +glean.MetricsPingScheduler +main +pool-23-thread-1 +pool-9-thread-1 +pool-9-thread-2 +queued-work-looper +``` + +Note that `arch_disk_io_*` represents the kotlin io dispatcher. diff --git a/mobile/android/fenix/docs/Logging-Crash-Information.md b/mobile/android/fenix/docs/Logging-Crash-Information.md new file mode 100644 index 0000000000..7222bd8375 --- /dev/null +++ b/mobile/android/fenix/docs/Logging-Crash-Information.md @@ -0,0 +1,56 @@ +# Logging Crash Information + +## Retrieving crash reports from the application +* Open Firefox +* Tap on the `3 dot menu` +* Tap `Settings` +* Scroll to the bottom of Settings +* Tap `About Firefox` +* Tap `Crashes` +* Tap on the Socorro link +* Copy and paste that address into a new [Github Issue: 🐞 Bug report](https://github.com/mozilla-mobile/fenix/issues/new/choose) or an existing issue +* If you have many crash reports it can be helpful to include several of the recent crash URLs + + + + + +## Using adb logcat to get crash information + +Please use the above directions as a first way to get info. Use the following directions if your crash does not show up in the crash window of Firefox or if the crash prevents you from accessing the settings of Firefox. + +To get information about a crash you will need an Android device that reproduces a crash, a computer running Windows, macOS or Linux and a USB cable that connects your device to your computer. + +### Configuring your phone +* Enable Developer Mode + * On stock Android open the Android Settings and use the search at the top to find `build number` + * Tap the build number 7 times + * For other devices use your favorite search engine or YouTube to find steps for your device. +* Enable Android USB debugging + * On stock Android Open the Android Settings and use the search at the top to find `USB debugging` and enable it +* Connect your device to the computer using a USB cable + +### Downloading the Android SDK Platform tools +* Download the [Android SDK Platform tools](https://developer.android.com/studio/releases/platform-tools) for your operating system +* Use your operating system tools to extract the zip file that was downloaded + +### Checking that adb can see your phone and is authorized +* Connect your device to the computer using a USB cable +* Open a command prompt or terminal and change to the directory to the platform tools directory that was extracted +* On Windows run the command `adb devices` on macOS and Linux run `./adb devices` +* If it returns unauthorized you will need to authorize the phone to connect to the computer by accepting the connection dialog on the phone + +### Reproducing the crash +* Connect your device to the computer using a USB cable +* Open a command prompt or terminal and change to the directory to the platform tools directory that was extracted +* On Windows run `adb logcat -v time` on macOS and Linux run `./adb logcat -v time` +* Reproduce the crash +* Submit the crash report(s) +* Unplug the device +* Copy all the information in the terminal +* Paste the information into a Gist https://gist.github.com/ and save logcat information +* Add the Gist URL to the issue for the crash + +### Optional Cleanup +* It is recommended to disable USB debugging once you are done collecting this information +* You can remove the Android SDK Platform tools by deleting the folders and zip file diff --git a/mobile/android/fenix/docs/Metric-Feature-Tags.md b/mobile/android/fenix/docs/Metric-Feature-Tags.md new file mode 100644 index 0000000000..dbd7e177a6 --- /dev/null +++ b/mobile/android/fenix/docs/Metric-Feature-Tags.md @@ -0,0 +1,37 @@ +# Metric Feature Tags + +To help find metrics in the [Glean Dictionary] and other tools, metrics should contain tag metadata corresponding to the +feature area(s) that they belong to. In the case of Firefox for Android, tag information is based off of the GitHub feature labels for this repository: + +https://github.com/mozilla-mobile/fenix/labels?q=feature%3A + +You can see how tag information is rendered here: + +https://dictionary.telemetry.mozilla.org/apps/fenix?itemType=tags&page=1 + +## Adding feature tags to metrics + +Adding tag information to a metric used to involve editing the [Glean Annotations repository], but you can now add this +information directly when adding or modifying `metrics.yaml`. Just add a section called `metadata` to the metric and add a list of tags that correspond to it. + +For example: + +```yaml + search_bar_tapped: + type: event + description: | + A user tapped the search bar + metadata: + tags: + - Search + ... +``` + +## Updating the feature tags + +The set of valid tags is documented in a file called `tags.yaml`, but should never be updated by hand. +If a feature labels is ever added or removed, you can synchronize the tags file in the source tree by running `./tools/update-glean-tags.py` in the root of the repository. +Note that a tag *must* be specified in `tags.yaml` for it to be usable in a metric, so if a tag is removed from `tags.yaml` all uses of it must be removed from `metrics.yaml`. + +[Glean Dictionary]: https://dictionary.telemetry.mozilla.org +[Glean Annotations repository]: https://github.com/mozilla/glean-annotations diff --git a/mobile/android/fenix/docs/Secret-settings-debug-menu-instructions.md b/mobile/android/fenix/docs/Secret-settings-debug-menu-instructions.md new file mode 100644 index 0000000000..c824d9cde3 --- /dev/null +++ b/mobile/android/fenix/docs/Secret-settings-debug-menu-instructions.md @@ -0,0 +1,9 @@ +# Debug settings menu instructions + +Instructions to enable the "secret settings" in the Settings menu for Firefox Nightly. This lets you access "Secret Settings" and "Secret Debug Info" in the Settings menu. + +1. Tap on three-dot menu in the toolbar +2. Tap on Settings +3. Scroll down and select "About Firefox Nightly" +4. Tap the Firefox logo until you see the "Debug menu: (#) click(s) left to enable" helper message +5. Once the Debug menu has been enabled, go back to Settings and you will see "Secret Settings" and "Secret Debug Info" menu items diff --git a/mobile/android/fenix/docs/Telemetry-implementation,-reviews,-renewals.md b/mobile/android/fenix/docs/Telemetry-implementation,-reviews,-renewals.md new file mode 100644 index 0000000000..81f3ea5825 --- /dev/null +++ b/mobile/android/fenix/docs/Telemetry-implementation,-reviews,-renewals.md @@ -0,0 +1,109 @@ +# Telemetry - Implementation, Reviews, Renewals + +See https://github.com/mozilla-mobile/fenix/wiki/Telemetry-Checklist for the steps to implement new probes. + +# Creating Glean Annotations + +Glean Annotations repository: https://github.com/mozilla/glean-annotations + +See [Add a Glean Annotation for an event](https://github.com/mozilla-mobile/fenix/wiki/Add-a-Glean-Annotation-for-an-event) for instructions. + +More info [here](https://mozilla.github.io/glean-annotations/contributing/creating/) + + +# Data review + +Data reviews are needed on all PRs that add new telemetry or modify existing telemetry. + +1. The implementer must complete the forms for [data renewal](https://github.com/mozilla/data-review/blob/main/renewal_request.md) or [a new data request](https://github.com/mozilla/data-review/blob/main/request.md) and put them as a comment in their PR. +2. Once the form is complete, contact a [Data Steward](https://wiki.mozilla.org/Data_Collection) to arrange a review. Note: a data review does not replace code review! The PR should not land without both a data review and a code review. +3. Once the data review is complete, add the link to the approval in the `data_reviews` sub-section of your metric in the `metrics.yaml` file. +Example: + +``` +download_notification: + resume: + type: event + description: | + A user resumed a download in the download notification + bugs: + - https://github.com/mozilla-mobile/fenix/issues/5583 + data_reviews: + - https://github.com/mozilla-mobile/fenix/pull/6554 + - https://github.com/mozilla-mobile/fenix/pull/13958#issuecomment-676857877 + - https://github.com/mozilla-mobile/fenix/pull/18143 + data_sensitivity: + - interaction + notification_emails: + - fenix-core@mozilla.com + expires: "2021-07-01" +``` + +When a telemetry probe is being renewed, do not remove the old data review links from `metrics.yaml`. The new approval should be added to the existing list. + +Make sure you are selecting the correct Category of data that is being collected: https://wiki.mozilla.org/Data_Collection#Data_Collection_Categories + +# Renewing existing telemetry + +1. Collect a list of metrics from metrics.yaml file in Fenix that will be expiring in that month + a. Currently compiling these in [this doc](https://docs.google.com/document/d/1NGlnTa9TPyTnd3ciUPbwujbITjkX8p8vJybXcZrrM2w/edit#) + b. This should be done at least a few weeks prior to the events/metrics' expiration date + c. Including metric name, original data review PR link, description (if it’s unclear from the name) +2. Figure out the owner for each of the metrics (who needs to give the OK to renew/remove) + a. Most renewals will need product approval + b. Other approvals could come from the engineering team (e.g. `preferences.remote_debugging_enabled`), GV, App Services, Performance (e.g. `startup.timeline.framework_primary`), etc. +3. Answer any open questions for the metric owners, and get approval from them to: + a. Renew the metric (for how long? 6 months? 1 year?) + b. Choose not to renew (but not delete) + c. Choose to remove the metric + d. Renew the metric and set to never expire (this should only be for business critical metrics) +4. Fill out the [renewal request](https://github.com/mozilla/data-review/blob/main/renewal_request.md) for each metric, with question 3 (“Why was the initial period of collection insufficient?”) being answered by the owner +5. Open a PR for the renewals + a. Sometimes it’s easier to split up the renewals into multiple PRs if there are multiple owners: e.g. [product renewals](https://github.com/mozilla-mobile/fenix/pull/21788), [perf renewals](https://github.com/mozilla-mobile/fenix/pull/21315), [Fission renewals](https://github.com/mozilla-mobile/fenix/pull/21779) for December 2021 +6. Get a data review for your [renewal request](https://github.com/mozilla/data-review/blob/main/renewal_request.md) and update each of the metrics’ data-reviews in metrics.yaml to reflect this + +## Approval process + +For each telemetry probe that we want to renew, the data-review will ask us [these questions](https://github.com/mozilla/data-review/blob/main/renewal_request.md). Each probe/group of related probes should have answers to those questions ([example](https://github.com/mozilla-mobile/fenix/pull/20517#issuecomment-887038794)). + +### Example renewal data request +``` +# Request for Data Collection Renewal + +`search_shortcuts:` + selected +1) Provide a link to the initial Data Collection Review Request for this collection. +- https://github.com/mozilla-mobile/fenix/pull/1202#issuecomment-476870449 +- https://github.com/mozilla-mobile/fenix/pull/15713#issuecomment-703972068 +- https://github.com/mozilla-mobile/fenix/pull/19924#issuecomment-861423789 + +2) When will this collection now expire? 08/01/2022 +3) Why was the initial period of collection insufficient? +Important for revenue tracking and optimization. + +`Toolbar_settings:` + changed_position: +1) Provide a link to the initial Data Collection Review Request for this collection. +- https://github.com/mozilla-mobile/fenix/pull/6608 +- https://github.com/mozilla-mobile/fenix/pull/15713#issuecomment-703972068 +- https://github.com/mozilla-mobile/fenix/pull/19924#issuecomment-861423789 + +2) When will this collection now expire? 08/01/2022 +3) Why was the initial period of collection insufficient? +The data didn’t initially tell us what we wanted (there were bugs), so we want to continue tracking this so we can answer our questions. + +`login_dialog:` + displayed: +cancelled +saved +never_save +1) Provide a link to the initial Data Collection Review Request for this collection. +- https://github.com/mozilla-mobile/fenix/pull/13050 +- https://github.com/mozilla-mobile/fenix/pull/19924#issuecomment-861423789 + +2) When will this collection now expire? 08/01/2022 +3) Why was the initial period of collection insufficient? +Still need to optimize this feature and we want trends from 6+mo of data. +``` + +For product-defined telemetry, this will involve meeting with a product manager and discussing each probe. There are three options: renew the probe for another length of time (usually 6 months), let the probe expire to evaluate later if the probe is still needed, or remove the probe entirely. diff --git a/mobile/android/fenix/docs/Test-telemetry-pings.md b/mobile/android/fenix/docs/Test-telemetry-pings.md new file mode 100644 index 0000000000..dcaa5099f1 --- /dev/null +++ b/mobile/android/fenix/docs/Test-telemetry-pings.md @@ -0,0 +1,76 @@ +# Test Telemetry Pings + +Watch a step by step [video](https://user-images.githubusercontent.com/6579541/170517089-7266b93e-7ff8-4ebb-ae01-4f2a7e558c66.mp4) + +1. To send data by default. apply this patch: +``` diff + +diff --git a/app/src/main/java/org/mozilla/fenix/FenixApplication.kt b/app/src/main/java/org/mozilla/fenix/FenixApplication.kt + +index 4cb11de43..0c6fab136 100644 + +--- a/app/src/main/java/org/mozilla/fenix/FenixApplication.kt + ++++ b/app/src/main/java/org/mozilla/fenix/FenixApplication.kt + +@@ -293,9 +293,7 @@ open class FenixApplication : LocaleAwareApplication(), Provider { + + } + + + + private fun startMetricsIfEnabled() { + +- if (settings().isTelemetryEnabled) { + +- components.analytics.metrics.start(MetricServiceType.Data) + +- } + ++ components.analytics.metrics.start(MetricServiceType.Data) + + + + if (settings().isMarketingTelemetryEnabled) { + + components.analytics.metrics.start(MetricServiceType.Marketing) + +diff --git a/app/src/main/java/org/mozilla/fenix/components/metrics/MetricController.kt b/app/src/main/java/org/mozilla/fenix/components/metrics/MetricController.kt + +index c38ebb62d..3ae102d97 100644 + +--- a/app/src/main/java/org/mozilla/fenix/components/metrics/MetricController.kt + ++++ b/app/src/main/java/org/mozilla/fenix/components/metrics/MetricController.kt + +@@ -50,7 +50,7 @@ interface MetricController { + + isMarketingDataTelemetryEnabled: () -> Boolean, + + settings: Settings + + ): MetricController { + +- return if (BuildConfig.TELEMETRY) { + ++ return if (true) { + + ReleaseMetricController( + + services, + + isDataTelemetryEnabled, + +``` + +2. Trigger your pings. +3. Sends the ping sing this command: +``` +adb shell am start -n org.mozilla.fenix.debug/mozilla.telemetry.glean.debug.GleanDebugActivity \ + --ez logPings true \ + --es sendPing metrics \ + --es debugViewTag test-metrics-ping +``` +4. See the results on https://debug-ping-preview.firebaseapp.com/ + +The parameters `sendPing` can be `metrics` or `events` depending or your needs, additionally `debugViewTag` can be customize to your preferred tag `debugViewTag your-metrics-ping`. diff --git a/mobile/android/fenix/docs/Working-with-Strings.md b/mobile/android/fenix/docs/Working-with-Strings.md new file mode 100644 index 0000000000..2e51fbd516 --- /dev/null +++ b/mobile/android/fenix/docs/Working-with-Strings.md @@ -0,0 +1,29 @@ +# Working with Strings + +## Marking an unused string to be removed + +Removing strings manually could cause crashes in **Beta** and **Release** versions 💥 , as the removed strings could be [uplifted to release branches](https://github.com/mozilla-mobile/fenix/pull/20364) by mistake, where strings are still needed. For this reason, we need a special process to remove them. + +Any landed string that is not removed while in development in Nightly will persist through 3 Firefox releases (Nightly, Beta, Release) before we can remove them from our code base. For example, +if you want to remove a string that has already shipped prior to **Firefox Nightly 93**, the same string will still be in-use in **Firefox Beta 92** and **Firefox Release 91**. This means the string will be marked as unused and removed in 93 while still riding the train, and it can be removed safely when **Firefox Release 93** no longer ships, for instance, **Firefox Release 94** and beyond. + +To keep us safe when you want to remove strings from nightly: + +1. Add these attributes to the target strings `moz:removedIn="<<ACTUAL_NIGHTLY_VERSION>>"` and `tools:ignore="UnusedResources"`. + +```xml + <string name="onboarding_close" moz:removedIn="93" tools:ignore="UnusedResources">Close</string> +``` +Example PR https://github.com/mozilla-mobile/fenix/pull/20980. + +## When to remove an unused string and how + +Strings that have been tagged with `moz:removedIn` attributes are safe to be removed after the marked version is no longer shipping and no longer in-use or needed. + +Consult the [Firefox release calendar](https://wiki.mozilla.org/Release_Management/Calendar). Let's say the Beta cut just happened and we are at Firefox Nightly 109, Firefox Beta 108 and Firefox Release 107. Everything marked with `moz:removedIn` <= 106 can now be removed. + +You only need to remove the en-US strings within [values/strings.xml](https://searchfox.org/mozilla-mobile/source/fenix/app/src/main/res/values/strings.xml), and this change will propagate to the other locales. + +## Future + +It would be nice to add some automatization to delete the strings that have the `moz:removedIn` attributes where a full cycle has happened (3 releases versions from the actual release version). diff --git a/mobile/android/fenix/docs/adjust.md b/mobile/android/fenix/docs/adjust.md new file mode 100644 index 0000000000..e498771392 --- /dev/null +++ b/mobile/android/fenix/docs/adjust.md @@ -0,0 +1,64 @@ +# Adjust + +Firefox for Android tracks certain types of installs using a third-party install tracking framework called Adjust. The intention is to determine the origin of Firefox for Android installs by answering the question, “Did this user on this device install Firefox for Android in response to a specific advertising campaign performed by Mozilla?” + +The framework consists of a software development kit (SDK) linked into Firefox for Android and a data-collecting Internet service backend run by the German company [adjust GmbH](https://www.adjust.com). The Adjust SDK is open source and MIT licensed. It is hosted at [https://github.com/adjust/android_sdk](https://github.com/adjust/android_sdk). Firefox for Android pulls in an unmodified copy of the SDK using Gradle. The [Dependencies.kt](https://github.com/mozilla-mobile/fenix/blob/main/buildSrc/src/main/java/Dependencies.kt) contains the version of the framework that is being used. The SDK is documented at [https://docs.adjust.com](https://docs.adjust.com). + +## Adjust Integration + +The Adjust framework is abstracted via the [AdjustMetricService](https://github.com/mozilla-mobile/fenix/blob/main/app/src/main/java/org/mozilla/fenix/components/metrics/AdjustMetricsService.kt) class. All interaction with Adjust happens via this class. + +## Adjust Messages + +The Adjust SDK collects and sends one type of message to the Adjust backend: + +* At the start of a new application session, a *Session Message* is sent with basic system info and how often the app has been used since the last time. + +The message is documented below in more detail of what is sent in each HTTP request. All messages are posted to a secure endpoint at `https://app.adjust.com`. They are all `application/x-www-form-urlencoded` HTTP `POST` requests. + +### Session Message + +#### Request + +``` +bundle_id: org.mozilla.fenix +tracking_enabled: 0 +language: en +country: CA +app_version: 4.2 +device_name: Pixel 2 +app_version_short: 2.0 +needs_response_details: 0 +attribution_deeplink: 1 +session_count: 1 +os_name: android +event_buffering_enabled: 0 +idfv: 8D452BFB-0692-4E8C-9DE0-7578486A872E +hardware_name: J127AP +app_token: xxxxxxxxxxxx +os_version: 10.1 +environment: production +created_at: 2016-11-10T20:34:39.720Z-0500 +device_type: phone +idfa: 00000000-0000-0000-0000-000000000000 +sent_at: 2016-11-10T20:34:39.787Z-0500 +``` + +These parameters (including ones not exposed to Mozilla) are documented at [https://partners.adjust.com/placeholders/](https://partners.adjust.com/placeholders/) + +#### Response + +If the application was successfully attributed to a specific campaign, details for that campaign are sent back as a JSON response: + +``` +{ "app_token": "xxxxxxxxxxxx", + "adid": "00000000000000000000", + "attribution" { "tracker_token": "xxxxxx", + "tracker_name": "Network::CAMPAIGN::ADGROUP::CREATIVE", + "network": "Network", + "campaign":"CAMPAIGN", + "adgroup":"ADGROUP", + "creative":"CREATIVE" } } +``` + +The application has no use for this information and ignores it. diff --git a/mobile/android/fenix/docs/architecture-overview.md b/mobile/android/fenix/docs/architecture-overview.md new file mode 100644 index 0000000000..ce4c7cc046 --- /dev/null +++ b/mobile/android/fenix/docs/architecture-overview.md @@ -0,0 +1,173 @@ +# Architecture Overview + +## Unidirectional data flow + +Firefox for Android's presentation layer architecture is based on the concept of "unidirectional data flow." This is a popular approach in client side development, especially on the web, and is core to Redux, MVI, Elm Architecture, and Flux. Our architecture is not identical to any of these (and they are not identical to each other), but the base concepts are the same. For a basic understanding of the motivations and approach, see [the official Redux docs](https://redux.js.org/basics/data-flow). For an article on when unidirectional data flow is and is not a good approach, see [this](https://medium.com/swlh/the-case-for-flux-379b7d1982c6). These are both written from the perspective of React.js developers, but the concepts are largely the same. + +Our largest deviation from these architectures is that while they each recommend one large, global store of data, we have a single store per screen and several other global stores. This carries both benefits and drawbacks, which will be covered later in this document. + +## Important types + +## <a name="store">Store</a> + +### **Overview** + +A store of State. + +See [mozilla.components.lib.state.Store](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/lib/state/src/main/java/mozilla/components/lib/state/Store.kt) + +Holds app State. + +Receives [Actions](#action), which are used to compute new State using [Reducers](#reducer) and can have [Middlewares](#middleware) attached which respond to and manipulate actions. + +### **Description** +Maintains a [State](#state), a [Reducer](#reducer) to compute new State, and [Middleware](#middleware). Whenever the Store receives a new [Action](#action) via `store.dispatch(action)`, it will first pass the action through its chain of [Middleware](#middleware). These middleware can initiate side-effects in response, or even consume or change the action. Finally, the Store computes new State using previous State and the new action in the [Reducer](#reducer). The result is then stored as the new State, and published to all consumers of the store. + +It is recommended that consumers rely as much as possible on observing State updates from the store instead of reading State directly. This ensures that the most up to date State is always used. This can prevent subtle bugs around call order, as all observers are notified of the same State change before a new change is applied. + +There are several global stores like `AppStore` and `BrowserStore`, as well as Stores scoped to individual screens. Screen-based Stores can be persisted across configuration changes, but are generally created and destroyed during fragment transactions. This means that data that must be shared across Stores should be lifted to a global Store or should be passed as arguments to the new fragment. + +Screen-based Stores should be created using [StoreProvider.get](https://github.com/mozilla-mobile/firefox-android/blob/main/fenix/app/src/main/java/org/mozilla/fenix/components/StoreProvider.kt). + +------- + +## <a name="state">State</a> +### **Overview** +Description of the state of a screen or other area of the app. + +See [mozilla.components.lib.state.State](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/lib/state/src/main/java/mozilla/components/lib/state/State.kt) + +### **Description** +Simple, immutable data object that contains all of the backing data required to display a screen. This should ideally only include Kotlin/Java data types which can be easily tested, avoiding Android platform types. This is especially true of large, expensive types like `Context` or `View` which should never be included in State. + +As much as possible, the State object should be an accurate, 1:1 representation of what is actually shown on the screen. That is to say, the screen should look exactly the same any time a State with the same values is emitted, regardless of any previous changes. This is not always possible as Android UI elements are very stateful, but it is a good goal to aim for. + +One major benefit of rendering a screen based on a State object is its impact on testing. UI tests are notoriously difficult to build and maintain. If we are able to build a simple, reproducible [View](#view) (i.e., if we can trust that the View will render as expected), that allows us to test our UI by verifying the correctness of our State object. + +This also gives us a major advantage when debugging. If the UI looks wrong, check the State object. If it's correct, the problem is in the View. If not, check that the correct [Action](#action) was sent. If so, the problem is in the reducer. If not, check the component that sent the Action. This helps us quickly narrow down problems. + +------- + +## <a name="action">Action</a> +### **Overview** +Simple description of a State change or a user interaction. Dispatched to Stores. + +See [mozilla.components.lib.state.Action](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/lib/state/src/main/java/mozilla/components/lib/state/Action.kt) + +### **Description** +Simple data object that carries information about a [State](#state) change to a [Store](#store). An Action describes _something that happened_, and carries any data relevant to that change. For example, `HistoryFragmentAction.ChangeEmptyState(isEmpty = true)`, captures that the State of the history fragment has become empty. + +------- + +## <a name="reducer">Reducer</a> +### **Overview** +Pure function used to create new [State](#state) objects. + +See [mozilla.components.lib.state.Reducer](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/lib/state/src/main/java/mozilla/components/lib/state/Reducer.kt) + +Referenced by: [Store](#store) + +### **Description** +A function that accepts the previous State and an [Action](#action), then combines them in order to return the new State. It is important that all Reducers remain [pure](https://en.wikipedia.org/wiki/Pure_function). This allows us to test Reducers based only on their inputs, without requiring that we take into account the state of the rest of the app. + +Note that the Reducer is always called serially, as state could be lost if it were ever executed in parallel. + +------- + +## <a name="middleware">Middleware</a> +### **Overview** +A Middleware sits between the store and the reducer. It provides an extension point between dispatching an action, and the moment it reaches the reducer. + +### **Description** + +A Middleware responds to actions by performing side-effects, and can also be used to rewrite an Action, intercept an Action, or dispatch additional Actions. + +The Store will create a chain of Middleware instances and invoke them in order. Every Middleware can decide to continue the chain or disrupt the chain. A Middleware has no knowledge of what comes before or after it in the chain. + +------- + +## <a name="view">View</a> +### **Overview** +Initializes UI elements, then updates them in response to [State](#state) changes + +Observes: [Store](#store) + +### **Description** +The view defines the mapping of State to UI. This can include XML bindings, Composables, or anything in between. + +Views should be as dumb as possible, and should include little or no conditional logic outside of determining which branch of a view tree to display based on State. Ideally, each primitive value in a State object is set on some field of a UI element. + +Views set listeners to UI elements, which trigger dispatches of [Actions](#action) to [Stores](#store). + +In some cases, it can be appropriate to initiate side-effects from the view when observing State updates from the Store. For example, a menu might be displayed. + +------- + +## Important notes +- Unlike other common implementations of unidirectional data flow, which typically have one global Store of data, we maintain smaller Stores for each screen and several global Stores. + - There is often no need to maintain UI state for views that are destroyed, and this allows us to to operate within the physical hardware constraints presented by Android development, such as having more limited memory resources. +- Stores that are local to a feature or screen should usually be persisted across configuration changes in a ViewModel by using [StoreProvider.get](https://github.com/mozilla-mobile/firefox-android/blob/main/fenix/app/src/main/java/org/mozilla/fenix/components/StoreProvider.kt). + +------- + +## Simplified Example +When reading through live code trying to understand an architecture, it can be difficult to find canonical examples, and often hard to locate the most important aspects. This is a simplified example of a basic history screen that includes a list of history items and which can be opened, multi-selected, and deleted. + +The following are links to the example versions of the architectural components listed above. + +- [HistoryFragment](./architectureexample/HistoryFragmentExample.kt) +- [HistoryStore](./architectureexample/HistoryStoreExample.kt) +- [HistoryState](./architectureexample/HistoryStoreExample.kt) +- [HistoryReducer](./architectureexample/HistoryStoreExample.kt) +- [HistoryNavigationMiddleware](./architectureexample/HistoryNavigationMiddlewareExample.kt) +- [HistoryStorageMiddleware](./architectureexample/HistoryStorageMiddlewareExample.kt) +- [HistoryTelemetryMiddleware](./architectureexample/HistoryTelemetryMiddlewareExample.kt) + +------- + +## Historical architecture components +There are some out-of-date architecture components that may still be seen throughout the app. The original documentation for these components is captured below in order to preserve context, but should be removed once these components are refactored out. + +For more context on when and why these components were removed, see [the RFC proposing their removal](https://github.com/mozilla-mobile/firefox-android/pull/1466). + +### Known Limitations (of historical components, copied from above) +- Many [Interactors](#interactor) have only one dependency, on a single [Controller](#controller). In these cases, they typically just forward each method call on and serve as a largely unnecessary layer. They do, however, 1) maintain consistency with the rest of the architecture, and 2) make it easier to add new Controllers in the future. + +------- + +## <a name="interactor"/>Interactor +### Overview +Called in response to a direct user action. Delegates to something else + +Called by: [View](#view) + +Calls: [Controllers](#controller), other Interactors + +### Description +This is the first object called whenever the user performs an action. Typically this will result in code in the [View](#view) that looks something like `some_button.onClickListener { interactor.onSomeButtonClicked() } `. It is the Interactors job to delegate this button click to whichever object should handle it. + +Interactors may hold references to multiple other Interactors and Controllers, in which case they delegate specific methods to their appropriate handlers. This helps prevent bloated Controllers that both perform logic and delegate to other objects. + +Sometimes an Interactor will only reference a single Controller. In these cases, the Interactor will simply forward calls to equivalent calls on the Controller. The Interactor does very little in these cases, and exists only to be consistent with the rest of the app. + +Note that prior to the introduction of Controllers, Interactors handled the responsibilities of both objects. **You may still find this pattern in some parts of the codebase,** but it is being actively refactored out. + +------- + +## <a name="controller"/>Controller +### Overview +Determines how the app should be updated whenever something happens + +Called by: [Interactor](#interactor) + +Calls: [Store](#store), library code (e.g., forward a back-press to Android, trigger an FxA login, navigate to a new Fragment, use an Android Components UseCase, etc) + +### Description +This is where much of the business logic of the app lives. Whenever called by an Interactor, a Controller will do one of the three following things: +- Create a new [Action](#action) that describes the necessary change, and send it to the Store +- Navigate to a new fragment via the NavController. Optionally include any state necessary to create this new fragment +- Interact with some third party manager. Typically these will update their own internal state and then emit changes to an observer, which will be used to update our Store + +Controllers can become very complex, and should be unit tested thoroughly whenever their methods do more than delegate simple calls to other objects. + +------- diff --git a/mobile/android/fenix/docs/architectureexample/HistoryFragmentExample.kt b/mobile/android/fenix/docs/architectureexample/HistoryFragmentExample.kt new file mode 100644 index 0000000000..f3fa6e3ee5 --- /dev/null +++ b/mobile/android/fenix/docs/architectureexample/HistoryFragmentExample.kt @@ -0,0 +1,60 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +// This is example code for the 'Simplified Example' section of +// /docs/architecture-overview.md +class HistoryFragment : Fragment() { + + private val store by lazy { + StoreProvider.get(this) { + HistoryStore( + initialState = HistoryState.initial, + middleware = listOf( + HistoryNavigationMiddleware(findNavController()) + HistoryStorageMiddleware(HistoryStorage()), + HistoryTelemetryMiddleware(), + ) + ) + } + } + + override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?): View? { + return ComposeView(requireContext()).apply { + setContent { + HistoryScreen(store) + } + } + } +} + +@Composable +private fun HistoryScreen(store: HistoryStore) { + val state = store.observeAsState(initialValue = HistoryState.initial) { state -> state } + val listState = rememberLazyListState() + LazyColumn(listState) { + if (state.selectedItems.isNotEmpty()) { + HistoryMultiSelectHeader( + onDeleteSelectedClick = { + store.dispatch(HistoryAction.DeleteItems(state.selectedItems)) + } + ) + } else { + HistoryHeader( + onDeleteAllClick = { store.dispatch(HistoryAction.DeleteItems(state.items)) } + ) + } + items(items = state.displayItems, key = { item -> item.id } ) { item -> + val isSelected = state.selectedItems.find { selectedItem -> + selectdItem == item + } + HistoryItem( + item = item, + isSelected = isSelected, + onClick = { store.dispatch(HistoryAction.OpenItem(item)) }, + onLongClick = { store.dispatch(HistoryAction.ToggleItemSelection(item)) }, + onDeleteClick = { store.dispatch(HistoryAction.DeleteItems(listOf(item))) }, + ) + } + } +} diff --git a/mobile/android/fenix/docs/architectureexample/HistoryNavigationMiddlewareExample.kt b/mobile/android/fenix/docs/architectureexample/HistoryNavigationMiddlewareExample.kt new file mode 100644 index 0000000000..49f3d42f2b --- /dev/null +++ b/mobile/android/fenix/docs/architectureexample/HistoryNavigationMiddlewareExample.kt @@ -0,0 +1,29 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +// This is example code for the 'Simplified Example' section of +// /docs/architecture-overview.md +class HistoryNavigationMiddleware( + private val navController: NavController, +) : Middleware<HistoryState, HistoryAction> { + override fun invoke( + context: MiddlewareContext<HistoryState, HistoryAction>, + next: (HistoryAction) -> Unit, + action: HistoryAction, + ) { + // This middleware won't need to manipulate the action, so the action can be passed through + // the middleware chain before the side-effects are initiated + next(action) + when(action) { + is HistoryAction.OpenItem -> { + navController.openToBrowserAndLoad( + searchTermOrURL = item.url, + newTab = true, + from = BrowserDirection.FromHistory, + ) + } + else -> Unit + } + } +} diff --git a/mobile/android/fenix/docs/architectureexample/HistoryStorageMiddlewareExample.kt b/mobile/android/fenix/docs/architectureexample/HistoryStorageMiddlewareExample.kt new file mode 100644 index 0000000000..674adca851 --- /dev/null +++ b/mobile/android/fenix/docs/architectureexample/HistoryStorageMiddlewareExample.kt @@ -0,0 +1,39 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +// This is example code for the 'Simplified Example' section of +// /docs/architecture-overview.md +class HistoryStorageMiddleware( + private val storage: HistoryStorage + private val scope: CoroutineScope, +) : Middleware<HistoryState, HistoryAction> { + override fun invoke( + context: MiddlewareContext<HistoryState, HistoryAction>, + next: (HistoryAction) -> Unit, + action: HistoryAction, + ) { + // This middleware won't need to manipulate the action, so the action can be passed through + // the middleware chain before the side-effects are initiated + next(action) + when(action) { + is HistoryAction.Init -> { + scope.launch { + val history = storage.load() + context.store.dispatch(HistoryAction.ItemsChanged(history)) + } + } + is HistoryAction.DeleteItems -> { + scope.launch { + val currentItems = context.state.items + if (storage.delete(action.items) is HistoryStorage.Success) { + context.store.dispatch( + HistoryAction.DeleteFinished() + ) + } + } + } + else -> Unit + } + } +} diff --git a/mobile/android/fenix/docs/architectureexample/HistoryStoreExample.kt b/mobile/android/fenix/docs/architectureexample/HistoryStoreExample.kt new file mode 100644 index 0000000000..c53d8d2981 --- /dev/null +++ b/mobile/android/fenix/docs/architectureexample/HistoryStoreExample.kt @@ -0,0 +1,64 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +// This is example code for the 'Simplified Example' section of +// /docs/architecture-overview.md +class HistoryStore( + private val initialState: HistoryState, + private val middleware: List<Middleware<HistoryState, HistoryAction>> +) : Store<HistoryState, Reducer<HistoryState, HistoryAction>>(initialState, middleware, ::reducer) { + init { + // This will ensure that middlewares can take any actions they need to during initialization + dispatch(HistoryAction.Init) + } +} + +sealed class HistoryAction { + object Init : HistoryAction() + data class ItemsChanged(val items: List<History>) : HistoryAction() + data class DeleteItems(val items: List<History>) : HistoryAction() + data class DeleteFinished() : HistoryAction() + data class ToggleItemSelection(val item: History) : HistoryAction() + data class OpenItem(val item: History) : HistoryAction() +} + +data class HistoryState( + val items: List<History>, + val selectedItems: List<History>, + val itemsBeingDeleted: List<History>, + companion object { + val initial = HistoryState( + items = listOf(), + selectedItems = listOf(), + itemsBeingDeleted = listOf(), + ) + } +) { + val displayItems = items.filter { item -> + item !in itemsBeingDeleted + } +} + +fun reducer(oldState: HistoryState, action: HistoryAction): HistoryState = when (action) { + is HistoryAction.ItemsChanged -> oldState.copy(items = action.items) + is HistoryAction.DeleteItems -> oldState.copy(itemsBeingDeleted = action.items) + is HistoryAction.DeleteFinished -> oldState.copy( + items = oldState.items - oldState.itemsBeingDeleted, + itemsBeingDeleted = listOf(), + ) + is HistoryAction.ToggleItemSelection -> { + if (oldState.selectedItems.contains(action.item)) { + oldState.copy(selectedItems = oldState.selectedItems - action.item) + } else { + oldState.copy(selectedItems = oldState.selectedItems + action.item) + } + } + else -> Unit +} + +data class History( + val id: Int, + val title: String, + val url: Uri, +) diff --git a/mobile/android/fenix/docs/architectureexample/HistoryTelemetryMiddlewareExample.kt b/mobile/android/fenix/docs/architectureexample/HistoryTelemetryMiddlewareExample.kt new file mode 100644 index 0000000000..9bda08ceeb --- /dev/null +++ b/mobile/android/fenix/docs/architectureexample/HistoryTelemetryMiddlewareExample.kt @@ -0,0 +1,22 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +// This is example code for the 'Simplified Example' section of +// /docs/architecture-overview.md +class HistoryTelemetryMiddleware : Middleware<HistoryState, HistoryAction> { + override fun invoke( + context: MiddlewareContext<HistoryState, HistoryAction>, + next: (HistoryAction) -> Unit, + action: HistoryAction, + ) { + // This middleware won't need to manipulate the action, so the action can be passed through + // the middleware chain before the side-effects are initiated + next(action) + when(action) { + is HistoryAction.DeleteItems -> History.itemsDeleted.record() + is HistoryAction.OpenItem -> History.itemOpened.record() + else -> Unit + } + } +} diff --git a/mobile/android/fenix/docs/certificates.md b/mobile/android/fenix/docs/certificates.md new file mode 100644 index 0000000000..1716bae036 --- /dev/null +++ b/mobile/android/fenix/docs/certificates.md @@ -0,0 +1,18 @@ +# Certificates + +The certificates for the official Fenix builds (`nightly`, `beta` and `production`) +can be accessed from the `certificates` folder in the root of this repository + +## Fingerprints + +Note: we currently only have the Fenix nightly deployed, and it's using the +production key at the moment. + +Channel | Fingerprint type | Fingerprint +---|---|--- +Nightly | SHA-1 | `2A:D2:D7:C0:80:BB:08:97:6B:46:D2:BB:50:5A:74:F1:FE:C9:5D:B5` +Nightly | SHA-256 | `77:EA:C4:CE:ED:36:AF:EF:BA:76:17:99:31:DD:4C:C1:95:AB:0C:D5:4B:AF:35:5D:21:5E:7B:BD:D2:8E:40:2A` +Beta | SHA-1 | `A1:B4:3E:62:20:43:41:FC:04:2B:15:89:77:0A:A9:18:AC:F7:AD:F9` +Beta | SHA-256 | `F5:62:C0:8F:30:77:86:86:D2:A4:7B:85:8F:45:E9:EF:35:70:83:08:5C:B2:89:1A:96:C4:09:F3:60:E9:CA:B9` +Production | SHA-1 | `0C:B5:98:40:E8:44:E4:00:56:BA:8F:23:3D:F1:C7:64:A7:DB:45:68` +Production | SHA-256 | `50:04:77:90:88:E7:F9:88:D5:BC:5C:C5:F8:79:8F:EB:F4:F8:CD:08:4A:1B:2A:46:EF:D4:C8:EE:4A:EA:F2:11` diff --git a/mobile/android/fenix/docs/crash-reporting.md b/mobile/android/fenix/docs/crash-reporting.md new file mode 100644 index 0000000000..979e4f3437 --- /dev/null +++ b/mobile/android/fenix/docs/crash-reporting.md @@ -0,0 +1,177 @@ +# Crash Reporting + +Firefox for Android uses a few libraries for crash and exception reporting. This kind of reporting gives Mozilla invaluable insight as to why Firefox for Android crashes or incorrectly behaves. It is one of the key methods we use to improve the product in terms of stability. + +This page documents the types of crash reporting, how the various parts interact, and what kind of data is sent back to Mozilla. + +Documentation for the specific libraries is included in the [Android Components Crash Reporting README](https://github.com/mozilla-mobile/android-components/blob/main/components/lib/crash/README.md). + +## Glean crash ping + +[Glean SDK](https://mozilla.github.io/glean/book/index.html) is a Mozilla open source telemetry library, which Firefox for Android uses to collect app telemetry. It can also collect crash counts as a labeled counter with each label corresponding to a specific type of crash (such as `native_code_crash`, `unhandled_exception`). + +The Glean crash ping format is documented [here](https://github.com/mozilla-mobile/android-components/blob/main/components/lib/crash/docs/metrics.md). + +To opt in or out of Glean telemetry reporting, visit the Data collection menu under Settings. + +## Breadcrumbs + +[Breadcrumbs](https://github.com/mozilla-mobile/android-components/blob/main/components/support/base/src/main/java/mozilla/components/support/base/crash/Breadcrumb.kt) are trail of events that are sent with each crash report to both Socorro and Sentry. + +### Events + +In [HomeActivity](https://github.com/mozilla-mobile/fenix/blob/main/app/src/main/java/org/mozilla/fenix/HomeActivity.kt) when `onDestinationChanged` occurs, the destination fragment's name and and whether it is a custom tab is added to the breadcrumbs. + +## Socorro + +[Socorro](https://wiki.mozilla.org/Socorro) is a Mozilla open source project for [crash statistics](https://crash-stats.mozilla.org/). Firefox for Android uses Socorro to track native GeckoView crashes. Crash reports contain a signature, classifications, and a number of improved fields (e.g. OS, product, version) - you can read more about what is sent in these fields in the [Socorro report documentation](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Crash_reporting/Understanding_crash_reports). + +These crashes contain hardware information and some app metadata, but no personally identifiable information is visible in the report. A few privacy-sensitive parts are only available to users who have "minidump access", which is a relatively small number of users with specific rules they must follow. + +A sample Firefox for Android crash report can be found [here](https://crash-stats.mozilla.org/report/index/bbbcc019-f30c-4fbb-8cbd-543940190923). + +A crash report is only sent when the user confirms and submits it through the crash reporter notification or dialog. Crash reports are never automatically sent. + +## Sentry + +[Sentry](https://sentry.io) is an open source crash reporting and aggregation platform. Both the client SDK, [github.com/getsentry/sentry-java](https://github.com/getsentry/sentry-java), and the server, [github.com/getsentry/sentry](https://github.com/getsentry/sentry), are open source. + +A crash report is only sent when the user confirms and submits it through the crash reporter notification or dialog. Crash reports are never automatically sent. + +### High-Level Summary + +The server is hosted and maintained by Mozilla. There are no third-parties involved, all crash reports are sent directly from Firefox for Android to the Sentry server hosted by Mozilla. + +On the client side Sentry is invisible. There are no parts to interact with. It reports crashes and fatal errors back to Mozilla in the background. + +On the server side there is a dashboard that the Firefox for Android team uses to look at incoming crash reports. The dashboard lets us inspect the crash report in detail and for example see where in the application the crash happened, what version of the application was used and what version of Android OS was active. Below is an overview of all the attributes that are part of a crash report. + +### Sentry Reports + +A typical Sentry crash report contains three categories of data: device, application, crash. It also contains some metadata about the crash report: +``` + "id": "6ae18611d6c649529a5eda0e48f42cb4", +// ... + "datetime": "2018-03-30T23:55:03.000000Z", +// ... + "received": 1522454183.0, +``` + +To clarify, `id` is a unique identifier for this crash report, *not a unique identifier for the user sending the report.* We explicitly disable the ability to uniquely identify users from their crash reports. + +#### Device Information + +Sentry collects basic information about the device the application is running on. Both static (device type) and dynamic (memory in use, device orientation). + +``` +"contexts": { + "device": { + "screen_resolution":"1920x1080", + "battery_level":44.0, + "orientation":"portrait", + "family":"ONEPLUS", + "model_id":"PQ3A.190705.003", + "type":"device", + "low_memory":false, + "simulator":false, + "free_storage":21314179072, + "storage_size":56416321536, + "screen_dpi":420, + "free_memory":2506031104, + "memory_size":6005846016, + "online":true, + "charging":false, + "model":"ONEPLUS A5000", + "screen_density":2.625, + "arch":"arm64-v8a", + "brand":"OnePlus", + "manufacturer":"OnePlus" + }, +// ... + "os":{ + "rooted":true, + "kernel_version":"4.4.184-sigmaKernel-v11.0", + "version":"9", + "build":"PQ3A.190705.003", + "type":"os", + "name":"Android" + } +} +``` + +### Application Information + +Sentry collects basic information about the Firefox for Android app. + +``` + "app":{ + "app_identifier":"org.mozilla.fenix", + "app_name":"Firefox Preview", + "app_start_time":"2019-09-23T21:00:05Z", + "app_version":"1.4.1", + "type":"app", + "app_build":12531634 + }, + "sdk":{ + "version":"1.7.10-598d4", + "name":"sentry-java" + } +``` + +### Crash Information + +#### Stack trace + +Every crash report contains a *stack trace*, which shows what functions in the Firefox for Android code led to this crash. It includes names of Android framework functions and Firefox for Android functions. Here's an excerpt of three lines from the stack trace: + +``` + "sentry.interfaces.Exception": { + "exc_omitted": null, + "values": [ + { + "stacktrace": { + "frames": [ + { + "function": "main", + "abs_path": "ZygoteInit.java", + "module": "com.android.internal.os.ZygoteInit", + "in_app": false, + "lineno": 801, + "filename": "ZygoteInit.java" + }, + { + "function": "run", + "abs_path": "ZygoteInit.java", + "module": "com.android.internal.os.ZygoteInit$MethodAndArgsCaller", + "in_app": false, + "lineno": 911, + "filename": "ZygoteInit.java" + }, + { + "function": "invoke", + "abs_path": "Method.java", + "in_app": false, + "module": "java.lang.reflect.Method", + "filename": "Method.java" +}, +``` + +##### Exception message + +The first line of every stack trace in every crash report contains a *reason* - why did this crash happen. This reason is provided by the developers who wrote the code that decide the app is in an error state. These developers include the Firefox for Android team at Mozilla, the Android framework, the Java programming language, and any libraries Mozilla bundles to develop Firefox for Android. + +Java, the Android framework, and Mozilla are diligent about making sure that no personally identifiable information is put in any of these messages. We keep them technical and to the point. We at Mozilla stay on top of our dependencies to ensure they're not including personally identifiable information as well. + +Here's an example message generated by Java: +``` +java.lang.StringIndexOutOfBoundsException: length=0; regionStart=20; regionLength=20 +``` + +Example of a Firefox for Android generated message: +``` +java.lang.StringIndexOutOfBoundsException: Cannot create negative-length String +``` + +##### Raw data dump + +In the explanations above, some redundant fields and field considered less important were omitted for brevity. To review these omissions, [this is an example of the raw data the server receives](https://gist.github.com/mcomella/50622aef817b40a20714b8550fb19991). This is up-to-date as of March 30, 2018. diff --git a/mobile/android/fenix/docs/experiments.md b/mobile/android/fenix/docs/experiments.md new file mode 100644 index 0000000000..e52f4ec458 --- /dev/null +++ b/mobile/android/fenix/docs/experiments.md @@ -0,0 +1,8 @@ +# Experiments + +Fenix uses Mozilla's [Experiments service for experimentation](https://mozilla.github.io/experimenter-docs/fenix-engineers). +Refer to the instructions [here](https://mozilla.github.io/experimenter-docs/fenix-engineers/) for details on how to configure them. + +## Experiment Opt Out + +Users can opt out of experiments at any time via Settings -> Data collection diff --git a/mobile/android/fenix/docs/index.rst b/mobile/android/fenix/docs/index.rst new file mode 100644 index 0000000000..d6ddb75f62 --- /dev/null +++ b/mobile/android/fenix/docs/index.rst @@ -0,0 +1,42 @@ +================================= +Fenix +================================= + +Specific documentation on a few topics is available at: + +.. toctree:: + :maxdepth: 1 + + Acronym Dictionary <Acronym-dictionary.md> + Addressing a performance regression <Addressing-a-performance-regression.md> + Architecture Decisions <Architecture-Decisions.md> + Architecture Overview <architecture-overview.md> + Crash Monitoring <Crash-Monitoring.md> + Creating a release branch<Creating-a-release-branch.md> + Data practises <Data-Practices.md> + Development Test Plan <Development-Test-Plan.md> + Fennec Migration <Fennec-Migration.md> + Firebase Cloud Messaging for WebPush <Firebase-Cloud-Messaging-for-WebPush.md> + Firefox for Android Team Processes <Firefox-for-Android-Team-Processes.md> + Guide to merging contributor PRs <Guide-to-merging-contributor-PRs.md> + Home <Home.md> + Implementing Telemetry <Implementing-Telemetry.md> + Implementing a Nimbus Experiment <Implementing-a-Nimbus-Experiment.md> + List of Fenix Threads <List-of-fenix-threads.md> + Logging Crash Information <Logging-Crash-Information.md> + Metric Feature Tags <Metric-Feature-Tags.md> + Debug settings menu instructions <Secret-settings-debug-menu-instructions.md> + Telemetry - Implementation, Reviews, Renewals <Telemetry-implementation,-reviews,-renewals.md> + Test Telemetry Pings <Test-telemetry-pings.md> + Working with Strings <Working-with-Strings.md> + Adjust <adjust.md> + Certificates <certificates.md> + Crash Reporting <crash-reporting.md> + Experiments <experiments.md> + l10n Screenshots Tests <l10nScreenshotsTests.md> + Manual Testing <manual-testing.md> + Metrics Definitions <metrics.md> + Release Checklist <release-checklist.md> + Substituting Local GeckoView <substituting-local-gv.md> + Sync Integration Tests <syncIntegrationTests.md> + Telemetry <telemetry.md> diff --git a/mobile/android/fenix/docs/l10nScreenshotsTests.md b/mobile/android/fenix/docs/l10nScreenshotsTests.md new file mode 100644 index 0000000000..637e290895 --- /dev/null +++ b/mobile/android/fenix/docs/l10nScreenshotsTests.md @@ -0,0 +1,34 @@ +# l10n Screenshots Test + +### Screenshtos Tests +We are using [`screengrab`](https://docs.fastlane.tools/getting-started/android/screenshots/) which works with fastlane to automate the process of capturing screenshots. +All the l10n screenshots are generated through the ui tests. These particular tests run as part of the screenshots package (`app/src/androidTest/mozilla/fenix/ui/screenshots`) + +### Run tests locally from Android Studio +Navigate to `app/src/androidTest/mozilla/fenix/ui/screenshots`, once in that directory, run the full test suite or a specific test by clicking on the `>` button. + +By running them manually you can check whether the test works or not but screenshots will not be saved. + +### Run tests locally from command line +1. Install the gem: +`sudo gem install screengrab` + +2. From command line run: +`fastlane screengrab --test_instrumentation_runner "androidx.test.runner.AndroidJUnitRunner"` + +The package configuration, apk paths as well as the locales are set in [Screengrab file](https://github.com/mozilla-mobile/fenix/blob/073fd8939067bc7a367d8db497bcf53fbd24cdd2/fastlane/Screengrabfile#L5). +In case there is a change there the file has to be modified accordingly. +Before launching that command, there has to be an emulator running. + +Once the build and tests finsish, screenshots will be saved in the root directory: `fastlane/metadata/android` +If there is a failure and screenshots are not saved, it may be necessary to create these folders manually first. + +## Run tests on CI +Currently there is a cron job scheduled weekly to run these tests on Monday. +It is set [here](https://github.com/mozilla-mobile/fenix/blob/5a8a7f549946fc8ad6ccf31f8c9c6bc2180aaed2/.cron.yml#L27). And the test results can be seen in [treeherder](https://treeherder.mozilla.org/jobs?repo=fenix&fromchange=67fb033f1bc8b772b64a1fda410632dd7e59450e&selectedTaskRun=NngGd-kXTtGGDpI9sMl2Bw.0), see previous link as an example. + +### TBD +So far, the tests run and the results are checked to be sure these tests work but the screenshots are not taken/saved on CI. That could be done in the future if there is a request from l10n team or any team interested in that. +See how it is done on [iOS](https://github.com/mozilla-mobile/firefox-ios/wiki/Screenshots-for-Localization) which integrates Taskcluster as job/task orchestrator, Bitrise to run the tests to take the screenshots and Taskcluster again to store and publicly share the screenshots. + +For any doubt or comment about these tests, please contact [Mobile Test Engineering](https://mana.mozilla.org/wiki/display/MTE/Mobile+Test+Engineering) team, slack: [#mobile-testeng](https://mozilla.slack.com/archives/C02KDDS9QM9) diff --git a/mobile/android/fenix/docs/manual-testing.md b/mobile/android/fenix/docs/manual-testing.md new file mode 100644 index 0000000000..a2e9e059e6 --- /dev/null +++ b/mobile/android/fenix/docs/manual-testing.md @@ -0,0 +1,125 @@ +# Softvision Mobile QA - Fenix testing tasks and process +============= + +Overview +-------- + +## Release +- Frequency: Fenix release schedule +- Tasks performed by the QA team: + - Smoke and sanity testing + - Exploratory testing + - Localization testing + - Bug triage +- Specific Release testing tasks: none +- Feature coverage: yes +- Bug verification coverage: uplifts + + +## Beta +- Frequency: Fenix release schedule +- Tasks performed by the QA team: + - Smoke and sanity testing + - Exploratory testing + - Localization testing + - Bug triage +- Specific Beta testing tasks: + - Full functional & UI testing + - TalkBack & Accessibility testing + - Full Search testing +- Feature coverage: yes +- Bug verification coverage: uplifts + +## Nightly +- Frequency: daily +- Tasks performed by the QA team + - Smoke and sanity testing + - Exploratory testing + - Bug triage +- Specific Nightly testing tasks: + - Bug verification (qa label) + - Feature testing + - Test case creation (including a11y) +- Feature coverage: yes +- Bug verification coverage: yes + +### Device defaults +- Device coverage: (unless otherwise specified): Pixel, Samsung, Xiaomi, OnePlus, Huawei + - Phone & tablets + - Android version: all + +## Detailed informations about the tasks performed + +#### Full-functional & UI testing +- Duration: 2 days +- Frequency: + - Upon Geckoview release (Beta 1) + - After Geckoview release, depending on the issues uplifted (if > 10 issues) +- Description: + - Set of tests that cover all functionalities + - 2 runs: 1 tablet, and 1 for phone + +#### Smoke & sanity testing +- Duration: 1 day +- Frequency: + - Release & Beta: Fenix release schedule + - Nightly: 2-3 times per week (depending of other tasks priority) +- Description: + - Small suite of tests focused on all major functionalities + +#### Feature testing +- Duration: based on feature complexity +- Frequency: when a new feature is implemented +- Description: + - Creation of test cases (a11y included) + - Feature bug verification (also duplicates, if it is the case) + - Exploratory testing around the new implementation and different areas that might be affected + +#### Bug verification (qa label & uplifts) +- Duration: based on bug complexity +- Frequency: daily/when qa label is added to fixed bugs +- Description: + - Different devices covered + - Verify the steps provided in the description on an affected build, in order to reproduce the bug (if it wasn't earlier) and on the build that contains the patch, to confirm the fix. + +#### Localization testing +- Duration + - Beta: 9hrs + - Release: 6hrs +- Frequency: Upon Geckoview release +- Description + - Suite of tests based on the most important languages and pseudo locale tests. + - Additionally, the number of languages listed ( Fenix settings) are verified to be the same number as in Pontoon. + +#### Search testing +- Duration: 1 day +- Frequency: Upon Geckoview release +- Description + - Set of tests that cover the interaction of users with URL bar, search engines & search codes (VPN). + +#### Accessibility testing +- Duration + - TalkBack: 1,5 day + - Scanner app: 1 day +- Frequency: Upon Geckoview release +- Description + - Tests are focused on the important functionalities + - TalkBack: check for issues when interacting with the app and the description of actions that are being performed + - Scanner: Menus, snackbars, others are being scanned in order to find suggestions for text contrast and touch target size + +#### Bug triage +- Duration: based on issue complexity +- Frequency: + - Daily/depending on the impact logged by the users +- Description: + - Issue investigation based on the information provided by the user + +#### Exploratory testing +- Duration: based on area tested +- Frequency (performed with): + - Smoke & sanity testing + - Full functional & UI tests + - Bug verification + - Bug triage +- Description: + - Testing scenarios that are not covered in test runs diff --git a/mobile/android/fenix/docs/metrics.md b/mobile/android/fenix/docs/metrics.md new file mode 100644 index 0000000000..24532087cc --- /dev/null +++ b/mobile/android/fenix/docs/metrics.md @@ -0,0 +1,5 @@ +# Metrics Definitions + +Metrics definitions for Firefox for Android have moved to the [Glean Dictionary](https://dictionary.telemetry.mozilla.org/apps/fenix). + +This file is kept only for historical reference. diff --git a/mobile/android/fenix/docs/release-checklist.md b/mobile/android/fenix/docs/release-checklist.md new file mode 100644 index 0000000000..0a2b954b47 --- /dev/null +++ b/mobile/android/fenix/docs/release-checklist.md @@ -0,0 +1,38 @@ +# Release Checklist + +## Overview ## + +Firefox for Android roughly follows the [Firefox Gecko release schedule](https://wiki.mozilla.org/Release_Management/Calendar#Calendars). +This means we cut a Beta every 4 weeks, with a full cycle (~4 weeks) of baking on Beta before going to Production release. + +The [Firefox for Android release schedule](https://docs.google.com/spreadsheets/d/1HotjliSCGOp2nTkfXrxv8qYcurNpkqLWBKbbId6ovTY/edit#gid=0) contains more details related to specific Mobile handoffs. + +### Requirements +- JIRA access +- Bugzilla account +- Sentry access + +## Firefox for Android Release +There are two releases this covers: the current changes in the Fenix Nightly channel that is going out to Beta, and the current Beta that is going to Production. + +## Cutting a Beta + +- [ ] Review [FeatureFlags](https://github.com/mozilla-mobile/fenix/blob/main/app/src/main/java/org/mozilla/fenix/FeatureFlags.kt) to determine if there are features that need to be enabled (or disabled) for Beta and Production release of Fenix. This will be a discussion with PO, PMs, EMs. +- [ ] Make a new Beta: Follow instructions [here](https://github.com/mozilla-mobile/fenix/wiki/Creating-a-release-branch) and notify the Release Management team (slack: #releaseduty-mobile). QA team is notified that a Beta release has been captured and they will run tests for Beta release sign-off +- [ ] Once there is GREEN QA signoff, the Release Management team (slack: #releaseduty-mobile) pushes the Beta version in the [Google Play Console](https://play.google.com/console/) +- [ ] Check Sentry each day for issues on [Firefox Beta](https://sentry.prod.mozaws.net/operations/firefox-beta/) and if nothing concerning, Release Management team bumps releases to 25%. Subsequent Beta builds are bumped to 100% assuming no blocking issues arise. +### Bugfix uplifts / Beta Product Integrity +- [ ] If bugs are considered release blocker then find someone to fix them on main and the milestone branch (cherry-pick / uplift) + - [ ] Add the uplift request to the appropriate row in the [Uplifts document](https://docs.google.com/spreadsheets/d/1qIvHpcQ3BqJtlzV5T4M1MhbWVxkNiG-ToeYnWEBW4-I/edit#gid=0). Ask for approval of uplift from Release Owner [amedyne](https://github.com/amedyne) and then notify Release Management team (slack: #releaseduty-mobile) of the uplift changes +- Note: Beta release versions are captured at least once a week during the Beta cycle. + + +### Production Release Candidate +- Production Release Candidate is captured on the third week of Beta by the Release Management team (slack: #releaseduty-mobile). This is then sent to Quality Assurance for Production Release Testing Sign-off. + + +## Production Release +- [ ] Once there is GREEN QA signoff, the Production Release Candidate is pushed to the Alpha testing track in [Google Play Console](https://play.google.com/console/u/0/developers/7083182635971239206/app/4972519468758466290/releases/overview) by the Release Management team (slack: #releaseduty-mobile) +- [ ] If nothing is concerning, release management officially tags the Release Candidate as Production release, (usually 1 week after 1st Release Candidate) +- [ ] Check Sentry for new crashes. Follow instructions for [Crash Monitoring](https://github.com/mozilla-mobile/fenix/wiki/Crash-Monitoring). File issues and triage. +- [ ] If nothing concerning, release management bumps releases(5%, 25%, 100%) diff --git a/mobile/android/fenix/docs/substituting-local-gv.md b/mobile/android/fenix/docs/substituting-local-gv.md new file mode 100644 index 0000000000..27605c3caa --- /dev/null +++ b/mobile/android/fenix/docs/substituting-local-gv.md @@ -0,0 +1,76 @@ +# Substituting local GeckoView + +### 1. Manually publish local GeckoView to local Maven + +Publish our local GeckoView to our local maven: +```sh +./mach build && ./mach gradle \ + geckoview:publishWithGeckoBinariesDebugPublicationToMavenLocal \ + exoplayer2:publishDebugPublicationToMavenLocal +``` + +:warning: **This needs to be run every time you make changes.** :warning: + +You need to copy the version in the logs or run: +```sh +./mach build | grep version +``` +(ex. `115.0.20230511122045-SNAPSHOT`) + +### 2. Modify Fenix to consume local GV +Update the build.gradle and Gecko.kt file in Fenix (see the diffs below). Remember to update the GV version with the version you found in step 2! + +*fenix/build.gradle* +```diff +diff --git a/fenix/build.gradle b/fenix/build.gradle +index 6a635a4818..4c8cc28995 100644 +--- a/fenix/build.gradle ++++ b/fenix/build.gradle +@@ -5,6 +5,7 @@ import org.mozilla.fenix.gradle.tasks.GithubDetailsTask + buildscript { + // This logic is duplicated in the allprojects block: I don't know how to fix that. + repositories { ++ mavenLocal() + maven { + name "Mozilla Nightly" + url "https://nightly.maven.mozilla.org/maven2" +@@ -90,6 +91,7 @@ plugins { + allprojects { + // This logic is duplicated in the buildscript block: I don't know how to fix that. + repositories { ++ mavenLocal() + maven { + name "Mozilla Nightly" + url "https://nightly.maven.mozilla.org/maven2" +``` +*Gecko.kt* +```diff +diff --git a/android-components/plugins/dependencies/src/main/java/Gecko.kt b/android-components/plugins/dependencies/src/main/java/Gecko.kt +index bed3fb0161..2d3a19a96e 100644 +--- a/android-components/plugins/dependencies/src/main/java/Gecko.kt ++++ b/android-components/plugins/dependencies/src/main/java/Gecko.kt +@@ -9,7 +9,7 @@ object Gecko { + /** + * GeckoView Version. + */ +- const val version = "115.0.20230511131014" ++ const val version = "115.0.20230511122045-SNAPSHOT" + + /** + * GeckoView channel +@@ -23,7 +23,7 @@ object Gecko { + enum class GeckoChannel( + val artifactName: String, + ) { +- NIGHTLY("geckoview-nightly-omni"), ++ NIGHTLY("geckoview-default-omni"), + BETA("geckoview-beta-omni"), + RELEASE("geckoview-omni"), + } + +``` + +### 3. Build fenix with local GV +Now sync your gradle changes and build! + +An easy way to confirm you are using a local GV is switching your Android Studio project tool window to "Project" and looking in the root directory called "External Libraries" for "GeckoView". You should see something like `Gradle: org.mozilla.geckoview-default-omni:115.0.20230511122045-SNAPSHOT@aar` diff --git a/mobile/android/fenix/docs/syncIntegrationTests.md b/mobile/android/fenix/docs/syncIntegrationTests.md new file mode 100644 index 0000000000..d16ac75a35 --- /dev/null +++ b/mobile/android/fenix/docs/syncIntegrationTests.md @@ -0,0 +1,28 @@ +# Sync Integration Tests + +### Sync Integration Tests +The aim of these tests is to check that the synchronization is working between Fenix and Desktop. The intention is to add tests for History, Bookmarks, Tabs and Logins. +At this moment only tests for History and Bookmarks are defined. + +### Steps to Run +To run these tests you will need Python 2 and pipenv installed. Once you have these, make sure you're in the `syncintegration` directory and run the following: + +`$ pipenv install` +`$ pipenv run pytest` + +When a test is launched a stage account is created. That will be used both in Desktop and Fenix to be sure that what is saved in one place is shown in the other. + +The process for example for History item Desktop -> Fenix, would be: +- Desktop is launched, user signed in and history item created. +- Android sim is launched (Pixel 3 API28), Fenix app starts and same user is signed in, then we go to History list and verify that the item is there. + + +### Results +Due to the set up necessary these tests do not run as part of the regular CI, via Taskcluster. +The idea is to have them running on Jenkins periodically (TBD how often). +Once they finish there is a slack notificattion received informing about the result (so far that is configured for #firefox-ios-alerts) + +A html file is generated with all the info, for each step to make it easy to debug in case of failure. + +## Notes +More detailed info can be found [`here`](https://docs.google.com/document/d/1dhxlbGQBA6aJi2Xz-CsJZuGJPRReoL7nfm9cYu4HcZI/edit?usp=sharing) diff --git a/mobile/android/fenix/docs/telemetry.md b/mobile/android/fenix/docs/telemetry.md new file mode 100644 index 0000000000..2cd13390db --- /dev/null +++ b/mobile/android/fenix/docs/telemetry.md @@ -0,0 +1,13 @@ +# Telemetry + +Fenix uses Mozilla's telemetry service (Glean) to measure feature performance and engagement. + +## Glean pings and metrics + +By using the Glean SDK, Fenix can send the pings the SDK owns and defines, as documented [in the Glean SDK docs](https://mozilla.github.io/glean/book/user/pings/index.html). + +Additional metrics or pings defined by Fenix are documented in the [Glean Dictionary](https://dictionary.telemetry.mozilla.org/apps/fenix). + +## Crash reporting + +See [here](https://github.com/mozilla-mobile/fenix/blob/main/docs/crash-reporting.md) for details on crash reporting in Firefox for Android. |