diff options
Diffstat (limited to 'mobile/android/focus-android/docs')
20 files changed, 1317 insertions, 0 deletions
diff --git a/mobile/android/focus-android/docs/Adjust-Usage.md b/mobile/android/focus-android/docs/Adjust-Usage.md new file mode 100644 index 0000000000..03d85363f1 --- /dev/null +++ b/mobile/android/focus-android/docs/Adjust-Usage.md @@ -0,0 +1,66 @@ +# Adjust Usage + +> **If there is anything in this document that is not clear, is incorrect, or that requires more detail, please file a request through a [Github](https://github.com/mozilla-mobile/focus-android/issues) or [Bugzilla](https://bugzilla.mozilla.org/enter_bug.cgi?product=Focus&component=General). Also feel free to submit corrections or additional information.** + +Firefox Focus (but not Firefox Klar) tracks certain types of installs using a third-party install tracking framework called Adjust. The intention is to determine the origin of Firefox Focus installs by answering the question, “Did this user on this device install Firefox Focus in response to a specific advertising campaign performed by Mozilla?” + +The framework consists of a software development kit (SDK) linked into Firefox Focus and a data-collecting Internet service backend run by the German company [adjust GmbH](https://www.adjust.com). The Adjust SDK is open source and MIT licensed. It is hosted at [https://github.com/adjust/android_sdk](https://github.com/adjust/android_sdk). Firefox Focus pulls in an unmodified copy of the SDK using Gradle. The [build.gradle](https://github.com/mozilla-mobile/focus-android/blob/master/app/build.gradle) contains the version of the framework that is being used. The SDK is documented at [https://docs.adjust.com](https://docs.adjust.com). + +## Adjust Integration + +The Adjust framework is abstracted via the [AdjustHelper](https://github.com/mozilla-mobile/focus-android/blob/master/app/src/focusRelease/java/org/mozilla/focus/utils/AdjustHelper.java) class. All interaction with Adjust happens via this class. + +## Adjust Messages + +The Adjust SDK collects and sends one type of message to the Adjust backend: + +* At the start of a new application session, a *Session Message* is sent with basic system info and how often the app has been used since the last time. + +The message is documented below in more detail of what is sent in each HTTP request. All messages are posted to a secure endpoint at `https://app.adjust.com`. They are all `application/x-www-form-urlencoded` HTTP `POST` requests. + +### Session Message + +#### Request + +``` +bundle_id: org.mozilla.focus +tracking_enabled: 0 +language: en +country: CA +app_version: 4.2 +device_name: Pixel 2 +app_version_short: 2.0 +needs_response_details: 0 +attribution_deeplink: 1 +session_count: 1 +os_name: android +event_buffering_enabled: 0 +idfv: 8D452BFB-0692-4E8C-9DE0-7578486A872E +hardware_name: J127AP +app_token: xxxxxxxxxxxx +os_version: 10.1 +environment: production +created_at: 2016-11-10T20:34:39.720Z-0500 +device_type: phone +idfa: 00000000-0000-0000-0000-000000000000 +sent_at: 2016-11-10T20:34:39.787Z-0500 +``` + +These parameters (including ones not exposed to Mozilla) are documented at [https://partners.adjust.com/placeholders/](https://partners.adjust.com/placeholders/) + +#### Response + +If the application was successfully attributed to a specific campaign, details for that campaign are sent back as a JSON response: + +``` +{ "app_token": "xxxxxxxxxxxx", + "adid": "00000000000000000000", + "attribution" { "tracker_token": "xxxxxx", + "tracker_name": "Network::CAMPAIGN::ADGROUP::CREATIVE", + "network": "Network", + "campaign":"CAMPAIGN", + "adgroup":"ADGROUP", + "creative":"CREATIVE" } } +``` + +The application has no use for this information and ignores it. diff --git a/mobile/android/focus-android/docs/Architecture-Decisions.md b/mobile/android/focus-android/docs/Architecture-Decisions.md new file mode 100644 index 0000000000..f42a5431c5 --- /dev/null +++ b/mobile/android/focus-android/docs/Architecture-Decisions.md @@ -0,0 +1,92 @@ +# Architecture Decisions + +This is a collection of records for "architecturally significant" decisions. [Why?](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions) + +--- + +## Observing state (LiveData) + +* **ADR-3** +* Status: Accepted +* Affected version: 2.0+ + +### Context + +During the refactoring of Focus for Android to support multitasking it became apparent that our callback-based approach wasn't good enough anymore. Previously the current browser state was used to update the UI only. Now with multiple concurrent sessions this state needed to be preserved and restored while the user switched between tabs. In addition to that we wanted to update multiple UI components based on the current state (partially) for which a single callback implementation became very complex fast. + +Having observers for small chunks of data and a "reactive style" seemed to be a better approach. There are a lot of libraries that offer this functionality already, namely: RxJava, event buses (e.g. Otto and EventBus), Architecture Components (LiveData), Agera and others. + +### Decision + +LiveData objects from Android's Architecture Components library are used to wrap data that updates asynchronously. UI components that depend on this data can either get the current state or subscribe in order to get updates whenever the data changes. + +Reasons: + +* The Architecture Components library is written and maintained by the core Android team at Google (-> high quality and visibility) +* Parts of the library are going to be used by the support libraries. Therefore they will get included into Focus for Android sooner or later anyways. +* The library is pretty small. +* The library takes care of the Activity/Fragment lifecycle (Avoids common errors). + +### Consequences + +* LiveData was designed to be used with `ViewModel` instances. So far we do not use `ViewModel`s and are also observing LiveData objects from non-UI components. This is not problematic but can cause problems in future versions of the library - although unlikely. + +--- + +## Browser engine: GeckoView vs. WebView + +* **ADR-2** +* Status: Accepted +* Affected app versions: 1.0+ + +### Context + +To render web content Firefox Focus needs to use a "web browser engine". So far all browsers from Mozilla, and Firefox for Android in particular, are using the [Gecko engine](https://en.wikipedia.org/wiki/Gecko_(software)). + +Android itself ships with a [WebView](https://developer.android.com/guide/webapps/webview.html) component that is (on new Android versions) based on Chromium/Chrome ([Blink engine](https://en.wikipedia.org/wiki/Blink_(web_engine))). + +A number of existing Android browsers (e.g. Opera, Brave) are built on top of the Blink rendering engine or are a direct fork of Chromium. + +### Decision + +Firefox Focus is going to be build on top of Android's WebView. + +Reasons for using WebView: +* At the time of writing the Focus for Android prototype GeckoView already existed but it wasn't in a state that it could be used outside of Firefox for Android reliably. In addition to that there wasn't a stable API comparable offering the feature set of WebView. +* APK size has been a long-term concern of the Firefox for Android team. A large APK size has been problematic for partnership deals and distribution in countries where bandwidth is limited or expensive. GeckoView is roughly 30 MB in size, while WebView is part of the Android system and is basically "free". Prototype builds of Focus for Android based on WebView were less than 3 MB in size. + +In addition to that there will be a build configuration that uses GeckoView. The GeckoView version will be guaranteed to compile; but keeping feature parity or keeping the build bug free is not a goal of the team. At this time the GeckoView version is a tech demo to explore its future potential only. + +Forking Chromium (or using Blink) was considered a too large investment in the prototype stage. + +### Consequences + +WebView has a complex API. Nevertheless it doesn't allow us to do heavy low-level customization that we could do if we would own the web browser engine (e.g. GeckoView). It remains to be seen whether this limitation will prevent feature development in a way that forces us to ship a browser engine with Focus for Android. + +--- + +## Minimum supported Android version: 5.0+ (API 21+) + +* **ADR-1** +* Status: Accepted +* Affected app versions: 1.0+ + +### Context + +Every app needs to define a minimum supported SDK version. This is usually a trade-off between how many users can be reached ([Android version distribution](https://developer.android.com/about/dashboards/index.html)) and what platform features the app needs to support ([Android API level overview](https://developer.android.com/guide/topics/manifest/uses-sdk-element.html#ApiLevels)). + +### Decision + +Focus will support Android versions 5 and higher (API 21+). This decisions is primarily driven by the following platform features that are not available on earlier versions of Android: + +* [WebView.shouldInterceptRequest()](shouldInterceptRequest): Our content blocking content uses the implementation of the callback that lets us inspect the request object. This implementation is only available on Android 21+. + +* [UI features](https://developer.android.com/training/material/shadows-clipping.html) like `elevation` allow us to build and prototype a "material" UI without needing to backport functionality. + +### Consequences + +* At the point of writing this ADR we will cover 73.4% of the Android market with that decision [(*)](https://developer.android.com/about/dashboards/index.html). + +* Android 4.4 (KitKat) still covers 17.1%. Those users can't install and use Focus. While UI features can be backported, the extended WebView APIs can not. Only a GeckoView based version of Focus would be able to support Android 4.4 or lower. + +--- diff --git a/mobile/android/focus-android/docs/Battery-Debugging.md b/mobile/android/focus-android/docs/Battery-Debugging.md new file mode 100644 index 0000000000..e24e22e22a --- /dev/null +++ b/mobile/android/focus-android/docs/Battery-Debugging.md @@ -0,0 +1,37 @@ +# Battery Debugging + +Android Performance Profiling can be quite tricky in spite of the wealth of good tools available. This doc is to explain how to track down battery and performance issues. + +## Battery Historian + +Whenever battery issues arise on Android, the first Google suggestion is to use the Battery Historian tool to parse an Android bug report and show what caused the battery drain. + +While this is a good option for most Android apps, it proves next to useless for analyzing most battery issues in Focus and Klar. This is because Focus and Klar uses a foreground service that remains running as long as your web pages are open. The service is required to keep the app in the foreground so data can remain in memory rather than being persisted to disk. This keeps the app from being killed by the OS and sending all your precious web session and page data to the garbage collector. + +Battery Historian starts from the premise that most battery problems are caused by excessive wake locks. It helps the programmer by showing wake locks, when the WiFi or cellular radio is being kept awake, and the rate of battery drain at different times. Focus and Klar do not require wake locks and they wouldn't be interesting, since the apps remain in the foreground most of the time the app is running. + +Analyzing Battery Historian results can still prove interesting. [Here are the instructions](https://developer.android.com/studio/profile/battery-historian) to try. + +## Android Studio Profiler + +If Battery Historian does not point at the radio or screen being kept alive as the issue, the next thing is to look for excessive CPU activity. There are several tools to look into this, but the easiest to get started is the profiler built into Android Studio. + +Open the project in Android Studio. If there is no Android Profiler tab on the bottom of the screen, open it from View -> Tool Windows -> Android Profiler. + +Make sure you're running a debug version of the app and choose the device and debuggable process from the dropdowns at the top of the Profiler. + +Click the CPU graph to get additional CPU detail. Choose instrumented to increase the resolution of the data. If the battery drain is happening while the phone is idle, you can easily check to find out what is causing the CPU activity, since there shouldn't be much at all. + +[Here's more](https://developer.android.com/studio/profile/android-profiler) on how to use the Android Profiler in general. [Here's more on the CPU Profiler](https://developer.android.com/studio/profile/cpu-profiler) specifically. + +## Systrace + +Systrace does not add that much to what Android Studio's Profiler already provides and the interface is a clunky webpage. You can still try it if you like. [Here are the instructions](https://developer.android.com/studio/command-line/systrace). + +## Traceview + +While you can get a lot of useful and interesting information from the Android Profiler and Systrace, it's helpful to get full call stacks. The easiest way to do this is to use the older Android profiling tool, Traceview. You can select from the code when to start and stop tracing with "Debug.startMethodTracing("name")" and "Debug.stopMethodTracing()" + +When you run the code, it'll generate a trace file in the files folder on the SDcard. You can read the trace file either by double-clicking it in Android Studio's Device File Explorer or even better by opening it in Android Device Monitor. You can start Android Device Monitor in recent Android Studio versions by running $ANDROID_SDK/tools/monitor. + +[Here's more detail on using TraceView](https://developer.android.com/studio/profile/traceview). diff --git a/mobile/android/focus-android/docs/Content-blocking.md b/mobile/android/focus-android/docs/Content-blocking.md new file mode 100644 index 0000000000..0b1bf45a26 --- /dev/null +++ b/mobile/android/focus-android/docs/Content-blocking.md @@ -0,0 +1,58 @@ +# Content blocking + +## Outdated Content. +Focus is using the same technology as Firefox for desktop, all the content blocking is happening inside of the Gecko engine web engine. content blocking list can be seen here https://github.com/mozilla-services/shavar-prod-lists. + +## Tracking protection: lists and general overview + +We use the disconnect tracking protection lists, these consist of: + +- The deny-list: a list of domains to block +- The entitylist: an override list to unblock certain domains for certain other domains. +- The google_mapping: a list of modifications for the deny-list. + +### The deny-list + +The deny-list contains list of domains that should be blocked. Any resources that are hosted +on these domains should be blocked. + +The source lists contain multiple categories, we use **Advertising**, **Social**, **Analytics**, **Content** (aka. "other content trackers"). +We use some items from the **Disconnect** list, those items are moved into the **Social** list when parsing. We ignore **Legacy Disconnect** and +**Legacy Social**. You can see the code parsing these lists at +[BlockListProcessor.java](../app/src/webkit/java/org/mozilla/focus/webkit/matcher/BlocklistProcessor.java). + +The google_mapping is similar to the main list - these items are simply addded to the existing categories mentioned above. (The google entries +in the main **Disconnect** list are discarded since we use those from google_mapping.json.) + +Note: each category contains a list of site owners: those names are discarded, we insert every single domain for every owner into the same list. + +### Entitylist: the override list + +Some domains need to be unblocked when visiting another domain belonging to the same owner. E.g. we usually block "googleapis.com", but when visiting e.g. google.de, +we unblock "googleapis.com". This is done in the entitylist: for every domain that belongs to a "property" in the entitylist, we unblock all domains listed in +"resources". + +## Implementation + +WebView calls the WebViewClient.shouldInterceptRequest() callback every time a resource is to be loaded - this permits us to intercept resource loading, and is how we +can perform content blocking on Android. (See [BlockListProcessor.java](../app/src/webkit/java/org/mozilla/focus/webkit/TrackingProtectionWebViewClient.java) ). + +We then just need to verify every resource URL to determine whether it can be loaded in that callback: we use a custom trie-based domain matching implementation for +Focus for Android. This is different from Focus iOS: Focus for iOS was originally a content blocking safari plugin, and used the iOS content blocking API +( https://developer.apple.com/library/content/documentation/Extensions/Conceptual/ContentBlockingRules/Introduction/Introduction.html ). +That API uses a specific blocklist format, therefore firefox-ios converts the disconnect lists into the iOS format at build-time. + +Focus-ios browser was implemented later, and reuses the iOS content blocking list format (iOS webview was unable to make use of the content blocking API +at that time, and therefore implemented its own URL matching). This is a regex based format: Focus-ios (browser) therefore stores a list of regexes, and iterates +over that list to check whether a given resource URL matches. That approach means that focus-ios only needs one copy of the blocklists, however this isn't ideal for performance. + +As mentioned, Focus for Android uses a custom Trie implementation instead of iterating over regexes. This does mean that we aren't reusing iOS's blocklist +format, but it also permits for ~140x faster domain lookup when compared to a port of the iOS domain lookup implementation. The entitylist is similar, +and we use extended versions of the same Trie for the domain overrides. See [UrlMatcher](../app/src/webkit/java/org/mozilla/focus/webkit/matcher/UrlMatcher.java) for +the actual matcher implementation. + + +## Miscellaneous notes +- We do not make any internet connection because every blocklist is built into Firefox Focus. When you enable a blocklsit in settings, our app will load the selected blocklist from disk. We will provide updated lists as an "App update". +- We discard the site owners and names when parsing the blocklists, that makes it harder to keep track of which trackers have been blocked for a given site. That data would probably have to be added to the blocklist trie if we want a better breakdown of trackers. +- Our Trie search is recursive, and uses a slightly silly "TrieString" to take care of string reversal and substrings. An iterative implementation would probably be better, and would avoid substrings. Such an implementation would be slightly more complex since we would have to keep track of string indexes, but reduces overhead. Given that current performance is acceptable, there isn't huge value in actually working on this. diff --git a/mobile/android/focus-android/docs/Crash-Reporting-with-Sentry.md b/mobile/android/focus-android/docs/Crash-Reporting-with-Sentry.md new file mode 100644 index 0000000000..2e6f5d4f6c --- /dev/null +++ b/mobile/android/focus-android/docs/Crash-Reporting-with-Sentry.md @@ -0,0 +1,147 @@ +# Crash Reporting with Sentry + +> **If there is anything in this document that is not clear, is incorrect, or that requires more detail, please file a request through a [Github](https://github.com/mozilla-mobile/focus-android/issues).** + +Focus Android uses [Sentry](https://sentry.io) for crash and exception reporting. This kind of reporting gives Mozilla invaluable insight as to why Focus crashes or incorrectly behaves. It is one of the key methods we use to improve the product in terms of stability. + +This page explains how Sentry works, how the various parts interact and what kind of data it sends back to Mozilla. + +## High-Level Summary +[Sentry](https://sentry.io) is an open source crash reporting and aggregation platform. Both the client SDK, [github.com/getsentry/sentry-java](https://github.com/getsentry/sentry-java), and the server, [github.com/getsentry/sentry](https://github.com/getsentry/sentry), are open source. + +The server is hosted and maintained by Mozilla. There are no third-parties involved, all crash reports to directly from Focus Android to the Sentry server hosted by Mozilla. + +On the client side Sentry is invisible. There are no parts to interact with. It reports crashes and fatal errors back to Mozilla in the background. Sentry is enabled when the *Send Usage Data* switch in the *Focus* settings is enabled by the user. By default this switch is enabled in Focus and is an *opt-out* mechanism. In Klar, by default this switch is disabled and is an *opt-in* mechanism. + +On the server side there is a dashboard that the Focus team uses to look at incoming crash reports. The dashboard lets us inspect the crash report in detail and for example see where in the application the crash happened, what version of the application was used and what version of Android OS was active. Below is an overview of all the attributes that are part of a crash report. + +## Sentry Reports + +A typical Sentry crash report contains three categories of data: device, application, crash. It also contains some metadata about the crash report: +``` + "id": "6ae18611d6c649529a5eda0e48f42cb4", +// ... + "datetime": "2018-03-30T23:55:03.000000Z", +// ... + "received": 1522454183.0, +``` + +To clarify, `id` is a unique identifier for this crash report, *not a unique identifier for the user sending the report.* We explicitly disable the ability to uniquely identify users from their crash reports. + +### Device Information + +Sentry collects basic information about the device the application is running on. Both static (device type) and dynamic (memory in use, device orientation). + +``` +"contexts": { + "device": { + "screen_resolution": "1920x1080", + "battery_level": 100.0, + "orientation": "landscape", + "family": "AFTN", + "model_id": "NS6212", + "type": "device", + "low_memory": false, + "simulator": false, + "free_storage": 3967590400, + "storage_size": 5735825408, + "screen_dpi": 320, + "free_memory": 543588352, + "memory_size": 1392164864, + "online": true, + "charging": true, + "model": "AFTN", + "screen_density": 2.0, + "arch": "armeabi-v7a", + "brand": "Amazon", + "manufacturer": "Amazon" + }, +// ... + "os": { + "rooted": false, + "kernel_version": "Linux version 3.14.29 (build@14-use1b-b-42) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #1 SMP PREEMPT Fri Jan 19 00:36:45 UTC 2018", + "version": "7.1.2", + "build": "NS6212", + "type": "os", + "name": "Android" + } +}, +``` + +### Application Information + +Sentry collects basic information about the Focus app. + +``` + "app": { + "app_identifier": "org.mozilla.focus", + "app_name": "Focus", + "app_start_time": "2018-03-30T16:55:03Z", + "app_version": "2.1", + "type": "app", + "app_build": 11 +// ... + "sdk": { + "client_ip": "63.245.222.193", + "version": "1.7.2-02be9", + "name": "sentry-java" +``` + +### Crash Information + +#### Stack trace + +Every crash report contains a *stack trace*, which shows what functions in the Focus code led to this crash. It includes names of Android framework functions and Focus functions. Here's an excerpt of three lines from the stack trace: + +``` + "sentry.interfaces.Exception": { + "exc_omitted": null, + "values": [ + { + "stacktrace": { + "frames": [ + { + "function": "main", + "abs_path": "ZygoteInit.java", + "module": "com.android.internal.os.ZygoteInit", + "in_app": false, + "lineno": 801, + "filename": "ZygoteInit.java" + }, + { + "function": "run", + "abs_path": "ZygoteInit.java", + "module": "com.android.internal.os.ZygoteInit$MethodAndArgsCaller", + "in_app": false, + "lineno": 911, + "filename": "ZygoteInit.java" + }, + { + "function": "invoke", + "abs_path": "Method.java", + "in_app": false, + "module": "java.lang.reflect.Method", + "filename": "Method.java" +}, +``` + +#### Exception message +The first line of every stack trace in every crash report contains a *reason* - why did this crash happen. This reason is provided by the developers who wrote the code that decide the app is in an error state. These developers include the Focus team at Mozilla, the Android framework, the Java programming language, and any libraries Mozilla bundles to develop Focus. + +Java, the Android framework, and Mozilla are diligent about making sure that no personally identifiable information is put in any of these messages. We keep them technical and to the point. We at Mozilla stay on top of our dependencies to ensure they're not including personally identifiable information as well. + +Here's an example message generated by Java: +``` +java.lang.StringIndexOutOfBoundsException: length=0; regionStart=20; regionLength=20 +``` + +Example of a Focus generated message: +``` +java.lang.StringIndexOutOfBoundsException: Cannot create negative-length String +``` + +#### Raw data dump +In the explanations above, some redundant fields and field considered less important were omitted for brevity. To review these omissions, [this is an example of the raw data the server receives](https://gist.github.com/mcomella/50622aef817b40a20714b8550fb19991). This is up-to-date as of March 30, 2018. + +## For developers +For developer documentation such as how to enable Sentry in your builds, see [`SentryWrapper.kt`](https://github.com/mozilla-mobile/focus-android/blob/master/app/src/main/java/org/mozilla/focus/telemetry/SentryWrapper.kt) in the code base. diff --git a/mobile/android/focus-android/docs/Development-Custom-GeckoView.md b/mobile/android/focus-android/docs/Development-Custom-GeckoView.md new file mode 100644 index 0000000000..81525b7595 --- /dev/null +++ b/mobile/android/focus-android/docs/Development-Custom-GeckoView.md @@ -0,0 +1,90 @@ +# Development with Custom GeckoView + +If you are an engineer working on Gecko(View) then you might be interested in building Focus/Klar with your own build of GeckoView. + +For this you will need to: + +* Checkout mozilla-central (or another branch) +* Setup your system to build Firefox for Android +* Package a GeckoView [AAR](https://developer.android.com/studio/projects/android-library.html) +* Modify your Focus gradle configuration to use your custom GeckoView AAR. + +## Setup build system + +Follow the [Build instructions](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build) to setup a Firefox for Android build (ARM or x86). + +A minimal `mozconfig` for GeckoView development might look like this (ARM): + +``` +ac_add_options --enable-application=mobile/android +ac_add_options --target=arm-linux-androideabi +# For x86 use: ac_add_options --target=i386-linux-android + +ac_add_options --with-android-sdk="<path-to-android-SDK>" +ac_add_options --with-android-ndk="<path-to-android-NDK>" +``` + +## Package GeckoView AAR + +After setting up your build system you should be able to build and package Firefox for Android: + +```bash +./mach build +./mach package +``` + +Now you can create the GeckoView AAR from the compiled code: + +```bash +./mach android archive-geckoview +``` + +This should create a file named `geckoview-*.aar` in your build output folder (MOZ_OBJDIR): + +```bash +$ ls <your-output-directory>/gradle/build/mobile/android/geckoview/outputs/aar +geckoview-official-withGeckoBinaries-noMinApi-release.aar +``` + +## Point your Focus/Klar build to your AAR + +In your Focus/Klar checkout open `app/build.gradle` (__not__ the build.gradle file in the root directory!) and locate the `repositories` block. This block defines where gradle will look for dependencies. Add the absolute path to your AAR as follows: + +```groovy +repositories { + // ... + + flatDir( + name: 'localBuild', + dirs: '<absolute path to AAR>' + ) +} +``` + +Now locate the ```dependencies``` block. This block defines which dependencies are needed to compile the application. Locate the already existing `armImplementation` and `x86Implementation` statements. Those are currently pointing to AARs that are pulled from our build servers. Replace the correct one (x86 / ARM) to use the name of your local AAR: + +```groovy +dependencies { + // ... + + // armImplementation "org.mozilla:geckoview-nightly-armeabi-v7a:60.0a1" + armImplementation ( + name: 'geckoview-official-withGeckoBinaries-noMinApi-release', + ext: 'aar' + ) + x86Implementation "org.mozilla:geckoview-nightly-x86:60.0a1" + + // ... +} +``` + +Now either build the `klarArmDebug` or `klarX86Debug` build variant from Android Studio (Running "Sync Project with Gradle files" once might be needed) or build and install from the command line: + +```bash +./gradlew installKlarArmDebug +./gradlew installKlarX86Debug +``` + +Finally, the default renderer might be set as Webview. You can check for the presence or absence of the Gecko logo at `focus:about` to verify. You may change rendering engine settings at `focus:test` and then press back for the app to restart. This should store your engine preference until you uninstall or clear data. + +You can also change the default engine at `focus-android/app/src/debug/java/org/mozilla/focus/web/Config.kt`. Setting `DEFAULT_NEW_RENDERER` to `true` will use GeckoView. diff --git a/mobile/android/focus-android/docs/Feature-&-Issue-workflow.md b/mobile/android/focus-android/docs/Feature-&-Issue-workflow.md new file mode 100644 index 0000000000..02c43f2ed7 --- /dev/null +++ b/mobile/android/focus-android/docs/Feature-&-Issue-workflow.md @@ -0,0 +1,54 @@ +# Feature & Issue workflow + +## High-Level Steps + +| Issue opened| ➡️ |Triaged| ➡️ |Ready in/for Backlog| ➡️ | Ready for UX | ➡️ | Ready for Eng| ➡️ |Eng done| ➡️ |Ready for review| ➡️ |Ready for QA| ➡️ |Done +|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------ + +## Details + +**Issue opened** +- Issue is created (by anybody) +- Ready for triage, where the issue gets either closed, commented on, left in triage for more info, assigned labels, and milestones + +**Triaged** +- Is triaged, which means issue has a P label, has a milestone assigned (can be backlog) +- Issue if possible can already get estimation, T-shirt size + +**Ready in/for Backlog** +- Issue is in Backlog milestone +- Issue includes enough information (follows template) that allows team members to estimate, understand the scope, user benefit, the what, and acceptance criteria +- Request for probes/KPIs reviewed and approved by product and data analyst +- Once estimated, it's ready to be moved into a sprint + +**Ready for UX** +- When UX picks up issue, assigns themselves to the issue +- UX to provide mocks, attach to Github issue +- Once UX is done, ready for eng, UX resource to unassign themselves, and use "ready for eng breakdown" label +- Can be skipped if no UX is needed + +**Ready for Eng** +- Eng should only pick up issues that are ready for Eng and assigned to a sprint/milestone (not Backlog) +- Copy/content strategist has provided strings + +**Eng done** +- Issue is eng done only if PR was submitted AND PR was reviewed +- PR is closed but not issue +- Issue will be assigned by eng to product manager (PM assigns it to other stakeholders where appropriate) +- Strings exported + +**Ready for Review** +- During sprints: Once PR is closed, the next day Nightly should have the changes and stakeholders can verify +- During sprint review: everyone can review, comment +- Once stakeholders(Marketing, PM) are happy with result, PM assigns "ready for QA" label + +**Ready for QA** +- QA to watch for issues with "ready for QA" label AND in current sprint or major milestone +- QA to assign the ticket they are currently working on to themselves +- Once QA verified and completed issue, un-assign themselves from issue +- Outline and record test steps in TestRail +- Identify new test(s) for automation and create github issue with "automation" label +- Closes issue + +**Done** +- Issue is closed, no assignees and assigned sprint or major milestone diff --git a/mobile/android/focus-android/docs/GeckoView-(In-Progress).md b/mobile/android/focus-android/docs/GeckoView-(In-Progress).md new file mode 100644 index 0000000000..1691579ab1 --- /dev/null +++ b/mobile/android/focus-android/docs/GeckoView-(In-Progress).md @@ -0,0 +1,6 @@ +# Running GeckoView in a debug environment + +Currently there are two ways to use GeckoView in development. + +1. You can navigate to `focus:test` and flip the toggle to switch to the new engine +2. In [`Config.kt`](https://github.com/mozilla-mobile/focus-android/blob/master/app/src/debug/java/org/mozilla/focus/web/Config.kt) you can set `DEFAULT_NEW_RENDERER` to `true`. diff --git a/mobile/android/focus-android/docs/Home.md b/mobile/android/focus-android/docs/Home.md new file mode 100644 index 0000000000..b5daab333f --- /dev/null +++ b/mobile/android/focus-android/docs/Home.md @@ -0,0 +1,40 @@ +# Firefox Focus for Android + +> Browse like no one’s watching. The new Firefox Focus automatically blocks a wide range of ads and online trackers — from the moment you launch it to the second you leave it. Easily erase your history, passwords and cookies, so you won’t get followed by things like unwanted ads. + +https://www.mozilla.org/firefox/focus/ + +*Don't see what you're looking for? Check out our shared docs: https://github.com/mozilla-mobile/firefox-android/tree/main/docs/shared.* + +## User support + +* Support articles: https://support.mozilla.org/en-US/products/focus-firefox/firefox-focus-android +* Support forum: https://support.mozilla.org/en-US/questions/focus-firefox?show=all + +## Download + +* Google Play - Firefox Focus: https://play.google.com/store/apps/details?id=org.mozilla.focus +* Google Play - Firefox Klar: https://play.google.com/store/apps/details?id=org.mozilla.klar +* APKs: https://github.com/mozilla-mobile/firefox-android/releases + +## Contribute + +* [Contributing (Writing code, translating the app, testing the app)](https://github.com/mozilla-mobile/firefox-android/blob/main/docs/shared/android/CONTRIBUTING.md) +* [List of issues](https://bugzilla.mozilla.org/describecomponents.cgi?product=Focus) +* [Mozilla Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/) + +## Communication +* Send an email to our [public Firefox Focus mailing list](https://mail.mozilla.org/listinfo/firefox-focus-public): firefox-focus-public@mozilla.com +* Send an email to our internal Firefox Focus mailing list (mozilla.com email required): firefox-focus@mozilla.com + +## Developer Documentation + +* [Architecture decisions](https://github.com/mozilla-mobile/firefox-android/blob/main/focus-android/docs/Architecture-Decisions.md) +* [Automation](https://github.com/mozilla-mobile/firefox-android/blob/main/docs/shared/android/automation.md) +* [Content blocking](https://github.com/mozilla-mobile/firefox-android/blob/main/focus-android/docs/Content-blocking.md) +* [Multisession architecture](https://github.com/mozilla-mobile/firefox-android/blob/main/focus-android/docs/Multisession-architecture.md) +* [Localization](https://github.com/mozilla-mobile/focus-android/wiki/Localization) +* [Telemetry](https://github.com/mozilla-mobile/firefox-android/blob/main/focus-android/docs/Telemetry.md) + +## Product Management +* [Release schedule 2018](https://wiki.mozilla.org/Mobile/Focus/Android/Train_Schedule) diff --git a/mobile/android/focus-android/docs/Homescreen-Tips.md b/mobile/android/focus-android/docs/Homescreen-Tips.md new file mode 100644 index 0000000000..ce4ebb4ec0 --- /dev/null +++ b/mobile/android/focus-android/docs/Homescreen-Tips.md @@ -0,0 +1,35 @@ +# Homescreen Tips + +## Homescreen Tips Explained 💡 + +Homescreen tips launched in Firefox Focus 7.0 in October 2018. They allow users to get more information about the product without having to research on their own or dig around in settings. Tips also enable users to _quickly_ access portions of the settings menu they may not know how to navigate otherwise. All tips have either a deep link to settings or a link to their corresponding SUMO article (if available). + +<img width="300" src="https://user-images.githubusercontent.com/4400286/43972144-4dbf9508-9c88-11e8-8660-3e16030cdc45.png"> + +## How Tips are Displayed +Currently, tips are shown based on the following constraints: +1. We will never show a user a tip from a feature they have _already engaged with_ +2. Tips are shown in a random order, and all have an equal chance of being shown +3. Only three tips are displayed per session before we revert back to showing "Browse. Erase. Repeat." +4. Tips are displayed on the home screen after launch, and are refreshed after each erase. + +Some notes to keep in mind: +* clicking a SUMO link does not count as engaging with the feature +* the "disable tips" tip is shown less frequently and is _guaranteed_ to show on the third launch of the app. +* a "session" is defined as every time the app is loaded into memory (i.e. has not been force quit). + +## List of Tips +Below is a list of tips available with their corresponding deep links: + +* "Open every link in Firefox Focus. Set Firefox Focus as default browser" -> Settings deep link +* "Turn off tips on the Firefox Focus start screen" -> Settings deep link +* "Turn off tips on the start screen" -> Settings deep link +* "Trusted site? Allowlist disables Tracking Protection for sites you know and trust." -> Settings deep link +* "Site behaving unexpectedly? Try turning off Tracking Protection" -> None +* "Get one-tap access to sites you use most. Menu > Add to Home screen" -> [SUMO article](https://support.mozilla.org/en-US/kb/add-web-page-shortcuts-your-home-screen) +* "Autocomplete URLs for sites you use most. Long-press any URL in the address bar" -> [SUMO article](https://support.mozilla.org/en-US/kb/autocomplete-settings-firefox-focus-address-bar) +* "Open a link in a new tab. Long-press any link on a page" -> [SUMO article](https://support.mozilla.org/en-US/kb/open-new-tab-firefox-focus-android) +* "Rather see the full desktop site? Menu > Request desktop site" -> [SUMO article](https://support.mozilla.org/en-US/kb/switch-desktop-view-firefox-focus-android) + +## The Future of Tips +If tips are received well, there are plans to update them in the future. A major change in consideration is hosting them server-side so they can be updated on the fly, rather than having to push an update to the app in order to change them. An android component for this feature is also being investigated. diff --git a/mobile/android/focus-android/docs/Multisession-architecture.md b/mobile/android/focus-android/docs/Multisession-architecture.md new file mode 100644 index 0000000000..465f4249ac --- /dev/null +++ b/mobile/android/focus-android/docs/Multisession-architecture.md @@ -0,0 +1,19 @@ +# Multisession Architecture + +To support multiple simultaneous browsing sessions the architecture of Focus has been refactored to strictly separate sessions (and associated data) from the UI that displays this data. + +Sessions are managed by a global `SessionManager` instance. UI components display data hold by `Session` objects. + +The new multisession architecture makes use of Android's [Architecture Components](https://developer.android.com/topic/libraries/architecture/index.html). The list of sessions in the `SessionManager` and the data inside a `Session` is returned as [LiveData](https://developer.android.com/topic/libraries/architecture/livedata.html) objects. UI components can observe those LiveData objects for changes and update the UI state if needed. + + + +* _BrowserFragment_ is the fragment that displays the browser chrome and web content. A _BrowserFragment_ is always connected to a _Session_ (1:1). + +* _IWebView_ is a generic interface for the view that renders web content. At runtime it's implemented by either [WebView](https://developer.android.com/reference/android/webkit/WebView.html) or GeckoView. State changes will be reported to a class that implements _IWebView.Callback_. + +* With the multisession architecture _IWebView.Callback_ is implemented by _SessionCallbackProxy_. The purpose of this class is to update the _Session_ object for this browsing session. Currently _SessionCallbackProxy_ will still delegate some callback methods to the BrowserFragment. This is expected to be removed in a follow-up refactoring eventually. + +* _BrowserFragment_ displays data hold by the _Session_ object. Data that changes periodically (e.g. URL, progress, ..) are represented as [LiveData](https://developer.android.com/topic/libraries/architecture/livedata.html) objects and can be observed so that the UI can be updated whenever the state changes. + +* Sessions are switched by displaying a new _BrowserFragment_ for a different _Session_ object. diff --git a/mobile/android/focus-android/docs/Recommended-pre-push-hook.md b/mobile/android/focus-android/docs/Recommended-pre-push-hook.md new file mode 100644 index 0000000000..a9d377205c --- /dev/null +++ b/mobile/android/focus-android/docs/Recommended-pre-push-hook.md @@ -0,0 +1,30 @@ +# Recommended pre-push hook + +If you want to reduce your PR turn-around time, I'd recommend adding a +pre-push hook: this script will stop a push if the unit tests or linters +fail, finding the failures before it hits TaskCluster (which takes +forever to dig through the logs): +```sh +#!/bin/sh + +./gradlew -q \ + checkstyle \ + ktlint \ + pmd \ + detektCheck \ + app:assembleFocusArmDebug + + +# Tasks omitted because they take a long time to run: +# - unit test on all variants +# - UI tests +# - lint (compiles all variants) +``` + +To use it: +1. Create a file with these ^ contents (exclude the "\`") at `<repo>/.git/hooks/pre-push` +1. Make it executable: `chmod 755 <repo>/.git/hooks/pre-push` + +And it will run before pushes. Notes: +- Run `git push ... --no-verify` to push without making the check +- It takes ~30 seconds to run. If you think this hook takes too long, you can remove the unit test line and it becomes almost instant. diff --git a/mobile/android/focus-android/docs/Release-Process.md b/mobile/android/focus-android/docs/Release-Process.md new file mode 100644 index 0000000000..e39789ba31 --- /dev/null +++ b/mobile/android/focus-android/docs/Release-Process.md @@ -0,0 +1,58 @@ +# Release Process + +## Creating a new Release Branch + +1. Create a branch name with the format `releases_v[version]` (for example: `releases_v87.0`). +2. Pin the Android Components version to the final release version with the format `[version].0.0`. + + For example: + + | FILE | CURRENT VERSION | RELEASE VERSION | + |--------------------|--------------------|-----------------| + | AndroidComponents | 73.0.20210223143117| 73.0.0 | + + For reference, the AC release checklist can be found at https://mozac.org/contributing/release-checklist. + Instead of updating the AC version manually, you could also wait for automation to update the [ac version](https://github.com/mozilla-mobile/focus-android/actions/workflows/update-ac.yml). + +3. In the GitHub UI, create a branch on the upstream with the format `releases_v[version]`. +4. Push the `releases_v[version]` branch as a PR with the commit message "Set version to [version]" and commit into the upstream `releases_v[version]` branch. +6. Create a new [milestone](https://github.com/mozilla-mobile/focus-android/milestones) for the next release and close the existing milestone. +7. Announce the new `releases_v[version]` branch on Slack in #releaseduty-mobile. + +Automation should take over after you land your PR into the upstream `releases_v[version]` branch. You can verify by clicking on the branch in the UI, and looking for the green/yellow dot that will list links to the running build tasks. + +## After the Beta release + + Update the `versionName` from `build.gradle(Module:focus-android.app)` to the next Focus beta release version.This should be the `[version + 1]`. + For example, if the release cut was for `90`, we're looking to prepare the latest nightly for `91`. + For example: + + | FILE | CURRENT VERSION | RELEASE VERSION | + |--------------------|--------------------|-----------------| + | build.gradle | 90 | 91 | + +## Renew telemetry + +After the Beta cut, another task is to renew/remove all soon to expire telemetry probes. What we're looking for is to create a list of telemetry that will expire in `[release_version add 2]`. See [Firefox Release Calendar](https://wiki.mozilla.org/Release_Management/Calendar) for the current Release version. There is a script that will help with finding these soon to expire telemetry. + +1. Use the helper `python3 tools/data_renewal_generate.py release_version+2` to detected and generate files that will help create the following files: + - `[release_version add 2]`_expiry_list.csv + - `[release_version add 2]`_renewal_request.txt +2. Upload the `[release_version add 2]`_expiry_list.csv to Google sheet in this [shared Google Drive](https://drive.google.com/drive/folders/1_ertMvn59eE9JmN721RqOjW6nNtxq9oS?usp=sharing) and contact product to review. For each telemetry listed answer decide for: + - Renew the metric (for how long? Add 12 versions?) + - Choose not to renew (but not delete) + - Choose to remove the metric + - Renew the metric and set to never expire (this should only be for business critical metrics) +3. Note that `metrics.yaml` is also modified. Once the review is over, continue to modify `metrics.yaml` to match the decision made in the Google sheet. Make sure to add the PR link and if the telemetry never expires, add the email of the owner as contact. +4. Create a PR for review. Modify `[release_version add 2]`_renewal_request.txt and paste it to the PR for data review. Make sure to add the answers for: + - When will this collection now expire? + - Why was the initial period of collection insufficient? + + +## Ask for Help + +If you run into any problems, please ask any questions on Slack in #releaseduty-mobile. + +References: +- https://github.com/mozilla-mobile/focus-android/tree/releases_v96.0 +- https://github.com/mozilla-mobile/focus-android/pull/6012 diff --git a/mobile/android/focus-android/docs/Release-tracks.md b/mobile/android/focus-android/docs/Release-tracks.md new file mode 100644 index 0000000000..ad0c147901 --- /dev/null +++ b/mobile/android/focus-android/docs/Release-tracks.md @@ -0,0 +1,53 @@ +# Release tracks + + + +We have three public release tracks for Focus/Klar: + +* **Production**: That's the version everyone sees on Google Play. We rollout releases gradually in stages. + +* **Beta**: This track is for Beta builds. We use this track to test builds with a wider audience and to uncover problems/crashes that may prevent us from releasing the build. We may push multiple releases to this track until we feel positive about publishing to the *production* track. + +* **Nightly**: We publish automated builds on this track every 24 hours. Those builds reflect the latest state of the repository (master). This track allows users to experience the newest Firefox Focus/Klar innovations in an unstable environment and provide feedback on features and performance to help determine what makes the final release. Nightly builds can be unstable and contain incomplete or defect features. + + +## Nightly + + + +* Updates every 24 hours. +* Latest development state +* Can be unstable and contain incomplete or defect features + +### How to sign up for Nightly + +To get Focus Nightly on your device, follow these steps: + +1) Visit https://groups.google.com/g/firefox-focus-pre-release and join the Google Group +2) After you have joined the group opt-in to receive Nightly builds, again with the same Google account: https://play.google.com/apps/testing/org.mozilla.focus.nightly +3) Download Firefox Focus (Nightly) from Google Play: https://play.google.com/store/apps/details?id=org.mozilla.focus.nightly + +Make sure you use the same Google Account for both steps. + + +## Beta + + + +* Updates to the Beta track will typically be 1-2 times weekly. + +### How to sign up for beta +To get Focus Beta on your device, follow these steps: +1) Visit https://groups.google.com/g/firefox-focus-pre-release and join the Google Group +2) After you have joined the group opt-in to receive Beta builds, again with the same Google account: https://play.google.com/apps/testing/org.mozilla.focus.beta +3) Download Firefox Focus (Beta) from Google Play: https://play.google.com/store/apps/details?id=org.mozilla.focus.beta + +Make sure you use the same Google Account for both steps. + + +### Manual downloads + +The latest Beta/Release builds can be downloaded manually from: +https://github.com/mozilla-mobile/focus-android/releases + +Note that manually downloaded builds **do not update automatically**. diff --git a/mobile/android/focus-android/docs/Removing-strings.md b/mobile/android/focus-android/docs/Removing-strings.md new file mode 100644 index 0000000000..02a3cd86c5 --- /dev/null +++ b/mobile/android/focus-android/docs/Removing-strings.md @@ -0,0 +1,29 @@ +# Removing strings + +## Marking an unused string to be removed + +Removing strings manually could cause crashes in **Beta** and **Release** versions 💥 , as the removed strings could be uplifted to release branches by mistake, where strings are still needed. For this reason, we need a special process to remove them. + +Any landed string that is not removed while in development in Nightly will persist through 3 Firefox releases (Nightly, Beta, Release) before we can remove them from our code base. For example, +if you want to remove a string that has already shipped prior to **Firefox Nightly 93**, the same string will still be in-use in **Firefox Beta 92** and **Firefox Release 91**. This means the string will be marked as unused and removed in 93 while still riding the train, and it can be removed safely when **Firefox Release 93** no longer ships, for instance, **Firefox Release 94** and beyond. + +To keep us safe when you want to remove strings from nightly: + +1. Add these attribute to the target strings `moz:removedIn="<<ACTUAL_NIGHTLY_VERSION>>"` and `tools:ignore="UnusedResources"`. + +```xml + <string name="onboarding_close" moz:removedIn="93" tools:ignore="UnusedResources">Close</string> +``` +Example commit https://github.com/mozilla-mobile/focus-android/pull/6291/files + +## When to remove an unused string and how + +Strings that have been tagged with `moz:removedIn` attributes are safe to be removed after the marked version is no longer shipping and no longer in-use or needed. + +Consult the [Firefox release calendar](https://wiki.mozilla.org/Release_Management/Calendar). Let's say the Beta cut just happened and we are at Firefox Nightly 109, Firefox Beta 108 and Firefox Release 107. Everything marked with `moz:removedIn` <= 106 can now be removed. + +You only need to remove the en-US strings within [values/strings.xml](https://github.com/mozilla-mobile/focus-android/blob/main/app/src/main/res/values/strings.xml), and this change will propagate to the other locales. + +## Future + +It would be nice to add some automatization to delete the strings that have the `moz:removedIn` attributes where a full cycle has happen (3 releases versions from the actual release version). diff --git a/mobile/android/focus-android/docs/Sprint-Process.md b/mobile/android/focus-android/docs/Sprint-Process.md new file mode 100644 index 0000000000..7f51230fe4 --- /dev/null +++ b/mobile/android/focus-android/docs/Sprint-Process.md @@ -0,0 +1,47 @@ +# Sprint Process + +Focus follows a 2-week sprint cycle with 6-week milestone releases. Dot-releases with bugfixes are released every two weeks. Our upcoming train schedule is [here](https://wiki.mozilla.org/Mobile/Focus/Android/Train_Schedule). + +## Issue naming and labels + +### Labels +Priority labels are based on the [Bugzilla triage process][triage priority] and set during triage to determine when they'll be worked on: +* `P1`: Issues for the current 2-week sprint. + * [Open engineering issues](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3AP1%20NOT%20%5Bux%5D%20in%3Atitle%20) + * [Open UX issues](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3AP1%20ux%20in%3Atitle%20) +* `P2`: Issues for the 6-week milestone release. + * [Open engineering issues](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3AP2%20NOT%20%5Bux%5D%20in%3Atitle%20) + * [Open UX issues](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3AP2%20ux%20in%3Atitle%20) +* `P3`: Backlog +* `P5`: Will not fix but will accept a patch + +Other labels: +* `addressed`: Label for excluding items from triage. Should be used for [meta] items. + +### Issue Prefixes +* `[meta]`: larger issues that need to be broken down, into a `[breakdown]` issue and issues for its smaller parts. This should include a checklist of all the issues (including the `[breakdown]` issue). + + These do NOT get a P* label, but should be in a milestone. +* `[breakdown]`: issue to track the work of breaking down a larger bug + +## Triage - Weekly +- ([Link](https://github.com/mozilla-mobile/focus-android/issues?q=is%3Aissue+is%3Aopen+-label%3AP1+-label%3AP2+-label%3AP3+-label%3AP4+-label%3AP5+-label%3Aaddressed+-label%3Ablocked+sort%3Aupdated-desc+no%3Amilestone)) Assign priority labels (P1, P2, ...) to bugs without priority labels or the `addressed` label +- ([Link](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3AP1+no%3Amilestone+)) Ensure all P1 labels are assigned a milestone +- ([Link](https://github.com/mozilla-mobile/focus-android/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3AP2+no%3Amilestone+)) Ensure all P2 labels are assigned a milestone + +## Weekly Bug management (alternating) + +### Sprint planning +Deciding what goes into a sprint, promote P2 issues to P1. After every release, this meeting is for deciding what goes into the upcoming milestone - add issues to the milestone and set them as P2. + +### Backlog grooming +Handling Triage overflow, adding to the contributor bug lists, looking at milestone lists. + +[triage priority]: https://wiki.mozilla.org/Bugmasters/Process/Triage#Weekly_or_More_Frequently_.28depending_on_the_component.29 +## Monthly Roadmap Planning +Plan and prioritize features for upcoming milestones - an update will be emailed out. + +Plan and facilitate workweeks +release schedule +trying to work with things without proper training +too much observing diff --git a/mobile/android/focus-android/docs/Telemetry.md b/mobile/android/focus-android/docs/Telemetry.md new file mode 100644 index 0000000000..a6d202a93e --- /dev/null +++ b/mobile/android/focus-android/docs/Telemetry.md @@ -0,0 +1,345 @@ +# Telemetry +For clients that have "send anonymous usage data" enabled Focus sends a "core" ping and an "event" ping to Mozilla's telemetry service. Sending telemetry can be disabled in the app's settings. Builds of "Focus for Android" have telemetry enabled by default ("opt-out") while builds of "Klar for Android" have telemetry disabled by default. + +## Core ping + +Focus for Android creates and tries to send a "core" ping whenever the app goes to the background. This core ping uses the same format as Firefox for Android and is [documented on firefox-source-docs.mozilla.org](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/core-ping.html). + +## Event ping + +In addition to the core ping an event ping for UI telemetry is generated and sent as soon as the app is sent to the background. + +### Settings + +As part of the event ping the most recent state of the user's setting is sent (default values in **bold**): + +| Setting | Key | Value(*) +|--------------------------|---------------------------------|---------------------- +| Default search engine | pref_search_engine | **bundled engine name**/`custom`(**) +| Block ad trackers | pref_privacy_block_ads | **true**/false +| Block analytics trackers | pref_privacy_block_analytics | **true**/false +| Block social trackers | pref_privacy_block_social | **true**/false +| Block content trackers | pref_privacy_block_other | true/**false** +| Block web fonts | pref_performance_block_webfonts | true/**false** +| Block images | pref_performance_block_images | true/**false** +| Locale override | pref_locale | **empty string**/`locale-code`(***) +| Default browser | pref_default_browser | true/**false** +| Stealth mode | pref_secure | **true**/false +| Autocomplete (Default) | pref_autocomplete_preinstalled | true/false +| Autocomplete (Custom) | pref_autocomplete_custom | true/false +| Show tips on Firefox Focus home screen | pref_key_tips | **true**/false + +(*) All values are sent as a `String`. + +(**) If the default is one of the bundled engines, the name of the engine. Otherwise, just `custom`. + +(***) An **empty string** value indicates "System Default" locale is selected. + +### Events + +The event ping contains a list of events ([see event format on readthedocs.io](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/events.html)) for the following actions: + +#### Sessions + +| Event | category | method | object | value | +|------------------------------------------|----------|------------|--------|--------| +| Start session (App is in the foreground) | action | foreground | app | | +| Stop session (App is in the background) | action | background | app | | + +#### Browsing + +| Event | category | method | object | value | extras. | +|----------------------------------------|----------|-----------------------|------------|--------|------------| +| URL entered | action | type_url | search_bar | | | +| Search entered | action | type_query | search_bar | | | +| Search hint clicked ("Search for ..") | action | select_query | search_bar | | | +| Link from third-party app | action | intent_url | app | | | +| Text selection action from app | action | text_selection_intent | app | | | +| Long press on image / link | action | long_press | browser | | | +| "Pull to refresh" | action | swipe | browser | reload | | +| Autofill popup is shown | action | show | autofill | | | +| Autofill performed | action | click | autofill | | | +| Enable Search Suggestion from prompt | action | click | search_suggestion_prompt | true/false | | + +#### Custom Tabs + +| Event | category | method | object | value | extra | +|---------------------------|----------|-------------------|----------------------|------------|------------| +| Custom tab opened | action | intent_custom_tab | app | | `options`* | +| "Close" button clicked | action | click | custom_tab_close_bu | | `tabs`* | +| "Action" button clicked | action | click | custom_tab_action_bu | | | +| Custom menu item selected | action | open | menu | custom_tab | | + +(*) `options` is a JSON map of enabled custom tab options as requested by the third-party app, e.g. [hasToolbarColor, hasCloseButton] + +(*) `tabs` is a JSON map containing the position of the currently selected tab and the total number of open tabs: + +```JavaScript +{ + "selected": "2" // Currently selected tab (Zero-based numbering; -1 if no tab is selected) + "total": "5" // Total number of open tabs +} +``` + +#### Context Menu + +| Event | category | method | object | value | +|----------------------------------------|----------|--------|---------------------|------------| +| Link context menu dismissed | action | cancel | browser_contextmenu | link | +| Share Link menu item selected | action | share | browser_contextmenu | link | +| Copy link menu item selected | action | copy | browser_contextmenu | link | +| Image context menu dismissed | action | cancel | browser_contextmenu | image | +| Share Image menu item selected | action | share | browser_contextmenu | image | +| Copy Image menu item selected | action | copy | browser_contextmenu | image | +| Save Image menu item selected | action | save | browser_contextmenu | image | +| Image with Link context menu dismissed | action | cancel | browser_contextmenu | image+link | +| Open link in new tab | action | open | browser_contextmenu | tab | +| Open new tab in Focus from customtab context menu | action | open | browser_contextmenu | full_browser | + + + +#### Erasing session + +| Event | category | method | object | value | extras | +|----------------------------------------|----------|-------------|---------------------|------------|---------| +| Floating action button clicked | action | click | erase_button | | `tabs`* | +| Back button clicked: Home screen | action | click | back_button | erase_home | `tabs`* | +| Back button clicked: Previous app | action | click | back_button | erase_app | `tabs`* | +| Notification clicked | action | click | notification | erase | `tabs`* | +| Notification "Erase and Open" click | action | click | notification_action | erase_open | `tabs`* | +| Home screen shortcut clicked | action | click | shortcut | erase | `tabs`* | +| Erase history in tabs tray | action | click | tabs_tray | erase | `tabs`* | +| App removed by user from "recent apps" | action | click | recent_apps | erase | `tabs`* | + +(*) `tabs` is a JSON map containing the position of the currently selected tab and the total number of open tabs: + +```JavaScript +{ + "selected": "2" // Currently selected tab (Zero-based numbering; -1 if no tab is selected) + "total": "5" // Total number of open tabs +} +``` + +#### Menu + +| Event | category | method | object | value | extras | +|---------------------------------------------|----------|----------|-----------------|--------------|----------| +| Enable/Disable content blocking for session | action | click | blocking_switch | true/false | | +| Share URL with third-party app | action | share | menu | | | +| Open default app for URL | action | open | menu | default | | +| Open Firefox | action | open | menu | firefox | | +| Open with ... (Show app chooser) | action | open | menu | selection | | +| Open full browser from custom tab | action | click | menu | full_browser | | +| "Download Firefox" clicked (in app chooser) | action | install | app | firefox | | +| "What's new" | action | click | menu | whats_new | `state`* | +| Reload current page | action | click | menu | reload | | +| Report site issue | action | click | menu | report_issue | | +| Find in Page | action | click | menu | find_in_page | | +| Request desktop site | action | click | desktop_request_check| true/false | | + +(*) `state` is a JSON map containing information about whether the menu item was highlighted: + +```JavaScript +{ + "highlighted": "true" // Whether the menu item was highlighted or not +} +``` + +#### Notification + +| Event | category | method | object | value | extras | +|---------------------------------------|----------|----------|---------------------|------------|---------| +| Notification clicked (Erase) | action | click | notification | erase | | +| Action "Open" clicked | action | click | notification_action | open | | +| Action "Erase and Open" clicked | action | click | notification_action | erase_open | `tabs`* | + +(*) `tabs` is a JSON map containing the position of the currently selected tab and the total number of open tabs: + +```JavaScript +{ + "selected": "2" // Currently selected tab (Zero-based numbering; -1 if no tab is selected) + "total": "5" // Total number of open tabs +} +``` +#### Page Load Time Histogram + + +| Event | category | method | object | extras| +|-----------------|----------|--------------|---------| ----- | +| Histogram for Page Load Times for Foreground Session | histogram | foreground | browser | histogram*| + +(*) There are 200 extras attached to this event, each a bucket of 100 ms each, with the key as the minimum value in the bucket and the value as the corresponding number of events in the bucket. Anything over 20,000 is put in the last bucket. For example: +``` +{”0":"2"} +{“100”:"3"} +... +{"19900", "4"} +``` + +#### URI Count + +| Event | category | method | object | extra | +|---------------------------------------------|----------|----------|---------------------|-----------------| +| The count of the total non-unique http(s) URIs visited in a subsession, including page reloads, after the session has been restored. This does not include background page requests and URIs from embedded pages or private browsing | action | open | browser | `{"total_uri_count": num }` | +| The count of the unique domains visited in a subsession, after the session has been restored. Subdomains under eTLD are aggregated after the first level (i.e. test.example.com and other.example.com are only counted once). This does not include background page requests and domains from embedded pages or private browsing. | action | open | browser | `{"unique_domains_count": num }` | + + +#### Downloads + +| Event | category | method | object | value | +|---------------------------------------------|----------|----------|---------------------|-----------------| +| Clicked "Download" | action | click | download_dialog | download | +| Clicked "Cancel" | action | click | download_dialog | cancel | + +#### Open Focus From Icon + +| Event | category | method | object | value | +|---------------------------------------------|----------|----------|---------------------|-----------------| +| Launched Focus From Icon | action | click | app_icon | open | +| Resume Focus From Icon | action | click | app_icon | resume | + +#### Add to Home screen + +| Event | category | method | object | value | +|-----------------------------------------|----------|----------|--------------------------|-------------------| +| Clicked "Add to Home screen" in dialog | action | click | add_to_homescreen_dialog | add_to_homescreen | +| Clicked "Cancel" in dialog | action | click | add_to_homescreen_dialog | cancel | +| Open Focus from Home screen shortcut | action | click | homescreen_shortcut | open | + +#### Share to Focus Event + +| Event | category | method | object | value | +|---------------------------------------------|----------|----------|---------------------|-----------------| +| Shared URL to Focus | action | share_intent | app | url | +| Shared Search Terms to Focus | action | share_intent | app | search | + +#### Settings + +| Event | category | method | object | value | extras | +|--------------------------------|----------|----------|-------------------------|-------|-------------------------- +| Setting changed | action | change | setting | <key> | `{ "to": <value> }` +| Autocomplete domain added | action | save | autocomplete_domain | | `{ "source": <value> }` +| Autocomplete domain removed | action | remove | autocomplete_domain | | `{ "total": 5 }` +| Autocomplete domain reordered | action | reorder | autocomplete_domain | | `options*` +| Open Exceptions Setting | action | open | allowlist | | +| Remove Exceptions Domains | action | remove | allowlist | |`{ "total": 5 }` +| Remove All Exceptions Domains | action | remove_all |allowlist | | +| Default search engine clicked | action | open | search_engine_setting | | +| Change default search engine | action | save | search_engine_setting | | `{"source": src* }` +| Select "Remove" engines screen | action | remove | search_engine_setting | | +| Delete search engines | remove | remove | remove_search_engines | |`{"selected": num* }` +| Restore bundled engines | action | restore | search_engine_setting | | +| Select "Add another engine" | action | show | custom_search_engine | | +| Save custom search engine | action | save | custom_search_engine | | `{"success": bool* }` +| Click "Add search engine" ℹ️ | action | click | search_engine_learn_more| | +| Open with default browser prompt| action | show | make_default_browser_open_with| | +| Settings default browser prompt| action | show | make_default_browser_settings| | + + +(*) `options` is a JSON map containing: + +```JavaScript +{ + "from": 5 + "to": 2 +} +``` + +(*) `src` can be either `bundled` or `custom` + +(*) `num` number of engines selected for deletion + +(*) `bool` true if successfully saved, false if validation error + +#### Firstrun + +| Event | category | method | object | value | +|---------------------------------------------|----------|----------|---------------------|--------------| +| Showing a first run page | action | show | firstrun | `page`* | +| Skip button pressed | action | click | firstrun | skip | +| Finish button pressed | action | click | firstrun | finish | + +(*) Page numbers start at 0. Initially when the firstrun tour is shown an event for the first page (0) is fired. + +#### Multitasking / Tabs + +| Event | category | method | object | value | extras | +|--------------------------------------------|----------|----------|---------------------|-------|---------| +| Context menu: Open link in new tab | action | open | browser_contextmenu | tab | `tabs`* | +| Open the tabs tray | action | show | tabs_tray | | | +| Close the tabs tray (back / click outside) | action | hide | tabs_tray | | | +| Switch to tab in tabs tray | action | click | tabs_tray | tab | `tabs`* | +| Erase history in tabs tray | action | click | tabs_tray | erase | `tabs`* | + +(*) `tabs` is a JSON map containing the position of the currently selected tab and the total number of open tabs: + +```JavaScript +{ + "selected": "2" // Currently selected tab (Zero-based numbering; -1 if no tab is selected) + "total": "5" // Total number of open tabs +} +``` + +#### Homescreen Tips + +| Event | category | method | object | value | +|------------------------------------------|----------|------------|--------|--------| +| Open in new tab tip displayed | action | show | tip | open_in_new_tab_tip | | +| Add to homescreen tip displayed | action | show | tip | add_to_homescreen_tip | | +| Disable tracking protection tip displayed | action | show | tip | disable_tracking_protection_tip | | +| Disable tips on home screen tip displayed | action | show | tip | disable_tips_tip | | +| Set default browser tip displayed | action | show | tip | default_browser_tip | | +| Autocomplete URL tip displayed | action | show | tip | add_autocomplete_url_tip | | +| Open in new tab tip tapped | action | click | tip | open_in_new_tab_tip | | +| Add to homescreen tip tapped | action | click | tip | add_to_homescreen_tip | | +| Disable tips tip tapped | action | click | tip | disable_tips_tip | | | +| Set default browser tip tapped | action | click | tip | default_browser_tip | | +| Autocomplete URL tip tapped | action | click | tip | add_autocomplete_url_tip | | +| Homescreen tips enabled/disabled | action | click | tip | add_to_homescreen_tip | | +| Survey tip displayed | action | show | tip | survey_tip | | +| Survey tip tapped | action | click | tip | survey_tip | | +| Survey (es) tip displayed | action | show | tip | survey_tip_es | | +| Survey (es) tip tapped | action | click | tip | survey_tip_es | | +| Survey (fr) tip displayed | action | show | tip | survey_tip_fr | | +| Survey (fr) tip tapped | action | click | tip | survey_tip_fr | | + +#### SSL Errors + +| Event | category | method | object | extras | +|--------------------------------------------|----------|----------|---------|---------| +| SSL Error From Page | error | page | browser |`error`*| +| SSL Error From Resource | error | resource | browser |`error`* | + +(*)`error` is a JSON map containing the primary SSL Error + +```JavaScript +{ + "error_code": "SSL_DATE_INVALID" // Primary SSL Error +} +``` + +| Possible Error Codes | +|----------------------| +| SSL_DATE_INVALID | +| SSL_EXPIRED | +|SSL_IDMISMATCH | +|SSL_NOTYETVALID | +|SSL_UNTRUSTED | +|SSL_INVALID | +|Undefined SSL Error | + +### Limits + +* An event ping will contain up to but no more than 500 events +* No more than 40 pings per type (core/event) are stored on disk for upload at a later time +* No more than 100 pings are sent per day + +## Implementation notes + +* Event pings are generated (and stored on disk) whenever the onStop() callback of the main activity is triggered. This happens whenever the main screen of the app is no longer visible (The app is in the background or another screen is displayed on top of the app). + +* Whenever we are storing pings we are also scheduling an upload. We are using Android’s JobScheduler API for that. This allows the system to run the background task whenever it is convenient and certain criterias are met. The only criteria we are specifying is that we require an active network connection. In most cases this job is executed immediately after the app is in the background. + +* Whenever an upload fails we are scheduling a retry. The first retry will happen after 30 seconds (or later if there’s no active network connection at this time). For further retries a exponential backoff policy is used: [30 seconds] * 2 ^ (num_failures - 1) + +* An earlier retry of the upload can happen whenever the app is coming to the foreground and sent to the background again (the previous scheduled job is reset and we are starting all over again). diff --git a/mobile/android/focus-android/docs/UI-Test.md b/mobile/android/focus-android/docs/UI-Test.md new file mode 100644 index 0000000000..a6b126297b --- /dev/null +++ b/mobile/android/focus-android/docs/UI-Test.md @@ -0,0 +1,54 @@ +# UI Test + +To run automated tests on firefox focus, you will need a few things to get started. +1. Working installation of git and a local clone of the this Focus repository. You can do: + + git clone https://github.com/mozilla-mobile/focus-android.git + +2. For this walkthrough, you will also need Android Studio, which can be found at [android website](https://developer.android.com/studio) . Follow the instructions. For the most part, Android Studio is going to set itself up with the repository all you have to do is open the directory. You may also need to install JDK as well. + +3. From there you will need either an android device (Preferably Nexus 4 or more recent) or setup emulators in Android Studio +To create an emulator: + 1. Open **Android Studio**, and navigate to the **Tools** drop down menu + 2. Navigate to **Android** pop out selection on the list and select "**AVD Manager**". A window will open from which you can manage your virtual devices + 3. select "**Create Virtual Device**" + 4. Select the phone tab for devices and select either the nexus devices or Google Pixel (preferably) + 5. you will need to select images with API level 22 ~ 26, download and install them by selecting download next to their names. + 6. Select the images to be installed, and click next. + 7. You can set the name of the device and the configuration, but is recommended to set following configurations changed: + * **Camera: None (For both Front and Back)** + * **Graphics: Automatic** + * **Multi-Core CPU: Unchecked** + + 8. **Device Testing**: If you wish to run tests on device, you need to enable Developer Options in the Settings menu (this is usually achieved by tapping **About Phone** -> **Build Number** 7 times), and enable '**USB Debugging**.' Then connect the phone to PC via USB cable. + + Our CI currently uses the following HW/SW configuration: + + _Firefox Focus_ + * Nexus 6, API 23 simulator + * Pixel 2, API 26 simulator + + _Firefox Klar_ + * Nexus 9, virtual, API Level 26 + +4. You are now ready to run the automated tests! + 1. Click the bottom left corner of Android Studio, and enable "**Build Variants**." The Build Variants value should be set to one of the following: + + on device: '**focusArmDebug**' or '**klarArmDebug**' + + simulator: '**focusX86Debug**' or '**klarX86Debug**' + + 2. Click on the drop down on the top left of Android Studio below the path view, and select '**Tests**' + 3. navigate the sub-directories to the folder locations of the tests to be run. For example, UI Tests are located in **focus-android/app/src/androidTest/java/org.mozilla.focus.activity** + 4. To run all tests in folder right click on the folder and select run. For individual tests you will do the same but right click on the specific test. + 5. select your simulator or device (connected via ADB). + +From there Android Studio will compile and install Focus on the simulator or device and start running the tests. + +If you just need to run Focus, select '**app**' from the **Configurations** pull-down menu and press Run button. + +You may need to follow along the first time to grant the app permissions to access the different parts of android such as storage access or the tests will fail, or manually turn on Storage permissions for Focus. + +It should be noted that not all tests will pass in simulator environment. Those tests are designed to run on our CI setup, and while most will not have issues, **BadURLTest** will fail on vanilla simulator environment where no Google Play store is installed, because it'll try to decode market:// url. Also, **AdBlockingTest** is suppressed, because while it passed locally, it was failing on our CI due to the network setup. I have left the test script intact for future references. + +(Drafted by [1Jamie](https://github.com/1jamie)) diff --git a/mobile/android/focus-android/docs/index.rst b/mobile/android/focus-android/docs/index.rst new file mode 100644 index 0000000000..2243097b6a --- /dev/null +++ b/mobile/android/focus-android/docs/index.rst @@ -0,0 +1,28 @@ +================================= +Focus for Android +================================= + +Specific documentation on a few topics is available at: + +.. toctree:: + :maxdepth: 1 + + Adjust Usage <Adjust-Usage.md> + Architecture Decisions <Architecture-Decisions.md> + Battery Debugging <Battery-Debugging.md> + Content blocking <Content-blocking.md> + Crash Reporting with Sentry <Crash-Reporting-with-Sentry.md> + Development Custom GeckoView <Development-Custom-GeckoView.md> + Feature & Issue workflow <Feature-&-Issue-workflow.md> + Running GeckoView in a debug environment <GeckoView-(In-Progress).md> + Home <Home.md> + Homescreen Tips <Homescreen-Tips.md> + Multisession architecture <Multisession-architecture.md> + Recommended pre-push hook <Recommended-pre-push-hook.md> + Release Process <Release-Process.md> + Release tracks <Release-tracks.md> + Removing strings<Removing-strings.md> + Sprint Process <Sprint-Process.md> + Telemetry <Telemetry.md> + UI Test <UI-Test.md> + l10n Screenshot Generation <l10n-Screenshot-Generation.md> diff --git a/mobile/android/focus-android/docs/l10n-Screenshot-Generation.md b/mobile/android/focus-android/docs/l10n-Screenshot-Generation.md new file mode 100644 index 0000000000..432358dd59 --- /dev/null +++ b/mobile/android/focus-android/docs/l10n-Screenshot-Generation.md @@ -0,0 +1,29 @@ +# l10n Screenshot Generation + +This wiki describes the steps to generate l10n screenshots on a device. + +## Prerequisites +You need to install [Fastlane Screengrab](https://docs.fastlane.tools/getting-started/android/screenshots/) in order to run screenshots test. Make sure all dependencies are installed as well. Gradle dependencies and screengrab config file is in the root folder of the repo. + +Open a console, go to Focus folder, and type ```export LC_ALL="en_US.UTF-8"``` . Fastlane needs this value to be set, otherwise the test will crash on some locales. + +Enable [this](https://github.com/mozilla-mobile/focus-android/blob/master/app/src/androidTest/java/org/mozilla/focus/screenshots/ScreenshotTest.java#L71) line and disable the line below. Make sure the import is properly added. ```HostScreencapScreenshotStrategy``` was developed to generate screenshots on Taskcluster, but it is currently not being used. In order to take device screenshots, you need ```UiAutomatorScreenshotStrategy```. + +## Build +Build test runner and apk by executing ```./gradlew assembleFocusArmDebug assembleFocusArmDebugAndroidTest``` + +## Configure Screengrabfile +```Screengrabfile``` contains the configuration for Screengrab execution, including locale list. The locale list can be found in: https://pontoon.mozilla.org/projects/focus-for-android/. Make sure the locales array is up to date. + +Note, that, ``` clear_previous_screenshots``` is set to true initially, but if you ran the tests and a handful of locales have failed, instead of rerunning the whole test, you should update locales array to only contain failed locales, and set this value to false. This way you won't lose successful locale screenshots. + +## Execute Test +Make sure the device is connected via USB, and run ```bundle exec fastlane screengrab run``` command. Currently, this will take about 3 hours to run through all locales. + +## Compress outputs +The screenshots are located in ```fastlane/metadata/android``` folder. Go to this folder, and resize the image to reduce the file size by running ```find . -name "*.png" | xargs mogrify -resize 50%``` + +On OS X, you can compress the file size further by using [ImageOptim](https://imageoptim.com/mac). ImageOptim has a bug where too many files are loaded, it slows down and eventually cannot process images. To circumvent this issue, ```find . -type f -iname \*png -print0 | xargs -0 -t -n 100 /Applications/ImageOptim.app/Contents/MacOS/ImageOptim``` command can be run on ```android``` folder, where it'll open only 100 images, close, and reopen with next 100 images. + +Rename ```screenshots.html``` to ```index.html``` +Upload the screenshots to https://github.com/npark-mozilla/npark-mozilla.github.io repo, if needed for general viewing. Or you can choose to upload to Google drive and share link. |