diff options
Diffstat (limited to 'mobile/android/android-components/docs/contribute')
11 files changed, 791 insertions, 0 deletions
diff --git a/mobile/android/android-components/docs/contribute/app_services_upgrade.md b/mobile/android/android-components/docs/contribute/app_services_upgrade.md new file mode 100644 index 0000000000..b79b71ae70 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/app_services_upgrade.md @@ -0,0 +1,36 @@ +--- +layout: page +title: Application Services version upgrades +permalink: /contributing/app-services-upgrades +--- + +Upgrading to a newer [version of App Services][as_version] upgrades the dependencies for multiple components, so it can be prudent to run a quick smoke test before it lands to ensure new functionality works and there are no regressions in the existing components. + +## Automated PR testing + +A quick first step is to run the automated smoke test on the Android Components PR by commenting: + +``` +bors try +``` + +This will run the `test-ui-browser` task, which will execute on-device tests of a sample browser built against the A-S version. If it's totally broken, that should blow up. + +## Fenix + +The next step is to build a [local version of Android Components][local_build] against [Fenix][fenix] and test the following items: + - Check general storage components works by adding/editing/removing: bookmarks, history + - Sign-in/Sign-up for Sync + - QR code pairing for Sync + - Changing settings in “Choose what to sync” updates on desktop and vice versa + - Rename device and verify on desktop + - Sync Now button works + - Browse passwords, auto-fill, save new passwords, update them + - Receiving tab works + - This needs push messaging keys, but we can test this without keys by using the “Sync Now” button + - Sending tabs + - Synced Tabs shows results from another device and vice versa + +[as_version]: https://github.com/mozilla-mobile/android-components/blob/main/buildSrc/src/main/java/Dependencies.kt#L33 +[local_build]: /contributing/testing-components-inside-app +[fenix]: https://github.com/mozilla-mobile/fenix diff --git a/mobile/android/android-components/docs/contribute/architecture.md b/mobile/android/android-components/docs/contribute/architecture.md new file mode 100644 index 0000000000..4b6ed25037 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/architecture.md @@ -0,0 +1,66 @@ +--- +layout: page +title: Architecture and Overview +permalink: /contributing/architecture +--- + +# Architecture and Overview + +Our main design goal is to provide independently reusable Android components. We strive to keep dependencies between components as minimal as possible. However, standalone components aren't always feasible, which is why we have grouped components based on their interactions and dependencies. + +On the lowest level, we provide standalone UI components (e.g. autocomplete, progressbar, colors) as well as independent service and support libraries (e.g. Telemetry, Kotlin extensions and Utilities). + +The second level consist of so called `Concept` modules. These are abstractions that describe contracts for component implementations such as `Engine` or `Session Storage`, which may in turn have multiple implementations. The purpose of these concepts is to allow for customization and pluggability. Therefore, where available, components should always depend on concept modules (not their implementations) to support bringing in alternative implementations. + +On top of `Concept` modules we provide `Browser` components. These components provide browser-specific functionality by implementing concepts and using lower level components. + +On the highest level, we provide `Feature` components which provide use case implementations (e.g search, load URL). Features can connect multiple `Browser` components with concepts, and will therefore depend on other components. + +The following diagram does not contain all available components. See [Components]({{ site.baseurl }}/components/) for a complete and up-to-date list. + +``` + ┌─────────────────────┬───────────────────────────────────────────────────────────────────────┐ + │ │ ┌───────────────────────────────────────────────────────────────────┐ │ + │ │ │ Feature │ │ + │ Features combine │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ browser components │ │ Session │ Toolbar │ Search │ │ │ + │ with concepts │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ └───────────────────────────────────────────────────────────────────┘ │ + ├─────────────────────┼───────────────────────────────────────────────────────────────────────┤ + │ │ ┌───────────────────────────────────────────────────────────────────┐ │ + │ │ │ Browser │ │ + │ Browser components │ │ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ │ │ + │ may implement │ │ Engine-Gecko Search Toolbar Errorpages │ │ + │ concepts and use │ │ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ │ │ + │ lower level │ │ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ┐ │ │ + │ components │ │ Engine-System Session Domains Menu │ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘ │ │ + │ │ └───────────────────────────────────────────────────────────────────┘ │ + ├─────────────────────┼───────────────────────────────────────────────────────────────────────┤ + │ │ ┌───────────────────────────────────────────────────────────────────┐ │ + │ Abstractions and │ │ Concept │ │ + │ contracts for │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ component │ │ Engine │ Toolbar │ │ │ + │ implementations │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ └───────────────────────────────────────────────────────────────────┘ │ + ├─────────────────────├───────────────────────────────────────────────────────────────────────┤ + │ │ ┌────────────────────────────────┐ ┌────────────────────────────────┐ │ + │ │ │ UI │ │ Service │ │ + │ Standalone │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ components │ │ Autocomplete │ │ │ Telemetry │ │ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ └────────────────────────────────┘ │ + │ │ │ Progress │ │ ┌────────────────────────────────┐ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ Support │ │ + │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ │ Colors │ │ │ Kotlin extensions │ │ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ │ Fonts │ │ │ Utilities │ │ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ + │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ + │ │ │ Icons │ │ │ │ │ + │ │ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ │ │ │ + │ │ └────────────────────────────────┘ └────────────────────────────────┘ │ + └─────────────────────┴───────────────────────────────────────────────────────────────────────┘ +``` diff --git a/mobile/android/android-components/docs/contribute/code_coverage.md b/mobile/android/android-components/docs/contribute/code_coverage.md new file mode 100644 index 0000000000..3a3eb6f540 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/code_coverage.md @@ -0,0 +1,25 @@ +--- +layout: page +title: Code coverage +permalink: /contributing/code-coverage +--- + +# Code Coverage + +> In computer science, test coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs. A program with high test coverage, measured as a percentage, has had more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage. ([Wikipedia](https://en.wikipedia.org/wiki/Code_coverage)) + +# Automated reports + +For pull requests and main pushes we generate code coverage reports on taskcluster and they can be accessed via the GitHub checks. + +# Generating reports locally + +Locally you can generate a coverage report for a module with the following command: +```bash +./gradlew -Pcoverage <module>:build +``` + +After that you'll find an HTML report at the following location: +``` +components/<path to module>/build/reports/jacoco/jacocoTestReport/html/index.html +``` diff --git a/mobile/android/android-components/docs/contribute/components_inside_app.md b/mobile/android/android-components/docs/contribute/components_inside_app.md new file mode 100644 index 0000000000..e67469115f --- /dev/null +++ b/mobile/android/android-components/docs/contribute/components_inside_app.md @@ -0,0 +1,148 @@ +--- +layout: page +title: Working on unreleased component code in an app +permalink: /contributing/testing-components-inside-app +--- + +Sometimes it may be helpful to test component code you are working on inside an end-user app that is using this component. + +## Avoid depending on apps living outside of the repository + +Before trying to integrate a modified component into an *external* app try to re-create a test scenario **inside** the repository: + +* Add **Unit tests**: Can you reproduce your scenario as a unit test with the help of [Robolectric](http://robolectric.org/)? Unit tests will create a reproducible scenario for every developer and can prevent regressions in the future. + +* Add or modify a **sample app**: Especially for visual changes an app running on a test device may be needed. Can your scenario be reproduced with a new or existing sample app inside the repository? The sample app will make it easier for other developers to *see* and understand your scenario. And it will make it easier to re-test the same scenario with future versions of a component. + +* Add a **user interface test** for a sample app: The UI test will prevent regressions in your sample app scenario and allows other developers to *replay* your scenario when changing component code. + +## Testing local components code + +Even if you are able to reproduce the scenario *inside* the repository you may still want to test your code with an external app consuming the component. You can do that by *publishing* the component to your *local* Maven repository and configuring the app to pull the dependency from there. This can be achieved manually, or via an automated flow described below. + +### Automated flow: using auto-publication to test local component code + +*android-component* repository contains scripts necessary to automatically determine if there are any local changes, publish them to a local maven repository, and to configure consuming application to use the latest published local version. + +Add the following to your project's `settings.gradle`. This code will execute during the project's Gradle initialization phase, and will trigger android-component's auto-publication scripts when `local.properties` is configured. + +``` +def runCmd(cmd, workingDir, successMessage, captureStdout=true) { + def proc = cmd.execute(null, new File(workingDir)) + def standardOutput = captureStdout ? new ByteArrayOutputStream() : System.out + proc.consumeProcessOutput(standardOutput, System.err) + proc.waitFor() + + if (proc.exitValue() != 0) { + throw new GradleException("Process '${cmd}' finished with non-zero exit value ${proc.exitValue()}"); + } else { + log(successMessage) + } + return captureStdout ? standardOutput : null +} + +Properties localProperties = null +String settingAndroidComponentsPath = "autoPublish.android-components.dir" + +if (file('local.properties').canRead()) { + localProperties = new Properties() + localProperties.load(file('local.properties').newDataInputStream()) + log('Loaded local.properties') +} + +if (localProperties != null) { + localProperties.each { prop -> + gradle.ext.set("localProperties.${prop.key}", prop.value) + } + + String androidComponentsLocalPath = localProperties.getProperty(settingAndroidComponentsPath) + + if (androidComponentsLocalPath != null) { + log("Enabling automatic publication of android-components from: $androidComponentsLocalPath") + def publishAcCmd = ["./automation/publish_to_maven_local_if_modified.py"] + runCmd(publishAcCmd, androidComponentsLocalPath, "Published android-components for local development.", false) + } else { + log("Disabled auto-publication of android-components. Enable it by settings '$settingAndroidComponentsPath' in local.properties") + } +} +``` + +Also, add the following to application's `build.gradle`. This will configure your project to use the latest locally published version of `android-components`. +``` +if (gradle.hasProperty('localProperties.autoPublish.android-components.dir')) { + ext.acSrcDir = gradle."localProperties.autoPublish.android-components.dir" + apply from: "../${acSrcDir}/substitute-local-ac.gradle" +} +``` + +Finally, to enable this workflow, in your project's `local.properties` file, add the following: +``` +autoPublish.android-components.dir=../android-components +``` + +With all of the above done, your project will now be built against your local checkout of `android-components`. This automation is practically zero-cost - if there are no local changes in `android-components`, no additional work will be performed. + +To disable this flow and test against a released version, comment out the `autoPublish` line in `local.properties`. + +#### Hints for working with an auto-publication workflow +- it may be worth it to clean up your local .m2 directory now-and-then, as old builds start to accumulate +- after making changes to `android-components`, press `sync with gradle` in your project's Android Studio to see those changes reflected in your project +- simply pressing `play` in your project's Android Studio should always produce a build with latest `android-components`, even if you didn't `sync` beforehand. + +### Manual flow: using a local Maven repository to test local component code + +This is the fully manual version of the above flow. Generally not recommended for day-to-day use since it's error-prone and more cumbersome. + +#### Setup version number + +In the *android-components* repository update the version number [in the configuration](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/.buildconfig.yml#L1) so that the app will be required to "download" this new version. Try to avoid picking a version number that was or will be officially released. Otherwise you may "pollute" your gradle cache with unofficial releases. To follow [the Maven convention](https://maven.apache.org/guides/getting-started/index.html#What_is_a_SNAPSHOT_version) you may want to use a `-SNAPSHOT` suffix. + +#### Publish to local maven repository + +After changing the components code you can publish your version to your local maven repository. You can either publish all components or just a specific ones: + +```bash +# Publish all components to your local Maven repository: +$ ./gradlew publishToMavenLocal + +# Only publish a single component (ui-autocomplete): +$ ./gradlew ui-autocomplete:publishToMavenLocal +``` + +#### Using a component from the local maven repository + +In your app project locate the `repositories` block inside your build.gradle and add `mavenLocal()` to the list: + +```groovy +repositories { + mavenLocal() + google() + mavenCentral() +} +``` + +Also add `mavenLocal()` to the build.gradle in the 'app' module: + +```groovy +buildscript { + repositories { + mavenLocal() + } +} +``` + +*Note*: There may be two `repositories` blocks in your root build.gradle. One for `buildscript` and one for `allprojects`. Make sure to add it to the `allprojects` block or to a build.gradle file inside a specific gradle module. + +After that update the version number of your component dependency in your app's build files. Gradle will now resolve the dependency from your local maven repository and you can test your changes inside the app. + +#### Iterating and caching + +For every change in the android components repository you will need to run the `install` task to generate a new artifact and publish it to your local Maven repository. + +When using a snapshot version (see above) in an app Gradle should ignore its cache and always try to use the latest version of the artifact. + +To enforce using the latest version of a dependency you can try the `--refresh-dependencies` Gradle command line option or setting up a [ResolutionStrategy with a cache time of 0](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.ResolutionStrategy.html) in your app project. + +## Automated Snapshots + +Snapshots are build daily from the `main` branch and published on [snapshots.maven.mozilla.org](https://snapshots.maven.mozilla.org). diff --git a/mobile/android/android-components/docs/contribute/deprecating.md b/mobile/android/android-components/docs/contribute/deprecating.md new file mode 100644 index 0000000000..8bb6bc5638 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/deprecating.md @@ -0,0 +1,38 @@ +--- +layout: page +title: Deprecating components and code +permalink: /contributing/deprecating +--- + +The *Android Components* team follows a [rapid release process with major releases coming out frequently]({{ site.baseurl }}/contributing/versioning). Therefore API breakage can happen more often when switching to a new version. The *Android Components* team tries to reduce API breakage and uses the following deprecation process to give app teams notice of upcoming changes as early as possible. Deprecated APIs are removed not earlier than 2 releases after the release that introduced the deprecation warning. + +## Avoiding API breakage + +Before breaking or deprecating an API consider options to provide new functionality without breaking the (old) API. + +Examples: +* When adding a new parameter to a method consider adding a default value so that defining a value for this parameter is not required and existing code using the method still compiles. +* When introducing new constructor parameters consider adding an additional constructor and keep/deprecate the existing constructor. + +In general if possible we try to keep "compile compatibility" when an app project upgrades to a new *Android Components* version. + +## When to Deprecate + +Deprecating a public API is needed whenever we introduce a new API and at the same time keep the old API around so that app teams have time to migrate. The goal of deprecating an API should always be to eventually remove it. + +No API guarantees are made for snapshot releases and so far unreleased/unused components. Unreleased APIs can break/change before they are released or used. + +## How to Deprecate + +* Add the **[@Deprecated](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-deprecated/index.html)** annotation to the deprecated API. Try to use the `message` parameter of the annotation to explain the deprecation. If possible use the `replaceWith` parameter to specify a code fragment which should be used as a replacement for the deprecated API usage. + + ```Kotlin + @Deprecated("Use `loadAsync` instead", ReplaceWith("loadAsync(context)")) + suspend fun load(context: Context): Deferred<SearchEngineList> { + // ... + } + ``` + +* Mention the deprecated API and the new API (if there's one) in the **changelog**. +* **File a new issue** about removing the deprecated API. Mention the target version for the removal in the issue title. The default time for removal is 2 releases after the release that introduced the deprecation warning. For example when deprecating an API in release 12.0.0 then file an issue to remove the API in release 14.0.0. For larger changes that require a major refactoring in app projects (e.g. migrating to a whole new component) a longer deprecation cycle can be chosen. The [cross-repo search engine "SearchFox"](https://searchfox.org/mozilla-mobile/source) is a helpful tool to find API usage in other Mozilla mobile projects. +* For larger API changes and breakage consider **sending a mail to the [android-components mailing list](https://lists.mozilla.org/listinfo/android-components)** to notify all consumers of the upcoming change. If a specific team at Mozilla will be required to do a large refactoring to accommodate the API change then try to coordinate the deprecation process and timing with that team. diff --git a/mobile/android/android-components/docs/contribute/design_axioms.md b/mobile/android/android-components/docs/contribute/design_axioms.md new file mode 100644 index 0000000000..40e84ceca1 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/design_axioms.md @@ -0,0 +1,39 @@ +--- +layout: page +title: Design Axioms +permalink: /contributing/design-axioms +--- + +## 💚 100% Kotlin + +Kotlin is the language of choice for components we develop, and it is the recommended language for application projects integrating components. We make use of Kotlin idioms and official Kotlin extension libraries (e.g. Coroutines). + +Java interoperability is not a goal of this project. We may add annotations for Java interoperability (e.g. `@JvmStatic`) at the request of a consumer to facilitate integration, but we are not considering replacing Kotlin idioms for the sake of operability. + +## 🔄 Third-party dependencies + +As providers of a framework, we strive to be independent of other third-party libraries unless strictly needed (e.g. Android Jetpack libraries). + +It is our goal to let our consumers decide what third-party libraries fit their needs, without our components making this choice or introducing an additional, duplicated stack of third-party frameworks. + +Additionally, by limiting external dependencies, we intent to keep the byte size of our components and the consumer apps as small as possible. + +## 🧰 Simple and applicable + +We are building components that directly satisfy the needs of our consumers. We tend to not design components in a vacuum without a defined use case. Components are preferred to be simple and "to the point". Additional functionality is added as needed, never in advance without knowing if it will ever be used (or how). + +## 🎨 Customizability + +Different consumers have different requirements. If possible, we try to provide customization options (e.g. styling) as needed by the consumers. We aim to balance customizability and complexity. When too complex of customization options are required, we prefer alternative component implementations. + +## 🧩 Pluggability + +Even with customization options, not all components will be able to satisfy all needs of all consumers. As such, we strive to depend on interfaces (allowing different implementations) instead of concrete classes. At a component level we prefer the same abstraction and try to depend on "concept components" (e.g. `concept-toolbar`) and their interfaces instead of components providing a specific implementation (e.g. `browser-toolbar`). + +## ✅ Testing + +We strive to have a high code coverage with a high quality suite of tests. Tests are meant to prevent regressions, exercise various parts of the code base, prove correctness and assert an always shippable state. + +## 📓 API Documentation + +While we strive to keep the public API surface as simple and self-explanatory as possible, we provide KDocs for all public API methods to describe the methods purpose, parameters, return types, and possible exceptions, if applicable. diff --git a/mobile/android/android-components/docs/contribute/hosting_code_in_repository.md b/mobile/android/android-components/docs/contribute/hosting_code_in_repository.md new file mode 100644 index 0000000000..c4d0ab02e5 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/hosting_code_in_repository.md @@ -0,0 +1,103 @@ +--- +layout: page +title: Hosting Android code in the Android Components repository +permalink: /contributing/hosting-android-code-in-repository +--- + +When developing a new (Mozilla) Android component - especially one that wraps **Rust** code for multiple platforms - one decision to make is where the Android (Kotlin/Java) code should live. Should it live in a project repository that also contains the Rust code or should it live in the *firefox-android* repository alongside the other Android code? + +This document lists the advantages you get *for free* when hosting your Android code in the *firefox-android* repository (consuming pre-compiled binaries of your Rust code). + +## Build + +#### Managed build system + +The *Android components* team maintains and updates the build process for local development and in automation. + +This includes: +* Updating the build system and code to use the latest Android SDK, build tools and Gradle version +* Updating code across all components to avoid using deprecated APIs + +## Testing + +Having all components in the same repository means that they will be tested as a whole. We can more easily guarantee that all components work well together. We will be doing this with unit tests and integration tests. + +This is specially important for service components, and becomes much more important once we are going to introduce more UI components that will need to use the service components. + +## Releasing and Publishing + +#### Frequent automated releases + +The *Android components* team releases frequently (currently weekly) and the process is fully automated. Code changes will be part of the next release without any further steps. + +#### Publishing + +Release artifacts are automatically published on maven.mozilla.org and are immediately available to consumers. + +The *Android components* team is planning to release SNAPSHOT builds for every successful *main* merge. This will allow consuming apps to test approved changes immediately after merge without needing to wait for a release. + +#### Consistent versioning and compatibility + +All components are released together with a shared version number. This guarantees that all components with the same version number are compatible. With that a consuming app can use the components in the same setup as they were developed and tested in the *firefox-android* repository. + +#### Less Fragmentation + +All components will be together in the same repository in a clearly named and hierarchical structure. + +## Documentation + +When all components are in the same project, it is much simpler to generate consistent and linked documentation. As an example, see the [service-firefox-accounts](https://mozilla-mobile.github.io/android-components/api/0.19.1/service-firefox-accounts/index.html) API Reference that is automatically generated and published when we do a release. + +This is not just a developer convenience, it is also highly convenient for users of the components (internal and external). It gives them a consistent view of the project as a whole. + +## Tooling + +The *Android components* team evaluates, integrates and maintains tools that run in automation for improving code quality, keeping a consistent code style, avoiding bugs and detecting performance issues. Those tools are getting run automatically for all pull requests and merges. + +Currently we are using: +* [Android Lint](https://developer.android.com/studio/write/lint) - *"Android Lint is a tool which scans Android project sources for potential bugs."* +* [ktlint](https://github.com/shyiko/ktlint) - *"An anti-bikeshedding Kotlin linter with built-in formatter"* +* [detekt](https://github.com/arturbosch/detekt) - *"A static code analysis tool for the Kotlin programming language"* + +In addition to that the *Android components* team started to write [custom lint rules](https://github.com/mozilla-mobile/firefox-android/tree/main/android-components/components/tooling/lint) that enforce component related rules (e.g. "Use the provided logging class in components instead of android.util.Log"). + +## Services + +#### Integration of third-party services + +The *Android components* team evaluates, integrates and maintains third-party services for code quality, testing and automation purposes. + +Currently we are using: +* *Firebase Device Lab* for running tests on real devices. + +The *Android components* team is currently evaluating [Gradle Enterprise](https://gradle.com/) to speed up builds and improve build reliability. + +## Development and code quality + +#### Consistent component APIs + +The *Android components* team makes sure all components have consistent APIs, and use the same patterns and conventions so that consumers can rely on component interoperability. + +#### Base component + +The *Android components* team maintains a *base component* (`support-base`) that offers basic building blocks for components. This avoids code duplication ("Every components implements its own Observable") and allows the app to configure certain behaviors in a consistent way. + +For example the *base component* offers components a logging class for logging inside components. The consuming app can configure how, if and when something gets logged for all components using this logging class. A similar setup is planned for telemetry and other functionality where the component needs to invoke something that the consuming app should be in control of. + +## Dependencies + +#### Optimized dependency graph + +The *Android components* team ensures that the dependency graph is as minimal as possible by avoiding duplicates and version conflicts. This is much easier to do in a monorepo with a single buildchain. If fact, keeping dependency versions consistent across many different repositories is hard and somewhat unrealistic. + +#### Consistent releases + +The *Android components* team has a high release cadence. A release is cut at the end of every sprint. This makes updates of all components available to internal and external consumers at the end of every week. + +## Ownership + +You remain the owner of your components code. Each component can have a [CODEOWNERS](https://help.github.com/articles/about-codeowners/) file that allows us to specify who owns and reviews the code. This means individual teams can be easily responsible for a subtree of the components project that they own. + +## Conclusion + +The *Android components* team recommends hosting Android component code (Kotlin) in the *firefox-android* repository for the reasons mentioned above. diff --git a/mobile/android/android-components/docs/contribute/merge_day.md b/mobile/android/android-components/docs/contribute/merge_day.md new file mode 100644 index 0000000000..8d2d4f2bf4 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/merge_day.md @@ -0,0 +1,146 @@ +--- +layout: page +title: Merge day process +permalink: /contributing/merge-day +--- + +## What is merge day? + +Firefox and Gecko(View) are released in multiple release channels: `Nightly`, `Beta`, `Release`. Those versions are maintained in separate repositories (`mozilla-central`, `mozilla-beta`, `mozilla-release`). Right before the release of a new version of Firefox, on "merge day", those repositories get merged so that: The `Beta` version becomes the `Release` version. The `Nightly` version comes the `Beta` version and the `Nightly` version gets a higher version number. + + + +As an example: +* GeckoView Beta 70.0 (`mozilla-beta`) -> GeckoView Release 70.0 (`mozilla-release`) +* GeckoView Nightly 71.0 (`mozilla-central`) -> GeckoView Beta 71.0 (`mozilla-beta`) +* GeckoView Nightly 71.0 (`mozilla-central`) -> GeckoView Nightly 72.0 (`mozilla-central`) + +Since the *Android Components* project uses separate components for tracking the GeckoView release channels we need to perform the same changes in the *Android Components* repository by moving the code accordingly and updating the dependency versions. + +For example: + +* `browser-engine-gecko-beta` (using GeckoView Beta 70.0) -> `browser-engine-gecko-release` (using GeckoView Release 70.0) +* `browser-engine-gecko-nightly` (using GeckoView Nightly 71.0) -> `browser-engine-gecko-beta` (using GeckoView Beta 71.0) +* `browser-engine-gecko-nightly` -> Starts using GeckoView 72.0 + +A new release happens roughly every 6 weeks. See the release calendar: +https://wiki.mozilla.org/Release_Management/Calendar + +## Process + +### Preparation + +Before starting the "Merge day" process make sure that all new major versions for all release channels of GeckoView (nightly, beta, release) are available on +https://maven.mozilla.org/. + +* Release: https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview/ +* Beta: https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview-beta/ +* Nightly: https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview-nightly/ + +### Beta -> Release + +First we delete the existing code of the `browser-engine-gecko` component and replace it with the code of the `browser-engine-gecko-beta` component: + +``` +rm -rf components/browser/engine-gecko/src +cp -R components/browser/engine-gecko-beta/src components/browser/engine-gecko/ +``` + +In addition to that we need to replace the metrics file: + +``` +cp -R components/browser/engine-gecko-beta/metrics.yaml components/browser/engine-gecko +``` + +After that we update `Gecko.kt` to use the latest GeckoView release version (used by `browser-engine-gecko`) from https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview/ + +``` +vi buildSrc/src/main/java/Gecko.kt +``` + +Finally we build and test the component to see whether there are any issues. + +``` +./gradlew browser-engine-gecko:test +``` + +See the "Build failures" section if the build or tests fail. + +If the build and test passes then commit this change. + +Example commit message: +`Issue #xxxx: (Merge day) browser-engine-gecko-beta (70) -> browser-engine-gecko (70)` + +### Nightly -> Beta + +Now we are removing the existing code of the `browser-engine-gecko-beta` component and replace it with the code of the `browser-engine-gecko-nightly` component: + +``` +rm -rf components/browser/engine-gecko-beta/src +cp -R components/browser/engine-gecko-nightly/src components/browser/engine-gecko-beta +``` + +In addition to that we need to replace the metrics file: + +``` +cp -R components/browser/engine-gecko-nightly/metrics.yaml components/browser/engine-gecko-beta +``` + +After that we update `Gecko.kt` to use the latest GeckoView beta version (previously used by `browser-engine-gecko-nightly`) from https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview-beta/ + +``` +vi buildSrc/src/main/java/Gecko.kt +``` + +Finally we build and test the component to see whether there are any issues. + +``` +./gradlew browser-engine-gecko-beta:test +``` + +See the "Build failures" section if the build or tests fail. + +If the build and test passes then commit this change. + +Example commit message: +`Issue #xxxx: (Merge day) browser-engine-gecko-nightly (71) -> browser-engine-gecko-beta (71)` + + +### Nightly + +Finally we need to update the nightly component. For this component we do not need to copy any code and instead can update the version of the component immediately from https://maven.mozilla.org/?prefix=maven2/org/mozilla/geckoview/geckoview-nightly/ + +``` +vi buildSrc/src/main/java/Gecko.kt +``` + +Again we are building and running the tests of the component: + +``` +./gradlew browser-engine-gecko-nightly:test +``` + +See the "Build failures" section if the build or tests fail. + +If the build and test passes then commit this change. + +Example commit message: +`Issue #xxxx: (Merge day) browser-engine-gecko-nightly -> 72.` + +### Changelog + +Finally add an entry to the changelog: + +``` +* **browser-engine-gecko**, **browser-engine-gecko-beta**, **browser-engine-gecko-nightly** + * **Merge day!** + * `browser-engine-gecko-release`: GeckoView 70.0 + * `browser-engine-gecko-beta`: GeckoView 71.0 + * `browser-engine-gecko-nightly`: GeckoView 72.0 +``` + +### Build failures + +* ***Gradle***: A common reason for build failures is changes in the Gradle configuration or the list of dependencies. Since we do not copy `build.gradle` (because it contains channel specific configuration) we need to manually compare the files to see if there's something to change. + +* ***Breaking API changes***: Especially when updating the Nightly component it can happen that there are breaking API changes that landed in GeckoView just right after the new version was started. So the first time we see them is after merge day when updating the major version of the dependency. In this case we need to fix those breaking changes manually like other breaking changes that happen in "normal" development phases. diff --git a/mobile/android/android-components/docs/contribute/release_checklist.md b/mobile/android/android-components/docs/contribute/release_checklist.md new file mode 100644 index 0000000000..e7276b127c --- /dev/null +++ b/mobile/android/android-components/docs/contribute/release_checklist.md @@ -0,0 +1,94 @@ +--- +layout: page +title: Release checklist +permalink: /contributing/release-checklist +--- + +These are instructions for preparing a release branch for Firefox Android and starting the next Nightly development cycle. + +## [Release Engineering / Release Management] Beta release branch creation & updates + +**This part is covered by the Release Engineering and Release Management teams. The dev team should not perform these steps.** + +1. Release Management emails the release-drivers list to begin merge day activities. For example, `Please merge mozilla-central to mozilla-beta, bump central to 123.` A message will also be sent to the #releaseduty Matrix channel. +2. The Release Engineering engineer on duty (“relduty”) triggers the Taskcluster merge day hooks. The hooks automate the following tasks: + - Migrating tip of mozilla-central to mozilla-beta + - Updating version.txt on mozilla-central for the new Nightly version + - Updating version.txt on mozilla-beta to change the version string from `[beta_version].0a1` to `[beta_version].0b1` +3. When the merge day tasks are completed, relduty will respond to the release-drivers thread and #releaseduty channel that merges are completed. This also serves as notification to the dev team that they can start the new Nightly development cycle per the steps given in the next section ⬇️ +4. In [ApplicationServices.kt](https://hg.mozilla.org/mozilla-central/file/default/mobile/android/android-components/plugins/dependencies/src/main/java/ApplicationServices.kt): + - Set `VERSION` to `[major-version].0` + - Set `CHANNEL` to `ApplicationServicesChannel.RELEASE` + + ```diff + --- a/mobile/android/android-components/plugins/dependencies/src/main/java/ApplicationServices.kt + +++ b/mobile/android/android-components/plugins/dependencies/src/main/java/ApplicationServices.kt + -val VERSION = "121.20231118050255" + -val CHANNEL = ApplicationServicesChannel.NIGHTLY + +val VERSION = "121.0" + +val CHANNEL = ApplicationServicesChannel.RELEASE + ``` + - Create a commit named `Switch to Application Services Release`. (Note: application-services releases directly after the nightly cycle, there's no beta cycle). +5. Once all of the above commits have landed, create a new `Firefox Android (Android-Components, Fenix, Focus)` release in [Ship-It](https://shipit.mozilla-releng.net/) and continue with the release checklist per normal practice. + +## [Dev team] Starting the next Nightly development cycle + +**Please handle this part once Release Management gives you the go.** + +Now that we made the Beta cut, we can remove all the unused strings marked moz:removedIn <= `[release_version subtract 1]`. `[release_version]` should follow the Firefox Release version. See [Firefox Release Calendar](https://wiki.mozilla.org/Release_Management/Calendar) for the current Release version. We will also want to bump the Android Component's [changelog.md](https://hg.mozilla.org/mozilla-central/file/default/mobile/android/android-components/docs/changelog.md) with the new Nightly development section. + +0. Wait for greenlight coming from Release Engineering (see #3 above). +1. File a Bugzilla issue named "Start the Nightly `[nightly_version]` development cycle". +2. Search and remove all strings marked `moz:removedIn="[release_version subtract 1]"` across Fenix, Focus and Android Components. Please avoid removing strings in the localized `strings.xml` and limit changes only to `values/strings.xml`. +3. Add the next Nightly in development section in the [changelog.md](https://hg.mozilla.org/mozilla-central/file/default/mobile/android/android-components/docs/changelog.md).: + - Add a new `[nightly_version].0 (In Development)` section for the next Nightly version with the next commit and milestone numbers. + - Update the `[beta_version].0` section, update the links to `release_v[beta_version]` and specifying the correct commit ranges. This should equate to changing `/blob/main/` to `/blob/releases_v[beta_version]/`. + + ```diff + diff --git a/docs/changelog.md b/docs/changelog.md + index 9e95d0e2adc..d901ed38cdd 100644 + --- a/docs/changelog.md + +++ b/docs/changelog.md + @@ -4,12 +4,18 @@ title: Changelog + permalink: /changelog/ + --- + + -# 114.0 (In Development) + -* [Commits](https://github.com/mozilla-mobile/firefox-android/compare/releases_v113..main) + +# 115.0 (In Development) + +* [Commits](https://github.com/mozilla-mobile/firefox-android/compare/releases_v114..main) + * [Dependencies](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/plugins/dependencies/src/main/java/DependenciesPlugin.kt) + * [Gecko](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/plugins/dependencies/src/main/java/Gecko.kt) + * [Configuration](https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/.config.yml) + + +# 114.0 + +* [Commits](https://github.com/mozilla-mobile/firefox-android/compare/releases_v113..releases_v114) + +* [Dependencies](https://github.com/mozilla-mobile/firefox-android/blob/releases_v114/android-components/plugins/dependencies/src/main/java/DependenciesPlugin.kt) + +* [Gecko](https://github.com/mozilla-mobile/firefox-android/blob/releases_v114/android-components/plugins/dependencies/src/main/java/Gecko.kt) + +* [Configuration](https://github.com/mozilla-mobile/firefox-android/blob/releases_v114/android-components/.config.yml) + + + * * **browser-state** + * 🌟 Added `DownloadState`.`openInApp` to indicate whether or not the file associated with the download should be opened in a third party app after downloaded successfully, for more information see [bug 1829371](https://bugzilla.mozilla.org/show_bug.cgi?id=1829371) and [bug 1829372](https://bugzilla.mozilla.org/show_bug.cgi?id=1829372). + ``` + +4. Submit a patch for review and land. + +### [Dev Team] Renew telemetry + +After the Beta cut, another task is to remove all soon to expire telemetry probes. What we're looking for is to create a list of telemetry that will expire in `[nightly_version add 1]`. See [Firefox Release Calendar](https://whattrainisitnow.com/calendar/) for the current Release version. There is a script that will help with finding these soon to expire telemetry. + +1. Use the helper in the mobile/android/fenix/tools folder `python3 data_renewal_generate.py [nightly_version add 1]` to detect and generate files that will help create the following files: + - `[nightly_version add 1]`_expiry_list.csv +2. File an issue for removing expired telemetry to address the expired metrics. See [Bug 1881336](https://bugzilla.mozilla.org/show_bug.cgi?id=1881336) for an example. +3. Remove the expired metrics. See [example](https://github.com/mozilla-mobile/firefox-android/pull/5700). + +### [Dev Team] Add SERP Telemetry json dump + +After the beta cut, another task is to add SERP telemetry json to the [search-telemetry-v2.json](https://hg.mozilla.org/mozilla-central/file/default/mobile/android/android-components/components/feature/search/src/main/assets/search/search_telemetry_v2.json) +The dump is to be fetched from the desktop telemetry dump located at [desktop-search-telemetry-v2.json](https://searchfox.org/mozilla-central/source/services/settings/dumps/main/search-telemetry-v2.json) + +### Ask for Help + +- Issues related to releases `#releaseduty` on Element +- Topics about CI (and the way we use Taskcluster) `#Firefox CI` on Element +- Breakage in PRs due to Gradle issues or GV upgrade problems `#mobile-android-team` on Slack diff --git a/mobile/android/android-components/docs/contribute/updating_tracking_protection_lists.md b/mobile/android/android-components/docs/contribute/updating_tracking_protection_lists.md new file mode 100644 index 0000000000..17f7fbba59 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/updating_tracking_protection_lists.md @@ -0,0 +1,61 @@ +--- +layout: page +title: Updating the tracking protecting lists process +permalink: /contributing/update-tracking-protection-list +--- + +## What is tracking protection? + +It is a feature that prevents trackers from collecting your personal information like your browsing habits and interests. In `GeckoEngine` it comes [built-in in Gecko(View)](https://mozilla.github.io/geckoview/javadoc/mozilla-central/org/mozilla/geckoview/ContentBlockingController.ContentBlockingException.html). For the `SystemEngine` we have to [created our own implementation](https://github.com/vladikoff/android-components/blob/main/components/browser/engine-system/src/main/java/mozilla/components/browser/engine/system/matcher/UrlMatcher.kt), +which relays on different lists that indicates what should be considered as a tracker or not. + +## The lists + +The following lists are kept outside of the Android Components repository in the [shavar-prod-lists](https://github.com/mozilla-services/shavar-prod-lists) repository. + + +| AC | Shavar | Purpose | +|-----------------------|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [domain_blocklist.json][1] | [disconnect-blocklist.json][2] | This blocklist is the core of tracking protection in Firefox. [For more information][3]. | +| [domain_safelist.json][4] | [disconnect-entitylist.json][5] | It is used to allow third-party subresources that are wholly owned by the same company that owns the top-level website that the user is visiting for more details. [For more information][6].| + + + +## Updating process + +For every FireFox release, a new branch gets created following the same pattern as [merge-day](https://mozac.org/contributing/merge-day), +where `main` contains `nightly`'s lists, the higher branch number contains `Beta` and the higher branch number before contains `Stable`. +That means that every [merge-day](https://mozac.org/contributing/merge-day), we need to update the [domain_blocklist.json][1] and [domain_safelist.json][4] files with their counterpart on [shavar-prod-lists](https://github.com/mozilla-services/shavar-prod-lists). + +### Preparation +1. Go to the [shavar-prod-lists](https://github.com/mozilla-services/shavar-prod-lists) repository +2. Switch to the higher branch number (Beta). +3. Copy the content of [disconnect-blocklist.json][2] -> [domain_blocklist.json][1] +4. Run all the tests on `browser-engine-system` 🤞 +5. 🔴 `If` something fails proceed to verify what's causing the issue and address it. +6. ✅ `Else` proceed to copy the content of [disconnect-entitylist.json][5] -> [domain_safelist.json][4] +7. Run all the tests on `browser-engine-system` 🤞 +8. 🔴 `If` something fails proceed to verify what's causing the issue and address it. +9. ✅ `Else` proceed to commit your changes using the guideline below. + +To keep track of every update, we include the [shavar-prod-lists](https://github.com/mozilla-services/shavar-prod-lists) commit hash as part of our commit message. +This way we can know which version of the lists we have on AC. + +``` +Closes #6163: Update tracking protection lists + +domain_blocklist.json maps to disconnect-blocklist.json +commit(shavar-prod-lists) d5755856f4eeab4ce5e8fb7896600ed7768368e5 + +domain_safelist.json maps to disconnect-entitylist.json +commit(shavar-prod-lists) 01dcca911aa7787fd835a1a19cef1012296f4eb7 +``` + +THE END! + +[1]: https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/browser/engine-system/src/main/res/raw/domain_blocklist.json +[2]: https://github.com/mozilla-services/shavar-prod-lists/blob/master/disconnect-blocklist.json +[3]: https://github.com/mozilla-services/shavar-prod-lists/blob/master/README.md#disconnect-blocklist.json +[4]: https://github.com/mozilla-mobile/firefox-android/blob/main/android-components/components/browser/engine-system/src/main/res/raw/domain_safelist.json +[5]: https://github.com/mozilla-services/shavar-prod-lists/blob/master/disconnect-entitylist.json +[6]: https://github.com/mozilla-services/shavar-prod-lists/blob/master/README.md#disconnect-entitylistjson diff --git a/mobile/android/android-components/docs/contribute/versioning.md b/mobile/android/android-components/docs/contribute/versioning.md new file mode 100644 index 0000000000..19a7b59a66 --- /dev/null +++ b/mobile/android/android-components/docs/contribute/versioning.md @@ -0,0 +1,35 @@ +--- +layout: page +title: Versioning and release process +permalink: /contributing/versioning +--- + +The *Android components* project uses a similar [versioning and release process as Firefox](https://wiki.mozilla.org/Release_Management/Release_Process) (New major version releases at fixed intervals). + +## Major releases + +* Major releases happen at fixed intervals (currently on Tuesday every week) +* Every release increments the major version (e.g. 1.0.0 -> 2.0.0 -> 3.0.0) +* Major releases are tagged on and shipped from the `main` branch. + +## Point releases + +* Point releases happen on demand whenever an app project requires a new version but cannot update to the latest release version yet or whenever a new release version is not available yet. +* Point releases are created from a release branch, created from a previously tagged release. +* The minor or patch version is increased depending on whether the release introduces backported functionality or just fixes bugs (e.g. 1.0.0 -> 1.1.0 or 2.0.0 -> 2.0.1). + +## Rationale and alternatives + +#### Release cycle and stability + +The *Android components* project follows a rapid release cycle to get new functionality into the hands of consuming apps as soon as possible. For this reason the project does not have a long release stability phase (e.g. 1.0.0 -> 1.0.1 -> 1.0.2 -> 1.1.0 -> 1.1.1). + +App teams are encouraged to update to the latest release frequently. The *Android Components* team tries to minimize the amount of breaking API changes. Frequent updates minimize the amount of API changes when updating *Android Components*. + +#### Versioning + +All *Android Components* are released together using a unified version number. This makes it easy for the *Android Components* team to guarantee that this set of components is guaranteed and tested to work together. + +The downside of a shared version is the loss of [semantic versioning](https://semver.org/) semantics on a per component level. It is entirely possible that a component is released using a new major version and there's not a single code change in this component in the new release. Instead the [changelog](https://mozac.org/changelog/) is the primary mechanism to communicate major and minor changes in a component. + +At this time individually semantically versioned components would introduce a too large maintenance overhead for the *Android Components* team (Individual releases, release manager/owner for every component, guaranteeing compatibility across many versions). |