diff options
Diffstat (limited to 'third_party/libwebrtc/build/android/docs')
8 files changed, 1415 insertions, 0 deletions
diff --git a/third_party/libwebrtc/build/android/docs/README.md b/third_party/libwebrtc/build/android/docs/README.md new file mode 100644 index 0000000000..5ee0ca638f --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/README.md @@ -0,0 +1,16 @@ +# Android Build Docs + +* [//docs/android_build_instructions.md](/docs/android_build_instructions.md) +* [//docs/android_dynamic_feature_modules.md](/docs/android_dynamic_feature_modules.md) +* [build_config.md](build_config.md) +* [coverage.md](coverage.md) +* [java_toolchain.md](java_toolchain.md) +* [java_optimization.md](java_optimization.md) +* [lint.md](lint.md) +* [life_of_a_resource.md](life_of_a_resource.md) +* [../incremental_install/README.md](../incremental_install/README.md) +* [//docs/ui/android/bytecode_rewriting.md](/docs/ui/android/bytecode_rewriting.md) +* [go/doubledown](https://goto.google.com/doubledown) (Googlers only) + +See also: +* [//build/README.md](../../README.md) diff --git a/third_party/libwebrtc/build/android/docs/build_config.md b/third_party/libwebrtc/build/android/docs/build_config.md new file mode 100644 index 0000000000..8f752a6691 --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/build_config.md @@ -0,0 +1,168 @@ +# Introduction + +This document describes the `.build_config.json` files that are used by the +Chromium build system for Android-specific targets like APK, resources, +and more. + +[TOC] + +# I. Overview of .build_config.json files: + +The Android build requires performing computations about dependencies in +various targets, which are not possible with the GN build language. To address +this, `.build_config.json` files are written during the build to store the needed +per-target information as JSON files. + +They are always written to `$target_gen_dir/${target_name}.build_config.json`. + +Many scripts under [`build/android/gyp/`](build/android_gyp/), which are used +during the build, can also accept parameter arguments using +`@FileArg references`, which look like: + + --some-param=@FileArg(<filename>:<key1>:<key2>:..<keyN>) + +This placeholder will ensure that `<filename>` is read as a JSON file, then +return the value at `[key1][key2]...[keyN]` for the `--some-param` option. + +Apart from that, the scripts do not need to know anything about the structure +of `.build_config.json` files (but the GN rules that invoke them do and select +which `@FileArg()` references to use). + +For a concrete example, consider the following GN fragment: + +```gn +# From //ui/android/BUILD.gn: +android_resources("ui_java_resources") { + custom_package = "org.chromium.ui" + resource_dirs = [ "java/res" ] + deps = [ + ":ui_strings_grd", + ] +} +``` + +This will end up generating the following JSON file under +`$CHROMIUM_OUTPUT_DIR/gen/ui/android/ui_java_resources.build_config.json`: + +```json +{ + "deps_info": { + "deps_configs": [ + "gen/ui/android/ui_strings_grd.build_config.json" + ], + "name": "ui_java_resources.build_config.json", + "package_name": "org.chromium.ui", + "path": "gen/ui/android/ui_java_resources.build_config.json", + "r_text": "gen/ui/android/ui_java_resources_R.txt", + "resources_dirs": [ + "../../ui/android/java/res" + ], + "resources_zip": "resource_zips/ui/android/ui_java_resources.resources.zip", + "srcjar": "gen/ui/android/ui_java_resources.srcjar", + "type": "android_resources" + }, + "gradle": {}, + "resources": { + "dependency_zips": [ + "resource_zips/ui/android/ui_strings_grd.resources.zip" + ], + "extra_package_names": [], + } +} +``` + +NOTE: All path values in `.build_config.json` files are relative to your +`$CHROMIUM_OUTPUT_DIR`. + +# II. Generation of .build_config.json files: + +They are generated by the GN [`write_build_config()`](gn_write_build_config) +internal template, which ends up invoking +[`write_build_config.py`](write_build_config_py). For our example above, this +is with the following parameters: + +``` +python ../../build/android/gyp/write_build_config.py \ + --type=android_resources \ + --depfile gen/ui/android/ui_java_resources__build_config_crbug_908819.d \ + --deps-configs=\[\"gen/ui/android/ui_strings_grd.build_config.json\"\] \ + --build-config gen/ui/android/ui_java_resources.build_config.json \ + --resources-zip resource_zips/ui/android/ui_java_resources.resources.zip \ + --package-name org.chromium.ui \ + --r-text gen/ui/android/ui_java_resources_R.txt \ + --resource-dirs=\[\"../../ui/android/java/res\"\] \ + --srcjar gen/ui/android/ui_java_resources.srcjar +``` + +Note that *most* of the content of the JSON file comes from command-line +parameters, but not all of it. + +In particular, the `resources['dependency_zips']` entry was computed by +inspecting the content of all dependencies (here, only +`ui_string_grd.build_config.json`), and collecting their +`deps_configs['resources_zip']` values. + +Because a target's `.build_config.json` file will always be generated after +that of all of its dependencies, +[`write_build_config.py`](write_build_config_py) can traverse the +whole (transitive) set of direct *and* indirect dependencies for a given target +and extract useful information out of it. + +This is the kind of processing that cannot be done at the GN language level, +and is very powerful for Android builds. + + +# III. Usage of .build_config.json files: + +In addition to being parsed by `write_build_config.py`, when they are listed +in the `--deps-configs` of a given target, the `.build_config.json` files are used +by other scripts under [build/android/gyp/] to build stuff. + +For example, the GN `android_resources` template uses it to invoke the +[`process_resources.py`] script with the following command, in order to +generate various related files (e.g. `ui_java_resources_R.txt`): + +```sh +python ../../build/android/gyp/process_resources.py \ + --depfile gen/ui/android/ui_java_resources_1.d \ + --android-sdk-jar ../../third_party/android_sdk/public/platforms/android-29/android.jar \ + --aapt-path ../../third_party/android_sdk/public/build-tools/29.0.2/aapt \ + --dependencies-res-zips=@FileArg\(gen/ui/android/ui_java_resources.build_config.json:resources:dependency_zips\) \ + --extra-res-packages=@FileArg\(gen/ui/android/ui_java_resources.build_config.json:resources:extra_package_names\) \ + --resource-dirs=\[\"../../ui/android/java/res\"\] \ + --debuggable \ + --resource-zip-out resource_zips/ui/android/ui_java_resources.resources.zip \ + --r-text-out gen/ui/android/ui_java_resources_R.txt \ + --srcjar-out gen/ui/android/ui_java_resources.srcjar \ + --non-constant-id \ + --custom-package org.chromium.ui \ + --shared-resources +``` + +Note the use of `@FileArg()` references here, to tell the script where to find +the information it needs. + + +# IV. Format of .build_config.json files: + +Thanks to `@FileArg()` references, Python build scripts under +[`build/android/gyp/`](build/android/gyp/) do not need to know anything +about the internal format of `.build_config.json` files. + +This format is decided between internal GN build rules and +[`write_build_config.py`][write_build_config_py]. Since these changes rather +often, the format documentation is kept inside the Python script itself, but +can be extracted as a Markdown file and visualized with the following commands: + +```sh +# Extract .build_config.json format documentation +build/android/gyp/write_build_config.py \ + --generate-markdown-format-doc > /tmp/format.md + +# Launch a browser to visualize the format documentation. +python tools/md_browser/md_browser.py -d /tmp /tmp/format.md +``` + +[build/android/gyp/]: https://chromium.googlesource.com/chromium/src/build/+/main/android/gyp/ +[gn_write_build_config]: https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?q=write_build_config&sq=package:chromium +[write_build_config_py]: https://chromium.googlesource.com/chromium/src/build/+/main/android/gyp/write_build_config.py diff --git a/third_party/libwebrtc/build/android/docs/class_verification_failures.md b/third_party/libwebrtc/build/android/docs/class_verification_failures.md new file mode 100644 index 0000000000..e3e474539e --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/class_verification_failures.md @@ -0,0 +1,286 @@ +# Class Verification Failures + +[TOC] + +## What's this all about? + +This document aims to explain class verification on Android, how this can affect +app performance, how to identify problems, and chromium-specific solutions. For +simplicity, this document focuses on how class verification is implemented by +ART, the virtual machine which replaced Dalvik starting in Android Lollipop. + +## What is class verification? + +The Java language requires any virtual machine to _verify_ the class files it +loads and executes. Generally, verification is extra work the virtual machine is +responsible for doing, on top of the work of loading the class and performing +[class initialization][1]. + +A class may fail verification for a wide variety of reasons, but in practice +it's usually because the class's code refers to unknown classes or methods. An +example case might look like: + +```java +public class WindowHelper { + // ... + public boolean isWideColorGamut() { + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O_MR1) { + return mWindow.isWideColorGamut(); + } + return false; + } +} +``` + +### Why does that fail? + +In this example, `WindowHelper` is a helper class intended to help callers +figure out wide color gamut support, even on pre-OMR1 devices. However, this +class will fail class verification on pre-OMR1 devices, because it refers to +[`Window#isWideColorGamut()`][2] (new-in-OMR1), which appears to be an undefined +method. + +### Huh? But we have an SDK check! + +SDK checks are completely irrelevant for class verification. Although readers +can see we'll never call the new-in-OMR1 API unless we're on >= OMR1 devices, +the Oreo version of ART doesn't know `isWideColorGamut()` was added in next +year's release. From ART's perspective, we may as well be calling +`methodWhichDoesNotExist()`, which would clearly be unsafe. + +All the SDK check does is protect us from crashing at runtime if we call this +method on Oreo or below. + +### Class verification on ART + +While the above is a mostly general description of class verification, it's +important to understand how the Android runtime handles this. + +Since class verification is extra work, ART has an optimization called **AOT +("ahead-of-time") verification**¹. Immediately after installing an app, ART will +scan the dex files and verify as many classes as it can. If a class fails +verification, this is usually a "soft failure" (hard failures are uncommon), and +ART marks the class with the status `RetryVerificationAtRuntime`. + +`RetryVerificationAtRuntime`, as the name suggests, means ART must try again to +verify the class at runtime. ART does so the first time you access the class +(right before class initialization/`<clinit>()` method). However, depending on +the class, this verification step can be very expensive (we've observed cases +which take [several milliseconds][3]). Since apps tend to initialize most of +their classes during startup, verification significantly increases startup time. + +Another minor cost to failing class verification is that ART cannot optimize +classes which fail verification, so **all** methods in the class will perform +slower at runtime, even after the verification step. + +*** aside +¹ AOT _verification_ should not be confused with AOT _compilation_ (another ART +feature). Unlike compilation, AOT verification happens during install time for +every application, whereas recent versions of ART aim to apply AOT compilation +selectively to optimize space. +*** + +## Chromium's solution + +In Chromium, we try to avoid doing class verification at runtime by +manually out-of-lining all Android API usage like so: + +```java +public class ApiHelperForOMR1 { + public static boolean isWideColorGamut(Window window) { + return window.isWideColorGamut(); + } +} + +public class WindowHelper { + // ... + public boolean isWideColorGamut() { + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O_MR1) { + return ApiHelperForOMR1.isWideColorGamut(mWindow); + } + return false; + } +} +``` + +This pushes the class verification failure out of `WindowHelper` and into the +new `ApiHelperForOMR1` class. There's no magic here: `ApiHelperForOMR1` will +fail class verification on Oreo and below, for the same reason `WindowHelper` +did previously. + +The key is that, while `WindowHelper` is used on all API levels, it only calls +into `ApiHelperForOMR1` on OMR1 and above. Because we never use +`ApiHelperForOMR1` on Oreo and below, we never load and initialize the class, +and thanks to ART's lazy runtime class verification, we never actually retry +verification. **Note:** `list_class_verification_failures.py` will still list +`ApiHelperFor*` classes in its output, although these don't cause performance +issues. + +### Creating ApiHelperFor\* classes + +There are several examples throughout the code base, but such classes should +look as follows: + +```java +/** + * Utility class to use new APIs that were added in O_MR1 (API level 27). + * These need to exist in a separate class so that Android framework can successfully verify + * classes without encountering the new APIs. + */ +@VerifiesOnOMR1 +@TargetApi(Build.VERSION_CODES.O_MR1) +public class ApiHelperForOMR1 { + private ApiHelperForOMR1() {} + + // ... +} +``` + +* `@VerifiesOnO_MR1`: this is a chromium-defined annotation to tell proguard + (and similar tools) not to inline this class or its methods (since that would + defeat the point of out-of-lining!) +* `@TargetApi(Build.VERSION_CODES.O_MR1)`: this tells Android Lint it's OK to + use OMR1 APIs since this class is only used on OMR1 and above. Substitute + `O_MR1` for the [appropriate constant][4], depending when the APIs were + introduced. +* Don't put any `SDK_INT` checks inside this class, because it must only be + called on >= OMR1. + +### Out-of-lining if your method has a new type in its signature + +Sometimes you'll run into a situation where a class **needs** to have a method +which either accepts a parameter which is a new type or returns a new type +(e.g., externally-facing code, such as WebView's glue layer). Even though it's +impossible to write such a class without referring to the new type, it's still +possible to avoid failing class verification. ART has a useful optimization: if +your class only moves a value between registers (i.e., it doesn't call any +methods or fields on the value), then ART will not check for the existence of +that value's type. This means you can write your class like so: + +```java +public class FooBar { + // FooBar needs to have the getNewTypeInAndroidP method, but it would be + // expensive to fail verification. This method will only be called on >= P + // but other methods on the class will be used on lower OS versions (and + // also can't be factored into another class). + public NewTypeInAndroidP getNewTypeInAndroidP() { + assert Build.VERSION.SDK_INT >= Build.VERSION_CODES.P; + // Stores a NewTypeInAndroidP in the return register, but doesn't do + // anything else with it + return ApiHelperForP.getNewTypeInAndroidP(); + } + + // ... +} + +@VerifiesOnP +@TargetApi(Build.VERSION_CODES.P) +public class ApiHelperForP { + public static NewTypeInAndroidP getNewTypeInAndroidP() { + return new NewTypeInAndroidP(); + } + + // ... +} +``` + +**Note:** this only works in ART (L+), not Dalvik (KitKat and earlier). + +## Investigating class verification failures + +Class verification is generally surprising and nonintuitive. Fortunately, the +ART team have provided tools to investigate errors (and the chromium team has +built helpful wrappers). + +### Listing failing classes + +The main starting point is to figure out which classes fail verification (those +which ART marks as `RetryVerificationAtRuntime`). This can be done for **any +Android app** (it doesn't have to be from the chromium project) like so: + +```shell +# Install the app first. Using Chrome as an example. +autoninja -C out/Default chrome_public_apk +out/Default/bin/chrome_public_apk install + +# List all classes marked as 'RetryVerificationAtRuntime' +build/android/list_class_verification_failures.py --package="org.chromium.chrome" +W 0.000s Main Skipping deobfuscation because no map file was provided. +first.failing.Class +second.failing.Class +... +``` + +"Skipping deobfuscation because no map file was provided" is a warning, since +many Android applications (including Chrome's release builds) are built with +proguard (or similar tools) to obfuscate Java classes and shrink code. Although +it's safe to ignore this warning if you don't obfuscate Java code, the script +knows how to deobfuscate classes for you (useful for `is_debug = true` or +`is_java_debug = true`): + +```shell +build/android/list_class_verification_failures.py --package="org.chromium.chrome" \ + --mapping=<path/to/file.mapping> # ex. out/Release/apks/ChromePublic.apk.mapping +android.support.design.widget.AppBarLayout +android.support.design.widget.TextInputLayout +... +``` + +Googlers can also download mappings for [official +builds](http://go/webview-official-builds). + +### Understanding the reason for the failure + +ART team also provide tooling for this. You can configure ART on a rooted device +to log all class verification failures (during installation), at which point the +cause is much clearer: + +```shell +# Enable ART logging (requires root). Note the 2 pairs of quotes! +adb root +adb shell setprop dalvik.vm.dex2oat-flags '"--runtime-arg -verbose:verifier"' + +# Restart Android services to pick up the settings +adb shell stop && adb shell start + +# Optional: clear logs which aren't relevant +adb logcat -c + +# Install the app and check for ART logs +adb install -d -r out/Default/apks/ChromePublic.apk +adb logcat | grep 'dex2oat' +... +... I dex2oat : Soft verification failures in boolean org.chromium.content.browser.selection.SelectionPopupControllerImpl.b(android.view.ActionMode, android.view.Menu) +... I dex2oat : boolean org.chromium.content.browser.selection.SelectionPopupControllerImpl.b(android.view.ActionMode, android.view.Menu): [0xF0] couldn't find method android.view.textclassifier.TextClassification.getActions ()Ljava/util/List; +... I dex2oat : boolean org.chromium.content.browser.selection.SelectionPopupControllerImpl.b(android.view.ActionMode, android.view.Menu): [0xFA] couldn't find method android.view.textclassifier.TextClassification.getActions ()Ljava/util/List; +... +``` + +*** note +**Note:** you may want to avoid `adb` wrapper scripts (ex. +`out/Default/bin/chrome_public_apk install`). These scripts cache the package +manager state to optimize away idempotent installs. However in this case, we +**do** want to trigger idempotent installs, because we want to re-trigger AOT +verification. +*** + +In the above example, `SelectionPopupControllerImpl` fails verification on Oreo +(API 26) because it refers to [`TextClassification.getActions()`][5], which was +added in Pie (API 28). If `SelectionPopupControllerImpl` is used on pre-Pie +devices, then `TextClassification.getActions()` must be out-of-lined. + +## See also + +* Bugs or questions? Contact ntfschr@chromium.org +* ART team's Google I/O talks: [2014](https://youtu.be/EBlTzQsUoOw) and later + years +* Analysis of class verification in Chrome and WebView (Google-only + [doc](http://go/class-verification-chromium-analysis)) +* Presentation on class verification in Chrome and WebView (Google-only + [slide deck](http://go/class-verification-chromium-slides)) + +[1]: https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html#jvms-5.5 +[2]: https://developer.android.com/reference/android/view/Window.html#isWideColorGamut() +[3]: https://bugs.chromium.org/p/chromium/issues/detail?id=838702 +[4]: https://developer.android.com/reference/android/os/Build.VERSION_CODES +[5]: https://developer.android.com/reference/android/view/textclassifier/TextClassification.html#getActions() diff --git a/third_party/libwebrtc/build/android/docs/coverage.md b/third_party/libwebrtc/build/android/docs/coverage.md new file mode 100644 index 0000000000..09fc29955c --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/coverage.md @@ -0,0 +1,83 @@ +# Android code coverage instructions + +These are instructions for collecting code coverage data for android +instrumentation and JUnit tests. + +[TOC] + +## How JaCoCo coverage works + +In order to use JaCoCo code coverage, we need to create build time pre-instrumented +class files and runtime **.exec** files. Then we need to process them using the +[build/android/generate_jacoco_report.py](https://source.chromium.org/chromium/chromium/src/+/main:build/android/generate_jacoco_report.py) script. + +## How to collect coverage data + +1. Use the following GN build arguments: + + ```gn + target_os = "android" + use_jacoco_coverage = true + ``` + + Now when building, pre-instrumented files will be created in the build directory. + +2. Run tests, with option `--coverage-dir <directory>`, to specify where to save + the .exec file. For example, you can run chrome JUnit tests: + `out/Debug/bin/run_chrome_junit_tests --coverage-dir /tmp/coverage`. + +3. The coverage results of JUnit and instrumentation tests will be merged + automatically if they are in the same directory. + +## How to generate coverage report + +1. Now we have generated .exec files already. We can create a JaCoCo HTML/XML/CSV + report using `generate_jacoco_report.py`, for example: + + ```shell + build/android/generate_jacoco_report.py \ + --format html \ + --output-dir /tmp/coverage_report/ \ + --coverage-dir /tmp/coverage/ \ + --sources-json-dir out/Debug/ \ + ``` + Then an index.html containing coverage info will be created in output directory: + + ``` + [INFO] Loading execution data file /tmp/coverage/testTitle.exec. + [INFO] Loading execution data file /tmp/coverage/testSelected.exec. + [INFO] Loading execution data file /tmp/coverage/testClickToSelect.exec. + [INFO] Loading execution data file /tmp/coverage/testClickToClose.exec. + [INFO] Loading execution data file /tmp/coverage/testThumbnail.exec. + [INFO] Analyzing 58 classes. + ``` + +2. For XML and CSV reports, we need to specify `--output-file` instead of `--output-dir` since + only one file will be generated as XML or CSV report. + ```shell + build/android/generate_jacoco_report.py \ + --format xml \ + --output-file /tmp/coverage_report/report.xml \ + --coverage-dir /tmp/coverage/ \ + --sources-json-dir out/Debug/ \ + ``` + + or + + ```shell + build/android/generate_jacoco_report.py \ + --format csv \ + --output-file /tmp/coverage_report/report.csv \ + --coverage-dir /tmp/coverage/ \ + --sources-json-dir out/Debug/ \ + ``` +3. If generating coverage and there are duplicate class files, as can happen + when generating coverage for downstream targets, use the + `--include-substr-filter` option to choose jars in the desired directory. Eg. + for generating coverage report for Clank internal repo + ```shell + build/android/generate_jacoco_report.py --format html \ + --output-dir /tmp/coverage_report/ --coverage-dir /tmp/coverage/ \ + --sources-json-dir out/java_coverage/ \ + --include-substr-filter obj/clank + ``` diff --git a/third_party/libwebrtc/build/android/docs/java_optimization.md b/third_party/libwebrtc/build/android/docs/java_optimization.md new file mode 100644 index 0000000000..0ba0d50358 --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/java_optimization.md @@ -0,0 +1,149 @@ +# Optimizing Java Code + +This doc describes how Java code is optimized in Chrome on Android and how to +deal with issues caused by the optimizer. For tips on how to write optimized +code, see [//docs/speed/binary_size/optimization_advice.md#optimizing-java-code](/docs/speed/binary_size/optimization_advice.md#optimizing-java-code). + +[TOC] + +## ProGuard vs R8 + +ProGuard is the original open-source tool used by many Android applications to +perform whole-program bytecode optimization. [R8](https://r8.googlesource.com/r8), +is a re-implementation that is used by Chrome (and the default for Android Studio). +The terms "ProGuard" and "R8" are used interchangeably within Chromium but +generally they're meant to refer to the tool providing Java code optimizations. + +## What does ProGuard do? + +1. Shrinking: ProGuard will remove unused code. This is especially useful + when depending on third party libraries where only a few functions are used. + +2. Obfuscation: ProGuard will rename classes/fields/methods to use shorter + names. Obfuscation is used for minification purposes only (not security). + +3. Optimization: ProGuard performs a series of optimizations to shrink code + further through various approaches (ex. inlining, outlining, class merging, + etc). + +## Build Process + +ProGuard is enabled only for release builds of Chrome because it is a slow build +step and breaks Java debugging. It can also be enabled manually via the GN arg: +```is_java_debug = false``` + +### ProGuard configuration files + +Most GN Java targets can specify ProGuard configuration files by setting the +`proguard_configs` variable. [//base/android/proguard](/base/android/proguard) +contains common flags shared by most Chrome applications. + +### GN build rules + +When `is_java_debug = false` and a target has enabled ProGuard, the `proguard` +step generates the `.dex` files for the application. The `proguard` step takes +as input a list of `.jar` files, runs R8/ProGuard on those `.jar` files, and +produces the final `.dex` file(s) that will be packaged into your `.apk` + +## Deobfuscation + +Obfuscation can be turned off for local builds while leaving ProGuard enabled +by setting `enable_proguard_obfuscation = false` in GN args. + +There are two main methods for deobfuscating Java stack traces locally: +1. Using APK wrapper scripts (stacks are automatically deobfuscated) + * `$OUT/bin/chrome_public_apk logcat` # Run adb logcat + * `$OUT/bin/chrome_public_apk run` # Launch chrome and run adb logcat + +2. Using `java_deobfuscate` + * build/android/stacktrace/java_deobfuscate.py $OUT/apks/ChromePublic.apk.mapping < logcat.txt` + * ProGuard mapping files are located beside APKs (ex. + `$OUT/apks/ChromePublic.apk` and `$OUT/apks/ChromePublic.apk.mapping`) + +Helpful links for deobfuscation: + +* [Internal bits about how mapping files are archived][proguard-site] +* [More detailed deobfuscation instructions][proguard-doc] +* [Script for deobfuscating official builds][deob-official] + +[proguard-site]: http://goto.google.com/chrome-android-proguard +[proguard-doc]: http://goto.google.com/chromejavadeobfuscation +[deob-official]: http://goto.google.com/chrome-android-official-deobfuscation + +## Debugging common failures + +ProGuard failures are often hard to debug. This section aims to outline some of +the more common errors. + +### Classes expected to be discarded + +The `-checkdiscard` directive can be used to ensure that certain items are +removed by ProGuard. A common use of `-checkdiscard` it to ensure that ProGuard +optimizations do not regress in their ability to remove code, such as code +intended only for debug builds, or generated JNI classes that are meant to be +zero-overhead abstractions. Annotating a class with +[@CheckDiscard][checkdiscard] will add a `-checkdiscard` rule automatically. + +[checkdiscard]: /base/android/java/src/org/chromium/base/annotations/CheckDiscard.java + +``` +Item void org.chromium.base.library_loader.LibraryPrefetcherJni.<init>() was not discarded. +void org.chromium.base.library_loader.LibraryPrefetcherJni.<init>() +|- is invoked from: +| void org.chromium.base.library_loader.LibraryPrefetcher.asyncPrefetchLibrariesToMemory() +... more code path lines +|- is referenced in keep rule: +| obj/chrome/android/chrome_public_apk/chrome_public_apk.resources.proguard.txt:104:1 + +Error: Discard checks failed. +``` + +Things to check + * Did you add code that is referenced by code path in the error message? + * If so, check the original class for why the `CheckDiscard` was added + originally and verify that the reason is still valid with your change (may + need git blame to do this). + * Try the extra debugging steps listed in the JNI section below. + +### JNI wrapper classes not discarded + +Proxy native methods (`@NativeMethods`) use generated wrapper classes to provide +access to native methods. We rely on ProGuard to fully optimize the generated +code so that native methods aren't a source of binary size bloat. The above +error message is an example when a JNI wrapper class wasn't discarded (notice +the name of the offending class). + * The ProGuard rule pointed to in the error message isn't helpful (just tells + us a code path that reaches the not-inlined class). + * Common causes: + * Caching the result of `ClassNameJni.get()` in a member variable. + * Passing a native wrapper method reference instead of using a lambda (i.e. + `Jni.get()::methodName` vs. `() -> Jni.get.methodName()`). + * For more debugging info, add to `base/android/proguard/chromium_code.flags`: + ``` + -whyareyounotinlining class org.chromium.base.library_loader.LibraryPrefetcherJni { + <init>(); + } + ``` + +### Duplicate classes + +``` +Type YourClassName is defined multiple times: obj/jar1.jar:YourClassName.class, obj/jar2.jar:YourClassName.class +``` + +Common causes: + * Multiple targets with overlapping `srcjar_deps`: + * Each `.srcjar` can only be depended on by a single Java target in any + given APK target. `srcjar_deps` are just a convenient way to depend on + generated files and should be treated like source files rather than + `deps`. + * Solution: Wrap the `srcjar` in an `android_library` target or have only a + single Java target depend on the `srcjar` and have other targets depend on + the containing Java target instead. + * Accidentally enabling APK level generated files for multiple targets that + share generated code (ex. Trichrome or App Bundles): + * Solution: Make sure the generated file is only added once. + +Debugging ProGuard failures isn't easy, so please message java@chromium.org +or [file a bug](crbug.com/new) with `component=Build os=Android` for any +issues related to Java code optimization. diff --git a/third_party/libwebrtc/build/android/docs/java_toolchain.md b/third_party/libwebrtc/build/android/docs/java_toolchain.md new file mode 100644 index 0000000000..4a39175472 --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/java_toolchain.md @@ -0,0 +1,284 @@ +# Chromium's Java Toolchain + +This doc aims to describe the Chrome build process that takes a set of `.java` +files and turns them into a `classes.dex` file. + +[TOC] + +## Core GN Target Types + +The following have `supports_android` and `requires_android` set to false by +default: +* `java_library()`: Compiles `.java` -> `.jar` +* `java_prebuilt()`: Imports a prebuilt `.jar` file. + +The following have `supports_android` and `requires_android` set to true. They +also have a default `jar_excluded_patterns` set (more on that later): +* `android_library()` +* `android_java_prebuilt()` + +All target names must end with "_java" so that the build system can distinguish +them from non-java targets (or [other variations](https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?rcl=ec2c17d7b4e424e060c3c7972842af87343526a1&l=20)). + +Most targets produce two separate `.jar` files: +* Device `.jar`: Used to produce `.dex.jar`, which is used on-device. +* Host `.jar`: For use on the host machine (`junit_binary` / `java_binary`). + * Host `.jar` files live in `lib.java/` so that they are archived in + builder/tester bots (which do not archive `obj/`). + +## From Source to Final Dex + +### Step 1: Create interface .jar with turbine or ijar + +For prebuilt `.jar` files, use [//third_party/ijar] to create interface `.jar` +from prebuilt `.jar`. + +For non-prebuilt targets, use [//third_party/turbine] to create interface `.jar` +from `.java` source files. Turbine is much faster than javac, and so enables +full compilation to happen more concurrently. + +What are interface jars?: + +* The contain `.class` files with all non-public symbols and function bodies + removed. +* Dependant targets use interface `.jar` files to skip having to be rebuilt + when only private implementation details change. + +[//third_party/ijar]: /third_party/ijar/README.chromium +[//third_party/turbine]: /third_party/turbine/README.chromium + +### Step 2a: Compile with javac + +This step is the only step that does not apply to prebuilt targets. + +* All `.java` files in a target are compiled by `javac` into `.class` files. + * This includes `.java` files that live within `.srcjar` files, referenced + through `srcjar_deps`. +* The `classpath` used when compiling a target is comprised of `.jar` files of + its deps. + * When deps are library targets, the Step 1 `.jar` file is used. + * When deps are prebuilt targets, the original `.jar` file is used. + * All `.jar` processing done in subsequent steps does not impact compilation + classpath. +* `.class` files are zipped into an output `.jar` file. +* There is **no support** for incremental compilation at this level. + * If one source file changes within a library, then the entire library is + recompiled. + * Prefer smaller targets to avoid slow compiles. + +### Step 2b: Compile with ErrorProne + +This step can be disabled via GN arg: `use_errorprone_java_compiler = false` + +* Concurrently with step 1a: [ErrorProne] compiles java files and checks for bug + patterns, including some [custom to Chromium][ep_plugins]. +* ErrorProne used to replace step 1a, but was changed to a concurrent step after + being identified as being slower. + +[ErrorProne]: https://errorprone.info/ +[ep_plugins]: /tools/android/errorprone_plugin/ + +### Step 3: Desugaring (Device .jar Only) + +This step happens only when targets have `supports_android = true`. It is not +applied to `.jar` files used by `junit_binary`. + +* `//third_party/bazel/desugar` converts certain Java 8 constructs, such as + lambdas and default interface methods, into constructs that are compatible + with Java 7. + +### Step 4: Instrumenting (Device .jar Only) + +This step happens only when this GN arg is set: `use_jacoco_coverage = true` + +* [Jacoco] adds instrumentation hooks to methods. + +[Jacoco]: https://www.eclemma.org/jacoco/ + +### Step 5: Filtering + +This step happens only when targets that have `jar_excluded_patterns` or +`jar_included_patterns` set (e.g. all `android_` targets). + +* Remove `.class` files that match the filters from the `.jar`. These `.class` + files are generally those that are re-created with different implementations + further on in the build process. + * E.g.: `R.class` files - a part of [Android Resources]. + * E.g.: `GEN_JNI.class` - a part of our [JNI] glue. + * E.g.: `AppHooksImpl.class` - how `chrome_java` wires up different + implementations for [non-public builds][apphooks]. + +[JNI]: /base/android/jni_generator/README.md +[Android Resources]: life_of_a_resource.md +[apphooks]: /chrome/android/java/src/org/chromium/chrome/browser/AppHooksImpl.java + +### Step 6: Per-Library Dexing + +This step happens only when targets have `supports_android = true`. + +* [d8] converts `.jar` files containing `.class` files into `.dex.jar` files + containing `classes.dex` files. +* Dexing is incremental - it will reuse dex'ed classes from a previous build if + the corresponding `.class` file is unchanged. +* These per-library `.dex.jar` files are used directly by [incremental install], + and are inputs to the Apk step when `enable_proguard = false`. + * Even when `is_java_debug = false`, many apk targets do not enable ProGuard + (e.g. unit tests). + +[d8]: https://developer.android.com/studio/command-line/d8 +[incremental install]: /build/android/incremental_install/README.md + +### Step 7: Apk / Bundle Module Compile + +* Each `android_apk` and `android_bundle_module` template has a nested + `java_library` target. The nested library includes final copies of files + stripped out by prior filtering steps. These files include: + * Final `R.java` files, created by `compile_resources.py`. + * Final `GEN_JNI.java` for [JNI glue]. + * `BuildConfig.java` and `NativeLibraries.java` (//base dependencies). + +[JNI glue]: /base/android/jni_generator/README.md + +### Step 8: Final Dexing + +This step is skipped when building using [Incremental Install]. + +When `is_java_debug = true`: +* [d8] merges all library `.dex.jar` files into a final `.mergeddex.jar`. + +When `is_java_debug = false`: +* [R8] performs whole-program optimization on all library `lib.java` `.jar` + files and outputs a final `.r8dex.jar`. + * For App Bundles, R8 creates a `.r8dex.jar` for each module. + +[Incremental Install]: /build/android/incremental_install/README.md +[R8]: https://r8.googlesource.com/r8 + +## Test APKs with apk_under_test + +Test APKs are normal APKs that contain an `<instrumentation>` tag within their +`AndroidManifest.xml`. If this tag specifies an `android:targetPackage` +different from itself, then Android will add that package's `classes.dex` to the +test APK's Java classpath when run. In GN, you can enable this behavior using +the `apk_under_test` parameter on `instrumentation_test_apk` targets. Using it +is discouraged if APKs have `proguard_enabled=true`. + +### Difference in Final Dex + +When `enable_proguard=false`: +* Any library depended on by the test APK that is also depended on by the + apk-under-test is excluded from the test APK's final dex step. + +When `enable_proguard=true`: +* Test APKs cannot make use of the apk-under-test's dex because only symbols + explicitly kept by `-keep` directives are guaranteed to exist after + ProGuarding. As a work-around, test APKs include all of the apk-under-test's + libraries directly in its own final dex such that the under-test apk's Java + code is never used (because it is entirely shadowed by the test apk's dex). + * We've found this configuration to be fragile, and are trying to [move away + from it](https://bugs.chromium.org/p/chromium/issues/detail?id=890452). + +### Difference in GEN_JNI.java +* Calling native methods using [JNI glue] requires that a `GEN_JNI.java` class + be generated that contains all native methods for an APK. There cannot be + conflicting `GEN_JNI` classes in both the test apk and the apk-under-test, so + only the apk-under-test has one generated for it. As a result this, + instrumentation test APKs that use apk-under-test cannot use native methods + that aren't already part of the apk-under-test. + +## How to Generate Java Source Code +There are two ways to go about generating source files: Annotation Processors +and custom build steps. + +### Annotation Processors +* These are run by `javac` as part of the compile step. +* They **cannot** modify the source files that they apply to. They can only + generate new sources. +* Use these when: + * an existing Annotation Processor does what you want + (E.g. Dagger, AutoService, etc.), or + * you need to understand Java types to do generation. + +### Custom Build Steps +* These use discrete build actions to generate source files. + * Some generate `.java` directly, but most generate a zip file of sources + (called a `.srcjar`) to simplify the number of inputs / outputs. +* Examples of existing templates: + * `jinja_template`: Generates source files using [Jinja]. + * `java_cpp_template`: Generates source files using the C preprocessor. + * `java_cpp_enum`: Generates `@IntDef`s based on enums within `.h` files. + * `java_cpp_strings`: Generates String constants based on strings defined in + `.cc` files. +* Custom build steps are preferred over Annotation Processors because they are + generally easier to understand, and can run in parallel with other steps + (rather than being tied to compiles). + +[Jinja]: https://palletsprojects.com/p/jinja/ + +## Static Analysis & Code Checks + +We use several tools for static analysis. + +### [ErrorProne](https://errorprone.info/) +* Runs as part of normal compilation. Controlled by GN arg: `use_errorprone_java_compiler`. +* Most useful check: + * Enforcement of `@GuardedBy` annotations. +* List of enabled / disabled checks exists [within compile_java.py](https://cs.chromium.org/chromium/src/build/android/gyp/compile_java.py?l=30) + * Many checks are currently disabled because there is work involved in fixing + violations they introduce. Please help! +* Custom checks for Chrome: + * [//tools/android/errorprone_plugin/src/org/chromium/tools/errorprone/plugin/](/tools/android/errorprone_plugin/src/org/chromium/tools/errorprone/plugin/) +* Use ErrorProne checks when you need something more sophisticated than pattern + matching. +* Checks run on the entire codebase, not only on changed lines. +* Does not run when `chromium_code = false` (e.g. for //third_party). + +### [Android Lint](https://developer.android.com/studio/write/lint) +* Runs as part of normal compilation. Controlled by GN arg: `disable_android_lint` +* Most useful check: + * Enforcing `@TargetApi` annotations (ensure you don't call a function that + does not exist on all versions of Android unless guarded by an version + check). +* List of disabled checks: + * [//build/android/lint/suppressions.xml](/build/android/lint/suppressions.xml) +* Custom lint checks [are possible](lint_plugins), but we don't have any. +* Checks run on the entire codebase, not only on changed lines. +* Does not run when `chromium_code = false` (e.g. for //third_party). + +[lint_plugins]: http://tools.android.com/tips/lint-custom-rules + +### [Bytecode Processor](/build/android/bytecode/) +* Performs a single check: + * That target `deps` are not missing any entries. + * In other words: Enforces that targets do not rely on indirect dependencies + to populate their classpath. +* Checks run on the entire codebase, not only on changed lines. + +### [PRESUBMIT.py](/PRESUBMIT.py): +* Checks for banned patterns via `_BANNED_JAVA_FUNCTIONS`. + * (These should likely be moved to checkstyle). +* Checks for a random set of things in `ChecksAndroidSpecificOnUpload()`. + * Including running Checkstyle. + * (Some of these other checks should likely also be moved to checkstyle). +* Checks run only on changed lines. + +### [Checkstyle](https://checkstyle.sourceforge.io/) +* Checks Java style rules that are not covered by clang-format. + * E.g.: Unused imports and naming conventions. +* Allows custom checks to be added via XML. Here [is ours]. +* Preferred over adding checks directly in PRESUBMIT.py because the tool + understands `@SuppressWarnings` annotations. +* Checks run only on changed lines. + +[is ours]: /tools/android/checkstyle/chromium-style-5.0.xml + +### [clang-format](https://clang.llvm.org/docs/ClangFormat.html) +* Formats `.java` files via `git cl format`. +* Can be toggle on/off with code comments. + ```java + // clang-format off + ... non-formatted code here ... + // clang-format on + ``` +* Does not work great for multiple annotations or on some lambda expressions, + but is generally agreed it is better than not having it at all. diff --git a/third_party/libwebrtc/build/android/docs/life_of_a_resource.md b/third_party/libwebrtc/build/android/docs/life_of_a_resource.md new file mode 100644 index 0000000000..5e46ef66af --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/life_of_a_resource.md @@ -0,0 +1,289 @@ +# Life of an Android Resource + +[TOC] + +## Overview + +This document describes how [Android Resources][android resources] +are built in Chromium's build system. It does not mention native resources +which are [processed differently][native resources]. + +[android resources]: https://developer.android.com/guide/topics/resources/providing-resources +[native resources]: https://www.chromium.org/developers/tools-we-use-in-chromium/grit/grit-users-guide + +The steps consume the following files as inputs: +* `AndroidManifest.xml` + * Including `AndroidManifest.xml` files from libraries, which get merged + together +* res/ directories + +The steps produce the following intermediate files: +* `R.srcjar` (contains `R.java` files) +* `R.txt` +* `.resources.zip` + +The steps produce the following files within an `.apk`: +* `AndroidManifest.xml` (a binary xml file) +* `resources.arsc` (contains all values and configuration metadata) +* `res/**` (drawables and layouts) +* `classes.dex` (just a small portion of classes from generated `R.java` files) + + +## The Build Steps + +Whenever you try to compile an apk or library target, resources go through the +following steps: + +### 1. Constructs .build\_config files: + +Inputs: +* GN target metadata +* Other `.build_config.json` files + +Outputs: +* Target-specific `.build_config.json` file + +`write_build_config.py` is run to record target metadata needed by future steps. +For more details, see [build_config.md](build_config.md). + + +### 2. Prepares resources: + +Inputs: +* Target-specific `.build_config.json` file +* Files listed as `sources` + +Outputs: +* Target-specific `resources.zip` (contains all resources listed in `sources`). +* Target-specific `R.txt` (list of all resources, including dependencies). + +`prepare_resources.py` zips up the target-specific resource files and generates +`R.txt`. No optimizations, crunching, etc are done on the resources. + +**The following steps apply only to apk & bundle targets (not to library +targets).** + +### 3. Create target-specific R.java files + +Inputs: +* `R.txt` from dependencies. + +Outputs: +* Target-specific (placeholder) `R.java` file. + +A target-specific `R.java` is generated for each `android_library()` target that +sets `resources_package`. Resource IDs are not known at this phase, so all +values are set as placeholders. This copy of `R` classes are discarded and +replaced with new copies at step 4. + +Example placeholder R.java file: +```java +package org.chromium.mypackage; + +public final class R { + public static class anim { + public static int abc_fade_in = 0; + public static int abc_fade_out = 0; + ... + } + ... +} +``` + +### 4. Finalizes apk resources: + +Inputs: +* Target-specific `.build_config.json` file +* Dependencies' `R.txt` files +* Dependencies' `resources.zip` files + +Output: +* Packaged `resources zip` (named `foo.ap_`) containing: + * `AndroidManifest.xml` (as binary xml) + * `resources.arsc` + * `res/**` +* Final `R.txt` + * Contains a list of resources and their ids (including of dependencies). +* Final `R.java` files + * See [What are `R.java` files and how are they generated]( + #how-r_java-files-are-generated) + + +#### 4(a). Compiles resources: + +For each library / resources target your apk depends on, the following happens: +* Use a regex (defined in the apk target) to remove select resources (optional). +* Convert png images to webp for binary size (optional). +* Move drawables in mdpi to non-mdpi directory ([why?](http://crbug.com/289843)) +* Use `aapt2 compile` to compile xml resources to binary xml (references to + other resources will now use the id rather than the name for faster lookup at + runtime). +* `aapt2 compile` adds headers/metadata to 9-patch images about which parts of + the image are stretchable vs static. +* `aapt2 compile` outputs a zip with the compiled resources (one for each + dependency). + + +#### 4(b). Links resources: + +After each dependency is compiled into an intermediate `.zip`, all those zips +are linked by the `aapt2 link` command which does the following: +* Use the order of dependencies supplied so that some resources clober each + other. +* Compile the `AndroidManifest.xml` to binary xml (references to resources are + now using ids rather than the string names) +* Create a `resources.arsc` file that has the name and values of string + resources as well as the name and path of non-string resources (ie. layouts + and drawables). +* Combine the compiled resources into one packaged resources apk (a zip file + with an `.ap_` extension) that has all the resources related files. + + +#### 4(c). Optimizes resources: + +Targets can opt into the following optimizations: +1) Resource name collapsing: Maps all resources to the same name. Access to + resources via `Resources.getIdentifier()` no longer work unless resources are + [allowlisted](#adding-resources-to-the-allowlist). +2) Resource filename obfuscation: Renames resource file paths from e.g.: + `res/drawable/something.png` to `res/a`. Rename mapping is stored alongside + APKs / bundles in a `.pathmap` file. Renames are based on hashes, and so are + stable between builds (unless a new hash collision occurs). +3) Unused resource removal: Referenced resources are extracted from the + optimized `.dex` and `AndroidManifest.xml`. Resources that are directly or + indirectly used by these files are removed. + +## App Bundles and Modules: + +Processing resources for bundles and modules is slightly different. Each module +has its resources compiled and linked separately (ie: it goes through the +entire process for each module). The modules are then combined to form a +bundle. Moreover, during "Finalizing the apk resources" step, bundle modules +produce a `resources.proto` file instead of a `resources.arsc` file. + +Resources in a dynamic feature module may reference resources in the base +module. During the link step for feature module resources, the linked resources +of the base module are passed in. However, linking against resources currently +works only with `resources.arsc` format. Thus, when building the base module, +resources are compiled as both `resources.arsc` and `resources.proto`. + +## Debugging resource related errors when resource names are obfuscated + +An example message from a stacktrace could be something like this: +``` +java.lang.IllegalStateException: Could not find CoordinatorLayout descendant +view with id org.chromium.chrome:id/0_resource_name_obfuscated to anchor view +android.view.ViewStub{be192d5 G.E...... ......I. 0,0-0,0 #7f0a02ad +app:id/0_resource_name_obfuscated} +``` + +`0_resource_name_obfuscated` is the resource name for all resources that had +their name obfuscated/stripped during the optimize resources step. To help with +debugging, the `R.txt` file is archived. The `R.txt` file contains a mapping +from resource ids to resource names and can be used to get the original resource +name from the id. In the above message the id is `0x7f0a02ad`. + +For local builds, `R.txt` files are output in the `out/*/apks` directory. + +For official builds, Googlers can get archived `R.txt` files next to archived +apks. + +### Adding resources to the allowlist + +If a resource is accessed via `getIdentifier()` it needs to be allowed by an +aapt2 resources config file. The config file looks like this: + +``` +<resource type>/<resource name>#no_obfuscate +``` +eg: +``` +string/app_name#no_obfuscate +id/toolbar#no_obfuscate +``` + +The aapt2 config file is passed to the ninja target through the +`resources_config_paths` variable. To add a resource to the allowlist, check +where the config is for your target and add a new line for your resource. If +none exist, create a new config file and pass its path in your target. + +### Webview resource ids + +The first two bytes of a resource id is the package id. For regular apks, this +is `0x7f`. However, Webview is a shared library which gets loaded into other +apks. The package id for webview resources is assigned dynamically at runtime. +When webview is loaded it calls this [R file's][Base Module R.java File] +`onResourcesLoaded()` function to have the correct package id. When +deobfuscating webview resource ids, disregard the first two bytes in the id when +looking it up in the `R.txt` file. + +Monochrome, when loaded as webview, rewrites the package ids of resources used +by the webview portion to the correct value at runtime, otherwise, its resources +have package id `0x7f` when run as a regular apk. + +[Base Module R.java File]: https://cs.chromium.org/chromium/src/out/android-Debug/gen/android_webview/system_webview_apk/generated_java/gen/base_module/R.java + +## How R.java files are generated + +`R.java` contain a set of nested static classes, each with static fields +containing ids. These ids are used in java code to reference resources in +the apk. + +There are three types of `R.java` files in Chrome. +1. Root / Base Module `R.java` Files +2. DFM `R.java` Files +3. Per-Library `R.java` Files + +### Root / Base Module `R.java` Files +Contain base android resources. All `R.java` files can access base module +resources through inheritance. + +Example Root / Base Module `R.java` File: +```java +package gen.base_module; + +public final class R { + public static class anim { + public static final int abc_fade_in = 0x7f010000; + public static final int abc_fade_out = 0x7f010001; + public static final int abc_slide_in_top = 0x7f010007; + } + public static class animator { + public static final int design_appbar_state_list_animator = 0x7f020000; + } +} +``` + +### DFM `R.java` Files +Extend base module root `R.java` files. This allows DFMs to access their own +resources as well as the base module's resources. + +Example DFM Root `R.java` File +```java +package gen.vr_module; + +public final class R { + public static class anim extends gen.base_module.R.anim { + } + public static class animator extends gen.base_module.R.animator { + public static final int design_appbar_state_list_animator = 0x7f030000; + } +} +``` + +### Per-Library `R.java` Files +Generated for each `android_library()` target that sets `resources_package`. +First a placeholder copy is generated in the `android_library()` step, and then +a final copy is created during finalization. + +Example final per-library `R.java`: +```java +package org.chromium.chrome.vr; + +public final class R { + public static final class anim extends + gen.vr_module.R.anim {} + public static final class animator extends + gen.vr_module.R.animator {} +} +``` diff --git a/third_party/libwebrtc/build/android/docs/lint.md b/third_party/libwebrtc/build/android/docs/lint.md new file mode 100644 index 0000000000..67e2f8bf3e --- /dev/null +++ b/third_party/libwebrtc/build/android/docs/lint.md @@ -0,0 +1,140 @@ +# Lint + +Android's [**lint**](https://developer.android.com/tools/help/lint.html) is a +static analysis tool that Chromium uses to catch possible issues in Java code. + +This is a list of [**checks**](http://tools.android.com/tips/lint-checks) that +you might encounter. + +[TOC] + +## How Chromium uses lint + +Chromium only runs lint on apk or bundle targets that explicitly set +`enable_lint = true`. Some example targets that have this set are: + + - `//chrome/android:monochrome_public_bundle` + - `//android_webview/support_library/boundary_interfaces:boundary_interface_example_apk` + - `//remoting/android:remoting_apk` + +## My code has a lint error + +If lint reports an issue in your code, there are several possible remedies. +In descending order of preference: + +### Fix it + +While this isn't always the right response, fixing the lint error or warning +should be the default. + +### Suppress it locally + +Java provides an annotation, +[`@SuppressWarnings`](https://developer.android.com/reference/java/lang/SuppressWarnings), +that tells lint to ignore the annotated element. It can be used on classes, +constructors, methods, parameters, fields, or local variables, though usage in +Chromium is typically limited to the first three. You do not need to import it +since it is in the `java.lang` package. + +Like many suppression annotations, `@SuppressWarnings` takes a value that tells +**lint** what to ignore. It can be a single `String`: + +```java +@SuppressWarnings("NewApi") +public void foo() { + a.methodThatRequiresHighSdkLevel(); +} +``` + +It can also be a list of `String`s: + +```java +@SuppressWarnings({ + "NewApi", + "UseSparseArrays" + }) +public Map<Integer, FakeObject> bar() { + Map<Integer, FakeObject> shouldBeASparseArray = new HashMap<Integer, FakeObject>(); + another.methodThatRequiresHighSdkLevel(shouldBeASparseArray); + return shouldBeASparseArray; +} +``` + +For resource xml files you can use `tools:ignore`: + +```xml +<?xml version="1.0" encoding="utf-8"?> +<resources xmlns:tools="http://schemas.android.com/tools"> + <!-- TODO(crbug/###): remove tools:ignore once these colors are used --> + <color name="hi" tools:ignore="NewApi,UnusedResources">@color/unused</color> +</resources> +``` + +The examples above are the recommended ways of suppressing lint warnings. + +### Suppress it in a `lint-suppressions.xml` file + +**lint** can be given a per-target XML configuration file containing warnings or +errors that should be ignored. Each target defines its own configuration file +via the `lint_suppressions_file` gn variable. It is usually defined near its +`enable_lint` gn variable. + +These suppressions files should only be used for temporarily ignoring warnings +that are too hard (or not possible) to suppress locally, and permanently +ignoring warnings only for this target. To permanently ignore a warning for all +targets, add the warning to the `_DISABLED_ALWAYS` list in +[build/android/gyp/lint.py](https://source.chromium.org/chromium/chromium/src/+/main:build/android/gyp/lint.py). +Disabling globally makes lint a bit faster. + +The exception to the above rule is for warnings that affect multiple languages. +Feel free to suppress those in lint-suppressions.xml files since it is not +practical to suppress them in each language file and it is a lot of extra bloat +to list out every language for every violation in lint-baseline.xml files. + +Here is an example of how to structure a suppressions XML file: + +```xml +<?xml version="1.0" encoding="utf-8" ?> +<lint> + <!-- Chrome is a system app. --> + <issue id="ProtectedPermissions" severity="ignore"/> + <issue id="UnusedResources"> + <!-- 1 raw resources are accessed by URL in various places. --> + <ignore regexp="gen/remoting/android/.*/res/raw/credits.*"/> + <!-- TODO(crbug.com/###): Remove the following line. --> + <ignore regexp="The resource `R.string.soon_to_be_used` appears to be unused"/> + </issue> +</lint> +``` + +## What are `lint-baseline.xml` files for? + +Baseline files are to help us introduce new lint warnings and errors without +blocking on fixing all our existing code that violate these new errors. Since +they are generated files, they should **not** be used to suppress lint warnings. +One of the approaches above should be used instead. Eventually all the errors in +baseline files should be either fixed or ignored permanently. + +The following are some common scenarios where you may need to update baseline +files. + +### I updated `cmdline-tools` and now there are tons of new errors! + +This happens every time lint is updated, since lint is provided by +`cmdline-tools`. + +Baseline files are defined via the `lint_baseline_file` gn variable. It is +usually defined near a target's `enable_lint` gn variable. To regenerate the +baseline file, delete it and re-run the lint target. The command will fail, but +the baseline file will have been generated. + +This may need to be repeated for all targets that have set `enable_lint = true`, +including downstream targets. Downstream baseline files should be updated and +first to avoid build breakages. Each target has its own `lint_baseline_file` +defined and so all these files can be removed and regenerated as needed. + +### I updated `library X` and now there are tons of new errors! + +This is usually because `library X`'s aar contains custom lint checks and/or +custom annotation definition. Follow the same procedure as updates to +`cmdline-tools`. |