From 43a97878ce14b72f0981164f87f2e35e14151312 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 7 Apr 2024 11:22:09 +0200 Subject: Adding upstream version 110.0.1. Signed-off-by: Daniel Baumann --- toolkit/mozapps/update/docs/BackgroundUpdates.rst | 224 +++++++++++++++++++++ .../update/docs/MaintenanceServiceTests.rst | 103 ++++++++++ .../update/docs/SettingUpAnUpdateServer.rst | 218 ++++++++++++++++++++ toolkit/mozapps/update/docs/index.rst | 10 + 4 files changed, 555 insertions(+) create mode 100644 toolkit/mozapps/update/docs/BackgroundUpdates.rst create mode 100644 toolkit/mozapps/update/docs/MaintenanceServiceTests.rst create mode 100644 toolkit/mozapps/update/docs/SettingUpAnUpdateServer.rst create mode 100644 toolkit/mozapps/update/docs/index.rst (limited to 'toolkit/mozapps/update/docs') diff --git a/toolkit/mozapps/update/docs/BackgroundUpdates.rst b/toolkit/mozapps/update/docs/BackgroundUpdates.rst new file mode 100644 index 0000000000..c692f4958a --- /dev/null +++ b/toolkit/mozapps/update/docs/BackgroundUpdates.rst @@ -0,0 +1,224 @@ +================== +Background Updates +================== + +The purpose of the background update system is to perform application updates +during times when Firefox is not running. It was originally implemented in `bug +1689520 `__. + +The system has three main tasks it needs to handle: + +1. :ref:`Determining whether background updates are possible ` + +2. :ref:`Scheduling background tasks ` + +3. :ref:`Checking for updates ` + +Architecturally, the background task is an instance of Firefox running in a +special background mode, not a separate tool. This allows it to leverage +existing functionality in Firefox, including the existing update code, but also +keep acceptable performance characteristics for a background task by controlling +and limiting the parts of Firefox that are loaded. + +Everything in this document applies only to Microsoft Windows systems. In the +future, we would like to extend background update support to macOS (see `bug +1653435 `__), however +support for Linux and other Unix variants is not planned due to the variation in +OS-level scheduling affordances across distributions/configurations. + +Lifecycle +========= + +When background updates are possible, the background update task will be invoked +every 7 hours (by default). The first invocation initiates an update download +which proceeds after the task exits using Windows BITS. The second invocation +prepares and stages the update. The third invocation installs the update as it +starts up, and then checks for a newer update, possibly initiating another +update download. The cycle then continues. If the user launches Firefox at any +point in this process, it will take over. If the background update task is +invoked while Firefox proper is running, the task exits without doing any work. +In the future, the second invocation will stage and then restart to finish +installing the update, rather than waiting for the third invocation (see `bug +1704855 `__). + +.. _background-updates-determining: + +Determining whether background updates are possible +=================================================== + +Configuration +------------- + +Updating Firefox, by definition, is an operation that applies to a Firefox +installation. However, Firefox configuration is generally done via preference +values and other files which are stored in a Firefox profile, and in general +profiles do not correspond 1:1 with installations. This raises the question of +how the configuration for something like the background updater should be +managed. We deal with this question in two different ways. + +There are two main preferences specifically relevant to updates. Those +are ``app.update.auto``, which controls whether updates should be +downloaded automatically at all, even if Firefox is running, and +``app.update.background.enabled``, to specifically control whether to +use the background update system. We store these preferences in the +update root directory, which is located in a per-installation location +outside of any profile. Any profile loaded in that installation can +observe and control these settings. + +But there are some other pieces of state which absolutely must come from a +profile, such as the telemetry client ID and logging level settings (see +`BackgroundTasksUtils.jsm `__). + +This means that, in addition to our per-installation prefs, we also need +to be able to identify and load a profile. To do that, we leverage `the profile +service `__ +to determine what the default profile for the installation would be if we were +running a normal browser session, and the background updater always uses it. + +Criteria +-------- + +The default profile must satisfy several conditions in order for background +updates to be scheduled. None of these confounding factors are present in fully +default configurations, but some are relatively common. See +`BackgroundUpdate.REASON `__ +for all the details. + +In order for the background task to be scheduled: + +- The per-installation ``app.update.background.enabled`` pref must be + true + +- The per-installation ``app.update.auto`` pref must be true (the + default) + +- The installation must have been created by an installer executable and not by + manually extracting an archive file + +- The current OS user must be capable of updating the installation based on its + file system permissions, either by having permission to write to application + files directly or by using the Mozilla Maintenance Service (which also + requires that it be installed and enabled, as it is by default) + +- BITS must be enabled via ``app.update.BITS.enabled`` (the default) + +- Firefox proxy server settings must not be configured (the default) + +- ``app.update.langpack.enabled`` must be false, or otherwise there must be no + langpacks installed. Background tasks cannot update addons such as langpacks, + because they are installed into a profile, and langpacks that are not + precisely matched with the version of Firefox that is installed can cause + YSOD failures (see `bug 1647443 `__), + so background updating in the presence of langpacks is too risky. + +If any per-installation prefs are changed while the default profile is not +running, the background update task will witness the changed prefs during its +next scheduled run, and exit if appropriate. The background task will not be +unscheduled at that point; that is delayed until a browser session is run with +the default profile (it should be possible for the background update task to +unschedule itself, but currently we prefer the simplicity of handling all +scheduling tasks from a single location). + +In the extremely unusual case when prefs belonging to the default profile are +modified outside of Firefox (with a text editor, say), then the +background task will generally pick up those changes with no action needed, +because it will fish the changed settings directly from the profile. + +.. _background-updates-scheduling: + +Scheduling background tasks +=========================== + +We use OS-level scheduling mechanisms to schedule the command ``firefox +--backgroundtask backgroundupdate`` to run on a particular cadence. This cadence +is controlled by the ``app.update.background.interval`` preference, which +defaults to 7 hours. + +On Windows, we use the `Task Scheduler +API `__; +on macOS this will use +`launchd `__. +For platform-specific scheduling details, see the +`TaskScheduler.jsm `__ +module. + +These background tasks are scheduled per OS user and run with that user’s +permissions. No additional privileges are requested or needed, regardless of the +user account's status, because we have already verified that either the user has +all the permissions they need or that the Maintenance Service can be used. + +Scheduling is done from within Firefox (or a background task) itself. To +reduce shared state, only the *default* Firefox profile will interact +with the OS-level task scheduling mechanism. + +.. _background-updates-checking: + +Checking for updates +==================== + +After verifying all the preconditions and exiting immediately if any do not +hold, the ``backgroundupdate`` task then verifies that it is the only Firefox +instance running (as determined by a multi-instance lock, see `bug +1553982 `__), since +otherwise it would be unsafe to continue performing any update work. + +The task then fishes configuration settings from the default profile, namely: + +- A subset of update specific preferences, such as ``app.update.log`` + +- Data reporting preferences, to ensure the task respects the user’s choices + +- The (legacy) Telemetry client ID, so that background update Telemetry + can be correlated with other Firefox Telemetry + +The background task creates a distinct profile for itself to load, because a +profile must be present in order for most of the Firefox code that it relies on +to function. This distinct profile is non-ephemeral, i.e., persistent, but not +visible to users: see `bug 1775132 +`__ + +After setting up this profile and reading all the configuration we need +into it, the regular +`UpdateService.jsm `__ +check process is initiated. To the greatest extent possible, this process is +identical to what happens during any regular browsing session. + +Specific topics +=============== + +User interface +-------------- + +The background update task must not produce any user-visible interface. If it +did, whatever appeared would be \*disembodied\*, unconnected to any usage of +Firefox itself and appearing to a user as a weird, scary popup that came out of +nowhere. To this end, we disable all UI within the updater when invoking +from a background task. See `bug +1696276 `__. + +This point also means that we cannot prompt for user elevation (on Windows this +would mean a UAC prompt) from within the task, so we have to make very sure that +we will be able to perform an update without needing to elevate. By default on +Windows we are able to do this because of the presence of the Maintenance +Service, but it may be disabled or not installed, so we still have to check. + +Staging +------- + +The background update task will follow the update staging setting in the user’s +default profile. The default setting is to enable staging, so most users will +have it. In the future, background update tasks will recognize when an update +has been staged and try to restart to finalize the staged update (see `bug +1704855 `__). Background +tasks cannot finalize a staged update in all cases however; for one example, see +`bug 1695797 `__, where we +ensure that background tasks do not finalize a staged update while other +instances of the application are running. + +Staging is enabled by default because it provides a marked improvement in +startup time for a browsing session. Without staging, browser startup following +retrieving an update would be blocked on extracting the update archive and +patching each individual application file. Staging does all of that in advance, +so that all that needs to be done to complete an update (and therefore all that +needs to be done during the startup path), is to move the already patched (that +is, staged) files into place, a much faster and less resource intensive job. diff --git a/toolkit/mozapps/update/docs/MaintenanceServiceTests.rst b/toolkit/mozapps/update/docs/MaintenanceServiceTests.rst new file mode 100644 index 0000000000..b954b572f8 --- /dev/null +++ b/toolkit/mozapps/update/docs/MaintenanceServiceTests.rst @@ -0,0 +1,103 @@ +Maintenance Service Tests +========================= + +The automated tests for the Mozilla Maintenance Service are a bit tricky. They +are located in ``toolkit/mozapps/update/tests/unit_service_updater/`` and they +allow for automated testing of application update using the Service. + +In automation, everything gets signed and the tests properly check that things +like certificate verification work. If, however, you want to run the tests +locally, the MAR and the updater binary will not be signed. Thus, the +verification of those certificates will fail and the tests will not run +properly. + +We don't want these tests to just always fail if someone runs large amounts of +tests locally. To avoid this, the tests basically just unconditionally pass if +you run them locally and don't take the time to set them up properly. + +If you want them to actually run locally, you will need to set up your +environment properly. + +Setting Up to Run the Tests Locally +----------------------------------- + +In order to run the service tests locally, we have to bypass much of the +certificate verification. Thus, this method may not be helpful if you need to +test that feature in particular. + +Add Fallback Key to Registry +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, you will need to add the fallback key to the registry. Normally, the +Firefox installer writes some certificate information to a registry key in an +installation-specific place. In testing, however, we can't get the permissions +to write this key, nor can we have the test environment predict every possible +installation directory that we might test with. To get around this problem, if +the Service can't find the installation-specific key, it will check a static +fallback location. + +The easiest way to correctly set up the fallback key is to copy the text below +into a ``.reg`` file and then double click it in the file browser to merge it +into the registry. + +.. code:: + + Windows Registry Editor Version 5.00 + + [HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\MaintenanceService\3932ecacee736d366d6436db0f55bce4] + + [HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\MaintenanceService\3932ecacee736d366d6436db0f55bce4\0] + "issuer"="DigiCert SHA2 Assured ID Code Signing CA" + "name"="Mozilla Corporation" + + [HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\MaintenanceService\3932ecacee736d366d6436db0f55bce4\1] + "issuer"="DigiCert Assured ID Code Signing CA-1" + "name"="Mozilla Corporation" + + [HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\MaintenanceService\3932ecacee736d366d6436db0f55bce4\2] + "issuer"="Mozilla Fake CA" + "name"="Mozilla Fake SPC" + +Build without Certificate Verification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To disable certificate verification, add this build flag to your ``mozconfig`` +file: + +.. code:: + + ac_add_options --enable-unverified-updates + +You will need to rebuild for this to take effect. + +Copy the Maintenance Service Binary +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This step will assume that you already have the Maintenance Service installed. + +First, move the existing Maintenance Service binary out of the way. It will +initially be located at +``C:\Program Files (x86)\Mozilla Maintenance Service\maintenanceservice.exe``. +An easy way to do this is to append ``.bak`` to its name. You should probably +restore your original Maintenance Service binary when you are done testing. + +Now, copy the Maintenance Service binary that you built into that directory. +It will be located at ``\dist\bin\maintenanceservice.exe``. + +If you make changes to the Maintenance Service and rebuild, you will have to +repeat this step. + +Running the Tests +----------------- + +You should now be ready run a service test: + +.. code:: + + ./mach test toolkit/mozapps/update/tests/unit_service_updater/ + +Or run all of them: + +.. code:: + + ./mach test toolkit/mozapps/update/tests/unit_service_updater diff --git a/toolkit/mozapps/update/docs/SettingUpAnUpdateServer.rst b/toolkit/mozapps/update/docs/SettingUpAnUpdateServer.rst new file mode 100644 index 0000000000..cecc81a1d9 --- /dev/null +++ b/toolkit/mozapps/update/docs/SettingUpAnUpdateServer.rst @@ -0,0 +1,218 @@ +Setting Up An Update Server +=========================== + +The goal of this document is to provide instructions for installing a +locally-served Firefox update. + +Obtaining an update MAR +----------------------- + +Updates are served as MAR files. There are two common ways to obtain a +MAR to use: download a prebuilt one, or build one yourself. + +Downloading a MAR +~~~~~~~~~~~~~~~~~ + +Prebuilt Nightly MARs can be found +`here `__ on +archive.mozilla.org. Be sure that you use the one that matches your +machine's configuration. For example, if you want the Nightly MAR from +2019-09-17 for a 64 bit Windows machine, you probably want the MAR +located at +https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-17-09-36-29-mozilla-central/firefox-71.0a1.en-US.win64.complete.mar. + +Prebuilt MARs for release and beta can be found +`here `__. Beta +builds are those with a ``b`` in the version string. After locating the +desired version, the MARs will be in the ``update`` directory. You want +to use the MAR labelled ``complete``, not a partial MAR. Here is an +example of an appropriate MAR file to use: +https://archive.mozilla.org/pub/firefox/releases/69.0b9/update/win64/en-US/firefox-69.0b9.complete.mar. + +Building a MAR +~~~~~~~~~~~~~~ + +Building a MAR locally is more complicated. Part of the problem is that +MARs are signed by Mozilla and so you cannot really build an "official" +MAR yourself. This is a security measure designed to prevent anyone from +serving malicious updates. If you want to use a locally-built MAR, the +copy of Firefox being updated will need to be built to allow un-signed +MARs. See :ref:`Building Firefox ` +for more information on building Firefox locally. In order to use a locally +built MAR, you will need to put this line in the mozconfig file in root of the +build directory (create it if it does not exist): + +.. code:: + + ac_add_options --enable-unverified-updates + +Firefox should otherwise be built normally. After building, you may want +to copy the installation of Firefox elsewhere. If you update the +installation without moving it, attempts at further incremental builds +will not work properly, and a clobber will be needed when building next. +To move the installation, first call ``./mach package``, then copy +``/dist/firefox`` elsewhere. The copied directory will be your +install directory. + +If you are running Windows and want the `Mozilla Maintenance +Service `__ +to be used, there are a few additional steps to be taken here. First, +the maintenance service needs to be "installed". Most likely, a +different maintenance service is already installed, probably at +``C:\Program Files (x86)\Mozilla Maintenance Service\maintenanceservice.exe``. +Backup that file to another location and replace it with +``/dist/bin/maintenanceservice.exe``. Don't forget to restore +the backup when you are done. Next, you will need to change the +permissions on the Firefox install directory that you created. Both that +directory and its parent directory should have permissions preventing +the current user from writing to it. + +Now that you have a build of Firefox capable of using a locally-built +MAR, it's time to build the MAR. First, build Firefox the way you want +it to be after updating. If you want it to be the same before and after +updating, this step is unnecessary and you can use the same build that +you used to create the installation. Then run these commands, +substituting ````, ````, ```` and +```` appropriately: + +.. code:: bash + + $ ./mach package + $ touch "/dist/firefox/precomplete" + $ MAR="/dist/host/bin/mar.exe" MOZ_PRODUCT_VERSION= MAR_CHANNEL_ID= ./tools/update-packaging/make_full_update.sh "/dist/firefox" + +For macOS you should use these commands: + +.. code:: bash + + $ ./mach package + $ touch "/dist/firefox/Firefox.app/Contents/Resources/precomplete" + $ MAR="/dist/host/bin/mar.exe" MOZ_PRODUCT_VERSION= MAR_CHANNEL_ID= ./tools/update-packaging/make_full_update.sh "/dist/firefox/Firefox.app" + +For a local build, ```` can be ``default``, and ```` +can be the value from ``browser/config/version.txt`` (or something +arbitrarily large like ``2000.0a1``). + +.. container:: blockIndicator note + + Note: It can be a bit tricky to get the ``make_full_update.sh`` + script to accept paths with spaces. + +Serving the update +------------------ + +Preparing the update files +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, create the directory that updates will be served from and put the +MAR file in it. Then, create a file within called ``update.xml`` with +these contents, replacing ````, ```` and ```` with +the MAR's filename, its sha512 hash, and its file size in bytes. + +:: + + + + + + + + +If you've downloaded the MAR you're using, you'll find the sha512 value +in a file called SHA512SUMS in the root of the release directory on +archive.mozilla.org for a release or beta build (you'll have to search +it for the file name of your MAR, since it includes the sha512 for every +file that's part of that release), and for a nightly build you'll find a +file with a .checksums extension adjacent to your MAR that contains that +information (for instance, for the MAR file at +https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-17-09-36-29-mozilla-central/firefox-71.0a1.en-US.win64.complete.mar, +the file +https://archive.mozilla.org/pub/firefox/nightly/2019/09/2019-09-17-09-36-29-mozilla-central/firefox-71.0a1.en-US.win64.checksums +contains the sha512 for that file as well as for all the other win64 +files that are part of that nightly release). + +If you've built your own MAR, you can obtain its sha512 checksum by +running the following command, which should work in Linux, macOS, or +Windows in the MozillaBuild environment: + +.. code:: + + shasum --algorithm 512 + +On Windows, you can get the exact file size in bytes for your MAR by +right clicking on it in the file explorer and selecting Properties. +You'll find the correct size in bytes at the end of the line that begins +"Size", **not** the one that begins "Size on disk". Be sure to remove +the commas when you paste this number into the XML file. + +On macOS, you can get the exact size of your MAR by running the command: + +.. code:: + + stat -f%z + +Or on Linux, the same command would be: + +.. code:: + + stat --format "%s" + +Starting your update server +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Now, start an update server to serve the update files on port 8000. An +easy way to do this is with Python. Remember to navigate to the correct +directory before starting the server. This is the Python2 command: + +.. code:: bash + + $ python -m SimpleHTTPServer 8000 + +or, this is the Python3 command: + +.. code:: bash + + $ python3 -m http.server 8000 + +.. container:: blockIndicator note + + If you aren't sure that you started the server correctly, try using a + web browser to navigate to ``http://127.0.0.1:8000/update.xml`` and + make sure that you get the XML file you created earlier. + +Installing the update +--------------------- + +You may want to start by deleting any pending updates to ensure that no +previously found updates interfere with installing the desired update. +You can use this command with Firefox's browser console to determine the +update directory: + +.. code:: + + ChromeUtils.importESModule("resource://gre/modules/FileUtils.sys.mjs").FileUtils.getDir("UpdRootD", [], false).path + +Once you have determined the update directory, close Firefox, browse to +the directory and remove the subdirectory called ``updates``. + +| Next, you need to change the update URL to point to the local XML + file. This can be done most reliably with an enterprise policy. The + policy file location depends on the operating system you are using. +| Windows/Linux: ``/distribution/policies.json`` +| macOS: ``/Contents/Resources/distribution/policies.json`` +| Create the ``distribution`` directory, if necessary, and put this in + ``policies.json``: + +:: + + { + "policies": { + "AppUpdateURL": "http://127.0.0.1:8000/update.xml" + } + } + +Now you are ready to update! Launch Firefox out of its installation +directory and navigate to the Update section ``about:preferences``. You +should see it downloading the update to the update directory. Since the +transfer is entirely local this should finish quickly, and a "Restart to +Update" button should appear. Click it to restart and apply the update. diff --git a/toolkit/mozapps/update/docs/index.rst b/toolkit/mozapps/update/docs/index.rst new file mode 100644 index 0000000000..d6cc9f5e64 --- /dev/null +++ b/toolkit/mozapps/update/docs/index.rst @@ -0,0 +1,10 @@ +================== +Application Update +================== + +.. toctree:: + :maxdepth: 1 + + BackgroundUpdates + MaintenanceServiceTests + SettingUpAnUpdateServer -- cgit v1.2.3