Adding upstream version 86.0.1.upstream/86.0.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

6 files changed, 440 insertions, 0 deletions

diff --git a/toolkit/crashreporter/docs/Using_the_Mozilla_symbol_server.rst b/toolkit/crashreporter/docs/Using_the_Mozilla_symbol_server.rst
new file mode 100644
index 0000000000..c636e41994
--- /dev/null
+++ b/toolkit/crashreporter/docs/Using_the_Mozilla_symbol_server.rst
@@ -0,0 +1,167 @@
+Using the Mozilla symbol server
+===============================
+
+The Mozilla project runs a symbol server for trunk Firefox nightly and
+release builds on Windows. Symbols are available for at least 30
+previous days worth of nightly builds, and Firefox releases from
+2.0.0.4. This allows debugging of those builds without forcing all users
+to download large debugging files. The server functions like
+`Microsoft's symbol server <https://docs.microsoft.com/windows/win32/dxtecharts/debugging-with-symbols>`__
+so the documentation there can be useful.
+
+Note that because Mozilla release builds are heavily optimized,
+debugging is not always easy. The debugger will not be able to show you
+the content of all variables and the execution path can seem strange
+because of inlining, tail calls, and other compiler optimizations. The
+only workaround is to build an unoptimized local build.
+
+The official symbol server URL for Firefox is ``https://symbols.mozilla.org/``.
+You cannot visit this URL directly: you must add it to the symbol path of your
+debugging tool. In the examples below, a local cache directory is used to avoid
+repeatedly fetching the PDB from the server. Replace
+``C:\Users\myname\symbols`` with an appropriate cache directory on your
+machine.
+
+Using the symbol server in Microsoft Visual C++
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+|The Tools menu of Visual C++ with the Options item selected| |The
+Visual C++ 2017 Symbols pane of the Options dialog with the Mozilla
+symbol server configured|
+
+Using the symbol server in Windbg
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Windbg symbol path is configured with a string value delimited with
+asterisk characters. To use only the Mozilla symbol server, add the
+following entry to your symbol path (**note:** you can replace ``C:\symcache\``
+with any writable directory on your computer, if you'd prefer a different
+location for downloaded symbols):
+
+.. code::
+
+    SRV*c:\symcache\*https://symbols.mozilla.org/
+
+Set this string as ``_NT_SYMBOL_PATH`` in the environment, using
+the Windbg menus, or by typing the ``.sympath`` command. If you
+would like to get symbols from Microsoft's symbol server as well, you
+should list that first (**note:** you can replace ``C:\symcache\``
+with any writable directory on your computer, if you'd prefer a different
+location for downloaded symbols):
+
+.. code::
+
+    SRV*c:\symcache\*http://msdl.microsoft.com/download/symbols;SRV*c:\symcache\*https://symbols.mozilla.org/
+
+|Image:symbol-server-windbg-menu.jpg| |The WinDbg Symbol Search Path
+dialog with the Mozilla symbol server configured|
+
+Downloading symbols using symchk.exe
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can also download the symbols using symchk.exe, part of Microsoft's
+Debugging Tools for Windows. The command should look like this (again,
+you can replace ``C:\symcache\`` with any writable directory on
+your computer, if you'd prefer a different location for downloaded
+symbols):
+
+::
+
+    "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\symchk.exe" /r "C:\Program Files\Mozilla Firefox/*" /s SRV*C:\symcache\*https://symbols.mozilla.org/
+
+Note the ``\*`` after the Mozilla directory. The output of this command should
+be similar to:
+
+::
+
+    SYMCHK: api-ms-win-core-file-l1-2-0.dll FAILED  - api-ms-win-core-file-l1-2-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-core-file-l2-1-0.dll FAILED  - api-ms-win-core-file-l2-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-core-localization-l1-2-0.dll FAILED  - api-ms-win-core-localization-l1-2-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-core-processthreads-l1-1-1.dll FAILED  - api-ms-win-core-processthreads-l1-1-1.pdb mismatched or not found
+    SYMCHK: api-ms-win-core-synch-l1-2-0.dll FAILED  - api-ms-win-core-synch-l1-2-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-core-timezone-l1-1-0.dll FAILED  - api-ms-win-core-timezone-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-conio-l1-1-0.dll FAILED  - api-ms-win-crt-conio-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-convert-l1-1-0.dll FAILED  - api-ms-win-crt-convert-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-environment-l1-1-0.dll FAILED  - api-ms-win-crt-environment-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-filesystem-l1-1-0.dll FAILED  - api-ms-win-crt-filesystem-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-heap-l1-1-0.dll FAILED  - api-ms-win-crt-heap-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-locale-l1-1-0.dll FAILED  - api-ms-win-crt-locale-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-math-l1-1-0.dll FAILED  - api-ms-win-crt-math-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-multibyte-l1-1-0.dll FAILED  - api-ms-win-crt-multibyte-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-private-l1-1-0.dll FAILED  - api-ms-win-crt-private-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-process-l1-1-0.dll FAILED  - api-ms-win-crt-process-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-runtime-l1-1-0.dll FAILED  - api-ms-win-crt-runtime-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-stdio-l1-1-0.dll FAILED  - api-ms-win-crt-stdio-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-string-l1-1-0.dll FAILED  - api-ms-win-crt-string-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-time-l1-1-0.dll FAILED  - api-ms-win-crt-time-l1-1-0.pdb mismatched or not found
+    SYMCHK: api-ms-win-crt-utility-l1-1-0.dll FAILED  - api-ms-win-crt-utility-l1-1-0.pdb mismatched or not found
+    SYMCHK: d3dcompiler_47.dll   FAILED  - D3DCompiler_47.pdb mismatched or not found
+    SYMCHK: maintenanceservice_installer.exe FAILED  - Built without debugging information.
+    SYMCHK: msvcp140.dll         FAILED  - msvcp140.amd64.pdb mismatched or not found
+    SYMCHK: ucrtbase.dll         FAILED  - ucrtbase.pdb mismatched or not found
+    SYMCHK: vcruntime140.dll     FAILED  - vcruntime140.amd64.pdb mismatched or not found
+    SYMCHK: helper.exe           FAILED  - Built without debugging information.
+
+    SYMCHK: FAILED files = 27
+    SYMCHK: PASSED + IGNORED files = 60
+
+Downloading symbols on Linux / Mac OS X
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you are on Linux and running GDB 7.9 or newer, you can use `this GDB
+Python script <https://gist.github.com/luser/193572147c401c8a965c>`__ to
+automatically fetch symbols. You will need to source this script before
+loading symbols (the part where it spends a few seconds loading each .so
+when you attach gdb). If you want to reload symbols, you can try:
+
+::
+
+    nosharedlibrary
+    sharedlibrary lib
+
+On older GDB and Mac OS X there is a `Python script to download symbols
+from the Mozilla symbol
+server <http://hg.mozilla.org/users/jwatt_jwatt.org/fetch-symbols>`__
+for :ref:`gdb <Debugging Firefox with GDB>`, Shark and other software
+that uses symbols. Note that the symbol file for the XUL library is very
+large and takes some time to download. This may make it appear as if the
+script has gotten stuck, but it will continue.
+
+Symbol indices are named like so:
+symbols.mozilla.org/{lowercased:Name}/{lowercased:Name}-{Version}-{Platform}-{BuildID}-symbols.txt.
+The Platform is either 'Darwin' (for Mac) or 'Linux'. The rest of values
+are based on the contents of the application.ini file under the [App]
+heading: For example, the Thunderbird 3.1b2 release with
+Name=Thunderbird, Version=3.1b2, BuildID=20100430125415 would have a
+filename of "thunderbird-3.1b2-Linux-20100430125415-symbols.txt" under
+the thunderbird directory at symbols.mozilla.org. Its contents are a
+list of paths to files, all relative to the directory the
+BLAH-symbols.txt file is found in.
+
+The source server
+~~~~~~~~~~~~~~~~~
+
+In addition to symbols, Mozilla also has a
+:ref:`source server <Using The Mozilla Source Server>`, letting you do
+source-level debugging and inspection on demand.
+
+Troubleshooting: Symbols will not download
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If symbols will not download no matter what you do, the problem may be
+that Internet Explorer has been set to the **Work Offline** mode. You
+will not receive any warnings of this in Windbg, Visual C++ or Visual
+Studio. Even using the command line with symchk.exe to download symbols
+will fail. This is because Microsoft uses Internet Explorer's internet &
+proxy settings to download the symbol files. Check the File menu of
+Internet Explorer to ensure "Work Offline" is unchecked.
+
+.. |The Tools menu of Visual C++ with the Options item selected| image:: img/vs_tools_options.png
+   :class: internal
+.. |The Visual C++ 2017 Symbols pane of the Options dialog with the Mozilla symbol server configured| image:: img/snip_20170901070042.png
+   :class: internal
+.. |Image:symbol-server-windbg-menu.jpg| image:: img/=Symbol-server-windbg-menu.jpg
+   :class: internal
+.. |The WinDbg Symbol Search Path dialog with the Mozilla symbol server configured| image:: img/snip_20170901081816.png
+   :class: internal
+
diff --git a/toolkit/crashreporter/docs/img/snip_20170901070042.png b/toolkit/crashreporter/docs/img/snip_20170901070042.png
new file mode 100644
index 0000000000..94f266d91e
--- /dev/null
+++ b/toolkit/crashreporter/docs/img/snip_20170901070042.png
diff --git a/toolkit/crashreporter/docs/img/snip_20170901081816.png b/toolkit/crashreporter/docs/img/snip_20170901081816.png
new file mode 100644
index 0000000000..550f0d28ea
--- /dev/null
+++ b/toolkit/crashreporter/docs/img/snip_20170901081816.png
diff --git a/toolkit/crashreporter/docs/img/symbol-server-windbg-menu.jpg b/toolkit/crashreporter/docs/img/symbol-server-windbg-menu.jpg
new file mode 100644
index 0000000000..000bc761d7
--- /dev/null
+++ b/toolkit/crashreporter/docs/img/symbol-server-windbg-menu.jpg
diff --git a/toolkit/crashreporter/docs/img/vs_tools_options.png b/toolkit/crashreporter/docs/img/vs_tools_options.png
new file mode 100644
index 0000000000..c16d2094a6
--- /dev/null
+++ b/toolkit/crashreporter/docs/img/vs_tools_options.png
diff --git a/toolkit/crashreporter/docs/index.rst b/toolkit/crashreporter/docs/index.rst
new file mode 100644
index 0000000000..f41cd448b3
--- /dev/null
+++ b/toolkit/crashreporter/docs/index.rst
@@ -0,0 +1,273 @@
+==============
+Crash Reporter
+==============
+
+Overview
+========
+
+The **crash reporter** is a subsystem to record and manage application
+crash data.
+
+While the subsystem is known as *crash reporter*, it helps to think of
+it more as a *process dump manager*. This is because the heart of this
+subsystem is really managing process dump files and these files are
+created not only from process crashes but also from hangs and other
+exceptional events.
+
+The crash reporter subsystem is composed of a number of pieces working
+together.
+
+Breakpad
+   Breakpad is a library and set of tools to make collecting process
+   information (notably dumps from crashes) easy. Breakpad is a 3rd
+   party project (originally developed by Google) that is imported into
+   the tree.
+
+Dump files
+   Breakpad produces files called *dump files* that hold process data
+   (stacks, heap data, etc).
+
+Crash Reporter Client
+   The crash reporter client is a standalone executable that is launched
+   to handle dump files. This application optionally submits crashes to
+   Mozilla (or the configured server).
+
+Minidump Analyzer
+   The minidump analyzer is a standalone executable that is launched by the
+   crash reporter client or by the browser itself to extract stack traces from
+   the dump files generated during a crash. It appends the stack traces to the
+   .extra file associated with the crash dump.
+
+Ping Sender
+   The ping sender is a standalone executable that is launched by the crash
+   reporter client to deliver a crash ping to our telemetry servers. The ping
+   sender is used to speed up delivery of the crash ping which would otherwise
+   have to wait for Firefox to be restarted in order to be sent.
+
+How Main-Process Crash Handling Works
+=====================================
+
+The crash handler is hooked up very early in the Gecko process lifetime.
+It all starts in ``XREMain::XRE_mainInit()`` from ``nsAppRunner.cpp``.
+Assuming crash reporting is enabled, this startup function registers an
+exception handler for the process and tells the crash reporter subsystem
+about basic metadata such as the application name and version.
+
+The registration of the crash reporter exception handler doubles as
+initialization of the crash reporter itself. This happens in
+``CrashReporter::SetExceptionHandler()`` from ``nsExceptionHandler.cpp``.
+The crash reporter figures out what application to use for reporting
+dumped crashes and where to store these dump files on disk. The Breakpad
+exception handler (really just a mechanism for dumping process state) is
+initialized as part of this function. The Breakpad exception handler is
+a ``google_breakpad::ExceptionHandler`` instance and it's stored as
+``gExceptionHandler``.
+
+As the application runs, various other systems may write *annotations*
+or *notes* to the crash reporter to indicate state of the application,
+help with possible reasons for a current or future crash, etc. These are
+performed via ``CrashReporter::AnnotateCrashReport()`` and
+``CrashReporter::AppendAppNotesToCrashReport()`` from
+``nsExceptionHandler.h``.
+
+For well running applications, this is all that happens. However, if a
+crash or similar exceptional event occurs (such as a hang), we need to
+write a crash report.
+
+When an event worthy of writing a dump occurs, the Breakpad exception
+handler is invoked and Breakpad does its thing. When Breakpad has
+finished, it calls back into ``CrashReporter::MinidumpCallback()`` from
+``nsExceptionHandler.cpp`` to tell the crash reporter about what was
+written.
+
+``MinidumpCallback()`` performs a number of actions once a dump has been
+written. It writes a file with the time of the crash so other systems can
+easily determine the time of the last crash. It supplements the dump
+file with an *extra* file containing Mozilla-specific metadata. This data
+includes the annotations set via ``CrashReporter::AnnotateCrashReport()``
+as well as time since last crash, whether garbage collection was active at
+the time of the crash, memory statistics, etc.
+
+If the *crash reporter client* is enabled, ``MinidumpCallback()`` invokes
+it. It simply tries to create a new *crash reporter client* process (e.g.
+*crashreporter.exe*) with the path to the written minidump file as an
+argument.
+
+The *crash reporter client* performs a number of roles. There's a lot going
+on, so you may want to look at ``main()`` in ``crashreporter.cpp``. First,
+stack traces are extracted from the dump via the *minidump analyzer* tool.
+The resulting traces are appended to the .extra file of the crash together with
+the SHA256 hash of the minidump file. Once this
+is done a crash ping is assembled holding the same information as the one
+generated by the ```CrashManager``` and it's sent to the telemetry servers via
+the *ping sender* program. The UUID of the ping is then stored in the extra
+file; the ```CrashManager``` will later pick it up and generate a new ping
+with the same UUID so that the telemetry server can deduplicate both pings.
+Then, the
+*crash reporter client* verifies that the dump data is sane. If it isn't
+(e.g. required metadata is missing), the dump data is ignored. If dump data
+looks sane, the dump data
+is moved into the *pending* directory for the configured data directory
+(defined via the ``MOZ_CRASHREPORTER_DATA_DIRECTORY`` environment variable
+or from the UI). Once this is done, the main crash reporter UI is displayed
+via ``UIShowCrashUI()``. The crash reporter UI is platform specific: there
+are separate versions for Windows, OS X, and various \*NIX presentation
+flavors (such as GTK). The basic gist is a dialog is displayed to the user
+and the user has the opportunity to submit this dump data to a remote
+server.
+
+If a dump is submitted via the crash reporter, the raw dump files are
+removed from the *pending* directory and a file containing the
+crash ID from the remote server for the submitted dump is created in the
+*submitted* directory.
+
+If the user chooses not to submit a dump in the crash reporter UI, the dump
+files are deleted.
+
+And that's pretty much what happens when a crash/dump is written!
+
+Plugin and Child Process Crashes
+================================
+
+Crashes in plugin and child processes are also managed by the crash
+reporting subsystem.
+
+Child process crashes are handled by the ``mozilla::dom::CrashReporterParent``
+class defined in ``dom/ipc``. When a child process crashes, the toplevel IPDL
+actor should check for it by calling TakeMinidump in its ``ActorDestroy``
+Method: see ``mozilla::plugins::PluginModuleParent::ActorDestroy`` and
+``mozilla::plugins::PluginModuleParent::ProcessFirstMinidump``. That method
+is responsible for calling
+``mozilla::dom::CrashReporterParent::GenerateCrashReportForMinidump`` with
+appropriate crash annotations specific to the crash. All child-process
+crashes are annotated with a ``ProcessType`` annotation, such as "content" or
+"plugin".
+
+Once the minidump file has been generated the
+``mozilla::dom::CrashReporterHost`` is notified of the crash. It will first
+try to extract the stack traces from the minidump file using the
+*minidump analyzer*. Then the stack traces will be stored in the extra file
+together with the rest of the crash annotations and finally the crash will be
+recorded by calling ```CrashService.addCrash()```. This last step adds the
+crash to the ```CrashManager``` database and automatically sends a crash ping
+with information about the crash.
+
+Submission of child process crashes is handled by application code. This
+code prompts the user to submit crashes in context-appropriate UI and then
+submits the crashes using ``CrashSubmit.jsm``.
+
+Memory Reports
+==============
+
+When a process detects that it is running low on memory, a memory report is
+saved. If the process crashes, the memory report will be included with the crash
+report. ``nsThread::SaveMemoryReportNearOOM()`` checks to see if the process is
+low on memory every 30 seconds at most and saves a report every 3 minutes at
+most. Since a child process cannot actually save to the hard drive, it instead
+notifies its parent process, which saves the report for it. If a crash does
+occur, the memory report is moved to the *pending* directory with the other dump
+data and an annotation is added to indicate the presence of the report. This
+happens in ``nsExceptionHandler.cpp``, but occurs in different functions
+depending on what process crashed. When the main process crashes, this happens
+in ``MinidumpCallback()``. When a child process crashes, it happens in
+``OnChildProcessDumpRequested()``, with the annotation being added in
+``WriteExtraData()``.
+
+Flash Process Crashes
+=====================
+
+On Windows Vista+, the Adobe Flash plugin creates two extra processes in its
+Firefox plugin to implement OS-level sandboxing. In order to catch crashes in
+these processes, Firefox injects a crash report handler into the process using the code at ``InjectCrashReporter.cpp``. When these crashes occur, the
+ProcessType=plugin annotation is present, and an additional annotation
+FlashProcessDump has the value "Sandbox" or "Broker".
+
+Plugin Hangs
+============
+
+Plugin hangs are handled as crash reports. If a plugin doesn't respond to an
+IPC message after 60 seconds, the plugin IPC code will take minidumps of all
+of the processes involved and then kill the plugin.
+
+In this case, there will be only one .ini file with the crash report metadata,
+but there will be multiple dump files: at least one for the browser process and
+one for the plugin process, and perhaps also additional dumps for the Flash
+sandbox and broker processes. All of these files are submitted together as a
+unit. Before submission, the filenames of the files are linked:
+
+- **uuid.ini** - *annotations, includes an additional_minidumps field*
+- **uuid.dmp** - *plugin process dump file*
+- **uuid-<other>.dmp** - *other process dump file as listed in additional_minidumps*
+
+about:crashes
+=============
+
+If the crash reporter subsystem is enabled, the *about:crashes*
+page will be registered with the application. This page provides
+information about previous and submitted crashes.
+
+It is also possible to submit crashes from *about:crashes*.
+
+Environment variables affecting crash reporting
+===============================================
+
+The exception handler and crash reporter client behavior can be altered by
+setting certain environment variables, some of these variables are used for
+testing but quite a few have only internal users.
+
+User-specified environment variables
+------------------------------------
+
+- ``MOZ_CRASHREPORTER`` - The opposite of MOZ_CRASHREPORTER_DISABLE, force
+  crash reporting on even if disabled in application.ini. You must use this to
+  enable crash reporting on debug builds.
+- ``MOZ_CRASHREPORTER_DISABLE`` - Disable Breakpad crash reporting completely
+  in non-debug builds. You can use this if you would rather use the JIT
+  debugger on Windows with the symbol server, for example.
+- ``MOZ_CRASHREPORTER_FULLDUMP`` - Store full application memory in the
+  minidump, so you can open it in a Microsoft debugger. Don't submit it to the
+  server. (Windows only.)
+- ``MOZ_CRASHREPORTER_NO_DELETE_DUMP`` - Don't delete the crash report dump
+  file after submitting it to the server. Minidumps will still be moved to the
+  "Crash Reports/pending" directory.
+- ``MOZ_CRASHREPORTER_NO_REPORT`` - Save the minidump file but don't launch the
+  crash reporting UI or send the report to the server. Minidumps will be stored
+  in the user's profile directory, in a subdirectory named "minidumps".
+- ``MOZ_CRASHREPORTER_SHUTDOWN`` - Save the minidump and then force the
+  application to close. This is useful for content crashes that don't normally
+  close the chrome (main application) processes. This variable would cause the
+  application to close as well.
+- ``MOZ_CRASHREPORTER_URL`` - Sets the URL that the crash reporter will submit
+  reports to.
+
+Environment variables used internally
+-------------------------------------
+
+- ``MOZ_CRASHREPORTER_AUTO_SUBMIT`` - When set causes the crash reporter client
+  to skip the UI flow and submit the crash report directly.
+- ``MOZ_CRASHREPORTER_DATA_DIRECTORY`` - Platform dependent data directory, the
+  pending crash reports will be stored in a subdirectory of this path. This
+  overrides the default one generated by the client's code.
+- ``MOZ_CRASHREPORTER_DUMP_ALL_THREADS`` - When set to 1 stack traces for
+  all threads are generated and sent in the crash ping, when not set only the
+  trace for the crashing thread will be generated instead.
+- ``MOZ_CRASHREPORTER_EVENTS_DIRECTORY`` - Path of the directory holding the
+  crash event files.
+- ``MOZ_CRASHREPORTER_PING_DIRECTORY`` - Path of the directory holding the
+  pending crash ping files.
+- ``MOZ_CRASHREPORTER_RESTART_ARG_<n>`` - Each of these variable specifies one
+  of the arguments that had been passed to the application, the crash reporter
+  client uses them for restarting it.
+- ``MOZ_CRASHREPORTER_RESTART_XUL_APP_FILE`` - If a XUL app file was specified
+  when starting the app it has to be stored in this variable so that the crash
+  reporter client can restart the application.
+- ``MOZ_CRASHREPORTER_STRINGS_OVERRIDE`` - Overrides the path used to load the
+  .ini file holding the strings used in the crash reporter client UI.
+
+Other topics
+============
+
+.. toctree::
+   :titlesonly:
+
+   Using_the_Mozilla_symbol_server