Adding upstream version 1:115.7.0.upstream/1%115.7.0upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 327 insertions, 0 deletions

diff --git a/tools/sanitizer/docs/tsan.rst b/tools/sanitizer/docs/tsan.rst
new file mode 100644
index 0000000000..77fb6c89d7
--- /dev/null
+++ b/tools/sanitizer/docs/tsan.rst
@@ -0,0 +1,327 @@
+Thread Sanitizer
+================
+
+What is Thread Sanitizer?
+--------------------------
+
+Thread Sanitizer (TSan) is a fast data race detector for C/C++ and Rust
+programs. It uses a compile-time instrumentation to check all non-race-free
+memory access at runtime. Unlike other tools, it understands compiler-builtin
+atomics and synchronization and therefore provides very accurate results
+with no false positives (except if unsupported synchronization primitives
+like inline assembly or memory fences are used). More information on how
+TSan works can be found on `the Thread Sanitizer wiki <https://github.com/google/sanitizers/wiki/ThreadSanitizerAlgorithm>`__.
+
+A `meta bug called tsan <https://bugzilla.mozilla.org/show_bug.cgi?id=tsan>`__
+is maintained to keep track of all the bugs found with TSan.
+
+A `blog post on hacks.mozilla.org <https://hacks.mozilla.org/2021/04/eliminating-data-races-in-firefox-a-technical-report/>`__ describes this project.
+
+Note that unlike other sanitizers, TSan is currently **only supported on Linux**.
+
+Downloading artifact builds
+---------------------------
+
+The easiest way to get Firefox builds with Thread Sanitizer is to download a
+continuous integration TSan build of mozilla-central (updated at least daily):
+
+-  mozilla-central optimized builds:
+   `linux <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.linux64-tsan-opt/artifacts/public/build/target.tar.bz2>`__
+
+The fuzzing team also offers a tool called ``fuzzfetch`` to download this and many
+other CI builds. It makes downloading and unpacking these builds much easier and
+can be used not just for fuzzing but for all purposes that require a CI build download.
+
+You can install ``fuzzfetch`` from
+`Github <https://github.com/MozillaSecurity/fuzzfetch>`__ or
+`via pip <https://pypi.org/project/fuzzfetch/>`__.
+
+Afterwards, you can run
+
+::
+
+   $ python -m fuzzfetch --tsan -n firefox-tsan
+
+to get the build mentioned above unpacked into a directory called ``firefox-tsan``.
+
+Creating Try builds
+-------------------
+
+If for some reason you can't use the pre-built binaries mentioned in the
+previous section (e.g. you need to test a patch), you can either build
+Firefox yourself (see the following section) or use the :ref:`try server <Pushing to Try>`
+to create the customized build for you. Pushing to try requires L1 commit
+access. If you don't have this access yet you can request access (see
+`Becoming A Mozilla
+Committer <https://www.mozilla.org/about/governance/policies/commit/>`__
+and `Mozilla Commit Access
+Policy <https://www.mozilla.org/about/governance/policies/commit/access-policy/>`__
+for the requirements).
+
+Using ``mach try fuzzy --full`` you can select the ``build-linux64-tsan/opt`` job
+and related tests (if required).
+
+Creating local builds on Linux
+------------------------------
+
+Build prerequisites
+~~~~~~~~~~~~~~~~~~~
+
+LLVM/Clang/Rust
+^^^^^^^^^^^^^^^
+
+The TSan instrumentation is implemented as an LLVM pass and integrated
+into Clang. We strongly recommend that you use the Clang version supplied
+as part of the ``mach bootstrap`` process, as we backported several required
+fixes for TSan on Firefox.
+
+Sanitizer support in Rust is genuinely experimental,
+so our build system only works with a specially patched version of Rust
+that we build in our CI. To install that specific version (or update to a newer
+version), run the following in the root of your mozilla-central checkout:
+
+::
+
+    ./mach artifact toolchain --from-build linux64-rust-dev
+    rm -rf ~/.mozbuild/rustc-sanitizers
+    mv rustc ~/.mozbuild/rustc-sanitizers
+    rustup toolchain link gecko-sanitizers ~/.mozbuild/rustc-sanitizers
+    rustup override set gecko-sanitizers
+
+``mach artifact`` will always download the ``linux64-rust-dev`` toolchain associated
+with the current mozilla central commit you have checked out. The toolchain should
+mostly behave like a normal rust nightly but we don't recommend using it for anything
+other than building gecko, just in case. Also note that
+``~/.mozbuild/rustc-sanitizers`` is just a reasonable default location -- feel
+free to "install" the toolchain wherever you please.
+
+Building Firefox
+~~~~~~~~~~~~~~~~
+
+Getting the source
+^^^^^^^^^^^^^^^^^^
+
+Using that or any later revision, all you need to do is to :ref:`get yourself
+a clone of mozilla-central <Mercurial overview>`.
+
+Adjusting the build configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Create the build configuration file ``mozconfig`` with the following
+content in your mozilla-central directory:
+
+::
+
+   # Combined .mozconfig file for TSan on Linux+Mac
+
+   mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-tsan
+
+   # Enable ASan specific code and build workarounds
+   ac_add_options --enable-thread-sanitizer
+
+   # This ensures that we also instrument Rust code.
+   export RUSTFLAGS="-Zsanitizer=thread"
+
+   # rustfmt is currently missing in Rust nightly
+   unset RUSTFMT
+
+   # Current Rust Nightly has warnings
+   ac_add_options --disable-warnings-as-errors
+
+   # These are required by TSan
+   ac_add_options --disable-jemalloc
+   ac_add_options --disable-crashreporter
+   ac_add_options --disable-elf-hack
+   ac_add_options --disable-profiling
+
+   # The Thread Sanitizer is not compatible with sandboxing
+   # (see bug 1182565)
+   ac_add_options --disable-sandbox
+
+   # Keep symbols to symbolize TSan traces later
+   export MOZ_DEBUG_SYMBOLS=1
+   ac_add_options --enable-debug-symbols
+   ac_add_options --disable-install-strip
+
+   # Settings for an opt build (preferred)
+   # The -gline-tables-only ensures that all the necessary debug information for ASan
+   # is present, but the rest is stripped so the resulting binaries are smaller.
+   ac_add_options --enable-optimize="-O2 -gline-tables-only"
+   ac_add_options --disable-debug
+
+   # Settings for a debug+opt build
+   #ac_add_options --enable-optimize
+   #ac_add_options --enable-debug
+
+
+Starting the build process
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Now you start the build process using the regular ``./mach build``
+command.
+
+Starting Firefox
+^^^^^^^^^^^^^^^^
+
+After the build has completed, ``./mach run`` with the usual options for
+running in a debugger (``gdb``, ``lldb``, ``rr``, etc.) work fine, as do
+the ``--disable-e10s`` and other options.
+
+Building only the JavaScript shell
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you want to build only the JavaScript shell instead of doing a full
+Firefox build, the build script below will probably help you to do so.
+Execute this script in the ``js/src/`` subdirectory and pass a directory
+name as the first parameter. The build will then be created in a new
+subdirectory with that name.
+
+::
+
+   #! /bin/sh
+
+   if [ -z $1 ] ; then
+        echo "usage: $0 <dirname>"
+   elif [ -d $1 ] ; then
+        echo "directory $1 already exists"
+   else
+        autoconf2.13
+        mkdir $1
+        cd $1
+        CC="/path/to/mozbuild/clang" \
+        CXX="/path/to/mozbuild/clang++" \
+        ../configure --disable-debug --enable-optimize="-O2 -gline-tables-only" --enable-thread-sanitizer --disable-jemalloc
+   fi
+
+Thread Sanitizer and Symbols
+----------------------------
+
+Unlike Address Sanitizer, TSan requires in-process symbolizing to work
+properly in the first place, as any kind of runtime suppressions will
+otherwise not work.
+
+Hence, it is required that you have a copy of ``llvm-symbolizer`` either
+in your ``PATH`` or pointed to by the ``TSAN_SYMBOLIZER_PATH`` environment
+variable. This binary is included in your local mozbuild directory, obtained
+by ``./mach bootstrap``.
+
+
+Runtime Suppressions
+--------------------
+
+TSan has the ability to suppress race reports at runtime. This can be used to
+silence a race while a fix is developed as well as to permanently silence a
+(benign) race that cannot be fixed.
+
+.. warning::
+       **Warning**: Many races *look* benign but are indeed not. Please read
+       the :ref:`FAQ section <Frequently Asked Questions about TSan>` carefully
+       and think twice before attempting to suppress a race.
+
+The runtime Suppression list is directly baked into Firefox at compile-time and
+located at `mozglue/build/TsanOptions.cpp <https://searchfox.org/mozilla-central/source/mozglue/build/TsanOptions.cpp>`__.
+
+.. warning::
+       **Important**: When adding a suppression, always make sure to include
+       the bug number. If the suppression is supposed to be permanent, please
+       add the string ``permanent`` in the same line as the bug number.
+
+.. warning::
+       **Important**: When adding a suppression for a *data race*, always make
+       sure to include a stack frame from **each** of the two race stacks.
+       Adding only one suppression for one stack can cause intermittent failures
+       that are later on hard to track. One exception to this rule is when suppressing
+       races on global variables. In that case, a single race entry with the name of
+       the variable is sufficient.
+
+Troubleshooting / Known Problems
+--------------------------------
+
+Known Sources of False Positives
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+TSan has a number of things that can cause false positives, namely:
+
+  * The use of memory fences (e.g. Rust Arc)
+  * The use of inline assembly for synchronization
+  * Uninstrumented code (e.g. external libraries) using compiler-builtins for synchronization
+  * A lock order inversion involving only a single thread can cause a false positive deadlock
+    report (see also https://github.com/google/sanitizers/issues/488).
+
+If none of these four items are involved, you should *never* assume that TSan is reporting
+a false positive to you without consulting TSan peers. It is very easy to misjudge a race
+to be a false positive because races can be highly complex and totally non-obvious due to
+compiler optimizations and the nature of parallel code.
+
+Intermittent Broken Stacks
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you intermittently see race reports where one stack is missing with a ``failed to restore the stack``
+message, this can indicate that a suppression is partially covering the race you are seeing.
+
+Any race where only one of the two stacks is matched by a runtime suppression will show up
+if that particular stack fails to symbolize for some reason. The usual solution is to search
+the suppressions for potential candidates and disable them temporarily to check if your race
+report now becomes mostly consistent.
+
+However, there are other reasons for broken TSan stacks, in particular if they are not intermittent.
+See also the ``history_size`` parameter in the `TSan flags <https://github.com/google/sanitizers/wiki/ThreadSanitizerFlags>`__.
+
+Intermittent Race Reports
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Unfortunately, the TSan algorithm does not guarantee, that a race is detected 100% of the
+time. Intermittent failures with TSan are (to a certain degree) to be expected and the races
+involved should be filed and fixed to solve the problem.
+
+.. _Frequently Asked Questions about TSan:
+
+Frequently Asked Questions about TSan
+-------------------------------------
+
+Why fix data races?
+~~~~~~~~~~~~~~~~~~~
+
+Data races are undefined behavior and can cause crashes as well as correctness issues.
+Compiler optimizations can cause racy code to have unpredictable and hard-to-reproduce behavior.
+
+At Mozilla, we have already seen several dangerous races, causing random
+`use-after-free crashes <https://bugzilla.mozilla.org/show_bug.cgi?id=1580288>`__,
+`intermittent test failures <https://bugzilla.mozilla.org/show_bug.cgi?id=1602009>`__,
+`hangs <https://bugzilla.mozilla.org/show_bug.cgi?id=1607008>`__,
+`performance issues <https://bugzilla.mozilla.org/show_bug.cgi?id=1615045>`__ and
+`intermittent asserts <https://bugzilla.mozilla.org/show_bug.cgi?id=1601940>`__. Such problems do
+not only decrease the quality of our code and user experience, but they also waste countless hours
+of developer time.
+
+Since it is very hard to judge if a particular race could cause such a situation, we
+have decided to fix all data races wherever possible, since doing so is often cheaper
+than analyzing a race.
+
+My race is benign, can we ignore it?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While it is possible to add a runtime suppression to ignore the race, we *strongly* encourage
+you to not do so, for two reasons:
+
+    1. Each suppressed race decreases the overall performance of the TSan build, as the race
+       has to be symbolized each time when it occurs. Since TSan is already in itself a slow
+       build, we need to keep the amount of suppressed races as low as possible.
+
+    2. Deciding if a race is truly benign is surprisingly hard. We recommend to read
+       `this blog post <http://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong>`__
+       and `this paper <https://www.usenix.org/legacy/events/hotpar11/tech/final_files/Boehm.pdf>`
+       on the effects of seemingly benign races.
+
+Valid reasons to suppress a confirmed benign race include performance problems arising from
+fixing the race or cases where fixing the race would require an unreasonable amount of work.
+
+Note that the use of atomics usually does not have the bad performance impact that developers
+tend to associate with it. If you assume that e.g. using atomics for synchronization will
+cause performance regressions, we suggest to perform a benchmark to confirm this. In many
+cases, the difference is not measurable.
+
+How does TSan work exactly?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+More information on how TSan works can be found on `the Thread Sanitizer wiki <https://github.com/google/sanitizers/wiki/ThreadSanitizerAlgorithm>`__.