diff options
Diffstat (limited to '')
| -rw-r--r-- | tools/sanitizer/docs/asan.rst | 398 |
1 files changed, 398 insertions, 0 deletions
diff --git a/tools/sanitizer/docs/asan.rst b/tools/sanitizer/docs/asan.rst new file mode 100644 index 0000000000..3b48accb09 --- /dev/null +++ b/tools/sanitizer/docs/asan.rst @@ -0,0 +1,398 @@ +Address Sanitizer +================= + +What is Address Sanitizer? +-------------------------- + +Address Sanitizer (ASan) is a fast memory error detector that detects +use-after-free and out-of-bound bugs in C/C++ programs. It uses a +compile-time instrumentation to check all reads and writes during the +execution. In addition, the runtime part replaces the ``malloc`` and +``free`` functions to check dynamically allocated memory. More +information on how ASan works can be found on `the Address Sanitizer +wiki <https://github.com/google/sanitizers/wiki/AddressSanitizer>`__. + +A `meta bug called asan-maintenance <https://bugzilla.mozilla.org/show_bug.cgi?id=asan-maintenance>`__ +is maintained to keep track of all the bugs found with ASan. + +Downloading artifact builds +--------------------------- + +For Linux and Windows users, the easiest way to get Firefox builds with +Address Sanitizer is to download a continuous integration asan build of +mozilla-central (updated at least daily): + +- mozilla-central optimized builds: + `linux <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.linux64-asan-opt/artifacts/public/build/target.tar.bz2>`__ + \| + `windows <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.win64-asan-opt/artifacts/public/build/target.zip>`__ + (recommended for testing) +- mozilla-central debug builds: + `linux <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.linux64-asan-debug/artifacts/public/build/target.tar.bz2>`__ + \| + `windows <https://firefox-ci-tc.services.mozilla.com/api/index/v1/task/gecko.v2.mozilla-central.latest.firefox.win64-asan-debug/artifacts/public/build/target.zip>`__ + (recommended for debugging if the optimized builds don't do the job) + +The fuzzing team also offers a tool called ``fuzzfetch`` to download these and many +other CI builds. It makes downloading and unpacking these builds much easier and +can be used not just for fuzzing but for all purposes that require a CI build download. + +You can install ``fuzzfetch`` from +`Github <https://github.com/MozillaSecurity/fuzzfetch>`__ or +`via pip <https://pypi.org/project/fuzzfetch/>`__. + +Afterwards, you can run e.g. + +:: + + $ python -m fuzzfetch --asan -n firefox-asan + +to get the optimized Linux ASan build mentioned above unpacked into a directory called ``firefox-asan``. +The ``--debug`` and ``--os`` switches can be used to get the other variants listed above. + +Creating Try builds +------------------- + +If for some reason you can't use the pre-built binaries mentioned in the +previous section (e.g. you want a non-Linux build or you need to test a +patch), you can either build Firefox yourself (see the following +section) or use the :ref:`try server <Try Server>` to +create the customized build for you. Pushing to try requires L1 commit +access. If you don't have this access yet you can request access (see +`Becoming A Mozilla +Committer <https://www.mozilla.org/about/governance/policies/commit/>`__ +and `Mozilla Commit Access +Policy <https://www.mozilla.org/about/governance/policies/commit/access-policy/>`__ +for the requirements). + +The tree contains `several mozconfig files for creating asan +builds <https://searchfox.org/mozilla-central/search?q=&case=true&path=browser%2Fconfig%2Fmozconfigs%2F*%2F*asan*>`__ +(the "nightly-asan" files create release builds, whereas the +"debug-asan" files create debug+opt builds). For Linux builds, the +appropriate configuration file is used by the ``linux64-asan`` target. +If you want to create a macOS or Windows build, you'll need to copy the +appropriate configuration file over the regular debug configuration +before pushing to try. For example: + +:: + + cp browser/config/mozconfigs/macosx64/debug-asan browser/config/mozconfigs/macosx64/debug + +You can then `push to Try in the usual +way </tools/try/index.html#using-try>`__ +and, once the build is complete, download the appropriate build +artifact. + +Creating local builds on Windows +-------------------------------- + +On Windows, ASan is supported only in 64-bit builds. + +Run ``mach bootstrap`` to get an updated clang-cl in your +``~/.mozbuild`` directory, then use the following +:ref:`mozconfig <Configuring Build Options>`: + +:: + + ac_add_options --target=x86_64-pc-mingw32 + ac_add_options --host=x86_64-pc-mingw32 + + ac_add_options --enable-address-sanitizer + ac_add_options --disable-jemalloc + + export CC="clang-cl.exe" + export CXX="clang-cl.exe" + + export LDFLAGS="clang_rt.asan_dynamic-x86_64.lib clang_rt.asan_dynamic_runtime_thunk-x86_64.lib" + CLANG_LIB_DIR="$(cd ~/.mozbuild/clang/lib/clang/*/lib/windows && pwd)" + export MOZ_CLANG_RT_ASAN_LIB_PATH="${CLANG_LIB_DIR}/clang_rt.asan_dynamic-x86_64.dll" + export LIB=$LIB:$CLANG_LIB_DIR + +If you want to use a different LLVM (see the :ref:`clang-cl instructions <Building Firefox On Windows>`), +alter CLANG_LIB_DIR as appropriate. + +If you launch an ASan build under WinDbg, you may see spurious +first-chance Access Violation exceptions. These come from ASan creating +shadow memory pages on demand, and can be ignored. Run ``sxi av`` to +ignore these exceptions. (You will still catch second-chance Access +Violation exceptions if you actually crash.) + +LeakSanitizer (LSan) is not supported on Windows. + +Creating local builds on Linux or Mac +------------------------------------- + +Build prerequisites +~~~~~~~~~~~~~~~~~~~ + +LLVM/Clang +^^^^^^^^^^ + +The ASan instrumentation is implemented as an LLVM pass and integrated +into Clang. Any clang version that is capable of compiling Firefox has +everything needed to do an ASAN build. + +Building Firefox +~~~~~~~~~~~~~~~~ + +Getting the source +^^^^^^^^^^^^^^^^^^ + +Using that or any later revision, all you need to do is to :ref:`get yourself +a clone of mozilla-central <Mercurial overview>`. + +Adjusting the build configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create the build configuration file ``mozconfig`` with the following +content in your mozilla-central directory: + +:: + + # Combined .mozconfig file for ASan on Linux+Mac + + mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-asan + + # Enable ASan specific code and build workarounds + ac_add_options --enable-address-sanitizer + + # Add ASan to our compiler flags + export CFLAGS="-fsanitize=address -Dxmalloc=myxmalloc -fPIC" + export CXXFLAGS="-fsanitize=address -Dxmalloc=myxmalloc -fPIC" + + # Additionally, we need the ASan flag during linking. Normally, our C/CXXFLAGS would + # be used during linking as well but there is at least one place in our build where + # our CFLAGS are not added during linking. + # Note: The use of this flag causes Clang to automatically link the ASan runtime :) + export LDFLAGS="-fsanitize=address" + + # These three are required by ASan + ac_add_options --disable-jemalloc + ac_add_options --disable-crashreporter + ac_add_options --disable-elf-hack + + # Keep symbols to symbolize ASan traces later + export MOZ_DEBUG_SYMBOLS=1 + ac_add_options --enable-debug-symbols + ac_add_options --disable-install-strip + + # Settings for an opt build (preferred) + # The -gline-tables-only ensures that all the necessary debug information for ASan + # is present, but the rest is stripped so the resulting binaries are smaller. + ac_add_options --enable-optimize="-O2 -gline-tables-only" + ac_add_options --disable-debug + + # Settings for a debug+opt build + #ac_add_options --enable-optimize + #ac_add_options --enable-debug + + # MacOSX only: Uncomment and adjust this path to match your SDK + # ac_add_options --with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk + +You may also need this, as seen in +``browser/config/mozconfigs/linux64/nightly-asan`` (the config file used +for Address Sanitizer builds used for automated testing): + +:: + + # ASan specific options on Linux + ac_add_options --enable-valgrind + +Starting the build process +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Now you start the build process using the regular ``./mach build`` +command. + +Starting Firefox +^^^^^^^^^^^^^^^^ + +After the build has completed, ``./mach run`` with the usual options for +running in a debugger (``gdb``, ``lldb``, ``rr``, etc.) work fine, as do +the ``--disable-e10s`` and other options. + +Building only the JavaScript shell +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you want to build only the JavaScript shell instead of doing a full +Firefox build, the build script below will probably help you to do so. +Execute this script in the ``js/src/`` subdirectory and pass a directory +name as the first parameter. The build will then be created in a new +subdirectory with that name. + +:: + + #! /bin/sh + + if [ -z $1 ] ; then + echo "usage: $0 <dirname>" + elif [ -d $1 ] ; then + echo "directory $1 already exists" + else + autoconf2.13 + mkdir $1 + cd $1 + CC="clang" \ + CXX="clang++" \ + CFLAGS="-fsanitize=address" \ + CXXFLAGS="-fsanitize=address" \ + LDFLAGS="-fsanitize=address" \ + ../configure --enable-debug --enable-optimize --enable-address-sanitizer --disable-jemalloc + fi + +Getting Symbols in Address Sanitizer Traces +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default, ASan traces are unsymbolized and only print the +binary/library and a memory offset instead. In order to get more useful +traces, containing symbols, there are two approaches. + +Using the LLVM Symbolizer (recommended) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +LLVM ships with a symbolizer binary that ASan will readily use to +immediately output symbolized traces. To use it, just set the +environment variable ``ASAN_SYMBOLIZER_PATH`` to reflect the location of +your ``llvm-symbolizer`` binary, before running the process. This +program is usually included in an LLVM distribution. Stacks without +symbols can also be post-processed, see below. + +.. warning:: + + .. note:: + + **Warning:** On OS X, the content sandbox prevents the symbolizer + from running. To use llvm-symbolizer on ASan output from a + content process, the content sandbox must be disabled. This can be + done by setting ``MOZ_DISABLE_CONTENT_SANDBOX=1`` in your run + environment. Setting this in .mozconfig has no effect. + + +Post-Processing Traces with asan_symbolize.py +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Instead of using the llvm-symbolizer binary, you can also pipe the +output through the ``asan_symbolize.py`` script, shipped with LLVM +(``$LLVM_HOME/projects/compiler-rt/lib/asan/scripts/asan_symbolize.py``), +often included in LLVM distributions. The disadvantage is that the +script will need to use ``addr2line`` to get the symbols, which means +that every library will have to be loaded into memory +(including``libxul``, which takes a bit). + +However, in certain situations it makes sense to use this script. For +example, if you have/received an unsymbolized trace, then you can still +use the script to turn it into a symbolized trace, given that you can +get the original binaries that produced the unsymbolized trace. In order +for the script to work in such cases, you need to ensure that the paths +in the trace point to the actual binaries, or change the paths +accordingly. + +Since the output of the ``asan_symbolize.py`` script is still mangled, +you might want to pipe the output also through ``c++filt`` afterwards. + +Troubleshooting / Known problems +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Cannot specify -o when generating multiple output files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you get the error +"``cannot specify -o when generating multiple output files"`` from +clang, disable ``elf-hack`` in your ``mozconfig`` to work around the +issue: + +:: + + ac_add_options --disable-elf-hack + +Optimized build +^^^^^^^^^^^^^^^ + +Since `an issue with -O2/-Os and +ASan <https://github.com/google/sanitizers/issues/20>`__ +has been resolved, the regular optimizations used by Firefox should work +without any problems. The optimized build has only a barely noticeable +speed penalty and seems to be even faster than regular debug builds. + +No "AddressSanitizer: **libc** interceptors initialized" shows after running ./mach run +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + $ ASAN_OPTIONS=verbosity=2 ./mach run + +Use the above command instead + +"An admin user name and password" is required to enter Developer Mode +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Please enable **Developer** **mode** by: + +:: + + $ /usr/sbin/DevToolsSecurity -enable + Developer mode is now enabled. + +Debugging issues that ASan finds +-------------------------------- + +When ASan discovers an issue it will simply print an error message and +exit the app. To stop the app in a debugger before ASan exits it, set a +breakpoint on ``__asan::ReportGenericError``. For more info on using +ASan and debugging issues that it uncovers, see the page `Address +sanitizer and a +debugger <https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger>`__ +page on the upstream wiki. + +``__asan_describe_address(pointer)`` issued at the debugger prompt or +even directly in the code allows outputting lots of information about +this memory address (thread and stack of allocation, of deallocation, +whether or not it is a bit outside a known buffer, thread and stack of +allocation of this buffer, etc.). This can be useful to understand where +some buffer that is not aligned was allocated, when doing SIMD work, for +example. + +`rr <https://rr-project.org/>`__ (Linux x86 only) works great with ASan +and combined, this combo allows doing some very powerful debugging +strategies. + +LeakSanitizer +------------- + +LeakSanitizer (LSan) is a special execution mode for regular ASan. It +takes advantage of how ASan tracks the set of live blocks at any given +point to print out the allocation stack of any block that is still alive +at shutdown, but is not reachable from the stack, according to a +conservative scan. This is very useful for detecting leaks of things +such as ``char*`` that do not participate in the usual Gecko shutdown +leak detection. LSan is supported on x86_64 Linux and OS X. + +LSan is enabled by default in ASan builds, as of more recent versions of +Clang. To make an ASan build not run LSan, set the environment variable +``ASAN_OPTIONS`` to ``detect_leaks=0`` (or add it as an entry to a +``:``-separated list if it is already set to something). If you want to +enable it when it is not for some reason, set it to 1 instead of 0. If +LSan is enabled and you are using a non-debug build, you will also want +to set the environment variable ``MOZ_CC_RUN_DURING_SHUTDOWN=1``, to +ensure that we run shutdown GCs and CCs to avoid spurious leaks. + +If an object that is reported by LSan is intentionally never freed, a +symbol can be added to ``build/sanitizers/lsan_suppressions.txt`` to get +LSan to ignore it. + +For some more information on LSan, see the `Leak Sanitizer wiki +page <https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer>`__. + + +A `meta bug called lsan <https://bugzilla.mozilla.org/show_bug.cgi?id=lsan>`__ +is maintained to keep track of all the bugs found with LSan. + + + +Frequently Asked Questions about ASan +------------------------------------- + +How does ASan work exactly? +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +More information on how ASan works can be found on `the Address Sanitizer wiki <https://github.com/google/sanitizers/wiki/AddressSanitizer>`__. |