diff options
Diffstat (limited to 'docs/contributing/debugging/debugging_a_minidump.rst')
| -rw-r--r-- | docs/contributing/debugging/debugging_a_minidump.rst | 202 |
1 files changed, 202 insertions, 0 deletions
diff --git a/docs/contributing/debugging/debugging_a_minidump.rst b/docs/contributing/debugging/debugging_a_minidump.rst new file mode 100644 index 0000000000..5638c29d85 --- /dev/null +++ b/docs/contributing/debugging/debugging_a_minidump.rst @@ -0,0 +1,202 @@ +Debugging A Minidump +==================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +The +`minidump <http://msdn.microsoft.com/en-us/library/windows/desktop/ms680369%28v=vs.85%29.aspx>`__ +file format contains data about a crash on Windows. It is used by +`rust-minidump <https://github.com/luser/rust-minidump>`__, +`Breakpad <https://wiki.mozilla.org/Breakpad>`__, and also by various +Windows debugging tools. Each minidump includes the following data. + +- Details about the exception which led to the crash. +- Information about each thread in the process: the address which was + executing and the register state at the time the process stopped. +- A list of shared libraries loaded into the process at the time of the + crash. +- The stack memory of each thread. +- The memory right around the crashing address. +- (Optional) Other memory regions, if requested by the application. +- (Optional) Other platform-specific data. + +Accessing minidumps from crash reports +-------------------------------------- + +Minidumps are not available to everyone. For details on how to gain +access and where to find minidump files for crash reports, consult the +:ref:`crash report documentation <Understanding Crash Reports>` + +Using rust-minidump's tooling +----------------------------------- + +Most of our crash-reporting infrastructure is based on rust-minidump. +The primary tool for this is the +`minidump-stackwalk <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk>`__ +CLI application, which includes extensive user documentation. + +That documentation includes a +`dedicated section <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk#analyzing-firefox-minidumps>`__ +on locally analyzing Firefox crashreports and minidumps. + +If you're looking for minidump_dump, it's included as part of +minidump-stackwalk. + +Using the MS Visual Studio debugger +----------------------------------- + +#. Set up the debugger to :ref:`use the Mozilla symbol + server <Using The Mozilla Symbol Server>` and + :ref:`source server <Using The Mozilla Source Server>`. +#. Double-click on the minidump file to open it in the debugger. +#. When it loads, click the green icon in the visual studio debugger + toolbar that looks like a play button. + +For Firefox releases older than Firefox 41, you will also need to +install the relevant release of Firefox (for example from +`here <https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/>`__), +and add the directory it is in (e.g., "C:\Program Files\Mozilla +Firefox 3.6 Beta 1\") to the same dialog in which you set up the +symbol server (in case the binary location in the minidump is not the +same as the one on your machine). Note that you can install the +relevant release anywhere. Just make sure to configure the symbol +server to the directory where you installed it. For releases from 41 +onward, the binaries are available on the symbol server. + +If this doesn't work, downloading the exact build and crashreporter +symbols full files. These can be found in treeherder / build folder. +Load Visual Studio, and go to file -> open -> minidump location. Click +on "Run Native", and Visual Studio will ask for the corresponding symbol +files. For each .dll you wish to have symbols for, you must go to a +console and go to the corresponding directory. E.g. (xul.dll should go +to xul.pdf in the crashreporter symbols directory). Each directory will +have a .pd\_ file. In a command shell run: "expand /r foo.pd\_". Then +point Visual Studio to this directory. + +Then you'll be able to examine: + ++------------------+-------------------------------------------------------------------------+ +| stack trace | The debugger shows the stack trace. You can right-click on any frame | +| | in the stack, and then choose "go to disassembly" or "go to source". | +| | (Choosing "go to disassembly" from the source view might not get you | +| | to the right place due to optimizations.) When looking at the | +| | source, beware that the debugging information will associate all | +| | inlined functions as part of the line into which they were inlined, | +| | and compiler (with PGO) inlines *very* aggressively (including | +| | inlining virtual functions). You can often figure out where you | +| | *really* are by reading the disassembly. | ++------------------+-------------------------------------------------------------------------+ +| registers | In the Registers tab (Debug->Windows->Registers if you don't have | +| | it open), you can look at the registers associated with each stack | +| | frame, but only at the current state (i.e., the time of the crash). | +| | Registers that Visual Studio can't figure out will be grayed-out and | +| | have the value 00000000. | ++------------------+-------------------------------------------------------------------------+ +| stack memory | You open a window (Memory 1, etc.) that shows contiguous segments of | +| | memory using the Debug->Windows->Memory menu item. You can then | +| | enter the address of the stack pointer (ESP register) in this window | +| | and look at the memory on the stack. (The minidump doesn't have the | +| | memory on the heap.) It's a good idea to change the "width" dropdown | +| | in the top right corner of the window from its default "Auto" to | +| | either "8" or "16" so that the memory display is word-aligned. If | +| | you're interested in pointers, which is usually the case, you can | +| | right click in this window and change the display to show 4-byte | +| | words (so that you don't have to reverse the order due to | +| | little-endianness). This view, combined with the disassembly, can | +| | often be used to reconstruct information beyond what in shown the | +| | function parameters. | ++------------------+-------------------------------------------------------------------------+ +| local variables | In the Watch 1 (etc.) window (which, if you don't have open, you can | +| | iget from Debug->Windows->Watch), you can type an expression | +| | (e.g., the name of a local variable) and the debugger will show you | +| | its value (although it sometimes gets confused). If you're looking | +| | at a pointer to a variable that happens to be on the stack, you can | +| | even examine member variables by just typing expressions. If Visual | +| | Studio can't figure something out from the minidump, it might show | +| | you 00000000 (is this true?). | ++------------------+-------------------------------------------------------------------------+ + +Using minidump-2-core on Linux +------------------------------ + +The `Breakpad +source <https://chromium.googlesource.com/breakpad/breakpad/+/master/>`__ +contains a tool called +`minidump-2-core <https://chromium.googlesource.com/breakpad/breakpad/+/master/src/tools/linux/md2core/>`__, +which converts Linux minidumps into core files. If you checkout and +build Breakpad, the binary will be at +``src/tools/linux/md2core/minidump-2-core``. Running the binary with the +path to a Linux minidump will generate a core file on stdout which can +then be loaded in gdb as usual. You will need to manually download the +matching Firefox binaries, but then you can use the :ref:`GDB Python +script <Downloading symbols on Linux / Mac OS X>` to download symbols. + +The ``minidump-2-core`` source does not currently handle processing +minidumps from a different CPU architecture than the system it was +built for. If you want to use it on an ARM dump, for example, you may +need to build the tool for ARM and run it under QEMU. + +Using other tools to inspect minidump data +------------------------------------------ + +Ted has a few tools that can be built against an already-built copy of +Breakpad to do more targeted inspection. All of these tools assume you +have checked out their source in a directory next to the breakpad +checkout, and that you have built Breakpad in an objdir named +``obj-breakpad`` at the same level. + +- `stackwalk-http <https://hg.mozilla.org/users/tmielczarek_mozilla.com/stackwalk-http/>`__ + is a version of the breakpad's minidump_stackwalk that can fetch symbols + over HTTP, and also has the Mozilla symbol server URL baked in. If you + run it like ``stackwalk /path/to/dmp /tmp/syms`` it will print the stack + trace and save the symbols it downloaded in ``/tmp/syms``. Note that + symbols are only uploaded to the symbol server for nightly and + release builds, not per-change builds. +- `dump-lookup <https://hg.mozilla.org/users/tmielczarek_mozilla.com/dump-lookup/>`__ + takes a minidump and prints values on the stack that are potential + return addresses. This is useful when a stack trace looks truncated + or otherwise wrong. It needs symbol files to produce useful output, + so you will generally want to have run ``stackwalk-http`` to download + them first. +- `get-minidump-instructions <https://hg.mozilla.org/users/tmielczarek_mozilla.com/get-minidump-instructions/>`__ + retrieves and displays the memory range surrounding the faulting + instruction pointer from a minidump. You will almost always want to + run it with the ``--disassemble`` option, which will make it send the + bytes through ``objdump`` to display the disassembled instructions. + If you also give it a path to symbols (see ``stackwalk-http`` above) + it can download the matching source files from hg.mozilla.org and + display source interleaved with the disassembly. +- `minidump-modules <http://hg.mozilla.org/users/tmielczarek_mozilla.com/minidump-modules>`__ + takes a minidump and prints the list of modules from the crashed + process. It will print the full path to each module, whereas the + Socorro UI only prints the filename for each module for privacy + reasons. It also accepts a -v option to print the debug ID for each + module, and a -d option to print relative paths to the symbol files + that would be used instead of the module filenames. + +Getting a stack trace from a crashed B2G process +------------------------------------------------ + +#. Get the minidump file in the phone at + /data/b2g/mozilla/\*.default/minidump/. You can use `adb + pull <http://developer.android.com/tools/help/adb.html>`__ for that. +#. Build the debug symbols using the command ./build.sh buildsymbols + inside the B2G tree. The symbol files will be generated in + $OBJDIR/dist/crashreporter-symbols. +#. Build and install + `google-breakpad <https://code.google.com/p/google-breakpad/>`__. +#. Use the + `minidump-stackwalk <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk>`__ + tool to get the stack trace. + +.. code:: bash + + Example: + + $ cd B2G + $ adb pull /data/b2g/mozilla/*.default/minidump/*.dmp . + $ls *.dmp + 71788789-197e-d769-67167423-4e7aef32.dmp + $ minidump-stackwalk 71788789-197e-d769-67167423-4e7aef32.dmp objdir-debug/dist/crashreporter-symbols/ |