summaryrefslogtreecommitdiffstats
path: root/docs/performance/memory/dmd.md
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--docs/performance/memory/dmd.md489
1 files changed, 489 insertions, 0 deletions
diff --git a/docs/performance/memory/dmd.md b/docs/performance/memory/dmd.md
new file mode 100644
index 0000000000..ebd6b5a2f8
--- /dev/null
+++ b/docs/performance/memory/dmd.md
@@ -0,0 +1,489 @@
+# Dark Matter Detector (DMD)
+
+DMD (short for "dark matter detector") is a heap profiler within
+Firefox. It has four modes.
+
+- "Dark Matter" mode. In this mode, DMD tracks the contents of the
+ heap, including which heap blocks have been reported by memory
+ reporters. It helps us reduce the "heap-unclassified" value in
+ Firefox's about:memory page, and also detects if any heap blocks
+ are reported twice. Originally, this was the only mode that DMD had,
+ which explains DMD's name. This is the default mode.
+- "Live" mode. In this mode, DMD tracks the current contents of the
+ heap. You can dump that information to file, giving a profile of the
+ live heap blocks at that point in time. This is good for
+ understanding how memory is used at an interesting point in time,
+ such as peak memory usage.
+- "Cumulative" mode. In this mode, DMD tracks both the past and
+ current contents of the heap. You can dump that information to file,
+ giving a profile of the heap usage for the entire session. This is
+ good for finding parts of the code that cause high heap churn, e.g.
+ by allocating many short-lived allocations.
+- "Heap scanning" mode. This mode is like live mode, but it also
+ records the contents of every live block in the log. This can be
+ used to investigate leaks by figuring out which objects might be
+ holding references to other objects.
+
+## Building and Running
+
+### Nightly Firefox
+
+The easiest way to use DMD is with the normal Nightly Firefox build,
+which has DMD already enabled in the build. To have DMD active while
+running it, you just need to set the environment variable `DMD=1` when
+running. For instance, on OSX, you can run something like:
+
+ DMD=1 /Applications/Firefox\ Nightly.app/Contents/MacOS/firefox
+
+You can tell it is working by going to `about:memory` and looking for
+"Save DMD Output". If DMD has been properly enabled, the "Save"
+button won't be grayed out. Look at the "Trigger" section below to
+see the full list of ways to get a DMD report once you have it
+activated. Note that the stack information you get will likely be less
+detailed, due to being unable to symbolicate. You will be able to get
+function names, but not line numbers.
+
+### Desktop Firefox
+
+#### Build
+
+Build Firefox with this option added to your mozconfig:
+
+ ac_add_options --enable-dmd
+
+If building via try server, modify
+`browser/config/mozconfigs/linux64/common-opt` or a similar file before
+pushing.
+
+#### Launch
+
+Use `mach run --dmd`; use `--mode` to choose the mode.
+
+On a Windows build done by the try server, [these
+instructions](https://bugzilla.mozilla.org/show_bug.cgi?id=936784#c69) from
+2013 may or may not be useful.
+
+#### Trigger
+
+There are a few ways to trigger a DMD snapshot. Most of these will also
+first get a memory report. When DMD is working on writing its output, it
+will print logging like this:
+
+ DMD[5222] opened /tmp/dmd-1414556492-5222.json.gz for writing
+ DMD[5222] Dump 1 {
+ DMD[5222] Constructing the heap block list...
+ DMD[5222] Constructing the stack trace table...
+ DMD[5222] Constructing the stack frame table...
+ DMD[5222] }
+
+You'll see separate output for each process. This step can take 10 or
+more seconds and may make Firefox freeze temporarily.
+
+If you see the "opened" line, it tells you where the file was saved.
+It's always in a temp directory, and the filenames are always of the
+form dmd-<pid>.
+
+The ways to trigger a DMD snapshot are:
+
+1. Visit about:memory and click the "Save" button under "Save DMD output".
+ The button won't be present in non-DMD builds, and will be grayed out
+ in DMD builds if DMD isn't enabled at start-up.
+
+2. If you wish to trigger DMD dumps from within C++ or JavaScript code,
+ you can use `nsIMemoryInfoDumper.dumpMemoryInfoToTempDir`. For example,
+ from JavaScript code you can do the following.
+
+ const Cc = Components.classes;
+ let mydumper = Cc["@mozilla.org/memory-info-dumper;1"]
+ .getService(Ci.nsIMemoryInfoDumper);
+ mydumper.dumpMemoryInfoToTempDir(identifier, anonymize, minimize);
+
+ This will dump memory reports and DMD output to the temporary
+ directory. `identifier` is a string that will be used for part of
+ the filename (or a timestamp will be used if it is an empty string);
+ `anonymize` is a boolean that indicates if the memory reports should
+ be anonymized; and `minimize` is a boolean that indicates if memory
+ usage should be minimized first.
+
+3. On Linux, you can send signal 34 to the firefox process, e.g.
+ with the following command.
+
+ $ killall -34 firefox
+
+4. The `MOZ_DMD_SHUTDOWN_LOG` environment variable, if set, triggers a DMD
+ run at shutdown; its value must be a directory where the logs will be
+ placed. This is mostly useful for debugging leaks. Which processes get
+ logged is controlled by the `MOZ_DMD_LOG_PROCESS` environment variable.
+ If this is not set, it will log all processes. It can be set to any valid
+ value of `XRE_GetProcessTypeString()` and will log only those processes.
+ For instance, if set to `default` it will only log the parent process. If
+ set to `tab`, it will log content processes only.
+
+ For example, if you have
+
+ MOZ_DMD_SHUTDOWN_LOG=~/dmdlogs/ MOZ_DMD_LOG_PROCESS=tab
+
+ then DMD will create logs at shutdown for content processes and save them to
+ `~/dmdlogs/`.
+
+**NOTE:**
+
+- To dump DMD data from content processes, you'll need to disable the
+ sandbox with `MOZ_DISABLE_CONTENT_SANDBOX=1`.
+- MOZ_DMD_SHUTDOWN_LOG must (currently) include the trailing separator
+ (\'\'/\")
+
+
+### Fennec
+
+**NOTE:**
+
+You'll note from the name of this section being "Fennec" that these instructions
+are very old. Hopefully they'll be more useful than not having them.
+
+**NOTE:**
+
+In order to use DMD on Fennec you will need root access on the Android
+device. Instructions on how to root your device is outside the scope of
+this document.
+
+
+#### Build
+
+Build with these options:
+
+ ac_add_options --enable-dmd
+
+#### Prep
+
+In order to prepare your device for running Fennec with DMD enabled, you
+will need to do a few things. First, you will need to push the libdmd.so
+library to the device so that it can by dynamically loaded by Fennec.
+You can do this by running:
+
+ adb push $OBJDIR/dist/bin/libdmd.so /sdcard/
+
+Second, you will need to make an executable wrapper for Fennec which
+sets an environment variable before launching it. (If you are familiar
+with the recommended "--es env0" method for setting environment
+variables when launching Fennec, note that you cannot use this method
+here because those are processed too late in the startup process. If you
+are not familiar with that method, you can ignore this parenthetical
+note.) First make the executable wrapper on your host machine using the
+editor of your choice. Name the file dmd_fennec and enter this as the
+contents:
+
+ #!/system/bin/sh
+ export MOZ_REPLACE_MALLOC_LIB=/sdcard/libdmd.so
+ exec "$@"
+
+If you want to use other DMD options, you can enter additional
+environment variables above. You will need to push this to the device
+and make it executable. Since you cannot mark files in /sdcard/ as
+executable, we will use /data/local/tmp for this purpose:
+
+ adb push dmd_fennec /data/local/tmp
+ adb shell
+ cd /data/local/tmp
+ chmod 755 dmd_fennec
+
+The final step is to make Android use the above wrapper script while
+launching Fennec, so that the environment variable is present when
+Fennec starts up. Assuming you have done a local build, the app
+identifier will be `org.mozilla.fennec_$USERNAME` (`$USERNAME` is your
+username on the host machine) and so we do this as shown below. If you
+are using a DMD-enabled try build, or build from other source, adjust
+the app identifier as necessary.
+
+ adb shell
+ su # You need root access for the setprop command to take effect
+ setprop wrap.org.mozilla.fennec_$USERNAME "/data/local/tmp/dmd_fennec"
+
+Once this is set up, starting the `org.mozilla.fennec_$USERNAME` app
+will use the wrapper script.
+
+#### Launch
+
+Launch Fennec either by tapping on the icon as usual, or from the
+command line (as before, be sure to replace
+`org.mozilla.fennec_$USERNAME` with the app identifier as appropriate).
+
+ adb shell am start -n org.mozilla.fennec_$USERNAME/.App
+
+#### Trigger
+
+Use the existing memory-report dumping hook:
+
+ adb shell am broadcast -a org.mozilla.gecko.MEMORY_DUMP
+
+In logcat, you should see output similar to this:
+
+ I/DMD (20731): opened /storage/emulated/0/Download/memory-reports/dmd-default-20731.json.gz for writing
+ ...
+ I/GeckoConsole(20731): nsIMemoryInfoDumper dumped reports to /storage/emulated/0/Download/memory-reports/unified-memory-report-default-20731.json.gz
+
+The path is where the memory reports and DMD reports get dumped to. You
+can pull them like so:
+
+ adb pull /sdcard/Download/memory-reports/dmd-default-20731.json.gz
+ adb pull /sdcard/Download/memory-reports/unified-memory-report-default-20731.json.gz
+
+## Processing the output
+
+DMD outputs one gzipped JSON file per process that contains a
+description of that process's heap. You can analyze these files (either
+gzipped or not) using `dmd.py`. On Nightly Firefox, `dmd.py` is included
+in the distribution. For instance on OS X, it is located in the
+directory `/Applications/Firefox Nightly.app/Contents/Resources/`. For
+Nightly, symbolication will fail, but you can at least get some
+information. In a local build, `dmd.py` will be located in the directory
+`$OBJDIR/dist/bin/`.
+
+Some platforms (Linux, Mac, Android) require stack fixing, which adds
+missing filenames, function names and line number information. This will
+occur automatically the first time you run `dmd.py` on the output file.
+This can take 10s of seconds or more to complete. (This will fail if
+your build does not contain symbols. However, if you have crash reporter
+symbols for your build -- as tryserver builds do -- you can use [this
+script](https://github.com/mstange/analyze-tryserver-profiles/blob/master/resymbolicate_dmd.py)
+instead: clone the whole repo, edit the paths at the top of
+`resymbolicate_dmd.py` and run it.) The simplest way to do this is to
+just run the `dmd.py` script on your DMD report while your working
+directory is `$OBJDIR/dist/bin`. This will allow the local libraries to
+be found and used.
+
+If you invoke `dmd.py` without arguments you will get output appropriate
+for the mode in which DMD was invoked.
+
+### "Dark matter" mode output
+
+For "dark matter" mode, `dmd.py`'s output describes how the live heap
+blocks are covered by memory reports. This output is broken into
+multiple sections.
+
+1. "Invocation". This tells you how DMD was invoked, i.e. what
+ options were used.
+2. "Twice-reported stack trace records". This tells you which heap
+ blocks were reported twice or more. The presence of any such records
+ indicates bugs in one or more memory reporters.
+3. "Unreported stack trace records". This tells you which heap blocks
+ were not reported, which indicate where additional memory reporters
+ would be most helpful.
+4. "Once-reported stack trace records": like the "Unreported stack
+ trace records" section, but for blocks reported once.
+5. "Summary": gives measurements of the total heap, and the
+ unreported/once-reported/twice-reported portions of it.
+
+The "Twice-reported stack trace records" and "Unreported stack trace
+records" sections are the most important, because they indicate ways in
+which the memory reporters can be improved.
+
+Here's an example stack trace record from the "Unreported stack trace
+records" section.
+
+ Unreported {
+ 150 blocks in heap block record 283 of 5,495
+ 21,600 bytes (20,400 requested / 1,200 slop)
+ Individual block sizes: 144 x 150
+ 0.00% of the heap (16.85% cumulative)
+ 0.02% of unreported (94.68% cumulative)
+ Allocated at {
+ #01: replace_malloc (/home/njn/moz/mi5/go64dmd/memory/replace/dmd/../../../../memory/replace/dmd/DMD.cpp:1286)
+ #02: malloc (/home/njn/moz/mi5/go64dmd/memory/build/../../../memory/build/replace_malloc.c:153)
+ #03: moz_xmalloc (/home/njn/moz/mi5/memory/mozalloc/mozalloc.cpp:84)
+ #04: nsCycleCollectingAutoRefCnt::incr(void*, nsCycleCollectionParticipant*) (/home/njn/moz/mi5/go64dmd/dom/xul/../../dist/include/nsISupportsImpl.h:250)
+ #05: nsXULElement::Create(nsXULPrototypeElement*, nsIDocument*, bool, bool,mozilla::dom::Element**) (/home/njn/moz/mi5/dom/xul/nsXULElement.cpp:287)
+ #06: nsXBLContentSink::CreateElement(char16_t const**, unsigned int, mozilla::dom::NodeInfo*, unsigned int, nsIContent**, bool*, mozilla::dom::FromParser) (/home/njn/moz/mi5/dom/xbl/nsXBLContentSink.cpp:874)
+ #07: nsCOMPtr<nsIContent>::StartAssignment() (/home/njn/moz/mi5/go64dmd/dom/xml/../../dist/include/nsCOMPtr.h:753)
+ #08: nsXMLContentSink::HandleStartElement(char16_t const*, char16_t const**, unsigned int, unsigned int, bool) (/home/njn/moz/mi5/dom/xml/nsXMLContentSink.cpp:1007)
+ }
+ }
+
+It tells you that there were 150 heap blocks that were allocated from
+the program point indicated by the "Allocated at" stack trace, that
+these blocks took up 21,600 bytes, that all 150 blocks had a size of 144
+bytes, and that 1,200 of those bytes were "slop" (wasted space caused
+by the heap allocator rounding up request sizes). It also indicates what
+percentage of the total heap size and the unreported portion of the heap
+these blocks represent.
+
+Within each section, records are listed from largest to smallest.
+
+Once-reported and twice-reported stack trace records also have stack
+traces for the report point(s). For example:
+
+ Reported at {
+ #01: mozilla::dmd::Report(void const*) (/home/njn/moz/mi2/memory/replace/dmd/DMD.cpp:1740) 0x7f68652581ca
+ #02: CycleCollectorMallocSizeOf(void const*) (/home/njn/moz/mi2/xpcom/base/nsCycleCollector.cpp:3008) 0x7f6860fdfe02
+ #03: nsPurpleBuffer::SizeOfExcludingThis(unsigned long (*)(void const*)) const (/home/njn/moz/mi2/xpcom/base/nsCycleCollector.cpp:933) 0x7f6860fdb7af
+ #04: nsCycleCollector::SizeOfIncludingThis(unsigned long (*)(void const*), unsigned long*, unsigned long*, unsigned long*, unsigned long*, unsigned long*) const (/home/njn/moz/mi2/xpcom/base/nsCycleCollector.cpp:3029) 0x7f6860fdb6b1
+ #05: CycleCollectorMultiReporter::CollectReports(nsIMemoryMultiReporterCallback*, nsISupports*) (/home/njn/moz/mi2/xpcom/base/nsCycleCollector.cpp:3075) 0x7f6860fde432
+ #06: nsMemoryInfoDumper::DumpMemoryReportsToFileImpl(nsAString_internal const&) (/home/njn/moz/mi2/xpcom/base/nsMemoryInfoDumper.cpp:626) 0x7f6860fece79
+ #07: nsMemoryInfoDumper::DumpMemoryReportsToFile(nsAString_internal const&, bool, bool) (/home/njn/moz/mi2/xpcom/base/nsMemoryInfoDumper.cpp:344) 0x7f6860febaf9
+ #08: mozilla::(anonymous namespace)::DumpMemoryReportsRunnable::Run() (/home/njn/moz/mi2/xpcom/base/nsMemoryInfoDumper.cpp:58) 0x7f6860fefe03
+ }
+
+You can tell which memory reporter made the report by the name of the
+`MallocSizeOf` function near the top of the stack trace. In this case it
+was the cycle collector's reporter.
+
+By default, DMD does not record an allocation stack trace for most
+blocks, to make it run faster. The decision on whether to record is done
+probabilistically, and larger blocks are more likely to have an
+allocation stack trace recorded. All unreported blocks that lack an
+allocation stack trace will end up in a single record. For example:
+
+ Unreported {
+ 420,010 blocks in heap block record 2 of 5,495
+ 29,203,408 bytes (27,777,288 requested / 1,426,120 slop)
+ Individual block sizes: 2,048 x 3; 1,024 x 103; 512 x 147; 496 x 7; 480 x 31; 464 x 6; 448 x 50; 432 x 41; 416 x 28; 400 x 53; 384 x 43; 368 x 216; 352 x 141; 336 x 58; 320 x 104; 304 x 5,130; 288 x 150; 272 x 591; 256 x 6,017; 240 x 1,372; 224 x 93; 208 x 488; 192 x 1,919; 176 x 18,903; 160 x 1,754; 144 x 5,041; 128 x 36,709; 112 x 5,571; 96 x 6,280; 80 x 40,738; 64 x 37,925; 48 x 78,392; 32 x 136,199; 16 x 31,001; 8 x 4,706
+ 3.78% of the heap (10.24% cumulative)
+ 21.24% of unreported (57.53% cumulative)
+ Allocated at {
+ #01: (no stack trace recorded due to --stacks=partial)
+ }
+ }
+
+In contrast, stack traces are always recorded when a block is reported,
+which means you can end up with records like this where the allocation
+point is unknown but the reporting point *is* known:
+
+ Once-reported {
+ 104,491 blocks in heap block record 13 of 4,689
+ 10,392,000 bytes (10,392,000 requested / 0 slop)
+ Individual block sizes: 512 x 124; 256 x 242; 192 x 813; 128 x 54,664; 64 x 48,648
+ 1.35% of the heap (48.65% cumulative)
+ 1.64% of once-reported (59.18% cumulative)
+ Allocated at {
+ #01: (no stack trace recorded due to --stacks=partial)
+ }
+ Reported at {
+ #01: mozilla::dmd::DMDFuncs::Report(void const*) (/home/njn/moz/mi5/go64dmd/memory/replace/dmd/../../../../memory/replace/dmd/DMD.cpp:1646)
+ #02: WindowsMallocSizeOf(void const*) (/home/njn/moz/mi5/dom/base/nsWindowMemoryReporter.cpp:189)
+ #03: nsAttrAndChildArray::SizeOfExcludingThis(unsigned long (*)(void const*)) const (/home/njn/moz/mi5/dom/base/nsAttrAndChildArray.cpp:880)
+ #04: mozilla::dom::FragmentOrElement::SizeOfExcludingThis(unsigned long (*)(void const*)) const (/home/njn/moz/mi5/dom/base/FragmentOrElement.cpp:2337)
+ #05: nsINode::SizeOfIncludingThis(unsigned long (*)(void const*)) const (/home/njn/moz/mi5/go64dmd/parser/html/../../../dom/base/nsINode.h:307)
+ #06: mozilla::dom::NodeInfo::NodeType() const (/home/njn/moz/mi5/go64dmd/dom/base/../../dist/include/mozilla/dom/NodeInfo.h:127)
+ #07: nsHTMLDocument::DocAddSizeOfExcludingThis(nsWindowSizes*) const (/home/njn/moz/mi5/dom/html/nsHTMLDocument.cpp:3710)
+ #08: nsIDocument::DocAddSizeOfIncludingThis(nsWindowSizes*) const (/home/njn/moz/mi5/dom/base/nsDocument.cpp:12820)
+ }
+ }
+
+The choice of whether to record an allocation stack trace for all blocks
+is controlled by an option (see below).
+
+### "Live" mode output
+
+
+For "live" mode, dmd.py's output describes what live heap blocks are
+present. This output is broken into multiple sections.
+
+1. "Invocation". This tells you how DMD was invoked, i.e. what
+ options were used.
+2. "Live stack trace records". This tells you which heap blocks were
+ present.
+3. "Summary": gives measurements of the total heap.
+
+The individual records are similar to those output in "dark matter"
+mode.
+
+### "Cumulative" mode output
+
+For "cumulative" mode, dmd.py's output describes how the live heap
+blocks are covered by memory reports. This output is broken into
+multiple sections.
+
+1. "Invocation". This tells you how DMD was invoked, i.e. what
+ options were used.
+2. "Cumulative stack trace records". This tells you which heap blocks
+ were allocated during the session.
+3. "Summary": gives measurements of the total (cumulative) heap.
+
+The individual records are similar to those output in "dark matter"
+mode.
+
+### "Scan" mode output
+
+For "scan" mode, the output of `dmd.py` is the same as "live" mode.
+A separate script, `block_analyzer.py`, can be used to find out
+information about which blocks refer to a particular block.
+`dmd.py --clamp-contents` needs to be run on the log first. See [this
+other page](heap_scan_mode.md) for an
+overview of how to use heap scan mode to fix a leak involving refcounted
+objects.
+
+## Options
+
+### Runtime
+
+When you run `mach run --dmd` you can specify additional options to
+control how DMD runs. Run `mach help run` for documentation on these.
+
+The most interesting one is `--mode`. Acceptable values are
+`dark-matter` (the default), `live`, `cumulative`, and `scan`.
+
+Another interesting one is `--stacks`. Acceptable values are `partial`
+(the default) and `full`. In the former case most blocks will not have
+an allocation stack trace recorded. However, because larger blocks are
+more likely to have one recorded, most allocated bytes should have an
+allocation stack trace even though most allocated blocks do not. Use
+`--stacks=full` if you want complete information, but note that DMD will
+run substantially slower in that case.
+
+The options may also be put in the environment variable DMD, or set DMD
+to 1 to enable DMD with default options (dark-matter and partial
+stacks).
+
+### Post-processing
+
+`dmd.py` also takes options that control how it works. Run `dmd.py -h`
+for documentation. The following options are the most interesting ones.
+
+- `-f` / `--max-frames`. By default, records show up to 8 stack
+ frames. You can choose a smaller number, in which case more
+ allocations will be aggregated into each record, but you'll have
+ less context. Or you can choose a larger number, in which cases
+ allocations will be split across more records, but you will have
+ more context. There is no single best value, but values in the range
+ 2..10 are often good. The maximum is 24.
+
+- `-a` / `--ignore-alloc-fns`. Many allocation stack traces start
+ with multiple frames that mention allocation wrapper functions, e.g.
+ `js_calloc()` calls `replace_calloc()`. This option filters these
+ out. It often helps improve the quality of the output when using a
+ small `--max-frames` value.
+
+- `-s` / `--sort-by`. This controls how records are sorted. Acceptable
+ values are `usable` (the default), `req`, `slop` and `num-blocks`.
+
+- `--clamp-contents`. For a heap scan log, this performs a
+ conservative pointer analysis on the contents of each block,
+ changing any value that is a pointer into the middle of a live block
+ into a pointer to the start of that block. All other values are
+ changes to null. In addition, all trailing nulls are removed from
+ the block contents.
+
+As an example that combines multiple options, if you apply the following
+command to a profile obtained in "live" mode:
+
+ dmd.py -r -f 2 -a -s slop
+
+it will give you a good idea of where the major sources of slop are.
+
+`dmd.py` can also compute the difference between two DMD output files,
+so long as those files were produced in the same mode. Simply pass it
+two filenames instead of one to get the difference.
+
+## Which heap blocks are reported?
+
+At this stage you might wonder how DMD knows, in "dark matter" mode,
+which allocations have been reported and which haven't. DMD only knows
+about heap blocks that are measured via a function created with one of
+the following two macros:
+
+ MOZ_DEFINE_MALLOC_SIZE_OF
+ MOZ_DEFINE_MALLOC_SIZE_OF_ON_ALLOC
+
+Fortunately, most of the existing memory reporters do this. See
+[Performance/Memory_Reporting](https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Memory_reporting "Platform/Memory Reporting")
+for more details about how memory reporters are written.