diff options
Diffstat (limited to '')
| -rw-r--r-- | tools/profiler/docs/markers-guide.rst | 485 |
1 files changed, 485 insertions, 0 deletions
diff --git a/tools/profiler/docs/markers-guide.rst b/tools/profiler/docs/markers-guide.rst new file mode 100644 index 0000000000..82fe6f3cda --- /dev/null +++ b/tools/profiler/docs/markers-guide.rst @@ -0,0 +1,485 @@ +Markers +======= + +Markers are packets of arbitrary data that are added to a profile by the Firefox code, usually to +indicate something important happening at a point in time, or during an interval of time. + +Each marker has a name, a category, some common optional information (timing, backtrace, etc.), +and an optional payload of a specific type (containing arbitrary data relevant to that type). + +.. note:: + This guide explains C++ markers in depth. To learn more about how to add a + marker in JavaScript or Rust, please take a look at their documentation + in :doc:`instrumenting-javascript` or :doc:`instrumenting-rust` respectively. + +Example +------- + +Short example, details below. + +Note: Most marker-related identifiers are in the ``mozilla`` namespace, to be added where necessary. + +.. code-block:: c++ + + // Record a simple marker with the category of DOM. + PROFILER_MARKER_UNTYPED("Marker Name", DOM); + + // Create a marker with some additional text information. (Be wary of printf!) + PROFILER_MARKER_TEXT("Marker Name", JS, MarkerOptions{}, "Additional text information."); + + // Record a custom marker of type `ExampleNumberMarker` (see definition below). + PROFILER_MARKER("Number", OTHER, MarkerOptions{}, ExampleNumberMarker, 42); + +.. code-block:: c++ + + // Marker type definition. + struct ExampleNumberMarker { + // Unique marker type name. + static constexpr Span<const char> MarkerTypeName() { return MakeStringSpan("number"); } + // Data specific to this marker type, serialized to JSON for profiler.firefox.com. + static void StreamJSONMarkerData(SpliceableJSONWriter& aWriter, int aNumber) { + aWriter.IntProperty("number", aNumber); + } + // Where and how to display the marker and its data. + static MarkerSchema MarkerTypeDisplay() { + using MS = MarkerSchema; + MS schema(MS::Location::MarkerChart, MS::Location::MarkerTable); + schema.SetChartLabel("Number: {marker.data.number}"); + schema.AddKeyLabelFormat("number", "Number", MS::Format::Number); + return schema; + } + }; + + +How to Record Markers +--------------------- + +Header to Include +^^^^^^^^^^^^^^^^^ + +If the compilation unit only defines and records untyped, text, and/or its own markers, include +`the main profiler markers header <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkers.h>`_: + +.. code-block:: c++ + + #include "mozilla/ProfilerMarkers.h" + +If it also records one of the other common markers defined in +`ProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkerTypes.h>`_, +include that one instead: + +.. code-block:: c++ + + #include "mozilla/ProfilerMarkerTypes.h" + +And if it uses any other profiler functions (e.g., labels), use +`the main Gecko Profiler header <https://searchfox.org/mozilla-central/source/tools/profiler/public/GeckoProfiler.h>`_ +instead: + +.. code-block:: c++ + + #include "GeckoProfiler.h" + +The above works from source files that end up in libxul, which is true for the majority +of Firefox source code. But some files live outside of libxul, such as mfbt, in which +case the advice is the same but the equivalent headers are from the Base Profiler instead: + +.. code-block:: c++ + + #include "mozilla/BaseProfilerMarkers.h" // Only own/untyped/text markers + #include "mozilla/BaseProfilerMarkerTypes.h" // Only common markers + #include "BaseProfiler.h" // Markers and other profiler functions + +Untyped Markers +^^^^^^^^^^^^^^^ + +Untyped markers don't carry any information apart from common marker data: +Name, category, options. + +.. code-block:: c++ + + PROFILER_MARKER_UNTYPED( + // Name, and category pair. + "Marker Name", OTHER, + // Marker options, may be omitted if all defaults are acceptable. + MarkerOptions(MarkerStack::Capture(), ...)); + +``PROFILER_MARKER_UNTYPED`` is a macro that simplifies the use of the main +``profiler_add_marker`` function, by adding the appropriate namespaces, and a surrounding +``#ifdef MOZ_GECKO_PROFILER`` guard. + +1. Marker name + The first argument is the name of this marker. This will be displayed in most places + the marker is shown. It can be a literal C string, or any dynamic string object. +2. `Category pair name <https://searchfox.org/mozilla-central/define?q=M_174bb0de187ee7d9>`_ + Choose a category + subcategory from the `the list of categories <https://searchfox.org/mozilla-central/define?q=M_174bb0de187ee7d9>`_. + This is the second parameter of each ``SUBCATEGORY`` line, for instance ``LAYOUT_Reflow``. + (Internally, this is really a `MarkerCategory <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerCategory>`_ + object, in case you need to construct it elsewhere.) +3. `MarkerOptions <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerOptions>`_ + See the options below. It can be omitted if there are no other arguments, ``{}``, or + ``MarkerOptions()`` (no specified options); only one of the following option types + alone; or ``MarkerOptions(...)`` with one or more of the following options types: + + * `MarkerThreadId <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerThreadId>`_ + Rarely used, as it defaults to the current thread. Otherwise it specifies the target + "thread id" (aka "track") where the marker should appear; This may be useful when + referring to something that happened on another thread (use ``profiler_current_thread_id()`` + from the original thread to get its id); or for some important markers, they may be + sent to the "main thread", which can be specified with ``MarkerThreadId::MainThread()``. + * `MarkerTiming <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerTiming>`_ + This specifies an instant or interval of time. It defaults to the current instant if + left unspecified. Otherwise use ``MarkerTiming::InstantAt(timestamp)`` or + ``MarkerTiming::Interval(ts1, ts2)``; timestamps are usually captured with + ``TimeStamp::Now()``. It is also possible to record only the start or the end of an + interval, pairs of start/end markers will be matched by their name. *Note: The + upcoming "marker sets" feature will make this pairing more reliable, and also + allow more than two markers to be connected*. + * `MarkerStack <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerStack>`_ + By default, markers do not record a "stack" (or "backtrace"). To record a stack at + this point, in the most efficient manner, specify ``MarkerStack::Capture()``. To + record a previously captured stack, first store a stack into a + ``UniquePtr<ProfileChunkedBuffer>`` with ``profiler_capture_backtrace()``, then pass + it to the marker with ``MarkerStack::TakeBacktrace(std::move(stack))``. + * `MarkerInnerWindowId <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerInnerWindowId>`_ + If you have access to an "inner window id", consider specifying it as an option, to + help profiler.firefox.com to classify them by tab. + +Text Markers +^^^^^^^^^^^^ + +Text markers are very common, they carry an extra text as a fourth argument, in addition to +the marker name. Use the following macro: + +.. code-block:: c++ + + PROFILER_MARKER_TEXT( + // Name, category pair, options. + "Marker Name", OTHER, {}, + // Text string. + "Here are some more details." + ); + +As useful as it is, using an expensive ``printf`` operation to generate a complex text +comes with a variety of issues string. It can leak potentially sensitive information +such as URLs can be leaked during the profile sharing step. profiler.firefox.com cannot +access the information programmatically. It won't get the formatting benefits of the +built-in marker schema. Please consider using a custom marker type to separate and +better present the data. + +Other Typed Markers +^^^^^^^^^^^^^^^^^^^ + +From C++ code, a marker of some type ``YourMarker`` (details about type definition follow) can be +recorded like this: + +.. code-block:: c++ + + PROFILER_MARKER( + "YourMarker name", OTHER, + MarkerOptions(MarkerTiming::IntervalUntilNowFrom(someStartTimestamp), + MarkerInnerWindowId(innerWindowId))), + YourMarker, "some string", 12345, "http://example.com", someTimeStamp); + +After the first three common arguments (like in ``PROFILER_MARKER_UNTYPED``), there are: + +4. The marker type, which is the name of the C++ ``struct`` that defines that type. +5. A variadic list of type-specific argument. They must match the number of, and must + be convertible to, ``StreamJSONMarkerData`` parameters as specified in the marker type definition. + +"Auto" Scoped Interval Markers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To capture time intervals around some important operations, it is common to store a timestamp, do the work, +and then record a marker, e.g.: + +.. code-block:: c++ + + void DoTimedWork() { + TimeStamp start = TimeStamp::Now(); + DoWork(); + PROFILER_MARKER_TEXT("Timed work", OTHER, MarkerTiming::IntervalUntilNowFrom(start), "Details"); + } + +`RAII <https://en.cppreference.com/w/cpp/language/raii>`_ objects automate this, by recording the time +when the object is constructed, and later recording the marker when the object is destroyed at the end +of its C++ scope. +This is especially useful if there are multiple scope exit points. + +``AUTO_PROFILER_MARKER_TEXT`` is `the only one implemented <https://searchfox.org/mozilla-central/define?q=M_ac7b392646edf5a5>`_ at this time. + +.. code-block:: c++ + + void MaybeDoTimedWork(bool aDoIt) { + AUTO_PROFILER_MARKER_TEXT("Timed work", OTHER, "Details"); + if (!aDoIt) { /* Marker recorded here... */ return; } + DoWork(); + /* ... or here. */ + } + +Note that these RAII objects only record one marker. In some situation, a very long +operation could be missed if it hasn't completed by the end of the profiling session. +In this case, consider recording two distinct markers, using +``MarkerTiming::IntervalStart()`` and ``MarkerTiming::IntervalEnd()``. + +Where to Define New Marker Types +-------------------------------- + +The first step is to determine the location of the marker type definition: + +* If this type is only used in one function, or a component, it can be defined in a + local common place relative to its use. +* For a more common type that could be used from multiple locations: + + * If there is no dependency on XUL, it can be defined in the Base Profiler, which can + be used in most locations in the codebase: + `mozglue/baseprofiler/public/BaseProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/mozglue/baseprofiler/public/BaseProfilerMarkerTypes.h>`__ + + * However, if there is a XUL dependency, then it needs to be defined in the Gecko Profiler: + `tools/profiler/public/ProfilerMarkerTypes.h <https://searchfox.org/mozilla-central/source/tools/profiler/public/ProfilerMarkerTypes.h>`__ + +.. _how-to-define-new-marker-types: + +How to Define New Marker Types +------------------------------ + +Each marker type must be defined once and only once. +The definition is a C++ ``struct``, its identifier is used when recording +markers of that type in C++. +By convention, the suffix "Marker" is recommended to better distinguish them +from non-profiler entities in the source. + +.. code-block:: c++ + + struct YourMarker { + +Marker Type Name +^^^^^^^^^^^^^^^^ + +A marker type must have a unique name, it is used to keep track of the type of +markers in the profiler storage, and to identify them uniquely on profiler.firefox.com. +(It does not need to be the same as the ``struct``'s name.) + +This name is defined in a special static member function ``MarkerTypeName``: + +.. code-block:: c++ + + // … + static constexpr Span<const char> MarkerTypeName() { + return MakeStringSpan("YourMarker"); + } + +Marker Type Data +^^^^^^^^^^^^^^^^ + +All markers of any type have some common data: A name, a category, options like +timing, etc. as previously explained. + +In addition, a certain marker type may carry zero of more arbitrary pieces of +information, and they are always the same for all markers of that type. + +These are defined in a special static member function ``StreamJSONMarkerData``. + +The first function parameters is always ``SpliceableJSONWriter& aWriter``, +it will be used to stream the data as JSON, to later be read by +profiler.firefox.com. + +.. code-block:: c++ + + // … + static void StreamJSONMarkerData(SpliceableJSONWriter& aWriter, + +The following function parameters is how the data is received as C++ objects +from the call sites. + +* Most C/C++ `POD (Plain Old Data) <https://en.cppreference.com/w/cpp/named_req/PODType>`_ + and `trivially-copyable <https://en.cppreference.com/w/cpp/named_req/TriviallyCopyable>`_ + types should work as-is, including ``TimeStamp``. +* Character strings should be passed using ``const ProfilerString8View&`` (this handles + literal strings, and various ``std::string`` and ``nsCString`` types, and spans with or + without null terminator). Use ``const ProfilerString16View&`` for 16-bit strings such as + ``nsString``. +* Other types can be used if they define specializations for ``ProfileBufferEntryWriter::Serializer`` + and ``ProfileBufferEntryReader::Deserializer``. You should rarely need to define new + ones, but if needed see how existing specializations are written, or contact the + `perf-tools team for help <https://chat.mozilla.org/#/room/#profiler:mozilla.org>`_. + +Passing by value or by reference-to-const is recommended, because arguments are serialized +in binary form (i.e., there are no optimizable ``move`` operations). + +For example, here's how to handle a string, a 64-bit number, another string, and +a timestamp: + +.. code-block:: c++ + + // … + const ProfilerString8View& aString, + const int64_t aBytes, + const ProfilerString8View& aURL, + const TimeStamp& aTime) { + +Then the body of the function turns these parameters into a JSON stream. + +When this function is called, the writer has just started a JSON object, so +everything that is written should be a named object property. Use +``SpliceableJSONWriter`` functions, in most cases ``...Property`` functions +from its parent class ``JSONWriter``: ``NullProperty``, ``BoolProperty``, +``IntProperty``, ``DoubleProperty``, ``StringProperty``. (Other nested JSON +types like arrays or objects are not supported by the profiler.) + +As a special case, ``TimeStamps`` must be streamed using ``aWriter.TimeProperty(timestamp)``. + +The property names will be used to identify where each piece of data is stored and +how it should be displayed on profiler.firefox.com (see next section). + +Here's how the above functions parameters could be streamed: + +.. code-block:: c++ + + // … + aWriter.StringProperty("myString", aString); + aWriter.IntProperty("myBytes", aBytes); + aWriter.StringProperty("myURL", aURL); + aWriter.TimeProperty("myTime", aTime); + } + +.. _marker-type-display-schema: + +Marker Type Display Schema +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Now that we have defined how to stream type-specific data (from Firefox to +profiler.firefox.com), we need to describe where and how this data will be +displayed on profiler.firefox.com. + +The static member function ``MarkerTypeDisplay`` returns an opaque ``MarkerSchema`` +object, which will be forwarded to profiler.firefox.com. + +.. code-block:: c++ + + // … + static MarkerSchema MarkerTypeDisplay() { + +The ``MarkerSchema`` type will be used repeatedly, so for convenience we can define +a local type alias: + +.. code-block:: c++ + + // … + using MS = MarkerSchema; + +First, we construct the ``MarkerSchema`` object to be returned at the end. + +One or more constructor arguments determine where this marker will be displayed in +the profiler.firefox.com UI. See the `MarkerSchema::Location enumeration for the +full list <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerSchema%3A%3ALocation>`_. + +Here is the most common set of locations, showing markers of that type in both the +Marker Chart and the Marker Table panels: + +.. code-block:: c++ + + // … + MS schema(MS::Location::MarkerChart, MS::Location::MarkerTable); + +Some labels can optionally be specified, to display certain information in different +locations: ``SetChartLabel``, ``SetTooltipLabel``, and ``SetTableLabel``; or +``SetAllLabels`` to define all of them the same way. + +The arguments is a string that may refer to marker data within braces: + +* ``{marker.name}``: Marker name. +* ``{marker.data.X}``: Type-specific data, as streamed with property name "X" from ``StreamJSONMarkerData`` (e.g., ``aWriter.IntProperty("X", aNumber);`` + +For example, here's how to set the Marker Chart label to show the marker name and the +``myBytes`` number of bytes: + +.. code-block:: c++ + + // … + schema.SetChartLabel("{marker.name} – {marker.data.myBytes}"); + +profiler.firefox.com will apply the label with the data in a consistent manner. For +example, with this label definition, it could display marker information like the +following in the Firefox Profiler's Marker Chart: + + * "Marker Name – 10B" + * "Marker Name – 25.204KB" + * "Marker Name – 512.54MB" + +For implementation details on this processing, see `src/profiler-logic/marker-schema.js <https://github.com/firefox-devtools/profiler/blob/main/src/profile-logic/marker-schema.js>`_ +in the profiler's front-end. + +Next, define the main display of marker data, which will appear in the Marker +Chart tooltips and the Marker Table sidebar. + +Each row may either be: + +* A dynamic key-value pair, using one of the ``MarkerSchema::AddKey...`` functions. Each function is given: + + * Key: Element property name as streamed in ``StreamJSONMarkerData``. + * Label: Optional prefix. Defaults to the key name. + * Format: How to format the data element value, see `MarkerSchema::Format for details <https://searchfox.org/mozilla-central/define?q=T_mozilla%3A%3AMarkerSchema%3A%3AFormat>`_. + * Searchable: Optional boolean, indicates if the value is used in searches, defaults to false. + +* Or a fixed label and value strings, using ``MarkerSchema::AddStaticLabelValue``. + +.. code-block:: c++ + + // … + schema.AddKeyLabelFormatSearchable( + "myString", "My String", MS::Format::String, true); + schema.AddKeyLabelFormat( + "myBytes", "My Bytes", MS::Format::Bytes); + schema.AddKeyLabelFormat( + "myUrl", "My URL", MS::Format::Url); + schema.AddKeyLabelFormat( + "myTime", "Event time", MS::Format::Time); + +Finally the ``schema`` object is returned from the function: + +.. code-block:: c++ + + // … + return schema; + } + +Any other ``struct`` member function is ignored. There could be utility functions used by the above +compulsory functions, to make the code clearer. + +And that is the end of the marker definition ``struct``. + +.. code-block:: c++ + + // … + }; + +Performance Considerations +-------------------------- + +During profiling, it is best to reduce the amount of work spent doing profiler +operations, as they can influence the performance of the code that you want to profile. + +Whenever possible, consider passing simple types to marker functions, such that +``StreamJSONMarkerData`` will do the minimum amount of work necessary to serialize +the marker type-specific arguments to its internal buffer representation. POD types +(numbers) and strings are the easiest and cheapest to serialize. Look at the +corresponding ``ProfileBufferEntryWriter::Serializer`` specializations if you +want to better understand the work done. + +Avoid doing expensive operations when recording markers. E.g.: ``printf`` of +different things into a string, or complex computations; instead pass the +``printf``/computation arguments straight through to the marker function, so that +``StreamJSONMarkerData`` can do the expensive work at the end of the profiling session. + +Marker Architecture Description +------------------------------- + +The above sections should give all the information needed for adding your own marker +types. However, if you are wanting to work on the marker architecture itself, this +section will describe how the system works. + +TODO: + * Briefly describe the buffer and serialization. + * Describe the template strategy for generating marker types + * Describe the serialization and link to profiler front-end docs on marker processing (if they exist) |