From 9f153fbfec0fb9c9ce38e749a7c6f4a5e115d4e9 Mon Sep 17 00:00:00 2001
From: Daniel Baumann <daniel.baumann@progress-linux.org>
Date: Thu, 19 Sep 2024 06:14:33 +0200
Subject: Merging upstream version 4.4.0.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
---
 doc/wsug_src/wsug_mate.adoc | 2401 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 2401 insertions(+)
 create mode 100644 doc/wsug_src/wsug_mate.adoc

(limited to 'doc/wsug_src/wsug_mate.adoc')

diff --git a/doc/wsug_src/wsug_mate.adoc b/doc/wsug_src/wsug_mate.adoc
new file mode 100644
index 00000000..9df35551
--- /dev/null
+++ b/doc/wsug_src/wsug_mate.adoc
@@ -0,0 +1,2401 @@
+[#ChMate]
+
+== MATE
+
+[#ChMateIntroduction]
+
+=== Introduction
+
+MATE: Meta Analysis and Tracing Engine
+
+What is MATE? Well, to keep it very short, with MATE you can create user
+configurable extension(s) of the display filter engine.
+
+MATE's goal is to enable users to filter frames based on information extracted
+from related frames or information on how frames relate to each other. MATE
+was written to help troubleshooting gateways and other systems where a "use"
+involves more protocols. However, MATE can be used as well to analyze other
+issues regarding an interaction between packets like response times,
+incompleteness of transactions, presence/absence of certain attributes in a
+group of Protocol Data Units (PDUs) and more.
+
+MATE is a Wireshark plugin that allows the user to specify how different
+frames are related to each other. To do so, MATE extracts data from the frames'
+tree and then, using that information, tries to group the frames based on how
+MATE is configured. Once the PDUs are related, MATE will create a "protocol"
+tree with fields the user can filter with. The fields will be almost the same
+for all the related frames, so one can filter a complete session spanning
+several frames containing more protocols based on an attribute appearing in
+some related frame. Other than that MATE allows to filter frames based on
+response times, number of PDUs in a group and a lot more.
+
+So far MATE has been used to:
+
+* Filter all packets of a call using various protocols knowing just the
+calling number. (MATE's original goal)
+* Filter all packets of all calls using various protocols based on the release
+cause of one of its "segments".
+* Extrapolate slow transactions from very "dense" captures. (finding requests
+that timeout)
+* Find incomplete transactions (no responses)
+* Follow requests through more gateways/proxies.
+* more...
+
+[#ChMateGettingStarted]
+
+=== Getting Started
+
+These are the steps to try out MATE:
+
+* Run Wireshark and check if the plugin is installed (MATE should
+appear in Help->About Wireshark:Plugins)
+* Get a configuration file e.g., <<File_tcp_mate,tcp.mate>> (see {wireshark-wiki-url}Mate/Examples[Mate/Examples]
+for more) and place it somewhere on your harddisk.
+* Go to Edit->Preferences...->Protocols->MATE and set the Configuration Filename to the file
+you want to use and restart Wireshark.
+* Load a corresponding capture file (e.g.,
+{wireshark-wiki-url}uploads/27707187aeb30df68e70c8fb9d614981/http.cap[http.cap]) and see if MATE
+has added some new display filter fields, something like: `mate tcp_pdu:1->tcp_ses:1`
+or, at prompt: `path_to/wireshark -o "mate.config: tcp.mate" -r http.cap`.
+
+If everything went well, your packet details might look something like this:
+
+.Packet Details - MATE TCP Session (tcp.mate)
+image::images/ws-mate-tcp-output.png[]
+
+[#ChMateManual]
+
+=== MATE Overview
+
+==== Introduction
+
+MATE creates a filterable tree based on information contained in frames that
+share some relationship with information obtained from other frames. The way
+these relationships are made is described in a configuration file. The
+configuration file tells MATE what makes a PDU and how to relate it to other
+PDUs.
+
+MATE analyzes each frame to extract relevant information from the "protocol"
+tree of that frame. The extracted information is contained in MATE PDUs;
+these contain a list of relevant attributes taken from the tree. From now on, I
+will use the term "PDU" to refer to the objects created by MATE containing the
+relevant information extracted from the frame; I'll use "frame" to refer to the
+"raw" information extracted by the various dissectors that pre-analyzed the frame.
+
+For every PDU, MATE checks if it belongs to an existing "Group of PDUs" (GOP).
+If it does, it assigns the PDU to that GOP and moves any new relevant attributes
+to the GOP's attribute list. How and when do PDUs belong to GOPs is described
+in the configuration file as well.
+
+Every time a GOP is assigned a new PDU, MATE will check if it matches the
+conditions to make it belong to a "Group of Groups" (GOG). Naturally the
+conditions that make a GOP belong to a GOG are taken from the configuration
+file as well.
+
+Once MATE is done analyzing the frame it will be able to create a "protocol"
+tree for each frame based on the PDUs, the GOPs they belong to and naturally any
+GOGs the former belongs to.
+
+How to tell MATE what to extract, how to group it and then how to relate those
+groups is made using AVPs and AVPLs.
+
+Information in MATE is contained in Attribute Value Pairs (AVPs). AVPs are made
+of two strings: the name and the value. AVPs are used in the configuration and
+there they have an operator as well. There are various ways AVPs can be matched
+against each other using those operators.
+
+AVPs are grouped into AVP Lists (AVPLs). PDUs, GOPs and GOGs have an AVPL each.
+Their AVPLs will be matched in various ways against others coming from the
+configuration file.
+
+MATE will be instructed how to extract AVPs from frames in order to create a PDU
+with an AVPL. It will be instructed as well, how to match that AVPL against the
+AVPLs of other similar PDUs in order to relate them. In MATE the relationship
+between PDUs is a GOP, it has an AVPL as well. MATE will be configured with other
+AVPLs to operate against the GOP's AVPL to relate GOPs together into GOGs.
+
+A good understanding on how AVPs and AVPLs work is fundamental to understand how
+MATE works.
+
+===== About MATE
+
+MATE was originally written by Luis Ontanon, a Telecommunications systems
+troubleshooter, as a way to save time filtering out the packets of a single call
+from huge capture files using just the calling number. Later he used the time he
+had saved to make it flexible enough to work with protocols other than the ones
+he was directly involved with.
+
+[#AVP]
+==== Attribute Value Pairs (AVP)
+
+Information used by MATE to relate different frames is contained in Attribute
+Value Pairs (AVPs). AVPs are made of two strings - the name and the value. When
+AVPs are used in the configuration, an operator is defined as well. There are
+various ways AVPs can be matched against each other using those operators.
+
+----
+  avp_name="avp's value"
+  another_name= "1234 is the value"
+----
+
+The name is a string used to refer to a "type" of an AVP. Two AVPs won't match
+unless their names are identical.
+
+
+The name must start with a lowercase letter (a-z) and can contain only alphanumeric characters
+(a-zA-Z0-9) and the special characters "_", "-", and ".". The name ends with an operator.
+
+You should not use uppercase characters in names, or names that start with “.” or
+“_”. Capitalized names are reserved for configuration parameters (we'll call them
+keywords); nothing forbids you from using capitalized strings for other things as
+well but it probably would be confusing. I'll avoid using capitalized words for
+anything but the keywords in this document, the reference manual, the examples
+and the base library. Names that start with a “.” would be very confusing as well
+because in the old grammar, AVPL transforms use names starting with a “.” to
+indicate they belong to the replacement AVPL.
+
+The value is a string that is either set in the configuration (for configuration
+AVPs) or by Wireshark while extracting interesting fields from a frame's tree.
+The values extracted from fields use the same representation as they do in filter
+strings except that no quotes are used.
+
+*The value will be dealt with as a string even if it is a number.* If there are
+any spaces in the value, the value must be between quotes "".
+Values that are also keywords such as True and False should also be wrapped
+in quotes ("True", "False").
+
+----
+   ip_addr=10.10.10.11
+   tcp_port=1234
+   binary_data=01:23:45:67:89:ab:cd:ef
+   parameter12=0x23aa
+   parameter_with_spaces="this value has spaces"
+----
+
+The way two AVPs with the same name might match is described by the operator.
+Remember two AVPs won't match unless their names are identical. In MATE, match
+operations are always made between the AVPs extracted from frames (called data
+AVPs) and the configuration's AVPs.
+
+Currently defined MATE AVP match operators are:
+
+* <<Equal,Equal>> _=_ will match if the string given completely matches the data
+AVP's value string
+* <<NotEqual,Not Equal>> _!_ will match only if the given value string is not equal to
+the data AVP's value string
+* <<OneOf,One Of>> _{}_ will match if one of the possible strings listed is equal to
+the data AVP's value string
+* <<StartsWith,Starts With>> _^_ will match if the string given matches the first
+characters of the data AVP's value string
+* <<EndsWith,Ends With>> _$_ will match if the string given matches the last characters
+of the data AVP's value string
+* <<Contains,Contains>> _~_ will match if the string given matches any substring of the
+data AVP's value string
+* <<LowerThan,Lower Than>> _<_ will match if the data AVP's value string is semantically
+lower than the string given
+* <<HigherThan,Higher Than>> _>_ will match if the data AVP's value string is semantically
+higher than the string given
+* <<Exists,Exists>> _?_ (the ? can be omitted) will match as far as a data AVP of the
+given name exists
+
+==== AVP lists (AVPL)
+
+An AVPL is a set of diverse AVPs that can be matched against other AVPLs. Every
+PDU, GOP and GOG has an AVPL that contains the information regarding it. The
+rules that MATE uses to group PDUs and GOPs are AVPL operations.
+
+*There will never be two identical AVPs in a given AVPL.* However, we can have
+more than one AVP with the same name in an AVPL as long as their values are
+different.
+
+Some AVPL examples:
+----
+  ( addr=10.20.30.40, addr=192.168.0.1, tcp_port=21, tcp_port=32534, user_cmd=PORT,
+        data_port=12344, data_addr=192.168.0.1 )
+  ( addr=10.20.30.40, addr=192.168.0.1, channel_id=22:23, message_type=Setup,
+        calling_number=1244556673 )
+  ( addr=10.20.30.40, addr=192.168.0.1, ses_id=01:23:45:67:89:ab:cd:ef )
+  ( user_id=pippo, calling_number=1244556673, assigned_ip=10.23.22.123 )
+----
+
+In MATE there are two types of AVPLs:
+
+* data AVPLs that contain information extracted from frames.
+* configuration AVPLs that come from the configuration and are used to tell MATE how
+to relate items based on their data AVPLs.
+
+Data AVPLs can be operated against configuration AVPLs in various ways:
+
+* <<Loose,Loose Match>>: Will match if at least one of the AVPs of each AVPL
+match. If it matches it will return an AVPL containing all AVPs from the data
+AVPL that did match the configuration AVPs.
+* <<Every,"Every" Match>>: Will match if none of the AVPs of the configuration AVPL
+fails to match a present AVP in the data AVPL, even if not all of the
+configuration AVPs have a match. If it matches it will return an AVPL containing
+all AVPs from the data AVPL that did match one AVP in the configuration AVPL.
+* <<Strict,Strict Match>>: Will match if and only if every one of the configuration
+AVPs have at least one match in the data AVPL. If it matches it will return
+an AVPL containing the AVPs from the data AVPL that matched.
+* There's also a <<Merge,Merge>> operation that is to be performed between AVPLs
+where all the AVPs that don't exist in the data AVPL but exist in the configuration
+will be added to the data AVPL.
+* Other than that, there are <<Transform,Transforms>> - a combination
+of a match AVPL and an AVPL to merge.
+
+=== MATE Frame Analysis
+
+MATE's analysis of a frame is performed in three phases:
+
+* In the first phase, MATE attempts to extract a MATE PDU from the frame's
+protocol tree. MATE will create a PDU if MATE's config has a _Pdu_ declaration
+whose _Proto_ is contained in the frame.
+
+* In the second phase, if a PDU has been extracted from the frame, MATE will try
+to group it to other PDUs into a GOP (Group of PDUs) by matching the key
+criteria given by a _Gop_ declaration. If there is no GOP yet with the key
+criteria for the PDU, MATE will try to create a new GOP for it if it matches the
+_Start_ criteria given in the _Gop_ declaration.
+
+* In the third phase, if there's a GOP for the PDU, MATE will try to group this
+GOP with other GOPs into a GOG (Group of Groups) using the criteria given by the
+_Member_ criteria of a _Gog_ declaration.
+
+.MATE Analysis (PDU->GOP->GOG) flowchart
+image::images/ws-mate-analysis.png[]
+
+The extraction and matching logic comes from MATE's configuration; MATE's
+configuration file is specified by the _mate.config_ preference. By default it is
+an empty string which means: do not configure MATE.
+
+The config file tells MATE what to look for in frames; How to make PDUs out of
+it; How will PDUs be related to other similar PDUs into GOPs; And how GOPs
+relate into GOGs.
+
+The MATE configuration file is a list of declarations. There are 4 types of
+declarations: _Transform_, _Pdu_, _Gop_, and _Gog_. A _Transform_ block must be
+before any of the other block declarations that may use it.
+
+==== Create PDUs (Phase 1)
+
+MATE will look in the tree of every frame to see if there is useful data to
+extract, and if there is, it will create one or more PDU objects containing the
+useful information.
+
+The first part of MATE's analysis is the "PDU extraction".
+
+===== PDU data extraction
+
+MATE will make a PDU for each different proto field of _Proto_ type present in the
+frame. MATE will fetch from the field's tree those fields that are defined in
+the <<Pdu>> declaration whose initial offset in the frame is within the
+boundaries of the current _Proto_ and those of the given _Transport_ and _Payload_
+statements.
+
+----
+Pdu dns_pdu Proto dns Transport ip {
+    Extract addr From ip.addr;
+    Extract dns_id From dns.id;
+    Extract dns_resp From dns.flags.response;
+};
+----
+
+.Wireshark window - fields for PDU extraction
+image::images/ws-mate-dns_pane.png[]
+
+Once MATE has found a _Proto_ field for which to create a PDU from the frame it
+will move backwards in the frame looking for the respective _Transport_ fields.
+After that it will create AVPs named as each of those given in the rest of the
+AVPL for every instance of the fields declared as its values.
+
+.Frame fields mapped to PDU attributes
+image::images/ws-mate-dns_pdu.png[]
+
+Sometimes we need information from more than one _Transport_ protocol. In that
+case MATE will check the frame looking backwards to look for the various
+_Transport_ protocols in the given stack. MATE will choose only the closest
+transport boundary per "protocol" in the frame.
+
+This way we'll have all PDUs for every _Proto_ that appears in a frame match its
+relative transports.
+
+----
+Pdu isup_pdu Proto isup Transport mtp3/ip {
+    Extract addr From ip.addr;
+
+    Extract m3pc From mtp3.dpc;
+    Extract m3pc From mtp3.opc;
+
+    Extract cic From isup.cic;
+    Extract isup_msg From isup.message_type;
+};
+----
+
+.Frame containing multiple PDUs
+image::images/ws-mate-isup_over_mtp3_over_ip.png[]
+
+This allows to assign the right _Transport_ to the PDU avoiding duplicate
+transport protocol entries (in case of tunneled ip over ip for example).
+
+----
+Pdu ftp_pdu Proto ftp Transport tcp/ip {
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract ftp_cmd From ftp.command;
+};
+----
+
+.Frame with encapsulated (tunneled) fields
+image::images/ws-mate-ftp_over_gre.png[]
+
+Other than the mandatory _Transport_ there is also an optional _Payload_
+statement, which works pretty much as _Transport_ but refers to elements after
+the _Proto_'s range. It is useful in those cases where the payload protocol
+might not appear in a PDU but nevertheless the PDU belongs to the same category.
+
+----
+Pdu mmse_over_http_pdu Proto http Transport tcp/ip {
+
+    Payload mmse;
+
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+
+    Extract content From http.content_type;
+    Extract host From http.host;
+    Extract http_rq From http.request;
+    Extract method From http.request.method;
+    Extract resp From http.response.code;
+
+    Extract msg_type From mmse.message_type;
+    Extract notify_status From mmse.status;
+    Extract send_status From mmse.response_status;
+    Extract trx From mmse.transaction_id;
+};
+----
+
+.Extract from Payload fields
+image::images/ws-mate-mmse_over_http.png[]
+
+===== Conditions on which to create PDUs
+
+There might be cases in which we won't want MATE to create a PDU unless some of
+its extracted attributes meet or do not meet some criteria. For that we use the
+_Criteria_ statements of the _Pdu_ declarations.
+
+----
+Pdu isup_pdu Proto isup Transport mtp3/ip {
+    ...
+
+    // MATE will create isup_pdu PDUs only when there is not a point code '1234'
+    Criteria Reject Strict (m3pc=1234);
+};
+
+Pdu ftp_pdu Proto ftp Transport tcp/ip {
+    ...
+
+    // MATE will create ftp_pdu PDUs only when they go to port 21 of our ftp_server
+    Criteria Accept Strict (addr=10.10.10.10, port=21);
+};
+----
+
+The _Criteria_ statement is given an action (_Accept_ or _Reject_), a match type
+(_Strict_, _Loose_ or _Every_) and an AVPL against which to match the currently
+extracted one.
+
+===== Transforming the attributes of a PDU
+
+Once the fields have been extracted into the PDU's AVPL, MATE will apply any
+declared _Transform_ to it. The way transforms are applied and how they work
+is described later on. However, it's useful to know that once the AVPL for the
+PDU is created, it may be transformed before being analyzed. That way we can
+massage the data to simplify the analysis.
+
+===== MATE's PDU tree
+
+Every successfully created PDU will add a MATE tree to the frame dissection. If
+the PDU is not related to any GOP, the tree for the PDU will contain just the
+PDU's info. If it is assigned to a GOP, the tree will also contain the GOP items,
+and the same applies for the GOG level.
+
+----
+mate dns_pdu:1
+    dns_pdu: 1
+        dns_pdu time: 3.750000
+        dns_pdu Attributes
+            dns_resp: False
+            dns_id: 0x8cac
+            addr: 10.194.4.11
+            addr: 10.194.24.35
+----
+
+The PDU's tree contains some filterable fields
+
+* _mate.dns_pdu_ will contain the number of the "dns_pdu" PDU
+* _mate.dns_pdu.RelativeTime_ will contain the time passed since the beginning
+of the capture in seconds
+* the tree will contain the various attributes of the PDU as well, these will
+all be strings (to be used in filters as "10.0.0.1", not as 10.0.0.1)
+** mate.dns_pdu.dns_resp
+** mate.dns_pdu.dns_id
+** mate.dns_pdu.addr
+
+==== Grouping PDUs together (GOP) (Phase 2)
+
+Once MATE has created the PDUs it passes to the PDU analysis phase. During the
+PDU analysis phase MATE will try to group PDUs of the same type into 'Groups of
+PDUs' (aka *GOP*++s++) and copy some AVPs from the PDU's AVPL to the GOP's AVPL.
+
+.Grouping PDUs (GOP) flowchart
+image::images/ws-mate-pdu_analysis.png[]
+
+===== What can belong to a GOP
+
+Given a PDU, the first thing MATE will do is to check if there is any GOP
+declaration in the configuration for the given PDU type. If so, it will use its
+_Match_ AVPL to match it against the PDU's AVPL; if they don't match, the
+analysis phase is done. If there is a match, the AVPL is the GOP's candidate key
+which will be used to search the index of GOPs for the GOP to which to assign
+the current PDU. If there is no such GOP and this PDU does not match the
+_Start_ criteria of a _Gop_ declaration for the PDU type, the PDU will remain
+unassigned and only the analysis phase will be done.
+
+----
+Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) {...};
+Gop dns_req On dns_pdu Match (addr, addr, dns_id) {...};
+Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) {...};
+----
+
+===== Start of a GOP
+
+If there was a match, the candidate key will be used to search the index of GOPs
+to see if there is already a GOP matching the GOP's key the same way. If there
+is such a match in the GOPs collection, and the PDU doesn't match the _Start_
+AVPL for its type, the PDU will be assigned to the matching GOP. If it is a
+_Start_ match, MATE will check whether or not that GOP has been already
+stopped. If the GOP has been stopped, a new GOP will be created and will replace
+the old one in the index of GOPs.
+
+----
+Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) {
+    Start (ftp_cmd=USER);
+};
+
+Gop dns_req On dns_pdu Match (addr, addr, dns_id) {
+    Start (dns_resp="True");
+};
+
+Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) {
+    Start (isup_msg=1);
+};
+----
+
+If no _Start_ is given for a GOP, a PDU whose AVPL matches an existing GOP's
+key will act as the start of a GOP.
+
+===== What goes into the GOP's AVPL
+
+Once we know a GOP exists and the PDU has been assigned to it, MATE will copy
+into the GOP's AVPL all the attributes matching the key plus any AVPs of the
+PDU's AVPL matching the _Extra_ AVPL.
+
+----
+Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) {
+    Start (ftp_cmd=USER);
+    Extra (pasv_prt, pasv_addr);
+};
+
+Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) {
+    Start (isup_msg=1);
+    Extra (calling, called);
+};
+----
+
+===== End of a GOP
+
+Once the PDU has been assigned to the GOP, MATE will check whether or not the
+PDU matches the _Stop_, if it happens, MATE will mark the GOP as stopped. Even
+after stopped, a GOP may get assigned new PDUs matching its key, unless such
+PDU matches _Start_. If it does, MATE will instead create a new GOP starting
+with that PDU.
+
+----
+Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) {
+    Start (ftp_cmd=USER);
+    Stop (ftp_cmd=QUIT); // The response to the QUIT command will be assigned to the same GOP
+    Extra (pasv_prt, pasv_addr);
+};
+
+Gop dns_req On dns_pdu Match (addr, addr, dns_id) {
+    Start (dns_resp="False");
+    Stop (dns_resp="True");
+};
+
+Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) {
+    Start (isup_msg=1); // IAM
+    Stop (isup_msg=16); // RLC
+    Extra (calling, called);
+};
+----
+
+If no _Stop_ criterium is stated for a given GOP, the GOP will be stopped as
+soon as it is created. However, as with any other GOP, PDUs matching the GOP's
+key will still be assigned to the GOP unless they match a _Start_ condition,
+in which case a new GOP using the same key will be created. To group multiple
+PDUs that match the _Start_, add a bogus _Stop_ such as
+----
+Gop frame_ses On frame_pdu Match (frame_time) {
+    Start (frame_time);
+    Stop (frame_time="FOO");
+};
+----
+
+===== GOP's tree
+
+For every frame containing a PDU that belongs to a GOP, MATE will create a tree
+for that GOP.
+
+The example below represents the tree created by the _dns_pdu_ and _dns_req_
+examples.
+
+----
+...
+MATE dns_pdu:6->dns_req:1
+    dns_pdu: 6
+        dns_pdu time: 2.103063
+        dns_pdu time since beginning of Gop: 2.103063
+        dns_pdu Attributes
+            dns_resp: True
+            dns_id: 0x8cac
+            addr: 10.194.4.11
+            addr: 10.194.24.35
+    dns_req: 1
+        GOP Key:  addr=10.194.4.11; addr=10.194.24.35; dns_id=0x8cac;
+        dns_req Attributes
+            dns_id: 0x8cac
+            addr: 10.194.4.11
+            addr: 10.194.24.35
+        dns_req Times
+            dns_req start time: 0.000000
+            dns_req hold time: 2.103063
+            dns_req duration: 2.103063
+        dns_req number of PDUs: 2
+            Start PDU: in frame 1
+            Stop PDU: in frame 6 (2.103063 : 2.103063)
+
+----
+
+Other than the PDU's tree, this one contains information regarding the
+relationship between the PDUs that belong to the GOP. That way we have:
+
+* mate.dns_req which contains the id of this dns_req GOP. This will be present
+in frames that belong to dns_req GOPs.
+* mate.dns_req.dns_id and mate.dns_req.addr which represent the values of the
+attributes copied into the GOP.
+* the timers of the GOP
+** mate.dns_req.StartTime time (in seconds) passed since beginning of capture
+until GOP's start.
+** mate.dns_req.Time time passed between the start PDU and the stop PDU assigned
+to this GOP (only created if a Stop criterion has been declared for the GOP and
+a matching PDU has arrived).
+** mate.dns_req.Duration time passed between the start PDU and the last PDU
+assigned to this GOP.
+* mate.dns_req.NumOfPdus the number of PDUs that belong to this GOP
+** mate.dns_req.Pdu a filterable list of frame numbers of the PDUs of this GOP
+
+===== GOP's timers
+
+Note that there are two "timers" for a GOP:
+
+* *Time*, which is defined only for GOPs that have been Stopped, and gives the
+time passed between the _Start_ and the _Stop_ PDUs.
+* *Duration*, which is defined for every GOP regardless of its state, and give
+the time passed between its _Start_ PDU and the last PDU that was assigned to
+that GOP.
+
+So:
+
+* we can filter for PDUs that belong to GOPs that have been Stopped with
+*mate.xxx.Time*
+* we can filter for PDUs that belong to unstopped GOPs with *mate.xxx &&
+!mate.xxx.Time*
+* we can filter for PDUs that belong to stopped GOPs using *mate.xxx.Duration*
+* we can filter for PDUs that belong to GOPs that have taken more (or less) time
+that 0.5s to complete with *mate.xxx.Time > 0.5* (you can try these also as
+color filters to find out when response times start to grow)
+
+==== Grouping GOPs together (GOG) (Phase 3)
+
+When GOPs are created, or whenever their AVPL changes, GOPs are (re)analyzed to
+check if they match an existent group of groups (GOG) or can create a new one.
+The GOP analysis is divided into two phases. In the first phase, the still
+unassigned GOP is checked to verify whether it belongs to an already existing
+GOG or may create a new one. The second phase eventually checks the GOG and
+registers its keys in the index of GOGs.
+
+.Grouping GOPs (GOG) flowchart
+image::images/ws-mate-gop_analysis.png[]
+
+There are several reasons for the author to believe that this feature needs to
+be reimplemented, so probably there will be deep changes in the way this is done
+in the near future. This section of the documentation reflects the version of
+MATE as of Wireshark 0.10.9; in future releases this will change.
+
+===== Declaring a Group Of Groups (GOG)
+
+The first thing we have to do configuring a GOG is to tell MATE that it exists.
+
+----
+Gog http_use {
+   ...
+};
+----
+
+===== Telling MATE what could be a GOG member
+
+Then we have to tell MATE what to look for a match in the candidate GOPs.
+
+----
+Gog http_use {
+    Member http_ses (host);
+    Member dns_req (host);
+};
+----
+
+===== Getting interesting data into the GOG
+
+Most often, also other attributes than those used for matching would be
+interesting. In order to copy from GOP to GOG other interesting attributes, we
+might use _Extra_ like we do for GOPs.
+
+----
+Gog http_use {
+    ...
+    Extra (cookie);
+};
+----
+
+===== GOG's tree
+
+----
+mate http_pdu:4->http_req:2->http_use:1
+    http_pdu: 4
+        http_pdu time: 1.309847
+        http_pdu time since beginning of Gop: 0.218930
+        http_req: 2
+            ... (the gop's tree for http_req: 2) ..
+        http_use: 1
+            http_use Attributes
+                host: www.example.com
+            http_use Times
+                http_use start time: 0.000000
+                http_use duration: 1.309847
+            number of GOPs: 3
+                dns_req: 1
+                    ... (the gop's tree for dns_req: 1) ..
+                http_req: 1
+                    ... (the gop's tree for http_req: 1) ..
+                http_req of current frame: 2
+----
+
+We can filter on:
+
+* the timers of the GOG
+** *mate.http_use.StartTime* time (in seconds) passed since beginning of capture until GOG's start.
+** *mate.http_use.Duration* time elapsed between the first frame of a GOG and the last one assigned to it.
+* the attributes passed to the GOG
+** *mate.http_use.host*
+* *mate.http_use.NumOfGops* the number of GOPs that belong to this GOG
+* *mate.http_use.GopStart* the start frame of a GOP
+* *mate.http_use.GopStop* the stop frame of a GOP
+
+==== Adjust data (AVPL Transforms)
+
+A Transform is a sequence of Match rules optionally completed with modification
+of the match result by an additional AVPL. Such modification may be an Insert
+(merge) or a Replace. Transforms can be used as helpers to manipulate an item's
+AVPL before it is processed further. They come to be very helpful in several
+cases.
+
+===== Syntax
+
+AVPL Transforms are declared in the following way:
+
+----
+Transform name {
+    Match [Strict|Every|Loose] match_avpl [Insert|Replace] modify_avpl;
+    ...
+};
+----
+
+The *name* is the handle to the AVPL transform. It is used to refer to the
+transform when invoking it later.
+
+The _Match_ declarations instruct MATE what and how to match against the data
+AVPL and how to modify the data AVPL if the match succeeds. They will be
+executed in the order they appear in the config file whenever they are invoked.
+
+The optional match type qualifier (_Strict_, _Every_, or _Loose_) is used
+to choose the <<Match,Match type>>; _Strict_ is the default value which
+may be omitted.
+
+The optional modification mode qualifier instructs MATE how the modify AVPL
+should be used:
+
+* the default value _Insert_ (which may be omitted) causes the _modify_avpl_
+to be *merged* to the existing data AVPL,
+* _Replace_ causes all the matching AVPs from the data AVPL to be
+*replaced* by the _modify_avpl_.
+
+The _modify_avpl_ may be an empty one; this comes useful in some cases for
+both _Insert_ and _Replace_ modification modes.
+
+----
+Transform rm_client_from_http_resp1 {
+    Match (http_rq); //first match wins so the request won't get the not_rq attribute inserted
+    Match Every (addr) Insert (not_rq); //this line won't be evaluated if the first one matched so not_rq won't be inserted to requests
+};
+
+Transform rm_client_from_http_resp2 {
+    Match (not_rq, client) Replace (); //replace "client and not_rq" with nothing
+};
+----
+
+Examples:
+
+----
+Transform insert_name_and {
+    Match Strict (host=10.10.10.10, port=2345) Insert (name=JohnDoe);
+};
+----
+
+adds name=JohnDoe to the data AVPL if it contains host=10.10.10.10 *and*
+port=2345
+
+----
+Transform insert_name_or {
+    Match Loose (host=10.10.10.10, port=2345) Insert (name=JohnDoe);
+};
+----
+
+adds name=JohnDoe to the data AVPL if it contains host=10.10.10.10 *or*
+port=2345
+
+----
+Transform replace_ip_address {
+    Match (host=10.10.10.10) Replace (host=192.168.10.10);
+};
+----
+
+replaces the original host=10.10.10.10 by host=192.168.10.10
+
+----
+Transform add_ip_address {
+    Match (host=10.10.10.10) (host=192.168.10.10);
+};
+----
+
+adds (inserts) host=192.168.10.10 to the AVPL, keeping the original
+host=10.10.10.10 in it too
+
+----
+ Transform replace_may_be_surprising {
+    Match Loose (a=aaaa, b=bbbb) Replace (c=cccc, d=dddd);
+ };
+----
+
+gives the following results:
+
+* (a=aaaa, b=eeee) gets transformed to (b=eeee, c=cccc, d=dddd) because a=aaaa
+did match so it got replaced while b=eeee did not match so it has been left
+intact,
+* (a=aaaa, b=bbbb) gets transformed to (c=cccc, d=dddd) because both a=aaaa and
+b=bbbb did match.
+
+===== Usage
+
+Once declared, Transforms can be added to the declarations of PDUs, GOPs or
+GOGs. This is done by adding the _Transform name_list_ statement to the
+declaration:
+
+----
+Pdu my_proto_pdu Proto my_proto Transport ip {
+    Extract addr From ip.addr;
+    ...
+    Transform my_pdu_transform[, other_pdu_transform[, yet_another_pdu_transform]];
+};
+----
+
+* In case of PDU, the list of transforms is applied against the PDU's AVPL
+after its creation.
+* In case of GOP and GOG, the list of transforms is applied against their
+respective AVPLs when they are created and every time they change.
+
+===== Operation
+
+.Applying Transform flowchart
+image::images/ws-mate-transform.png[]
+
+* A list of previously declared Transforms may be given to every Item (_Pdu_, _Gop_,
+or _Gog_), using the _Transform_ statement.
+* Every time the AVPL of an item changes, it will be operated against *all* the
+Transforms on the list given to that item. The Transforms on the list are
+applied left to right.
+* Inside each of the Transforms, the item's AVPL will be operated against the
+Transform's Match clauses starting from the topmost one, until all have been
+tried or until one of them succeeds.
+
+MATE's Transforms can be used for many different things, like:
+
+===== Multiple Start/Stop conditions for a GOP
+
+Using _Transforms_ we can add more than one start or stop condition to a GOP.
+
+----
+Transform start_cond {
+    Match (attr1=aaa,attr2=bbb) (msg_type=start);
+    Match (attr3=www,attr2=bbb) (msg_type=start);
+    Match (attr5^a) (msg_type=stop);
+    Match (attr6$z) (msg_type=stop);
+};
+
+Pdu pdu ... {
+    ...
+    Transform start_cond;
+}
+
+Gop gop ... {
+    Start (msg_type=start);
+    Stop (msg_type=stop);
+    ...
+}
+----
+
+===== Marking GOPs and GOGs to filter them easily
+
+----
+Transform marks {
+    Match (addr=10.10.10.10, user=john) (john_at_host);
+    Match (addr=10.10.10.10, user=tom) (tom_at_host);
+}
+
+...
+
+Gop my_gop ... {
+    ...
+    Transform marks;
+}
+----
+
+After that we can use a display filter *mate.my_gop.john_at_host* or
+*mate.my_gop.tom_at_host*
+
+===== Adding (Insert) direction knowledge to MATE
+
+----
+Transform direction_as_text {
+    Match (src=192.168.0.2, dst=192.168.0.3) Insert (direction=from_2_to_3);
+    Match (src=192.168.0.3, dst=192.168.0.2) Insert (direction=from_3_to_2);
+};
+
+Pdu my_pdu Proto my_proto Transport tcp/ip {
+    Extract src From ip.src;
+    Extract dst From ip.dst;
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract start From tcp.flags.syn;
+    Extract stop From tcp.flags.fin;
+    Extract stop From tcp.flags.rst;
+    Transform direction_as_text;
+}
+
+Gop my_gop On my_pdu Match (addr,addr,port,port) {
+    ...
+    Extra (direction);
+}
+----
+
+The original example (below) would delete _src_ and _dst_ then add _direction_.
+----
+Transform direction_as_text {
+    Match (src=192.168.0.2, dst=192.168.0.3) Replace (direction=from_2_to_3);
+    Match (src=192.168.0.3, dst=192.168.0.2) Replace (direction=from_3_to_2);
+};
+----
+
+===== NAT
+
+NAT can create problems when tracing, but we can easily work around it by
+Transforming the NATed IP address and the Ethernet address of the router into
+the non-NAT address:
+
+----
+Transform denat {
+    Match (addr=192.168.0.5, ether=01:02:03:04:05:06) Replace (addr=123.45.67.89);
+    Match (addr=192.168.0.6, ether=01:02:03:04:05:06) Replace (addr=123.45.67.90);
+    Match (addr=192.168.0.7, ether=01:02:03:04:05:06) Replace (addr=123.45.67.91);
+}
+
+Pdu my_pdu Proto my_proto transport tcp/ip/eth {
+    Extract ether From eth.addr;
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Transform denat;
+}
+----
+
+[#ChMateConfigurationTutorial]
+
+=== MATE's configuration tutorial
+
+We'll show a MATE configuration that first creates GOPs for every DNS and HTTP
+request, then it ties the GOPs together in a GOG based on the host. Finally,
+we'll separate into different GOGs request coming from different users.
+
+With this MATE configuration loaded we can:
+
+* use *mate.http_use.Duration > 5.5* to filter frames based on the time it takes
+to load a complete page from the DNS request to resolve its name until the last
+image gets loaded.
+* use *mate.http_use.client == "10.10.10.20" && mate.http_use.host == "www.example.com"*
+to isolate DNS and HTTP packets related to a visit of a certain user.
+* use *mate.http_req.Duration > 1.5* to filter all the packets of HTTP requests
+that take more than 1.5 seconds to complete.
+
+The complete config file is available on the Wireshark Wiki:
+{wireshark-wiki-url}Mate/Tutorial
+
+Note: This example uses _dns.qry.name_ which is defined since Wireshark
+version 0.10.9. Supposing you have a MATE plugin already installed you can test
+it with the current Wireshark version.
+
+==== A GOP for DNS requests
+
+First we'll tell MATE how to create a GOP for each DNS request/response.
+
+MATE needs to know what makes a DNS PDU. We describe it using a _Pdu_
+declaration:
+
+----
+Pdu dns_pdu Proto dns Transport ip {
+    Extract addr From ip.addr;
+    Extract dns_id From dns.id;
+    Extract dns_resp From dns.flags.response;
+};
+----
+
+Using _Proto dns_ we tell MATE to create PDUs every time it finds _dns_. Using
+_Transport ip_ we inform MATE that some of the fields we are interested are
+in the _ip_ part of the frame. Finally, we tell MATE to import _ip.addr_ as
+_addr_, _dns.id_ as _dns_id_ and _dns.flags.response_ as _dns_resp_.
+
+Once we've told MATE how to extract _dns_pdus_ we'll tell it how to match
+requests and responses and group them into a GOP. For this we'll use a _Gop_
+declaration to define the GOP, and then, _Start_ and _Stop_ statements to
+tell it when the GOP starts and ends.
+
+----
+Gop dns_req On dns_pdu Match (addr,addr,dns_id) {
+    Start (dns_resp="False");
+    Stop (dns_resp="True");
+};
+----
+
+Using the *Gop* declaration we tell MATE that the *Name* of the GOP is _dns_req_,
+that _dns_pdus_s can become members of the GOP, and what is the key used to match
+the PDUs to the GOP.
+
+The key for this GOP is _"addr, addr, dns_id"_. That means that in order to
+belong to the same GOP, _dns_pdus_ have to have both addresses and the
+_request id_ identical. We then instruct MATE that a _dns_req_ starts whenever
+a _dns_pdu_ matches _"dns_resp=++"++False++"++"_ and that it stops when another _dns_pdu_
+matches _"dns_resp=++"++True++"++"_.
+
+At this point, if we open a capture file using this configuration, we are able
+to use a display filter *mate.dns_req.Time > 1* to see only the packets of
+DNS requests that take more than one second to complete.
+
+We can use a display filter *mate.dns_req && ! mate.dns_req.Time* to find
+requests for which no response was given. *mate.xxx.Time* is set only for GOPs
+that have being stopped.
+
+==== A GOP for HTTP requests
+
+This other example creates a GOP for every HTTP request.
+
+----
+Pdu http_pdu Proto http Transport tcp/ip {
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract http_rq From http.request.method;
+    Extract http_rs From http.response;
+    DiscardPduData true;
+};
+
+Gop http_req On http_pdu Match (addr, addr, port, port) {
+    Start (http_rq);
+    Stop (http_rs);
+};
+----
+
+So, if we open a capture using this configuration
+
+* filtering with *mate.http_req.Time > 1* will give all the requests where the
+response header takes more than one second to come
+* filtering with *mate.http_req.Duration > 1.5* will show those request that
+take more than 1.5 seconds to complete.
+
+You have to know that *mate.xxx.Time* gives the time in seconds between the PDU
+matching the GOP *Start* clause and the PDU matching the GOP *Stop* clause (yes, you can create
+timers using this!). On the other hand, *mate.xxx.Duration* gives you the time
+passed between the GOP *Start* and the last PDU assigned to that GOP regardless
+whether it is a *Stop* or not. After the GOP *Stop*, PDUs matching the GOP's Key will
+still be assigned to the same GOP as far as they don't match the GOP *Start*, in
+which case a new GOP with the same key will be created.
+
+==== Getting DNS and HTTP together into a GOG
+
+We'll tie together to a single GOG all the HTTP packets belonging to requests
+and responses to a certain host and the DNS request and response used to resolve
+its domain name using the _Pdu_ and _Gop_ definitions of the previous examples
+
+To be able to group DNS and HTTP requests together, we need to import into the
+PDUs and GOPs some part of information that both those protocols share. Once the
+PDUs and GOPs have been defined, we can use _Extract_ (for PDUs) and
+_Extract_ (for GOPs) statements to tell MATE what other protocol fields are to
+be added to PDU's and GOP's AVPLs. We add the following statements to the
+appropriate declarations:
+
+----
+    Extract host From http.host; // to Pdu http_pdu as the last Extract in the list
+    Extra (host); // to Gop http_req after the Stop
+
+    Extract host From dns.qry.name; // to Pdu dns_pdu as the last Extract in the list
+    Extra (host); // to Gop dns_req after the Stop
+----
+
+Here we've told MATE to import _http.host_ into _http_pdu_ and _dns.qry.name_
+into _dns_pdu_ as _host_. We also have to tell MATE to copy the _host_
+attribute from the PDUs to the GOPs - we do this using _Extra_.
+
+Once we have all the data we need in PDUs and GOPs, we tell MATE what makes
+different GOPs belong to a certain GOG.
+
+----
+Gog http_use {
+    Member http_req (host);
+    Member dns_req (host);
+    Expiration 0.75;
+};
+----
+
+Using the _Gog_ declaration, we tell MATE to define a GOG type named
+_http_use_ whose expiration is 0.75 seconds after all the GOPs that belong to it
+had been stopped. After that time, an eventual new GOP with the same key match
+will create a new GOG instead of been added to the previous GOG.
+
+Using the _Member_ statements, we tell MATE that *http_req*s with the same
+*host* belong to the same GOG, same thing for *dns_req*s.
+
+So far we have instructed MATE to group every packet related to sessions towards
+a certain host. At this point if we open a capture file and:
+
+* a display filter *mate.http_use.Duration > 5* will show only those requests
+that have taken more than 5 seconds to complete starting from the DNS request
+and ending with the last packet of the HTTP responses.
+
+* a display filter *mate.http_use.host == "www.w3c.org"* will show all the
+packets (both DNS and HTTP) related to the requests directed to www.w3c.org
+
+==== Separating requests from multiple users
+
+"Houston: we've had a problem here."
+
+This configuration works fine if used for captures taken at the client's side
+but deeper in the network we'd got a real mess. Requests from many users get
+mixed together into _http_uses_. GOGs are created and stopped almost randomly
+(depending on the timing in which GOPs start and stop). How do we get requests
+from individual users separated from each other?
+
+MATE has a tool that can be used to resolve this kind of grouping issues. This
+tool are the _Transforms_. Once defined, they can be applied against PDUs,
+GOPs and GOGs and they might replace or insert more attributes based on what's
+there. We'll use them to create an attribute named *client*, using which we'll
+separate different requests.
+
+For DNS we need the ip.src of the request moved into the GOP only from the DNS
+request.
+
+So we first tell MATE to import ip.src as client:
+
+----
+    Extract client From ip.src;
+----
+
+Next, we tell MATE to replace ( *dns_resp="True", client* ) with just *dns_resp="True"* in
+the PDU. That way, we'll keep the attribute *client* only in the DNS request
+PDUs (i.e., packets coming from the client).To do so, we have to add a
+_Transform_ declaration (in this case, with just one clause) before the _Pdu_
+declaration which uses it:
+
+----
+Transform rm_client_from_dns_resp {
+    Match (dns_resp="True", client) Replace (dns_resp="True");
+};
+----
+
+Next, we invoke the transform by adding the following line after the _Extract_
+list of the dns_pdu PDU:
+
+----
+    Transform rm_client_from_dns_resp;
+----
+
+HTTP is a little trickier. We have to remove the attribute carrying ip.src from
+both the response and the "continuations" of the response, but as there is
+nothing to filter on for the continuations, we have to add a fake attribute
+first. And then we have to remove *client* when the fake attribute appears.
+This is possible due to the fact that the _Match_ clauses in the _Transform_
+are executed one by one until one of them succeeds. First, we declare another
+two _Transforms_:
+
+----
+Transform rm_client_from_http_resp1 {
+    Match (http_rq); //first match wins so the request won't get the not_rq attribute inserted
+    Match Every (addr) Insert (not_rq); //this line won't be evaluated if the first one matched so not_rq won't be inserted to requests
+};
+
+Transform rm_client_from_http_resp2 {
+    Match (not_rq, client) Replace (); //replace "client and not_rq" with nothing (will happen only in the response and eventual parts of it)
+};
+----
+
+Next, we add another _Extract_ statement to the _http_pdu_ declaration, and
+apply both _Transforms_ declared above in a proper order:
+
+----
+    Extract client From ip.src;
+    Transform rm_client_from_http_resp1, rm_client_from_http_resp2;
+----
+
+In MATE, all the _Transform_s listed for an item will be evaluated, while
+inside a single _Transform_, the evaluation will stop at the first successful
+_Match_ clause. That's why we first just match _http_rq_ to get out of the
+first sequence before adding the _not_rq_ attribute. Then we apply the second
+_Transform_ which removes both _not_rq_ and _client_ if both are there. Yes,
+_Transform_s are cumbersome, but they are very useful.
+
+Once we got all what we need in the PDUs, we have to tell MATE to copy the
+attribute _client_ from the PDUs to the respective GOPs, by adding client to
+_Extra_ lists of both _Gop_ declarations:
+
+----
+    Extra (host, client);
+----
+
+On top of that, we need to modify the old declarations of GOP key to new ones
+that include both _client_ and _host_. So we change the _Gog_ *Member*
+declarations the following way:
+
+----
+    Member http_req (host, client);
+    Member dns_req (host, client);
+----
+
+Now we got it, every "usage" gets its own GOG.
+
+[#ChMateConfigurationExamples]
+
+=== MATE configuration examples
+
+The following is a collection of various configuration examples for MATE. Many
+of them are useless because the "conversations" facility does a better job.
+Anyway they are meant to help users understanding how to configure MATE.
+
+[#File_tcp_mate]
+==== TCP session (tcp.mate)
+
+The following example creates a GOP out of every TCP session.
+
+----
+Transform add_tcp_stop {
+    Match (tcp_flags_reset="True") Insert (tcp_stop="True");
+    Match (tcp_flags_fin="True") Insert (tcp_stop="True");
+};
+
+Pdu tcp_pdu Proto tcp Transport ip {
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract tcp_start From tcp.flags.syn;
+    Extract tcp_flags_reset From tcp.flags.reset;
+    Extract tcp_flags_fin From tcp.flags.fin;
+    Transform add_tcp_stop;
+};
+
+Gop tcp_ses On tcp_pdu Match (addr, addr, port, port) {
+    Start (tcp_start="True");
+    Stop (tcp_stop="True");
+};
+
+Done;
+----
+
+This probably would do fine in 99.9% of the cases but 10.0.0.1:20->10.0.0.2:22 and 10.0.0.1:22->10.0.0.2:20 would both fall into the same gop if they happen to overlap in time.
+
+* filtering with *mate.tcp_ses.Time > 1* will give all the sessions that last more than one second
+* filtering with *mate.tcp_ses.NumOfPdus < 5* will show all tcp sessions that have less than 5 packets.
+* filtering with *mate.tcp_ses.Id == 3* will show all the packets for the third tcp session MATE has found
+
+==== a GOG for a complete FTP session
+
+This configuration allows to tie a complete passive FTP session (including the
+data transfer) in a single GOG.
+
+----
+Pdu ftp_pdu Proto ftp Transport tcp/ip {
+    Extract ftp_addr From ip.addr;
+    Extract ftp_port From tcp.port;
+    Extract ftp_resp From ftp.response.code;
+    Extract ftp_req From ftp.request.command;
+    Extract server_addr From ftp.passive.ip;
+    Extract server_port From ftp.passive.port;
+
+    LastPdu true;
+};
+
+Pdu ftp_data_pdu Proto ftp-data Transport tcp/ip{
+    Extract server_addr From ip.src;
+    Extract server_port From tcp.srcport;
+
+};
+
+Gop ftp_data On ftp_data_pdu Match (server_addr, server_port) {
+    Start (server_addr);
+};
+
+Gop ftp_ctl On ftp_pdu Match (ftp_addr, ftp_addr, ftp_port, ftp_port) {
+    Start (ftp_resp=220);
+    Stop (ftp_resp=221);
+    Extra (server_addr, server_port);
+};
+
+Gog ftp_ses {
+    Member ftp_ctl (ftp_addr, ftp_addr, ftp_port, ftp_port);
+    Member ftp_data (server_addr, server_port);
+};
+
+Done;
+----
+
+Note: not having anything to distinguish between ftp-data packets makes this
+config to create one GOP for every ftp-data packet instead of each transfer.
+Pre-started GOPs would avoid this.
+
+==== using RADIUS to filter SMTP traffic of a specific user
+
+Spying on people, in addition to being immoral, is illegal in many countries.
+This is an example meant to explain how to do it not an invitation to do so.
+It's up to the police to do this kind of job when there is a good reason to do
+so.
+
+----
+Pdu radius_pdu On radius Transport udp/ip {
+    Extract addr From ip.addr;
+    Extract port From udp.port;
+    Extract radius_id From radius.id;
+    Extract radius_code From radius.code;
+    Extract user_ip From radius.framed_addr;
+    Extract username From radius.username;
+}
+
+Gop radius_req On radius_pdu (radius_id, addr, addr, port, port) {
+    Start (radius_code {1|4|7} );
+    Stop (radius_code {2|3|5|8|9} );
+    Extra (user_ip, username);
+}
+
+// we define the smtp traffic we want to filter
+Pdu user_smtp Proto smtp Transport tcp/ip {
+    Extract user_ip From ip.addr;
+    Extract smtp_port From tcp.port;
+    Extract tcp_start From tcp.flags.syn;
+    Extract tcp_stop From tcp.flags.reset;
+}
+
+Gop user_smtp_ses On user_smtp (user_ip, user_ip, smtp_port!25) {
+    Start (tcp_start=1);
+    Stop (tcp_stop=1);
+}
+
+// with the following group of groups we'll group together the radius and the smtp
+// we set a long expiration to avoid the session expire on long pauses.
+Gog user_mail {
+    Expiration 1800;
+    Member radius_req (user_ip);
+    Member user_smtp_ses (user_ip);
+    Extra (username);
+}
+
+Done;
+----
+
+Filtering the capture file with *mate.user_mail.username == "theuser"* will
+filter the RADIUS packets and SMTP traffic for _"theuser"_.
+
+==== H323 Calls
+
+This configuration will create a GOG out of every call.
+
+----
+Pdu q931 Proto q931 Transport ip {
+    Extract addr From ip.addr;
+    Extract call_ref From q931.call_ref;
+    Extract q931_msg From q931.message_type;
+    Extract calling From q931.calling_party_number.digits;
+    Extract called From q931.called_party_number.digits;
+    Extract guid From h225.guid;
+    Extract q931_cause From q931.cause_value;
+};
+
+Gop q931_leg On q931 Match (addr, addr, call_ref) {
+    Start (q931_msg=5);
+    Stop (q931_msg=90);
+    Extra (calling, called, guid, q931_cause);
+};
+
+Pdu ras Proto h225.RasMessage Transport ip {
+    Extract addr From ip.addr;
+    Extract ras_sn From h225.requestSeqNum;
+    Extract ras_msg From h225.RasMessage;
+    Extract guid From h225.guid;
+};
+
+Gop ras_req On ras Match (addr, addr, ras_sn) {
+    Start (ras_msg {0|3|6|9|12|15|18|21|26|30} );
+    Stop (ras_msg {1|2|4|5|7|8|10|11|13|14|16|17|19|20|22|24|27|28|29|31});
+    Extra (guid);
+};
+
+Gog call {
+    Member ras_req (guid);
+    Member q931_leg (guid);
+    Extra (called,calling,q931_cause);
+};
+
+Done;
+----
+
+with this we can:
+
+* filter all signalling for a specific caller: *mate.call.caller == "123456789"*
+* filter all signalling for calls with a specific release cause: *mate.call.q931_cause == 31*
+* filter all signalling for very short calls: *mate.q931_leg.Time < 5*
+
+==== MMS
+
+With this example, all the components of an MMS send or receive will be tied
+into a single GOG. Note that this example uses the _Payload_ clause because
+MMS delivery uses MMSE over either HTTP or WSP. As it is not possible to relate
+the retrieve request to a response by the means of MMSE only (the request is
+just an HTTP GET without any MMSE), a GOP is made of HTTP PDUs but MMSE data
+need to be extracted from the bodies.
+
+----
+## WARNING: this example has been blindly translated from the "old" MATE syntax
+## and it has been verified that Wireshark accepts it. However, it has not been
+## tested against any capture file due to lack of the latter.
+
+Transform rm_client_from_http_resp1 {
+    Match (http_rq);
+    Match Every (addr) Insert (not_rq);
+};
+
+Transform rm_client_from_http_resp2 {
+    Match (not_rq,ue) Replace ();
+};
+
+Pdu mmse_over_http_pdu Proto http Transport tcp/ip {
+    Payload mmse;
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract http_rq From http.request;
+    Extract content From http.content_type;
+    Extract resp From http.response.code;
+    Extract method From http.request.method;
+    Extract host From http.host;
+    Extract content From http.content_type;
+    Extract trx From mmse.transaction_id;
+    Extract msg_type From mmse.message_type;
+    Extract notify_status From mmse.status;
+    Extract send_status From mmse.response_status;
+    Transform rm_client_from_http_resp1, rm_client_from_http_resp2;
+};
+
+Gop mmse_over_http On mmse_over_http_pdu Match (addr, addr, port, port) {
+    Start (http_rq);
+    Stop (http_rs);
+    Extra (host, ue, resp, notify_status, send_status, trx);
+};
+
+Transform mms_start {
+    Match Loose() Insert (mms_start);
+};
+
+Pdu mmse_over_wsp_pdu Proto wsp Transport ip {
+    Payload mmse;
+    Extract trx From mmse.transaction_id;
+    Extract msg_type From mmse.message_type;
+    Extract notify_status From mmse.status;
+    Extract send_status From mmse.response_status;
+    Transform mms_start;
+};
+
+Gop mmse_over_wsp On mmse_over_wsp_pdu Match (trx) {
+    Start (mms_start);
+    Stop (never);
+    Extra (ue, notify_status, send_status);
+};
+
+Gog mms {
+    Member mmse_over_http (trx);
+    Member mmse_over_wsp (trx);
+    Extra (ue, notify_status, send_status, resp, host, trx);
+    Expiration 60.0;
+};
+----
+
+[#ChMateConfigurationLibrary]
+
+=== MATE's configuration library
+
+The MATE library (will) contains GOP definitions for several protocols. Library
+protocols are included in your MATE config using: +_Action=Include;
+Lib=proto_name;_+.
+
+For Every protocol with a library entry, we'll find defined what from the PDU is
+needed to create a GOP for that protocol, eventually any criteria and the very
+essential GOP definition (i.e., __Gop__, _Start_ and _Stop_).
+
+[NOTE]
+====
+It seems that this code is written in the old syntax of MATE. So far it has not
+been transcribed into the new format. It may still form the basis to recreate
+these in the new format.
+====
+
+==== General use protocols
+
+===== TCP
+
+It will create a GOP for every TCP session. If it is used it should be the last
+one in the list. And every other proto on top of TCP should be declared with
+_LastPdu=TRUE;_ so that a TCP PDU is not created where another pdu type exists.
+
+----
+Transform add_tcp_stop {
+    Match (tcp_flags_reset="True") Insert (tcp_stop="True");
+    Match (tcp_flags_fin="True") Insert (tcp_stop="True");
+};
+
+Pdu tcp_pdu Proto tcp Transport ip {
+    Extract addr From ip.addr;
+    Extract port From tcp.port;
+    Extract tcp_start From tcp.flags.syn;
+    Extract tcp_flags_reset From tcp.flags.reset;
+    Extract tcp_flags_fin From tcp.flags.fin;
+    Transform add_tcp_stop;
+};
+
+Gop tcp_ses On tcp_pdu Match (addr, addr, port, port) {
+    Start (tcp_start="True");
+    Stop (tcp_stop="True");
+};
+
+Done;
+----
+
+===== DNS
+
+will create a GOP containing every request and its response (eventually
+retransmissions too).
+
+----
+Action=PduDef; Name=dns_pdu; Proto=dns; Transport=udp/ip; addr=ip.addr; port=udp.port; dns_id=dns.id; dns_rsp=dns.flags.response;
+
+Action=GopDef; Name=dns_req; On=dns_pdu; addr; addr; port!53; dns_id;
+Action=GopStart; For=dns_req; dns_rsp=0;
+Action=GopStop; For=dns_req; dns_rsp=1;
+----
+
+===== RADIUS
+
+A GOP for every transaction.
+
+----
+Action=PduDef; Name=radius_pdu; Proto=radius; Transport=udp/ip; addr=ip.addr; port=udp.port; radius_id=radius.id; radius_code=radius.code;
+
+Action=GopDef; Name=radius_req; On=radius_pdu; radius_id; addr; addr; port; port;
+Action=GopStart; For=radius_req; radius_code|1|4|7;
+Action=GopStop; For=radius_req; radius_code|2|3|5|8|9;
+----
+
+===== RTSP
+
+----
+Action=PduDef; Name=rtsp_pdu; Proto=rtsp; Transport=tcp/ip; addr=ip.addr; port=tcp.port; rtsp_method=rtsp.method;
+Action=PduExtra; For=rtsp_pdu; rtsp_ses=rtsp.session; rtsp_url=rtsp.url;
+
+Action=GopDef; Name=rtsp_ses; On=rtsp_pdu; addr; addr; port; port;
+Action=GopStart; For=rtsp_ses; rtsp_method=DESCRIBE;
+Action=GopStop; For=rtsp_ses; rtsp_method=TEARDOWN;
+Action=GopExtra; For=rtsp_ses; rtsp_ses; rtsp_url;
+----
+
+==== VoIP/Telephony
+
+Most protocol definitions here will create one GOP for every Call Leg unless
+stated.
+
+===== ISUP
+
+----
+Action=PduDef; Name=isup_pdu; Proto=isup; Transport=mtp3; mtp3pc=mtp3.dpc; mtp3pc=mtp3.opc; cic=isup.cic; isup_msg=isup.message_type;
+
+Action=GopDef; Name=isup_leg; On=isup_pdu; ShowPduTree=TRUE; mtp3pc; mtp3pc; cic;
+Action=GopStart; For=isup_leg; isup_msg=1;
+Action=GopStop; For=isup_leg; isup_msg=16;
+----
+
+===== Q931
+
+----
+Action=PduDef; Name=q931_pdu; Proto=q931; Stop=TRUE; Transport=tcp/ip; addr=ip.addr; call_ref=q931.call_ref; q931_msg=q931.message_type;
+
+Action=GopDef; Name=q931_leg; On=q931_pdu; addr; addr; call_ref;
+Action=GopStart; For=q931_leg; q931_msg=5;
+Action=GopStop; For=q931_leg; q931_msg=90;
+----
+
+===== H225 RAS
+
+----
+Action=PduDef; Name=ras_pdu; Proto=h225.RasMessage; Transport=udp/ip; addr=ip.addr; ras_sn=h225.RequestSeqNum; ras_msg=h225.RasMessage;
+Action=PduExtra; For=ras_pdu; guid=h225.guid;
+
+Action=GopDef; Name=ras_leg; On=ras_pdu; addr; addr; ras_sn;
+Action=GopStart; For=ras_leg; ras_msg|0|3|6|9|12|15|18|21|26|30;
+Action=GopStop; For=ras_leg; ras_msg|1|2|4|5|7|8|10|11|13|14|16|17|19|20|22|24|27|28|29|31;
+Action=GopExtra; For=ras_leg; guid;
+----
+
+===== SIP
+
+----
+Action=PduDef; Proto=sip_pdu; Transport=tcp/ip; addr=ip.addr; port=tcp.port; sip_method=sip.Method; sip_callid=sip.Call-ID; calling=sdp.owner.username;
+
+Action=GopDef; Name=sip_leg; On=sip_pdu; addr; addr; port; port;
+Action=GopStart; For=sip; sip_method=INVITE;
+Action=GopStop; For=sip; sip_method=BYE;
+----
+
+===== MEGACO
+
+Will create a GOP out of every transaction.
+
+To "tie" them to your call's GoG use: _Action=GogKey; Name=your_call; On=mgc_tr;
+addr!mgc_addr; megaco_ctx;_
+
+----
+Action=PduDef; Name=mgc_pdu; Proto=megaco; Transport=ip; addr=ip.addr; megaco_ctx=megaco.context; megaco_trx=megaco.transid; megaco_msg=megaco.transaction; term=megaco.termid;
+
+Action=GopDef; Name=mgc_tr; On=mgc_pdu; addr; addr; megaco_trx;
+Action=GopStart; For=mgc_tr; megaco_msg|Request|Notify;
+Action=GopStop; For=mgc_tr; megaco_msg=Reply;
+Action=GopExtra; For=mgc_tr; term^DS1; megaco_ctx!Choose one;
+----
+
+[#ChMateReferenceManual]
+
+=== MATE's reference manual
+
+==== Attribute Value Pairs (AVP)
+
+MATE uses AVPs for almost everything: to keep the data it has extracted from the
+frames' trees as well as to keep the elements of the configuration.
+
+These "pairs" (actually tuples) are made of a name, a value and, in case of
+configuration AVPs, an operator. Names and values are strings. AVPs with
+operators other than '=' are used only in the configuration and are used for
+matching AVPs of PDUs, GOPs and GOGs in the analysis phase.
+
+===== Name
+
+The name is a string used to refer to a type of AVP. Two attributes won't
+match unless their names are identical. Capitalized names are reserved for
+keywords (you can use them for your elements if you want but I think it's not
+the case). MATE attribute names can be used in Wireshark's display filters the
+same way like names of protocol fields provided by dissectors, but they are not
+just references to (or aliases of) protocol fields.
+
+===== Value
+
+The value is a string. It is either set in the configuration (for configuration
+AVPs) or by MATE while extracting interesting fields from a dissection tree
+and/or manipulating them later. The values extracted from fields use the same
+representation as they do in filter strings.
+
+==== AVP Operators (=,!,{},^,$,~,<,>,?)
+
+Currently only match operators are defined (there are plans to (re)add transform
+attributes but some internal issues have to be solved before that). The match
+operations are always performed between two operands: the value of an AVP stated
+in the configuration and the value of an AVP (or several AVPs with the same name)
+extracted from packet data (called "data AVPs"). It is not possible to match
+data AVPs to each other.
+
+The defined match operators are:
+
+* <<Equal,Equal>> _=_ test for equality, that is: either the value strings are identical
+or the match will fail.
+* <<NotEqual,Not Equal>> _!_ will match only if the value strings aren't equal.
+* <<OneOf,One Of>> _{}_ will match if one of the value strings listed is equal to the
+data AVP's string. Items inside the list's curly braces are
+separated with the | character.
+* <<StartsWith,Starts With>> _^_ will match if the configuration value string matches the
+first characters of the data AVP's value string.
+* <<EndsWith,Ends With>> _$_ will match if the configuration value string matches the
+last characters of the data AVP's value string.
+* <<Contains,Contains>> _~_ will match if the configuration value string matches a
+substring of the characters of the data AVP's value string.
+* <<LowerThan,Lower Than>> _<_ will match if the data AVP's value string is semantically
+lower than the configuration value string.
+* <<HigherThan,Higher Than>> _>_ will match if the data AVP's value string is semantically
+higher than the configuration value string.
+* <<Exists,Exists>> _?_ (can be omitted) will match if the AVP name matches, regardless
+what the value string is.
+
+[#Equal]
+===== Equal AVP Operator (=)
+
+This operator tests whether the values of the operator and the operand AVP are
+equal.
+
+Example::
+attrib=aaa *matches* attrib=aaa +
+attrib=aaa *does not match* attrib=bbb
+
+[#NotEqual]
+===== Not equal AVP operator (!)
+
+This operator matches if the value strings of two AVPs are not equal.
+
+Example::
+attrib=aaa matches attrib!bbb +
+attrib=aaa does not match attrib!aaa
+
+[#OneOf]
+===== "One of" AVP operator ({})
+
+The "one of" operator matches if the data AVP value is equal to one of the
+values listed in the "one of" AVP.
+
+Example::
+attrib=1 matches attrib{1|2|3} +
+attrib=2 matches attrib{1|2|3} +
+attrib=4 does not match attrib{1|2|3}
+
+[#StartsWith]
+===== "Starts with" AVP operator (^)
+
+The "starts with" operator matches if the first characters of the data AVP
+value are identical to the configuration AVP value.
+
+Example::
+attrib=abcd matches attrib^abc +
+attrib=abc matches attrib^abc +
+attrib=ab does not match attrib^abc +
+attrib=abcd does not match attrib^bcd +
+attrib=abc does not match attrib^abcd +
+
+[#EndsWith]
+===== "Ends with" operator ($)
+
+The ends with operator will match if the last bytes of the data AVP value are
+equal to the configuration AVP value.
+
+Example::
+attrib=wxyz matches attrib$xyz +
+attrib=yz does not match attrib$xyz +
+attrib=abc...wxyz does not match attrib$abc
+
+[#Contains]
+===== Contains operator (~)
+
+The "contains" operator will match if the data AVP value contains a string
+identical to the configuration AVP value.
+
+Example::
+attrib=abcde matches attrib~bcd +
+attrib=abcde matches attrib~abc +
+attrib=abcde matches attrib~cde +
+attrib=abcde does not match attrib~xyz
+
+[#LowerThan]
+===== "Lower than" operator (<)
+
+The "lower than" operator will match if the data AVP value is semantically lower
+than the configuration AVP value.
+
+Example::
+attrib=abc matches attrib<bcd +
+attrib=1 matches attrib<2 +
+but beware: attrib=10 does not match attrib<9 +
+attrib=bcd does not match attrib<abc +
+attrib=bcd does not match attrib<bcd +
+
+BUGS
+
+It should check whether the values are numbers and compare them numerically
+
+[#HigherThan]
+===== "Higher than" operator (>)
+
+The "higher than" operator will match if the data AVP value is semantically
+higher than the configuration AVP value.
+
+Examples
+
+attrib=bcd matches attrib>abc +
+attrib=3 matches attrib>2 +
+but beware: attrib=9 does not match attrib>10 +
+attrib=abc does not match attrib>bcd +
+attrib=abc does not match attrib>abc +
+
+BUGS
+
+It should check whether the values are numbers and compare them numerically
+
+[#Exists]
+===== Exists operator (?)
+
+The exists operator will always match as far as the two operands have the same
+name.
+
+Examples
+
+attrib=abc matches attrib? +
+attrib=abc matches attrib (this is just an alternative notation of the previous example) +
+obviously attrib=abc does not match other_attrib? +
+
+==== Attribute Value Pair List (AVPL)
+PDUs, GOPs and GOGs use an AVPL to contain the tracing information. An AVPL is
+an unsorted set of <<AVP,AVPs>> that can be matched against other AVPLs.
+
+[#Match]
+==== Operations between AVPLs (Match)
+
+There are three types of match operations that can be performed between AVPLs.
+The PDU's/GOP's/GOG's AVPL will be always one of the operands; the AVPL operator
+(match type) and the second operand AVPL will always come from the
+<<Config,configuration>>.
+Note that a diverse AVP match operator may be specified for each AVP in the
+configuration AVPL.
+
+An AVPL match operation returns a result AVPL. In <<Transform,Transform>>s, the
+result AVPL may be replaced by another AVPL. The replacement means that the
+existing data AVPs are dropped and the replacement AVPL from the
+<<Config,configuration>> is <<Merge,Merged>> to the data AVPL of the
+PDU/GOP/GOG.
+
+* <<Loose,Loose Match>>: Will match if at least one of the AVPs of the two
+operand AVPLs match. If it matches, it returns a result AVPL containing all AVPs
+from the data AVPL that did match the configuration's AVPs.
+* <<Every,"Every" Match>>: Will match if none of the AVPs of the configuration
+AVPL fails to match an AVP in the data AVPL, even if not all of the
+configuration AVPs have a match. If it matches, it returns a result AVPL
+containing all AVPs from the data AVPL that did match an AVP in the
+configuration AVPL.
+* <<Strict,Strict Match>>: Will match if and only if each of the AVPs in the
+configuration AVPL has at least one match in the data AVPL. If it matches, it
+returns a result AVPL containing those AVPs from the data AVPL that matched.
+
+[#Loose]
+===== Loose Match
+
+A loose match between AVPLs succeeds if at least one of the data AVPs matches at
+least one of the configuration AVPs. Its result AVPL contains all the data AVPs
+that matched.
+
+Loose matches are used in Extra operations against the <<Pdu,PDU>>'s AVPL to
+merge the result into <<Gop,GOP>>'s AVPL, and against <<Gop,GOP>>'s AVPL to
+merge the result into <<Gog,GOG>>'s AVPL. They may also be used in
+<<Criteria,Criteria>> and <<Transform,Transform>>s.
+
+[NOTE]
+====
+As of current (2.0.1), Loose Match does not work as described here, see
+https://gitlab.com/wireshark/wireshark/issues/12184[issue 12184]. Only use
+in Transforms and Criteria is effectively affected by the bug.
+====
+
+Loose Match Examples
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a?, attr_c=ccc) ==> (attr_a=aaa)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a=xxx; attr_c=ccc) ==> No Match!
+
+[#Every]
+===== Every Match
+
+An "every" match between AVPLs succeeds if none of the configuration's AVPs that
+have a counterpart in the data AVPL fails to match. Its result AVPL contains all
+the data AVPs that matched.
+
+These may only be used in <<Criteria,Criteria>> and <<Transform,Transform>>s.
+
+[NOTE]
+====
+As of current (2.0.1), Loose Match does not work as described here, see
+https://gitlab.com/wireshark/wireshark/-/issues/12184[issue 12184].
+====
+
+"Every" Match Examples
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c?, attr_d=ddd) ==> (attr_a=aaa, attr_c=xxx)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c=ccc) ==> No Match!
+
+(attr_a=aaa; attr_b=bbb; attr_c=xxx) Match Every (attr_a=xxx, attr_c=ccc) ==> No Match!
+
+[#Strict]
+===== Strict Match
+
+A Strict match between AVPLs succeeds if and only if every AVP in the
+configuration AVPL has at least one counterpart in the data AVPL and none of the
+AVP matches fails. The result AVPL contains all the data AVPs that matched.
+
+These are used between GOP keys (key AVPLs) and PDU AVPLs. They may also be used
+in <<Criteria,Criteria>> and <<Transform,Transform>>s.
+
+Examples
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c=xxx) ==> (attr_a=aaa, attr_c=xxx)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx, attr_c=yyy) Match Strict (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx, attr_c=yyy)
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c=ccc) ==> No Match!
+
+(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c?, attr_d?) ==> No Match!
+
+[#Merge]
+==== AVPL Merge
+
+An AVPL may be merged into another one. That would add to the latter every AVP
+from the former that does not already exist there.
+
+This operation is done
+
+* between the result of a key match and the GOP's or GOG's AVPL,
+* between the result of an Extra match and the GOP's or GOG's AVPL,
+* between the result of a <<Transform,Transform>> match and PDU's/GOP's AVPL. If
+the operation specified by the Match clause is Replace, the result AVPL of the
+match is removed from the item's AVPL before the modify_avpl is merged into it.
+
+Examples
+
+(attr_a=aaa, attr_b=bbb) "merge" (attr_a=aaa, attr_c=xxx) former becomes (attr_a=aaa, attr_b=bbb, attr_c=xxx)
+
+Can't have multiple "attr_a" with same value "aaa"
+
+(attr_a=aaa, attr_b=bbb) "merge" (attr_a=aaa, attr_a=xxx) former becomes (attr_a=aaa, attr_a=xxx, attr_b=bbb)
+
+Multiple "attr_a" with different values "aaa" and "xxx"
+
+(attr_a=aaa, attr_b=bbb) "merge" (attr_c=xxx, attr_d=ddd) former becomes (attr_a=aaa, attr_b=bbb, attr_c=xxx, attr_d=ddd)
+
+All AVP names are unique so resulting AVPL contains all AVPs from both AVPLs
+
+[#Config]
+=== Configuration Reference (mate.config)
+[#Pdu]
+==== PDU declaration block
+
+The following configuration AVPLs deal with PDU creation and data extraction.
+
+===== _Pdu_ declaration block header
+
+In each frame of the capture, MATE will look for source _proto_name_'s PDUs in
+the order in which the declarations appear in its configuration and will create
+PDUs of every type it can from that frame, unless specifically instructed that
+some PDU type is the last one to be looked for in the frame. If told so for a
+given type, MATE will extract all PDUs of that type and the previously declared
+types it finds in the frame but not those declared later.
+
+The complete declaration of a _Pdu_ looks as below; the mandatory order of the
+diverse clauses is as shown.
+
+----
+Pdu name Proto proto_name Transport {proto1[/proto2/proto3[/...]|mate}; {
+    Payload proto; //optional, no default value
+    Extract attribute From proto.field ; //may occur multiple times, at least once
+    Transform transform1[, transform2[, ...]]; //optional
+    Criteria {Accept|Reject} {Strict|Every|Loose} match_avpl; //optional
+    DropUnassigned {TRUE|FALSE}; //optional, default=FALSE
+    DiscardPduData {TRUE|FALSE}; //optional, default=FALSE
+    LastPdu {TRUE|FALSE}; //optional, default=FALSE
+};
+----
+
+====== Pdu name
+
+The _name_ is a mandatory attribute of a _Pdu_ declaration. It is chosen
+arbitrarily, except that each _name_ may only be used once in MATE's
+configuration, regardless the class of an item it is used for. The _name_ is
+used to distinguish between different types of PDUs, GOPs, and GOGs. The _name_
+is also used as part of the filterable fields' names related to this type of PDU
+which MATE creates.
+
+However, several _Pdu_ declarations may share the same _name_. In such case, all
+of them are created from each source PDU matching their _Proto_, _Transport_,
+and _Payload_ clauses, while the bodies of their declarations may be totally
+different from each other. Together with the _Accept_ (or _Reject_) clauses,
+this feature is useful when it is necessary to build the PDU's AVPL from
+different sets of source fields depending on contents (or mere presence) of
+other source fields.
+
+====== Proto and Transport clauses
+
+Every instance of the protocol _proto_name_ PDU in a frame will generate one
+PDU with the AVPs extracted from fields that are in the _proto_name_'s range
+and/or the ranges of underlying protocols specified by the _Transport_ list.
+It is a mandatory attribute of a _Pdu_ declaration. The _proto_name_ is the name
+of the protocol as used in Wireshark display filter.
+
+The PDU's _Proto_, and its _Transport_ list of protocols separated by / tell
+MATE which fields of a frame can get into the PDU's AVPL. In order that MATE
+would extract an attribute from a frame's protocol tree, the area representing
+the field in the hex display of the frame must be within the area of either the
+_Proto_ or its relative _Transport_++s++. _Transport_++s++ are chosen moving backwards
+from the protocol area, in the order they are given.
+
+_Proto http Transport tcp/ip_ does what you'd expect it to - it selects the
+nearest tcp range that precedes the current http range, and the nearest ip range
+that precedes that tcp range. If there is another ip range before the nearest
+one (e.g., in case of IP tunneling), that one is not going to be selected.
+_Transport_ tcp/ip/ip that "logically" should select the encapsulating IP header
+too doesn't work so far.
+
+Once we've selected the _Proto_ and _Transport_ ranges, MATE will fetch those
+protocol fields belonging to them whose extraction is declared using the
+_Extract_ clauses for the PDU type. The _Transport_ list is also mandatory,
+if you actually don't want to use any transport protocol, use _Transport mate_.
+(This didn't work until 0.10.9).
+
+===== Payload clause
+
+Other than the PDU's _Proto_ and its _Transport_ protocols, there is also a
+_Payload_ attribute to tell MATE from which ranges of _Proto_'s payload to
+extract fields of a frame into the PDU. In order to extract an attribute from a
+frame's tree the highlighted area of the field in the hex display must be within
+the area of the _Proto_'s relative payload(s). _Payload_++s++ are chosen moving
+forward from the protocol area, in the order they are given.
+_Proto http Transport tcp/ip Payload mmse_ will select the first mmse range
+after the current http range. Once we've selected the _Payload_ ranges, MATE
+will fetch those protocol fields belonging to them whose extraction is declared
+using the _Extract_ clauses for the PDU type.
+
+===== Extract clause
+
+Each _Extract_ clause tells MATE which protocol field value to extract as an AVP
+value and what string to use as the AVP name. The protocol fields are referred
+to using the names used in Wireshark display filters. If there is more than one
+such protocol field in the frame, each instance that fulfills the criteria
+stated above is extracted into its own AVP. The AVP names may be chosen
+arbitrarily, but to be able to match values originally coming from different
+PDUs (e.g., hostname from DNS query and a hostname from HTTP GET request) later
+in the analysis, identical AVP names must be assigned to them and the dissectors
+must provide the field values in identical format (which is not always the case).
+
+===== Transform clause
+
+The _Transform_ clause specifies a list of previously declared _Transform_++s++ to
+be performed on the PDU's AVPL after all protocol fields have been extracted to
+it. The list is always executed completely, left to right. On the contrary, the
+list of Match clauses inside each individual _Transform_ is executed only until
+the first match succeeds.
+
+[#Criteria]
+===== Criteria clause
+
+This clause tells MATE whether to use the PDU for analysis. It specifies a match
+AVPL, an AVPL <<Match,Match type>> (_Strict_, _Every_, or _Loose_) and the action to be
+performed (_Accept_ or _Reject_) if the match succeeds. Once every attribute has
+been extracted and eventual transform list has been executed, and if the
+_Criteria_ clause is present, the PDU's AVPL is matched against the match AVPL;
+if the match succeeds, the action specified is executed, i.e., the PDU is
+accepted or rejected. The default behaviors used if the respective keywords are
+omitted are _Strict_ and _Accept_. Accordingly, if the clause is omitted, all
+PDUs are accepted.
+
+===== DropUnassigned clause
+
+If set to _TRUE_, MATE will destroy the PDU if it cannot assign it to a GOP.
+If set to _FALSE_ (the default if not given), MATE will keep them.
+
+===== DiscardPduData clause
+
+If set to _TRUE_, MATE will delete the PDU's AVPL once it has analyzed it and
+eventually extracted some AVPs from it into the GOP's AVPL. This is useful to
+save memory (of which MATE uses a lot). If set to _FALSE_ (the default if not
+given), MATE will keep the PDU attributes.
+
+===== LastPdu clause
+
+If set to _FALSE_ (the default if not given), MATE will continue to look for
+PDUs of other types in the frame. If set to _TRUE_, it will not try to create
+PDUs of other types from the current frame, yet it will continue to try for the
+current type.
+
+[#Gop]
+==== GOP declaration block
+
+===== _Gop_ declaration block header
+
+Declares a Gop type and its candidate key.
+
+----
+Gop name On pduname Match key {
+    Start match_avpl; // optional
+    Stop match_avpl; // optional
+    Extra match_avpl; // optional
+    Transform transform_list; // optional
+    Expiration time; // optional
+    IdleTimeout time; // optional
+    Lifetime time; // optional
+    DropUnassigned [TRUE|FALSE]; //optional
+    ShowTree [NoTree|PduTree|FrameTree|BasicTree]; //optional
+    ShowTimes [TRUE|FALSE]; //optional, default TRUE
+};
+----
+
+====== Gop name
+
+The _name_ is a mandatory attribute of a _Gop_ declaration. It is chosen
+arbitrarily, except that each _name_ may only be used once in MATE's
+configuration, regardless the class of an item it is used for. The _name_ is
+used to distinguish between different types of PDUs, GOPs, and GOGs. The _name_
+is also used as part of the filterable fields' names related to this type of
+GOP which MATE creates.
+
+====== On clause
+
+The _name_ of PDUs which this type of GOP is supposed to be grouping. It is
+mandatory.
+
+====== Match clause
+
+Defines what AVPs form up the _key_ part of the GOP's AVPL (the GOP's _key_ AVPL
+or simply the GOP's _key_). All PDUs matching the _key_ AVPL of an active GOP
+are assigned to that GOP; a PDU which contains the AVPs whose attribute names
+are listed in the GOP's _key_ AVPL, but they do not strictly match any active
+GOP's _key_ AVPL, will create a new GOP (unless a _Start_ clause is given).
+When a GOP is created, the elements of its key AVPL are copied from the creating
+PDU.
+
+===== Start clause
+
+If given, it tells MATE what match_avpl must a PDU's AVPL match, in addition to
+matching the GOP's _key_, in order to start a GOP. If not given, any PDU whose
+AVPL matches the GOP's _key_ AVPL will act as a start for a GOP. The PDU's AVPs
+matching the match_avpl are not automatically copied into the GOP's AVPL.
+
+===== Stop clause
+
+If given, it tells MATE what match_avpl must a PDU's AVPL match, in addition to
+matching the GOP's _key_, in order to stop a GOP. If omitted, the GOP is
+"auto-stopped" - that is, the GOP is marked as stopped as soon as it is created.
+The PDU's AVPs matching the match_avpl are not automatically copied into the
+GOP's AVPL.
+
+===== Extra clause
+
+If given, tells MATE which AVPs from the PDU's AVPL are to be copied into the
+GOP's AVPL in addition to the GOP's key.
+
+===== Transform clause
+
+The _Transform_ clause specifies a list of previously declared _Transform_++s++ to
+be performed on the GOP's AVPL after the AVPs from each new PDU, specified by
+the _key_ AVPL and the _Extra_ clause's match_avpl, have been merged into it.
+The list is always executed completely, left to right. On the contrary, the list
+of _Match_ clauses inside each individual _Transform_ is executed only until
+the first match succeeds.
+
+===== Expiration clause
+
+A (floating) number of seconds after a GOP is _Stop_ ped during which further
+PDUs matching the _Stop_ ped GOP's key but not the _Start_ condition will still
+be assigned to that GOP. The default value of zero has an actual meaning of
+infinity, as it disables this timer, so all PDUs matching the _Stop_ ped GOP's
+key will be assigned to that GOP unless they match the _Start_ condition.
+
+===== IdleTimeout clause
+
+A (floating) number of seconds elapsed from the last PDU assigned to the GOP
+after which the GOP will be considered released. The default value of zero has
+an actual meaning of infinity, as it disables this timer, so the GOP won't be
+released even if no PDUs arrive - unless the _Lifetime_ timer expires.
+
+===== Lifetime clause
+
+A (floating) of seconds after the GOP _Start_ after which the GOP will be
+considered released regardless anything else. The default value of zero has an
+actual meaning of infinity.
+
+===== DropUnassigned clause
+
+Whether or not a GOP that has not being assigned to any GOG should be discarded.
+If _TRUE_, the GOP is discarded right after creation. If _FALSE_, the default,
+the unassigned GOP is kept. Setting it to _TRUE_ helps save memory and speed up
+filtering.
+
+===== TreeMode clause
+
+Controls the display of PDUs subtree of the GOP:
+
+* _NoTree_: completely suppresses showing the tree
+* _PduTree_: the tree is shown and shows the PDUs by PDU Id
+* _FrameTree_: the tree is shown and shows the PDUs by the frame number in which
+they are
+* _BasicTree_: needs investigation
+
+===== ShowTimes clause
+
+Whether or not to show the times subtree of the GOP. If _TRUE_, the default,
+the subtree with the timers is added to the GOP's tree. If _FALSE_, the subtree
+is suppressed.
+
+[#Gog]
+==== GOG declaration block
+
+===== _Gog_ declaration block header
+
+Declares a Gog type and its candidate key.
+
+----
+Gog name {
+    Member gopname (key); // mandatory, at least one
+    Extra match_avpl; // optional
+    Transform transform_list; // optional
+    Expiration time; // optional, default 2.0
+    GopTree [NoTree|PduTree|FrameTree|BasicTree]; // optional
+    ShowTimes [TRUE|FALSE]; // optional, default TRUE
+};
+----
+
+====== Gog name
+
+The _name_ is a mandatory attribute of a _Gog_ declaration. It is chosen
+arbitrarily, except that each _name_ may only be used once in MATE's
+configuration, regardless the class of an item it is used for. The _name_ is
+used to distinguish between different types of PDUs, GOPs, and GOGs. The _name_
+is also used as part of the filterable fields' names related to this type of
+GOG which MATE creates.
+
+===== Member clause
+
+Defines the _key_ AVPL for the GOG individually for each GOP type _gopname_.
+All _gopname_ type GOPs whose _key_ AVPL matches the corresponding _key_ AVPL
+of an active GOG are assigned to that GOG; a GOP which contains the AVPs whose
+attribute names are listed in the GOG's corresponding _key_ AVPL, but they do
+not strictly match any active GOG's _key_ AVPL, will create a new GOG. When a
+GOG is created, the elements of its _key_ AVPL are copied from the creating GOP.
+
+Although the _key_ AVPLs are specified separately for each of the Member
+_gopname_++s++, in most cases they are identical, as the very purpose of a GOG is
+to group together GOPs made of PDUs of different types.
+
+===== Extra clause
+
+If given, tells MATE which AVPs from any of the GOP's AVPL are to be copied
+into the GOG's AVPL in addition to the GOG's key.
+
+===== Expiration clause
+
+A (floating) number of seconds after all the GOPs assigned to a GOG have been
+released during which new GOPs matching any of the session keys should still be
+assigned to the existing GOG instead of creating a new one. Its value can range
+from 0.0 to infinite. Defaults to 2.0 seconds.
+
+===== Transform clause
+
+The _Transform_ clause specifies a list of previously declared _Transform_++s++ to
+be performed on the GOG's AVPL after the AVPs from each new GOP, specified by
+the _key_ AVPL and the _Extra_ clause's match_avpl, have been merged into it.
+The list is always executed completely, left to right. On the contrary, the list
+of _Match_ clauses inside each individual _Transform_ is executed only until
+the first match succeeds.
+
+===== TreeMode clause
+
+Controls the display of GOPs subtree of the GOG:
+
+* _NoTree_: completely suppresses showing the tree
+* _BasicTree_: needs investigation
+* _FullTree_: needs investigation
+
+===== ShowTimes clause
+
+Whether or not to show the times subtree of the GOG. If _TRUE_, the default,
+the subtree with the timers is added to the GOG's tree. If _FALSE_, the subtree
+is suppressed.
+
+[#Transform]
+==== Transform declaration block
+
+A Transform is a sequence of Match rules optionally followed by an instruction
+how to modify the match result using an additional AVPL. Such modification may
+be an Insert (merge) or a Replace. The syntax is as follows:
+
+----
+Transform name {
+    Match [Strict|Every|Loose] match_avpl [[Insert|Replace] modify_avpl] ; // may occur multiple times, at least once
+};
+----
+
+For examples of Transforms, check the <<ChMateManual,Manual>> page.
+
+TODO: migrate the examples here?
+
+The list of Match rules inside a Transform is processed top to bottom;
+the processing ends as soon as either a Match rule succeeds or all have been
+tried in vain.
+
+Transforms can be used as helpers to manipulate an item's AVPL before the item
+is processed further. An item declaration may contain a Transform clause
+indicating a list of previously declared Transforms. Regardless whether the
+individual transforms succeed or fail, the list is always executed completely
+and in the order given, i.e., left to right.
+
+In MATE configuration file, a Transform must be declared before declaring any
+item which uses it.
+
+==== Settings configuration AVPL
+
+[NOTE]
+====
+The *Settings* parameters have been moved to other configuration parameters
+or deprecated. Leave for now until rest of document is updated for current syntax.
+====
+
+The *Settings* config element is used to pass to MATE various operational
+parameters. the possible parameters are
+
+===== GogExpiration
+
+How long in seconds after all the GOPs assigned to a GOG have been released new
+GOPs matching any of the session keys should create a new GOG instead of being
+assigned to the previous one. Its value can range from 0.0 to infinite.
+Defaults to 2.0 seconds.
+
+===== DiscardPduData
+
+Whether or not the AVPL of every PDU should be deleted after it was being
+processed (saves memory). It can be either _TRUE_ or _FALSE_. Defaults to _TRUE_.
+Setting it to _FALSE_ can save you from a headache if your config does not work.
+
+===== DiscardUnassignedPdu
+
+Whether PDUs should be deleted if they are not assigned to any GOP. It can be
+either _TRUE_ or _FALSE_. Defaults to _FALSE_. Set it to _TRUE_ to save memory
+if unassigned PDUs are useless.
+
+===== DiscardUnassignedGop
+
+Whether GOPs should be deleted if they are not assigned to any session. It can
+be either _TRUE_ or _FALSE_. Defaults to _FALSE_. Setting it to _TRUE_ saves
+memory.
+
+===== ShowPduTree
+
+===== ShowGopTimes
+
+==== Debugging Stuff
+
+The following settings are used to debug MATE and its configuration. All levels
+are integers ranging from 0 (print only errors) to 9 (flood me with junk),
+defaulting to 0.
+
+===== Debug declaration block header
+
+----
+Debug {
+    Filename "path/name"; //optional, no default value
+    Level [0-9]; //optional, generic debug level
+    Pdu Level [0-9]; //optional, specific debug level for Pdu handling
+    Gop Level [0-9]; //optional, specific debug level for Gop handling
+    Gog Level [0-9]; //optional, specific debug level for Gog handling
+};
+----
+
+====== Filename clause
+
+The  {{{path/name}}} is a full path to the file to which debug output is to be
+written. Non-existent file will be created, existing file will be overwritten
+at each opening of a capture file. If the statement is missing, debug messages
+are written to console, which means they are invisible on Windows.
+
+====== Level clause
+
+Sets the level of debugging for generic debug messages. It is an integer
+ranging from 0 (print only errors) to 9 (flood me with junk).
+
+====== Pdu Level clause
+
+Sets the level of debugging for messages regarding PDU creation. It is an
+integer ranging from 0 (print only errors) to 9 (flood me with junk).
+
+====== Gop Level clause
+
+Sets the level of debugging for messages regarding PDU analysis (that is how do
+they fit into ?GOPs). It is an integer ranging from 0 (print only errors) to 9
+(flood me with junk).
+
+====== Gog Level clause
+
+Sets the level of debugging for messages regarding GOP analysis (that is how do
+they fit into ?GOGs). It is an integer ranging from 0 (print only errors) to 9
+(flood me with junk).
+
+===== Settings Example
+----
+Action=Settings; SessionExpiration=3.5; DiscardPduData=FALSE;
+----
+
+==== Action=Include
+
+Will include a file to the configuration.
+
+----
+Action=Include; {Filename=filename;|Lib=libname;}
+----
+
+===== Filename
+
+The filename of the file to include. If it does not begin with '/' it will look
+for the file in the current path.
+
+===== Lib
+
+The name of the lib config to include. will look for libname.mate in
+wiresharks_dir/matelib.
+
+===== Include Example
+----
+Action=Include; Filename=rtsp.mate;
+----
+
+This will include the file called "rtsp.mate" into the current config.
-- 
cgit v1.2.3

