diff options
Diffstat (limited to 'docbook/wsug_src/wsug_mate.adoc')
| -rw-r--r-- | docbook/wsug_src/wsug_mate.adoc | 2318 |
1 files changed, 0 insertions, 2318 deletions
diff --git a/docbook/wsug_src/wsug_mate.adoc b/docbook/wsug_src/wsug_mate.adoc deleted file mode 100644 index 6de78c70..00000000 --- a/docbook/wsug_src/wsug_mate.adoc +++ /dev/null @@ -1,2318 +0,0 @@ -[#ChMate] - -== MATE - -[#ChMateIntroduction] - -=== Introduction - -MATE: Meta Analysis and Tracing Engine - -What is MATE? Well, to keep it very short, with MATE you can create user -configurable extension(s) of the display filter engine. - -MATE's goal is to enable users to filter frames based on information extracted -from related frames or information on how frames relate to each other. MATE -was written to help troubleshooting gateways and other systems where a "use" -involves more protocols. However, MATE can be used as well to analyze other -issues regarding an interaction between packets like response times, -incompleteness of transactions, presence/absence of certain attributes in a -group of PDUs and more. - -MATE is a Wireshark plugin that allows the user to specify how different -frames are related to each other. To do so, MATE extracts data from the frames' -tree and then, using that information, tries to group the frames based on how -MATE is configured. Once the PDUs are related, MATE will create a "protocol" -tree with fields the user can filter with. The fields will be almost the same -for all the related frames, so one can filter a complete session spanning -several frames containing more protocols based on an attribute appearing in -some related frame. Other than that MATE allows to filter frames based on -response times, number of PDUs in a group and a lot more. - -So far MATE has been used to: - -* Filter all packets of a call using various protocols knowing just the -calling number. (MATE's original goal) -* Filter all packets of all calls using various protocols based on the release -cause of one of its "segments". -* Extrapolate slow transactions from very "dense" captures. (finding requests -that timeout) -* Find incomplete transactions (no responses) -* Follow requests through more gateways/proxies. -* more... - -[#ChMateGettingStarted] - -=== Getting Started - -These are the steps to try out MATE: - -* Run Wireshark and check if the plugin is installed correct (MATE should -appear in Help->About->Plugins) -* Get a configuration file e.g., tcp.mate (see {wireshark-wiki-url}Mate/Examples[Mate/Examples] -for more) and place it somewhere on your harddisk. -* Go to Preferences->Protocols->MATE and set the config filename to the file -you want to use (you don't have to restart Wireshark) -* Load a corresponding capture file (e.g., -{wireshark-wiki-url}uploads/27707187aeb30df68e70c8fb9d614981/http.cap[http.cap]) and see if MATE -has added some new display filter fields, something like: `mate tcp_pdu:1->tcp_ses:1` -or, at prompt: `path_to/wireshark -o "mate.config: tcp.mate" -r http.cap`. - -If anything went well, your packet details might look something like this: - -image::images/ws-mate-tcp-output.png[] - -[#ChMateManual] - -=== MATE Manual - -==== Introduction - -MATE creates a filterable tree based on information contained in frames that -share some relationship with information obtained from other frames. The way -these relationships are made is described in a configuration file. The -configuration file tells MATE what makes a PDU and how to relate it to other -PDUs. - -MATE analyzes each frame to extract relevant information from the "protocol" -tree of that frame. The extracted information is contained in MATE PDUs; -these contain a list of relevant attributes taken from the tree. From now on, I -will use the term "PDU" to refer to the objects created by MATE containing the -relevant information extracted from the frame; I'll use "frame" to refer to the -"raw" information extracted by the various dissectors that pre-analyzed the frame. - -For every PDU, MATE checks if it belongs to an existing "Group of PDUs" (Gop). -If it does, it assigns the PDU to that Gop and moves any new relevant attributes -to the Gop's attribute list. How and when do PDUs belong to Gops is described -in the configuration file as well. - -Every time a Gop is assigned a new PDU, MATE will check if it matches the -conditions to make it belong to a "Group of Groups" (Gog). Naturally the -conditions that make a Gop belong to a Gog are taken from the configuration -file as well. - -Once MATE is done analyzing the frame it will be able to create a "protocol" -tree for each frame based on the PDUs, the Gops they belong to and naturally any -Gogs the former belongs to. - -How to tell MATE what to extract, how to group it and then how to relate those -groups is made using AVPs and AVPLs. - -Information in MATE is contained in Attribute/Value Pairs (AVPs). AVPs are made -of two strings: the name and the value. AVPs are used in the configuration and -there they have an operator as well. There are various ways AVPs can be matched -against each other using those operators. - -AVPs are grouped into AVP Lists (AVPLs). PDUs, Gops and Gogs have an AVPL each. -Their AVPLs will be matched in various ways against others coming from the -configuration file. - -MATE will be instructed how to extract AVPs from frames in order to create a PDU -with an AVPL. It will be instructed as well, how to match that AVPL against the -AVPLs of other similar PDUs in order to relate them. In MATE the relationship -between PDUs is a Gop, it has an AVPL as well. MATE will be configured with other -AVPLs to operate against the Gop's AVPL to relate Gops together into Gogs. - -A good understanding on how AVPs and AVPLs work is fundamental to understand how -MATE works. - -[#AVP] -==== Attribute Value Pairs - -Information used by MATE to relate different frames is contained in Attribute/ -Value Pairs (AVPs). AVPs are made of two strings - the name and the value. When -AVPs are used in the configuration, an operator is defined as well. There are -various ways AVPs can be matched against each other using those operators. - ----- - avp_name="avp's value" - another_name= "1234 is the value" ----- - -The name is a string used to refer to a "kind" of an AVP. Two AVPs won't match -unless their names are identical. - -You should not use uppercase characters in names, or names that start with “.” or -“_”. Capitalized names are reserved for configuration parameters (we'll call them -keywords); nothing forbids you from using capitalized strings for other things as -well but it probably would be confusing. I'll avoid using capitalized words for -anything but the keywords in this document, the reference manual, the examples -and the base library. Names that start with a “.” would be very confusing as well -because in the old grammar, AVPL transformations use names starting with a “.” to -indicate they belong to the replacement AVPL. - -The value is a string that is either set in the configuration (for configuration -AVPs) or by Wireshark while extracting interesting fields from a frame's tree. -The values extracted from fields use the same representation as they do in filter -strings except that no quotes are used. - -The name can contain only alphanumeric characters, "_", and ".". The name ends -with an operator. - -The value will be dealt with as a string even if it is a number. If there are -any spaces in the value, the value must be between quotes "". - ----- - ip_addr=10.10.10.11, - tcp_port=1234, - binary_data=01:23:45:67:89:ab:cd:ef, - parameter12=0x23aa, - parameter_with_spaces="this value has spaces" ----- - -The way two AVPs with the same name might match is described by the operator. -Remember two AVPs won't match unless their names are identical. In MATE, match -operations are always made between the AVPs extracted from frames (called data -AVPs) and the configuration's AVPs. - -Currently defined MATE's AVP match operators are: - -* <<Equal,Equal>> _=_ will match if the string given completely matches the data -AVP's value string -* <<NotEqual,Not Equal>> _!_ will match only if the given value string is not equal to -the data AVP's value string -* <<OneOf,One Of>> _{}_ will match if one of the possible strings listed is equal to -the data AVP's value string -* <<StartsWith,Starts With>> _^_ will match if the string given matches the first -characters of the data AVP's value string -* <<EndsWith,Ends With>> _$_ will match if the string given matches the last characters -of the data AVP's value string -* <<Contains,Contains>> _~_ will match if the string given matches any substring of the -data AVP's value string -* <<LowerThan,Lower Than>> _<_ will match if the data AVP's value string is semantically -lower than the string given -* <<HigherThan,Higher Than>> _>_ will match if the data AVP's value string is semantically -higher than the string given -* <<Exists,Exists>> _?_ (the ? can be omitted) will match as far as a data AVP of the -given name exists - -==== AVP lists - -An AVPL is a set of diverse AVPs that can be matched against other AVPLs. Every -PDU, Gop and Gog has an AVPL that contains the information regarding it. The -rules that MATE uses to group Pdus and Gops are AVPL operations. - -There will never be two identical AVPs in a given AVPL. However, we can have -more than one AVP with the same name in an AVPL as long as their values are -different. - -Some AVPL examples: ----- - ( addr=10.20.30.40, addr=192.168.0.1, tcp_port=21, tcp_port=32534, user_cmd=PORT, data_port=12344, data_addr=192.168.0.1 ) - ( addr=10.20.30.40, addr=192.168.0.1, channel_id=22:23, message_type=Setup, calling_number=1244556673 ) - ( addr=10.20.30.40, addr=192.168.0.1, ses_id=01:23:45:67:89:ab:cd:ef ) - ( user_id=pippo, calling_number=1244556673, assigned_ip=10.23.22.123 ) ----- - -In MATE there are two types of AVPLs: - -* data AVPLs that contain information extracted from frames. -* operation AVPLs that come from the configuration and are used to tell MATE how -to relate items based on their data AVPLs. - -Data AVPLs can be operated against operation AVPLs in various ways: - -* <<Loose,Loose Match>>: Will match if at least one of the AVPs of each AVPL -match. If it matches it will return an AVPL containing all AVPs from the operand -AVPL that did match the operator's AVPs. -* <<Every,"Every" Match>>: Will match if none of the AVPs of the operator AVPL -fails to match a present AVP in the operand AVPL, even if not all of the -operator's AVPs have a match. If it matches it will return an AVPL containing -all AVPs from the operand AVPL that did match one AVP in the operator AVPL. -* <<Strict,Strict Match>>: Will match if and only if every one of the operator's -AVPs have at least one match in the operand AVPL. If it matches it will return -an AVPL containing the AVPs from the operand that matched. -* There's also a <<Merge,Merge>> operation that is to be performed between AVPLs -where all the AVPs that don't exist in the operand AVPL but exist in the operand -will be added to the operand AVPL. -* Other than that, there are <<Transform,Transformations>> - a combination -of a match AVPL and an AVPL to merge. - -==== MATE Analysis - -MATE's analysis of a frame is performed in three phases: - -* In the first phase, MATE attempts to extract a MATE Pdu from the frame's -protocol tree. MATE will create a Pdu if MATE's config has a _Pdu_ declaration -whose _Proto_ is contained in the frame. - -* In the second phase, if a Pdu has been extracted from the frame, MATE will try -to group it to other Pdus into a Gop (Group of Pdus) by matching the key -criteria given by a _Gop_ declaration. If there is no Gop yet with the key -criteria for the Pdu, MATE will try to create a new Gop for it if it matches the -_Start_ criteria given in the Gop declaration. - -* In the third phase, if there's a Gop for the Pdu, MATE will try to group this -Gop with other Gops into a Gog (Group of Groups) using the criteria given by the -_Member_ criteria of a Gog declaration. - -image::images/ws-mate-analysis.png[] - -The extraction and matching logic comes from MATE's configuration; MATE's -configuration file is declared by the _mate.config_ preference. By default it is -an empty string which means: do not configure MATE. - -The config file tells MATE what to look for in frames; How to make PDUs out of -it; How will PDUs be related to other similar PDUs into Gops; And how Gops -relate into Gogs. - -The MATE configuration file is a list of declarations. There are 4 types of -declarations: _Transform_, _Pdu_, _Gop_ and _Gog_. - -===== Mate's PDU's - -MATE will look in the tree of every frame to see if there is useful data to -extract, and if there is, it will create one or more PDU objects containing the -useful information. - -The first part of MATE's analysis is the "PDU extraction"; there are various -"Actions" that are used to instruct MATE what has to be extracted from the -current frame's tree into MATE's PDUs. - -====== PDU data extraction - -MATE will make a Pdu for each different proto field of Proto type present in the -frame. MATE will fetch from the field's tree those fields that are defined in -the <<Pdu>> declaration whose initial offset in the frame is within the -boundaries of the current Proto and those of the given Transport and Payload -statements. - ----- -Pdu dns_pdu Proto dns Transport ip { - Extract addr From ip.addr; - Extract dns_id From dns.id; - Extract dns_resp From dns.flags.response; -}; ----- -MATE will make a Pdu for each different proto field of Proto type present in the -frame. MATE will fetch from the field's tree those fields that are defined in -the <<Pdu>> AVPL whose initial offset in the frame is within the boundaries of -the current Proto and those of the various assigned Transports. - -image::images/ws-mate-dns_pane.png[] - -Once MATE has found a _Proto_ field for which to create a Pdu from the frame it -will move backwards in the frame looking for the respective _Transport_ fields. -After that it will create AVPs named as each of those given in the rest of the -AVPL for every instance of the fields declared as its values. - -image::images/ws-mate-dns_pdu.png[] - -Sometimes we need information from more than one _Transport_ protocol. In that -case MATE will check the frame looking backwards to look for the various -_Transport_ protocols in the given stack. MATE will choose only the closest -transport boundary per "protocol" in the frame. - -This way we'll have all Pdus for every _Proto_ that appears in a frame match its -relative transports. - ----- -Pdu isup_pdu Proto isup Transport mtp3/ip { - Extract m3pc From mtp3.dpc; - Extract m3pc From mtp3.opc; - Extract cic From isup.cic; - Extract addr From ip.addr; - Extract isup_msg From isup.message_type; -}; ----- - -image::images/ws-mate-isup_over_mtp3_over_ip.png[] - -This allows to assign the right _Transport_ to the Pdu avoiding duplicate -transport protocol entries (in case of tunneled ip over ip for example). - ----- -Pdu ftp_pdu Proto ftp Transport tcp/ip { - Extract addr From ip.addr; - Extract port From tcp.port; - Extract ftp_cmd From ftp.command; -}; ----- - -image::images/ws-mate-ftp_over_gre.png[] - -Other than the mandatory _Transport_ there is also an optional _Payload_ -statement, which works pretty much as _Transport_ but refers to elements after -the _Proto_'s range. It is useful in those cases where the payload protocol -might not appear in a Pdu but nevertheless the Pdu belongs to the same category. - ----- -Pdu mmse_over_http_pdu Proto http Transport tcp/ip { - - Payload mmse; - - Extract addr From ip.addr; - Extract port From tcp.port; - Extract method From http.request.method; - Extract content From http.content_type; - Extract http_rq From http.request; - Extract resp From http.response.code; - Extract host From http.host; - Extract trx From mmse.transaction_id; - Extract msg_type From mmse.message_type; - Extract notify_status From mmse.status; - Extract send_status From mmse.response_status; -}; ----- - -image::images/ws-mate-mmse_over_http.png[] - -====== Conditions on which to create PDUs - -There might be cases in which we won't want MATE to create a PDU unless some of -its extracted attributes meet or do not meet some criteria. For that we use the -_Criteria_ statements of the _Pdu_ declarations. - ----- -Pdu isup_pdu Proto isup Transport mtp3/ip { - ... - - // MATE will create isup_pdu PDUs only when there is not a point code '1234' - Criteria Reject Strict (m3pc=1234); -}; - -Pdu ftp_pdu Proto ftp Transport tcp/ip { - ... - - // MATE will create ftp_pdu PDUs only when they go to port 21 of our ftp_server - Criteria Accept Strict (addr=10.10.10.10, port=21); -}; ----- - -The _Criteria_ statement is given an action (_Accept_ or _Reject_), a match mode -(_Strict_, _Loose_ or _Every_) and an AVPL against which to match the currently -extracted one. - -====== Transforming the attributes of a PDU - -Once the fields have been extracted into the Pdu's AVPL, MATE will apply any -declared transformation to it. The way transforms are applied and how they work -is described later on. However, it's useful to know that once the AVPL for the -Pdu is created, it may be transformed before being analyzed. That way we can -massage the data to simplify the analysis. - -====== MATE's PDU tree - -Every successfully created Pdu will add a MATE tree to the frame dissection. If -the Pdu is not related to any Gop, the tree for the Pdu will contain just the -Pdu's info, if it is assigned to a Gop, the tree will also contain the Gop items, -and the same applies for the Gog level. - ----- -mate dns_pdu:1 - dns_pdu: 1 - dns_pdu time: 3.750000 - dns_pdu Attributes - dns_resp: 0 - dns_id: 36012 - addr: 10.194.4.11 - addr: 10.194.24.35 ----- - -The Pdu's tree contains some filterable fields - -* _mate.dns_pdu_ will contain the number of the "dns_pdu" Pdu -* _mate.dns_pdu.RelativeTime_ will contain the time passed since the beginning -of the capture in seconds -* the tree will contain the various attributes of the Pdu as well, these will -all be strings (to be used in filters as "10.0.0.1", not as 10.0.0.1) -** mate.dns_pdu.dns_resp -** mate.dns_pdu.dns_id -** mate.dns_pdu.addr - -===== Grouping Pdus together (Gop) - -Once MATE has created the Pdus it passes to the Pdu analysis phase. During the -PDU analysis phase MATE will try to group Pdus of the same type into 'Groups of -Pdus' (aka *Gop*s) and copy some AVPs from the Pdu's AVPL to the Gop's AVPL. - -image::images/ws-mate-pdu_analysis.png[] - -====== What can belong to a Gop - -Given a Pdu, the first thing MATE will do is to check if there is any Gop -declaration in the configuration for the given Pdu type. If so, it will use its -_Match_ AVPL to match it against the Pdu's AVPL; if they don't match, the -analysis phase is done. If there is a match, the AVPL is the Gop's candidate key -which will be used to search the Gop's index for the Gop to which to assign -the current PDU. If there is no such Gop and this Pdu does not match the -_Start_ criteria of a Gop declaration for the Pdu type, the Pdu will remain -unassigned and only the analysis phase will be done. - ----- -Gop ftp_ses On ftp_pdu Match (addr, addr, port, port); -Gop dns_req On dns_pdu Match (addr, addr, dns_id); -Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic); ----- - -====== Start of a Gop - -If there was a match, the candidate key will be used to search the Gop's index -to see if there is already a Gop matching the Gop's key the same way. If there -is such a match in the Gops collection, and the PDU doesn't match the _Start_ -AVPL for its kind, the PDU will be assigned to the matching Gop. If it is a -_Start_ match, MATE will check whether or not that Gop has been already -stopped. If the Gop has been stopped, a new Gop will be created and will replace -the old one in the Gop's index. - ----- -Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) { - Start (ftp_cmd=USER); -}; - -Gop dns_req On dns_pdu Match (addr, addr, dns_id) { - Start (dns_resp=0); -}; - -Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) { - Start (isup_msg=1); -}; ----- - -If no _Start_ is given for a Gop, a Pdu whose AVPL matches an existing Gog's -key will act as the start of a Gop. - -====== What goes into the Gop's AVPL - -Once we know a Gop exists and the Pdu has been assigned to it, MATE will copy -into the Gop's AVPL all the attributes matching the key plus any AVPs of the -Pdu's AVPL matching the _Extra_ AVPL. - ----- -Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) { - Start (ftp_cmd=USER); - Extra (pasv_prt, pasv_addr); -}; - -Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) { - Start (isup_msg=1); - Extra (calling, called); -}; ----- - -====== End of a Gop - -Once the Pdu has been assigned to the Gop, MATE will check whether or not the -Pdu matches the _Stop_, if it happens, MATE will mark the Gop as stopped. Even -after stopped, a Gop may get assigned new Pdus matching its key, unless such -Pdu matches _Start_. If it does, MATE will instead create a new Gop starting -with that Pdu. - ----- -Gop ftp_ses On ftp_pdu Match (addr, addr, port, port) { - Start (ftp_cmd=USER); - Stop (ftp_cmd=QUIT); // The response to the QUIT command will be assigned to the same Gop - Extra (pasv_prt, pasv_addr); -}; - -Gop dns_req On dns_pdu Match (addr, addr, dns_id) { - Start (dns_resp=0); - Stop (dns_resp=1); -}; - -Gop isup_leg On isup_pdu Match (m3pc, m3pc, cic) { - Start (isup_msg=1); // IAM - Stop (isup_msg=16); // RLC - Extra (calling, called); -}; ----- - -If no _Stop_ criterium is stated for a given Gop, the Gop will be stopped as -soon as it is created. However, as with any other Gop, Pdus matching the Gop's -key will still be assigned to the Gop unless they match a _Start_ condition, -in which case a new Gop using the same key will be created. - -===== Gop's tree - -For every frame containing a Pdu that belongs to a Gop, MATE will create a tree -for that Gop. - -The example below represents the tree created by the _dns_pdu_ and _dns_req_ -examples. - ----- -... -mate dns_pdu:6->dns_req:1 - dns_pdu: 6 - dns_pdu time: 2.103063 - dns_pdu time since beginning of Gop: 2.103063 - dns_req: 1 - dns_req Attributes - dns_id: 36012 - addr: 10.194.4.11 - addr: 10.194.24.35 - dns_req Times - dns_req start time: 0.000000 - dns_req hold time: 2.103063 - dns_req duration: 2.103063 - dns_req number of PDUs: 2 - Start PDU: in frame 1 - Stop PDU: in frame 6 (2.103063 : 2.103063) - dns_pdu Attributes - dns_resp: 1 - dns_id: 36012 - addr: 10.194.4.11 - addr: 10.194.24.35 ----- - -Other than the pdu's tree, this one contains information regarding the -relationship between the Pdus that belong to the Gop. That way we have: - -* mate.dns_req which contains the id of this dns_req Gop. This will be present -in frames that belong to dns_req Gops. -* mate.dns_req.dns_id and mate.dns_req.addr which represent the values of the -attributes copied into the Gop. -* the timers of the Gop -** mate.dns_req.StartTime time (in seconds) passed since beginning of capture -until Gop's start. -** mate.dns_req.Time time passed between the start Pdu and the stop Pdu assigned -to this Gop (only created if a Stop criterion has been declared for the Gop and -a matching Pdu has arrived). -** mate.dns_req.Duration time passed between the start Pdu and the last Pdu -assigned to this Gop. -* mate.dns_req.NumOfPdus the number of Pdus that belong to this Gop -** a filterable list of frame numbers of the pdus of this Gop - -====== Gop's timers - -Note that there are two "timers" for a Gop: - -* *Time*, which is defined only for Gops that have been Stopped, and gives the -time passed between the _Start_ and the _Stop_ Pdus. -* *Duration*, which is defined for every Gop regardless of its state, and give -the time passed between its _Start_ Pdu and the last Pdu that was assigned to -that Gop. - -So: - -* we can filter for Pdus that belong to Gops that have been Stopped with -*mate.xxx.Time* -* we can filter for Pdus that belong to unstopped Gops with *mate.xxx && -mate.xxx.Time* -* we can filter for Pdus that belong to stopped Gops using *mate.xxx.Duration* -* we can filter for Pdus that belong to Gops that have taken more (or less) time -that 0.5s to complete with *mate.xxx.Time > 0.5* (you can try these also as -color filters to find out when response times start to grow) - -===== Grouping Gops together (Gog) - -When Gops are created, or whenever their AVPL changes, Gops are (re)analyzed to -check if they match an existent group of groups (Gog) or can create a new one. -The Gop analysis is divided into two phases. In the first phase, the still -unassigned Gop is checked to verify whether it belongs to an already existing -Gog or may create a new one. The second phase eventually checks the Gog and -registers its keys in the Gogs index. - -image::images/ws-mate-gop_analysis.png[] - -There are several reasons for the author to believe that this feature needs to -be reimplemented, so probably there will be deep changes in the way this is done -in the near future. This section of the documentation reflects the version of -MATE as of Wireshark 0.10.9; in future releases this will change. - -====== Declaring a Group Of Groups - -The first thing we have to do configuring a Gog is to tell MATE that it exists. - ----- -Gog web_use { - ... -}; ----- - -====== Telling MATE what could be a Gog member - -Then we have to tell MATE what to look for a match in the candidate Gops. - ----- -Gog web_use { - Member http_ses (host); - Member dns_req (host); -}; ----- - -====== Getting interesting data into the Gop - -Most often, also other attributes than those used for matching would be -interesting. In order to copy from Gop to Gog other interesting attributes, we -might use _Extra_ like we do for Gops. - ----- -Gog web_use { - ... - Extra (cookie); -}; ----- - -====== Gog's tree - ----- -mate http_pdu:4->http_req:2->http_use:1 - http_pdu: 4 - http_pdu time: 1.309847 - http_pdu time since beginning of Gop: 0.218930 - http_req: 2 - ... (the gop's tree for http_req: 2) .. - http_use: 1 - http_use Attributes - host: www.example.com - http_use Times - http_use start time: 0.000000 - http_use duration: 1.309847 - number of GOPs: 3 - dns_req: 1 - ... (the gop's tree for dns_req: 1) .. - http_req: 1 - ... (the gop's tree for http_req: 1) .. - http_req of current frame: 2 ----- - -We can filter on: - -* *mate.http_use.Duration* time elapsed between the first frame of a Gog and the last one assigned to it. -* the attributes passed to the Gog -** *mate.http_use.host* - -===== AVPL Transforms - -A Transform is a sequence of Match rules optionally completed with modification -of the match result by an additional AVPL. Such modification may be an Insert -(merge) or a Replace. Transforms can be used as helpers to manipulate an item's -AVPL before it is processed further. They come to be very helpful in several -cases. - -====== Syntax - -AVPL Transformations are declared in the following way: - ----- - Transform name { - Match [Strict|Every|Loose] match_avpl [Insert|Replace] modify_avpl ; - ... - }; ----- - -The *name* is the handle to the AVPL transformation. It is used to refer to the -transform when invoking it later. - -The _Match_ declarations instruct MATE what and how to match against the data -AVPL and how to modify the data AVPL if the match succeeds. They will be -executed in the order they appear in the config file whenever they are invoked. - -The optional match mode qualifier (_Strict_, _Every_, or _Loose_) is used -to choose the match mode as explained above; _Strict_ is a default value which -may be omitted. - -The optional modification mode qualifier instructs MATE how the modify AVPL -should be used: - -* the default value _Insert_ (which may be omitted) causes the _modify_avpl_ -to be *merged* to the existing data AVPL, -* the _Replace_ causes all the matching AVPs from the data AVPL to be -*replaced* by the _modify_avpl_. - -The _modify_avpl_ may be an empty one; this comes useful in some cases for -both _Insert_ and _Replace_ modification modes. - -Examples: - ----- - Transform insert_name_and { - Match Strict (host=10.10.10.10, port=2345) Insert (name=JohnDoe); -}; ----- - -adds name=JohnDoe to the data AVPL if it contains host=10.10.10.10 *and* -port=2345 - ----- -Transform insert_name_or { - Match Loose (host=10.10.10.10, port=2345) Insert (name=JohnDoe); -}; ----- - -adds name=JohnDoe to the data AVPL if it contains host=10.10.10.10 *or* -port=2345 - ----- -Transform replace_ip_address { - Match (host=10.10.10.10) Replace (host=192.168.10.10); -}; ----- - -replaces the original host=10.10.10.10 by host=192.168.10.10 - ----- -Transform add_ip_address { - Match (host=10.10.10.10) (host=192.168.10.10); -}; ----- - -adds (inserts) host=192.168.10.10 to the AVPL, keeping the original -host=10.10.10.10 in it too - ----- - Transform replace_may_be_surprising { - Match Loose (a=aaaa, b=bbbb) Replace (c=cccc, d=dddd); - }; ----- - -gives the following results: - -* (a=aaaa, b=eeee) gets transformed to (b=eeee, c=cccc, d=dddd) because a=aaaa -did match so it got replaced while b=eeee did not match so it has been left -intact, -* (a=aaaa, b=bbbb) gets transformed to (c=cccc, d=dddd) because both a=aaaa and -b=bbbb did match. - -====== Usage - -Once declared, Transforms can be added to the declarations of PDUs, Gops or -Gogs. This is done by adding the _Transform name_list_ statement to the -declaration: - ----- -Pdu my_proto_pdu Proto my_proto Transport ip { - Extract addr From ip.addr; - ... - Transform my_pdu_transform[, other_pdu_transform[, yet_another_pdu_transform]]; -}; ----- - -* In case of PDU, the list of transforms is applied against the PDU's AVPL -after its creation. -* In case of Gop and Gog, the list of transforms is applied against their -respective AVPLs when they are created and every time they change. - -===== Operation - -image::images/ws-mate-transform.png[] - -* A list of previously declared Transforms may be given to every Item (Pdu, Gop, -or Gog), using the Transform statement. -* Every time the AVPL of an item changes, it will be operated against *all* the -Transforms on the list given to that item. The Transforms on the list are -applied left to right. -* Inside each of the Transforms, the item's AVPL will be operated against the -Transform's Match clauses starting from the topmost one, until all have been -tried or until one of them succeeds. - -MATE's Transforms can be used for many different things, like: - -====== Multiple Start/Stop conditions for a Gop - -Using _Transforms_ we can add more than one start or stop condition to a Gop. - ----- -Transform start_cond { - Match (attr1=aaa,attr2=bbb) (msg_type=start); - Match (attr3=www,attr2=bbb) (msg_type=start); - Match (attr5^a) (msg_type=stop); - Match (attr6$z) (msg_type=stop); -}; - -Pdu pdu ... { - ... - Transform start_cond; -} - -Gop gop ... { - Start (msg_type=start); - Stop (msg_type=stop); - ... -} ----- - -====== Marking Gops and Gogs to filter them easily - ----- -Transform marks { - Match (addr=10.10.10.10, user=john) (john_at_host); - Match (addr=10.10.10.10, user=tom) (tom_at_host); -} - -... - -Gop my_gop ... { - ... - Transform marks; -} ----- - -After that we can use a display filter *mate.gop.john_at_host* or -*mate.gop.tom_at_host* - -====== Adding direction knowledge to MATE - ----- -Transform direction_as_text { - Match (src=192.168.0.2, dst=192.168.0.3) Replace (direction=from_2_to_3); - Match (src=192.168.0.3, dst=192.168.0.2) Replace (direction=from_3_to_2); -}; - -Pdu my_pdu Proto my_proto Transport tcp/ip { - Extract src From ip.src; - Extract dst From ip.dst; - Extract addr From ip.addr; - Extract port From tcp.port; - Extract start From tcp.flags.syn; - Extract stop From tcp.flags.fin; - Extract stop From tcp.flags.rst; - Transform direction_as_text; -} - -Gop my_gop On my_pdu Match (addr,addr,port,port) { - ... - Extra (direction); -} ----- - -====== NAT - -NAT can create problems when tracing, but we can easily work around it by -Transforming the NATed IP address and the Ethernet address of the router into -the non-NAT address: - ----- -Transform denat { - Match (addr=192.168.0.5, ether=01:02:03:04:05:06) Replace (addr=123.45.67.89); - Match (addr=192.168.0.6, ether=01:02:03:04:05:06) Replace (addr=123.45.67.90); - Match (addr=192.168.0.7, ether=01:02:03:04:05:06) Replace (addr=123.45.67.91); -} - -Pdu my_pdu Proto my_proto transport tcp/ip/eth { - Extract ether From eth.addr; - Extract addr From ip.addr; - Extract port From tcp.port; - Transform denat; -} ----- - -==== About MATE - -MATE was originally written by Luis Ontanon, a Telecommunications systems -troubleshooter, as a way to save time filtering out the packets of a single call -from huge capture files using just the calling number. Later he used the time he -had saved to make it flexible enough to work with protocols other than the ones -he was directly involved with. - -[#ChMateConfigurationTutorial] - -=== MATE's configuration tutorial - -We'll show a MATE configuration that first creates Gops for every DNS and HTTP -request, then it ties the Gops together in a Gop based on the host. Finally, -we'll separate into different Gogs request coming from different users. - -With this MATE configuration loaded we can: - -* use *mate.http_use.Duration > 5.5* to filter frames based on the time it takes -to load a complete page from the DNS request to resolve its name until the last -image gets loaded. -* use *mate.http_use.client == "10.10.10.20" && mate.http_use.host == "www.example.com"* -to isolate DNS and HTTP packets related to a visit of a certain user. -* use *mate.http_req.Duration > 1.5* to filter all the packets of HTTP requests -that take more than 1.5 seconds to complete. - -The complete config file is available on the Wireshark Wiki: -https://gitlab.com/wireshark/wireshark/-/wikis/Mate/Tutorial - -Note: This example uses _dns.qry.name_ which is defined since Wireshark -version 0.10.9. Supposing you have a mate plugin already installed you can test -it with the current Wireshark version. - -==== A Gop for DNS requests - -First we'll tell MATE how to create a Gop for each DNS request/response. - -MATE needs to know what makes a DNS PDU. We describe it this using a Pdu -declaration: - ----- -Pdu dns_pdu Proto dns Transport ip { - Extract addr From ip.addr; - Extract dns_id From dns.id; - Extract dns_resp From dns.flags.response; -}; ----- - -Using _Proto dns_ we tell MATE to create Pdus every time it finds _dns_. Using -_Transport ip_ we inform MATE that some of the fields we are interested are -in the _ip_ part of the frame. Finally, we tell MATE to import _ip.addr_ as -_addr_, _dns.id_ as _dns_id_ and _dns.flags.response_ as _dns_resp_. - -Once we've told MATE how to extract _dns_pdus_ we'll tell it how to match -requests and responses and group them into a Gop. For this we'll use a _Gop_ -declaration to define the Gop, and then, _Start_ and _Stop_ statements to -tell it when the Gop starts and ends. - ----- -Gop dns_req On dns_pdu Match (addr,addr,dns_id) { - Start (dns_resp=0); - Stop (dns_resp=1); -}; ----- - -Using the *Gop* declaration we tell MATE that the *Name* of the Gop is _dns_req_, -that _dns_pdus_ can become members of the Gop, and what is the key used to match -the Pdus to the Gop. - -The key for this Gop is _"addr, addr, dns_id"_. That means that in order to -belong to the same Gop, _dns_pdus_ have to have both addresses and the -_request id_ identical. We then instruct MATE that a _dns_req_ starts whenever -a _dns_pdu_ matches _"dns_resp=0"_ and that it stops when another _dns_pdu_ -matches _"dns_resp=1"_. - -At this point, if we open a capture file using this configuration, we are able -to use a display filter *mate.dns_req.Time > 1* to see only the packets of -DNS requests that take more than one second to complete. - -We can use a display filter *mate.dns_req && ! mate.dns_req.Time* to find -requests for which no response was given. *mate.xxx.Time* is set only for Gops -that have being stopped. - -==== A Gop for HTTP requests - -This other example creates a Gop for every HTTP request. - ----- -Pdu http_pdu Proto http Transport tcp/ip { - Extract addr From ip.addr; - Extract port From tcp.port; - Extract http_rq From http.request.method; - Extract http_rs From http.response; - DiscardPduData true; -}; - -Gop http_req On http_pdu Match (addr, addr, port, port) { - Start (http_rq); - Stop (http_rs); -}; ----- - -So, if we open a capture using this configuration - -* filtering with *mate.http_req.Time > 1* will give all the requests where the -response header takes more than one second to come -* filtering with *mate.http_req.Duration > 1.5* will show those request that -take more than 1.5 seconds to complete. - -You have to know that *mate.xxx.Time* gives the time in seconds between the pdu -matching the GopStart and the Pdu matching the GopStop (yes, you can create -timers using this!). On the other hand, *mate.xxx.Duration* gives you the time -passed between the GopStart and the last pdu assigned to that Gop regardless -whether it is a stop or not. After the GopStop, Pdus matching the Gop's Key will -still be assigned to the same Gop as far as they don't match the GopStart, in -which case a new Gop with the same key will be created. - -==== Getting DNS and HTTP together into a Gog - -We'll tie together to a single Gog all the http packets belonging to requests -and responses to a certain host and the dns request and response used to resolve -its domain name using the Pdu and Gop definitions of the previous examples - -To be able to group DNS and HTTP requests together, we need to import into the -Pdus and Gops some part of information that both those protocols share. Once the -Pdus and Gops have been defined, we can use _Extract_ (for Pdus) and -_Extract_ (for Gops) statements to tell MATE what other protocol fields are to -be added to Pdus' and Gops' AVPLs. We add the following statements to the -appropriate declarations: - ----- -Extract host From http.host; // to Pdu http_pdu as the last Extract in the list -Extra (host); // to Gop http_req after the Stop - -Extract host From dns.qry.name; // to Pdu dns_pdu as the last Extract in the list -Extra (host); // to Gop dns_req after the Stop ----- - -Here we've told MATE to import _http.host_ into _http_pdu_ and _dns.qry.name_ -into _dns_pdu_ as _host_. We also have to tell MATE to copy the _host_ -attribute from the Pdus to the Gops, we do this using _Extra_. - -Once we have all the data we need in Pdus and Gops, we tell MATE what makes -different Gops belong to a certain Gog. - ----- -Gog http_use { - Member http_req (host); - Member dns_req (host); - Expiration 0.75; -}; ----- - -Using the _Gog_ declaration, we tell MATE to define a Gog type _Named_ -_http_use_ whose expiration is 0.75 seconds after all the Gops that belong to it -had been stopped. After that time, an eventual new Gop with the same key match -will create a new Gog instead of been added to the previous Gog. - -Using the _Member_ statements, we tell MATE that *http_req*s with the same -*host* belong to the same Gog, same thing for *dns_req*s. - -So far we have instructed mate to group every packet related to sessions towards -a certain host. At this point if we open a capture file and: - -* a display filter *mate.http_use.Duration > 5* will show only those requests -that have taken more than 5 seconds to complete starting from the DNS request -and ending with the last packet of the http responses. - -* a display filter *mate.http_use.host == "www.w3c.org"* will show all the -packets (both DNS and HTTP) related to the requests directed to www.w3c.org - -==== Separating requests from multiple users - -"Houston: we've had a problem here." - -This configuration works fine if used for captures taken at the client's side -but deeper in the network we'd got a real mess. Requests from many users get -mixed together into _http_uses_. Gogs are created and stopped almost randomly -(depending on the timing in which Gops start and stop). How do we get requests -from individual users separated from each other? - -MATE has a tool that can be used to resolve this kind of grouping issues. This -tool are the _Transforms_. Once defined, they can be applied against Pdus, -Gops and Gogs and they might replace or insert more attributes based on what's -there. We'll use them to create an attribute named client, using which we'll -separate different requests. - -For DNS we need the ip.src of the request moved into the Gop only from the DNS -request. - -So we first tell MATE to import ip.src as client: - ----- -Extract client From ip.src; ----- - -Next, we tell MATE to replace ( *dns_resp=1, client* ) with just *dns_resp=1* in -the Pdu. That way, we'll keep the attribute *client* only in the DNS request -Pdus (i.e., packets coming from the client).To do so, we have to add a -_Transform_ declaration (in this case, with just one clause) before the Pdu -declaration which uses it: - ----- -Transform rm_client_from_dns_resp { - Match (dns_resp=1, client) Replace (dns_resp=1); -}; ----- - -Next, we invoke the transform by adding the following line after the _Extract_ -list of the dns_pdu Pdu: - ----- - Transform rm_client_from_dns_resp; ----- - -HTTP is a little trickier. We have to remove the attribute carrying ip.src from -both the response and the "continuations" of the response, but as there is -nothing to filter on for the continuations, we have to add a fake attribute -first. And then we have to remove client when the fake attribute appears. -This is possible due to the fact that the _Match_ clauses in the _Transform_ -are executed one by one until one of them succeeds. First, we declare another -two _Transforms_: - ----- -Transform rm_client_from_http_resp1 { - Match (http_rq); //first match wins so the request won't get the not_rq attribute inserted - Match Every (addr) Insert (not_rq); //this line won't be evaluated if the first one matched so not_rq won't be inserted to requests -}; - -Transform rm_client_from_http_resp2 { - Match (not_rq, client) Replace (); //replace "client and not_rq" with nothing (will happen only in the response and eventual parts of it) -}; ----- - -Next, we add another _Extract_ statement to the _http_pdu_ declaration, and -apply both _Transforms_ declared above in a proper order: - ----- - Extract client From ip.src; - Transform rm_client_from_http_resp1, rm_client_from_http_resp2; ----- - -In MATE, all the _Transform_s listed for an item will be evaluated, while -inside a single _Transform_, the evaluation will stop at the first successful -_Match_ clause. That's why we first just match _http_rq_ to get out of the -first sequence before adding the _not_rq_ attribute. Then we apply the second -_Transform_ which removes both _not_rq_ and _client_ if both are there. Yes, -_Transform_s are cumbersome, but they are very useful. - -Once we got all what we need in the Pdus, we have to tell MATE to copy the -attribute _client_ from the Pdus to the respective Gops, by adding client to -_Extra_ lists of both Gop declarations: - ----- -Extra (host, client); ----- - -On top of that, we need to modify the old declarations of Gop key to new ones -that include both _client_ and _host_. So we change the Gog *Member* -declarations the following way: - ----- - Member http_req (host, client); - Member dns_req (host, client); ----- - -Now we got it, every "usage" gets its own Gog. - -[#ChMateConfigurationExamples] - -=== MATE configuration examples - -The following is a collection of various configuration examples for MATE. Many -of them are useless because the "conversations" facility does a better job. -Anyway they are meant to help users understanding how to configure MATE. - -==== TCP session - -The following example creates a GoP out of every TCP session. - ----- -Pdu tcp_pdu Proto tcp Transport ip { - Extract addr From ip.addr; - Extract port From tcp.port; - Extract tcp_start From tcp.flags.syn; - Extract tcp_stop From tcp.flags.reset; - Extract tcp_stop From tcp.flags.fin; -}; - -Gop tcp_ses On tcp_pdu Match (addr, addr, port, port) { - Start (tcp_start=1); - Stop (tcp_stop=1); -}; - -Done; ----- - -This probably would do fine in 99.9% of the cases but 10.0.0.1:20->10.0.0.2:22 and 10.0.0.1:22->10.0.0.2:20 would both fall into the same gop if they happen to overlap in time. - -* filtering with *mate.tcp_ses.Time > 1* will give all the sessions that last less than one second -* filtering with *mate.tcp_ses.NumOfPdus < 5* will show all tcp sessions that have less than 5 packets. -* filtering with *mate.tcp_ses.Id == 3* will show all the packets for the third tcp session MATE has found - -==== a Gog for a complete FTP session - -This configuration allows to tie a complete passive ftp session (including the -data transfer) in a single Gog. - ----- -Pdu ftp_pdu Proto ftp Transport tcp/ip { - Extract ftp_addr From ip.addr; - Extract ftp_port From tcp.port; - Extract ftp_resp From ftp.response.code; - Extract ftp_req From ftp.request.command; - Extract server_addr From ftp.passive.ip; - Extract server_port From ftp.passive.port; - - LastPdu; -}; - -Pdu ftp_data_pdu Proto ftp-data Transport tcp/ip{ - Extract server_addr From ip.src; - Extract server_port From tcp.srcport; - -}; - -Gop ftp_data On ftp_data_pdu (server_addr, server_port) { - Start (server_addr); -}; - -Gop ftp_ctl On ftp_pdu (ftp_addr, ftp_addr, ftp_port, ftp_port) { - Start (ftp_resp=220); - Stop (ftp_resp=221); - Extra (server_addr, server_port); -}; - -Gog ftp_ses { - Member ftp_ctl (ftp_addr, ftp_addr, ftp_port, ftp_port); - Member ftp_data (server_addr, server_port); -}; - -Done; ----- - -Note: not having anything to distinguish between ftp-data packets makes this -config to create one Gop for every ftp-data packet instead of each transfer. -Pre-started Gops would avoid this. - -==== using RADIUS to filter SMTP traffic of a specific user - -Spying on people, in addition to being immoral, is illegal in many countries. -This is an example meant to explain how to do it not an invitation to do so. -It's up to the police to do this kind of job when there is a good reason to do -so. - ----- -Pdu radius_pdu On radius Transport udp/ip { - Extract addr From ip.addr; - Extract port From udp.port; - Extract radius_id From radius.id; - Extract radius_code From radius.code; - Extract user_ip From radius.framed_addr; - Extract username From radius.username; -} - -Gop radius_req On radius_pdu (radius_id, addr, addr, port, port) { - Start (radius_code {1|4|7} ); - Stop (radius_code {2|3|5|8|9} ); - Extra (user_ip, username); -} - -// we define the smtp traffic we want to filter -Pdu user_smtp Proto smtp Transport tcp/ip { - Extract user_ip From ip.addr; - Extract smtp_port From tcp.port; - Extract tcp_start From tcp.flags.syn; - Extract tcp_stop From tcp.flags.reset; -} - -Gop user_smtp_ses On user_smtp (user_ip, user_ip, smtp_port!25) { - Start (tcp_start=1); - Stop (tcp_stop=1); -} - -// with the following group of groups we'll group together the radius and the smtp -// we set a long expiration to avoid the session expire on long pauses. -Gog user_mail { - Expiration 1800; - Member radius_req (user_ip); - Member user_smtp_ses (user_ip); - Extra (username); -} - -Done; ----- - -Filtering the capture file with *mate.user_mail.username == "theuser"* will -filter the radius packets and smtp traffic for _"theuser"_. - -==== H323 Calls - -This configuration will create a Gog out of every call. - ----- -Pdu q931 Proto q931 Transport ip { - Extract addr From ip.addr; - Extract call_ref From q931.call_ref; - Extract q931_msg From q931.message_type; - Extract calling From q931.calling_party_number.digits; - Extract called From q931.called_party_number.digits; - Extract guid From h225.guid; - Extract q931_cause From q931.cause_value; -}; - -Gop q931_leg On q931 Match (addr, addr, call_ref) { - Start (q931_msg=5); - Stop (q931_msg=90); - Extra (calling, called, guid, q931_cause); -}; - -Pdu ras Proto h225.RasMessage Transport ip { - Extract addr From ip.addr; - Extract ras_sn From h225.requestSeqNum; - Extract ras_msg From h225.RasMessage; - Extract guid From h225.guid; -}; - -Gop ras_req On ras Match (addr, addr, ras_sn) { - Start (ras_msg {0|3|6|9|12|15|18|21|26|30} ); - Stop (ras_msg {1|2|4|5|7|8|10|11|13|14|16|17|19|20|22|24|27|28|29|31}); - Extra (guid); -}; - -Gog call { - Member ras_req (guid); - Member q931_leg (guid); - Extra (called,calling,q931_cause); -}; - -Done; ----- - -with this we can: - -* filter all signalling for a specific caller: *mate.call.caller == "123456789"* -* filter all signalling for calls with a specific release cause: *mate.call.q931_cause == 31* -* filter all signalling for very short calls: *mate.q931_leg.Time < 5* - -==== MMS - -With this example, all the components of an MMS send or receive will be tied -into a single Gog. Note that this example uses the _Payload_ clause because -MMS delivery uses MMSE over either HTTP or WSP. As it is not possible to relate -the retrieve request to a response by the means of MMSE only (the request is -just an HTTP GET without any MMSE), a Gop is made of HTTP Pdus but MMSE data -need to be extracted from the bodies. - ----- -## WARNING: this example has been blindly translated from the "old" MATE syntax -## and it has been verified that Wireshark accepts it. However, it has not been -## tested against any capture file due to lack of the latter. - -Transform rm_client_from_http_resp1 { - Match (http_rq); - Match Every (addr) Insert (not_rq); -}; - -Transform rm_client_from_http_resp2 { - Match (not_rq,ue) Replace (); -}; - -Pdu mmse_over_http_pdu Proto http Transport tcp/ip { - Payload mmse; - Extract addr From ip.addr; - Extract port From tcp.port; - Extract http_rq From http.request; - Extract content From http.content_type; - Extract resp From http.response.code; - Extract method From http.request.method; - Extract host From http.host; - Extract content From http.content_type; - Extract trx From mmse.transaction_id; - Extract msg_type From mmse.message_type; - Extract notify_status From mmse.status; - Extract send_status From mmse.response_status; - Transform rm_client_from_http_resp1, rm_client_from_http_resp2; -}; - -Gop mmse_over_http On mmse_over_http_pdu Match (addr, addr, port, port) { - Start (http_rq); - Stop (http_rs); - Extra (host, ue, resp, notify_status, send_status, trx); -}; - -Transform mms_start { - Match Loose() Insert (mms_start); -}; - -Pdu mmse_over_wsp_pdu Proto wsp Transport ip { - Payload mmse; - Extract trx From mmse.transaction_id; - Extract msg_type From mmse.message_type; - Extract notify_status From mmse.status; - Extract send_status From mmse.response_status; - Transform mms_start; -}; - -Gop mmse_over_wsp On mmse_over_wsp_pdu Match (trx) { - Start (mms_start); - Stop (never); - Extra (ue, notify_status, send_status); -}; - -Gog mms { - Member mmse_over_http (trx); - Member mmse_over_wsp (trx); - Extra (ue, notify_status, send_status, resp, host, trx); - Expiration 60.0; -}; ----- - -[#ChMateConfigurationLibrary] - -=== MATE's configuration library - -The MATE library (will) contains GoP definitions for several protocols. Library -protocols are included in your MATE config using: +_Action=Include; -Lib=proto_name;_+. - -For Every protocol with a library entry, we'll find defined what from the PDU is -needed to create a GoP for that protocol, eventually any criteria and the very -essential GoP definition (i.e., __GopDef__, _GopStart_ and _GopStop_). - -[NOTE] -==== -It seems that this code is written in the old syntax of MATE. So far it has not -been transcribed into the new format. It may still form the basis to recreate -these in the new format. -==== - -==== General use protocols - -===== TCP - -It will create a GoP for every TCP session, If it is used it should be the last -one in the list. And every other proto on top of TCP should be declared with -_Stop=TRUE;_ so the a TCP PDU is not created where we got already one going on. - ----- - Action=PduDef; Name=tcp_pdu; Proto=tcp; Transport=ip; addr=ip.addr; port=tcp.port; tcp_start=tcp.flags.syn; tcp_stop=tcp.flags.fin; tcp_stop=tcp.flags.reset; - Action=GopDef; Name=tcp_session; On=tcp_pdu; addr; addr; port; port; - Action=GopStart; For=tcp_session; tcp_start=1; - Action=GopStop; For=tcp_session; tcp_stop=1; ----- - -===== DNS - -will create a GoP containing every request and its response (eventually -retransmissions too). - ----- -Action=PduDef; Name=dns_pdu; Proto=dns; Transport=udp/ip; addr=ip.addr; port=udp.port; dns_id=dns.id; dns_rsp=dns.flags.response; - -Action=GopDef; Name=dns_req; On=dns_pdu; addr; addr; port!53; dns_id; -Action=GopStart; For=dns_req; dns_rsp=0; -Action=GopStop; For=dns_req; dns_rsp=1; ----- - -===== RADIUS - -A Gop for every transaction. - ----- -Action=PduDef; Name=radius_pdu; Proto=radius; Transport=udp/ip; addr=ip.addr; port=udp.port; radius_id=radius.id; radius_code=radius.code; - -Action=GopDef; Name=radius_req; On=radius_pdu; radius_id; addr; addr; port; port; -Action=GopStart; For=radius_req; radius_code|1|4|7; -Action=GopStop; For=radius_req; radius_code|2|3|5|8|9; ----- - -===== RTSP - ----- -Action=PduDef; Name=rtsp_pdu; Proto=rtsp; Transport=tcp/ip; addr=ip.addr; port=tcp.port; rtsp_method=rtsp.method; -Action=PduExtra; For=rtsp_pdu; rtsp_ses=rtsp.session; rtsp_url=rtsp.url; - -Action=GopDef; Name=rtsp_ses; On=rtsp_pdu; addr; addr; port; port; -Action=GopStart; For=rtsp_ses; rtsp_method=DESCRIBE; -Action=GopStop; For=rtsp_ses; rtsp_method=TEARDOWN; -Action=GopExtra; For=rtsp_ses; rtsp_ses; rtsp_url; ----- - -==== VoIP/Telephony - -Most protocol definitions here will create one Gop for every Call Leg unless -stated. - -===== ISUP - ----- -Action=PduDef; Name=isup_pdu; Proto=isup; Transport=mtp3; mtp3pc=mtp3.dpc; mtp3pc=mtp3.opc; cic=isup.cic; isup_msg=isup.message_type; - -Action=GopDef; Name=isup_leg; On=isup_pdu; ShowPduTree=TRUE; mtp3pc; mtp3pc; cic; -Action=GopStart; For=isup_leg; isup_msg=1; -Action=GopStop; For=isup_leg; isup_msg=16; ----- - -===== Q931 - ----- -Action=PduDef; Name=q931_pdu; Proto=q931; Stop=TRUE; Transport=tcp/ip; addr=ip.addr; call_ref=q931.call_ref; q931_msg=q931.message_type; - -Action=GopDef; Name=q931_leg; On=q931_pdu; addr; addr; call_ref; -Action=GopStart; For=q931_leg; q931_msg=5; -Action=GopStop; For=q931_leg; q931_msg=90; ----- - -===== H225 RAS - ----- -Action=PduDef; Name=ras_pdu; Proto=h225.RasMessage; Transport=udp/ip; addr=ip.addr; ras_sn=h225.RequestSeqNum; ras_msg=h225.RasMessage; -Action=PduExtra; For=ras_pdu; guid=h225.guid; - -Action=GopDef; Name=ras_leg; On=ras_pdu; addr; addr; ras_sn; -Action=GopStart; For=ras_leg; ras_msg|0|3|6|9|12|15|18|21|26|30; -Action=GopStop; For=ras_leg; ras_msg|1|2|4|5|7|8|10|11|13|14|16|17|19|20|22|24|27|28|29|31; -Action=GopExtra; For=ras_leg; guid; ----- - -===== SIP - ----- -Action=PduDef; Proto=sip_pdu; Transport=tcp/ip; addr=ip.addr; port=tcp.port; sip_method=sip.Method; sip_callid=sip.Call-ID; calling=sdp.owner.username; - -Action=GopDef; Name=sip_leg; On=sip_pdu; addr; addr; port; port; -Action=GopStart; For=sip; sip_method=INVITE; -Action=GopStop; For=sip; sip_method=BYE; ----- - -===== MEGACO - -Will create a Gop out of every transaction. - -To "tie" them to your call's GoG use: _Action=GogKey; Name=your_call; On=mgc_tr; -addr!mgc_addr; megaco_ctx;_ - ----- -Action=PduDef; Name=mgc_pdu; Proto=megaco; Transport=ip; addr=ip.addr; megaco_ctx=megaco.context; megaco_trx=megaco.transid; megaco_msg=megaco.transaction; term=megaco.termid; - -Action=GopDef; Name=mgc_tr; On=mgc_pdu; addr; addr; megaco_trx; -Action=GopStart; For=mgc_tr; megaco_msg|Request|Notify; -Action=GopStop; For=mgc_tr; megaco_msg=Reply; -Action=GopExtra; For=mgc_tr; term^DS1; megaco_ctx!Choose one; ----- - -[#ChMateReferenceManual] - -=== MATE's reference manual - -==== Attribute Value Pairs - -MATE uses AVPs for almost everything: to keep the data it has extracted from the -frames' trees as well as to keep the elements of the configuration. - -These "pairs" (actually tuples) are made of a name, a value and, in case of -configuration AVPs, an operator. Names and values are strings. AVPs with -operators other than '=' are used only in the configuration and are used for -matching AVPs of Pdus, GoPs and GoGs in the analysis phase. - -===== Name - -The name is a string used to refer to a class of AVPs. Two attributes won't -match unless their names are identical. Capitalized names are reserved for -keywords (you can use them for your elements if you want but I think it's not -the case). MATE attribute names can be used in Wireshark's display filters the -same way like names of protocol fields provided by dissectors, but they are not -just references to (or aliases of) protocol fields. - -===== Value - -The value is a string. It is either set in the configuration (for configuration -AVPs) or by MATE while extracting interesting fields from a dissection tree -and/or manipulating them later. The values extracted from fields use the same -representation as they do in filter strings. - -===== Operators - -Currently only match operators are defined (there are plans to (re)add transform -attributes but some internal issues have to be solved before that). The match -operations are always performed between two operands: the value of an AVP stated -in the configuration and the value of an AVP (or several AVPs with the same name) -extracted from packet data (called "data AVPs"). It is not possible to match -data AVPs to each other. - -The defined match operators are: - -* <<Equal,Equal>> _=_ test for equality, that is: either the value strings are identical -or the match will fail. -* <<NotEqual,Not Equal>> _!_ will match only if the value strings aren't equal. -* <<OneOf,One Of>> _{}_ will match if one of the value strings listed is equal to the -data AVP's string. Items inside the list's curly braces are -separated with the | character. -* <<StartsWith,Starts With>> _^_ will match if the configuration value string matches the -first characters of the data AVP's value string. -* <<EndsWith,Ends With>> _$_ will match if the configuration value string matches the -last characters of the data AVP's value string. -* <<Contains,Contains>> _~_ will match if the configuration value string matches a -substring of the characters of the data AVP's value string. -* <<LowerThan,Lower Than>> _<_ will match if the data AVP's value string is semantically -lower than the configuration value string. -* <<HigherThan,Higher Than>> _>_ will match if the data AVP's value string is semantically -higher than the configuration value string. -* <<Exists,Exists>> _?_ (can be omitted) will match if the AVP name matches, regardless -what the value string is. - -[#Equal] -====== Equal AVP Operator - -This operator tests whether the values of the operator and the operand AVP are -equal. - -Example:: -attrib=aaa *matches* attrib=aaa + -attrib=aaa *does not match* attrib=bbb - -[#NotEqual] -====== Not equal AVP operator - -This operator matches if the value strings of two AVPs are not equal. - -Example:: -attrib=aaa matches attrib!bbb + -attrib=aaa does not match attrib!aaa - -[#OneOf] -====== "One of" AVP operator - -The "one of" operator matches if the data AVP value is equal to one of the -values listed in the "one of" AVP. - -Example:: -attrib=1 matches attrib{1|2|3} + -attrib=2 matches attrib{1|2|3} + -attrib=4 does not match attrib{1|2|3} - -[#StartsWith] -====== "Starts with" AVP operator - -The "starts with" operator matches if the first characters of the data AVP -value are identical to the configuration AVP value. - -Example:: -attrib=abcd matches attrib^abc + -attrib=abc matches attrib^abc + -attrib=ab does not match attrib^abc + -attrib=abcd does not match attrib^bcd + -attrib=abc does not match attrib^abcd + - -[#EndsWith] -====== "Ends with" operator - -The ends with operator will match if the last bytes of the data AVP value are -equal to the configuration AVP value. - -Example:: -attrib=wxyz matches attrib$xyz + -attrib=yz does not match attrib$xyz + -attrib=abc...wxyz does not match attrib$abc - -[#Contains] -====== Contains operator - -The "contains" operator will match if the data AVP value contains a string -identical to the configuration AVP value. - -Example:: -attrib=abcde matches attrib~bcd + -attrib=abcde matches attrib~abc + -attrib=abcde matches attrib~cde + -attrib=abcde does not match attrib~xyz - -[#LowerThan] -====== "Lower than" operator - -The "lower than" operator will match if the data AVP value is semantically lower -than the configuration AVP value. - -Example:: -attrib=abc matches attrib<bcd + -attrib=1 matches attrib<2 + -but beware: attrib=10 does not match attrib<9 + -attrib=bcd does not match attrib<abc + -attrib=bcd does not match attrib<bcd + - -BUGS - -It should check whether the values are numbers and compare them numerically - -[#HigherThan] -====== "Higher than" operator - -The "higher than" operator will match if the data AVP value is semantically -higher than the configuration AVP value. - -Examples - -attrib=bcd matches attrib>abc + -attrib=3 matches attrib>2 + -but beware: attrib=9 does not match attrib>10 + -attrib=abc does not match attrib>bcd + -attrib=abc does not match attrib>abc + - -BUGS - -It should check whether the values are numbers and compare them numerically - -[#Exists] -====== Exists operator - -The exists operator will always match as far as the two operands have the same -name. - -Examples - -attrib=abc matches attrib? + -attrib=abc matches attrib (this is just an alternative notation of the previous example) + -obviously attrib=abc does not match other_attrib? + - -==== Attribute/Value Pair List (AVPL) -Pdus, GoPs and GoGs use an AVPL to contain the tracing information. An AVPL is -an unsorted set of <<AVP,AVPs>> that can be matched against other AVPLs. - -===== Operations between AVPLs - -There are three types of match operations that can be performed between AVPLs. -The Pdu's/GoP's/GoG's AVPL will be always one of the operands; the AVPL operator -(match type) and the second operand AVPL will always come from the -<<Config,configuration>>. -Note that a diverse AVP match operator may be specified for each AVP in the -configuration AVPL. - -An AVPL match operation returns a result AVPL. In <<Transform,Transform>>s, the -result AVPL may be replaced by another AVPL. The replacement means that the -existing data AVPs are dropped and the replacement AVPL from the -<<Config,configuration>> is <<Merge,Merged>> to the data AVPL of the -Pdu/GoP/GoG. - -* <<Loose,Loose Match>>: Will match if at least one of the AVPs of the two -operand AVPLs match. If it matches, it returns a result AVPL containing all AVPs -from the data AVPL that did match the configuration's AVPs. -* <<Every,"Every" Match>>: Will match if none of the AVPs of the configuration -AVPL fails to match an AVP in the data AVPL, even if not all of the -configuration AVPs have a match. If it matches, it returns a result AVPL -containing all AVPs from the data AVPL that did match an AVP in the -configuration AVPL. -* <<Strict,Strict Match>>: Will match if and only if each of the AVPs in the -configuration AVPL has at least one match in the data AVPL. If it matches, it -returns a result AVPL containing those AVPs from the data AVPL that matched. - -[#Loose] -====== Loose Match - -A loose match between AVPLs succeeds if at least one of the data AVPs matches at -least one of the configuration AVPs. Its result AVPL contains all the data AVPs -that matched. - -Loose matches are used in Extra operations against the <<Pdu,Pdu>>'s AVPL to -merge the result into <<Gop,Gop>>'s AVPL, and against <<Gop,Gop>>'s AVPL to -merge the result into <<Gog,Gog>>'s AVPL. They may also be used in -<<Criteria,Criteria>> and <<Transform,Transform>>s. - -[NOTE] -==== -As of current (2.0.1), Loose Match does not work as described here, see -https://gitlab.com/wireshark/wireshark/issues/12184[issue 12184]. Only use -in Transforms and Criteria is effectively affected by the bug. -==== - -Loose Match Examples - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a?, attr_c=ccc) ==> (attr_a=aaa) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Loose (attr_a=xxx; attr_c=ccc) ==> No Match! - -[#Every] -====== Every Match - -An "every" match between AVPLs succeeds if none of the configuration's AVPs that -have a counterpart in the data AVPL fails to match. Its result AVPL contains all -the data AVPs that matched. - -These may only be used in <<Criteria,Criteria>> and <<Transform,Transform>>s. - -[NOTE] -==== -As of current (2.0.1), Loose Match does not work as described here, see -https://gitlab.com/wireshark/wireshark/-/issues/12184[issue 12184]. -==== - -"Every" Match Examples - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c?, attr_d=ddd) ==> (attr_a=aaa, attr_c=xxx) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Every (attr_a?, attr_c=ccc) ==> No Match! - -(attr_a=aaa; attr_b=bbb; attr_c=xxx) Match Every (attr_a=xxx, attr_c=ccc) ==> No Match! - -[#Strict] -====== Strict Match - -A Strict match between AVPLs succeeds if and only if every AVP in the -configuration AVPL has at least one counterpart in the data AVPL and none of the -AVP matches fails. The result AVPL contains all the data AVPs that matched. - -These are used between Gop keys (key AVPLs) and Pdu AVPLs. They may also be used -in <<Criteria,Criteria>> and <<Transform,Transform>>s. - -Examples - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c=xxx) ==> (attr_a=aaa, attr_c=xxx) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx, attr_c=yyy) Match Strict (attr_a?, attr_c?) ==> (attr_a=aaa, attr_c=xxx, attr_c=yyy) - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c=ccc) ==> No Match! - -(attr_a=aaa, attr_b=bbb, attr_c=xxx) Match Strict (attr_a?, attr_c?, attr_d?) ==> No Match! - -[#Merge] -====== AVPL Merge - -An AVPL may be merged into another one. That would add to the latter every AVP -from the former that does not already exist there. - -This operation is done - -* between the result of a key match and the Gop's or Gog's AVPL, -* between the result of an Extra match and the Gop's or Gog's AVPL, -* between the result of a <<Transform,Transform>> match and Pdu's/Gop's AVPL. If -the operation specified by the Match clause is Replace, the result AVPL of the -match is removed from the item's AVPL before the modify_avpl is merged into it. - -Examples - -(attr_a=aaa, attr_b=bbb) Merge (attr_a=aaa, attr_c=xxx) former becomes (attr_a=aaa, attr_b=bbb, attr_c=xxx) - -(attr_a=aaa, attr_b=bbb) Merge (attr_a=aaa, attr_a=xxx) former becomes (attr_a=aaa, attr_a=xxx, attr_b=bbb) - -(attr_a=aaa, attr_b=bbb) Merge (attr_c=xxx, attr_d=ddd) former becomes (attr_a=aaa, attr_b=bbb, attr_c=xxx, attr_d=ddd) - -[#Transform] -====== Transforms - -A Transform is a sequence of Match rules optionally followed by an instruction -how to modify the match result using an additional AVPL. Such modification may -be an Insert (merge) or a Replace. The syntax is as follows: - ----- -Transform name { - Match [Strict|Every|Loose] match_avpl [[Insert|Replace] modify_avpl] ; // may occur multiple times, at least once -}; ----- - -For examples of Transforms, check the <<ChMateManual,Manual>> page. - -TODO: migrate the examples here? - -The list of Match rules inside a Transform is processed top to bottom; -the processing ends as soon as either a Match rule succeeds or all have been -tried in vain. - -Transforms can be used as helpers to manipulate an item's AVPL before the item -is processed further. An item declaration may contain a Transform clause -indicating a list of previously declared Transforms. Regardless whether the -individual transforms succeed or fail, the list is always executed completely -and in the order given, i.e., left to right. - -In MATE configuration file, a Transform must be declared before declaring any -item which uses it. - -[#Config] -=== Configuration AVPLs -[#Pdu] -==== Pdsu's configuration actions - -The following configuration AVPLs deal with PDU creation and data extraction. - -===== Pdu declaration block header - -In each frame of the capture, MATE will look for source _proto_name_'s PDUs in -the order in which the declarations appear in its configuration and will create -Pdus of every type it can from that frame, unless specifically instructed that -some Pdu type is the last one to be looked for in the frame. If told so for a -given type, MATE will extract all Pdus of that type and the previously declared -types it finds in the frame but not those declared later. - -The complete declaration of a Pdu looks as below; the mandatory order of the -diverse clauses is as shown. - ----- - Pdu name Proto proto_name Transport proto1[/proto2/proto3[/...]]] { - Payload proto; //optional, no default value - Extract attribute From proto.field ; //may occur multiple times, at least once - Transform (transform1[, transform2[, ...]]); //optional - Criteria [{Accept|Reject}] [{Strict|Every|Loose} match_avpl]; - DropUnassigned {true|false}; //optional, default=false - DiscardPduData {true|false}; //optional, default=false - LastExtracted {true|false}; //optional, default=false - }; ----- - -===== Pdu name - -The _name_ is a mandatory attribute of a Pdu declaration. It is chosen -arbitrarily, except that each _name_ may only be used once in MATE's -configuration, regardless the class of an item it is used for. The _name_ is -used to distinguish between different types of Pdus, Gops, and Gogs. The _name_ -is also used as part of the filterable fields' names related to this type of Pdu -which MATE creates. - -However, several Pdu declarations may share the same _name_. In such case, all -of them are created from each source PDU matching their _Proto_, _Transport_, -and _Payload_ clauses, while the bodies of their declarations may be totally -different from each other. Together with the _Accept_ (or _Reject_) clauses, -this feature is useful when it is necessary to build the Pdu's AVPL from -different sets of source fields depending on contents (or mere presence) of -other source fields. - -====== Proto and Transport clauses - -Every instance of the protocol _proto_name_ PDU in a frame will generate one -Pdu with the AVPs extracted from fields that are in the _proto_name_'s range -and/or the ranges of underlying protocols specified by the _Transport_ list. -It is a mandatory attribute of a Pdu declaration. The _proto_name_ is the name -of the protocol as used in Wireshark display filter. - -The Pdu's _Proto_, and its _Transport_ list of protocols separated by / tell -MATE which fields of a frame can get into the Pdu's AVPL. In order that MATE -would extract an attribute from a frame's protocol tree, the area representing -the field in the hex display of the frame must be within the area of either the -_Proto_ or its relative _Transport_ s. _Transport_ s are chosen moving backwards -from the protocol area, in the order they are given. - -_Proto http Transport tcp/ip_ does what you'd expect it to - it selects the -nearest tcp range that precedes the current http range, and the nearest ip range -that precedes that tcp range. If there is another ip range before the nearest -one (e.g., in case of IP tunneling), that one is not going to be selected. -_Transport_ tcp/ip/ip that "logically" should select the encapsulating IP header -too doesn't work so far. - -Once we've selected the _Proto_ and _Transport_ ranges, MATE will fetch those -protocol fields belonging to them whose extraction is declared using the -_Extract_ clauses for the Pdu type. The _Transport_ list is also mandatory, -if you actually don't want to use any transport protocol, use _Transport mate_. -(This didn't work until 0.10.9). - -====== Payload clause - -Other than the Pdu's _Proto_ and its _Transport_ protocols, there is also a -_Payload_ attribute to tell MATE from which ranges of _Proto_'s payload to -extract fields of a frame into the Pdu. In order to extract an attribute from a -frame's tree the highlighted area of the field in the hex display must be within -the area of the _Proto_'s relative payload(s). _Payload_ s are chosen moving -forward from the protocol area, in the order they are given. -_Proto http Transport tcp/ip Payload mmse_ will select the first mmse range -after the current http range. Once we've selected the _Payload_ ranges, MATE -will fetch those protocol fields belonging to them whose extraction is declared -using the _Extract_ clauses for the Pdu type. - -====== Extract clause - -Each _Extract_ clause tells MATE which protocol field value to extract as an AVP -value and what string to use as the AVP name. The protocol fields are referred -to using the names used in Wireshark display filters. If there is more than one -such protocol field in the frame, each instance that fulfills the criteria -stated above is extracted into its own AVP. The AVP names may be chosen -arbitrarily, but to be able to match values originally coming from different -Pdus (e.g., hostname from DNS query and a hostname from HTTP GET request) later -in the analysis, identical AVP names must be assigned to them and the dissectors -must provide the field values in identical format (which is not always the case). - -====== Transform clause - -The _Transform_ clause specifies a list of previously declared _Transform_ s to -be performed on the Pdu's AVPL after all protocol fields have been extracted to -it. The list is always executed completely, left to right. On the contrary, the -list of Match clauses inside each individual _Transform_ is executed only until -the first match succeeds. - -[#Criteria] -====== Criteria clause - -This clause tells MATE whether to use the Pdu for analysis. It specifies a match -AVPL, an AVPL match type (_Strict_, _Every_, or _Loose_) and the action to be -performed (_Accept_ or _Reject_) if the match succeeds. Once every attribute has -been extracted and eventual transform list has been executed, and if the -_Criteria_ clause is present, the Pdu's AVPL is matched against the match AVPL; -if the match succeeds, the action specified is executed, i.e., the Pdu is -accepted or rejected. The default behaviors used if the respective keywords are -omitted are _Strict_ and _Accept_. Accordingly, if the clause is omitted, all -Pdus are accepted. - -====== DropUnassigned clause - -If set to _TRUE_, MATE will destroy the Pdu if it cannot assign it to a Gop. -If set to _FALSE_ (the default if not given), MATE will keep them. - -====== DiscardPduData clause - -If set to _TRUE_, MATE will delete the Pdu's AVPL once it has analyzed it and -eventually extracted some AVPs from it into the Gop's AVPL. This is useful to -save memory (of which MATE uses a lot). If set to _FALSE_ (the default if not -given), MATE will keep the Pdu attributes. - -====== LastExtracted clause - -If set to _FALSE_ (the default if not given), MATE will continue to look for -Pdus of other types in the frame. If set to _TRUE_, it will not try to create -Pdus of other types from the current frame, yet it will continue to try for the -current type. - -[#Gop] -===== Gop's configuration actions - -====== Gop declaration block header - -Declares a Gop type and its prematch candidate key. - ----- - Gop name On pduname Match key { - Start match_avpl; // optional - Stop match_avpl; // optional - Extra match_avpl; // optional - Transform transform_list; // optional - Expiration time; // optional - IdleTimeout time; // optional - Lifetime time; // optional - DropUnassigned [TRUE|FALSE]; //optional - ShowTree [NoTree|PduTree|FrameTree|BasicTree]; //optional - ShowTimes [TRUE|FALSE]; //optional, default TRUE - }; ----- - -====== Gop name - -The _name_ is a mandatory attribute of a Gop declaration. It is chosen -arbitrarily, except that each _name_ may only be used once in MATE's -configuration, regardless the class of an item it is used for. The _name_ is -used to distinguish between different types of Pdus, Gops, and Gogs. The _name_ -is also used as part of the filterable fields' names related to this type of -Gop which MATE creates. - -====== On clause - -The _name_ of Pdus which this type of Gop is supposed to be groupping. It is -mandatory. - -====== Match clause - -Defines what AVPs form up the _key_ part of the Gop's AVPL (the Gop's _key_ AVPL -or simply the Gop's _key_). All Pdus matching the _key_ AVPL of an active Gop -are assigned to that Gop; a Pdu which contains the AVPs whose attribute names -are listed in the Gop's _key_ AVPL, but they do not strictly match any active -Gop's _key_ AVPL, will create a new Gop (unless a _Start_ clause is given). -When a Gop is created, the elements of its key AVPL are copied from the creating -Pdu. - -====== Start clause - -If given, it tells MATE what match_avpl must a Pdu's AVPL match, in addition to -matching the Gop's _key_, in order to start a Gop. If not given, any Pdu whose -AVPL matches the Gop's _key_ AVPL will act as a start for a Gop. The Pdu's AVPs -matching the match_avpl are not automatically copied into the Gop's AVPL. - -====== Stop clause - -If given, it tells MATE what match_avpl must a Pdu's AVPL match, in addition to -matching the Gop's key, in order to stop a Gop. If omitted, the Gop is -"auto-stopped" - that is, the Gop is marked as stopped as soon as it is created. -The Pdu's AVPs matching the match_avpl are not automatically copied into the -Gop's AVPL. - -====== Extra clause - -If given, tells MATE which AVPs from the Pdu's AVPL are to be copied into the -Gop's AVPL in addition to the Gop's key. - -====== Transform clause - -The _Transform_ clause specifies a list of previously declared _Transform_ s to -be performed on the Gop's AVPL after the AVPs from each new Pdu, specified by -the key AVPL and the _Extra_ clause's match_avpl, have been merged into it. -The list is always executed completely, left to right. On the contrary, the list -of _Match_ clauses inside each individual _Transform_ is executed only until -the first match succeeds. - -====== Expiration clause - -A (floating) number of seconds after a Gop is _Stop_ ped during which further -Pdus matching the _Stop_ ped Gop's key but not the _Start_ condition will still -be assigned to that Gop. The default value of zero has an actual meaning of -infinity, as it disables this timer, so all Pdus matching the _Stop_ ped Gop's -key will be assigned to that Gop unless they match the _Start_ condition. - -====== IdleTimeout clause - -A (floating) number of seconds elapsed from the last Pdu assigned to the Gop -after which the Gop will be considered released. The default value of zero has -an actual meaning of infinity, as it disables this timer, so the Gop won't be -released even if no Pdus arrive - unless the _Lifetime_ timer expires. - -====== Lifetime clause - -A (floating) of seconds after the Gop _Start_ after which the Gop will be -considered released regardless anything else. The default value of zero has an -actual meaning of infinity. - -====== DropUnassigned clause - -Whether or not a Gop that has not being assigned to any Gog should be discarded. -If _TRUE_, the Gop is discarded right after creation. If _FALSE_, the default, -the unassigned Gop is kept. Setting it to _TRUE_ helps save memory and speed up -filtering. - -====== TreeMode clause - -Controls the display of Pdus subtree of the Gop: - -* _NoTree_: completely suppresses showing the tree -* _PduTree_: the tree is shown and shows the Pdus by Pdu Id -* _FrameTree_: the tree is shown and shows the Pdus by the frame number in which -they are -* _BasicTree_: needs investigation - -====== ShowTimes clause - -Whether or not to show the times subtree of the Gop. If _TRUE_, the default, -the subtree with the timers is added to the Gop's tree. If _FALSE_, the subtree -is suppressed. - -[#Gog] -===== Gog's configuration actions - -====== Gop declaration block header - -Declares a Gog type and its prematch candidate key. - ----- - Gog name { - Member gopname (key); // mandatory, at least one - Extra match_avpl; // optional - Transform transform_list; // optional - Expiration time; // optional, default 2.0 - GopTree [NoTree|PduTree|FrameTree|BasicTree]; // optional - ShowTimes [TRUE|FALSE]; // optional, default TRUE - }; ----- - -====== Gop name - -The _name_ is a mandatory attribute of a Gog declaration. It is chosen -arbitrarily, except that each _name_ may only be used once in MATE's -configuration, regardless the class of an item it is used for. The _name_ is -used to distinguish between different types of Pdus, Gops, and Gogs. The _name_ -is also used as part of the filterable fields' names related to this type of -Gop which MATE creates. - -====== Member clause - -Defines the _key_ AVPL for the Gog individually for each Gop type _gopname_. -All _gopname_ type Gops whose _key_ AVPL matches the corresponding _key_ AVPL -of an active Gog are assigned to that Gog; a Gop which contains the AVPs whose -attribute names are listed in the Gog's corresponding _key_ AVPL, but they do -not strictly match any active Gog's _key_ AVPL, will create a new Gog. When a -Gog is created, the elements of its _key_ AVPL are copied from the creating Gop. - -Although the _key_ AVPLs are specified separately for each of the Member -_gopname_ s, in most cases they are identical, as the very purpose of a Gog is -to group together Gops made of Pdus of different types. - -====== Extra clause - -If given, tells MATE which AVPs from any of the Gop's AVPL are to be copied -into the Gog's AVPL in addition to the Gog's key. - -====== Expiration clause - -A (floating) number of seconds after all the Gops assigned to a Gog have been -released during which new Gops matching any of the session keys should still be -assigned to the existing Gog instead of creating a new one. Its value can range -from 0.0 to infinite. Defaults to 2.0 seconds. - -====== Transform clause - -The _Transform_ clause specifies a list of previously declared _Transform_ s to -be performed on the Gog's AVPL after the AVPs from each new Gop, specified by -the _key_ AVPL and the _Extra_ clause's match_avpl, have been merged into it. -The list is always executed completely, left to right. On the contrary, the list -of _Match_ clauses inside each individual _Transform_ is executed only until -the first match succeeds. - -====== TreeMode clause - -Controls the display of Gops subtree of the Gog: - -* _NoTree_: completely suppresses showing the tree -* _BasicTree_: needs investigation -* _FullTree_: needs investigation - -====== ShowTimes clause - -Whether or not to show the times subtree of the Gog. If _TRUE_, the default, -the subtree with the timers is added to the Gog's tree. If _FALSE_, the subtree -is suppressed. - -===== Settings Config AVPL - -The *Settings* config element is used to pass to MATE various operational -parameters. the possible parameters are - -====== GogExpiration - -How long in seconds after all the gops assigned to a gog have been released new -gops matching any of the session keys should create a new gog instead of being -assigned to the previous one. Its value can range from 0.0 to infinite. -Defaults to 2.0 seconds. - -====== DiscardPduData - -Whether or not the AVPL of every Pdu should be deleted after it was being -processed (saves memory). It can be either _TRUE_ or _FALSE_. Defaults to _TRUE_. -Setting it to _FALSE_ can save you from a headache if your config does not work. - -====== DiscardUnassignedPdu - -Whether Pdus should be deleted if they are not assigned to any Gop. It can be -either _TRUE_ or _FALSE_. Defaults to _FALSE_. Set it to _TRUE_ to save memory -if unassigned Pdus are useless. - -====== DiscardUnassignedGop - -Whether GoPs should be deleted if they are not assigned to any session. It can -be either _TRUE_ or _FALSE_. Defaults to _FALSE_. Setting it to _TRUE_ saves -memory. - -====== ShowPduTree - -====== ShowGopTimes - -===== Debugging Stuff - -The following settings are used to debug MATE and its configuration. All levels -are integers ranging from 0 (print only errors) to 9 (flood me with junk), -defaulting to 0. - -====== Debug declaration block header - ----- - Debug { - Filename "path/name"; //optional, no default value - Level [0-9]; //optional, generic debug level - Pdu Level [0-9]; //optional, specific debug level for Pdu handling - Gop Level [0-9]; //optional, specific debug level for Gop handling - Gog Level [0-9]; //optional, specific debug level for Gog handling - }; ----- - -====== Filename clause - -The {{{path/name}}} is a full path to the file to which debug output is to be -written. Non-existent file will be created, existing file will be overwritten -at each opening of a capture file. If the statement is missing, debug messages -are written to console, which means they are invisible on Windows. - -====== Level clause - -Sets the level of debugging for generic debug messages. It is an integer -ranging from 0 (print only errors) to 9 (flood me with junk). - -====== Pdu Level clause - -Sets the level of debugging for messages regarding Pdu creation. It is an -integer ranging from 0 (print only errors) to 9 (flood me with junk). - -====== Gop Level clause - -Sets the level of debugging for messages regarding Pdu analysis (that is how do -they fit into ?GoPs). It is an integer ranging from 0 (print only errors) to 9 -(flood me with junk). - -====== Gog Level clause - -Sets the level of debugging for messages regarding GoP analysis (that is how do -they fit into ?GoGs). It is an integer ranging from 0 (print only errors) to 9 -(flood me with junk). - -====== Settings Example ----- -Action=Settings; SessionExpiration=3.5; DiscardPduData=FALSE; ----- - -===== Action=Include - -Will include a file to the configuration. - ----- -Action=Include; {Filename=filename;|Lib=libname;} ----- - -====== Filename - -The filename of the file to include. If it does not begin with '/' it will look -for the file in the current path. - -====== Lib - -The name of the lib config to include. will look for libname.mate in -wiresharks_dir/matelib. - -====== Include Example ----- -Action=Include; Filename=rtsp.mate; ----- - -This will include the file called "rtsp.mate" into the current config. |