diff options
Diffstat (limited to 'src/spdk/dpdk/doc/guides/sample_app_ug')
78 files changed, 25887 insertions, 0 deletions
diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/bbdev_app.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/bbdev_app.rst new file mode 100644 index 00000000..653f972f --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/bbdev_app.rst @@ -0,0 +1,133 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2017 Intel Corporation + +.. bbdev_app: + +Loop-back Sample Application using Baseband Device (bbdev) +========================================================== + +The baseband sample application is a simple example of packet processing using +the Data Plane Development Kit (DPDK) for baseband workloads using Wireless +Device abstraction library. + +Overview +-------- + +The Baseband device sample application performs a loop-back operation using a +baseband device capable of transceiving data packets. +A packet is received on an ethernet port -> enqueued for downlink baseband +operation -> dequeued from the downlink baseband device -> enqueued for uplink +baseband operation -> dequeued from the baseband device -> then the received +packet is compared with the baseband operations output. Then it's looped back to +the ethernet port. + +* The MAC header is preserved in the packet + +Limitations +----------- + +* Only one baseband device and one ethernet port can be used. + +Compiling the Application +------------------------- + +#. DPDK needs to be built with ``baseband_turbo_sw`` PMD driver enabled along + with ``FLEXRAN SDK`` Libraries. Refer to *SW Turbo Poll Mode Driver* + documentation for more details on this. + +#. Go to the example directory: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/bbdev_app + +#. Set the target (a default target is used if not specified). For example: + + .. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + + See the *DPDK Getting Started Guide* for possible RTE_TARGET values. + +#. Build the application: + + .. code-block:: console + + make + +Running the Application +----------------------- + +The application accepts a number of command line options: + +.. code-block:: console + + $ ./build/bbdev [EAL options] -- [-e ENCODING_CORES] [-d DECODING_CORES] / + [-p ETH_PORT_ID] [-b BBDEV_ID] + +where: + +* ``e ENCODING_CORES``: hexmask for encoding lcored (default = 0x2) +* ``d DECODING_CORES``: hexmask for decoding lcores (default = 0x4) +* ``p ETH_PORT_ID``: ethernet port ID (default = 0) +* ``b BBDEV_ID``: BBDev ID (default = 0) + +The application requires that baseband devices is capable of performing +the specified baseband operation are available on application initialization. +This means that HW baseband device/s must be bound to a DPDK driver or +a SW baseband device/s (virtual BBdev) must be created (using --vdev). + +To run the application in linuxapp environment with the turbo_sw baseband device +using the whitelisted port running on 1 encoding lcore and 1 decoding lcore +issue the command: + +.. code-block:: console + + $ ./build/bbdev --vdev='baseband_turbo_sw' -w <NIC0PCIADDR> -c 0x38 --socket-mem=2,2 \ + --file-prefix=bbdev -- -e 0x10 -d 0x20 + +where, NIC0PCIADDR is the PCI addresse of the Rx port + +This command creates one virtual bbdev devices ``baseband_turbo_sw`` where the +device gets linked to a corresponding ethernet port as whitelisted by +the parameter -w. +3 cores are allocated to the application, and assigned as: + + - core 3 is the master and used to print the stats live on screen, + + - core 4 is the encoding lcore performing Rx and Turbo Encode operations + + - core 5 is the downlink lcore performing Turbo Decode, validation and Tx + operations + + +Refer to the *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + +Using Packet Generator with baseband device sample application +-------------------------------------------------------------- + +To allow the bbdev sample app to do the loopback, an influx of traffic is required. +This can be done by using DPDK Pktgen to burst traffic on two ethernet ports, and +it will print the transmitted along with the looped-back traffic on Rx ports. +Executing the command below will generate traffic on the two whitelisted ethernet +ports. + +.. code-block:: console + + $ ./pktgen-3.4.0/app/x86_64-native-linuxapp-gcc/pktgen -c 0x3 \ + --socket-mem=1,1 --file-prefix=pg -w <NIC1PCIADDR> -- -m 1.0 -P + +where: + +* ``-c COREMASK``: A hexadecimal bitmask of cores to run on +* ``--socket-mem``: Memory to allocate on specific sockets (use comma separated values) +* ``--file-prefix``: Prefix for hugepage filenames +* ``-w <NIC1PCIADDR>``: Add a PCI device in white list. The argument format is <[domain:]bus:devid.func>. +* ``-m <string>``: Matrix for mapping ports to logical cores. +* ``-P``: PROMISCUOUS mode + + +Refer to *The Pktgen Application* documents for general information on running +Pktgen with DPDK applications. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/cmd_line.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/cmd_line.rst new file mode 100644 index 00000000..b38a5b33 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/cmd_line.rst @@ -0,0 +1,158 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Command Line Sample Application +=============================== + +This chapter describes the Command Line sample application that +is part of the Data Plane Development Kit (DPDK). + +Overview +-------- + +The Command Line sample application is a simple application that +demonstrates the use of the command line interface in the DPDK. +This application is a readline-like interface that can be used +to debug a DPDK application, in a Linux* application environment. + +.. note:: + + The rte_cmdline library should not be used in production code since + it is not validated to the same standard as other DPDK libraries. + See also the "rte_cmdline library should not be used in production code due to limited testing" item + in the "Known Issues" section of the Release Notes. + +The Command Line sample application supports some of the features of the GNU readline library such as, completion, +cut/paste and some other special bindings that make configuration and debug faster and easier. + +The application shows how the rte_cmdline application can be extended to handle a list of objects. +There are three simple commands: + +* add obj_name IP: Add a new object with an IP/IPv6 address associated to it. + +* del obj_name: Delete the specified object. + +* show obj_name: Show the IP associated with the specified object. + +.. note:: + + To terminate the application, use **Ctrl-d**. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling` + +The application is located in the ``cmd_line`` sub-directory. + +Running the Application +----------------------- + +To run the application in linuxapp environment, issue the following command: + +.. code-block:: console + + $ ./build/cmdline -l 0-3 -n 4 + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +EAL Initialization and cmdline Start +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first task is the initialization of the Environment Abstraction Layer (EAL). +This is achieved as follows: + +.. code-block:: c + + int main(int argc, char **argv) + { + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + +Then, a new command line object is created and started to interact with the user through the console: + +.. code-block:: c + + cl = cmdline_stdin_new(main_ctx, "example> "); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + +The cmd line_interact() function returns when the user types **Ctrl-d** and in this case, +the application exits. + +Defining a cmdline Context +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A cmdline context is a list of commands that are listed in a NULL-terminated table, for example: + +.. code-block:: c + + cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *) &cmd_obj_del_show, + (cmdline_parse_inst_t *) &cmd_obj_add, + (cmdline_parse_inst_t *) &cmd_help, + NULL, + }; + +Each command (of type cmdline_parse_inst_t) is defined statically. +It contains a pointer to a callback function that is executed when the command is parsed, +an opaque pointer, a help string and a list of tokens in a NULL-terminated table. + +The rte_cmdline application provides a list of pre-defined token types: + +* String Token: Match a static string, a list of static strings or any string. + +* Number Token: Match a number that can be signed or unsigned, from 8-bit to 32-bit. + +* IP Address Token: Match an IPv4 or IPv6 address or network. + +* Ethernet* Address Token: Match a MAC address. + +In this example, a new token type obj_list is defined and implemented +in the parse_obj_list.c and parse_obj_list.h files. + +For example, the cmd_obj_del_show command is defined as shown below: + +.. code-block:: c + + struct cmd_obj_add_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t name; + struct object *obj; + }; + + static void cmd_obj_del_show_parsed(void *parsed_result, struct cmdline *cl, attribute ((unused)) void *data) + { + /* ... */ + } + + cmdline_parse_token_string_t cmd_obj_action = TOKEN_STRING_INITIALIZER(struct cmd_obj_del_show_result, action, "show#del"); + + parse_token_obj_list_t cmd_obj_obj = TOKEN_OBJ_LIST_INITIALIZER(struct cmd_obj_del_show_result, obj, &global_obj_list); + + cmdline_parse_inst_t cmd_obj_del_show = { + .f = cmd_obj_del_show_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "Show/del an object", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_obj_action, + (void *)&cmd_obj_obj, + NULL, + }, + }; + +This command is composed of two tokens: + +* The first token is a string token that can be show or del. + +* The second token is an object that was previously added using the add command in the global_obj_list variable. + +Once the command is parsed, the rte_cmdline application fills a cmd_obj_del_show_result structure. +A pointer to this structure is given as an argument to the callback function and can be used in the body of this function. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/compiling.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/compiling.rst new file mode 100644 index 00000000..a2d75ed2 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/compiling.rst @@ -0,0 +1,95 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +Compiling the Sample Applications +================================= + +This section explains how to compile the DPDK sample applications. + +To compile all the sample applications +-------------------------------------- + + +Set the path to DPDK source code if its not set: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + +Go to DPDK source: + + .. code-block:: console + + cd $RTE_SDK + +Build DPDK: + + .. code-block:: console + + make defconfig + make + +Build the sample applications: + + .. code-block:: console + + export RTE_TARGET=build + make -C examples + +For other possible ``RTE_TARGET`` values and additional information on +compiling see +:ref:`Compiling DPDK on Linux <linux_gsg_compiling_dpdk>` or +:ref:`Compiling DPDK on FreeBSD <building_from_source>`. +Applications are output to: ``$RTE_SDK/examples/app-dir/build`` or +``$RTE_SDK/examples/app-dir/$RTE_TARGET``. + + +In the example above the compiled application is written to the ``build`` subdirectory. +To have the applications written to a different location, +the ``O=/path/to/build/directory`` option may be specified in the make command. + + .. code-block:: console + + make O=/tmp + +To build the applications for debugging use the ``DEBUG`` option. +This option adds some extra flags, disables compiler optimizations and +sets verbose output. + + .. code-block:: console + + make DEBUG=1 + + +To compile a single application +------------------------------- + +Set the path to DPDK source code: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + +Go to DPDK source: + + .. code-block:: console + + cd $RTE_SDK + +Build DPDK: + + .. code-block:: console + + make defconfig + make + +Go to the sample application directory. Unless otherwise specified the sample +applications are located in ``$RTE_SDK/examples/``. + + +Build the application: + + .. code-block:: console + + export RTE_TARGET=build + make diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/dist_app.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/dist_app.rst new file mode 100644 index 00000000..1f68d081 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/dist_app.rst @@ -0,0 +1,134 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Distributor Sample Application +============================== + +The distributor sample application is a simple example of packet distribution +to cores using the Data Plane Development Kit (DPDK). + +Overview +-------- + +The distributor application performs the distribution of packets that are received +on an RX_PORT to different cores. When processed by the cores, the destination +port of a packet is the port from the enabled port mask adjacent to the one on +which the packet was received, that is, if the first four ports are enabled +(port mask 0xf), ports 0 and 1 RX/TX into each other, and ports 2 and 3 RX/TX +into each other. + +This application can be used to benchmark performance using the traffic +generator as shown in the figure below. + +.. _figure_dist_perf: + +.. figure:: img/dist_perf.* + + Performance Benchmarking Setup (Basic Environment) + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``distributor`` sub-directory. + +Running the Application +----------------------- + +#. The application has a number of command line options: + + .. code-block:: console + + ./build/distributor_app [EAL options] -- -p PORTMASK + + where, + + * -p PORTMASK: Hexadecimal bitmask of ports to configure + +#. To run the application in linuxapp environment with 10 lcores, 4 ports, + issue the command: + + .. code-block:: console + + $ ./build/distributor_app -l 1-9,22 -n 4 -- -p f + +#. Refer to the DPDK Getting Started Guide for general information on running + applications and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The distributor application consists of four types of threads: a receive +thread (``lcore_rx()``), a distributor thread (``lcore_dist()``), a set of +worker threads (``lcore_worker()``), and a transmit thread(``lcore_tx()``). +How these threads work together is shown in :numref:`figure_dist_app` below. +The ``main()`` function launches threads of these four types. Each thread +has a while loop which will be doing processing and which is terminated +only upon SIGINT or ctrl+C. + +The receive thread receives the packets using ``rte_eth_rx_burst()`` and will +enqueue them to an rte_ring. The distributor thread will dequeue the packets +from the ring and assign them to workers (using ``rte_distributor_process()`` API). +This assignment is based on the tag (or flow ID) of the packet - indicated by +the hash field in the mbuf. For IP traffic, this field is automatically filled +by the NIC with the "usr" hash value for the packet, which works as a per-flow +tag. The distributor thread communicates with the worker threads using a +cache-line swapping mechanism, passing up to 8 mbuf pointers at a time +(one cache line) to each worker. + +More than one worker thread can exist as part of the application, and these +worker threads do simple packet processing by requesting packets from +the distributor, doing a simple XOR operation on the input port mbuf field +(to indicate the output port which will be used later for packet transmission) +and then finally returning the packets back to the distributor thread. + +The distributor thread will then call the distributor api +``rte_distributor_returned_pkts()`` to get the processed packets, and will enqueue +them to another rte_ring for transfer to the TX thread for transmission on the +output port. The transmit thread will dequeue the packets from the ring and +transmit them on the output port specified in packet mbuf. + +Users who wish to terminate the running of the application have to press ctrl+C +(or send SIGINT to the app). Upon this signal, a signal handler provided +in the application will terminate all running threads gracefully and print +final statistics to the user. + +.. _figure_dist_app: + +.. figure:: img/dist_app.* + + Distributor Sample Application Layout + + +Debug Logging Support +--------------------- + +Debug logging is provided as part of the application; the user needs to uncomment +the line "#define DEBUG" defined in start of the application in main.c to enable debug logs. + +Statistics +---------- + +The main function will print statistics on the console every second. These +statistics include the number of packets enqueued and dequeued at each stage +in the application, and also key statistics per worker, including how many +packets of each burst size (1-8) were sent to each worker thread. + +Application Initialization +-------------------------- + +Command line parsing is done in the same way as it is done in the L2 Forwarding Sample +Application. See :ref:`l2_fwd_app_cmd_arguments`. + +Mbuf pool initialization is done in the same way as it is done in the L2 Forwarding +Sample Application. See :ref:`l2_fwd_app_mbuf_init`. + +Driver Initialization is done in same way as it is done in the L2 Forwarding Sample +Application. See :ref:`l2_fwd_app_dvr_init`. + +RX queue initialization is done in the same way as it is done in the L2 Forwarding +Sample Application. See :ref:`l2_fwd_app_rx_init`. + +TX queue initialization is done in the same way as it is done in the L2 Forwarding +Sample Application. See :ref:`l2_fwd_app_tx_init`. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ethtool.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ethtool.rst new file mode 100644 index 00000000..47e09f6e --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ethtool.rst @@ -0,0 +1,116 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +Ethtool Sample Application +========================== + +The Ethtool sample application shows an implementation of an +ethtool-like API and provides a console environment that allows +its use to query and change Ethernet card parameters. The sample +is based upon a simple L2 frame reflector. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ethtool`` sub-directory. + +Running the Application +----------------------- + +The application requires an available core for each port, plus one. +The only available options are the standard ones for the EAL: + +.. code-block:: console + + ./ethtool-app/ethtool-app/${RTE_TARGET}/ethtool [EAL options] + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) +options. + +Using the application +--------------------- + +The application is console-driven using the cmdline DPDK interface: + +.. code-block:: console + + EthApp> + +From this interface the available commands and descriptions of what +they do as as follows: + +* ``drvinfo``: Print driver info +* ``eeprom``: Dump EEPROM to file +* ``module-eeprom``: Dump plugin module EEPROM to file +* ``link``: Print port link states +* ``macaddr``: Gets/sets MAC address +* ``mtu``: Set NIC MTU +* ``open``: Open port +* ``pause``: Get/set port pause state +* ``portstats``: Print port statistics +* ``regs``: Dump port register(s) to file +* ``ringparam``: Get/set ring parameters +* ``rxmode``: Toggle port Rx mode +* ``stop``: Stop port +* ``validate``: Check that given MAC address is valid unicast address +* ``vlan``: Add/remove VLAN id +* ``quit``: Exit program + + +Explanation +----------- + +The sample program has two parts: A background `packet reflector`_ +that runs on a slave core, and a foreground `Ethtool Shell`_ that +runs on the master core. These are described below. + +Packet Reflector +~~~~~~~~~~~~~~~~ + +The background packet reflector is intended to demonstrate basic +packet processing on NIC ports controlled by the Ethtool shim. +Each incoming MAC frame is rewritten so that it is returned to +the sender, using the port in question's own MAC address as the +source address, and is then sent out on the same port. + +Ethtool Shell +~~~~~~~~~~~~~ + +The foreground part of the Ethtool sample is a console-based +interface that accepts commands as described in `using the +application`_. Individual call-back functions handle the detail +associated with each command, which make use of the functions +defined in the `Ethtool interface`_ to the DPDK functions. + +Ethtool interface +----------------- + +The Ethtool interface is built as a separate library, and implements +the following functions: + +- ``rte_ethtool_get_drvinfo()`` +- ``rte_ethtool_get_regs_len()`` +- ``rte_ethtool_get_regs()`` +- ``rte_ethtool_get_link()`` +- ``rte_ethtool_get_eeprom_len()`` +- ``rte_ethtool_get_eeprom()`` +- ``rte_ethtool_set_eeprom()`` +- ``rte_ethtool_get_module_info()`` +- ``rte_ethtool_get_module_eeprom()`` +- ``rte_ethtool_get_pauseparam()`` +- ``rte_ethtool_set_pauseparam()`` +- ``rte_ethtool_net_open()`` +- ``rte_ethtool_net_stop()`` +- ``rte_ethtool_net_get_mac_addr()`` +- ``rte_ethtool_net_set_mac_addr()`` +- ``rte_ethtool_net_validate_addr()`` +- ``rte_ethtool_net_change_mtu()`` +- ``rte_ethtool_net_get_stats64()`` +- ``rte_ethtool_net_vlan_rx_add_vid()`` +- ``rte_ethtool_net_vlan_rx_kill_vid()`` +- ``rte_ethtool_net_set_rx_mode()`` +- ``rte_ethtool_get_ringparam()`` +- ``rte_ethtool_set_ringparam()`` diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/eventdev_pipeline.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/eventdev_pipeline.rst new file mode 100644 index 00000000..0ec02907 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/eventdev_pipeline.rst @@ -0,0 +1,145 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2017 Intel Corporation. + +Eventdev Pipeline Sample Application +==================================== + +The eventdev pipeline sample application is a sample app that demonstrates +the usage of the eventdev API using the software PMD. It shows how an +application can configure a pipeline and assign a set of worker cores to +perform the processing required. + +The application has a range of command line arguments allowing it to be +configured for various numbers worker cores, stages,queue depths and cycles per +stage of work. This is useful for performance testing as well as quickly testing +a particular pipeline configuration. + + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``examples`` sub-directory. + + + +Running the Application +----------------------- + +The application has a lot of command line options. This allows specification of +the eventdev PMD to use, and a number of attributes of the processing pipeline +options. + +An example eventdev pipeline running with the software eventdev PMD using +these settings is shown below: + + * ``-r1``: core mask 0x1 for RX + * ``-t1``: core mask 0x1 for TX + * ``-e4``: core mask 0x4 for the software scheduler + * ``-w FF00``: core mask for worker cores, 8 cores from 8th to 16th + * ``-s4``: 4 atomic stages + * ``-n0``: process infinite packets (run forever) + * ``-c32``: worker dequeue depth of 32 + * ``-W1000``: do 1000 cycles of work per packet in each stage + * ``-D``: dump statistics on exit + +.. code-block:: console + + ./build/eventdev_pipeline --vdev event_sw0 -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D + +The application has some sanity checking built-in, so if there is a function +(eg; the RX core) which doesn't have a cpu core mask assigned, the application +will print an error message: + +.. code-block:: console + + Core part of pipeline was not assigned any cores. This will stall the + pipeline, please check core masks (use -h for details on setting core masks): + rx: 0 + tx: 1 + +Configuration of the eventdev is covered in detail in the programmers guide, +see the Event Device Library section. + + +Observing the Application +------------------------- + +At runtime the eventdev pipeline application prints out a summary of the +configuration, and some runtime statistics like packets per second. On exit the +worker statistics are printed, along with a full dump of the PMD statistics if +required. The following sections show sample output for each of the output +types. + +Configuration +~~~~~~~~~~~~~ + +This provides an overview of the pipeline, +scheduling type at each stage, and parameters to options such as how many +flows to use and what eventdev PMD is in use. See the following sample output +for details: + +.. code-block:: console + + Config: + ports: 2 + workers: 8 + packets: 0 + priorities: 1 + Queue-prio: 0 + qid0 type: atomic + Cores available: 44 + Cores used: 10 + Eventdev 0: event_sw + Stages: + Stage 0, Type Atomic Priority = 128 + Stage 1, Type Atomic Priority = 128 + Stage 2, Type Atomic Priority = 128 + Stage 3, Type Atomic Priority = 128 + +Runtime +~~~~~~~ + +At runtime, the statistics of the consumer are printed, stating the number of +packets received, runtime in milliseconds, average mpps, and current mpps. + +.. code-block:: console + + # consumer RX= xxxxxxx, time yyyy ms, avg z.zzz mpps [current w.www mpps] + +Shutdown +~~~~~~~~ + +At shutdown, the application prints the number of packets received and +transmitted, and an overview of the distribution of work across worker cores. + +.. code-block:: console + + Signal 2 received, preparing to exit... + worker 12 thread done. RX=4966581 TX=4966581 + worker 13 thread done. RX=4963329 TX=4963329 + worker 14 thread done. RX=4953614 TX=4953614 + worker 0 thread done. RX=0 TX=0 + worker 11 thread done. RX=4970549 TX=4970549 + worker 10 thread done. RX=4986391 TX=4986391 + worker 9 thread done. RX=4970528 TX=4970528 + worker 15 thread done. RX=4974087 TX=4974087 + worker 8 thread done. RX=4979908 TX=4979908 + worker 2 thread done. RX=0 TX=0 + + Port Workload distribution: + worker 0 : 12.5 % (4979876 pkts) + worker 1 : 12.5 % (4970497 pkts) + worker 2 : 12.5 % (4986359 pkts) + worker 3 : 12.5 % (4970517 pkts) + worker 4 : 12.5 % (4966566 pkts) + worker 5 : 12.5 % (4963297 pkts) + worker 6 : 12.5 % (4953598 pkts) + worker 7 : 12.5 % (4974055 pkts) + +To get a full dump of the state of the eventdev PMD, pass the ``-D`` flag to +this application. When the app is terminated using ``Ctrl+C``, the +``rte_event_dev_dump()`` function is called, resulting in a dump of the +statistics that the PMD provides. The statistics provided depend on the PMD +used, see the Event Device Drivers section for a list of eventdev PMDs. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/exception_path.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/exception_path.rst new file mode 100644 index 00000000..a5590870 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/exception_path.rst @@ -0,0 +1,281 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Exception Path Sample Application +================================= + +The Exception Path sample application is a simple example that demonstrates the use of the DPDK +to set up an exception path for packets to go through the Linux* kernel. +This is done by using virtual TAP network interfaces. +These can be read from and written to by the DPDK application and +appear to the kernel as a standard network interface. + +Overview +-------- + +The application creates two threads for each NIC port being used. +One thread reads from the port and writes the data unmodified to a thread-specific TAP interface. +The second thread reads from a TAP interface and writes the data unmodified to the NIC port. + +The packet flow through the exception path application is as shown in the following figure. + +.. _figure_exception_path_example: + +.. figure:: img/exception_path_example.* + + Packet Flow + + +To make throughput measurements, kernel bridges must be setup to forward data between the bridges appropriately. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``exception_path`` sub-directory. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + .build/exception_path [EAL options] -- -p PORTMASK -i IN_CORES -o OUT_CORES + +where: + +* -p PORTMASK: A hex bitmask of ports to use + +* -i IN_CORES: A hex bitmask of cores which read from NIC + +* -o OUT_CORES: A hex bitmask of cores which write to NIC + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +The number of bits set in each bitmask must be the same. +The coremask -c or the corelist -l parameter of the EAL options should include IN_CORES and OUT_CORES. +The same bit must not be set in IN_CORES and OUT_CORES. +The affinities between ports and cores are set beginning with the least significant bit of each mask, that is, +the port represented by the lowest bit in PORTMASK is read from by the core represented by the lowest bit in IN_CORES, +and written to by the core represented by the lowest bit in OUT_CORES. + +For example to run the application with two ports and four cores: + +.. code-block:: console + + ./build/exception_path -l 0-3 -n 4 -- -p 3 -i 3 -o c + +Getting Statistics +~~~~~~~~~~~~~~~~~~ + +While the application is running, statistics on packets sent and +received can be displayed by sending the SIGUSR1 signal to the application from another terminal: + +.. code-block:: console + + killall -USR1 exception_path + +The statistics can be reset by sending a SIGUSR2 signal in a similar way. + +Explanation +----------- + +The following sections provide some explanation of the code. + +Initialization +~~~~~~~~~~~~~~ + +Setup of the mbuf pool, driver and queues is similar to the setup done in the :ref:`l2_fwd_app_real_and_virtual`. +In addition, the TAP interfaces must also be created. +A TAP interface is created for each lcore that is being used. +The code for creating the TAP interface is as follows: + +.. code-block:: c + + /* + * Create a tap network interface, or use existing one with same name. + * If name[0]='\0' then a name is automatically assigned and returned in name. + */ + + static int tap_create(char *name) + { + struct ifreq ifr; + int fd, ret; + + fd = open("/dev/net/tun", O_RDWR); + if (fd < 0) + return fd; + + memset(&ifr, 0, sizeof(ifr)); + + /* TAP device without packet information */ + + ifr.ifr_flags = IFF_TAP | IFF_NO_PI; + if (name && *name) + rte_snprinf(ifr.ifr_name, IFNAMSIZ, name); + + ret = ioctl(fd, TUNSETIFF, (void *) &ifr); + + if (ret < 0) { + close(fd); + return ret; + + } + + if (name) + snprintf(name, IFNAMSIZ, ifr.ifr_name); + + return fd; + } + +The other step in the initialization process that is unique to this sample application +is the association of each port with two cores: + +* One core to read from the port and write to a TAP interface + +* A second core to read from a TAP interface and write to the port + +This is done using an array called port_ids[], which is indexed by the lcore IDs. +The population of this array is shown below: + +.. code-block:: c + + tx_port = 0; + rx_port = 0; + + RTE_LCORE_FOREACH(i) { + if (input_cores_mask & (1ULL << i)) { + /* Skip ports that are not enabled */ + while ((ports_mask & (1 << rx_port)) == 0) { + rx_port++; + if (rx_port > (sizeof(ports_mask) * 8)) + goto fail; /* not enough ports */ + } + port_ids[i] = rx_port++; + } else if (output_cores_mask & (1ULL << i)) { + /* Skip ports that are not enabled */ + while ((ports_mask & (1 << tx_port)) == 0) { + tx_port++; + if (tx_port > (sizeof(ports_mask) * 8)) + goto fail; /* not enough ports */ + } + port_ids[i] = tx_port++; + } + } + +Packet Forwarding +~~~~~~~~~~~~~~~~~ + +After the initialization steps are complete, the main_loop() function is run on each lcore. +This function first checks the lcore_id against the user provided input_cores_mask and output_cores_mask to see +if this core is reading from or writing to a TAP interface. + +For the case that reads from a NIC port, the packet reception is the same as in the L2 Forwarding sample application +(see :ref:`l2_fwd_app_rx_tx_packets`). +The packet transmission is done by calling write() with the file descriptor of the appropriate TAP interface +and then explicitly freeing the mbuf back to the pool. + +.. code-block:: c + + /* Loop forever reading from NIC and writing to tap */ + + for (;;) { + struct rte_mbuf *pkts_burst[PKT_BURST_SZ]; + unsigned i; + + const unsigned nb_rx = rte_eth_rx_burst(port_ids[lcore_id], 0, pkts_burst, PKT_BURST_SZ); + + lcore_stats[lcore_id].rx += nb_rx; + + for (i = 0; likely(i < nb_rx); i++) { + struct rte_mbuf *m = pkts_burst[i]; + int ret = write(tap_fd, rte_pktmbuf_mtod(m, void*), + + rte_pktmbuf_data_len(m)); + rte_pktmbuf_free(m); + if (unlikely(ret<0)) + lcore_stats[lcore_id].dropped++; + else + lcore_stats[lcore_id].tx++; + } + } + +For the other case that reads from a TAP interface and writes to a NIC port, +packets are retrieved by doing a read() from the file descriptor of the appropriate TAP interface. +This fills in the data into the mbuf, then other fields are set manually. +The packet can then be transmitted as normal. + +.. code-block:: c + + /* Loop forever reading from tap and writing to NIC */ + + for (;;) { + int ret; + struct rte_mbuf *m = rte_pktmbuf_alloc(pktmbuf_pool); + + if (m == NULL) + continue; + + ret = read(tap_fd, m->pkt.data, MAX_PACKET_SZ); lcore_stats[lcore_id].rx++; + if (unlikely(ret < 0)) { + FATAL_ERROR("Reading from %s interface failed", tap_name); + } + + m->pkt.nb_segs = 1; + m->pkt.next = NULL; + m->pkt.data_len = (uint16_t)ret; + + ret = rte_eth_tx_burst(port_ids[lcore_id], 0, &m, 1); + if (unlikely(ret < 1)) { + rte_pktmuf_free(m); + lcore_stats[lcore_id].dropped++; + } + else { + lcore_stats[lcore_id].tx++; + } + } + +To set up loops for measuring throughput, TAP interfaces can be connected using bridging. +The steps to do this are described in the section that follows. + +Managing TAP Interfaces and Bridges +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Exception Path sample application creates TAP interfaces with names of the format tap_dpdk_nn, +where nn is the lcore ID. These TAP interfaces need to be configured for use: + +.. code-block:: console + + ifconfig tap_dpdk_00 up + +To set up a bridge between two interfaces so that packets sent to one interface can be read from another, +use the brctl tool: + +.. code-block:: console + + brctl addbr "br0" + brctl addif br0 tap_dpdk_00 + brctl addif br0 tap_dpdk_03 + ifconfig br0 up + +The TAP interfaces created by this application exist only when the application is running, +so the steps above need to be repeated each time the application is run. +To avoid this, persistent TAP interfaces can be created using openvpn: + +.. code-block:: console + + openvpn --mktun --dev tap_dpdk_00 + +If this method is used, then the steps above have to be done only once and +the same TAP interfaces can be reused each time the application is run. +To remove bridges and persistent TAP interfaces, the following commands are used: + +.. code-block:: console + + ifconfig br0 down + brctl delbr br0 + openvpn --rmtun --dev tap_dpdk_00 + diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/flow_classify.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/flow_classify.rst new file mode 100644 index 00000000..003ed035 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/flow_classify.rst @@ -0,0 +1,540 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2017 Intel Corporation. + +Flow Classify Sample Application +================================ + +The Flow Classify sample application is based on the simple *skeleton* example +of a forwarding application. + +It is intended as a demonstration of the basic components of a DPDK forwarding +application which uses the Flow Classify library API's. + +Please refer to the +:doc:`../prog_guide/flow_classify_lib` +for more information. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``flow_classify`` sub-directory. + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + cd ~/dpdk/examples/flow_classify + ./build/flow_classify -c 4 -n 4 -- --rule_ipv4="../ipv4_rules_file.txt" + +Please refer to the *DPDK Getting Started Guide*, section +:doc:`../linux_gsg/build_sample_apps` +for general information on running applications and the Environment Abstraction +Layer (EAL) options. + + +Sample ipv4_rules_file.txt +-------------------------- + +.. code-block:: console + + #file format: + #src_ip/masklen dst_ip/masklen src_port : mask dst_port : mask proto/mask priority + # + 2.2.2.3/24 2.2.2.7/24 32 : 0xffff 33 : 0xffff 17/0xff 0 + 9.9.9.3/24 9.9.9.7/24 32 : 0xffff 33 : 0xffff 17/0xff 1 + 9.9.9.3/24 9.9.9.7/24 32 : 0xffff 33 : 0xffff 6/0xff 2 + 9.9.8.3/24 9.9.8.7/24 32 : 0xffff 33 : 0xffff 6/0xff 3 + 6.7.8.9/24 2.3.4.5/24 32 : 0x0000 33 : 0x0000 132/0xff 4 + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + +ACL field definitions for the IPv4 5 tuple rule +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following field definitions are used when creating the ACL table during +initialisation of the ``Flow Classify`` application.. + +.. code-block:: c + + enum { + PROTO_FIELD_IPV4, + SRC_FIELD_IPV4, + DST_FIELD_IPV4, + SRCP_FIELD_IPV4, + DSTP_FIELD_IPV4, + NUM_FIELDS_IPV4 + }; + + enum { + PROTO_INPUT_IPV4, + SRC_INPUT_IPV4, + DST_INPUT_IPV4, + SRCP_DESTP_INPUT_IPV4 + }; + + static struct rte_acl_field_def ipv4_defs[NUM_FIELDS_IPV4] = { + /* first input field - always one byte long. */ + { + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint8_t), + .field_index = PROTO_FIELD_IPV4, + .input_index = PROTO_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, next_proto_id), + }, + /* next input field (IPv4 source address) - 4 consecutive bytes. */ + { + /* rte_flow uses a bit mask for IPv4 addresses */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint32_t), + .field_index = SRC_FIELD_IPV4, + .input_index = SRC_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, src_addr), + }, + /* next input field (IPv4 destination address) - 4 consecutive bytes. */ + { + /* rte_flow uses a bit mask for IPv4 addresses */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint32_t), + .field_index = DST_FIELD_IPV4, + .input_index = DST_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, dst_addr), + }, + /* + * Next 2 fields (src & dst ports) form 4 consecutive bytes. + * They share the same input index. + */ + { + /* rte_flow uses a bit mask for protocol ports */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint16_t), + .field_index = SRCP_FIELD_IPV4, + .input_index = SRCP_DESTP_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + sizeof(struct ipv4_hdr) + + offsetof(struct tcp_hdr, src_port), + }, + { + /* rte_flow uses a bit mask for protocol ports */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint16_t), + .field_index = DSTP_FIELD_IPV4, + .input_index = SRCP_DESTP_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + sizeof(struct ipv4_hdr) + + offsetof(struct tcp_hdr, dst_port), + }, + }; + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + +It then parses the flow_classify application arguments + +.. code-block:: c + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid flow_classify parameters\n"); + +The ``main()`` function also allocates a mempool to hold the mbufs +(Message Buffers) used by the application: + +.. code-block:: c + + mbuf_pool = rte_mempool_create("MBUF_POOL", + NUM_MBUFS * nb_ports, + MBUF_SIZE, + MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + rte_socket_id(), + 0); + +mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``port_init()`` function which is explained in the next section: + +.. code-block:: c + + RTE_ETH_FOREACH_DEV(portid) { + if (port_init(portid, mbuf_pool) != 0) { + rte_exit(EXIT_FAILURE, + "Cannot init port %" PRIu8 "\n", portid); + } + } + +The ``main()`` function creates the ``flow classifier object`` and adds an ``ACL +table`` to the flow classifier. + +.. code-block:: c + + struct flow_classifier { + struct rte_flow_classifier *cls; + }; + + struct flow_classifier_acl { + struct flow_classifier cls; + } __rte_cache_aligned; + + /* Memory allocation */ + size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct flow_classifier_acl)); + cls_app = rte_zmalloc(NULL, size, RTE_CACHE_LINE_SIZE); + if (cls_app == NULL) + rte_exit(EXIT_FAILURE, "Cannot allocate classifier memory\n"); + + cls_params.name = "flow_classifier"; + cls_params.socket_id = socket_id; + + cls_app->cls = rte_flow_classifier_create(&cls_params); + if (cls_app->cls == NULL) { + rte_free(cls_app); + rte_exit(EXIT_FAILURE, "Cannot create classifier\n"); + } + + /* initialise ACL table params */ + table_acl_params.name = "table_acl_ipv4_5tuple"; + table_acl_params.n_rule_fields = RTE_DIM(ipv4_defs); + table_acl_params.n_rules = FLOW_CLASSIFY_MAX_RULE_NUM; + memcpy(table_acl_params.field_format, ipv4_defs, sizeof(ipv4_defs)); + + /* initialise table create params */ + cls_table_params.ops = &rte_table_acl_ops, + cls_table_params.arg_create = &table_acl_params, + cls_table_params.type = RTE_FLOW_CLASSIFY_TABLE_ACL_IP4_5TUPLE; + + ret = rte_flow_classify_table_create(cls_app->cls, &cls_table_params); + if (ret) { + rte_flow_classifier_free(cls_app->cls); + rte_free(cls); + rte_exit(EXIT_FAILURE, "Failed to create classifier table\n"); + } + +It then reads the ipv4_rules_file.txt file and initialises the parameters for +the ``rte_flow_classify_table_entry_add`` API. +This API adds a rule to the ACL table. + +.. code-block:: c + + if (add_rules(parm_config.rule_ipv4_name)) { + rte_flow_classifier_free(cls_app->cls); + rte_free(cls_app); + rte_exit(EXIT_FAILURE, "Failed to add rules\n"); + } + +Once the initialization is complete, the application is ready to launch a +function on an lcore. In this example ``lcore_main()`` is called on a single +lcore. + +.. code-block:: c + + lcore_main(cls_app); + +The ``lcore_main()`` function is explained below. + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization used in the Basic +Forwarding application is shown below: + +.. code-block:: c + + static inline int + port_init(uint8_t port, struct rte_mempool *mbuf_pool) + { + struct rte_eth_conf port_conf = port_conf_default; + const uint16_t rx_rings = 1, tx_rings = 1; + struct ether_addr addr; + int retval; + uint16_t q; + + /* Configure the Ethernet device. */ + retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf); + if (retval != 0) + return retval; + + /* Allocate and set up 1 RX queue per Ethernet port. */ + for (q = 0; q < rx_rings; q++) { + retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL, mbuf_pool); + if (retval < 0) + return retval; + } + + /* Allocate and set up 1 TX queue per Ethernet port. */ + for (q = 0; q < tx_rings; q++) { + retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL); + if (retval < 0) + return retval; + } + + /* Start the Ethernet port. */ + retval = rte_eth_dev_start(port); + if (retval < 0) + return retval; + + /* Display the port MAC address. */ + rte_eth_macaddr_get(port, &addr); + printf("Port %u MAC: %02" PRIx8 " %02" PRIx8 " %02" PRIx8 + " %02" PRIx8 " %02" PRIx8 " %02" PRIx8 "\n", + port, + addr.addr_bytes[0], addr.addr_bytes[1], + addr.addr_bytes[2], addr.addr_bytes[3], + addr.addr_bytes[4], addr.addr_bytes[5]); + + /* Enable RX in promiscuous mode for the Ethernet device. */ + rte_eth_promiscuous_enable(port); + + return 0; + } + +The Ethernet ports are configured with default settings using the +``rte_eth_dev_configure()`` function and the ``port_conf_default`` struct. + +.. code-block:: c + + static const struct rte_eth_conf port_conf_default = { + .rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN } + }; + +For this example the ports are set up with 1 RX and 1 TX queue using the +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + retval = rte_eth_dev_start(port); + + +Finally the RX port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(port); + +The Add Rules function +~~~~~~~~~~~~~~~~~~~~~~ + +The ``add_rules`` function reads the ``ipv4_rules_file.txt`` file and calls the +``add_classify_rule`` function which calls the +``rte_flow_classify_table_entry_add`` API. + +.. code-block:: c + + static int + add_rules(const char *rule_path) + { + FILE *fh; + char buff[LINE_MAX]; + unsigned int i = 0; + unsigned int total_num = 0; + struct rte_eth_ntuple_filter ntuple_filter; + + fh = fopen(rule_path, "rb"); + if (fh == NULL) + rte_exit(EXIT_FAILURE, "%s: Open %s failed\n", __func__, + rule_path); + + fseek(fh, 0, SEEK_SET); + + i = 0; + while (fgets(buff, LINE_MAX, fh) != NULL) { + i++; + + if (is_bypass_line(buff)) + continue; + + if (total_num >= FLOW_CLASSIFY_MAX_RULE_NUM - 1) { + printf("\nINFO: classify rule capacity %d reached\n", + total_num); + break; + } + + if (parse_ipv4_5tuple_rule(buff, &ntuple_filter) != 0) + rte_exit(EXIT_FAILURE, + "%s Line %u: parse rules error\n", + rule_path, i); + + if (add_classify_rule(&ntuple_filter) != 0) + rte_exit(EXIT_FAILURE, "add rule error\n"); + + total_num++; + } + + fclose(fh); + return 0; + } + + +The Lcore Main function +~~~~~~~~~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function on the +available lcores. +The ``lcore_main`` function calls the ``rte_flow_classifier_query`` API. +For the Basic Forwarding application the ``lcore_main`` function looks like the +following: + +.. code-block:: c + + /* flow classify data */ + static int num_classify_rules; + static struct rte_flow_classify_rule *rules[MAX_NUM_CLASSIFY]; + static struct rte_flow_classify_ipv4_5tuple_stats ntuple_stats; + static struct rte_flow_classify_stats classify_stats = { + .stats = (void *)&ntuple_stats + }; + + static __attribute__((noreturn)) void + lcore_main(cls_app) + { + uint16_t port; + + /* + * Check that the port is on the same NUMA node as the polling thread + * for best performance. + */ + RTE_ETH_FOREACH_DEV(port) + if (rte_eth_dev_socket_id(port) > 0 && + rte_eth_dev_socket_id(port) != (int)rte_socket_id()) { + printf("\n\n"); + printf("WARNING: port %u is on remote NUMA node\n", + port); + printf("to polling thread.\n"); + printf("Performance will not be optimal.\n"); + + printf("\nCore %u forwarding packets. \n", + rte_lcore_id()); + printf("[Ctrl+C to quit]\n + } + + /* Run until the application is quit or killed. */ + for (;;) { + /* + * Receive packets on a port and forward them on the paired + * port. The mapping is 0 -> 1, 1 -> 0, 2 -> 3, 3 -> 2, etc. + */ + RTE_ETH_FOREACH_DEV(port) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + for (i = 0; i < MAX_NUM_CLASSIFY; i++) { + if (rules[i]) { + ret = rte_flow_classifier_query( + cls_app->cls, + bufs, nb_rx, rules[i], + &classify_stats); + if (ret) + printf( + "rule [%d] query failed ret [%d]\n\n", + i, ret); + else { + printf( + "rule[%d] count=%"PRIu64"\n", + i, ntuple_stats.counter1); + + printf("proto = %d\n", + ntuple_stats.ipv4_5tuple.proto); + } + } + } + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + } + +The main work of the application is done within the loop: + +.. code-block:: c + + for (;;) { + RTE_ETH_FOREACH_DEV(port) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + +Packets are received in bursts on the RX ports and transmitted in bursts on +the TX ports. The ports are grouped in pairs with a simple mapping scheme +using the an XOR on the port number:: + + 0 -> 1 + 1 -> 0 + + 2 -> 3 + 3 -> 2 + + etc. + +The ``rte_eth_tx_burst()`` function frees the memory buffers of packets that +are transmitted. If packets fail to transmit, ``(nb_tx < nb_rx)``, then they +must be freed explicitly using ``rte_pktmbuf_free()``. + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/flow_filtering.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/flow_filtering.rst new file mode 100644 index 00000000..bd0ae1e2 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/flow_filtering.rst @@ -0,0 +1,549 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2017 Mellanox Technologies, Ltd + +Basic RTE Flow Filtering Sample Application +=========================================== + +The Basic RTE flow filtering sample application is a simple example of a +creating a RTE flow rule. + +It is intended as a demonstration of the basic components RTE flow rules. + + +Compiling the Application +------------------------- + +To compile the application export the path to the DPDK source tree and go to +the example directory: + +.. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + + cd ${RTE_SDK}/examples/flow_filtering + +Set the target, for example: + +.. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + +See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. + +Build the application as follows: + +.. code-block:: console + + make + + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + ./build/flow -l 1 -n 1 + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + + +Explanation +----------- + +The example is build from 2 main files, +``main.c`` which holds the example logic and ``flow_blocks.c`` that holds the +implementation for building the flow rule. + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function located in ``main.c`` file performs the initialization +and runs the main loop function. + +The first task is to initialize the Environment Abstraction Layer (EAL). The +``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", 4096, 128, 0, + RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``init_port()`` function which is explained in the next section: + +.. code-block:: c + + init_port(); + +Once the initialization is complete, we set the flow rule using the +following code: + +.. code-block:: c + + /* create flow for send packet with */ + flow = generate_ipv4_flow(port_id, selected_queue, + SRC_IP, EMPTY_MASK, + DEST_IP, FULL_MASK, &error); + if (!flow) { + printf("Flow can't be created %d message: %s\n", + error.type, + error.message ? error.message : "(no stated reason)"); + rte_exit(EXIT_FAILURE, "error in creating flow"); + } + +In the last part the application is ready to launch the +``main_loop()`` function. Which is explained below. + + +.. code-block:: c + + main_loop(); + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization used in the flow filtering +application is shown below: + +.. code-block:: c + + init_port(void) + { + int ret; + uint16_t i; + struct rte_eth_conf port_conf = { + .rxmode = { + .split_hdr_size = 0, + .offloads = DEV_RX_OFFLOAD_CRC_STRIP, + }, + .txmode = { + .offloads = + DEV_TX_OFFLOAD_VLAN_INSERT | + DEV_TX_OFFLOAD_IPV4_CKSUM | + DEV_TX_OFFLOAD_UDP_CKSUM | + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_SCTP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO, + }, + }; + struct rte_eth_txconf txq_conf; + struct rte_eth_rxconf rxq_conf; + struct rte_eth_dev_info dev_info; + + printf(":: initializing port: %d\n", port_id); + ret = rte_eth_dev_configure(port_id, + nr_queues, nr_queues, &port_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: cannot configure device: err=%d, port=%u\n", + ret, port_id); + } + + rte_eth_dev_info_get(port_id, &dev_info); + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = port_conf.rxmode.offloads; + /* only set Rx queues: something we care only so far */ + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_rx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + &rxq_conf, + mbuf_pool); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Rx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + + txq_conf = dev_info.default_txconf; + txq_conf.offloads = port_conf.txmode.offloads; + + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_tx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + &txq_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Tx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + + rte_eth_promiscuous_enable(port_id); + ret = rte_eth_dev_start(port_id); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start:err=%d, port=%u\n", + ret, port_id); + } + + assert_link_status(); + + printf(":: initializing port: %d done\n", port_id); + } + +The Ethernet port is configured with default settings using the +``rte_eth_dev_configure()`` function and the ``port_conf_default`` struct: + +.. code-block:: c + + struct rte_eth_conf port_conf = { + .rxmode = { + .split_hdr_size = 0, + .offloads = DEV_RX_OFFLOAD_CRC_STRIP, + }, + .txmode = { + .offloads = + DEV_TX_OFFLOAD_VLAN_INSERT | + DEV_TX_OFFLOAD_IPV4_CKSUM | + DEV_TX_OFFLOAD_UDP_CKSUM | + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_SCTP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO, + }, + }; + + ret = rte_eth_dev_configure(port_id, nr_queues, nr_queues, &port_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: cannot configure device: err=%d, port=%u\n", + ret, port_id); + } + rte_eth_dev_info_get(port_id, &dev_info); + rxq_conf = dev_info.default_rxconf; + rxq_conf.offloads = port_conf.rxmode.offloads; + +For this example we are configuring number of rx and tx queues that are connected +to a single port. + +.. code-block:: c + + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_rx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + &rxq_conf, + mbuf_pool); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Rx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_tx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + &txq_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Tx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + +In the next step we create and apply the flow rule. which is to send packets +with destination ip equals to 192.168.1.1 to queue number 1. The detail +explanation of the ``generate_ipv4_flow()`` appears later in this document: + +.. code-block:: c + + flow = generate_ipv4_flow(port_id, selected_queue, + SRC_IP, EMPTY_MASK, + DEST_IP, FULL_MASK, &error); + +We are setting the RX port to promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(port_id); + +The last step is to start the port. + +.. code-block:: c + + ret = rte_eth_dev_start(port_id); + if (ret < 0) { + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err%d, port=%u\n", + ret, port_id); + } + + +The main_loop function +~~~~~~~~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function to handle +the main loop. For the flow filtering application the main_loop function +looks like the following: + +.. code-block:: c + + static void + main_loop(void) + { + struct rte_mbuf *mbufs[32]; + struct ether_hdr *eth_hdr; + uint16_t nb_rx; + uint16_t i; + uint16_t j; + + while (!force_quit) { + for (i = 0; i < nr_queues; i++) { + nb_rx = rte_eth_rx_burst(port_id, + i, mbufs, 32); + if (nb_rx) { + for (j = 0; j < nb_rx; j++) { + struct rte_mbuf *m = mbufs[j]; + + eth_hdr = rte_pktmbuf_mtod(m, + struct ether_hdr *); + print_ether_addr("src=", + ð_hdr->s_addr); + print_ether_addr(" - dst=", + ð_hdr->d_addr); + printf(" - queue=0x%x", + (unsigned int)i); + printf("\n"); + rte_pktmbuf_free(m); + } + } + } + } + /* closing and releasing resources */ + rte_flow_flush(port_id, &error); + rte_eth_dev_stop(port_id); + rte_eth_dev_close(port_id); + } + +The main work of the application is reading the packets from all +queues and printing for each packet the destination queue: + +.. code-block:: c + + while (!force_quit) { + for (i = 0; i < nr_queues; i++) { + nb_rx = rte_eth_rx_burst(port_id, i, mbufs, 32); + if (nb_rx) { + for (j = 0; j < nb_rx; j++) { + struct rte_mbuf *m = mbufs[j]; + eth_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); + print_ether_addr("src=", ð_hdr->s_addr); + print_ether_addr(" - dst=", ð_hdr->d_addr); + printf(" - queue=0x%x", (unsigned int)i); + printf("\n"); + rte_pktmbuf_free(m); + } + } + } + } + + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. Which results in closing the port and the device using +``rte_eth_dev_stop`` and ``rte_eth_dev_close`` + +The generate_ipv4_flow function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The generate_ipv4_rule function is responsible for creating the flow rule. +This function is located in the ``flow_blocks.c`` file. + +.. code-block:: c + + static struct rte_flow * + generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error) + { + struct rte_flow_attr attr; + struct rte_flow_item pattern[MAX_PATTERN_NUM]; + struct rte_flow_action action[MAX_PATTERN_NUM]; + struct rte_flow *flow = NULL; + struct rte_flow_action_queue queue = { .index = rx_q }; + struct rte_flow_item_eth eth_spec; + struct rte_flow_item_eth eth_mask; + struct rte_flow_item_vlan vlan_spec; + struct rte_flow_item_vlan vlan_mask; + struct rte_flow_item_ipv4 ip_spec; + struct rte_flow_item_ipv4 ip_mask; + + memset(pattern, 0, sizeof(pattern)); + memset(action, 0, sizeof(action)); + + /* + * set the rule attribute. + * in this case only ingress packets will be checked. + */ + memset(&attr, 0, sizeof(struct rte_flow_attr)); + attr.ingress = 1; + + /* + * create the action sequence. + * one action only, move packet to queue + */ + + action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; + action[0].conf = &queue; + action[1].type = RTE_FLOW_ACTION_TYPE_END; + + /* + * set the first level of the pattern (eth). + * since in this example we just want to get the + * ipv4 we set this level to allow all. + */ + memset(ð_spec, 0, sizeof(struct rte_flow_item_eth)); + memset(ð_mask, 0, sizeof(struct rte_flow_item_eth)); + eth_spec.type = 0; + eth_mask.type = 0; + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð_spec; + pattern[0].mask = ð_mask; + + /* + * setting the second level of the pattern (vlan). + * since in this example we just want to get the + * ipv4 we also set this level to allow all. + */ + memset(&vlan_spec, 0, sizeof(struct rte_flow_item_vlan)); + memset(&vlan_mask, 0, sizeof(struct rte_flow_item_vlan)); + pattern[1].type = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan_spec; + pattern[1].mask = &vlan_mask; + + /* + * setting the third level of the pattern (ip). + * in this example this is the level we care about + * so we set it according to the parameters. + */ + memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4)); + memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4)); + ip_spec.hdr.dst_addr = htonl(dest_ip); + ip_mask.hdr.dst_addr = dest_mask; + ip_spec.hdr.src_addr = htonl(src_ip); + ip_mask.hdr.src_addr = src_mask; + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ip_spec; + pattern[2].mask = &ip_mask; + + /* the final level must be always type end */ + pattern[3].type = RTE_FLOW_ITEM_TYPE_END; + + int res = rte_flow_validate(port_id, &attr, pattern, action, error); + if(!res) + flow = rte_flow_create(port_id, &attr, pattern, action, error); + + return flow; + } + +The first part of the function is declaring the structures that will be used. + +.. code-block:: c + + struct rte_flow_attr attr; + struct rte_flow_item pattern[MAX_PATTERN_NUM]; + struct rte_flow_action action[MAX_PATTERN_NUM]; + struct rte_flow *flow; + struct rte_flow_error error; + struct rte_flow_action_queue queue = { .index = rx_q }; + struct rte_flow_item_eth eth_spec; + struct rte_flow_item_eth eth_mask; + struct rte_flow_item_vlan vlan_spec; + struct rte_flow_item_vlan vlan_mask; + struct rte_flow_item_ipv4 ip_spec; + struct rte_flow_item_ipv4 ip_mask; + +The following part create the flow attributes, in our case ingress. + +.. code-block:: c + + memset(&attr, 0, sizeof(struct rte_flow_attr)); + attr.ingress = 1; + +The third part defines the action to be taken when a packet matches +the rule. In this case send the packet to queue. + +.. code-block:: c + + action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; + action[0].conf = &queue; + action[1].type = RTE_FLOW_ACTION_TYPE_END; + +The forth part is responsible for creating the pattern and is build from +number of step. In each step we build one level of the pattern starting with +the lowest one. + +Setting the first level of the pattern ETH: + +.. code-block:: c + + memset(ð_spec, 0, sizeof(struct rte_flow_item_eth)); + memset(ð_mask, 0, sizeof(struct rte_flow_item_eth)); + eth_spec.type = 0; + eth_mask.type = 0; + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð_spec; + pattern[0].mask = ð_mask; + +Setting the second level of the pattern VLAN: + +.. code-block:: c + + memset(&vlan_spec, 0, sizeof(struct rte_flow_item_vlan)); + memset(&vlan_mask, 0, sizeof(struct rte_flow_item_vlan)); + pattern[1].type = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan_spec; + pattern[1].mask = &vlan_mask; + +Setting the third level ip: + +.. code-block:: c + + memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4)); + memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4)); + ip_spec.hdr.dst_addr = htonl(dest_ip); + ip_mask.hdr.dst_addr = dest_mask; + ip_spec.hdr.src_addr = htonl(src_ip); + ip_mask.hdr.src_addr = src_mask; + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ip_spec; + pattern[2].mask = &ip_mask; + +Closing the pattern part. + +.. code-block:: c + + pattern[3].type = RTE_FLOW_ITEM_TYPE_END; + +The last part of the function is to validate the rule and create it. + +.. code-block:: c + + int res = rte_flow_validate(port_id, &attr, pattern, action, &error); + if (!res) + flow = rte_flow_create(port_id, &attr, pattern, action, &error); + diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/hello_world.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/hello_world.rst new file mode 100644 index 00000000..a2051f79 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/hello_world.rst @@ -0,0 +1,94 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Hello World Sample Application +============================== + +The Hello World sample application is an example of the simplest DPDK application that can be written. +The application simply prints an "helloworld" message on every enabled lcore. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``helloworld`` sub-directory. + +Running the Application +----------------------- + +To run the example in a linuxapp environment: + +.. code-block:: console + + $ ./build/helloworld -l 0-3 -n 4 + +Refer to *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of code. + +EAL Initialization +~~~~~~~~~~~~~~~~~~ + +The first task is to initialize the Environment Abstraction Layer (EAL). +This is done in the main() function using the following code: + +.. code-block:: c + + int + + main(int argc, char **argv) + + { + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + +This call finishes the initialization process that was started before main() is called (in case of a Linuxapp environment). +The argc and argv arguments are provided to the rte_eal_init() function. +The value returned is the number of parsed arguments. + +Starting Application Unit Lcores +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the EAL is initialized, the application is ready to launch a function on an lcore. +In this example, lcore_hello() is called on every available lcore. +The following is the definition of the function: + +.. code-block:: c + + static int + lcore_hello( attribute ((unused)) void *arg) + { + unsigned lcore_id; + + lcore_id = rte_lcore_id(); + printf("hello from core %u\n", lcore_id); + return 0; + } + +The code that launches the function on each lcore is as follows: + +.. code-block:: c + + /* call lcore_hello() on every slave lcore */ + + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + rte_eal_remote_launch(lcore_hello, NULL, lcore_id); + } + + /* call it on master lcore too */ + + lcore_hello(NULL); + +The following code is equivalent and simpler: + +.. code-block:: c + + rte_eal_mp_remote_launch(lcore_hello, NULL, CALL_MASTER); + +Refer to the *DPDK API Reference* for detailed information on the rte_eal_mp_remote_launch() function. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/client_svr_sym_multi_proc_app.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/client_svr_sym_multi_proc_app.png Binary files differnew file mode 100644 index 00000000..abd3ef34 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/client_svr_sym_multi_proc_app.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_app.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_app.svg new file mode 100644 index 00000000..f19ad9b7 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_app.svg @@ -0,0 +1,474 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2014-2017 Intel Corporation --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="526.94379" + height="379.53668" + id="svg4090" + version="1.1" + inkscape:version="0.92.1 r15371" + sodipodi:docname="dist_app.svg"> + <defs + id="defs4092"> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path10501" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4017" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4019" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4021" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4023" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4025" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4027" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4029" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4031" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4033" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4035" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path10498" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4039" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4041" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4043" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4045" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4047" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4049" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1" + inkscape:cx="401.32873" + inkscape:cy="130.13572" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:window-width="1920" + inkscape:window-height="1137" + inkscape:window-x="1912" + inkscape:window-y="-8" + inkscape:window-maximized="1" /> + <metadata + id="metadata4095"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-35.078263,-28.308125)"> + <rect + style="fill:none;stroke:#000000;stroke-width:0.81890059;stroke-opacity:0.98412697" + id="rect10443" + width="152.96732" + height="178.99617" + x="124.50176" + y="128.95552" /> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-2" + width="124.71397" + height="46.675529" + x="437.00507" + y="133.06113" /> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-0" + width="124.71397" + height="46.675529" + x="436.80811" + y="193.87207" /> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-9" + width="124.71397" + height="46.675529" + x="436.80811" + y="256.06277" /> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-7" + width="124.71397" + height="46.675529" + x="135.7057" + y="360.66928" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99566948;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Mstart)" + d="M 278.89497,147.51907 436.5713,146.78234" + id="path10486-2" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99290925;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Mstart)" + d="m 279.37092,206.8834 156.80331,-0.73671" + id="path10486-1" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99379504;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Mstart)" + d="m 279.19738,270.88669 157.15478,-0.73638" + id="path10486-4" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99820405;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 277.17846,166.20347 158.11878,-0.73842" + id="path10486-0-4" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99410033;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 277.47049,225.92925 157.32298,-0.73606" + id="path10486-0-7" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99566948;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="M 277.70474,289.26714 435.38107,288.5304" + id="path10486-0-77" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="345.02322" + y="134.82103" + id="text11995" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997" + x="345.02322" + y="134.82103" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Request burst</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="346.38663" + y="164.76628" + id="text11995-7" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3" + x="346.38663" + y="164.76628" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Mbuf Pointers</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="502.36844" + y="151.66222" + id="text11995-7-3" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5" + x="502.36844" + y="151.66222" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">WorkerThread1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="499.40103" + y="207.94502" + id="text11995-7-3-9" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-9" + x="499.40103" + y="207.94502" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">WorkerThread2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="500.1918" + y="266.59644" + id="text11995-7-3-82" + transform="scale(0.9399234,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-6" + x="500.1918" + y="266.59644" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">WorkerThreadN</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="193.79703" + y="362.85193" + id="text11995-7-3-6" + transform="scale(0.93992342,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-0" + x="193.79703" + y="362.85193" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">TX thread</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="175.78905" + y="207.26257" + id="text11995-7-3-3" + transform="scale(0.9399234,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-8" + x="175.78905" + y="207.26257" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Distributor Thread</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:0.75945646;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 49.600127,54.625621 85.546363,-0.79004" + id="path10486-0-4-5" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.75945646;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 135.70569,384.00706 -85.546361,0.79003" + id="path10486-0-4-5-7" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="73.342712" + y="44.196564" + id="text11995-7-8" + transform="scale(0.9399234,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-3" + x="73.342712" + y="44.196564" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Mbufs In</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="83.4814" + y="352.62543" + id="text11995-7-8-5" + transform="scale(0.93992342,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-3-1" + x="83.4814" + y="352.62543" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Mbufs Out</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1.01068497;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:0.98412697" + d="m 171.68192,308.06701 0.21464,27.84908 -8.6322,0.37082 -11.33877,0.17876 25.75778,13.51792 23.25799,10.16096 18.87014,-6.69841 31.83305,-15.77889 -10.75831,-0.30147 -10.37586,-0.40509 -0.22443,-28.8261 z" + id="path12188" + inkscape:connector-curvature="0" + inkscape:transform-center-y="7.0247597" + sodipodi:nodetypes="cccccccccccc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="193.68871" + y="309.26349" + id="text11995-7-3-6-2" + transform="scale(0.93992342,1.0639165)"><tspan + sodipodi:role="line" + x="193.68871" + y="309.26349" + id="tspan12214" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">SW Ring</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1.02106845;stroke-miterlimit:3;stroke-dasharray:none;stroke-opacity:0.98412697" + d="m 173.27214,75.568236 0.21464,28.424254 -8.6322,0.37848 -11.33877,0.18245 25.75778,13.79709 23.25799,10.37083 18.87013,-6.83675 31.83305,-16.10478 -10.75831,-0.30769 -10.37586,-0.41345 -0.22443,-29.421453 z" + id="path12188-5" + inkscape:connector-curvature="0" + inkscape:transform-center-y="7.1698404" + sodipodi:nodetypes="cccccccccccc" /> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-7-7" + width="124.71397" + height="46.675529" + x="138.18427" + y="28.832333" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="190.80019" + y="51.17778" + id="text11995-7-3-6-6" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-0-4" + x="190.80019" + y="51.17778" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">RX thread</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="196.38097" + y="90.224785" + id="text11995-7-3-6-2-9" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + x="196.38097" + y="90.224785" + id="tspan12214-8" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">SW Ring</tspan></text> + <rect + style="fill:none;stroke:#000000;stroke-width:0.99999988;stroke-opacity:0.98412697" + id="rect10445-7-7-5" + width="124.71397" + height="46.675529" + x="327.86566" + y="29.009106" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="387.27209" + y="45.36227" + id="text11995-7-3-6-6-3" + transform="scale(0.93992339,1.0639165)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-0-4-4" + x="387.27209" + y="45.36227" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif">Stats thread</tspan><tspan + sodipodi:role="line" + x="387.27209" + y="57.016178" + style="font-size:9.32312489px;line-height:1.25;font-family:sans-serif" + id="tspan165">(to console)</tspan></text> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_perf.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_perf.svg new file mode 100644 index 00000000..d5b2a2be --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/dist_perf.svg @@ -0,0 +1,462 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2014 Intel Corporation --> + + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="524.65625" + height="387.59375" + id="svg4116" + version="1.1" + inkscape:version="0.48.5 r10040" + sodipodi:docname="New document 2"> + <defs + id="defs4118"> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path10498" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path10501" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4038" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4040" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4042" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4044" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4046" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4048" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4050" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4052" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4054" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4056" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4058" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4060" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4062" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4064" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4066" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4068" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker4070" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4072" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="marker4074" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path4076" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.4,0,0,0.4,4,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1" + inkscape:cx="354.46875" + inkscape:cy="78.904643" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:window-width="1920" + inkscape:window-height="1017" + inkscape:window-x="-8" + inkscape:window-y="-8" + inkscape:window-maximized="1" /> + <metadata + id="metadata4121"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-20.53125,-22.84375)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-opacity:0.98412697" + id="rect10443" + width="165.52779" + height="376.84436" + x="21.023544" + y="24.286175" /> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-opacity:0.98412697" + id="rect10445" + width="156.95697" + height="386.59042" + x="387.73376" + y="23.352676" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:url(#Arrow1Mstart)" + d="m 188.27904,66.970932 195.99264,0.833121" + id="path10486" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="M 188.05945,91.53983 384.0521,90.566545" + id="path10486-0" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="460.4104" + y="292.91855" + id="text11995" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + x="460.4104" + y="292.91855" + id="tspan12218">Port2</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="459.06958" + y="59.738571" + id="text11995-7" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + id="tspan11997-3" + x="459.06958" + y="59.738571">Port0</tspan></text> + <text + xml:space="preserve" + style="font-size:15.28272438px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="474.06027" + y="184.77933" + id="text11995-7-3" + sodipodi:linespacing="125%" + transform="scale(0.8986678,1.1127582)"><tspan + sodipodi:role="line" + id="tspan11997-3-5" + x="474.06027" + y="184.77933">DPDK board</tspan></text> + <text + xml:space="preserve" + style="font-size:16.0002594px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="54.009655" + y="171.28656" + id="text11995-7-3-3" + sodipodi:linespacing="125%" + transform="scale(0.81894062,1.2210897)"><tspan + sodipodi:role="line" + id="tspan11997-3-5-8" + x="54.009655" + y="171.28656">Traffic Generator</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="459.46732" + y="91.195976" + id="text11995-7-8" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + id="tspan11997-3-3" + x="459.46732" + y="91.195976">Port1</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="460.15295" + y="326.05963" + id="text11995-7-3-6-2" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + x="460.15295" + y="326.05963" + id="tspan12214">Port3</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99199999, 1.98399994;stroke-dashoffset:0;marker-start:url(#Arrow1Mstart);display:inline" + d="m 389.46894,68.26125 12.5232,0 c 1.5,0 3,1.5 3,3 l 0,21.198419" + id="path10486-06" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="3" + sodipodi:nodetypes="ccc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99999997, 1.99999998;stroke-dashoffset:0" + d="m 398.43415,91.043274 -11.52714,0 0.98804,0" + id="path12267" + inkscape:connector-curvature="0" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:url(#Arrow1Mstart)" + d="m 190.33387,103.00575 195.99267,0.97328" + id="path10486-43" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="M 187.50304,56.857383 383.49569,55.884111" + id="path10486-0-9" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99199997, 1.98399998;stroke-dashoffset:0;marker-start:url(#Arrow1Mstart);display:inline" + d="m 388.42391,103.27876 27.61666,0 c 1.5,0 3,-1.5 3,-3 l 0,-41.462569" + id="path10486-06-7" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="3" + sodipodi:nodetypes="ccc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99999997, 1.99999994;stroke-dashoffset:0" + d="m 417.31173,56.402625 -26.65144,0 2.2844,0" + id="path12267-7" + inkscape:connector-curvature="0" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:url(#Arrow1Mstart)" + d="m 186.54351,319.92933 195.99264,0.83313" + id="path10486-07" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 185.45351,344.49822 195.99262,-0.97328" + id="path10486-0-3" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99199997, 1.98399993;stroke-dashoffset:0;marker-start:url(#Arrow1Mstart);display:inline" + d="m 387.7334,321.21965 12.52321,0 c 1.5,0 3,1.5 3,3 l 0,21.19843" + id="path10486-06-4" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="3" + sodipodi:nodetypes="ccc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99999997, 1.99999994;stroke-dashoffset:0" + d="m 396.69862,344.00166 -11.52714,0 0.98804,0" + id="path12267-9" + inkscape:connector-curvature="0" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:url(#Arrow1Mstart)" + d="m 190.33918,355.96416 195.99266,0.97327" + id="path10486-43-6" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-mid:none;marker-end:url(#Arrow1Mend)" + d="m 185.76751,309.8158 195.99266,-0.97331" + id="path10486-0-9-5" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.99199992;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99199997, 1.98399994;stroke-dashoffset:0;marker-start:url(#Arrow1Mstart);display:inline" + d="m 386.68838,356.23716 27.61666,0 c 1.5,0 3,-1.5 3,-3 l 0,-41.46255" + id="path10486-06-7-1" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="3" + sodipodi:nodetypes="ccc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:3;stroke-opacity:1;stroke-dasharray:0.99999995, 1.99999991;stroke-dashoffset:0" + d="m 415.57618,309.36103 -26.65143,0 2.28441,0" + id="path12267-7-0" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="168.01332" + y="295.95398" + id="text11995-9" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + x="168.01332" + y="295.95398" + id="tspan12218-9">Port2</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="166.67253" + y="62.774006" + id="text11995-7-6" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + id="tspan11997-3-8" + x="166.67253" + y="62.774006">Port0</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="167.07025" + y="94.231415" + id="text11995-7-8-3" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + id="tspan11997-3-3-4" + x="167.07025" + y="94.231415">Port1</tspan></text> + <text + xml:space="preserve" + style="font-size:11.9913578px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="167.75591" + y="329.09506" + id="text11995-7-3-6-2-8" + sodipodi:linespacing="125%" + transform="scale(0.91971036,1.0872988)"><tspan + sodipodi:role="line" + x="167.75591" + y="329.09506" + id="tspan12214-4">Port3</tspan></text> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/example_rules.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/example_rules.png Binary files differnew file mode 100644 index 00000000..4019b7df --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/example_rules.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/exception_path_example.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/exception_path_example.svg new file mode 100644 index 00000000..c19c5808 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/exception_path_example.svg @@ -0,0 +1,102 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2014 Intel Corporation --> + +<svg xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns="http://www.w3.org/2000/svg" height="330.16" width="568.88" version="1.0" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/"> + <g transform="matrix(0.136 0 0 .13439 -6.7377 -69.183)"> + <path d="m3004.4 514.8v2456.8h1228v-2456.8h-1228z" fill-rule="evenodd" fill="#fcd5b5"/> + <path d="m1359.6 514.8v2456.8h1644.8v-2456.8h-1644.8z" fill-rule="evenodd" fill="#d9d9d9"/> + <path d="m1096 1460v188.8h415.2v-188.8h-415.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m1089.2 1460c0-3.7 3.1-6.8 6.8-6.8h415.2c3.8 0 6.8 3.1 6.8 6.8v188.8c0 3.8-3 6.8-6.8 6.8h-415.2c-3.7 0-6.8-3-6.8-6.8zm13.6 188.8-6.8-6.8h415.2l-6.8 6.8v-188.8l6.8 6.8h-415.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1594.3033" x="1175.8816" font-family="Arial" fill="#ffffff">Port0</text> + <path d="m1096 1762.4v188.8h415.2v-188.8h-415.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m1089.2 1762.4c0-3.7 3.1-6.8 6.8-6.8h415.2c3.8 0 6.8 3.1 6.8 6.8v188.8c0 3.8-3 6.8-6.8 6.8h-415.2c-3.7 0-6.8-3-6.8-6.8zm13.6 188.8-6.8-6.8h415.2l-6.8 6.8v-188.8l6.8 6.8h-415.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1896.6992" x="1175.8816" font-family="Arial" fill="#ffffff">Port1</text> + <path d="m1096 2177.6v189.6h415.2v-189.6h-415.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m1089.2 2177.6c0-3.7 3.1-6.8 6.8-6.8h415.2c3.8 0 6.8 3.1 6.8 6.8v189.6c0 3.8-3 6.8-6.8 6.8h-415.2c-3.7 0-6.8-3-6.8-6.8zm13.6 189.6-6.8-6.8h415.2l-6.8 6.8v-189.6l6.8 6.8h-415.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2312.4934" x="1167.0984" font-family="Arial" fill="#ffffff">PortN</text> + <path d="m2060 892.8v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 892.8c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1008.4115" x="2124.8179" font-family="Arial" fill="#ffffff">CoreA0</text> + <path d="m2060 1157.6v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 1157.6c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1273.0079" x="2124.8179" font-family="Arial" fill="#ffffff">CoreA1</text> + <path d="m2060 1535.2v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 1535.2c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1651.0026" x="2116.0015" font-family="Arial" fill="#ffffff">CoreAN</text> + <path d="m2060 2026.4v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 2026.4c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2142.3958" x="2124.8179" font-family="Arial" fill="#ffffff">CoreB0</text> + <path d="m2060 2291.2v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 2291.2c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2406.9922" x="2124.8179" font-family="Arial" fill="#ffffff">CoreB1</text> + <path d="m2060 2669.6v151.2h491.2v-151.2h-491.2z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2053.2 2669.6c0-3.7 3.1-6.8 6.8-6.8h491.2c3.8 0 6.8 3.1 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-491.2c-3.7 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h491.2l-6.8 6.8v-151.2l6.8 6.8h-491.2l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2784.9868" x="2116.0015" font-family="Arial" fill="#ffffff">CoreBN</text> + <path d="m2834.4 892.8v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 892.8c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1008.4115" x="2921.5908" font-family="Arial" fill="#ffffff">tapA0</text> + <path d="m2834.4 1157.6v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 1157.6c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1273.0079" x="2921.5908" font-family="Arial" fill="#ffffff">tapA1</text> + <path d="m2834.4 1535.2v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 1535.2c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1651.0026" x="2912.8074" font-family="Arial" fill="#ffffff">tapAN</text> + <path d="m2834.4 2026.4v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 2026.4c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2142.3958" x="2921.5908" font-family="Arial" fill="#ffffff">t</text> + <text xml:space="preserve" font-size="106.4px" y="2142.3958" x="2951.1909" font-family="Arial" fill="#ffffff">apB0</text> + <path d="m2834.4 2291.2v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 2291.2c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2406.9922" x="2921.5908" font-family="Arial" fill="#ffffff">tapB1</text> + <path d="m2834.4 2669.6v151.2h453.6v-151.2h-453.6z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m2827.6 2669.6c0-3.6 3.2-6.8 6.8-6.8h453.6c3.8 0 6.8 3.2 6.8 6.8v151.2c0 3.8-3 6.8-6.8 6.8h-453.6c-3.6 0-6.8-3-6.8-6.8zm13.6 151.2-6.8-6.8h453.6l-6.8 6.8v-151.2l6.8 6.8h-453.6l6.8-6.8z" stroke="#385d8a" stroke-width=".2px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="2784.9868" x="2912.8074" font-family="Arial" fill="#ffffff">tapBN</text> + <path stroke-linejoin="round" d="m1510.7 1543.2 9-0.4 7.8-1.2 8.1-1.9 7.9-2.6 8-3.3 7.9-3.9 8-4.7 7.9-5.3 7.7-5.9 7.7-6.6 7.6-7.1 7.5-7.7 7.2-8.2 7.1-8.8 6.8-9.2 6.6-9.7 6.4-10.2 6.1-10.5 5.8-10.9 5.4-11.3 5.1-11.6 4.8-11.9 4.4-12.1 4-12.4 3.7-12.6 3.1-12.7 2.8-13 2.2-13.1 1.8-13.1 1.3-13.2 0.8-13.2 0.2-13.4 0.6-13.9 1.6-14.5 2.6-14.4 3.6-14.4 4.6-14.2 5.5-14 6.4-13.8 7.2-13.5 8-13.3 8.7-13 9.5-12.7 10.2-12.4 10.8-12.1 11.6-11.6 12-11.3 12.7-10.8 13.1-10.4 13.6-9.9 14.2-9.4 14.5-8.9 15-8.3 15.3-7.8 15.7-7.1 15.9-6.5 16.3-5.9c0.6-0.2 1.2-0.4 1.9-0.5l12.2-2.1 4 22.8-12.2 2.2 1.9-0.5-15.3 5.4-15.1 6.2-14.8 6.8-14.6 7.3-14.1 7.8-13.8 8.4-13.3 8.9-12.9 9.3-12.5 9.8-11.9 10.2-11.4 10.6-10.7 10.9-10.2 11.2-9.5 11.5-8.8 11.8-8.2 12.1-7.4 12.2-6.6 12.4-5.8 12.6-5 12.6-4.2 12.7-3.3 12.8-2.4 12.9-1.5 12.8-0.5 13.5-0.3 14.2-0.8 14.2-1.4 14.1-1.9 13.9-2.5 13.9-2.9 13.7-3.4 13.6-3.8 13.3-4.3 13.2-4.6 12.9-5.1 12.6-5.5 12.3-5.9 12-6.2 11.7-6.5 11.3-6.8 10.8-7.2 10.5-7.4 10-7.7 9.5-8 9.1-8.2 8.5-8.5 7.8-8.6 7.4-9 6.8-9.1 6-9.3 5.3-9.5 4.8-9.6 3.8-9.9 3.1-9.8 2.2-10.1 1.4-9 0.4zm428.8-589.4 120.1 15-108.6 53.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m1510.6 2261.2 9.5-0.5 8.3-1.3 8.3-2.1 8.3-3 8.4-3.7 8.4-4.5 8.4-5.2 8.3-6 8.2-6.7 8.1-7.4 8-8.1 7.8-8.7 7.7-9.4 7.5-10 7.2-10.4 6.9-11 6.8-11.5 6.4-12 6.1-12.4 5.7-12.7 5.5-13.2 5-13.4 4.7-13.9 4.2-14 3.8-14.3 3.3-14.5 2.9-14.6 2.4-14.8 1.9-15 1.4-15 0.8-15 0.3-15.1 0.5-15.7 1.6-16.2 2.5-16.1 3.5-16 4.4-15.8 5.3-15.6 6.1-15.5 6.9-15.2 7.8-14.9 8.5-14.6 9.2-14.3 9.9-13.9 10.5-13.6 11.2-13.1 11.7-12.7 12.3-12.2 12.8-11.7 13.2-11.2 13.8-10.6 14.2-10.1 14.5-9.4 14.9-8.8 15.3-8.1 15.6-7.3 26.4-8.6 7.1 22.1-23.6 7.4-14.6 7-14.4 7.6-14.1 8.2-13.8 8.9-13.4 9.4-12.9 10-12.6 10.6-12.1 11-11.6 11.6-11.1 11.9-10.5 12.4-9.9 12.7-9.3 13.1-8.6 13.3-8 13.7-7.3 13.9-6.4 14.1-5.8 14.3-4.9 14.4-4.1 14.6-3.2 14.7-2.4 14.7-1.4 14.7-0.5 15.3-0.3 15.9-0.9 15.9-1.5 15.9-2 15.6-2.5 15.6-3.1 15.5-3.5 15.2-4 15-4.5 14.8-4.9 14.4-5.3 14.3-5.7 13.8-6.1 13.5-6.5 13.1-6.8 12.7-7.1 12.2-7.5 11.8-7.8 11.2-8.1 10.7-8.3 10.1-8.6 9.6-8.9 8.9-9.1 8.3-9.3 7.6-9.6 6.8-9.7 6-10 5.3-10 4.4-10.4 3.5-10.5 2.5-10.5 1.5-9.4 0.5-1.3-23.1zm428.5-661.4 120.5 11.4-106.9 56.9-13.6-68.3z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2059.2 2114.1-9.7-0.4-10.5-1.2-10.6-2.1-10.4-2.8-10.2-3.6-10.1-4.4-9.9-5-9.7-5.6-9.6-6.2-9.3-6.9-9.1-7.4-8.8-8-8.6-8.3-8.3-9-8-9.3-7.7-9.9-7.4-10.1-7-10.6-6.7-11-6.3-11.2-5.9-11.6-5.5-11.8-5.1-12.2-4.6-12.3-4.2-12.6-3.6-12.8-3.2-12.8-2.6-13.1-2.1-13.2-1.5-13.2-0.9-13.4-0.4-13.3-0.4-12.7-1.4-11.9-2.3-12-3.2-11.8-4-11.9-4.7-11.8-5.6-11.7-6.3-11.5-7.1-11.4-7.8-11.3-8.5-11-9.1-10.7-9.7-10.5-10.3-10.2-10.9-9.9-11.4-9.5-11.9-9.2-12.4-8.7-12.8-8.3-13.2-7.8-13.6-7.4-13.8-6.8-14.3-6.3-14.4-5.7-21.4-5.5 5.9-22.4 24 6.3 15.4 6.1 15 6.7 14.7 7.3 14.4 7.8 14 8.3 13.6 8.8 13 9.2 12.7 9.8 12.1 10.2 11.6 10.5 11.1 10.9 10.3 11.3 9.9 11.6 9.1 11.9 8.4 12.2 7.7 12.5 7 12.7 6.1 13 5.3 13.1 4.4 13.3 3.5 13.6 2.5 13.5 1.5 13.7 0.5 13 0.3 12.4 0.8 12.2 1.4 12.3 1.9 12.2 2.5 12.1 2.9 12.1 3.4 11.9 3.8 11.6 4.4 11.6 4.6 11.3 5.1 11.1 5.5 10.8 5.9 10.5 6.2 10.2 6.5 9.8 6.8 9.4 7.1 9.1 7.4 8.6 7.6 8.2 7.8 7.7 8 7.3 8.2 6.7 8.3 6.1 8.4 5.6 8.6 5 8.6 4.3 8.7 3.8 8.6 3.1 8.8 2.5 8.7 1.8 8.8 1.1 9.6 0.4-0.9 23.1zm-439.5-505.7-108.5-54 120.3-14.6-11.8 68.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2059.3 2378.7-10.4-0.3-11.2-1.2-11-1.9-11-2.6-10.8-3.3-10.7-4-10.6-4.7-10.2-5.1-10.3-5.8-9.9-6.4-9.7-6.8-9.4-7.4-9.2-7.8-8.9-8.3-8.6-8.8-8.2-9.1-7.9-9.5-7.6-9.9-7.1-10.2-6.8-10.5-6.4-10.8-5.9-11.2-5.4-11.3-5-11.5-4.5-11.8-3.9-11.9-3.4-12.2-2.9-12.2-2.2-12.3-1.7-12.5-1-12.5-0.3-12.6-0.4-11.8-1.4-11-2.2-11-3-11.1-3.7-11-4.6-10.8-5.3-10.9-6.1-10.7-6.7-10.6-7.4-10.4-8.1-10.2-8.7-10-9.3-9.7-9.9-9.5-10.4-9.1-10.9-8.9-11.4-8.5-11.8-8.1-12.2-7.8-12.6-7.2-13-6.8-13.3-6.4-13.6-5.9-13.9-5.3 1.6 0.4-17.4-4 5.2-22.6 17.5 4c0.5 0.2 1 0.3 1.6 0.5l14.6 5.7 14.4 6.3 14.1 6.7 13.7 7.3 13.5 7.8 12.9 8.2 12.7 8.6 12.1 9.1 11.6 9.4 11.2 9.9 10.5 10.2 10 10.5 9.4 10.9 8.7 11 8.2 11.5 7.4 11.6 6.6 11.9 5.9 12.1 5.1 12.3 4.3 12.5 3.3 12.6 2.4 12.7 1.5 12.9 0.5 12.1 0.3 11.3 0.8 11.4 1.5 11.3 2 11.3 2.6 11.2 3.1 11.1 3.7 11 4 10.8 4.6 10.7 5 10.4 5.5 10.3 5.8 10 6.2 9.8 6.7 9.4 6.9 9.1 7.3 8.8 7.6 8.5 7.9 8 8.1 7.6 8.4 7.2 8.5 6.7 8.9 6.3 8.9 5.7 9 5.3 9.4 4.6 9.2 4.2 9.4 3.5 9.5 2.9 9.5 2.4 9.5 1.7 9.5 1 10.3 0.4zm-439.7-467.7-108.4-54.2 120.3-14.4z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2059.3 2756.9-10.4-0.3-11.1-1.1-10.9-1.7-11-2.4-10.8-3.1-10.7-3.7-10.4-4.2-10.4-4.9-10.1-5.3-9.9-5.9-9.7-6.3-9.5-6.9-9.1-7.2-9-7.8-8.5-8-8.3-8.4-8-8.9-7.5-9.2-7.2-9.4-6.8-9.9-6.4-10-5.9-10.2-5.5-10.6-5-10.8-4.4-10.9-4-11.2-3.5-11.2-2.8-11.5-2.3-11.5-1.6-11.6-1.1-11.7-0.3-11.7-0.4-10.9-1.4-10-2.2-10.1-2.9-10.1-3.7-10.1-4.6-10-5.3-9.9-6-10-6.7-9.7-7.4-9.6-8-9.4-8.7-9.2-9.3-9-9.8-8.8-10.4-8.5-10.9-8.1-11.4-7.9-11.8-7.5-12.3-7.2-12.6-6.7-13-6.3-13.3-6-13.6-5.4-13.9-4.9-15.7-3.3 4.8-22.7 18.8 4.1 14.6 5.3 14.4 5.7 14.1 6.3 13.7 6.7 13.5 7.2 12.9 7.6 12.6 8 12.1 8.4 11.6 8.8 11.2 9.1 10.6 9.4 10 9.8 9.4 10.1 8.8 10.3 8.2 10.6 7.4 10.8 6.7 11.1 5.9 11.3 5.1 11.5 4.3 11.6 3.4 11.8 2.4 11.9 1.5 12.1 0.5 11.2 0.3 10.4 0.8 10.4 1.5 10.4 2 10.3 2.5 10.3 3.1 10.2 3.6 10.2 4.1 9.9 4.6 9.9 4.9 9.6 5.5 9.5 5.8 9.2 6.2 9 6.6 8.8 6.9 8.4 7.3 8.1 7.6 7.8 7.9 7.4 8 7.1 8.5 6.7 8.5 6.2 8.8 5.8 8.9 5.3 9.2 4.8 9.2 4.4 9.4 3.8 9.4 3.3 9.5 2.7 9.5 2.2 9.6 1.5 9.6 1 10.3 0.4-0.7 23.1zm-439.1-431.2-109-52.9 120.1-15.8-11.1 68.7z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2551.2 957.2h26.5 12.9l12.7 0.1h12.2 11.7 11.1l10.4 0.1h9.5 8.5l7.7 0.1h3.4 2.9 2.8l2.6 0.1h1.9 1.7l1.8 0.1 0.6-0.1 1.6 0.1h-0.8 0.2l1.6 0.1v-0.1h1.4l2 0.1h1.8 2.6 2.5 3.1l3.6 0.1h18.4v23.2h-18.8l-3.2-0.1h-3.1-2.9-2.2-2.4l-1.4-0.1h-1.4-2 0.4-0.2-1.2l0.4-0.1h-1.4-1-1.7-2.3l-2.2-0.1h-2.8-3.2-3.3l-7.5-0.1h-8.6-9.5l-10.3-0.1h-11.2-11.6-12.3l-12.6-0.1h-12.9-26.5v-23.2zm167.4-22.5 115.9 35-116.1 34.6 0.2-69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2551.2 1221.2h26.5 12.9l12.7 0.1h12.2 11.7 11.1l10.4 0.1h9.5 8.5l7.7 0.1h3.4 2.9 2.8l2.6 0.1h1.9 1.7l1.8 0.1 0.6-0.1 1.6 0.1h-0.8 0.2l1.6 0.1v-0.1h1.4l2 0.1h1.8 2.6 2.5 3.1l3.6 0.1h18.4v23.2h-18.8l-3.2-0.1h-3.1-2.9-2.2-2.4l-1.4-0.1h-1.4-2 0.4-0.2-1.2l0.4-0.1h-1.4-1-1.7-2.3l-2.2-0.1h-2.8-3.2-3.3l-7.5-0.1h-8.6-9.5l-10.3-0.1h-11.2-11.6-12.3l-12.6-0.1h-12.9-26.5v-23.2zm167.4-22.5 115.9 35-116.1 34.6 0.2-69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2551.2 1599.6h26.5 12.9l12.7 0.1h12.2 11.7 11.1l10.4 0.1h9.5 8.5l7.7 0.1h3.4 2.9 2.8l2.6 0.1h1.9 1.7l1.8 0.1 0.6-0.1 1.6 0.1h-0.8 0.2l1.6 0.1v-0.1h1.4l2 0.1h1.8 2.6 2.5 3.1l3.6 0.1h18.4v23.2h-18.8l-3.2-0.1h-3.1-2.9-2.2-2.4l-1.4-0.1h-1.4-2 0.4-0.2-1.2l0.4-0.1h-1.4-1-1.7-2.3l-2.2-0.1h-2.8-3.2-3.3l-7.5-0.1h-8.6-9.5l-10.3-0.1h-11.2-11.6-12.3l-12.6-0.1h-12.9-26.5v-23.2zm167.4-22.5 115.9 35-116.1 34.6 0.2-69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2834.5 2114.9h-26.4-13l-12.6-0.1h-12.2-11.7-11.2-10.3l-9.5-0.1h-8.6-7.5l-3.5-0.1h-3-2.7-2.6-1.9l-2-0.1h-1.1-1-1.6 0.8-0.2l-1.6-0.1h-1.8-1.3-2.1l-2.6-0.1h-2.6-3.1-3.5l-18.5-0.1 0.1-23.2 18.8 0.1h3.1 3.1 3l2.2 0.1h2.1 2.1 1l2 0.1h-0.4 0.2 1.2-0.4 1l1.7 0.1h1.4 2.3 2.2 2.7 3.3l3.2 0.1h7.6 8.6l9.4 0.1h10.4 11.1 11.7 12.2l12.6 0.1h13 26.4v23.2zm-167.3 22.5-116-35 116.1-34.6-0.1 69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2834.5 2379.7h-26.4-13l-12.6-0.1h-12.2-11.7-11.2-10.3l-9.5-0.1h-8.6-7.5l-3.5-0.1h-3-2.7-2.6-1.9l-2-0.1h-1.1-1-1.6 0.8-0.2l-1.6-0.1h-1.8-1.3-2.1l-2.6-0.1h-2.6-3.1-3.5l-18.5-0.1 0.1-23.2 18.8 0.1h3.1 3.1 3l2.2 0.1h2.1 2.1 1l2 0.1h-0.4 0.2 1.2-0.4 1l1.7 0.1h1.4 2.3 2.2 2.7 3.3l3.2 0.1h7.6 8.6l9.4 0.1h10.4 11.1 11.7 12.2l12.6 0.1h13 26.4v23.2zm-167.3 22.5-116-35 116.1-34.6-0.1 69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m2834.5 2757.3h-26.4-13l-12.6-0.1h-12.2-11.7-11.2-10.3l-9.5-0.1h-8.6-7.5l-3.5-0.1h-3-2.7-2.6-1.9l-2-0.1h-1.1-1-1.6 0.8-0.2l-1.6-0.1h-1.8-1.3-2.1l-2.6-0.1h-2.6-3.1-3.5l-18.5-0.1 0.1-23.2 18.8 0.1h3.1 3.1 3l2.2 0.1h2.1 2.1 1l2 0.1h-0.4 0.2 1.2-0.4 1l1.7 0.1h1.4 2.3 2.2 2.7 3.3l3.2 0.1h7.6 8.6l9.4 0.1h10.4 11.1 11.7 12.2l12.6 0.1h13 26.4v23.2zm-167.3 22.5-116-35 116.1-34.6-0.1 69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m3288.2 957.2 16.2 0.4 16.6 0.8 16.6 1.2 16.6 1.8 16.4 2.2 16.4 2.8 16.4 3.2 16.4 3.6 16.2 4.2 16.2 4.6 16 5 16 5.4 16 6 15.8 6.2 15.6 6.8 15.4 7 15.4 7.6 15.4 7.8 15 8.2 15 8.6 14.8 9 14.6 9.2 14.4 9.8 14.2 9.8 14 10.4 14 10.6 13.6 10.8 13.4 11.4 26 23 25.2 24.4 24.2 25.6 23 26.4 11.2 13.6 10.8 13.6 10.6 14 10 14.4 10 14.2 9.4 14.6 9.2 14.8 9 14.8 8.4 15.2 8.2 15.2 7.8 15.4 7.4 15.6 7 15.6 6.6 15.8 6.2 16 5.8 16 5.4 16 5 16.2 4.6 16.4 4 16.4 3.8 16.4 3 16.6 2.8 16.6 2.2 16.6 1.8 16.6 1.2 16.8 0.8 16.6 0.2 16.8v16.8l-0.8 16.8-1.2 16.8-1.8 16.6-2.2 16.8-2.8 16.6-3.2 16.4-3.6 16.6-4 16.4-4.6 16.2-5 16.2-5.4 16.2-5.8 16-6.2 16-6.6 15.8-7 15.6-7.4 15.6-7.8 15.4-8 15.4-8.6 15-8.8 15-9.2 14.6-9.6 14.6-9.8 14.4-10.2 14.2-10.4 14-10.8 13.8-11.2 13.6-22.8 26.2-24.2 25.6-25.2 24.4-26 23.2-13.6 11.2-13.4 11-14 10.6-14 10.4-14.2 9.8-14.4 9.8-14.6 9.2-14.8 9-14.8 8.6-15.2 8.2-15.2 7.8-15.4 7.6-15.4 7-15.8 6.8-15.6 6.2-16 5.8-15.8 5.6-16.2 5-16 4.6-16.4 4-26 4.6-3.8-22.8 24.2-4.2 15.6-4 15.6-4.4 15.4-4.8 15.4-5.2 15.4-5.8 15.2-6 15-6.4 15-7 15-7.2 14.8-7.6 14.6-7.8 14.4-8.4 14.4-8.8 14-9 14-9.4 13.8-9.6 13.8-10 13.4-10.2 13.2-10.8 13-10.8 25.4-22.6 24.4-23.6 23.4-24.8 22.6-25.8 10.8-13.2 10.4-13.2 10.2-13.6 9.8-13.8 9.6-14 9.4-14.2 8.8-14.2 8.6-14.6 8.2-14.6 8-14.8 7.4-15 7.2-15 6.8-15.2 6.4-15.2 6-15.4 5.6-15.6 5.2-15.6 4.8-15.6 4.4-15.6 4-15.8 3.4-16 3.2-15.8 2.6-16 2.2-16 1.6-16 1.2-16 0.8-15.8v-16.4l-0.2-16-0.6-16.2-1.2-15.8-1.8-16.2-2.2-15.8-2.6-16-3-15.8-3.4-16-4-15.8-4.4-15.8-4.8-15.6-5.2-15.6-5.6-15.4-6-15.4-6.4-15.4-6.8-15-7.2-15.2-7.4-14.8-8-14.8-8.2-14.6-8.6-14.6-9-14.4-9.2-14-9.6-14-9.8-13.8-10.2-13.6-10.6-13.4-10.6-13-22.4-25.8-23.6-24.6-24.4-23.8-25.6-22.8-13-10.8-13.2-10.6-13.4-10.4-13.6-10-14-9.6-13.8-9.4-14.2-9-14.4-8.6-14.4-8.4-14.8-8-14.6-7.6-15-7.2-15-7-15.2-6.4-15.2-6-15.2-5.8-15.4-5.2-15.6-4.8-15.6-4.6-15.6-3.8-15.6-3.6-15.8-3.2-15.8-2.6-15.8-2.2-15.8-1.6-15.8-1.4-15.8-0.6-16.2-0.4 0.2-23.2zm119.8 1432.2-119.2-22.2 111.6-47 7.6 69.2z" stroke="#c0504d" stroke-width=".2px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m3288.2 1221.2 10.8 0.2 11.2 0.6 11 0.8 11 1 22 3.2 21.8 4.2 21.8 5.6 21.4 6.6 21.2 7.4 20.8 8.8 20.6 9.6 20 10.4 19.6 11.4 19.2 12.4 18.8 13 18 13.8 17.4 14.6 16.8 15.4 16.2 15.8 15.4 16.6 14.8 17 13.8 17.8 13 18 12 18.6 11.2 19 10.2 19.6 9.2 19.6 8 20.2 7 20.2 5.8 20.6 2.6 10.4 2 10.6 1.8 10.4 1.6 10.4 1.2 10.6 0.8 10.4 0.6 10.6 0.2 10.6v10.6l-0.6 10.8-0.8 10.6-1.2 10.4-1.6 10.4-1.8 10.6-2.2 10.4-2.4 10.4-5.6 20.4-7 20.4-8 20-9.2 19.8-10.2 19.6-11 19-12.2 18.6-13 18.2-13.8 17.6-14.6 17.2-15.4 16.6-16 15.8-16.8 15.4-17.6 14.6-18 13.8-18.6 13-19 12.4-19.8 11.4-20 10.6-20.4 9.6-20.8 8.6-21.2 7.6-25.6 6-5.2-22.6 23-5.2 20.2-7.2 19.8-8.2 19.4-9.2 19.2-10 18.8-11 18.2-11.8 18-12.4 17.2-13.2 16.8-14 16-14.6 15.4-15.2 14.8-15.8 14-16.4 13.2-17 12.4-17.2 11.6-17.8 10.6-18 9.6-18.4 8.6-18.8 7.6-18.8 6.6-19.2 5.4-19.6 2.4-9.8 2-9.6 1.8-9.8 1.4-9.8 1-9.8 0.8-9.8 0.6-9.6v-10.2l-0.2-10-0.4-9.6-0.8-10-1.2-9.6-1.4-9.8-1.6-9.8-2-9.8-2.4-9.6-5.4-19.2-6.4-19.2-7.6-19-8.8-18.8-9.6-18.4-10.6-18-11.4-17.8-12.6-17.4-13-16.8-14-16.4-14.8-15.8-15.6-15.4-16-14.6-16.8-14-17.2-13.2-17.8-12.6-18.4-11.6-18.8-11-19.2-10-19.4-9.2-19.8-8.2-20.2-7.4-20.4-6.2-20.4-5.2-20.8-4-20.8-3-10.4-1.2-10.4-0.6-10.6-0.6-10.6-0.2 0.2-23.2zm120.6 897.8-120-17 109.4-51.8 10.6 68.8z" stroke="#c0504d" stroke-width=".2px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m3288.2 1599.6 14.8 0.2 14.8 0.8 15 1 14.8 1.4 15 1.8 14.8 2.2 14.8 2.6 14.6 3 14.6 3.4 14.8 3.8 14.4 4 14.4 4.4 14.4 4.8 14.2 5.2 14 5.4 14.2 5.6 13.8 6.2 13.8 6.4 27 13.6 26.6 14.8 25.8 15.8 25.2 17 24.4 18 23.6 18.8 22.8 19.8 22 20.8 20.8 21.4 19.8 22.2 18.6 22.8 9 12 8.8 11.8 8.2 12 8 12.2 7.8 12.2 7.4 12.4 7 12.6 6.6 12.6 6.4 12.8 6 12.8 5.6 13 5.2 13 5 13.2 4.6 13.2 4 13.4 3.8 13.2 3.2 13.6 3 13.4 2.4 13.6 2 13.4 1.6 13.6 1.2 13.8 0.6 13.6 0.4 13.6-0.2 13.8-0.6 13.8-1 13.6-1.8 13.8-2 13.4-2.4 13.6-3 13.6-3.2 13.4-3.8 13.4-4 13.4-4.6 13.2-4.8 13.2-5.2 13-5.8 13-5.8 12.8-6.4 12.8-6.8 12.8-7 12.4-7.4 12.4-7.6 12.4-8 12.2-8.4 12-17.4 23.4-18.8 23-19.6 22.2-21 21.4-21.8 20.8-22.6 19.8-23.6 18.8-24.4 18-25.2 17-25.8 16-26.6 14.8-27 13.6-14 6.4-13.8 6.2-14 5.6-14.2 5.4-14.2 5.2-14.2 4.8-14.4 4.4-14.6 4-14.6 3.8-14.6 3.2-13.8 1.8-2.6-23 11.2-1.4 14.2-3.2 14-3.4 14-4 13.8-4.2 13.8-4.6 13.8-5 13.6-5.2 13.4-5.6 13.6-5.8 13.2-6.2 26.2-13.2 25.6-14.2 25-15.4 24.4-16.4 23.6-17.4 23-18.4 22-19.2 21-20 20.2-20.6 19.2-21.6 18-22 17-22.8 8-11.6 7.8-11.8 7.4-11.8 7-11.8 6.8-12.2 6.4-12 6-12.2 5.8-12.4 5.4-12.4 5-12.6 4.8-12.4 4.2-12.8 4-12.6 3.4-12.8 3.2-12.8 2.8-12.6 2.4-13 1.8-12.8 1.6-12.8 1-13 0.6-12.6 0.2-13.4-0.2-12.8-0.8-13-1-12.8-1.4-12.8-2-13-2.4-12.8-2.6-12.8-3.2-12.6-3.6-12.8-3.8-12.8-4.4-12.6-4.6-12.6-5-12.4-5.4-12.4-5.8-12.4-6-12.2-6.4-12.2-6.8-12-7.2-11.8-7.4-12-7.6-11.6-8-11.6-8.4-11.4-8.4-11.2-18.2-22.2-19-21.4-20.2-20.8-21.2-20-22-19.2-22.8-18.4-23.8-17.4-24.2-16.4-25.2-15.4-25.6-14.4-26.4-13.2-13.2-6.2-13.6-5.8-13.4-5.6-13.8-5.2-13.6-5-13.8-4.6-14-4.2-13.8-4-14.2-3.6-14-3.2-14.2-2.8-14-2.6-14.4-2.2-14.2-1.8-14.2-1.2-14.4-1.2-14.2-0.6-14.6-0.2 0.2-23.2zm119.6 1167.6-119.2-22.2 111.6-47 7.6 69.2z" stroke="#c0504d" stroke-width=".2px" fill="#c0504d"/> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="3128.3906" font-family="Arial" fill="#000000">Kernel</text> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="3436.3906" font-family="Arial" fill="#000000">-</text> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="3471.5906" font-family="Arial" fill="#000000">space: </text> + <text xml:space="preserve" font-size="106.4px" y="770.81787" x="3128.3906" font-family="Arial" fill="#000000">bridging/forwarding</text> + <path d="m56.8 1422.4v906.4h604.8v-906.4h-604.8z" fill-rule="evenodd" fill="#4f81bd"/> + <path stroke-linejoin="round" d="m50 1422.4c0-3.7 3.1-6.8 6.8-6.8h604.8c3.8 0 6.8 3.1 6.8 6.8v906.4c0 3.8-3 6.8-6.8 6.8h-604.8c-3.7 0-6.8-3-6.8-6.8zm13.6 906.4-6.8-6.8h604.8l-6.8 6.8v-906.4l6.8 6.8h-604.8l6.8-6.8z" stroke="#385d8a" stroke-width=".1px" fill="#385d8a"/> + <text xml:space="preserve" font-size="106.4px" y="1851.599" x="213.47821" font-family="Arial" fill="#ffffff">Traffic </text> + <text xml:space="preserve" font-size="106.4px" y="1979.599" x="119.07821" font-family="Arial" fill="#ffffff">Generator</text> + <path stroke-linejoin="round" d="m1510.7 1845 9.9-0.5 8.8-1.2 8.9-2.1 9-2.7 8.8-3.6 8.8-4.2 8.9-5 8.7-5.7 8.6-6.4 8.6-6.9 8.3-7.7 8.2-8.2 8.1-8.8 7.8-9.4 7.5-9.8 7.3-10.4 7.1-10.8 6.7-11.3 6.3-11.6 6.1-12 5.6-12.4 5.3-12.6 4.8-13 4.4-13.2 4-13.4 3.5-13.6 3-13.8 2.5-13.9 2-14 1.5-14.1 0.8-14.1 0.3-14.1 0.5-14.8 1.5-15.3 2.4-15.3 3.5-15.1 4.2-15 5.3-14.8 5.9-14.6 6.8-14.4 7.5-14.1 8.3-13.8 9-13.5 9.6-13.1 10.3-12.8 10.8-12.4 11.4-12 11.9-11.5 12.5-11.1 12.9-10.5 13.4-10.1 13.8-9.5 14.1-8.8 14.6-8.3 14.9-7.6 15.1-7l1.5-0.6 21.5-6.3 6.6 22.2-21.5 6.4 1.6-0.6-14.3 6.5-14 7.2-13.7 7.7-13.3 8.4-13 8.9-12.6 9.4-12.2 10-11.7 10.3-11.3 10.9-10.8 11.2-10.2 11.7-9.6 11.9-9 12.4-8.4 12.6-7.7 12.8-7 13.1-6.3 13.2-5.6 13.5-4.7 13.5-4 13.7-3.2 13.8-2.2 13.7-1.4 13.9-0.5 14.4-0.3 15.1-1 15.1-1.5 14.9-2.1 14.9-2.7 14.8-3.2 14.6-3.7 14.4-4.3 14.2-4.6 14-5.2 13.7-5.6 13.3-6 13.2-6.4 12.8-6.8 12.4-7.2 11.9-7.5 11.6-7.8 11.1-8.2 10.7-8.5 10-8.8 9.6-8.9 9-9.4 8.4-9.4 7.7-9.8 7.2-10 6.4-10.1 5.7-10.4 4.9-10.6 4.1-10.6 3.3-10.8 2.3-10.9 1.5-9.9 0.4zm428.4-623.6 120.5 11.4-106.9 56.9z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="1503.0131" font-family="Arial" fill="#000000">User</text> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="1727.8131" font-family="Arial" fill="#000000">-</text> + <text xml:space="preserve" font-size="106.4px" y="642.81787" x="1763.0131" font-family="Arial" fill="#000000">space:</text> + <text xml:space="preserve" font-size="106.4px" y="770.81787" x="1503.0131" font-family="Arial" fill="#000000">DPDK application</text> + <path stroke-linejoin="round" d="m768.1 1596.7 215-31.4 3.4 22.9-215.1 31.5-3.3-23zm18.2 44.3-119.9-17.7 109.8-51.2 10.1 68.9zm182-97 119.8 17.6-109.7 51.3-10.1-68.9z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m772.4 1853.1 212.9-0.4v23.2l-212.8 0.4-0.1-23.2zm11.7 46.4-116.1-34.6 116-35 0.1 69.6zm189.5-70 116.1 34.5-115.9 35.1-0.2-69.6z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path stroke-linejoin="round" d="m775.2 2204.9 216.5 41.1-4.3 22.8-216.5-41.1 4.3-22.8zm2.7 47.8-107.5-55.9 120.5-12.5-13 68.4zm206.7-31.7 107.5 55.8-120.4 12.6 12.9-68.4z" stroke="#c0504d" stroke-width=".1px" fill="#c0504d"/> + <path d="m1266 1988.4v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m1266 2064.4v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m1266 2139.6v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 1346v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 1422v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 1497.2v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 2479.6v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 2555.6v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m2286 2630.8v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 1346v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 1422v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 1497.2v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 2479.6v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 2555.6v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + <path d="m3042 2630.8v24h24.8v-24h-24.8z" fill-rule="evenodd" fill="#1f497d"/> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipsec_endpoints.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipsec_endpoints.svg new file mode 100644 index 00000000..e4aba4cc --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipsec_endpoints.svg @@ -0,0 +1,850 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="155.68507mm" + height="76.061203mm" + viewBox="0 0 551.64003 269.50821" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="endpoints.svg"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker18451" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path18453" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker18273" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path18275" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10612" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Send"> + <path + transform="matrix(-0.3,0,0,-0.3,0.69,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#cfce37;fill-opacity:1;fill-rule:evenodd;stroke:#cfce37;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path10614" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10270" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lstart"> + <path + transform="matrix(0.8,0,0,0.8,10,0)" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path10272" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Sstart" + orient="auto" + refY="0" + refX="0" + id="marker9102" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path9104" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#035002;fill-opacity:1;fill-rule:evenodd;stroke:#035002;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.2,0,0,0.2,1.2,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Sstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Sstart" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5945" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#d4ce00;fill-opacity:1;fill-rule:evenodd;stroke:#d4ce00;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.2,0,0,0.2,1.2,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient11398" + osb:paint="gradient"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop11400" /> + <stop + style="stop-color:#000000;stop-opacity:0;" + offset="1" + id="stop11402" /> + </linearGradient> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker9540" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path9542" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient9336" + osb:paint="solid"> + <stop + style="stop-color:#dc181a;stop-opacity:1;" + offset="0" + id="stop9338" /> + </linearGradient> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker7350" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7352" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path4871" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker7092" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path7094" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5822" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path5824" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Send" + orient="auto" + refY="0" + refX="0" + id="marker5369" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path5371" + style="fill:#035002;fill-opacity:1;fill-rule:evenodd;stroke:#035002;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-0.3,0,0,-0.3,0.69,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5013" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Tail" + orient="auto" + refY="0" + refX="0" + id="Tail" + style="overflow:visible" + inkscape:isstock="true"> + <g + id="g4907" + transform="scale(-1.2,-1.2)" + style="fill:#528ac6;fill-opacity:1;stroke:#000000;stroke-opacity:1"> + <path + id="path4909" + d="M -3.8048674,-3.9585227 0.54352094,0" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + <path + id="path4911" + d="M -1.2866832,-3.9585227 3.0617053,0" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + <path + id="path4913" + d="M 1.3053582,-3.9585227 5.6537466,0" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + <path + id="path4915" + d="M -3.8048674,4.1775838 0.54352094,0.21974226" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + <path + id="path4917" + d="M -1.2866832,4.1775838 3.0617053,0.21974226" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + <path + id="path4919" + d="M 1.3053582,4.1775838 5.6537466,0.21974226" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.80000001;stroke-linecap:round;stroke-opacity:1" + inkscape:connector-curvature="0" /> + </g> + </marker> + <marker + inkscape:stockid="Arrow2Send" + orient="auto" + refY="0" + refX="0" + id="Arrow2Send" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4904" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-0.3,0,0,-0.3,0.69,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0" + refX="0" + id="Arrow1Send" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4886" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#528ac6;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.2,0,0,-0.2,-1.2,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path4874" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5734-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path5736-7" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5734-4-3" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path5736-7-7" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5734-4-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path5736-7-1" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Send" + orient="auto" + refY="0" + refX="0" + id="marker5369-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path5371-1" + style="fill:#d4ce00;fill-opacity:1;fill-rule:evenodd;stroke:#d4ce00;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-0.3,0,0,-0.3,0.69,0)" /> + </marker> + <marker + inkscape:stockid="Arrow2Send" + orient="auto" + refY="0" + refX="0" + id="marker5369-7" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path5371-9" + style="fill:#38782d;fill-opacity:1;fill-rule:evenodd;stroke:#38782d;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-0.3,0,0,-0.3,0.69,0)" /> + </marker> + <marker + inkscape:stockid="Arrow2Send" + orient="auto" + refY="0" + refX="0" + id="marker5369-3-7" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path5371-1-0" + style="fill:#38782d;fill-opacity:1;fill-rule:evenodd;stroke:#38782d;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-0.3,0,0,-0.3,0.69,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker18451-7" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path18453-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart-7" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path4871-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker7350-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7352-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="2.8" + inkscape:cx="258.01346" + inkscape:cy="122.99559" + inkscape:document-units="cm" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1920" + inkscape:window-height="1017" + inkscape:window-x="1592" + inkscape:window-y="-8" + inkscape:window-maximized="1" + fit-margin-top="3" + fit-margin-right="1" + fit-margin-bottom="1" + fit-margin-left="1" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(194.35761,4.2637303)"> + <rect + style="opacity:1;fill:#528ac6;fill-opacity:1;stroke:none;stroke-opacity:1" + id="rect4140-5" + width="131.15564" + height="102.30846" + x="-179.57692" + y="140.22072" + ry="7.155364" /> + <rect + style="opacity:1;fill:#528ac6;fill-opacity:1;stroke:none;stroke-opacity:1" + id="rect4140-5-1" + width="131.15564" + height="102.30846" + x="213.55591" + y="140.22072" + ry="6.5590839" /> + <rect + style="opacity:1;fill:#528ac6;fill-opacity:1;stroke:none;stroke-opacity:1" + id="rect4140-5-2" + width="131.15564" + height="46.258114" + x="-179.57692" + y="6.3661909" + ry="5.3665228" /> + <rect + style="opacity:1;fill:#528ac6;fill-opacity:1;stroke:none;stroke-opacity:1" + id="rect4140-5-2-7" + width="131.15564" + height="46.258114" + x="213.55591" + y="6.3661909" + ry="6.5590839" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-182.36929" + y="188.59424" + id="text4210" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan4212" + x="-182.36929" + y="188.59424">ep0</tspan><tspan + sodipodi:role="line" + x="-182.36929" + y="208.69011" + id="tspan4214" /><tspan + sodipodi:role="line" + x="-182.36929" + y="228.786" + id="tspan4216" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="324.621" + y="188.49612" + id="text4218" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan4220" + x="324.621" + y="188.49612">ep1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-159.97678" + y="32.860863" + id="text4222" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan4224" + x="-159.97678" + y="32.860863">traffic gen</tspan><tspan + sodipodi:role="line" + x="-159.97678" + y="52.956738" + id="tspan4226" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="249.77068" + y="32.860863" + id="text4222-1" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan4224-1" + x="249.77068" + y="32.860863">traffic gen</tspan><tspan + sodipodi:role="line" + x="249.77068" + y="52.956738" + id="tspan4226-6" /></text> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)" + d="m -48.997848,184.52811 c 260.957198,-0.5963 260.957198,-0.5963 260.957198,-0.5963" + id="path5305" + inkscape:connector-curvature="0" /> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker10270)" + d="M -47.641399,212.81116 C 213.3158,212.21487 213.3158,212.21487 213.3158,212.21487" + id="path5305-2" + inkscape:connector-curvature="0" /> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker7350)" + d="m 262.8239,53.615939 c 0,85.864371 0,85.864371 0,85.864371" + id="path5922-0-6" + inkscape:connector-curvature="0" /> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart)" + d="m -146.8568,54.80849 c 0,85.86437 0,85.86437 0,85.86437" + id="path5922-0-1" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="246.78571" + y="500.57648" + id="text7750" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan7752" + x="246.78571" + y="500.57648" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-158.9595" + y="151.84753" + id="text7754" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7756" + x="-158.9595" + y="151.84753">2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-110.66317" + y="151.84753" + id="text7758" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7760" + x="-110.66317" + y="151.84753">3</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="267.07309" + y="151.84753" + id="text7762" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7764" + x="267.07309" + y="151.84753">2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="318.81448" + y="151.84753" + id="text7766" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7768" + x="318.81448" + y="151.84753">3</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="225.86406" + y="183.73483" + id="text7770" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7772" + x="225.86406" + y="183.73483">0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="226.18214" + y="211.56969" + id="text7774" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7776" + x="226.18214" + y="211.56969">1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:11.25px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="311.78571" + y="505.93362" + id="text7778" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan7780" + x="311.78571" + y="505.93362" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-65.239075" + y="183.73483" + id="text7782" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7784" + x="-65.239075" + y="183.73483">0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:18.08628845px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-64.921005" + y="211.56969" + id="text7786" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7788" + x="-64.921005" + y="211.56969">1</tspan></text> + <path + style="fill:none;fill-rule:evenodd;stroke:#dc181a;stroke-width:1.60606241;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:3.21212486, 9.63637456;stroke-dashoffset:0;stroke-opacity:1" + d="M -190.00162,248.33118 C -74.638406,174.62535 10.240093,56.871479 81.499676,57.515772 176.72224,59.019701 245.16177,197.27679 352.92565,246.54234" + id="path7850" + inkscape:connector-curvature="0" + sodipodi:nodetypes="ccc" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:center;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:middle;fill:#dc181a;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="84.985992" + y="98.449951" + id="text7876" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7878" + x="84.985992" + y="98.449951">UNPROTECTED</tspan><tspan + sodipodi:role="line" + x="84.985992" + y="118.54583" + id="tspan7886">cipher-text</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:center;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:middle;fill:#dc181a;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="84.899643" + y="20.937273" + id="text7880" + sodipodi:linespacing="125%" + inkscape:transform-center-x="48.100163" + inkscape:transform-center-y="3.577681" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan7882" + x="84.899643" + y="20.937273">PROTECTED</tspan><tspan + sodipodi:role="line" + x="84.899643" + y="41.03315" + id="tspan7884">clear-text</tspan></text> + <path + style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#035002;stroke-width:3.42781782;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker5369)" + d="m -101.57521,164.71986 c -0.0232,13.22578 17.528492,19.5572 32.486212,20.12768" + id="path9370" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#d4ce00;stroke-width:3.08351111;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker5369-3)" + d="m 231.46374,184.56991 c 19.27508,-1.72507 24.47287,-7.77038 31.06883,-19.63365" + id="path9370-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.8206749px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="60.45417" + y="251.78688" + id="text14635" + sodipodi:linespacing="125%" + transform="scale(0.9630889,1.0383257)"><tspan + sodipodi:role="line" + id="tspan14637" + x="60.45417" + y="251.78688">outbound</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:16.07670021px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="62.073208" + y="232.23116" + id="text14635-4" + sodipodi:linespacing="125%" + transform="scale(0.96291595,1.0385122)"><tspan + sodipodi:role="line" + id="tspan14637-2" + x="62.073208" + y="232.23116">inbound</tspan></text> + <path + style="fill:none;fill-rule:evenodd;stroke:#d4ce00;stroke-width:3.8278625;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Sstart)" + d="m -144.07995,171.20523 c 15.90052,30.60035 40.80354,39.04445 75.670299,43.29761" + id="path9370-1" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#035002;stroke-width:3.73783302;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#marker9102)" + d="m 236.99654,212.30897 c 40.08586,-4.29157 62.39356,-19.33069 76.11102,-48.84339" + id="path9370-8-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#d4ce00;stroke-width:4.01917505;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + d="m 32.274837,235.80819 c 18.037567,-0.2236 20.456397,-0.29813 20.456397,-0.29813" + id="path10604" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#035002;stroke-width:4.01917505;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + d="m 32.274837,256.82674 c 18.037567,-0.22362 20.456397,-0.29814 20.456397,-0.29814" + id="path10604-4" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7)" + d="m -100.78618,138.21108 c 0,-85.864363 0,-85.864363 0,-85.864363" + id="path5922-0-1-5" + inkscape:connector-curvature="0" /> + <path + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.60767007px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker7350-9)" + d="m 312.42811,137.49681 c 0,-85.864382 0,-85.864382 0,-85.864382" + id="path5922-0-6-9" + inkscape:connector-curvature="0" /> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipv4_acl_rule.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipv4_acl_rule.png Binary files differnew file mode 100644 index 00000000..017e8815 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ipv4_acl_rule.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/kernel_nic.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/kernel_nic.png Binary files differnew file mode 100644 index 00000000..bffb5c1d --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/kernel_nic.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_benchmark_setup.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_benchmark_setup.svg new file mode 100644 index 00000000..470226fe --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_benchmark_setup.svg @@ -0,0 +1,520 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2011 Intel Corporation --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="766.95715" + height="494.77206" + id="svg2" + version="1.1"> + <defs + id="defs4" /> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + showgrid="false" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + units="cm" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-135.8992,-434.6983)"> + <path + inkscape:connector-curvature="0" + id="4" + style="fill:none;stroke:#000000;stroke-width:2.12513781;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + d="m 140.50508,922.27019 123.45,0 0,-482.96601 -123.45,0 0,482.96601 z" /> + <text + xml:space="preserve" + x="157.20508" + y="692.83075" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;font-family:Arial;-inkscape-font-specification:Arial" + id="text4026" + sodipodi:linespacing="103%" /> + <text + id="6" + xml:space="preserve" + x="256.40509" + y="692.83075" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;font-family:Arial;-inkscape-font-specification:Arial" + sodipodi:linespacing="103%" /> + <path + inkscape:connector-curvature="0" + id="130" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 265.37162,514.23178 508.66341,0" /> + <path + inkscape:connector-curvature="0" + id="131" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 273.60035,514.35422 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <text + xml:space="preserve" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="162.14893" + y="885.45764" + id="text4028" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + id="tspan4030" + x="162.14893" + y="885.45764" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;font-family:Arial;-inkscape-font-specification:Arial">Traffic</tspan><tspan + sodipodi:role="line" + x="162.14893" + y="906.05762" + id="tspan4032" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;font-family:Arial;-inkscape-font-specification:Arial">Generator</tspan></text> + <path + inkscape:connector-curvature="0" + id="4-6" + style="fill:none;stroke:#000000;stroke-width:2.12513781;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + d="m 774.8005,924.8645 123.45,0 0,-482.96601 -123.45,0 0,482.96601 z" /> + <text + xml:space="preserve" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="796.44434" + y="888.05188" + id="text4028-8" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + id="tspan4030-2" + x="796.44434" + y="888.05188" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;font-family:Arial;-inkscape-font-specification:Arial">NUT</tspan><tspan + sodipodi:role="line" + x="796.44434" + y="908.65186" + id="tspan4032-0" + style="font-size:20px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;font-family:Arial;-inkscape-font-specification:Arial">(RTE)</tspan></text> + <path + inkscape:connector-curvature="0" + id="130-3" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 264.42725,533.68597 508.66341,0" /> + <path + inkscape:connector-curvature="0" + id="131-2" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 519.81423,513.99534 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-7" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 775.11271,514.27808 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:none" + id="rect4380" + width="57.578693" + height="29.294424" + x="358.14981" + y="499.76208" + rx="23.942125" + ry="23.942125" /> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="362.21854" + y="519.86017" + id="text4028-9" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="362.21854" + y="519.86017" + id="tspan4032-5" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">Flow 0</tspan></text> + <path + inkscape:connector-curvature="0" + id="131-2-1" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 509.92292,533.82453 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-0" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 263.67299,533.64775 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-7" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 764.48137,533.82453 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-31" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 264.7107,569.61938 508.6634,0" /> + <path + inkscape:connector-curvature="0" + id="131-1" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 272.93943,569.74182 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-3-3" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 263.76633,589.07357 508.66337,0" /> + <path + inkscape:connector-curvature="0" + id="131-2-5" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 519.1533,569.38294 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-7-7" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 774.4518,569.66568 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:none" + id="rect4380-6" + width="57.578693" + height="29.294424" + x="357.48889" + y="555.14972" + rx="23.942125" + ry="23.942125" /> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="361.55762" + y="575.2478" + id="text4028-9-8" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="361.55762" + y="575.2478" + id="tspan4032-5-5" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">Flow 1</tspan></text> + <path + inkscape:connector-curvature="0" + id="131-2-1-8" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 509.262,589.21214 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-0-7" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 263.01207,589.03535 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-7-9" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 765.94172,588.50503 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-34" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 266.17109,647.92558 508.66341,0" /> + <path + inkscape:connector-curvature="0" + id="131-9" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 274.39982,648.04802 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-3-5" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 265.22672,667.37977 508.66338,0" /> + <path + inkscape:connector-curvature="0" + id="131-2-11" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 520.6137,647.68914 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-7-6" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 775.9122,647.97188 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:none" + id="rect4380-8" + width="57.578693" + height="29.294424" + x="358.94928" + y="633.45587" + rx="23.942125" + ry="23.942125" /> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="363.01801" + y="653.55396" + id="text4028-9-1" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="363.01801" + y="653.55396" + id="tspan4032-5-1" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">Flow 2</tspan></text> + <path + inkscape:connector-curvature="0" + id="131-2-1-2" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 510.7224,667.51834 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-0-9" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 264.47246,667.34155 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-7-96" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 765.2808,667.51834 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-31-4" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 265.51017,703.31318 508.66343,0" /> + <path + inkscape:connector-curvature="0" + id="131-1-4" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 273.7389,703.43562 -10,5 c 1.6,-3.14999 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-3-3-9" + style="fill:none;stroke:#000000;stroke-width:2.08607888px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + d="m 264.5658,722.76737 508.6634,0" /> + <path + inkscape:connector-curvature="0" + id="131-2-5-9" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 519.9528,703.07674 -10,5 c 1.6,-3.14999 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-7-7-2" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 775.2513,703.35948 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:none" + id="rect4380-6-5" + width="57.578693" + height="29.294424" + x="358.28836" + y="688.84351" + rx="23.942125" + ry="23.942125" /> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="362.35709" + y="708.94159" + id="text4028-9-8-2" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="362.35709" + y="708.94159" + id="tspan4032-5-5-5" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">Flow 3</tspan></text> + <path + inkscape:connector-curvature="0" + id="131-2-1-8-3" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 510.0615,722.90594 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-0-7-9" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 263.81154,722.72915 10,5 c -1.6,-3.15 -1.6,-6.85 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="131-2-1-7-9-7" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + d="m 764.6199,722.90594 10,5 c -1.6,-3.15001 -1.6,-6.85001 0,-10 l 0,0 -10,5 z" /> + <path + inkscape:connector-curvature="0" + id="130-38" + style="fill:none;stroke:#000000;stroke-width:1.95051968;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.90103936, 3.90103936;stroke-dashoffset:0" + d="m 776.77917,533.43352 18.93346,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-5" + style="fill:none;stroke:#000000;stroke-width:1.95051968;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.90103936, 3.90103936;stroke-dashoffset:0" + d="m 776.27809,570.08799 18.93346,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-4" + style="fill:none;stroke:#000000;stroke-width:2.03578472;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.07156944, 4.07156944;stroke-dashoffset:0" + d="m 795.76128,534.45831 0,35.11152" /> + <path + inkscape:connector-curvature="0" + id="130-38-51" + style="fill:none;stroke:#000000;stroke-width:1.95051968;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.90103936, 3.90103936;stroke-dashoffset:0" + d="m 776.36881,667.40751 18.93347,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-5-0" + style="fill:none;stroke:#000000;stroke-width:1.95051968;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.90103936, 3.90103936;stroke-dashoffset:0" + d="m 775.86773,704.06198 18.93347,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-4-3" + style="fill:none;stroke:#000000;stroke-width:2.03578472;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.07156944, 4.07156944;stroke-dashoffset:0" + d="m 795.35093,668.4323 0,35.11152" /> + <path + inkscape:connector-curvature="0" + id="130-38-7" + style="fill:none;stroke:#000000;stroke-width:2.03061938;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.06123936, 4.06123936;stroke-dashoffset:0" + d="m 776.04259,515.37859 52.63775,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-7-7" + style="fill:none;stroke:#000000;stroke-width:2.03061938;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.06123936, 4.06123936;stroke-dashoffset:0" + d="m 776.70517,587.76566 52.63774,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-7-7-1" + style="fill:none;stroke:#000000;stroke-width:2.03595257;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.07190574, 4.07190574;stroke-dashoffset:0" + d="m 829.46781,518.11584 0,67.90517" /> + <path + inkscape:connector-curvature="0" + id="130-38-7-3" + style="fill:none;stroke:#000000;stroke-width:2.03061938;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.06123936, 4.06123936;stroke-dashoffset:0" + d="m 775.40198,648.44575 52.63775,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-7-7-5" + style="fill:none;stroke:#000000;stroke-width:2.03061938;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.06123936, 4.06123936;stroke-dashoffset:0" + d="m 776.06456,720.83282 52.63774,0" /> + <path + inkscape:connector-curvature="0" + id="130-38-7-7-1-0" + style="fill:none;stroke:#000000;stroke-width:2.03595257;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.07190574, 4.07190574;stroke-dashoffset:0" + d="m 828.8272,651.183 0,67.90517" /> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="249.25908" + y="517.75714" + id="text4028-9-14" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="249.25908" + y="517.75714" + id="tspan4032-5-19" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">0</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="248.77629" + y="591.84143" + id="text4028-9-14-8" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="248.77629" + y="591.84143" + id="tspan4032-5-19-8" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">1</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="249.80574" + y="651.65784" + id="text4028-9-14-1" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="249.80574" + y="651.65784" + id="tspan4032-5-19-6" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">2</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="249.32295" + y="727.74213" + id="text4028-9-14-8-8" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="249.32295" + y="727.74213" + id="tspan4032-5-19-8-6" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">3</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="843.30432" + y="519.68823" + id="text4028-9-14-9" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="843.30432" + y="519.68823" + id="tspan4032-5-19-0" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">0</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="842.82153" + y="589.77252" + id="text4028-9-14-8-3" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="842.82153" + y="589.77252" + id="tspan4032-5-19-8-8" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">1</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="843.85095" + y="671.58893" + id="text4028-9-14-1-2" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="843.85095" + y="671.58893" + id="tspan4032-5-19-6-8" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">2</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="843.36816" + y="725.67316" + id="text4028-9-14-8-8-6" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="843.36816" + y="725.67316" + id="tspan4032-5-19-8-6-3" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial">3</tspan></text> + <text + xml:space="preserve" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + x="514.69012" + y="789.7041" + id="text4028-1" + sodipodi:linespacing="103%"><tspan + sodipodi:role="line" + x="514.69012" + y="789.7041" + id="tspan4032-3" + style="font-size:16px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:102.99999714%;writing-mode:lr-tb;text-anchor:start;font-family:Arial;-inkscape-font-specification:Arial">...</tspan></text> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_encrypt_flow.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_encrypt_flow.svg new file mode 100644 index 00000000..492c3df0 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_encrypt_flow.svg @@ -0,0 +1,194 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export l2fwd-crypto-encrypt-flow.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + width="10.3779in" height="2.38075in" viewBox="0 0 747.207 171.414" xml:space="preserve" color-interpolation-filters="sRGB" + class="st15"> + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.25;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.25} + .st3 {fill:#ffc000;stroke:#40709c;stroke-width:0.75} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st5 {font-size:1em} + .st6 {fill:#4672c4;stroke:#40709c;stroke-width:0.75} + .st7 {fill:#538135;stroke:#40709c;stroke-width:0.75} + .st8 {marker-end:url(#mrkr4-58);stroke:#41719c;stroke-dasharray:3,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st9 {fill:#41719c;fill-opacity:1;stroke:#41719c;stroke-opacity:1;stroke-width:0.28409090909091} + .st10 {marker-end:url(#mrkr4-58);stroke:#41719c;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st11 {fill:none;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st12 {stroke:#c7c8c8;stroke-width:0.25} + .st13 {fill:none;stroke:none;stroke-width:0.25} + .st14 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em} + .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend4"> + <path d="M 2 1 L 0 0 L 2 -1 L 2 1 " style="stroke:none"/> + </g> + <marker id="mrkr4-58" class="st9" refX="-7.04" orient="auto" markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend4" transform="scale(-3.52,-3.52) "/> + </marker> + </defs> + <defs id="Filters"> + <filter id="filter_2"> + <feGaussianBlur stdDeviation="2"/> + </filter> + </defs> + <g> + <title>Page-1</title> + <g id="shape101-1" transform="translate(3.73674,-5.34781)"> + <title>Circle.53</title> + <desc>RX P0 Q0</desc> + <g id="shadow101-2" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st3"/> + <text x="21.35" y="141.75" class="st4">RX <tspan x="14.52" dy="1.2em" class="st5">P</tspan>0 Q0</text> </g> + <g id="shape102-8" transform="translate(101.797,-5.34781)"> + <title>Circle.56</title> + <desc>RX</desc> + <g id="shadow102-9" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st6"/> + <text x="21.35" y="147.75" class="st4">RX</text> </g> + <g id="shape103-14" transform="translate(395.977,-5.34781)"> + <title>Circle.57</title> + <desc>CRYPTO DEQ</desc> + <g id="shadow103-15" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st7"/> + <text x="10.52" y="141.75" class="st4">CRYPTO <tspan x="17.78" dy="1.2em" class="st5">DEQ</tspan></text> </g> + <g id="shape104-21" transform="translate(297.917,-5.34781)"> + <title>Circle.58</title> + <desc>CRYPTO ENQ</desc> + <g id="shadow104-22" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st7"/> + <text x="10.52" y="141.75" class="st4">CRYPTO <tspan x="17.63" dy="1.2em" class="st5">ENQ</tspan></text> </g> + <g id="shape105-28" transform="translate(690.158,-5.34781)"> + <title>Circle.73</title> + <desc>TX P0 Q0</desc> + <g id="shadow105-29" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st3"/> + <text x="21.63" y="141.75" class="st4">TX <tspan x="14.52" dy="1.2em" class="st5">P</tspan>0 Q0</text> </g> + <g id="shape106-35" transform="translate(494.037,-5.34781)"> + <title>Circle.74</title> + <desc>MAC</desc> + <g id="shadow106-36" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st6"/> + <text x="16.83" y="147.75" class="st4">MAC</text> </g> + <g id="shape107-41" transform="translate(199.857,-5.34781)"> + <title>Circle.61</title> + <desc>PAD</desc> + <g id="shadow107-42" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st6"/> + <text x="18.11" y="147.75" class="st4">PAD</text> </g> + <g id="shape108-47" transform="translate(592.097,-5.34781)"> + <title>Circle.62</title> + <desc>TX</desc> + <g id="shadow108-48" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st2"/> + </g> + <path d="M0 144.75 A26.6643 26.6643 0 0 1 53.33 144.75 A26.6643 26.6643 0 1 1 0 144.75 Z" class="st6"/> + <text x="21.63" y="147.75" class="st4">TX</text> </g> + <g id="shape109-53" transform="translate(57.0653,-24.9255)"> + <title>Dynamic connector.63</title> + <path d="M0 164.33 L37.69 164.33" class="st8"/> + </g> + <g id="shape110-59" transform="translate(155.125,-24.9255)"> + <title>Dynamic connector.65</title> + <path d="M0 164.33 L37.69 164.33" class="st10"/> + </g> + <g id="shape111-64" transform="translate(253.186,-24.9255)"> + <title>Dynamic connector.66</title> + <path d="M0 164.33 L37.69 164.33" class="st10"/> + </g> + <g id="shape112-69" transform="translate(351.246,-24.9255)"> + <title>Dynamic connector.67</title> + <path d="M0 164.33 L37.69 164.33" class="st8"/> + </g> + <g id="shape113-74" transform="translate(449.306,-24.9255)"> + <title>Dynamic connector.68</title> + <path d="M0 164.33 L37.69 164.33" class="st10"/> + </g> + <g id="shape114-79" transform="translate(547.366,-24.9255)"> + <title>Dynamic connector.69</title> + <path d="M0 164.33 L37.69 164.33" class="st10"/> + </g> + <g id="shape115-84" transform="translate(645.426,-24.9255)"> + <title>Dynamic connector.70</title> + <path d="M0 164.33 L37.69 164.33" class="st8"/> + </g> + <g id="shape116-89" transform="translate(174.599,68.9848) rotate(90)"> + <title>Left Brace</title> + <g id="shadow116-90" transform="matrix(1,0,0,1,1.97279,-0.345598)" class="st1"> + <path d="M28.35 171.41 A24.4921 16.4101 0 0 1 14.17 167.43 L14.17 95.44 L0 95.44 L14.17 95.44 L14.17 23.46 A24.4921 + 16.4101 0 0 1 28.35 19.47" class="st11"/> + </g> + <path d="M28.35 171.41 A24.4921 16.4101 0 0 1 14.17 167.43 L14.17 95.44 L0 95.44 L14.17 95.44 L14.17 23.46 A24.4921 16.4101 + 0 0 1 28.35 19.47" class="st12"/> + </g> + <g id="shape117-97" transform="translate(371.271,68.9848) rotate(90)"> + <title>Left Brace.74</title> + <g id="shadow117-98" transform="matrix(1,0,0,1,1.97279,-0.345598)" class="st1"> + <path d="M28.35 171.41 A23.1398 15.504 0 0 1 14.17 163.51 L14.17 95.44 L0 95.44 L14.17 95.44 L14.17 27.38 A23.1398 + 15.504 0 0 1 28.35 19.47" class="st11"/> + </g> + <path d="M28.35 171.41 A23.1398 15.504 0 0 1 14.17 163.51 L14.17 95.44 L0 95.44 L14.17 95.44 L14.17 27.38 A23.1398 15.504 + 0 0 1 28.35 19.47" class="st12"/> + </g> + <g id="shape118-105" transform="translate(212.048,-117.835)"> + <title>Sheet.118</title> + <desc>Stage 2: Pad packets and enqueue crypto operations</desc> + <rect x="0" y="118.085" width="127.559" height="53.3286" class="st13"/> + <text x="13.85" y="133.95" class="st14">Stage 2: Pad packets <tspan x="14.65" dy="1.2em" class="st5">and enqueue crypto </tspan><tspan + x="37.46" dy="1.2em" class="st5">operations</tspan></text> </g> + <g id="shape119-110" transform="translate(15.3756,-117.835)"> + <title>Sheet.119</title> + <desc>Stage 1: Read packets from port</desc> + <rect x="0" y="118.085" width="127.559" height="53.3286" class="st13"/> + <text x="10.71" y="141.15" class="st14">Stage 1: Read packets <tspan x="40.13" dy="1.2em" class="st5">from port</tspan></text> </g> + <g id="shape120-114" transform="translate(567.943,68.9848) rotate(90)"> + <title>Left Brace.78</title> + <g id="shadow120-115" transform="matrix(1,0,0,1,1.97279,-0.345598)" class="st1"> + <path d="M28.35 171.41 A37.9502 8.92454 0 0 1 14.17 166.08 L14.17 144.75 L0 144.75 L14.17 144.75 L14.17 123.42 A37.9502 + 8.92454 0 0 1 28.35 118.09" class="st11"/> + </g> + <path d="M28.35 171.41 A37.9502 8.92454 0 0 1 14.17 166.08 L14.17 144.75 L0 144.75 L14.17 144.75 L14.17 123.42 A37.9502 + 8.92454 0 0 1 28.35 118.09" class="st12"/> + </g> + <g id="shape121-122" transform="translate(371.106,-117.835)"> + <title>Sheet.121</title> + <desc>Stage 3: Dequeue processed crypto operations</desc> + <rect x="0" y="118.085" width="99.248" height="53.3286" class="st13"/> + <text x="6.71" y="133.95" class="st14">Stage 3: Dequeue <tspan x="7.83" dy="1.2em" class="st5">processed crypto </tspan><tspan + x="23.31" dy="1.2em" class="st5">operations</tspan></text> </g> + <g id="shape122-127" transform="translate(666.003,71.9952) rotate(90)"> + <title>Left Brace.80</title> + <g id="shadow122-128" transform="matrix(1,0,0,1,1.97279,-0.345598)" class="st1"> + <path d="M28.35 171.41 A22.2255 24.1253 0 0 1 14.17 161.02 L14.17 48.34 L0 48.34 L14.17 48.34 L14.17 -64.35 A22.2255 + 24.1253 0 0 1 28.35 -74.74" class="st11"/> + </g> + <path d="M28.35 171.41 A22.2255 24.1253 0 0 1 14.17 161.02 L14.17 48.34 L0 48.34 L14.17 48.34 L14.17 -64.35 A22.2255 + 24.1253 0 0 1 28.35 -74.74" class="st12"/> + </g> + <g id="shape123-135" transform="translate(553.887,-111.814)"> + <title>Sheet.123</title> + <desc>Stage 4: Modify Packet MAC header and transmit</desc> + <rect x="0" y="118.085" width="127.559" height="53.3286" class="st13"/> + <text x="8.01" y="133.95" class="st14">Stage 4: Modify Packet <tspan x="22.85" dy="1.2em" class="st5">MAC header and </tspan><tspan + x="43.12" dy="1.2em" class="st5">transmit </tspan></text> </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_virtenv_benchmark_setup.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_virtenv_benchmark_setup.png Binary files differnew file mode 100644 index 00000000..317df8aa --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_virtenv_benchmark_setup.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_vm2vm.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_vm2vm.svg new file mode 100644 index 00000000..b84dcb27 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/l2_fwd_vm2vm.svg @@ -0,0 +1,311 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="554.46204" + height="443.63278" + viewBox="0 0 554.46204 443.63279" + id="svg3917" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="l2_fwd_vm2vm.svg"> + <defs + id="defs3919"> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker8020" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path8022" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7177" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path7179" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6025" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + id="path6027" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path5351" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5348" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="powerstroke" + id="path-effect4780" + is_visible="true" + offset_points="0,0.5" + sort_points="true" + interpolator_type="Linear" + interpolator_beta="0.2" + start_linecap_type="zerowidth" + linejoin_type="round" + miter_limit="4" + end_linecap_type="zerowidth" + cusp_linecap_type="round" /> + <linearGradient + id="linearGradient4729" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop4731" /> + </linearGradient> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-5" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path5351-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path5351-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path5351-2-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1" + inkscape:cx="323.29803" + inkscape:cy="27.634604" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:snap-nodes="false" + inkscape:snap-bbox="true" + inkscape:window-width="1276" + inkscape:window-height="1400" + inkscape:window-x="1280" + inkscape:window-y="38" + inkscape:window-maximized="0" + units="px" + fit-margin-top="5" + fit-margin-left="5" + fit-margin-right="5" + fit-margin-bottom="5" /> + <metadata + id="metadata3922"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-0.56091356,-0.34416246)"> + <rect + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:2.10537624;stroke-opacity:1" + id="rect4727" + width="542.35669" + height="431.5274" + x="6.6136017" + y="6.3968506" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:30.53249741px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="237.30467" + y="33.252548" + id="text4735" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4737" + x="237.30467" + y="33.252548">Host</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#000000;stroke-opacity:1" + id="rect4739" + width="207.08128" + height="202.03053" + x="38.385803" + y="45.240112" /> + <rect + style="fill:none;fill-opacity:1;stroke:#000000;stroke-opacity:1" + id="rect4739-3" + width="207.08128" + height="202.03053" + x="301.53052" + y="44.22995" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:19.96650314px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="101.13004" + y="63.706543" + id="text4756" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4758" + x="101.13004" + y="63.706543">Guest1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:19.96650314px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="369.73492" + y="63.619873" + id="text4756-6" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4758-7" + x="369.73492" + y="63.619873">Guest2</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#000000;stroke-width:1;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect5336" + width="477.80215" + height="85.862968" + x="39.39595" + y="316.97116" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:23.81648636px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="237.96404" + y="398.79352" + id="text5338" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan5340" + x="237.96404" + y="398.79352">L2FWD</tspan></text> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.9760201px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)" + d="m 120.20815,247.27063 0,68.32236" + id="path5342" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.9760201px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend-5)" + d="m 382.84782,246.56645 0,68.32236" + id="path5342-5" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.9760201px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend-6)" + d="m 162.63455,316.66519 0,-68.32236" + id="path5342-9" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.9760201px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend-6-1)" + d="m 423.25391,315.65504 0,-68.32236" + id="path5342-9-7" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.60951841;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.82855511, 1.60951837;stroke-dashoffset:0;stroke-opacity:1" + d="m 119.48645,319.66266 0,47.47156 303.479,0 0,-51.26929" + id="path10412" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.1137104;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:3.3411313, 1.11371043;stroke-dashoffset:0;stroke-opacity:1" + d="m 162.67537,318.28501 0,31.19206 221.14177,0 0,-33.68743" + id="path10412-0" + inkscape:connector-curvature="0" /> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/load_bal_app_arch.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/load_bal_app_arch.png Binary files differnew file mode 100644 index 00000000..848e48ba --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/load_bal_app_arch.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/overlay_networking.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/overlay_networking.svg new file mode 100644 index 00000000..caf1c264 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/overlay_networking.svg @@ -0,0 +1,1847 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Generated by Microsoft Visio, SVG Export overlay_networking.svg Page-1 --> + +<svg + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="7.4165835in" + height="4.6068053in" + viewBox="0 0 533.99401 331.68998" + xml:space="preserve" + class="st29" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="overlay_networking.svg" + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-rule:evenodd;stroke-linecap:square;stroke-miterlimit:3"><metadata + id="metadata498"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title></dc:title></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="885" + inkscape:window-height="480" + id="namedview496" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:zoom="0.23838384" + inkscape:cx="340.46875" + inkscape:cy="137.425" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="0" + inkscape:current-layer="svg2" /><v:documentProperties + v:langID="1033" + v:viewMarkup="false"><v:userDefs><v:ud + v:nameU="msvSubprocessMaster" + v:prompt="" + v:val="VT4(Rectangle)" /><v:ud + v:nameU="msvNoAutoConnect" + v:val="VT0(1):26" /><v:ud + v:nameU="msvConvertTheme" /></v:userDefs></v:documentProperties><style + type="text/css" + id="style4"><![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#b2b2b2;fill-opacity:0.5;stroke:#b2b2b2;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5} + .st3 {fill:#d8d8d8;fill-opacity:0.7;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st4 {fill:#000000;font-family:Calibri;font-size:1.16666em} + .st5 {fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5} + .st6 {fill:#6b9bc7;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st7 {fill:#000000;font-family:Calibri;font-size:0.833336em} + .st8 {fill:#f6d5b9;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st9 {fill:#000000;font-family:Calibri;font-size:0.75em} + .st10 {fill:url(#grad0-21);stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st11 {fill:#000000;font-family:Calibri;font-size:1.00001em} + .st12 {fill:#00b0f0;fill-opacity:0.8;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st13 {fill:#00b0f0;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st14 {fill:#7030a0;fill-opacity:0.6;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st15 {fill:url(#grad0-121)} + .st16 {stroke:#0070c0;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st17 {fill:url(#grad0-128)} + .st18 {stroke:#d26d19;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st19 {stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5;stroke-width:1.5} + .st20 {stroke:#d26d19;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st21 {stroke:#0070c0;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st22 {stroke:#7030a0;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st23 {stroke:#43365a;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st24 {stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st25 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st26 {fill:url(#grad0-236)} + .st27 {stroke:#7030a0;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st28 {fill:#8fa350;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.5} + .st29 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]></style><defs + id="Patterns_And_Gradients"><linearGradient + id="grad0-21" + x1="-0.13807119" + y1="655.71232" + x2="175.84111" + y2="655.71232" + gradientTransform="scale(1.0499298,0.95244464)" + gradientUnits="userSpaceOnUse"><stop + offset="0" + stop-color="#f0f0f0" + stop-opacity="1" + id="stop8" /><stop + offset="1" + stop-color="#ffffff" + stop-opacity="1" + id="stop10" /></linearGradient><linearGradient + id="grad0-121" + x1="-0.15931553" + y1="5138.0002" + x2="31.958082" + y2="5138.0002" + gradientTransform="scale(6.5279263,0.15318801)" + gradientUnits="userSpaceOnUse"><stop + offset="0" + stop-color="#f0f0f0" + stop-opacity="1" + id="stop13" /><stop + offset="1" + stop-color="#0070c0" + stop-opacity="1" + id="stop15" /></linearGradient><linearGradient + id="grad0-128" + x1="0" + y1="0" + x2="1" + y2="0" + gradientTransform="matrix(0,-1,1,0,0,1)"><stop + offset="0" + stop-color="#f0f0f0" + stop-opacity="1" + id="stop18" /><stop + offset="1" + stop-color="#d26d19" + stop-opacity="1" + id="stop20" /></linearGradient><linearGradient + id="grad0-236" + x1="-0.19200153" + y1="4918.6482" + x2="31.120248" + y2="4918.6482" + gradientTransform="scale(6.2499501,0.16000128)" + gradientUnits="userSpaceOnUse"><stop + offset="0" + stop-color="#f0f0f0" + stop-opacity="1" + id="stop23" /><stop + offset="1" + stop-color="#7030a0" + stop-opacity="1" + id="stop25" /></linearGradient><linearGradient + inkscape:collect="always" + xlink:href="#grad0-128" + id="linearGradient4626" + gradientTransform="scale(7.734852,0.12928496)" + x1="0" + y1="6087.7926" + x2="38.210169" + y2="6087.7926" + gradientUnits="userSpaceOnUse" /><linearGradient + inkscape:collect="always" + xlink:href="#grad0-128" + id="linearGradient4628" + gradientTransform="scale(8.1000686,0.12345574)" + x1="0" + y1="6380.262" + x2="34.992296" + y2="6380.262" + gradientUnits="userSpaceOnUse" /></defs><g + v:mID="0" + v:index="1" + v:groupContext="foregroundPage" + id="g27" + transform="translate(-33.625,-174.25)"><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /><v:ud + v:nameU="msvThemeOrder" + v:val="VT0(0):26" /></v:userDefs><title + id="title29">Page-1</title><v:pageProperties + v:drawingScale="1" + v:pageScale="1" + v:drawingUnits="0" + v:shadowOffsetX="9" + v:shadowOffsetY="-9" /><v:layer + v:name="Flowchart" + v:index="0" /><v:layer + v:name="Connector" + v:index="1" /><g + id="shape3-1" + v:mID="3" + v:groupContext="shape" + transform="translate(85.5,-493.875)"><title + id="title32">Rounded rectangle</title><desc + id="desc34">Host</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="50.5382" + cy="730.125" + width="101.08" + height="123.75" /><g + id="shadow3-2" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 92.08,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-105.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -83.08,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 83.08,0 z" + class="st2" + id="path37" + inkscape:connector-curvature="0" + style="fill:#b2b2b2;fill-opacity:0.5;stroke:#b2b2b2;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 92.08,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-105.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -83.08,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 83.08,0 z" + class="st3" + id="path39" + inkscape:connector-curvature="0" + style="fill:#d8d8d8;fill-opacity:0.7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="37.400002" + y="784.71997" + class="st4" + v:langID="2052" + id="text41" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar />Host</text> +</g><g + id="shape10-6" + v:mID="10" + v:groupContext="shape" + transform="translate(33.75,-569.7)"><title + id="title44">Rounded rectangle.10</title><desc + id="desc46">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="19.4595" + cy="779.85" + width="38.92" + height="24.3" /><g + id="shadow10-7" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 29.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.3 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,767.7 a 9.00007,9.00007 -180 0 0 -9,9 l 0,6.3 a 9.00007,9.00007 -180 0 0 9,9 l 20.92,0 z" + class="st5" + id="path49" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 29.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.3 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,767.7 a 9.00007,9.00007 -180 0 0 -9,9 l 0,6.3 a 9.00007,9.00007 -180 0 0 9,9 l 20.92,0 z" + class="st6" + id="path51" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="12.35" + y="782.84998" + class="st7" + v:langID="2052" + id="text53" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape11-11" + v:mID="11" + v:groupContext="shape" + transform="translate(93.171,-566.775)"><title + id="title56">Rounded rectangle.11</title><desc + id="desc58">VNID 100</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="27.6645" + cy="783.9" + width="55.33" + height="16.2" /><g + id="shadow11-12" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 47.23,792 a 8.10006,8.10006 -180 0 0 8.1,-8.1 8.10006,8.10006 -180 0 0 -8.1,-8.1 l -39.13,0 A 8.10006,8.10006 -180 0 0 0,783.9 8.10006,8.10006 -180 0 0 8.1,792 l 39.13,0 z" + class="st5" + id="path61" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 47.23,792 a 8.10006,8.10006 -180 0 0 8.1,-8.1 8.10006,8.10006 -180 0 0 -8.1,-8.1 l -39.13,0 A 8.10006,8.10006 -180 0 0 0,783.9 8.10006,8.10006 -180 0 0 8.1,792 l 39.13,0 z" + class="st8" + id="path63" + inkscape:connector-curvature="0" + style="fill:#f6d5b9;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="10.44" + y="786.59998" + class="st9" + v:langID="2052" + id="text65" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 100</text> +</g><g + id="shape14-16" + v:mID="14" + v:groupContext="shape" + v:layerMember="0" + transform="translate(204.576,-381.375)"><title + id="title68">Object</title><desc + id="desc70">IP Network</desc><v:custProps><v:cp + v:nameU="Cost" + v:lbl="Cost" + v:type="7" + v:format="@" + v:langID="1033" /><v:cp + v:nameU="Duration" + v:lbl="Duration" + v:type="2" + v:langID="1033" /><v:cp + v:nameU="Resources" + v:lbl="Resources" + v:langID="1033" /></v:custProps><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(1,1,1,1)" + v:tabSpace="42.5197" /><v:textRect + cx="92.25" + cy="708.331" + width="184.5" + height="167.337" /><g + id="shadow14-17" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 26.52,755.72 a 21.2239,24.0427 -180 0 0 30.57,20.35 29.3964,33.3006 -180 0 0 53.17,-7.53 8.89171,10.0726 -180 0 0 9.59,10.87 33.8326,38.3261 -180 0 0 32.49,-15.89 21.0233,23.8155 -180 0 0 22.9,-15.06 36.2371,30.7434 -123.25 0 0 -13.86,-32.32 124.285,106.223 -59.84 0 0 23.08,-31.84 46.9314,46.8568 -180 0 0 -24.31,-42.9 28.0325,27.988 -180 0 0 -53.62,13.39 24.2803,21.6216 -26.25 0 0 -40.25,1.24 34.8986,32.2655 -147.51 0 0 -43.54,45.6 27.2819,27.2385 -180 0 0 3.78,54.09 z" + class="st5" + id="path73" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 26.52,755.72 a 21.2239,24.0427 -180 0 0 30.57,20.35 29.3964,33.3006 -180 0 0 53.17,-7.53 8.89171,10.0726 -180 0 0 9.59,10.87 33.8326,38.3261 -180 0 0 32.49,-15.89 21.0233,23.8155 -180 0 0 22.9,-15.06 36.2371,30.7434 -123.25 0 0 -13.86,-32.32 124.285,106.223 -59.84 0 0 23.08,-31.84 46.9314,46.8568 -180 0 0 -24.31,-42.9 28.0325,27.988 -180 0 0 -53.62,13.39 24.2803,21.6216 -26.25 0 0 -40.25,1.24 34.8986,32.2655 -147.51 0 0 -43.54,45.6 27.2819,27.2385 -180 0 0 3.78,54.09 z" + class="st10" + id="path75" + inkscape:connector-curvature="0" + style="fill:url(#grad0-21);stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="65.139999" + y="711.92999" + class="st11" + v:langID="2052" + id="text77" + style="font-size:12.00012016px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />IP Network</text> +</g><g + id="shape1-23" + v:mID="1" + v:groupContext="shape" + transform="translate(33.75,-521.1)"><title + id="title80">Rounded rectangle.1</title><desc + id="desc82">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="19.4595" + cy="779.85" + width="38.92" + height="24.3" /><g + id="shadow1-24" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 29.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.3 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,767.7 a 9.00007,9.00007 -180 0 0 -9,9 l 0,6.3 a 9.00007,9.00007 -180 0 0 9,9 l 20.92,0 z" + class="st5" + id="path85" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 29.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.3 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,767.7 a 9.00007,9.00007 -180 0 0 -9,9 l 0,6.3 a 9.00007,9.00007 -180 0 0 9,9 l 20.92,0 z" + class="st6" + id="path87" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="12.35" + y="782.84998" + class="st7" + v:langID="2052" + id="text89" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape22-28" + v:mID="22" + v:groupContext="shape" + transform="translate(93.171,-530.325)"><title + id="title92">Rounded rectangle.22</title><desc + id="desc94">VNID 101</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="27.6645" + cy="783.9" + width="55.33" + height="16.2" /><g + id="shadow22-29" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 47.23,792 a 8.10006,8.10006 -180 0 0 8.1,-8.1 8.10006,8.10006 -180 0 0 -8.1,-8.1 l -39.13,0 A 8.10006,8.10006 -180 0 0 0,783.9 8.10006,8.10006 -180 0 0 8.1,792 l 39.13,0 z" + class="st5" + id="path97" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 47.23,792 a 8.10006,8.10006 -180 0 0 8.1,-8.1 8.10006,8.10006 -180 0 0 -8.1,-8.1 l -39.13,0 A 8.10006,8.10006 -180 0 0 0,783.9 8.10006,8.10006 -180 0 0 8.1,792 l 39.13,0 z" + class="st12" + id="path99" + inkscape:connector-curvature="0" + style="fill:#00b0f0;fill-opacity:0.8;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="10.44" + y="786.59998" + class="st9" + v:langID="2052" + id="text101" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 101</text> +</g><g + id="shape23-33" + v:mID="23" + v:groupContext="shape" + transform="translate(85.5,-288)"><title + id="title104">Rounded rectangle.23</title><desc + id="desc106">Host</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="51.75" + cy="733.5" + width="103.5" + height="117" /><g + id="shadow23-34" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 94.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-99 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,675 a 9.00007,9.00007 -180 0 0 -9,9 l 0,99 a 9.00007,9.00007 -180 0 0 9,9 l 85.5,0 z" + class="st2" + id="path109" + inkscape:connector-curvature="0" + style="fill:#b2b2b2;fill-opacity:0.5;stroke:#b2b2b2;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 94.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-99 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,675 a 9.00007,9.00007 -180 0 0 -9,9 l 0,99 a 9.00007,9.00007 -180 0 0 9,9 l 85.5,0 z" + class="st3" + id="path111" + inkscape:connector-curvature="0" + style="fill:#d8d8d8;fill-opacity:0.7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="38.610001" + y="788.09998" + class="st4" + v:langID="2052" + id="text113" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar />Host</text> +</g><g + id="shape24-38" + v:mID="24" + v:groupContext="shape" + transform="translate(33.75,-362.1)"><title + id="title116">Rounded rectangle.24</title><desc + id="desc118">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="20.25" + cy="780.3" + width="40.51" + height="23.4" /><g + id="shadow24-39" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 31.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-5.4 a 9.00007,9.00007 -180 0 0 -9,-9 l -22.5,0 a 9.00007,9.00007 -180 0 0 -9,9 l 0,5.4 a 9.00007,9.00007 -180 0 0 9,9 l 22.5,0 z" + class="st5" + id="path121" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 31.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-5.4 a 9.00007,9.00007 -180 0 0 -9,-9 l -22.5,0 a 9.00007,9.00007 -180 0 0 -9,9 l 0,5.4 a 9.00007,9.00007 -180 0 0 9,9 l 22.5,0 z" + class="st6" + id="path123" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="13.14" + y="783.29999" + class="st7" + v:langID="2052" + id="text125" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape25-43" + v:mID="25" + v:groupContext="shape" + transform="translate(95.4166,-358.2)"><title + id="title128">Rounded rectangle.25</title><desc + id="desc130">VNID 100</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="28.7917" + cy="784.2" + width="57.59" + height="15.6" /><g + id="shadow25-44" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 49.78,792 a 7.80006,7.80006 -180 0 0 7.8,-7.8 7.80006,7.80006 -180 0 0 -7.8,-7.8 l -41.98,0 A 7.80006,7.80006 -180 0 0 0,784.2 7.80006,7.80006 -180 0 0 7.8,792 l 41.98,0 z" + class="st5" + id="path133" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 49.78,792 a 7.80006,7.80006 -180 0 0 7.8,-7.8 7.80006,7.80006 -180 0 0 -7.8,-7.8 l -41.98,0 A 7.80006,7.80006 -180 0 0 0,784.2 7.80006,7.80006 -180 0 0 7.8,792 l 41.98,0 z" + class="st8" + id="path135" + inkscape:connector-curvature="0" + style="fill:#f6d5b9;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="11.57" + y="786.90002" + class="st9" + v:langID="2052" + id="text137" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 100</text> +</g><g + id="shape31-48" + v:mID="31" + v:groupContext="shape" + transform="translate(33.75,-315.3)"><title + id="title140">Rounded rectangle.31</title><desc + id="desc142">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="20.6757" + cy="780.3" + width="41.36" + height="23.4" /><g + id="shadow31-49" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 32.35,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-5.4 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,768.6 a 9.00007,9.00007 -180 0 0 -9,9 l 0,5.4 a 9.00007,9.00007 -180 0 0 9,9 l 23.35,0 z" + class="st5" + id="path145" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 32.35,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-5.4 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,768.6 a 9.00007,9.00007 -180 0 0 -9,9 l 0,5.4 a 9.00007,9.00007 -180 0 0 9,9 l 23.35,0 z" + class="st6" + id="path147" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="13.56" + y="783.29999" + class="st7" + v:langID="2052" + id="text149" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape32-53" + v:mID="32" + v:groupContext="shape" + transform="translate(95.4166,-323.1)"><title + id="title152">Rounded rectangle.32</title><desc + id="desc154">VNID 101</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="28.7917" + cy="784.2" + width="57.59" + height="15.6" /><g + id="shadow32-54" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 49.78,792 a 7.80006,7.80006 -180 0 0 7.8,-7.8 7.80006,7.80006 -180 0 0 -7.8,-7.8 l -41.98,0 A 7.80006,7.80006 -180 0 0 0,784.2 7.80006,7.80006 -180 0 0 7.8,792 l 41.98,0 z" + class="st5" + id="path157" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 49.78,792 a 7.80006,7.80006 -180 0 0 7.8,-7.8 7.80006,7.80006 -180 0 0 -7.8,-7.8 l -41.98,0 A 7.80006,7.80006 -180 0 0 0,784.2 7.80006,7.80006 -180 0 0 7.8,792 l 41.98,0 z" + class="st13" + id="path159" + inkscape:connector-curvature="0" + style="fill:#00b0f0;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="11.57" + y="786.90002" + class="st9" + v:langID="2052" + id="text161" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 101</text> +</g><g + id="shape34-58" + v:mID="34" + v:groupContext="shape" + transform="translate(407.076,-491.625)"><title + id="title164">Rounded rectangle.34</title><desc + id="desc166">Host</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="48.4618" + cy="730.125" + width="96.93" + height="123.75" /><g + id="shadow34-59" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 87.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-105.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -78.92,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 78.92,0 z" + class="st2" + id="path169" + inkscape:connector-curvature="0" + style="fill:#b2b2b2;fill-opacity:0.5;stroke:#b2b2b2;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 87.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-105.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -78.92,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 78.92,0 z" + class="st3" + id="path171" + inkscape:connector-curvature="0" + style="fill:#d8d8d8;fill-opacity:0.7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="35.330002" + y="784.71997" + class="st4" + v:langID="2052" + id="text173" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar />Host</text> +</g><g + id="shape35-63" + v:mID="35" + v:groupContext="shape" + transform="translate(520.049,-580.313)"><title + id="title176">Rounded rectangle.35</title><desc + id="desc178">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="18.8889" + cy="779.625" + width="37.78" + height="24.75" /><g + id="shadow35-64" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st5" + id="path181" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st6" + id="path183" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="11.78" + y="782.63" + class="st7" + v:langID="2052" + id="text185" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape36-68" + v:mID="36" + v:groupContext="shape" + transform="translate(441.076,-582.375)"><title + id="title188">Rounded rectangle.36</title><desc + id="desc190">VNID 100</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="26.4444" + cy="783.75" + width="52.89" + height="16.5" /><g + id="shadow36-69" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st5" + id="path193" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st8" + id="path195" + inkscape:connector-curvature="0" + style="fill:#f6d5b9;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="9.2200003" + y="786.45001" + class="st9" + v:langID="2052" + id="text197" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 100</text> +</g><g + id="shape37-73" + v:mID="37" + v:groupContext="shape" + transform="translate(520.049,-541.125)"><title + id="title200">Rounded rectangle.37</title><desc + id="desc202">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="18.8889" + cy="779.625" + width="37.78" + height="24.75" /><g + id="shadow37-74" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st5" + id="path205" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st6" + id="path207" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="11.78" + y="782.63" + class="st7" + v:langID="2052" + id="text209" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape38-78" + v:mID="38" + v:groupContext="shape" + transform="translate(441.076,-551.438)"><title + id="title212">Rounded rectangle.38</title><desc + id="desc214">VNID 101</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="26.4444" + cy="783.75" + width="52.89" + height="16.5" /><g + id="shadow38-79" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st5" + id="path217" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st12" + id="path219" + inkscape:connector-curvature="0" + style="fill:#00b0f0;fill-opacity:0.8;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="9.2200003" + y="786.45001" + class="st9" + v:langID="2052" + id="text221" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 101</text> +</g><g + id="shape40-83" + v:mID="40" + v:groupContext="shape" + transform="translate(398.076,-288)"><title + id="title224">Rounded rectangle.40</title><desc + id="desc226">Host</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="52.9618" + cy="727.875" + width="105.93" + height="128.25" /><g + id="shadow40-84" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 96.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-110.25 a 9.00007,9.00007 -180 0 0 -9,-9 l -87.92,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 87.92,0 z" + class="st2" + id="path229" + inkscape:connector-curvature="0" + style="fill:#b2b2b2;fill-opacity:0.5;stroke:#b2b2b2;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 96.92,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-110.25 a 9.00007,9.00007 -180 0 0 -9,-9 l -87.92,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 87.92,0 z" + class="st3" + id="path231" + inkscape:connector-curvature="0" + style="fill:#d8d8d8;fill-opacity:0.7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="39.830002" + y="782.46997" + class="st4" + v:langID="2052" + id="text233" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar /><v:newlineChar />Host</text> +</g><g + id="shape41-88" + v:mID="41" + v:groupContext="shape" + transform="translate(522,-369.225)"><title + id="title236">Rounded rectangle.41</title><desc + id="desc238">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="21.1806" + cy="779.175" + width="42.37" + height="25.65" /><g + id="shadow41-89" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 33.36,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-7.65 a 9.00007,9.00007 -180 0 0 -9,-9 l -24.36,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 24.36,0 z" + class="st5" + id="path241" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 33.36,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-7.65 a 9.00007,9.00007 -180 0 0 -9,-9 l -24.36,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 24.36,0 z" + class="st6" + id="path243" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="14.07" + y="782.16998" + class="st7" + v:langID="2052" + id="text245" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape42-93" + v:mID="42" + v:groupContext="shape" + transform="translate(436.201,-369)"><title + id="title248">Rounded rectangle.42</title><desc + id="desc250">VNID 100</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="29.6528" + cy="783.45" + width="59.31" + height="17.1" /><g + id="shadow42-94" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 50.76,792 a 8.55007,8.55007 -180 0 0 8.55,-8.55 8.55007,8.55007 -180 0 0 -8.55,-8.55 l -42.21,0 A 8.55007,8.55007 -180 0 0 0,783.45 8.55007,8.55007 -180 0 0 8.55,792 l 42.21,0 z" + class="st5" + id="path253" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 50.76,792 a 8.55007,8.55007 -180 0 0 8.55,-8.55 8.55007,8.55007 -180 0 0 -8.55,-8.55 l -42.21,0 A 8.55007,8.55007 -180 0 0 0,783.45 8.55007,8.55007 -180 0 0 8.55,792 l 42.21,0 z" + class="st8" + id="path255" + inkscape:connector-curvature="0" + style="fill:#f6d5b9;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="12.43" + y="786.15002" + class="st9" + v:langID="2052" + id="text257" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 100</text> +</g><g + id="shape43-98" + v:mID="43" + v:groupContext="shape" + transform="translate(523.319,-317.925)"><title + id="title260">Rounded rectangle.43</title><desc + id="desc262">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="21.1806" + cy="779.175" + width="42.37" + height="25.65" /><g + id="shadow43-99" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 33.36,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-7.65 a 9.00007,9.00007 -180 0 0 -9,-9 l -24.36,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 24.36,0 z" + class="st5" + id="path265" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 33.36,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-7.65 a 9.00007,9.00007 -180 0 0 -9,-9 l -24.36,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 24.36,0 z" + class="st6" + id="path267" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="14.07" + y="782.16998" + class="st7" + v:langID="2052" + id="text269" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape44-103" + v:mID="44" + v:groupContext="shape" + transform="translate(436.201,-324.9)"><title + id="title272">Rounded rectangle.44</title><desc + id="desc274">VNID 102</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="29.6528" + cy="783.45" + width="59.31" + height="17.1" /><g + id="shadow44-104" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 50.76,792 a 8.55007,8.55007 -180 0 0 8.55,-8.55 8.55007,8.55007 -180 0 0 -8.55,-8.55 l -42.21,0 A 8.55007,8.55007 -180 0 0 0,783.45 8.55007,8.55007 -180 0 0 8.55,792 l 42.21,0 z" + class="st5" + id="path277" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 50.76,792 a 8.55007,8.55007 -180 0 0 8.55,-8.55 8.55007,8.55007 -180 0 0 -8.55,-8.55 l -42.21,0 A 8.55007,8.55007 -180 0 0 0,783.45 8.55007,8.55007 -180 0 0 8.55,792 l 42.21,0 z" + class="st14" + id="path279" + inkscape:connector-curvature="0" + style="fill:#7030a0;fill-opacity:0.6;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="12.43" + y="786.15002" + class="st9" + v:langID="2052" + id="text281" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 102</text> +</g><g + id="shape46-108" + v:mID="46" + v:groupContext="shape" + transform="translate(441.076,-518.438)"><title + id="title284">Rounded rectangle.46</title><desc + id="desc286">VNID 102</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="26.4444" + cy="783.75" + width="52.89" + height="16.5" /><g + id="shadow46-109" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st5" + id="path289" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 44.64,792 a 8.25006,8.25006 -180 0 0 8.25,-8.25 8.25006,8.25006 -180 0 0 -8.25,-8.25 l -36.39,0 A 8.25006,8.25006 -180 0 0 0,783.75 8.25006,8.25006 -180 0 0 8.25,792 l 36.39,0 z" + class="st14" + id="path291" + inkscape:connector-curvature="0" + style="fill:#7030a0;fill-opacity:0.6;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="9.2200003" + y="786.45001" + class="st9" + v:langID="2052" + id="text293" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VNID 102</text> +</g><g + id="shape47-113" + v:mID="47" + v:groupContext="shape" + transform="translate(520.049,-501.937)"><title + id="title296">Rounded rectangle.47</title><desc + id="desc298">VM</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="18.8889" + cy="779.625" + width="37.78" + height="24.75" /><g + id="shadow47-114" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st5" + id="path301" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 28.78,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-6.75 a 9.00007,9.00007 -180 0 0 -9,-9 l -19.78,0 a 9.00007,9.00007 -180 0 0 -9,9 L 0,783 a 9.00007,9.00007 -180 0 0 9,9 l 19.78,0 z" + class="st6" + id="path303" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="11.78" + y="782.63" + class="st7" + v:langID="2052" + id="text305" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM</text> +</g><g + id="shape48-118" + v:mID="48" + v:groupContext="shape" + transform="matrix(0.02167878,0.99976499,-0.99976499,0.02167878,940.314,236.405)"><title + id="title308">Arced arrow</title><v:userDefs><v:ud + v:nameU="ControlX" + v:val="VT0(0.125):0" /><v:ud + v:nameU="Scale" + v:val="VT0(1):26" /><v:ud + v:nameU="AntiScale" + v:val="VT0(1):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="ControlX" + v:prompt="" + v:val="VT0(0.0625):1" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><path + d="M 1.13,787.65 3.31,788.21 0,792 l -1.04,-4.92 2.17,0.57 z m 205.31,0 -2.18,0.56 3.31,3.79 1.05,-4.92 -2.18,0.57 z" + class="st15" + id="path310" + inkscape:connector-curvature="0" + style="fill:url(#grad0-121)" /><path + d="m 1.13,787.65 a 106.681,106.681 0 0 1 205.31,0" + class="st16" + id="path312" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="M 1.13,787.65 3.31,788.21 0,792 l -1.04,-4.92 2.17,0.57" + class="st16" + id="path314" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="m 206.44,787.65 -2.18,0.56 3.31,3.79 1.05,-4.92 -2.18,0.57" + class="st16" + id="path316" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape49-125" + v:mID="49" + v:groupContext="shape" + transform="matrix(-0.99814331,0.06090917,-0.06090917,-0.99814331,489.312,991.905)"><title + id="title319">Arced arrow.49</title><v:userDefs><v:ud + v:nameU="ControlX" + v:val="VT0(0.125):0" /><v:ud + v:nameU="Scale" + v:val="VT0(1):26" /><v:ud + v:nameU="AntiScale" + v:val="VT0(1):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="ControlX" + v:prompt="" + v:val="VT0(0.0625):1" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><path + d="M 2.72,788.42 4.51,789.78 0,792 l 0.93,-4.94 1.79,1.36 z m 290.11,0 -1.8,1.36 4.52,2.22 -0.93,-4.94 -1.79,1.36 z" + class="st17" + id="path321" + inkscape:connector-curvature="0" + style="fill:url(#linearGradient4626)" /><path + d="m 2.72,788.42 a 183.886,183.886 0 0 1 290.11,0" + class="st18" + id="path323" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="M 2.72,788.42 4.51,789.78 0,792 l 0.93,-4.94 1.79,1.36" + class="st18" + id="path325" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="m 292.83,788.42 -1.8,1.36 4.52,2.22 -0.93,-4.94 -1.79,1.36" + class="st18" + id="path327" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape50-132" + v:mID="50" + v:groupContext="shape" + transform="matrix(0.99916938,-0.04074983,0.04074983,0.99916938,120.726,-365.342)"><title + id="title330">Arced arrow.50</title><v:userDefs><v:ud + v:nameU="ControlX" + v:val="VT0(0.125):0" /><v:ud + v:nameU="Scale" + v:val="VT0(1):26" /><v:ud + v:nameU="AntiScale" + v:val="VT0(1):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="ControlX" + v:prompt="" + v:val="VT0(0.0625):1" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><path + d="m 3.79,789.57 1.21,1.9 -5,0.53 2.58,-4.32 1.21,1.89 z m 275.86,0 -1.22,1.9 5.01,0.53 -2.58,-4.32 -1.21,1.89 z" + class="st17" + id="path332" + inkscape:connector-curvature="0" + style="fill:url(#linearGradient4628)" /><path + d="m 3.79,789.57 a 259.249,259.249 0 0 1 275.86,0" + class="st18" + id="path334" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="m 3.79,789.57 1.21,1.9 -5,0.53 2.58,-4.32 1.21,1.89" + class="st18" + id="path336" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="m 279.65,789.57 -1.22,1.9 5.01,0.53 -2.58,-4.32 -1.21,1.89" + class="st18" + id="path338" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape4-138" + v:mID="4" + v:groupContext="shape" + transform="matrix(0.93831753,0.34577481,-0.34577481,0.93831753,348.104,-324.947)"><title + id="title341">Sheet.4</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow4-139" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.84909,0.853261)" + class="st1" + style="visibility:visible"><path + d="m 0,792 22.56,0" + class="st19" + id="path344" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 22.56,0" + class="st20" + id="path346" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape5-145" + v:mID="5" + v:groupContext="shape" + transform="matrix(0.98206713,-0.18853156,0.18853156,0.98206713,-74.2157,-312.797)"><title + id="title349">Sheet.5</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow5-146" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.14269,1.68566)" + class="st1" + style="visibility:visible"><path + d="m 0,792 20.69,0" + class="st19" + id="path352" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 20.69,0" + class="st21" + id="path354" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape6-152" + v:mID="6" + v:groupContext="shape" + transform="matrix(0.94671224,0.32208064,-0.32208064,0.94671224,327.757,-539.646)"><title + id="title357">Sheet.6</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow6-153" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.82706,0.89947)" + class="st1" + style="visibility:visible"><path + d="m 0,792 21.66,0" + class="st19" + id="path360" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 21.66,0" + class="st20" + id="path362" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape7-159" + v:mID="7" + v:groupContext="shape" + transform="matrix(0.96958947,-0.24473714,0.24473714,0.96958947,-121.163,-509.165)"><title + id="title365">Sheet.7</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow7-160" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.04379,1.74863)" + class="st1" + style="visibility:visible"><path + d="m 0,792 21.15,0" + class="st19" + id="path368" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 21.15,0" + class="st21" + id="path370" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape8-166" + v:mID="8" + v:groupContext="shape" + transform="matrix(0.99688828,-0.07882744,0.07882744,0.99688828,431.534,-588.161)"><title + id="title373">Sheet.8</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow8-167" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.32201,1.54903)" + class="st1" + style="visibility:visible"><path + d="m 0,792 26.16,0" + class="st19" + id="path376" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 26.16,0" + class="st20" + id="path378" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape13-173" + v:mID="13" + v:groupContext="shape" + transform="matrix(0.97299749,0.2308157,-0.2308157,0.97299749,676.771,-538.302)"><title + id="title381">Sheet.13</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow13-174" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.73349,1.06874)" + class="st1" + style="visibility:visible"><path + d="m 0,792 26.81,0" + class="st19" + id="path384" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 26.81,0" + class="st21" + id="path386" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape15-180" + v:mID="15" + v:groupContext="shape" + transform="matrix(0.90347299,0.42864501,-0.42864501,0.90347299,833.452,-450.238)"><title + id="title389">Sheet.15</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow15-181" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.91825,0.683753)" + class="st1" + style="visibility:visible"><path + d="m 0,792 28.87,0" + class="st19" + id="path392" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 28.87,0" + class="st22" + id="path394" + inkscape:connector-curvature="0" + style="stroke:#7030a0;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape16-187" + v:mID="16" + v:groupContext="shape" + transform="matrix(0.98587931,-0.16745741,0.16745741,0.98587931,362.881,-366.366)"><title + id="title397">Sheet.16</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow16-188" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.17853,1.6608)" + class="st1" + style="visibility:visible"><path + d="m 0,792 26.87,0" + class="st19" + id="path400" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 26.87,0" + class="st20" + id="path402" + inkscape:connector-curvature="0" + style="stroke:#d26d19;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape17-194" + v:mID="17" + v:groupContext="shape" + transform="matrix(0.99532092,0.09662438,-0.09662438,0.99532092,572.033,-329.744)"><title + id="title405">Sheet.17</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow17-195" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.5724,1.29412)" + class="st1" + style="visibility:visible"><path + d="m 0,792 27.94,0" + class="st19" + id="path408" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 27.94,0" + class="st23" + id="path410" + inkscape:connector-curvature="0" + style="stroke:#43365a;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape18-201" + v:mID="18" + v:groupContext="shape" + transform="matrix(0.56466455,-0.82532051,0.82532051,0.56466455,-464.654,-1.71391)"><title + id="title413">Sheet.18</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow18-202" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(-0.375346,2.00158)" + class="st1" + style="visibility:visible"><path + d="m 0,792 71.36,0" + class="st19" + id="path416" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 71.36,0" + class="st24" + id="path418" + inkscape:connector-curvature="0" + style="stroke:#404040;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape19-208" + v:mID="19" + v:groupContext="shape" + transform="matrix(0.46159689,0.8870898,-0.8870898,0.46159689,1074.07,13.0882)"><title + id="title421">Sheet.19</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow19-209" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.94211,-0.612711)" + class="st1" + style="visibility:visible"><path + d="m 0,792 57.58,0" + class="st19" + id="path424" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 57.58,0" + class="st24" + id="path426" + inkscape:connector-curvature="0" + style="stroke:#404040;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape20-215" + v:mID="20" + v:groupContext="shape" + transform="matrix(0.57372226,0.81904992,-0.81904992,0.57372226,835.264,-218.138)"><title + id="title429">Sheet.20</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow20-216" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(2.00559,-0.353273)" + class="st1" + style="visibility:visible"><path + d="m 0,792 62.72,0" + class="st19" + id="path432" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 62.72,0" + class="st25" + id="path434" + inkscape:connector-curvature="0" + style="stroke:#000000;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape21-222" + v:mID="21" + v:groupContext="shape" + transform="matrix(-0.68364325,0.72981635,-0.72981635,-0.68364325,985.087,779.949)"><title + id="title437">Sheet.21</title><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow21-223" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(0.0664758,-2.03538)" + class="st1" + style="visibility:visible"><path + d="m 0,792 43.41,0" + class="st19" + id="path440" + inkscape:connector-curvature="0" + style="stroke:#cdcdcd;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,792 43.41,0" + class="st24" + id="path442" + inkscape:connector-curvature="0" + style="stroke:#404040;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape68-229" + v:mID="68" + v:groupContext="shape" + transform="matrix(-0.78308598,0.62191345,-0.62191345,-0.78308598,933.635,852.514)"><title + id="title445">Arced arrow.68</title><v:userDefs><v:ud + v:nameU="ControlX" + v:val="VT0(0.125):0" /><v:ud + v:nameU="Scale" + v:val="VT0(1):26" /><v:ud + v:nameU="AntiScale" + v:val="VT0(1):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="ControlX" + v:prompt="" + v:val="VT0(0):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><path + d="m 0,792 a 329.322,329.322 0 0 1 367.87,0" + class="st16" + id="path447" + inkscape:connector-curvature="0" + style="stroke:#0070c0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape70-233" + v:mID="70" + v:groupContext="shape" + transform="matrix(0.02522082,-0.9996819,0.9996819,0.02522082,-355.547,438.576)"><title + id="title450">Arced arrow.70</title><v:userDefs><v:ud + v:nameU="ControlX" + v:val="VT0(0.125):0" /><v:ud + v:nameU="Scale" + v:val="VT0(1):26" /><v:ud + v:nameU="AntiScale" + v:val="VT0(1):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="ControlX" + v:prompt="" + v:val="VT0(0.06400492):1" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><path + d="M 1.04,787.51 3.29,788.03 0,792 l -1.2,-5.01 2.24,0.52 z m 191.22,0 -2.25,0.52 3.29,3.97 1.2,-5.01 -2.24,0.52 z" + class="st26" + id="path452" + inkscape:connector-curvature="0" + style="fill:url(#grad0-236)" /><path + d="m 1.04,787.51 a 98.7059,98.7059 0 0 1 191.22,0" + class="st27" + id="path454" + inkscape:connector-curvature="0" + style="stroke:#7030a0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="M 1.04,787.51 3.29,788.03 0,792 l -1.2,-5.01 2.24,0.52" + class="st27" + id="path456" + inkscape:connector-curvature="0" + style="stroke:#7030a0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /><path + d="m 192.26,787.51 -2.25,0.52 3.29,3.97 1.2,-5.01 -2.24,0.52" + class="st27" + id="path458" + inkscape:connector-curvature="0" + style="stroke:#7030a0;stroke-width:1;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape84-240" + v:mID="84" + v:groupContext="shape" + transform="translate(157.5,-510.75)"><title + id="title461">Sheet.84</title><desc + id="desc463">TEP</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="11.25" + cy="753.75" + width="22.5" + height="76.5" /><rect + x="0" + y="715.5" + width="22.5" + height="76.5" + class="st28" + id="rect465" + style="fill:#8fa350;stroke:#404040;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round" /><text + x="4.54" + y="756.45001" + class="st9" + v:langID="2052" + id="text467" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TEP</text> +</g><g + id="shape86-243" + v:mID="86" + v:groupContext="shape" + transform="translate(162,-315)"><title + id="title470">Sheet.86</title><desc + id="desc472">TEP</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="11.8125" + cy="753.75" + width="23.63" + height="76.5" /><rect + x="0" + y="715.5" + width="23.625" + height="76.5" + class="st28" + id="rect474" + style="fill:#8fa350;stroke:#404040;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round" /><text + x="5.0999999" + y="756.45001" + class="st9" + v:langID="2052" + id="text476" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TEP</text> +</g><g + id="shape87-246" + v:mID="87" + v:groupContext="shape" + transform="translate(405,-324)"><title + id="title479">Sheet.87</title><desc + id="desc481">TEP</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="11.8125" + cy="753.75" + width="23.63" + height="76.5" /><rect + x="0" + y="715.5" + width="23.625" + height="76.5" + class="st28" + id="rect483" + style="fill:#8fa350;stroke:#404040;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round" /><text + x="5.0999999" + y="756.45001" + class="st9" + v:langID="2052" + id="text485" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TEP</text> +</g><g + id="shape88-249" + v:mID="88" + v:groupContext="shape" + transform="translate(411.75,-513)"><title + id="title488">Sheet.88</title><desc + id="desc490">TEP</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="11.25" + cy="754.875" + width="22.5" + height="74.25" /><rect + x="0" + y="717.75" + width="22.5" + height="74.25" + class="st28" + id="rect492" + style="fill:#8fa350;stroke:#404040;stroke-width:0.5;stroke-linecap:round;stroke-linejoin:round" /><text + x="4.54" + y="757.58002" + class="st9" + v:langID="2052" + id="text494" + style="font-size:9px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TEP</text> +</g></g></svg>
\ No newline at end of file diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_1.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_1.svg new file mode 100644 index 00000000..db01d7c2 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_1.svg @@ -0,0 +1,799 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="449.57141" + height="187.34319" + viewBox="0 0 449.57143 187.34319" + id="svg2" + version="1.1" + inkscape:version="0.48.3.1 r9886" + sodipodi:docname="performance_thread_1.svg" + inkscape:export-filename="C:\Users\tkulasex\Documents\L-threads\model-v2.png" + inkscape:export-xdpi="90" + inkscape:export-ydpi="90"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker11487" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11489" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker11285" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11287" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker11107" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11109" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10757" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10759" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10421" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10423" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10273" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10275" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker9983" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path9985" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker9853" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path9855" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4248-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker4992-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path4994-2" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4254-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker4992-4-0" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path4994-2-9" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6-8" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4248-0-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5952-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path5954-4" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5952-2-1" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path5954-4-2" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6881-5" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path6883-0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-0" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-0-4" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-2-4" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10119-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend" + inkscape:collect="always"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10121-6" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker11487-0" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11489-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10585" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path10587" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10273-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10275-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10421-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10423-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-2" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10119" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend" + inkscape:collect="always"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10121" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10923" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend" + inkscape:collect="always"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10925" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10757-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10759-3" + inkscape:connector-curvature="0" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4" + inkscape:cx="138.23152" + inkscape:cy="-30.946457" + inkscape:document-units="px" + inkscape:current-layer="g4142-7" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:window-width="1920" + inkscape:window-height="1148" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" + width="744.09px" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-40.428564,-78.569476)"> + <g + transform="translate(7.9156519e-7,106.78572)" + id="g4142-7"> + <g + transform="translate(162.14285,0.35714094)" + id="g4177-1"> + <g + transform="translate(-160.49999,-56.592401)" + id="g4142-55-1"> + <rect + y="43.076488" + x="39.285713" + height="65" + width="38.57143" + id="rect4136-65-2" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-4-8" + y="62.447506" + x="-95.515633" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="62.447506" + x="-95.515633" + id="tspan4140-2-4" + sodipodi:role="line">Port 1</tspan></text> + </g> + <rect + y="93.269798" + x="-121.21429" + height="65" + width="38.57143" + id="rect4136-8-3-7" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 1.00000002;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-8-7-3" + y="-98.052498" + x="-145.70891" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="-98.052498" + x="-145.70891" + id="tspan4140-5-8-3" + sodipodi:role="line">Port 2</tspan></text> + <g + transform="translate(-158.35713,1.6218895)" + id="g4177-7-6"> + <rect + y="1.2907723" + x="132.85715" + height="46.42857" + width="94.285713" + id="rect4171-1-9" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + sodipodi:linespacing="125%" + id="text4173-0-0" + y="29.147915" + x="146.42856" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="29.147915" + x="146.42856" + id="tspan4175-6-1" + sodipodi:role="line">rx-thread</tspan></text> + </g> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="86.642853" + y="78.626976" + id="text5627-0-5" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan5629-8-6" + x="86.642853" + y="78.626976">rings</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10757)" + d="m -83.357144,17.912679 56.42858,4.28571" + id="path4239-3-5" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10923)" + d="m -82.808124,125.71821 53.57145,-9.28573" + id="path4239-0-3-6" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker10119)" + d="m 68.78571,29.341249 62.5,28.21429" + id="path5457-1-2" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + transform="translate(-161.92858,95.100119)" + id="g4177-7-6-7"> + <rect + y="1.2907723" + x="132.85715" + height="46.42857" + width="94.285713" + id="rect4171-1-9-8" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + sodipodi:linespacing="125%" + id="text4173-0-0-6" + y="29.147915" + x="146.42856" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="29.147915" + x="146.42856" + id="tspan4175-6-1-8" + sodipodi:role="line">rx-thread</tspan></text> + </g> + <g + transform="translate(249.5,-71.149881)" + id="g4142-5-1-2"> + <rect + y="43.076488" + x="39.285713" + height="65" + width="38.57143" + id="rect4136-6-5-3" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-3-3-5" + y="62.447506" + x="-95.515633" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="62.447506" + x="-95.515633" + id="tspan4140-7-3-5" + sodipodi:role="line">Port 1</tspan></text> + </g> + <rect + y="74.426659" + x="288.07141" + height="65" + width="38.57143" + id="rect4136-8-4-7-7" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 1.00000002;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-8-2-5-8" + y="311.23318" + x="-126.86578" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="311.23318" + x="-126.86578" + id="tspan4140-5-4-9-6" + sodipodi:role="line">Port 2</tspan></text> + <g + id="g5905-4" + transform="translate(-1.2142913,-215.16774)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431)" + d="M 226.28573,52.462339 287.7143,2.8194795" + id="path4984-4-07" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10421)" + d="m 227.09388,122.75669 60.35714,9.64286" + id="path4984-1-6-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + id="g5905-6-0" + transform="translate(0.21427875,-156.1499)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-9-1" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-0-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-7-8" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10273)" + d="m 227.19687,67.801919 58.92857,41.071411" + id="path4984-4-0-4" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10585)" + d="M 227.30382,110.24508 286.94667,24.530799" + id="path4984-4-0-0-7" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker11487)" + d="m 66.28572,118.8909 65.71429,-2.14285" + id="path5457-1-2-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + id="g5905-4-6" + transform="translate(-3.5000113,-277.43173)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker10119-2)" + d="M 68.35772,16.118199 127.64343,-6.3818105" + id="path5457-1-2-2" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431-3)" + d="m 224.52079,-13.531251 64.28571,2.14286" + id="path4984-4-07-4" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431-3-0)" + d="M 224.17025,2.1505695 287.02739,87.864849" + id="path4984-4-07-4-7" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + </g> + </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_2.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_2.svg new file mode 100644 index 00000000..48cf8338 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/performance_thread_2.svg @@ -0,0 +1,865 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="449.57141" + height="187.34319" + viewBox="0 0 449.57143 187.34319" + id="svg2" + version="1.1" + inkscape:version="0.48.3.1 r9886" + sodipodi:docname="performance_thread_2.svg" + inkscape:export-filename="C:\Users\tkulasex\Documents\L-threads\model-v2.png" + inkscape:export-xdpi="90" + inkscape:export-ydpi="90"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker11487" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11489" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker11285" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11287" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker11107" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11109" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10757" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10759" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10421" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10423" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10273" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10275" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker9983" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path9985" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker9853" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path9855" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4248-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker4992-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path4994-2" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4254-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker4992-4-0" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path4994-2-9" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6-8" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path4248-0-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5952-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path5954-4" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5952-2-1" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path5954-4-2" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6881-5" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + inkscape:connector-curvature="0" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path6883-0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-0" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-0-4" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-2-4" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-3-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-4-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10119-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend" + inkscape:collect="always"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10121-6" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker11487-0" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path11489-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10585" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path10587" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10273-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10275-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10421-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10423-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker10431-2" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path10433-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10119" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Mend" + inkscape:collect="always"> + <path + transform="matrix(-0.4,0,0,-0.4,-4,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10121" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10923" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend" + inkscape:collect="always"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10925" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker10757-4" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow1Lend"> + <path + transform="matrix(-0.8,0,0,-0.8,-10,0)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + id="path10759-3" + inkscape:connector-curvature="0" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4" + inkscape:cx="138.23152" + inkscape:cy="-30.946457" + inkscape:document-units="px" + inkscape:current-layer="g4177-1" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:window-width="1920" + inkscape:window-height="1148" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" + width="744.09px" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-40.428564,-78.569476)"> + <g + transform="translate(7.9156519e-7,106.78572)" + id="g4142-7"> + <g + transform="translate(162.14285,0.35714094)" + id="g4177-1"> + <g + transform="translate(-160.49999,-56.592401)" + id="g4142-55-1"> + <rect + y="43.076488" + x="39.285713" + height="65" + width="38.57143" + id="rect4136-65-2" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-4-8" + y="62.447506" + x="-95.515633" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="62.447506" + x="-95.515633" + id="tspan4140-2-4" + sodipodi:role="line">Port 1</tspan></text> + </g> + <rect + y="93.269798" + x="-121.21429" + height="65" + width="38.57143" + id="rect4136-8-3-7" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 1.00000002;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-8-7-3" + y="-98.052498" + x="-145.70891" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="-98.052498" + x="-145.70891" + id="tspan4140-5-8-3" + sodipodi:role="line">Port 2</tspan></text> + <g + transform="translate(-158.35713,1.6218895)" + id="g4177-7-6"> + <rect + y="1.2907723" + x="132.85715" + height="46.42857" + width="94.285713" + id="rect4171-1-9" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + sodipodi:linespacing="125%" + id="text4173-0-0" + y="29.147915" + x="146.42856" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="29.147915" + x="146.42856" + id="tspan4175-6-1" + sodipodi:role="line">rx-thread</tspan></text> + </g> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="86.642853" + y="78.626976" + id="text5627-0-5" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan5629-8-6" + x="86.642853" + y="78.626976">rings</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10757)" + d="m -83.357144,17.912679 56.42858,4.28571" + id="path4239-3-5" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10923)" + d="m -82.808124,125.71821 53.57145,-9.28573" + id="path4239-0-3-6" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker10119)" + d="m 68.78571,29.341249 62.5,28.21429" + id="path5457-1-2" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + transform="translate(-161.92858,95.100119)" + id="g4177-7-6-7"> + <rect + y="1.2907723" + x="132.85715" + height="46.42857" + width="94.285713" + id="rect4171-1-9-8" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + sodipodi:linespacing="125%" + id="text4173-0-0-6" + y="29.147915" + x="146.42856" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="29.147915" + x="146.42856" + id="tspan4175-6-1-8" + sodipodi:role="line">rx-thread</tspan></text> + </g> + <g + transform="translate(249.5,-71.149881)" + id="g4142-5-1-2"> + <rect + y="43.076488" + x="39.285713" + height="65" + width="38.57143" + id="rect4136-6-5-3" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-3-3-5" + y="62.447506" + x="-95.515633" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="62.447506" + x="-95.515633" + id="tspan4140-7-3-5" + sodipodi:role="line">Port 1</tspan></text> + </g> + <rect + y="74.426659" + x="288.07141" + height="65" + width="38.57143" + id="rect4136-8-4-7-7" + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 1.00000002;stroke-dashoffset:0" /> + <text + transform="matrix(0,-1,1,0,0,0)" + sodipodi:linespacing="125%" + id="text4138-8-2-5-8" + y="311.23318" + x="-126.86578" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + y="311.23318" + x="-126.86578" + id="tspan4140-5-4-9-6" + sodipodi:role="line">Port 2</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431)" + d="M 226.28573,52.462339 287.7143,2.8194795" + id="path4984-4-07" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10421)" + d="m 227.09388,122.75669 60.35714,9.64286" + id="path4984-1-6-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10273)" + d="m 227.19687,67.801919 58.92857,41.071411" + id="path4984-4-0-4" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10585)" + d="M 228.01811,113.10222 287.66096,27.387942" + id="path4984-4-0-0-7" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker11487)" + d="m 66.28572,118.8909 65.71429,-2.14285" + id="path5457-1-2-8" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + id="g5905-4-6" + transform="matrix(1,0,0,0.48279909,-0.64286832,-142.16523)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4.00000008, 2.00000004;stroke-dashoffset:0;marker-end:url(#marker10119-2)" + d="M 68.35772,16.118199 127.64343,-6.3818105" + id="path5457-1-2-2" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431-3)" + d="m 224.52079,-13.531251 64.28571,2.14286" + id="path4984-4-07-4" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker10431-3-0)" + d="M 224.17025,2.1505695 287.02739,87.864849" + id="path4984-4-07-4-7" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <g + id="g5905-4-6-5" + transform="matrix(1,0,0,0.45244466,-0.99999222,-110.73112)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7-6" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8-7" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5-0" + x="146.42856" + y="278.34433">tx-drain</tspan></text> + </g> + <g + id="g5905-4-6-2" + transform="matrix(1,0,0,0.48279909,1.3158755,-80.292458)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7-8" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5-6" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <g + id="g5905-4-6-5-9" + transform="matrix(1,0,0,0.45244466,0.95875552,-48.858358)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7-6-6" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8-7-4" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5-0-0" + x="146.42856" + y="278.34433">tx-drain</tspan></text> + </g> + <g + id="g5905-4-6-6" + transform="matrix(1,0,0,0.48279909,1.315876,-24.578174)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7-3" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8-78" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5-9" + x="146.42856" + y="278.34433">tx-thread</tspan></text> + </g> + <g + id="g5905-4-6-5-0" + transform="matrix(1,0,0,0.45244466,0.958756,6.8559263)"> + <rect + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:4, 1;stroke-dashoffset:0" + id="rect4171-9-0-0-7-6-0" + width="94.285713" + height="46.42857" + x="132.85715" + y="250.48721" /> + <text + xml:space="preserve" + style="font-size:15px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="146.42856" + y="278.34433" + id="text4173-9-2-6-8-7-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4175-0-7-3-5-0-3" + x="146.42856" + y="278.34433">tx-drain</tspan></text> + </g> + </g> + </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/pipeline_overview.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/pipeline_overview.png Binary files differnew file mode 100644 index 00000000..21820367 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/pipeline_overview.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/ptpclient.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ptpclient.svg new file mode 100644 index 00000000..fd78ef83 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ptpclient.svg @@ -0,0 +1,528 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="88.134132mm" + height="96.380394mm" + viewBox="0 0 312.28629 341.50533" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="ptpclient.svg"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow2Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow2Mend" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4256" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="scale(-0.6,-0.6)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lstart" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4247" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6746" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Lend"> + <path + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path6748" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker6526" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path6528" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker6276" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path6278" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6128" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="DotM"> + <path + transform="matrix(0.4,0,0,0.4,2.96,0.4)" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="m -2.5,-1 c 0,2.76 -2.24,5 -5,5 -2.76,0 -5,-2.24 -5,-5 0,-2.76 2.24,-5 5,-5 2.76,0 5,2.24 5,5 z" + id="path6130" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker6022" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Lend" + inkscape:collect="always"> + <path + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path6024" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5922" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="DotM"> + <path + transform="matrix(0.4,0,0,0.4,2.96,0.4)" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="m -2.5,-1 c 0,2.76 -2.24,5 -5,5 -2.76,0 -5,-2.24 -5,-5 0,-2.76 2.24,-5 5,-5 2.76,0 5,2.24 5,5 z" + id="path5924" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lend" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path4250" + style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="DotM" + orient="auto" + refY="0" + refX="0" + id="marker5676" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path5678" + d="m -2.5,-1 c 0,2.76 -2.24,5 -5,5 -2.76,0 -5,-2.24 -5,-5 0,-2.76 2.24,-5 5,-5 2.76,0 5,2.24 5,5 z" + style="fill:#0000ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.4,0,0,0.4,2.96,0.4)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker5600" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5602" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5510" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="DotM" + inkscape:collect="always"> + <path + transform="matrix(0.4,0,0,0.4,2.96,0.4)" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="m -2.5,-1 c 0,2.76 -2.24,5 -5,5 -2.76,0 -5,-2.24 -5,-5 0,-2.76 2.24,-5 5,-5 2.76,0 5,2.24 5,5 z" + id="path5512" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker4651" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4653" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker4605" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4607" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4232" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="DotM" + orient="auto" + refY="0" + refX="0" + id="DotM" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path4293" + d="m -2.5,-1 c 0,2.76 -2.24,5 -5,5 -2.76,0 -5,-2.24 -5,-5 0,-2.76 2.24,-5 5,-5 2.76,0 5,2.24 5,5 z" + style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.4,0,0,0.4,2.96,0.4)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path4229" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3402" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3398" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3394" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3378" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3374" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3370" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3366" + is_visible="true" /> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.06" + inkscape:cx="246.81474" + inkscape:cy="136.37428" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + showguides="false" + inkscape:window-width="1920" + inkscape:window-height="1053" + inkscape:window-x="94" + inkscape:window-y="69" + inkscape:window-maximized="1" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-33.920555,-581.05015)"> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.30149066;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + d="m 64.094156,600.92012 c 0,49.42495 0,321.26225 0,321.26225" + id="path3388" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1.47424495;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + d="m 291.64401,600.59617 0,321.95932" + id="path3390" + inkscape:connector-curvature="0" /> + <path + style="fill:#ff00ff;fill-rule:evenodd;stroke:#000000;stroke-width:1.28789771;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:2;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#DotM);marker-end:url(#marker6746)" + d="M 64.094156,623.68474 289.7829,665.38811" + id="path4223" + inkscape:connector-curvature="0" /> + <path + style="fill:#ff00ff;fill-rule:evenodd;stroke:#000000;stroke-width:1.28789771;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#marker5510);marker-end:url(#marker6526)" + d="M 63.199944,683.08006 290.44233,725.29448" + id="path5274" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="295.22058" + y="665.10974" + id="text5424" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan5426" + x="295.22058" + y="665.10974">T2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="242.44649" + y="657.08087" + id="text5428" + sodipodi:linespacing="100%" + transform="matrix(0.98289795,0.18415108,-0.18415108,0.98289795,0,0)"><tspan + sodipodi:role="line" + id="tspan5430" + x="242.44649" + y="657.08087" + style="font-size:17.5px">FOLLOW UP:T1</tspan></text> + <path + style="fill:#0000ff;fill-rule:evenodd;stroke:#000000;stroke-width:1.28789771;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#marker5676);marker-end:url(#Arrow2Lend)" + d="M 291.5001,774.36878 64.400801,816.4401" + id="path5586" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="-44.967201" + y="809.64362" + id="text5898" + sodipodi:linespacing="100%" + transform="matrix(0.98291896,-0.18403889,0.18403889,0.98291896,0,0)"><tspan + sodipodi:role="line" + id="tspan5900" + x="-44.967201" + y="809.64362" + style="font-size:17.5px">DELAY REQUEST</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="297.08099" + y="774.7981" + id="text5902" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan5904" + x="297.08099" + y="774.7981">T3</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="35.20845" + y="817.29871" + id="text5906" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan5908" + x="35.20845" + y="817.29871">T4</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="33.920555" + y="623.68475" + id="text5910" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan5912" + x="33.920555" + y="623.68475">T1</tspan></text> + <path + style="fill:#ff00ff;fill-rule:evenodd;stroke:#000000;stroke-width:1.28789771;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#marker6128);marker-end:url(#marker6022)" + d="M 63.971502,838.62056 290.6415,881.55049" + id="path5914" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="223.59686" + y="811.73932" + id="text6858" + sodipodi:linespacing="100%" + transform="matrix(0.98367106,0.17997568,-0.17997568,0.98367106,0,0)"><tspan + sodipodi:role="line" + id="tspan6860" + x="223.59686" + y="811.73932" + style="font-size:17.5px">DELAY RESPONSE:T4</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.39868927px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="698.62549" + y="-320.85892" + id="text6862" + sodipodi:linespacing="100%" + transform="matrix(0.0147127,0.99989176,-0.99989176,0.0147127,0,0)"><tspan + sodipodi:role="line" + id="tspan6864" + x="698.62549" + y="-320.85892">time</tspan></text> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:2.1464963;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend)" + d="m 337.72657,748.13938 0,66.11208" + id="path6866" + inkscape:connector-curvature="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.42257881px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="38.764343" + y="590.47479" + id="text7094" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan7096" + x="38.764343" + y="590.47479">master</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:47.51625061px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="571.69812" + y="61.796165" + id="text7098" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan7100" + x="571.69812" + y="61.796165" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:13.42257881px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="271.23392" + y="593.71478" + id="text7102" + sodipodi:linespacing="100%"><tspan + sodipodi:role="line" + id="tspan7104" + x="271.23392" + y="593.71478">slave</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20.3917141px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#800080;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="244.40062" + y="602.10443" + id="text4317" + sodipodi:linespacing="125%" + transform="matrix(0.98605316,0.16643065,-0.16643065,0.98605316,0,0)"><tspan + sodipodi:role="line" + id="tspan4319" + x="244.40062" + y="602.10443" + style="font-size:17.5px">SYNC</tspan></text> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/qos_sched_app_arch.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/qos_sched_app_arch.png Binary files differnew file mode 100644 index 00000000..63294367 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/qos_sched_app_arch.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/quickassist_block_diagram.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/quickassist_block_diagram.png Binary files differnew file mode 100644 index 00000000..61043eae --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/quickassist_block_diagram.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/ring_pipeline_perf_setup.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ring_pipeline_perf_setup.png Binary files differnew file mode 100644 index 00000000..d8820c08 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/ring_pipeline_perf_setup.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/server_node_efd.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/server_node_efd.svg new file mode 100644 index 00000000..9aee30bc --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/server_node_efd.svg @@ -0,0 +1,1254 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export efd_i6.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.2496in" height="5.89673in" + viewBox="0 0 593.971 424.565" xml:space="preserve" color-interpolation-filters="sRGB" class="st27"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"> + <v:userDefs> + <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/> + <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/> + </v:userDefs> + </v:documentProperties> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st5 {fill:#feffff;font-family:Calibri;font-size:0.75em} + .st6 {fill:none;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st7 {fill:none;stroke:#2e75b5;stroke-width:2.25} + .st8 {fill:#305497;stroke:#2e75b5;stroke-width:1} + .st9 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold} + .st10 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2)} + .st11 {fill:#5b9bd5} + .st12 {stroke:#c7c8c8;stroke-width:0.25} + .st13 {fill:#acccea;stroke:#c7c8c8;stroke-width:0.25} + .st14 {fill:#feffff;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st15 {fill:#ed7d31;stroke:#c7c8c8;stroke-width:0.25} + .st16 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25} + .st17 {marker-end:url(#mrkr5-212);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st18 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091} + .st19 {fill:none;stroke:#2e75b5;stroke-width:1} + .st20 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em} + .st21 {fill:none;stroke:none;stroke-width:0.25} + .st22 {font-size:1em} + .st23 {fill:#ffffff} + .st24 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st25 {marker-end:url(#mrkr5-444);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st26 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.37313432835821} + .st27 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-212" class="st18" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + <marker id="mrkr5-444" class="st26" v:arrowType="5" v:arrowSize="2" v:setback="4.69" refX="-4.69" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-2.68,-2.68) "/> + </marker> + </defs> + <defs id="Filters"> + <filter id="filter_2"> + <feGaussianBlur stdDeviation="2"/> + </filter> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <v:userDefs> + <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/> + </v:userDefs> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(319.501,-335.688)"> + <title>Rectangle.58</title> + <desc>Key 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow3-2" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="4.74" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key 1</text> </g> + <g id="shape4-7" v:mID="4" v:groupContext="shape" transform="translate(353.251,-335.688)"> + <title>Rectangle.59</title> + <desc>Action 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow4-8" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.62" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action 1</text> </g> + <g id="shape5-13" v:mID="5" v:groupContext="shape" transform="translate(400.501,-335.688)"> + <title>Rectangle.60</title> + <desc>Key 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow5-14" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="4.74" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key 2</text> </g> + <g id="shape6-19" v:mID="6" v:groupContext="shape" transform="translate(434.251,-335.688)"> + <title>Rectangle.61</title> + <desc>Action 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow6-20" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.62" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action 2</text> </g> + <g id="shape7-25" v:mID="7" v:groupContext="shape" transform="translate(481.501,-335.688)"> + <title>Rectangle.62</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow7-26" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape8-30" v:mID="8" v:groupContext="shape" transform="translate(515.251,-335.688)"> + <title>Rectangle.63</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow8-31" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(319.501,-313.188)"> + <title>Rectangle.64</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow9-36" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape10-40" v:mID="10" v:groupContext="shape" transform="translate(353.251,-313.188)"> + <title>Rectangle.65</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow10-41" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape11-45" v:mID="11" v:groupContext="shape" transform="translate(400.501,-313.188)"> + <title>Rectangle.66</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow11-46" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape12-50" v:mID="12" v:groupContext="shape" transform="translate(434.251,-313.188)"> + <title>Rectangle.67</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow12-51" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape13-55" v:mID="13" v:groupContext="shape" transform="translate(481.501,-313.188)"> + <title>Rectangle.68</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow13-56" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape14-60" v:mID="14" v:groupContext="shape" transform="translate(515.251,-313.188)"> + <title>Rectangle.69</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow14-61" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape15-65" v:mID="15" v:groupContext="shape" transform="translate(319.501,-277.188)"> + <title>Rectangle.70</title> + <desc>Key x</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow15-66" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.11" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key x</text> </g> + <g id="shape16-71" v:mID="16" v:groupContext="shape" transform="translate(353.251,-277.188)"> + <title>Rectangle.71</title> + <desc>Action x</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow16-72" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.99" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action x</text> </g> + <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(400.501,-277.188)"> + <title>Rectangle.72</title> + <desc>Key y</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow17-78" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.01" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key y</text> </g> + <g id="shape18-83" v:mID="18" v:groupContext="shape" transform="translate(434.251,-277.188)"> + <title>Rectangle.73</title> + <desc>Action y</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow18-84" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.89" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action y</text> </g> + <g id="shape19-89" v:mID="19" v:groupContext="shape" transform="translate(481.501,-277.188)"> + <title>Rectangle.74</title> + <desc>Key z</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow19-90" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.3" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key z</text> </g> + <g id="shape20-95" v:mID="20" v:groupContext="shape" transform="translate(515.251,-277.188)"> + <title>Rectangle.75</title> + <desc>Action z</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow20-96" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="5.18" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action z</text> </g> + <g id="shape21-101" v:mID="21" v:groupContext="shape" transform="translate(319.501,-240.687)"> + <title>Rectangle.76</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow21-102" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape22-106" v:mID="22" v:groupContext="shape" transform="translate(353.251,-240.687)"> + <title>Rectangle.77</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow22-107" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape23-111" v:mID="23" v:groupContext="shape" transform="translate(400.501,-240.687)"> + <title>Rectangle.78</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow23-112" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape24-116" v:mID="24" v:groupContext="shape" transform="translate(434.251,-240.687)"> + <title>Rectangle.79</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow24-117" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape25-121" v:mID="25" v:groupContext="shape" transform="translate(481.501,-240.687)"> + <title>Rectangle.80</title> + <desc>Key N</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow25-122" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.21" y="418.26" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key N</text> </g> + <g id="shape26-127" v:mID="26" v:groupContext="shape" transform="translate(515.251,-240.687)"> + <title>Rectangle.81</title> + <desc>Action N</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow26-128" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="5.67" y="418.26" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action N</text> </g> + <g id="shape27-133" v:mID="27" v:groupContext="shape" transform="translate(317.251,-231.687)"> + <title>Rectangle.82</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow27-134" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="289.065" width="245.25" height="135.5" class="st6"/> + </g> + <rect x="0" y="289.065" width="245.25" height="135.5" class="st7"/> + </g> + <g id="shape28-138" v:mID="28" v:groupContext="shape" transform="translate(328.501,-362.688)"> + <title>Sheet.28</title> + <desc>Local Table for N Specific Flows Serviced at Node 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="110.423" cy="418.94" width="220.85" height="11.25"/> + <rect x="0" y="413.315" width="220.846" height="11.25" class="st8"/> + <text x="5.77" y="421.94" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Local Table for N Specific Flows Serviced at Node 1</text> </g> + <g id="group34-141" transform="translate(66.0294,-165.569)" v:mID="34" v:groupContext="group"> + <v:custProps> + <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/> + <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/> + <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/> + </v:custProps> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/> + <v:ud v:nameU="SolSH" v:prompt="" v:val="VT15({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/> + <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <title>Load balancer</title> + <g id="shape35-142" v:mID="35" v:groupContext="shape" transform="translate(0,-7.33146)"> + <title>Sheet.35</title> + <g id="shadow35-143" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M54 367.23 L18 367.23 L0 377.86 L0 424.56 L72 424.56 L72 377.86 L54 367.23 Z" class="st10"/> + <path d="M0 377.86 L72 377.86" class="st6"/> + <path d="M54 367.23 L18 367.23 L0 377.86 L0 424.56 L72 424.56 L72 377.86 L54 367.23" class="st6"/> + </g> + <path d="M54 367.23 L18 367.23 L0 377.86 L0 424.56 L72 424.56 L72 377.86 L54 367.23 Z" class="st11"/> + <path d="M0 377.86 L72 377.86" class="st12"/> + <path d="M54 367.23 L18 367.23 L0 377.86 L0 424.56 L72 424.56 L72 377.86 L54 367.23" class="st12"/> + </g> + <g id="shape36-152" v:mID="36" v:groupContext="shape" transform="translate(8.03054,-12.9324)"> + <title>Sheet.36</title> + <g id="shadow36-153" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M45.34 421.81 L41.2 422.66 L44.12 424.56 L49.68 423.16 L48.75 417.51 L45.8 415.59 L46.69 419.68 L36.97 + 413.34 L35.6 415.45 L45.34 421.81 ZM50.83 405.36 L39.2 405.36 L39.2 407.88 L50.8 407.88 L47.82 410.83 + L51.34 410.83 L55.21 406.61 L51.32 402.39 L47.83 402.39 L50.83 405.36 ZM46.49 392.01 L36.75 398.37 + L38.13 400.48 L47.84 394.14 L46.96 398.23 L49.91 396.31 L50.84 390.66 L45.28 389.26 L42.36 391.16 + L46.49 392.01 ZM27.71 397.16 C22.66 397.16 18.58 401.25 18.58 406.29 C18.58 411.33 22.66 415.42 + 27.71 415.42 C32.75 415.42 36.84 411.33 36.84 406.29 C36.84 401.25 32.75 397.16 27.71 397.16 ZM27.71 + 400.04 C31.15 400.04 33.96 402.84 33.96 406.29 C33.96 409.74 31.15 412.54 27.71 412.54 C24.26 412.54 + 21.46 409.74 21.46 406.29 C21.46 402.84 24.26 400.04 27.71 400.04 ZM11.64 405.04 L0 405.04 L0 407.56 + L11.6 407.56 L8.62 410.51 L12.14 410.51 L16.01 406.29 L12.12 402.07 L8.64 402.07 L11.64 405.04 Z" + class="st2"/> + </g> + <path d="M45.34 421.81 L41.2 422.66 L44.12 424.56 L49.68 423.16 L48.75 417.51 L45.8 415.59 L46.69 419.68 L36.97 413.34 + L35.6 415.45 L45.34 421.81 ZM50.83 405.36 L39.2 405.36 L39.2 407.88 L50.8 407.88 L47.82 410.83 L51.34 + 410.83 L55.21 406.61 L51.32 402.39 L47.83 402.39 L50.83 405.36 ZM46.49 392.01 L36.75 398.37 L38.13 400.48 + L47.84 394.14 L46.96 398.23 L49.91 396.31 L50.84 390.66 L45.28 389.26 L42.36 391.16 L46.49 392.01 ZM27.71 + 397.16 C22.66 397.16 18.58 401.25 18.58 406.29 C18.58 411.33 22.66 415.42 27.71 415.42 C32.75 415.42 + 36.84 411.33 36.84 406.29 C36.84 401.25 32.75 397.16 27.71 397.16 ZM27.71 400.04 C31.15 400.04 33.96 + 402.84 33.96 406.29 C33.96 409.74 31.15 412.54 27.71 412.54 C24.26 412.54 21.46 409.74 21.46 406.29 + C21.46 402.84 24.26 400.04 27.71 400.04 ZM11.64 405.04 L0 405.04 L0 407.56 L11.6 407.56 L8.62 410.51 + L12.14 410.51 L16.01 406.29 L12.12 402.07 L8.64 402.07 L11.64 405.04 Z" class="st13"/> + </g> + </g> + <g id="shape37-157" v:mID="37" v:groupContext="shape" transform="translate(21.0294,-45.4375)"> + <title>Rectangle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow37-158" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="336.433" width="135" height="88.1315" class="st2"/> + </g> + <rect x="0" y="336.433" width="135" height="88.1315" class="st3"/> + </g> + <g id="shape38-162" v:mID="38" v:groupContext="shape" transform="translate(34.693,-126.438)"> + <title>Sheet.38</title> + <desc>EFD Table</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="49.3364" cy="415.565" width="98.68" height="18"/> + <rect x="0" y="406.565" width="98.6728" height="18" class="st8"/> + <text x="24.87" y="419.17" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>EFD Table</text> </g> + <g id="shape39-165" v:mID="39" v:groupContext="shape" transform="translate(30.0294,-99.4375)"> + <title>Rectangle.39</title> + <desc>Group_id</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="26.9182" cy="415.565" width="53.84" height="18"/> + <g id="shadow39-166" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="53.8364" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="53.8364" height="18" class="st15"/> + <text x="7.87" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Group_id</text> </g> + <g id="shape40-171" v:mID="40" v:groupContext="shape" transform="translate(93.193,-99.4375)"> + <title>Rectangle.40</title> + <desc>Hash index</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="26.9182" cy="415.565" width="53.84" height="18"/> + <g id="shadow40-172" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="53.8364" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="53.8364" height="18" class="st15"/> + <text x="4.64" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hash index</text> </g> + <g id="shape41-177" v:mID="41" v:groupContext="shape" transform="translate(30.193,-82.4275)"> + <title>Rectangle.41</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow41-178" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape42-182" v:mID="42" v:groupContext="shape" transform="translate(30.193,-66.8125)"> + <title>Rectangle.42</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow42-183" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape43-187" v:mID="43" v:groupContext="shape" transform="translate(30.1112,-52.1875)"> + <title>Rectangle.43</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow43-188" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape44-192" v:mID="44" v:groupContext="shape" transform="translate(93.0294,-81.4375)"> + <title>Rectangle.44</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow44-193" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape45-197" v:mID="45" v:groupContext="shape" transform="translate(93.193,-66.8125)"> + <title>Rectangle.45</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow45-198" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape46-202" v:mID="46" v:groupContext="shape" transform="translate(93.193,-52.1875)"> + <title>Rectangle.46</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow46-203" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st2"/> + </g> + <rect x="0" y="413.315" width="53.8364" height="11.25" class="st16"/> + </g> + <g id="shape47-207" v:mID="47" v:groupContext="shape" transform="translate(374.924,544.022) rotate(135)"> + <title>Sheet.47</title> + <path d="M-0 417.75 A40.674 18.0151 -156.2 0 0 40.24 422.15 L40.49 421.89" class="st17"/> + </g> + <g id="shape48-213" v:mID="48" v:groupContext="shape" transform="translate(21.0294,-19)"> + <title>Sheet.48</title> + <desc>Supports X*N Flows</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="67.5" cy="415.565" width="135" height="18"/> + <rect x="0" y="406.565" width="135" height="18" class="st19"/> + <text x="19.05" y="419.17" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Supports X*N Flows</text> </g> + <g id="shape49-216" v:mID="49" v:groupContext="shape" transform="translate(48.0294,-229.938)"> + <title>Sheet.49</title> + <desc>Frontend Server or Load Balancer</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="400.94" width="108" height="47.25"/> + <rect x="0" y="377.315" width="108" height="47.25" class="st21"/> + <text x="14.56" y="397.34" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Frontend Server<v:newlineChar/><tspan + x="13.16" dy="1.2em" class="st22">or Load Balancer </tspan> </text> </g> + <g id="group51-220" transform="translate(223.876,-310.938)" v:mID="51" v:groupContext="group"> + <v:custProps> + <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CPU" v:lbl="CPU" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Memory" v:lbl="Memory" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="OperatingSystem" v:lbl="Operating System" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="HardDriveSize" v:lbl="Hard Drive Capacity" v:prompt="" v:type="0" v:format="" + v:sortKey="Workstation" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Department" v:lbl="Department" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/> + <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Server)"/> + <v:cp v:nameU="BelongsTo" v:lbl="Belongs To" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0"/> + </v:custProps> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/> + <v:ud v:nameU="SolSH" v:prompt="" v:val="VT15({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/> + <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <title>Server</title> + <g id="shape52-221" v:mID="52" v:groupContext="shape" transform="translate(13.0183,0)"> + <title>Sheet.52</title> + <g id="shadow52-222" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="352.565" width="45.9634" height="72" class="st2"/> + </g> + <rect x="0" y="352.565" width="45.9634" height="72" class="st3"/> + </g> + <g id="shape53-226" v:mID="53" v:groupContext="shape" transform="translate(47.371,-30.7354)"> + <title>Sheet.53</title> + <g id="shadow53-227" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st2"/> + </g> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st13"/> + </g> + <g id="shape54-231" v:mID="54" v:groupContext="shape" transform="translate(30.51,-11.8022)"> + <title>Sheet.54</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(1)"/> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <g id="shadow54-232" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 + L-0 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" + class="st10"/> + </g> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 L-0 + 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" class="st23"/> + </g> + </g> + <g id="shape59-239" v:mID="59" v:groupContext="shape" transform="translate(277.876,-373.938)"> + <title>Sheet.59</title> + <path d="M-0 424.56 A111.108 53.2538 42.31 0 1 93.83 421.21 L94.14 421.41" class="st17"/> + </g> + <g id="shape60-244" v:mID="60" v:groupContext="shape" transform="translate(205.876,-283.938)"> + <title>Sheet.60</title> + <desc>Backend Server 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="408.124" width="108" height="32.8815"/> + <rect x="0" y="391.683" width="108" height="32.8815" class="st21"/> + <text x="11.93" y="411.72" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Backend Server 1</text> </g> + <g id="group61-247" transform="translate(223.876,-207.438)" v:mID="61" v:groupContext="group"> + <v:custProps> + <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CPU" v:lbl="CPU" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Memory" v:lbl="Memory" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="OperatingSystem" v:lbl="Operating System" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="HardDriveSize" v:lbl="Hard Drive Capacity" v:prompt="" v:type="0" v:format="" + v:sortKey="Workstation" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Department" v:lbl="Department" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/> + <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Server)"/> + <v:cp v:nameU="BelongsTo" v:lbl="Belongs To" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0"/> + </v:custProps> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/> + <v:ud v:nameU="SolSH" v:prompt="" v:val="VT15({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/> + <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <title>Server.61</title> + <g id="shape62-248" v:mID="62" v:groupContext="shape" transform="translate(13.0183,0)"> + <title>Sheet.62</title> + <g id="shadow62-249" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="352.565" width="45.9634" height="72" class="st2"/> + </g> + <rect x="0" y="352.565" width="45.9634" height="72" class="st3"/> + </g> + <g id="shape63-253" v:mID="63" v:groupContext="shape" transform="translate(47.371,-30.7354)"> + <title>Sheet.63</title> + <g id="shadow63-254" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st2"/> + </g> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st13"/> + </g> + <g id="shape64-258" v:mID="64" v:groupContext="shape" transform="translate(30.51,-11.8022)"> + <title>Sheet.64</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(1)"/> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <g id="shadow64-259" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 + L-0 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" + class="st10"/> + </g> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 L-0 + 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" class="st23"/> + </g> + </g> + <g id="shape65-266" v:mID="65" v:groupContext="shape" transform="translate(205.876,-180.437)"> + <title>Sheet.65</title> + <desc>Backend Server 2</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="408.124" width="108" height="32.8815"/> + <rect x="0" y="391.683" width="108" height="32.8815" class="st21"/> + <text x="11.93" y="411.72" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Backend Server 2</text> </g> + <g id="group66-269" transform="translate(219.029,-58.9375)" v:mID="66" v:groupContext="group"> + <v:custProps> + <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CPU" v:lbl="CPU" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Memory" v:lbl="Memory" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="OperatingSystem" v:lbl="Operating System" v:prompt="" v:type="0" v:format="" v:sortKey="Workstation" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="HardDriveSize" v:lbl="Hard Drive Capacity" v:prompt="" v:type="0" v:format="" + v:sortKey="Workstation" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Department" v:lbl="Department" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/> + <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Server)"/> + <v:cp v:nameU="BelongsTo" v:lbl="Belongs To" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0"/> + </v:custProps> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/> + <v:ud v:nameU="SolSH" v:prompt="" v:val="VT15({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/> + <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <title>Server.66</title> + <g id="shape67-270" v:mID="67" v:groupContext="shape" transform="translate(13.0183,0)"> + <title>Sheet.67</title> + <g id="shadow67-271" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="352.565" width="45.9634" height="72" class="st2"/> + </g> + <rect x="0" y="352.565" width="45.9634" height="72" class="st3"/> + </g> + <g id="shape68-275" v:mID="68" v:groupContext="shape" transform="translate(47.371,-30.7354)"> + <title>Sheet.68</title> + <g id="shadow68-276" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st2"/> + </g> + <ellipse cx="2.77848" cy="421.786" rx="2.77848" ry="2.77848" class="st13"/> + </g> + <g id="shape69-280" v:mID="69" v:groupContext="shape" transform="translate(30.51,-11.8022)"> + <title>Sheet.69</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(1)"/> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <g id="shadow69-281" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 + L-0 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" + class="st10"/> + </g> + <path d="M-0 424.56 L22.42 424.56 L22.42 422.76 L-0 422.76 L-0 424.56 ZM-0 419.11 L22.42 419.11 L22.42 417.31 L-0 + 417.31 L-0 419.11 ZM-0 413.65 L22.42 413.65 L22.42 411.84 L-0 411.84 L-0 413.65 Z" class="st23"/> + </g> + </g> + <g id="shape70-288" v:mID="70" v:groupContext="shape" transform="translate(201.029,-26.056)"> + <title>Sheet.70</title> + <desc>Backend Server X</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="408.124" width="108" height="32.8815"/> + <rect x="0" y="391.683" width="108" height="32.8815" class="st21"/> + <text x="11.86" y="411.72" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Backend Server X</text> </g> + <g id="shape71-291" v:mID="71" v:groupContext="shape" transform="translate(684.44,239.627) rotate(90)"> + <title>Sheet.71</title> + <path d="M0 424.56 L45 424.56" class="st24"/> + </g> + <g id="shape72-294" v:mID="72" v:groupContext="shape" transform="translate(6.85967,-22.443) rotate(-38.1076)"> + <title>Sheet.72</title> + <path d="M-0 424.56 A96.1331 44.4001 55.03 0 1 68.24 420.56 L68.51 420.79" class="st17"/> + </g> + <g id="shape73-299" v:mID="73" v:groupContext="shape" transform="translate(328.501,-135.937)"> + <title>Rectangle.73</title> + <desc>Key 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow73-300" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="4.74" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key 1</text> </g> + <g id="shape74-305" v:mID="74" v:groupContext="shape" transform="translate(362.251,-135.937)"> + <title>Rectangle.74</title> + <desc>Action 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow74-306" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.62" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action 1</text> </g> + <g id="shape75-311" v:mID="75" v:groupContext="shape" transform="translate(409.501,-135.937)"> + <title>Rectangle.75</title> + <desc>Key 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow75-312" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="4.74" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key 2</text> </g> + <g id="shape76-317" v:mID="76" v:groupContext="shape" transform="translate(443.251,-135.937)"> + <title>Rectangle.76</title> + <desc>Action 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow76-318" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.62" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action 2</text> </g> + <g id="shape77-323" v:mID="77" v:groupContext="shape" transform="translate(490.501,-135.937)"> + <title>Rectangle.77</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow77-324" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape78-328" v:mID="78" v:groupContext="shape" transform="translate(524.251,-135.937)"> + <title>Rectangle.78</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow78-329" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape79-333" v:mID="79" v:groupContext="shape" transform="translate(328.501,-113.437)"> + <title>Rectangle.79</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow79-334" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape80-338" v:mID="80" v:groupContext="shape" transform="translate(362.251,-113.437)"> + <title>Rectangle.80</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow80-339" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape81-343" v:mID="81" v:groupContext="shape" transform="translate(409.501,-113.437)"> + <title>Rectangle.81</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow81-344" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape82-348" v:mID="82" v:groupContext="shape" transform="translate(443.251,-113.437)"> + <title>Rectangle.82</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow82-349" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape83-353" v:mID="83" v:groupContext="shape" transform="translate(490.501,-113.437)"> + <title>Rectangle.83</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow83-354" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape84-358" v:mID="84" v:groupContext="shape" transform="translate(524.251,-113.437)"> + <title>Rectangle.84</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow84-359" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape85-363" v:mID="85" v:groupContext="shape" transform="translate(328.501,-77.4375)"> + <title>Rectangle.85</title> + <desc>Key x</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow85-364" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.11" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key x</text> </g> + <g id="shape86-369" v:mID="86" v:groupContext="shape" transform="translate(362.251,-77.4375)"> + <title>Rectangle.86</title> + <desc>Action x</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow86-370" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.99" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action x</text> </g> + <g id="shape87-375" v:mID="87" v:groupContext="shape" transform="translate(409.501,-77.4375)"> + <title>Rectangle.87</title> + <desc>Key y</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow87-376" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.01" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key y</text> </g> + <g id="shape88-381" v:mID="88" v:groupContext="shape" transform="translate(443.251,-77.4375)"> + <title>Rectangle.88</title> + <desc>Action y</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow88-382" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="4.89" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action y</text> </g> + <g id="shape89-387" v:mID="89" v:groupContext="shape" transform="translate(490.501,-77.4375)"> + <title>Rectangle.89</title> + <desc>Key z</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow89-388" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.3" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key z</text> </g> + <g id="shape90-393" v:mID="90" v:groupContext="shape" transform="translate(524.251,-77.4375)"> + <title>Rectangle.90</title> + <desc>Action z</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow90-394" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="5.18" y="418.56" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action z</text> </g> + <g id="shape91-399" v:mID="91" v:groupContext="shape" transform="translate(328.501,-40.9375)"> + <title>Rectangle.91</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow91-400" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape92-404" v:mID="92" v:groupContext="shape" transform="translate(362.251,-40.9375)"> + <title>Rectangle.92</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow92-405" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape93-409" v:mID="93" v:groupContext="shape" transform="translate(409.501,-40.9375)"> + <title>Rectangle.93</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow93-410" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + </g> + <g id="shape94-414" v:mID="94" v:groupContext="shape" transform="translate(443.251,-40.9375)"> + <title>Rectangle.94</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow94-415" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + </g> + <g id="shape95-419" v:mID="95" v:groupContext="shape" transform="translate(490.501,-40.9375)"> + <title>Rectangle.95</title> + <desc>Key N</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.75" cy="415.565" width="31.5" height="18"/> + <g id="shadow95-420" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="31.5" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="31.5" height="18" class="st3"/> + <text x="5.21" y="418.26" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Key N</text> </g> + <g id="shape96-425" v:mID="96" v:groupContext="shape" transform="translate(524.251,-40.9375)"> + <title>Rectangle.96</title> + <desc>Action N</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="21.375" cy="415.565" width="42.75" height="18"/> + <g id="shadow96-426" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="406.565" width="42.75" height="18" class="st2"/> + </g> + <rect x="0" y="406.565" width="42.75" height="18" class="st3"/> + <text x="5.67" y="418.26" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Action N</text> </g> + <g id="shape97-431" v:mID="97" v:groupContext="shape" transform="translate(326.251,-31.9375)"> + <title>Rectangle.97</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow97-432" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="289.065" width="245.25" height="135.5" class="st6"/> + </g> + <rect x="0" y="289.065" width="245.25" height="135.5" class="st7"/> + </g> + <g id="shape98-436" v:mID="98" v:groupContext="shape" transform="translate(337.501,-162.938)"> + <title>Sheet.98</title> + <desc>Local Table for N Specific Flows Serviced at Node X</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="110.423" cy="418.94" width="220.85" height="11.25"/> + <rect x="0" y="413.315" width="220.846" height="11.25" class="st8"/> + <text x="5.55" y="421.94" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Local Table for N Specific Flows Serviced at Node X</text> </g> + <g id="shape99-439" v:mID="99" v:groupContext="shape" transform="translate(-204.342,-29.4449) rotate(-53.7462)"> + <title>Sheet.99</title> + <path d="M0 424.56 L160.37 424.56" class="st25"/> + </g> + <g id="shape100-445" v:mID="100" v:groupContext="shape" transform="translate(-37.6568,-164.882) rotate(-24.444)"> + <title>Sheet.100</title> + <path d="M0 424.56 L101.71 424.56" class="st25"/> + </g> + <g id="shape101-450" v:mID="101" v:groupContext="shape" transform="translate(464.049,-50.8578) rotate(50.099)"> + <title>Sheet.101</title> + <path d="M0 424.56 L139.8 424.56" class="st25"/> + </g> + <g id="shape102-455" v:mID="102" v:groupContext="shape" transform="translate(372.376,-207.438)"> + <title>Sheet.102</title> + <desc>Supports N Flows</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="67.5" cy="415.565" width="135" height="18"/> + <rect x="0" y="406.565" width="135" height="18" class="st19"/> + <text x="25.15" y="419.17" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Supports N Flows</text> </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/sym_multi_proc_app.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/sym_multi_proc_app.png Binary files differnew file mode 100644 index 00000000..4aaebccd --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/sym_multi_proc_app.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/tep_termination_arch.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/tep_termination_arch.svg new file mode 100644 index 00000000..54f1655c --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/tep_termination_arch.svg @@ -0,0 +1,1400 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Generated by Microsoft Visio, SVG Export VXLAN.svg Page-1 --> + +<svg + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="4.9898949in" + height="6.1537709in" + viewBox="0 0 359.27243 443.07151" + xml:space="preserve" + class="st24" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="tep_termination_arch.svg" + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-rule:evenodd;stroke-linecap:square;stroke-miterlimit:3"><metadata + id="metadata389"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title></dc:title></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1875" + inkscape:window-height="1056" + id="namedview387" + showgrid="false" + inkscape:zoom="0.23838384" + inkscape:cx="236.40625" + inkscape:cy="204.93313" + inkscape:window-x="-2" + inkscape:window-y="-3" + inkscape:window-maximized="1" + inkscape:current-layer="svg2" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" /><v:documentProperties + v:langID="1033" + v:viewMarkup="false"><v:userDefs><v:ud + v:nameU="msvSubprocessMaster" + v:prompt="" + v:val="VT4(Rectangle)" /><v:ud + v:nameU="msvNoAutoConnect" + v:val="VT0(1):26" /><v:ud + v:nameU="msvConvertTheme" /></v:userDefs></v:documentProperties><style + type="text/css" + id="style4"><![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5} + .st3 {fill:#d1dab4;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st4 {fill:#6b9bc7;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st5 {fill:#000000;font-family:Calibri;font-size:0.916672em;font-weight:bold} + .st6 {fill:#92d050;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st7 {fill:#000000;font-family:Calibri;font-size:0.833336em} + .st8 {font-size:1em} + .st9 {fill:#3e75a6;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st10 {fill:#d8d8d8;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.24} + .st11 {fill:#404040;fill-opacity:1;stroke:#404040;stroke-opacity:1;stroke-width:0.08695652173913} + .st12 {fill:none;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em} + .st15 {fill:#000000;font-family:Calibri;font-size:1.16666em;font-weight:bold} + .st16 {fill:url(#grad0-72);stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st17 {fill:url(#grad0-83);stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st18 {fill:#f2c096;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st19 {fill:#e1ebf3;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st20 {fill:#000000;font-family:Calibri;font-size:1.16666em} + .st21 {font-size:0.857157em} + .st22 {fill:#d26d19;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st23 {fill:#a6c3dd;stroke:#404040;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st24 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]></style><defs + id="Patterns_And_Gradients"><linearGradient + id="grad0-72" + x1="0" + y1="0" + x2="1" + y2="0" + gradientTransform="matrix(0,-1,1,0,0,1)"><stop + offset="0" + stop-color="#efaf7a" + stop-opacity="1" + id="stop8" /><stop + offset="1" + stop-color="#f2bf96" + stop-opacity="1" + id="stop10" /></linearGradient><linearGradient + id="grad0-83" + x1="-0.024014981" + y1="1433.2565" + x2="86.829642" + y2="1433.2565" + gradientTransform="scale(1.9190145,0.52110079)" + gradientUnits="userSpaceOnUse"><stop + offset="0" + stop-color="#f0f0f0" + stop-opacity="1" + id="stop13" /><stop + offset="1" + stop-color="#ffffff" + stop-opacity="1" + id="stop15" /></linearGradient><linearGradient + inkscape:collect="always" + xlink:href="#grad0-72" + id="linearGradient4536" + gradientTransform="scale(5.2221044,0.19149368)" + x1="-0.023935719" + y1="4088.255" + x2="48.280345" + y2="4088.255" + gradientUnits="userSpaceOnUse" /><linearGradient + inkscape:collect="always" + xlink:href="#grad0-72" + id="linearGradient4538" + gradientTransform="scale(5.2221044,0.19149368)" + x1="-0.023935719" + y1="4088.255" + x2="48.280345" + y2="4088.255" + gradientUnits="userSpaceOnUse" /></defs><defs + id="Markers"><g + id="lend13"><path + d="M 3,1 0,0 3,-1 3,1" + style="stroke:none" + id="path19" + inkscape:connector-curvature="0" /></g><marker + id="mrkr13-28" + class="st11" + v:arrowType="13" + v:arrowSize="2" + v:setback="34.5" + refX="-34.5" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="overflow:visible;fill:#404040;fill-opacity:1;stroke:#404040;stroke-width:0.08695652;stroke-opacity:1"><use + xlink:href="#lend13" + transform="scale(-11.5,-11.5)" + id="use22" + x="0" + y="0" + width="100%" + height="100%" /></marker></defs><g + v:mID="0" + v:index="1" + v:groupContext="foregroundPage" + id="g24" + transform="translate(-116.875,-116.875)"><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /><v:ud + v:nameU="msvThemeOrder" + v:val="VT0(0):26" /></v:userDefs><title + id="title26">Page-1</title><v:pageProperties + v:drawingScale="1" + v:pageScale="1" + v:drawingUnits="0" + v:shadowOffsetX="9" + v:shadowOffsetY="-9" /><v:layer + v:name="Flowchart" + v:index="0" /><g + id="shape33-1" + v:mID="33" + v:groupContext="shape" + transform="translate(117,-432)"><title + id="title29">Rectangle.18</title><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow33-2" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="666" + width="342" + height="126" + class="st2" + id="rect32" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="666" + width="342" + height="126" + class="st3" + id="rect34" + style="fill:#d1dab4;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape10-5" + v:mID="10" + v:groupContext="shape" + transform="translate(216,-522)"><title + id="title37">Rounded rectangle.10</title><desc + id="desc39">Userspace vhost Interface</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="74.25" + cy="778.5" + width="148.5" + height="27" /><g + id="shadow10-6" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 139.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-9 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,765 a 9.00007,9.00007 -180 0 0 -9,9 l 0,9 a 9.00007,9.00007 -180 0 0 9,9 l 130.5,0 z" + class="st2" + id="path42" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 139.5,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-9 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,765 a 9.00007,9.00007 -180 0 0 -9,9 l 0,9 a 9.00007,9.00007 -180 0 0 9,9 l 130.5,0 z" + class="st4" + id="path44" + inkscape:connector-curvature="0" + style="fill:#6b9bc7;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="15.14" + y="781.79999" + class="st5" + v:langID="2052" + id="text46" + style="font-weight:bold;font-size:11.0000639px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Userspace vhost Interface </text> +</g><g + id="shape12-10" + v:mID="12" + v:groupContext="shape" + transform="translate(355.5,-481.5)"><title + id="title49">Rounded rectangle.12</title><desc + id="desc51">Encapsulation</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="45" + cy="785.25" + width="90" + height="13.5" /><g + id="shadow12-11" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st2" + id="path54" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st6" + id="path56" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="16.58" + y="788.25" + class="st7" + v:langID="1033" + id="text58" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Encapsulation</text> +</g><g + id="shape13-15" + v:mID="13" + v:groupContext="shape" + transform="translate(153,-483.75)"><title + id="title61">Rounded rectangle.13</title><desc + id="desc63">Decapsulation</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="36" + cy="785.25" + width="72" + height="13.5" /><g + id="shadow13-16" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="M 65.25,792 A 6.75005,6.75005 -180 0 0 72,785.25 6.75005,6.75005 -180 0 0 65.25,778.5 l -58.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 58.5,0 z" + class="st2" + id="path66" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="M 65.25,792 A 6.75005,6.75005 -180 0 0 72,785.25 6.75005,6.75005 -180 0 0 65.25,778.5 l -58.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 58.5,0 z" + class="st6" + id="path68" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="8.2200003" + y="788.25" + class="st7" + v:langID="1033" + id="text70" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />D<tspan + class="st8" + v:langID="2052" + id="tspan72" + style="font-size:10.00003242px">ecap</tspan>sulation</text> +</g><g + id="shape35-21" + v:mID="35" + v:groupContext="shape" + transform="translate(117,-324)"><title + id="title75">Rectangle.18.35</title><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow35-22" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="697.5" + width="342" + height="94.5" + class="st2" + id="rect78" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="697.5" + width="342" + height="94.5" + class="st9" + id="rect80" + style="fill:#3e75a6;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape15-25" + v:mID="15" + v:groupContext="shape" + transform="translate(259.654,-263.25)"><title + id="title83">Data store</title><v:userDefs><v:ud + v:nameU="CH" + v:val="VT0(5):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /></v:userDefs><path + d="m 0,600.13 a 28.3465,10.6299 0 0 1 56.69,0 28.3465,10.6299 0 0 1 -56.69,0 z m 0,0 0,181.24 a 28.3465,10.6299 -180 1 0 56.69,0 l 0,-181.24 a 28.3465,10.6299 0 0 1 -56.69,0 z" + class="st10" + id="path85" + inkscape:connector-curvature="0" + style="fill:#d8d8d8;stroke:#404040;stroke-width:0.23999999;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape16-29" + v:mID="16" + v:groupContext="shape" + transform="translate(324,-391.5)"><title + id="title88">Rounded rectangle.16</title><desc + id="desc90">Inner IP/L4 checksum offload</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="67.5" + cy="785.25" + width="135" + height="13.5" /><g + id="shadow16-30" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 128.25,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -121.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 121.5,0 z" + class="st2" + id="path93" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 128.25,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -121.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 121.5,0 z" + class="st6" + id="path95" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="8.2700005" + y="788.25" + class="st7" + v:langID="2052" + id="text97" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Inner IP/L4 checksum offload</text> +</g><g + id="shape17-34" + v:mID="17" + v:groupContext="shape" + transform="translate(144,-337.5)"><title + id="title100">Rounded rectangle.17</title><desc + id="desc102">UDP port configuration</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="54" + cy="785.25" + width="108" + height="13.5" /><g + id="shadow17-35" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 101.25,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -94.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 94.5,0 z" + class="st2" + id="path105" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 101.25,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -94.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 94.5,0 z" + class="st6" + id="path107" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="6.9299998" + y="788.25" + class="st7" + v:langID="1033" + id="text109" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />UDP port configuration</text> +</g><g + id="shape18-39" + v:mID="18" + v:groupContext="shape" + transform="translate(145.346,-357.75)"><title + id="title112">Rounded rectangle.18</title><desc + id="desc114">Inner IP/L4 csum verify</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="53.3268" + cy="785.25" + width="106.66" + height="13.5" /><g + id="shadow18-40" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 99.9,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -93.15,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 93.15,0 z" + class="st2" + id="path117" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 99.9,792 a 6.75005,6.75005 -180 0 0 6.75,-6.75 6.75005,6.75005 -180 0 0 -6.75,-6.75 l -93.15,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 93.15,0 z" + class="st6" + id="path119" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="6.6599998" + y="788.25" + class="st7" + v:langID="2052" + id="text121" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Inner IP/L4<tspan + class="st8" + v:langID="1033" + id="tspan123" + style="font-size:10.00003242px" /><tspan + class="st8" + v:langID="1033" + id="tspan125" + style="font-size:10.00003242px">csum</tspan> verify</text> +</g><g + id="shape19-46" + v:mID="19" + v:groupContext="shape" + transform="translate(162,-378)"><title + id="title128">Rounded rectangle.19</title><desc + id="desc130">Tunnel Filter</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="45" + cy="785.25" + width="90" + height="13.5" /><g + id="shadow19-47" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st2" + id="path133" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st6" + id="path135" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="18.290001" + y="788.25" + class="st7" + v:langID="1033" + id="text137" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Tunnel <tspan + class="st8" + v:langID="2052" + id="tspan139" + style="font-size:10.00003242px" />F<tspan + class="st8" + v:langID="2052" + id="tspan141" + style="font-size:10.00003242px">ilter</tspan></text> +</g><g + id="shape20-53" + v:mID="20" + v:groupContext="shape" + transform="translate(162,-398.25)"><title + id="title144">Rounded rectangle.20</title><desc + id="desc146">Packet Type</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="45" + cy="785.25" + width="90" + height="13.5" /><g + id="shadow20-54" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st2" + id="path149" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="M 83.25,792 A 6.75005,6.75005 -180 0 0 90,785.25 6.75005,6.75005 -180 0 0 83.25,778.5 l -76.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 76.5,0 z" + class="st6" + id="path151" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="20.530001" + y="788.25" + class="st7" + v:langID="1033" + id="text153" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Packet Type</text> +</g><g + id="shape27-58" + v:mID="27" + v:groupContext="shape" + transform="translate(326.25,-441)"><title + id="title156">Sheet.27</title><desc + id="desc158">DPDK Application</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="54" + cy="783" + width="108" + height="18" /><rect + x="0" + y="774" + width="108" + height="18" + class="st12" + id="rect160" + style="fill:none;stroke:none;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="10.11" + y="786.59998" + class="st13" + v:langID="2052" + id="text162" + style="font-weight:bold;font-size:12.00012016px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />DPDK Application</text> +</g><g + id="shape26-61" + v:mID="26" + v:groupContext="shape" + transform="translate(324,-279)"><title + id="title165">Sheet.26</title><desc + id="desc167">VXLAN Tunnel</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="45" + cy="778.5" + width="90" + height="27" /><rect + x="0" + y="765" + width="90" + height="27" + class="st12" + id="rect169" + style="fill:none;stroke:none;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="10.51" + y="782.09998" + class="st14" + v:langID="2052" + id="text171" + style="font-size:12.00012016px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VXLAN Tunnel</text> +</g><g + id="shape28-64" + v:mID="28" + v:groupContext="shape" + transform="translate(342,-333)"><title + id="title174">Sheet.28</title><desc + id="desc176">NIC</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="45" + cy="783" + width="90" + height="18" /><rect + x="0" + y="774" + width="90" + height="18" + class="st12" + id="rect178" + style="fill:none;stroke:none;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="36.400002" + y="787.20001" + class="st15" + v:langID="2052" + id="text180" + style="font-weight:bold;font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />NIC</text> +</g><g + id="shape29-67" + v:mID="29" + v:groupContext="shape" + transform="matrix(0,1,1,0,-517.5,270)"><title + id="title183">45 degree single</title><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow29-68" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 0,787.5 4.5,-4.5 0,2.97 247.5,0 0,3.06 -247.5,0 0,2.97 -4.5,-4.5 z" + class="st2" + id="path186" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,787.5 4.5,-4.5 0,2.97 247.5,0 0,3.06 -247.5,0 0,2.97 -4.5,-4.5 z" + class="st16" + id="path188" + inkscape:connector-curvature="0" + style="fill:url(#linearGradient4536);stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape30-73" + v:mID="30" + v:groupContext="shape" + transform="matrix(0,-1,-1,0,1093.5,522)"><title + id="title191">45 degree single.30</title><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow30-74" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(-1.44,-1.44)" + class="st1" + style="visibility:visible"><path + d="m 0,787.5 4.5,-4.5 0,2.97 247.5,0 0,3.06 -247.5,0 0,2.97 -4.5,-4.5 z" + class="st2" + id="path194" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 0,787.5 4.5,-4.5 0,2.97 247.5,0 0,3.06 -247.5,0 0,2.97 -4.5,-4.5 z" + class="st16" + id="path196" + inkscape:connector-curvature="0" + style="fill:url(#linearGradient4538);stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape14-78" + v:mID="14" + v:groupContext="shape" + v:layerMember="0" + transform="translate(207,-234)"><title + id="title199">Object</title><desc + id="desc201">WAN/LAN</desc><v:custProps><v:cp + v:nameU="Cost" + v:lbl="Cost" + v:type="7" + v:format="@" + v:langID="1033" /><v:cp + v:nameU="Duration" + v:lbl="Duration" + v:type="2" + v:langID="1033" /><v:cp + v:nameU="Resources" + v:lbl="Resources" + v:langID="1033" /></v:custProps><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(1,1,1,1)" + v:tabSpace="42.5197" /><v:textRect + cx="83.25" + cy="769.5" + width="166.5" + height="45" /><g + id="shadow14-79" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 23.93,782.24 a 19.1532,6.46551 -180 0 0 27.59,5.48 26.5285,8.95514 -180 0 0 47.98,-2.03 8.02422,2.70871 -180 0 0 8.66,2.93 30.5319,10.3066 -180 0 0 29.32,-4.28 18.9723,6.40442 -180 0 0 20.66,-4.05 29.3596,9.20857 -176.98 0 0 -12.51,-8.69 100.331,31.9328 -2.78 0 0 20.84,-8.56 42.3527,12.6007 -180 0 0 -21.95,-11.54 25.2977,7.52648 -180 0 0 -48.39,3.6 21.4707,5.9338 -1.58 0 0 -36.32,0.33 30.834,8.86245 -178.72 0 0 -39.29,12.27 24.6202,7.32494 -180 0 0 3.41,14.54 z" + class="st2" + id="path204" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 23.93,782.24 a 19.1532,6.46551 -180 0 0 27.59,5.48 26.5285,8.95514 -180 0 0 47.98,-2.03 8.02422,2.70871 -180 0 0 8.66,2.93 30.5319,10.3066 -180 0 0 29.32,-4.28 18.9723,6.40442 -180 0 0 20.66,-4.05 29.3596,9.20857 -176.98 0 0 -12.51,-8.69 100.331,31.9328 -2.78 0 0 20.84,-8.56 42.3527,12.6007 -180 0 0 -21.95,-11.54 25.2977,7.52648 -180 0 0 -48.39,3.6 21.4707,5.9338 -1.58 0 0 -36.32,0.33 30.834,8.86245 -178.72 0 0 -39.29,12.27 24.6202,7.32494 -180 0 0 3.41,14.54 z" + class="st17" + id="path206" + inkscape:connector-curvature="0" + style="fill:url(#grad0-83);stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="53.23" + y="773.70001" + class="st15" + v:langID="2052" + id="text208" + style="font-weight:bold;font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />WAN/LAN</text> +</g><g + id="shape11-85" + v:mID="11" + v:groupContext="shape" + transform="translate(236.25,-472.5)"><title + id="title211">Rounded rectangle.11</title><desc + id="desc213">TEP</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="54" + cy="774" + width="108" + height="36" /><g + id="shadow11-86" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 99,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-18 a 9.00007,9.00007 -180 0 0 -9,-9 l -90,0 a 9.00007,9.00007 -180 0 0 -9,9 l 0,18 a 9.00007,9.00007 -180 0 0 9,9 l 90,0 z" + class="st2" + id="path216" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 99,792 a 9.00007,9.00007 -180 0 0 9,-9 l 0,-18 a 9.00007,9.00007 -180 0 0 -9,-9 l -90,0 a 9.00007,9.00007 -180 0 0 -9,9 l 0,18 a 9.00007,9.00007 -180 0 0 9,9 l 90,0 z" + class="st18" + id="path218" + inkscape:connector-curvature="0" + style="fill:#f2c096;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="44.91" + y="777.59998" + class="st13" + v:langID="2052" + id="text220" + style="font-weight:bold;font-size:12.00012016px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TEP</text> +</g><g + id="shape36-90" + v:mID="36" + v:groupContext="shape" + transform="translate(117,-571.5)"><title + id="title223">Rectangle.18.36</title><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><g + id="shadow36-91" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="688.5" + width="342" + height="103.5" + class="st2" + id="rect226" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="688.5" + width="342" + height="103.5" + class="st19" + id="rect228" + style="fill:#e1ebf3;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape5-94" + v:mID="5" + v:groupContext="shape" + transform="translate(306,-594)"><title + id="title231">Rounded rectangle.5</title><desc + id="desc233">VM-2 (VNID:200)</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="61.3125" + cy="765" + width="122.63" + height="54" /><g + id="shadow5-95" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 113.62,792 a 9.00007,9.00007 -180 0 0 9.01,-9 l -0.01,-36 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,738 a 9.00007,9.00007 -180 0 0 -9,9 l 0,36 a 9.00007,9.00007 -180 0 0 9,9 l 104.62,0 z" + class="st2" + id="path236" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 113.62,792 a 9.00007,9.00007 -180 0 0 9.01,-9 l -0.01,-36 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,738 a 9.00007,9.00007 -180 0 0 -9,9 l 0,36 a 9.00007,9.00007 -180 0 0 9,9 l 104.62,0 z" + class="st9" + id="path238" + inkscape:connector-curvature="0" + style="fill:#3e75a6;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="16.93" + y="769.20001" + class="st20" + v:langID="2052" + id="text240" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM<tspan + class="st8" + v:langID="1033" + id="tspan242" + style="font-size:13.99991989px">-</tspan>2 (<tspan + class="st21" + id="tspan244" + style="font-size:12.00012875px">VNID</tspan><tspan + class="st21" + id="tspan246" + style="font-size:12.00012875px">:</tspan><tspan + class="st21" + id="tspan248" + style="font-size:12.00012875px">200</tspan><tspan + class="st21" + id="tspan250" + style="font-size:12.00012875px">)</tspan></text> +</g><g + id="shape3-104" + v:mID="3" + v:groupContext="shape" + transform="translate(135,-594)"><title + id="title253">Rounded rectangle</title><desc + id="desc255">VM-1 (VNID: 100)</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="61.3125" + cy="765" + width="122.63" + height="54" /><g + id="shadow3-105" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="m 113.62,792 a 9.00007,9.00007 -180 0 0 9.01,-9 l -0.01,-36 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,738 a 9.00007,9.00007 -180 0 0 -9,9 l 0,36 a 9.00007,9.00007 -180 0 0 9,9 l 104.62,0 z" + class="st2" + id="path258" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="m 113.62,792 a 9.00007,9.00007 -180 0 0 9.01,-9 l -0.01,-36 a 9.00007,9.00007 -180 0 0 -9,-9 L 9,738 a 9.00007,9.00007 -180 0 0 -9,9 l 0,36 a 9.00007,9.00007 -180 0 0 9,9 l 104.62,0 z" + class="st9" + id="path260" + inkscape:connector-curvature="0" + style="fill:#3e75a6;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="15.58" + y="769.20001" + class="st20" + v:langID="2052" + id="text262" + style="font-size:13.99991989px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />VM<tspan + class="st8" + v:langID="1033" + id="tspan264" + style="font-size:13.99991989px">-</tspan>1 (<tspan + class="st21" + id="tspan266" + style="font-size:12.00012875px">VNID</tspan><tspan + class="st21" + id="tspan268" + style="font-size:12.00012875px">: </tspan><tspan + class="st21" + id="tspan270" + style="font-size:12.00012875px">100</tspan><tspan + class="st21" + id="tspan272" + style="font-size:12.00012875px">)</tspan></text> +</g><g + id="shape23-114" + v:mID="23" + v:groupContext="shape" + transform="matrix(0.8626982,0.50571911,-0.50571911,0.8626982,624.813,-477.284)"><title + id="title275">Process boundary.23</title><v:userDefs><v:ud + v:nameU="CH" + v:val="VT0(5):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /></v:userDefs><rect + x="0" + y="789.16498" + width="75.634804" + height="2.83465" + class="st22" + id="rect277" + style="fill:#d26d19;stroke:none;stroke-width:0.75;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape22-116" + v:mID="22" + v:groupContext="shape" + transform="matrix(0.95221525,0.30542775,-0.30542775,0.95221525,412.466,-548.055)"><title + id="title280">Process boundary</title><v:userDefs><v:ud + v:nameU="CH" + v:val="VT0(5):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /></v:userDefs><rect + x="0" + y="789.16498" + width="125.234" + height="2.83465" + class="st22" + id="rect282" + style="fill:#d26d19;stroke:none;stroke-width:0.75;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape24-118" + v:mID="24" + v:groupContext="shape" + transform="matrix(-0.93547487,0.35339321,-0.35339321,-0.93547487,670.892,944.318)"><title + id="title285">Process boundary.24</title><v:userDefs><v:ud + v:nameU="CH" + v:val="VT0(5):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /></v:userDefs><rect + x="0" + y="789.16498" + width="108.234" + height="2.83465" + class="st22" + id="rect287" + style="fill:#d26d19;stroke:none;stroke-width:0.75;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape25-120" + v:mID="25" + v:groupContext="shape" + transform="matrix(-0.78441927,0.62023093,-0.62023093,-0.78441927,828.973,824.895)"><title + id="title290">Process boundary.25</title><v:userDefs><v:ud + v:nameU="CH" + v:val="VT0(5):26" /><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /></v:userDefs><rect + x="0" + y="789.16498" + width="61.670101" + height="2.83465" + class="st22" + id="rect292" + style="fill:#d26d19;stroke:none;stroke-width:0.75;stroke-linecap:round;stroke-linejoin:round" /></g><g + id="shape4-122" + v:mID="4" + v:groupContext="shape" + transform="translate(148.5,-587.25)"><title + id="title295">Rectangle</title><desc + id="desc297">vport-0</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="22.5" + cy="784.125" + width="45" + height="15.75" /><g + id="shadow4-123" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st2" + id="rect300" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st23" + id="rect302" + style="fill:#a6c3dd;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="7.5" + y="787.13" + class="st7" + v:langID="1033" + id="text304" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />v<tspan + class="st8" + v:langID="2052" + id="tspan306" + style="font-size:10.00003242px">port</tspan><tspan + class="st8" + v:langID="2052" + id="tspan308" + style="font-size:10.00003242px">-</tspan><tspan + class="st8" + v:langID="2052" + id="tspan310" + style="font-size:10.00003242px">0</tspan></text> +</g><g + id="shape2-130" + v:mID="2" + v:groupContext="shape" + transform="translate(202.5,-587.25)"><title + id="title313">Rectangle.2</title><desc + id="desc315">vport-1</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="22.5" + cy="784.125" + width="45" + height="15.75" /><g + id="shadow2-131" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st2" + id="rect318" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st23" + id="rect320" + style="fill:#a6c3dd;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="7.5" + y="787.13" + class="st7" + v:langID="1033" + id="text322" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />v<tspan + class="st8" + v:langID="2052" + id="tspan324" + style="font-size:10.00003242px">port</tspan><tspan + class="st8" + v:langID="2052" + id="tspan326" + style="font-size:10.00003242px">-</tspan><tspan + class="st8" + v:langID="2052" + id="tspan328" + style="font-size:10.00003242px">1</tspan></text> +</g><g + id="shape6-138" + v:mID="6" + v:groupContext="shape" + transform="translate(317.25,-587.25)"><title + id="title331">Rectangle.6</title><desc + id="desc333">vport-0</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="21.375" + cy="784.125" + width="42.75" + height="15.75" /><g + id="shadow6-139" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="776.25" + width="42.75" + height="15.75" + class="st2" + id="rect336" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="776.25" + width="42.75" + height="15.75" + class="st23" + id="rect338" + style="fill:#a6c3dd;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="6.3699999" + y="787.13" + class="st7" + v:langID="1033" + id="text340" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />v<tspan + class="st8" + v:langID="2052" + id="tspan342" + style="font-size:10.00003242px">port</tspan><tspan + class="st8" + v:langID="2052" + id="tspan344" + style="font-size:10.00003242px">-</tspan><tspan + class="st8" + v:langID="2052" + id="tspan346" + style="font-size:10.00003242px">0</tspan></text> +</g><g + id="shape7-146" + v:mID="7" + v:groupContext="shape" + transform="translate(369,-587.25)"><title + id="title349">Rectangle.7</title><desc + id="desc351">vport-1</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="22.5" + cy="784.125" + width="45" + height="15.75" /><g + id="shadow7-147" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st2" + id="rect354" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><rect + x="0" + y="776.25" + width="45" + height="15.75" + class="st23" + id="rect356" + style="fill:#a6c3dd;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="7.5" + y="787.13" + class="st7" + v:langID="1033" + id="text358" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />v<tspan + class="st8" + v:langID="2052" + id="tspan360" + style="font-size:10.00003242px">port</tspan><tspan + class="st8" + v:langID="2052" + id="tspan362" + style="font-size:10.00003242px">-</tspan><tspan + class="st8" + v:langID="2052" + id="tspan364" + style="font-size:10.00003242px">1</tspan></text> +</g><g + id="shape37-154" + v:mID="37" + v:groupContext="shape" + transform="translate(234,-648)"><title + id="title367">Sheet.37</title><desc + id="desc369">Guest</desc><v:userDefs><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="54" + cy="783" + width="108" + height="18" /><rect + x="0" + y="774" + width="108" + height="18" + class="st12" + id="rect371" + style="fill:none;stroke:none;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="39.459999" + y="786.59998" + class="st13" + v:langID="1033" + id="text373" + style="font-weight:bold;font-size:12.00012016px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />Guest</text> +</g><g + id="shape38-157" + v:mID="38" + v:groupContext="shape" + transform="translate(324,-357.75)"><title + id="title376">Rounded rectangle.38</title><desc + id="desc378">TSO offload</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(14):26" /><v:ud + v:nameU="msvThemeColors" + v:val="VT0(36):26" /><v:ud + v:nameU="msvThemeEffects" + v:val="VT0(16):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="36" + cy="785.25" + width="72" + height="13.5" /><g + id="shadow38-158" + v:groupContext="shadow" + v:shadowOffsetX="1.44" + v:shadowOffsetY="-1.44" + v:shadowType="1" + transform="translate(1.44,1.44)" + class="st1" + style="visibility:visible"><path + d="M 65.25,792 A 6.75005,6.75005 -180 0 0 72,785.25 6.75005,6.75005 -180 0 0 65.25,778.5 l -58.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 58.5,0 z" + class="st2" + id="path381" + inkscape:connector-curvature="0" + style="fill:#cdcdcd;fill-opacity:0.5;stroke:#cdcdcd;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5" /></g><path + d="M 65.25,792 A 6.75005,6.75005 -180 0 0 72,785.25 6.75005,6.75005 -180 0 0 65.25,778.5 l -58.5,0 A 6.75005,6.75005 -180 0 0 0,785.25 6.75005,6.75005 -180 0 0 6.75,792 l 58.5,0 z" + class="st6" + id="path383" + inkscape:connector-curvature="0" + style="fill:#92d050;stroke:#404040;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round" /><text + x="12.33" + y="788.25" + class="st7" + v:langID="2052" + id="text385" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000"><v:paragraph + v:horizAlign="1" /><v:tabList />TSO offload</text> +</g></g></svg>
\ No newline at end of file diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/test_pipeline_app.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/test_pipeline_app.png Binary files differnew file mode 100644 index 00000000..d322a9e4 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/test_pipeline_app.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/threads_pipelines.png b/src/spdk/dpdk/doc/guides/sample_app_ug/img/threads_pipelines.png Binary files differnew file mode 100644 index 00000000..27b5e86d --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/threads_pipelines.png diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_highlevel.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_highlevel.svg new file mode 100644 index 00000000..92f88267 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_highlevel.svg @@ -0,0 +1,710 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export vm_power_mgr_highlevel.svg Page-1 --> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2014 Intel Corporation --> + +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + width="7.96928in" height="6.37479in" viewBox="0 0 573.788 458.985" xml:space="preserve" color-interpolation-filters="sRGB" + class="st28"> + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#464646;fill-opacity:0.5;stroke:#464646;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:0.5} + .st3 {fill:url(#grad0-6);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st4 {fill:#000000;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st5 {fill:#464646;fill-opacity:0.5} + .st6 {stroke:#464646;stroke-linecap:butt;stroke-opacity:0.5;stroke-width:0.25} + .st7 {fill:url(#grad0-15)} + .st8 {stroke:#1f497d;stroke-linecap:butt;stroke-width:0.25} + .st9 {fill:#464646;fill-opacity:0.5;stroke:#464646;stroke-linecap:butt;stroke-opacity:0.5;visibility:hidden} + .st10 {fill:url(#grad0-15);stroke:#1f497d;stroke-linecap:butt;stroke-width:0.25;visibility:hidden} + .st11 {fill:#000000;font-family:Arial;font-size:0.666664em;font-weight:bold} + .st12 {fill:url(#grad0-15);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st13 {fill:#000000;font-family:Arial;font-size:0.666664em} + .st14 {font-size:1em} + .st15 {fill:url(#grad0-196);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st16 {fill:url(#grad0-229);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st17 {font-family:Symbol;font-size:1em;font-weight:normal} + .st18 {font-size:1em;font-weight:normal} + .st19 {fill:url(#grad0-282);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st20 {fill:url(#grad0-344);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st21 {fill:url(#grad0-357);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st22 {fill:url(#grad0-418);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st23 {fill:url(#grad0-431);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st24 {marker-end:url(#mrkr5-440);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st25 {fill:#1f497d;fill-opacity:1;stroke:#1f497d;stroke-opacity:1;stroke-width:0.28409090909091} + .st26 {marker-end:url(#mrkr5-440);marker-start:url(#mrkr5-460);stroke:#1f497d;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st27 {fill:#ffffff;stroke:none;stroke-linecap:butt;stroke-width:7.2} + .st28 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Patterns_And_Gradients"> + <linearGradient id="grad0-6" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#bdd194" stop-opacity="1"/> + <stop offset="1" stop-color="#a5c06b" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-15" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#b6c8de" stop-opacity="1"/> + <stop offset="1" stop-color="#9ab3d1" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-196" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#e9f4f8" stop-opacity="1"/> + <stop offset="1" stop-color="#e1f1f5" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-229" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#ffffff" stop-opacity="1"/> + <stop offset="1" stop-color="#ffffff" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-282" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#dc9e9c" stop-opacity="1"/> + <stop offset="1" stop-color="#ce7977" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-344" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#b6cc88" stop-opacity="1"/> + <stop offset="1" stop-color="#a5c06b" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-357" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#fab781" stop-opacity="1"/> + <stop offset="1" stop-color="#f89c52" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-418" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#acc8ea" stop-opacity="1"/> + <stop offset="1" stop-color="#8db3e2" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-431" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#e2e2e2" stop-opacity="1"/> + <stop offset="1" stop-color="#d8d8d8" stop-opacity="1"/> + </linearGradient> + <linearGradient id="grad0-437" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(90 0.5 0.5)"> + <stop offset="0" stop-color="#c4d2e3" stop-opacity="1"/> + <stop offset="1" stop-color="#9ab3d1" stop-opacity="1"/> + </linearGradient> + </defs> + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-440" class="st25" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + <marker id="mrkr5-460" class="st25" refX="5.8" orient="auto" markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(3.52) "/> + </marker> + </defs> + <g> + <title>Page-1</title> + <g id="shape1-1" transform="translate(18.3,-19.74)"> + <title>Box</title> + <desc>Host</desc> + <g id="shadow1-2" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="239.3" width="405.354" height="219.685" class="st2"/> + </g> + <rect x="0" y="239.3" width="405.354" height="219.685" class="st3"/> + <text x="191.19" y="451.39" class="st4">Host</text> </g> + <g id="group59-8" transform="translate(585.399,54.9035) rotate(76.5904)"> + <title>1-D single.59</title> + <g id="shape63-9"> + <title>Sheet.63</title> + <g id="shadow63-10" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape64-17"> + <title>Sheet.64</title> + <g id="shadow64-18" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape65-24" transform="translate(-2.82107,-2.88265)"> + <title>Sheet.65</title> + <g id="shadow65-25" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st10"/> + </g> + </g> + <g id="group54-29" transform="translate(656.289,-75.8304) rotate(58.6472)"> + <title>1-D single.54</title> + <g id="shape56-30"> + <title>Sheet.56</title> + <g id="shadow56-31" transform="matrix(1,0,0,1,1.97897,-0.480491)" class="st1"> + <path d="M0 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape57-37"> + <title>Sheet.57</title> + <g id="shadow57-38" transform="matrix(1,0,0,1,1.97897,-0.480491)" class="st1"> + <path d="M2.83 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L246.36 456.15 A1.41732 1.41732 0 0 1 247.78 457.57 L247.78 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L249.19 456.15 A6.33846 6.33846 -180 0 0 249.19 450.48 L248.41 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L247.78 449.06 A1.41732 1.41732 0 0 1 246.36 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape58-44" transform="translate(-3.21337,-2.88265)"> + <title>Sheet.58</title> + <g id="shadow58-45" transform="matrix(1,0,0,1,1.97897,-0.480491)" class="st1"> + <rect x="0" y="453.412" width="1.41455" height="5.57329" rx="0.707276" ry="0.707276" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.41455" height="5.57329" rx="0.707276" ry="0.707276" class="st10"/> + </g> + </g> + <g id="shape10-49" transform="translate(18.3,-298.953)"> + <title>Box.10</title> + <desc>VM 0</desc> + <g id="shadow10-50" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="317.253" width="201.26" height="141.732" class="st2"/> + </g> + <rect x="0" y="317.253" width="201.26" height="141.732" class="st3"/> + <text x="91.29" y="414.52" class="st11">VM 0</text> </g> + <g id="shape2-55" transform="translate(18.3,-48.7951)"> + <title>Box.2</title> + <desc>Core 0</desc> + <g id="shadow2-56" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 0</text> </g> + <g id="shape3-61" transform="translate(69.3236,-48.7951)"> + <title>Box.3</title> + <desc>Core 1</desc> + <g id="shadow3-62" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 1</text> </g> + <g id="shape4-67" transform="translate(120.347,-48.7951)"> + <title>Box.4</title> + <desc>Core 2</desc> + <g id="shadow4-68" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 2</text> </g> + <g id="shape5-73" transform="translate(171.371,-48.7951)"> + <title>Box.5</title> + <desc>Core 3</desc> + <g id="shadow5-74" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 3</text> </g> + <g id="shape6-79" transform="translate(222.394,-48.7951)"> + <title>Box.6</title> + <desc>Core 4</desc> + <g id="shadow6-80" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 4</text> </g> + <g id="shape7-85" transform="translate(273.418,-48.7951)"> + <title>Box.7</title> + <desc>Core 5</desc> + <g id="shadow7-86" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 5</text> </g> + <g id="shape8-91" transform="translate(324.442,-48.7951)"> + <title>Box.8</title> + <desc>Core 6</desc> + <g id="shadow8-92" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 6</text> </g> + <g id="shape9-97" transform="translate(375.465,-48.7951)"> + <title>Box.9</title> + <desc>Core 7</desc> + <g id="shadow9-98" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.09" y="443.31" class="st13">Core 7</text> </g> + <g id="shape11-103" transform="translate(18.3,-298.953)"> + <title>Box.11</title> + <desc>Virtual Core 0</desc> + <g id="shadow11-104" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>0</text> </g> + <g id="shape12-110" transform="translate(69.3236,-298.953)"> + <title>Box.12</title> + <desc>Virtual Core 1</desc> + <g id="shadow12-111" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>1</text> </g> + <g id="shape13-117" transform="translate(120.347,-298.953)"> + <title>Box.13</title> + <desc>Virtual Core 2</desc> + <g id="shadow13-118" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>2</text> </g> + <g id="shape14-124" transform="translate(171.371,-298.953)"> + <title>Box.14</title> + <desc>Virtual Core 3</desc> + <g id="shadow14-125" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>3</text> </g> + <g id="group16-131" transform="translate(333.737,-187.264) rotate(39.9929)"> + <title>1-D single</title> + <g id="shape17-132"> + <title>Sheet.17</title> + <g id="shadow17-133" transform="matrix(1,0,0,1,2.0287,0.177741)" class="st1"> + <path d="M0 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape18-139"> + <title>Sheet.18</title> + <g id="shadow18-140" transform="matrix(1,0,0,1,2.0287,0.177741)" class="st1"> + <path d="M2.83 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L328.75 456.15 A1.41732 1.41732 0 0 1 330.16 457.57 L330.16 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L331.58 456.15 A6.33846 6.33846 -180 0 0 331.58 450.48 L330.8 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L330.16 449.06 A1.41732 1.41732 0 0 1 328.75 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape19-146" transform="translate(-4.26978,-2.88265)"> + <title>Sheet.19</title> + <g id="shadow19-147" transform="matrix(1,0,0,1,2.0287,0.177741)" class="st1"> + <rect x="0" y="453.412" width="1.87959" height="5.57329" rx="0.939796" ry="0.939796" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.87959" height="5.57329" rx="0.939796" ry="0.939796" class="st10"/> + </g> + </g> + <g id="group20-151" transform="translate(534.375,54.9035) rotate(76.5904)"> + <title>1-D single.20</title> + <g id="shape21-152"> + <title>Sheet.21</title> + <g id="shadow21-153" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape22-159"> + <title>Sheet.22</title> + <g id="shadow22-160" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape23-166" transform="translate(-2.82107,-2.88265)"> + <title>Sheet.23</title> + <g id="shadow23-167" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st10"/> + </g> + </g> + <g id="group28-171" transform="translate(636.422,54.9035) rotate(76.5904)"> + <title>1-D single.28</title> + <g id="shape29-172"> + <title>Sheet.29</title> + <g id="shadow29-173" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape30-179"> + <title>Sheet.30</title> + <g id="shadow30-180" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L215.76 456.15 A1.41732 1.41732 0 0 1 217.18 457.57 L217.18 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L218.6 456.15 A6.33846 6.33846 -180 0 0 218.6 450.48 L217.81 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L217.18 449.06 A1.41732 1.41732 0 0 1 215.76 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape31-186" transform="translate(-2.82107,-2.88265)"> + <title>Sheet.31</title> + <g id="shadow31-187" transform="matrix(1,0,0,1,1.73469,-1.06679)" class="st1"> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.24186" height="5.57329" rx="0.620928" ry="0.620928" class="st10"/> + </g> + </g> + <g id="shape32-191" transform="translate(18.3,-417.299)"> + <title>Box.32</title> + <desc>DPDK Application</desc> + <g id="shadow32-192" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="435.599" width="201.26" height="23.3858" class="st2"/> + </g> + <rect x="0" y="435.599" width="201.26" height="23.3858" class="st15"/> + <text x="68.84" y="446.8" class="st13">DPDK Application</text> </g> + <g id="shape33-198" transform="translate(245.072,-298.953)"> + <title>Box.33</title> + <desc>VM 1</desc> + <g id="shadow33-199" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="317.253" width="99.2126" height="141.732" class="st2"/> + </g> + <rect x="0" y="317.253" width="99.2126" height="141.732" class="st3"/> + <text x="40.27" y="414.52" class="st11">VM 1</text> </g> + <g id="shape34-204" transform="translate(245.072,-298.953)"> + <title>Box.34</title> + <desc>Virtual Core 0</desc> + <g id="shadow34-205" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>0</text> </g> + <g id="shape35-211" transform="translate(296.095,-298.953)"> + <title>Box.35</title> + <desc>Virtual Core 1</desc> + <g id="shadow35-212" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st2"/> + </g> + <rect x="0" y="422.843" width="48.189" height="36.1417" class="st12"/> + <text x="12.76" y="438.51" class="st13">Virtual <tspan x="12.09" dy="1.2em" class="st14">Core </tspan>1</text> </g> + <g id="shape36-218" transform="translate(245.072,-417.299)"> + <title>Box.36</title> + <desc>DPDK Application</desc> + <g id="shadow36-219" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="435.599" width="99.2126" height="23.3858" class="st2"/> + </g> + <rect x="0" y="435.599" width="99.2126" height="23.3858" class="st15"/> + <text x="17.81" y="446.8" class="st13">DPDK Application</text> </g> + <g id="shape49-224" transform="translate(372.631,-334.488)"> + <title>Box.49</title> + <desc>DPDK VM Application Reuse librte_power interface, but provide...</desc> + <g id="shadow49-225" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="352.788" width="175.748" height="106.197" class="st2"/> + </g> + <rect x="0" y="352.788" width="175.748" height="106.197" class="st16"/> + <text x="4" y="363.99" class="st11">DPDK VM Application <tspan x="4" dy="1.212em" class="st17">·</tspan><tspan + class="st17"> </tspan><tspan class="st18">Reuse </tspan>librte_power<tspan class="st18"> </tspan><tspan + class="st18">interface</tspan><tspan class="st18">, </tspan><tspan class="st18">but </tspan><tspan x="22" + dy="1.204em" class="st18">provides a new implementation that </tspan><tspan x="22" dy="1.2em" class="st18">forwards frequency set requests to host </tspan><tspan + x="22" dy="1.2em" class="st18">via Virtio</tspan><tspan class="st18">-</tspan><tspan class="st18">Serial channel </tspan><tspan + x="4" dy="1.212em" class="st17">·</tspan><tspan class="st17"> </tspan><tspan class="st18">Each lcore has exclusive access to a </tspan><tspan + x="22" dy="1.204em" class="st18">single channel </tspan><tspan x="4" dy="1.212em" class="st17">·</tspan><tspan + class="st17"> </tspan><tspan class="st18">Sample application re</tspan><tspan class="st18">-</tspan><tspan + class="st18">uses </tspan>l3fwd_power <tspan x="4" dy="1.216em" class="st17">·</tspan><tspan class="st17"> </tspan><tspan + class="st18">A CLI for changing frequency from within </tspan><tspan x="22" dy="1.204em" class="st18">a VM is also included</tspan><tspan + class="st18">.</tspan></text> </g> + <g id="group37-257" transform="translate(769.581,100.51) rotate(82.4551)"> + <title>1-D single.37</title> + <g id="shape38-258"> + <title>Sheet.38</title> + <g id="shadow38-259" transform="matrix(1,0,0,1,1.61661,-1.23846)" class="st1"> + <path d="M0 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L0 450.48 + L0 456.15 Z" class="st5"/> + <path d="M0 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L0 450.48" + class="st6"/> + </g> + <path d="M0 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L0 450.48 L0 456.15 + Z" class="st7"/> + <path d="M0 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 0 0 + 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 0.334585 + -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L0 450.48" class="st8"/> + </g> + <g id="shape39-265"> + <title>Sheet.39</title> + <g id="shadow39-266" transform="matrix(1,0,0,1,1.61661,-1.23846)" class="st1"> + <path d="M2.83 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st6"/> + </g> + <path d="M2.83 456.15 L211.63 456.15 A1.41732 1.41732 0 0 1 213.05 457.57 L213.05 457.57 a0.334585 0.334585 -180 + 0 0 0.633846 0.149631 L214.47 456.15 A6.33846 6.33846 -180 0 0 214.47 450.48 L213.68 448.91 a0.334585 + 0.334585 -180 0 0 -0.633846 0.149631 L213.05 449.06 A1.41732 1.41732 0 0 1 211.63 450.48 L2.83 450.48 + A2.83465 2.83465 -180 0 0 0 453.32 L0 453.32 A2.83465 2.83465 -180 0 0 2.83 456.15" class="st8"/> + </g> + <g id="shape40-272" transform="translate(-2.76812,-2.88265)"> + <title>Sheet.40</title> + <g id="shadow40-273" transform="matrix(1,0,0,1,1.61661,-1.23846)" class="st1"> + <rect x="0" y="453.412" width="1.21855" height="5.57329" rx="0.609275" ry="0.609275" class="st9"/> + </g> + <rect x="0" y="453.412" width="1.21855" height="5.57329" rx="0.609275" ry="0.609275" class="st10"/> + </g> + </g> + <g id="shape15-277" transform="translate(18.3,-99.8187)"> + <title>Box.15</title> + <desc>OS/Hypervisor</desc> + <g id="shadow15-278" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="319.379" width="405.354" height="139.606" class="st2"/> + </g> + <rect x="0" y="319.379" width="405.354" height="139.606" class="st19"/> + <text x="174.89" y="452.58" class="st11">OS/Hypervisor</text> </g> + <g id="shape55-284" transform="translate(237.985,-159.346)"> + <title>Box.55</title> + <desc>Linux “userspace” power governor /sys/devices/system/cpu/cpuN...</desc> + <g id="shadow55-285" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="430.638" width="167.244" height="28.3465" class="st2"/> + </g> + <rect x="0" y="430.638" width="167.244" height="28.3465" class="st19"/> + <text x="16.72" y="441.84" class="st11">Linux “userspace” power governor <tspan x="8.93" dy="1.2em" class="st14">/</tspan>sys/devices/system/cpu/cpuN/cpufreq/</text> </g> + <g id="shape45-291" transform="translate(429.324,-60.1337)"> + <title>Box.45</title> + <desc>VM Power Monitor Accepts VM Commands over Virtio Serial endpo...</desc> + <g id="shadow45-292" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="254.89" width="130.519" height="204.094" class="st2"/> + </g> + <rect x="0" y="254.89" width="130.519" height="204.094" class="st16"/> + <text x="4" y="266.09" class="st11">VM Power Monitor <tspan x="4" dy="1.212em" class="st17">·</tspan><tspan + class="st17"> </tspan><tspan class="st18">Accepts VM Commands over </tspan><tspan x="22" dy="1.204em" + class="st18">Virtio Serial endpoints</tspan><tspan class="st18">, </tspan><tspan x="22" dy="1.2em" + class="st18">monitored via </tspan>epoll <tspan x="4" dy="1.212em" class="st17">·</tspan><tspan + class="st17"> </tspan><tspan class="st18">Commands include the </tspan><tspan x="22" dy="1.204em" + class="st18">virtual core to be modified</tspan><tspan class="st18">, </tspan><tspan x="22" dy="1.2em" + class="st18">using </tspan>libvirt <tspan class="st18">to get physical </tspan><tspan x="22" dy="1.2em" + class="st18">core mapping </tspan><tspan x="4" dy="1.212em" class="st17">·</tspan><tspan class="st17"> </tspan><tspan + class="st18">Uses librte</tspan><tspan class="st18">_</tspan><tspan class="st18">power to affect </tspan><tspan + x="22" dy="1.204em" class="st18">frequency changes via Linux </tspan><tspan x="22" dy="1.2em" class="st18">userspace power </tspan><tspan + x="22" dy="1.2em" class="st18">governor</tspan><tspan class="st18">(</tspan><tspan class="st18">APCI cpufreq</tspan><tspan + class="st18">) </tspan><tspan x="4" dy="1.212em" class="st17">·</tspan><tspan class="st17"> </tspan><tspan + class="st18">CLI</tspan><tspan class="st18">: </tspan><tspan class="st18">For adding VM channels </tspan><tspan + x="22" dy="1.204em" class="st18">to monitor</tspan><tspan class="st18">, </tspan><tspan class="st18">inspecting and </tspan><tspan + x="22" dy="1.2em" class="st18">changing channel state</tspan><tspan class="st18">, </tspan><tspan x="22" + dy="1.2em" class="st18">manually altering CPU </tspan><tspan x="22" dy="1.2em" class="st18">frequency</tspan><tspan + class="st18">. </tspan><tspan class="st18">Also allows for the </tspan><tspan x="22" dy="1.2em" + class="st18">changing of vCPU to pCPU </tspan><tspan x="22" dy="1.2em" class="st18">pinning</tspan><tspan + class="st18">.</tspan></text> </g> + <g id="shape53-339" transform="translate(26.8039,-116.827)"> + <title>Box.53</title> + <desc>VM Power Monitor Application</desc> + <g id="shadow53-340" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="342.41" width="150.236" height="116.575" class="st2"/> + </g> + <rect x="0" y="342.41" width="150.236" height="116.575" class="st20"/> + <text x="17.34" y="452.58" class="st11">VM Power Monitor Application</text> </g> + <g id="shape61-346" transform="translate(18.3,-396.394)"> + <title>Box.61</title> + <desc>librte_power(vm)</desc> + <g id="shadow61-347" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="438.079" width="201.26" height="20.9055" class="st2"/> + </g> + <rect x="0" y="438.079" width="201.26" height="20.9055" class="st20"/> + <text x="70.84" y="450.93" class="st13">librte_power(vm)</text> </g> + <g id="shape48-352" transform="translate(18.3,-366.275)"> + <title>Box.48</title> + <desc>lcore channel 0</desc> + <g id="shadow48-353" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">0</tspan></text> </g> + <g id="shape47-361" transform="translate(245.072,-396.394)"> + <title>Box.47</title> + <desc>librte_power(vm)</desc> + <g id="shadow47-362" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="438.079" width="99.2126" height="20.9055" class="st2"/> + </g> + <rect x="0" y="438.079" width="99.2126" height="20.9055" class="st20"/> + <text x="19.82" y="450.93" class="st13">librte_power(vm)</text> </g> + <g id="shape46-367" transform="translate(72.1583,-366.275)"> + <title>Box.46</title> + <desc>lcore channel 1</desc> + <g id="shadow46-368" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">1</tspan></text> </g> + <g id="shape60-375" transform="translate(126.017,-366.275)"> + <title>Box.60</title> + <desc>lcore channel 2</desc> + <g id="shadow60-376" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">2</tspan></text> </g> + <g id="shape62-383" transform="translate(177.04,-366.275)"> + <title>Box.62</title> + <desc>lcore channel 3</desc> + <g id="shadow62-384" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">3</tspan></text> </g> + <g id="shape50-391" transform="translate(245.072,-366.275)"> + <title>Box.50</title> + <desc>lcore channel 0</desc> + <g id="shadow50-392" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">0</tspan></text> </g> + <g id="shape52-399" transform="translate(301.765,-366.275)"> + <title>Box.52</title> + <desc>lcore channel 1</desc> + <g id="shadow52-400" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st2"/> + </g> + <rect x="0" y="428.867" width="42.5197" height="30.1181" class="st21"/> + <text x="11.7" y="436.73" class="st11">lcore <tspan x="6.14" dy="1.2em" class="st14">channel </tspan><tspan x="19.04" + dy="1.2em" class="st14">1</tspan></text> </g> + <g id="shape51-407" transform="translate(26.8039,-216.039)"> + <title>Box.51</title> + <desc>Endpoint Monitor(lcore channels)</desc> + <g id="shadow51-408" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st21"/> + <text x="7.09" y="452.82" class="st11">Endpoint Monitor(lcore channels)</text> </g> + <g id="shape25-413" transform="translate(26.8039,-198.854)"> + <title>Box.25</title> + <desc>Channel Manager</desc> + <g id="shadow25-414" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st22"/> + <text x="37.53" y="452.82" class="st11">Channel Manager</text> </g> + <g id="shape41-420" transform="translate(247.906,-215.862)"> + <title>Box.41</title> + <desc>QEMU</desc> + <g id="shadow41-421" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st21"/> + <text x="58.87" y="452.82" class="st11">QEMU</text> </g> + <g id="shape42-426" transform="translate(247.906,-198.854)"> + <title>Box.42</title> + <desc>libvirt</desc> + <g id="shadow42-427" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st23"/> + <text x="59.98" y="452.82" class="st11">libvirt</text> </g> + <g id="shape43-433" transform="translate(168.536,-231.718)"> + <title>Dynamic connector.43</title> + <path d="M0 465.98 L10.63 465.98 L10.63 466.16 L73.21 466.16" class="st24"/> + </g> + <g id="shape69-441" transform="translate(168.536,-190.527)"> + <title>Dynamic connector</title> + <path d="M0 458.98 L10.63 458.98 L10.63 475.99 L63.29 475.99" class="st24"/> + </g> + <g id="shape26-447" transform="translate(26.8039,-181.846)"> + <title>Box.26</title> + <desc>librte_power(Host)</desc> + <g id="shadow26-448" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st19"/> + <text x="35.53" y="452.82" class="st11">librte_power(Host)</text> </g> + <g id="shape68-453" transform="translate(168.536,-200.449)"> + <title>Dynamic connector.68</title> + <desc>Map vCPU to pCPU</desc> + <path d="M5.8 451.9 L6.16 451.9 L73.21 451.9" class="st26"/> + <rect x="4.11883" y="447.099" width="71.1324" height="9.59985" class="st27"/> + <text x="4.12" y="454.3" class="st13">Map vCPU to pCPU</text> </g> + <g id="shape27-464" transform="translate(26.8039,-164.838)"> + <title>Box.27</title> + <desc>VM Power CLI</desc> + <g id="shadow27-465" transform="matrix(1,0,0,1,1.44,1.44)" class="st1"> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st2"/> + </g> + <rect x="0" y="441.623" width="141.732" height="17.3622" class="st22"/> + <text x="44.2" y="452.82" class="st11">VM Power CLI</text> </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_vm_request_seq.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_vm_request_seq.svg new file mode 100644 index 00000000..1487cda9 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vm_power_mgr_vm_request_seq.svg @@ -0,0 +1,895 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2014 Intel Corporation --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + version="1.1" + width="912.44751" + height="664.9787" + id="svg5187" + inkscape:version="0.48.5 r10040" + sodipodi:docname="vm_power_mgr_vm_request_seq.svg"> + <metadata + id="metadata5497"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <defs + id="defs5495" /> + <sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1920" + inkscape:window-height="1005" + id="namedview5493" + showgrid="false" + fit-margin-top="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + fit-margin-left="0.1" + inkscape:zoom="1.5857392" + inkscape:cx="533.1241" + inkscape:cy="478.97029" + inkscape:window-x="1920" + inkscape:window-y="36" + inkscape:window-maximized="0" + inkscape:current-layer="Layer1000" /> + <g + id="Layer1000" + transform="matrix(1.3033705,0,0,1.3931773,-22.056332,168.16686)"> + <g + id="Layer1001" + transform="matrix(1.237886,0,0,1.2010974,-4.0390827,-71.427441)"> + <defs + id="defs5191"> + <filter + filterUnits="userSpaceOnUse" + id="shadow1" + color-interpolation-filters="sRGB"> + <feOffset + in="SourceGraphic" + result="offset" + dx="1.4400" + dy="1.4400" + id="feOffset5194" /> + <feColorMatrix + in="offset" + result="offset-color" + type="matrix" + values="0 0 0 0 0.2745 0 0 0 0 0.2745 0 0 0 0 0.2745 0 0 0 0.4980 0" + id="feColorMatrix5196" /> + <feMerge + id="feMerge5198"> + <feMergeNode + in="offset-color" + id="feMergeNode5200" /> + <feMergeNode + in="SourceGraphic" + id="feMergeNode5202" /> + </feMerge> + </filter> + </defs> + <defs + id="defs5204"> + <linearGradient + id="grad1"> + <stop + offset="0%" + stop-color="#ffffff" + stop-opacity="1.0000" + id="stop5207" /> + <stop + offset="100%" + stop-color="#ffffff" + stop-opacity="1.0000" + id="stop5209" /> + </linearGradient> + <linearGradient + xlink:href="#grad1" + id="grad2" + x1="164.37778" + y1="54.035988" + x2="164.37778" + y2="290.9747" + gradientTransform="scale(0.85432014,1.1705214)" + gradientUnits="userSpaceOnUse" /> + </defs> + <path + d="m 157.2997,317.4803 168.6846,0 0,-231.1182 -168.6846,0 0,231.1182 z" + style="fill:url(#grad2);fill-rule:evenodd;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round;filter:url(#shadow1)" + id="path5212" + inkscape:connector-curvature="0" /> + <defs + id="defs5214"> + <filter + filterUnits="userSpaceOnUse" + id="shadow2" + color-interpolation-filters="sRGB"> + <feOffset + in="SourceGraphic" + result="offset" + dx="1.4400" + dy="1.4400" + id="feOffset5217" /> + <feColorMatrix + in="offset" + result="offset-color" + type="matrix" + values="0 0 0 0 0.2745 0 0 0 0 0.2745 0 0 0 0 0.2745 0 0 0 0.4980 0" + id="feColorMatrix5219" /> + <feMerge + id="feMerge5221"> + <feMergeNode + in="offset-color" + id="feMergeNode5223" /> + <feMergeNode + in="SourceGraphic" + id="feMergeNode5225" /> + </feMerge> + </filter> + </defs> + <path + d="m 157.2997,317.4803 168.6846,0 0,-231.1182 -168.6846,0 0,231.1182 z" + style="fill:none;stroke:#1f497d;stroke-width:0.25;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;filter:url(#shadow2)" + id="path5227" + inkscape:connector-curvature="0" /> + </g> + <text + x="192.70265" + y="32.792809" + id="text5229" + transform="scale(1.0151991,0.98502847)" + style="font-size:14.63223553px"> + <tspan + font-size="8.0000" + id="tspan5231" + style="font-size:9.75482368px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">Loop: for each epoll event</tspan> + </text> + </g> + <g + id="Layer1002" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 52.7934,331.6535 0,-272.4094" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5234" + inkscape:connector-curvature="0" /> + <text + x="-79.411301" + y="195.44881" + transform="matrix(0,-1,1,0,-142.6554,248.2422)" + id="text5236"> + <tspan + font-size="12.0000" + id="tspan5238" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1003" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1004"> + <path + d="m 17.2272,59.2441 71.1324,0 0,-25.2284 -71.1324,0 0,25.2284 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5242" + inkscape:connector-curvature="0" /> + <path + d="m 17.2272,59.2441 71.1324,0 0,-25.2284 -71.1324,0 0,25.2284 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5244" + inkscape:connector-curvature="0" /> + </g> + <text + x="21.2272" + y="46.629902" + id="text5246"> + <tspan + font-size="8.0000" + id="tspan5248" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">librte_power(VM)</tspan> + </text> + </g> + <text + x="334.86655" + y="44.000031" + id="text5250" + transform="scale(0.98193323,1.0183992)" + style="font-size:19.71731567px"> + <tspan + font-weight="bold" + font-size="36.0000" + id="tspan5252" + style="font-size:59.15194702px;font-weight:bold;fill:#cdcdcd;fill-opacity:1;stroke-opacity:1;font-family:Calibri">Sequence</tspan> + </text> + <g + id="Layer1005" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 129.4472,331.6535 0,-272.4094" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5255" + inkscape:connector-curvature="0" /> + <text + x="-2.7574999" + y="195.44881" + transform="matrix(0,-1,1,0,-66.0016,324.896)" + id="text5257"> + <tspan + font-size="12.0000" + id="tspan5259" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1006" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1007"> + <path + d="m 89.6487,59.2441 79.5971,0 0,-25.2284 -79.5971,0 0,25.2284 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5263" + inkscape:connector-curvature="0" /> + <path + d="m 89.6487,59.2441 79.5971,0 0,-25.2284 -79.5971,0 0,25.2284 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5265" + inkscape:connector-curvature="0" /> + </g> + <text + x="93.648697" + y="46.629902" + id="text5267"> + <tspan + font-size="8.0000" + id="tspan5269" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">guest_channel(VM)</tspan> + </text> + </g> + <g + id="Layer1008" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 240.9449,354.9562 0,-293.7369" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5272" + inkscape:connector-curvature="0" /> + <text + x="98.076401" + y="208.0878" + transform="matrix(0,-1,1,0,32.8571,449.0327)" + id="text5274"> + <tspan + font-size="12.0000" + id="tspan5276" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1009" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1010"> + <path + d="m 195.3689,61.2193 91.152,0 0,-27.2036 -91.152,0 0,27.2036 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5280" + inkscape:connector-curvature="0" /> + <path + d="m 195.3689,61.2193 91.152,0 0,-27.2036 -91.152,0 0,27.2036 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5282" + inkscape:connector-curvature="0" /> + </g> + <text + x="199.3689" + y="47.6175" + id="text5284"> + <tspan + font-size="8.0000" + id="tspan5286" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">channel_monitor(Host)</tspan> + <tspan + font-size="8.0000" + id="tspan5288" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1011" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 361.5266,352.9811 0,-293.737" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5291" + inkscape:connector-curvature="0" /> + <text + x="218.6581" + y="206.11259" + transform="matrix(0,-1,1,0,155.414,567.6392)" + id="text5293"> + <tspan + font-size="12.0000" + id="tspan5295" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1012" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1013"> + <path + d="m 313.5013,59.2441 96.0505,0 0,-27.2035 -96.0505,0 0,27.2035 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5299" + inkscape:connector-curvature="0" /> + <path + d="m 313.5013,59.2441 96.0505,0 0,-27.2035 -96.0505,0 0,27.2035 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5301" + inkscape:connector-curvature="0" /> + </g> + <text + x="317.50131" + y="45.6423" + id="text5303"> + <tspan + font-size="8.0000" + id="tspan5305" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">channel_manager(Host)</tspan> + <tspan + font-size="8.0000" + id="tspan5307" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1014" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 458.3741,331.6535 0,-272.4094" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5310" + inkscape:connector-curvature="0" /> + <text + x="326.1694" + y="195.44881" + transform="matrix(0,-1,1,0,262.9253,653.8229)" + id="text5312"> + <tspan + font-size="12.0000" + id="tspan5314" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1015" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1016"> + <path + d="m 413.466,59.2441 89.8163,0 0,-25.2284 -89.8163,0 0,25.2284 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5318" + inkscape:connector-curvature="0" /> + <path + d="m 413.466,59.2441 89.8163,0 0,-25.2284 -89.8163,0 0,25.2284 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5320" + inkscape:connector-curvature="0" /> + </g> + <text + x="417.466" + y="46.629902" + id="text5322"> + <tspan + font-size="8.0000" + id="tspan5324" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">power_manager(Host)</tspan> + </text> + </g> + <g + id="Layer1017" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 249.6969,310.3937 0,-209.7638 -9,0 0,209.7638 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5327" + inkscape:connector-curvature="0" /> + <path + d="m 249.6969,310.3937 0,-209.7638 -9,0 0,209.7638 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5329" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1018" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5332"> + <marker + id="endMarker1" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5335" /> + </marker> + </defs> + <path + d="m 249.6969,154.6299 a 23.4,468 90 0 1 0,18" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker1)" + id="path5337" + inkscape:connector-curvature="0" /> + <text + x="289.6969" + y="163.6299" + id="text5339"> + <tspan + font-size="8.0000" + id="tspan5341" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">process_request</tspan> + </text> + </g> + <g + id="Layer1019" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5344"> + <marker + id="endMarker2" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5347" /> + </marker> + </defs> + <path + d="m 249.6969,190.6299 108.6323,-0.7086" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker2)" + id="path5349" + inkscape:connector-curvature="0" /> + <text + x="271.9838" + y="179.6738" + id="text5351"> + <tspan + font-size="8.0000" + id="tspan5353" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">get_pcpu_mask()</tspan> + </text> + </g> + <g + id="Layer1020" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 367.3292,225.9213 0,-36 -9,0 0,36 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5356" + inkscape:connector-curvature="0" /> + <path + d="m 367.3292,225.9213 0,-36 -9,0 0,36 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5358" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1021" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5361"> + <marker + id="startMarker1" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="9" + refY="5"> + <polyline + points="10,0 0,5 10,10 9,5" + fill="solid" + id="polyline5364" /> + </marker> + </defs> + <path + d="m 249.6969,226.6299 108.6323,-0.7086" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44;marker-start:url(#startMarker1)" + id="path5366" + inkscape:connector-curvature="0" /> + <text + x="282.43311" + y="215.6738" + id="text5368"> + <tspan + font-size="8.0000" + id="tspan5370" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">pcpu_mask</tspan> + </text> + </g> + <g + id="Layer1022" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 544.3667,331.6535 0,-272.4094" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44" + id="path5373" + inkscape:connector-curvature="0" /> + <text + x="412.1619" + y="195.44881" + transform="matrix(0,-1,1,0,348.9179,739.8155)" + id="text5375"> + <tspan + font-size="12.0000" + id="tspan5377" + style="font-size:12px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial" /> + </text> + </g> + <g + id="Layer1023" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <g + id="Layer1024"> + <path + d="m 506.5757,59.2441 75.5819,0 0,-25.2284 -75.5819,0 0,25.2284 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5381" + inkscape:connector-curvature="0" /> + <path + d="m 506.5757,59.2441 75.5819,0 0,-25.2284 -75.5819,0 0,25.2284 z" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5383" + inkscape:connector-curvature="0" /> + </g> + <text + x="510.57571" + y="46.629902" + id="text5385"> + <tspan + font-size="8.0000" + id="tspan5387" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">librte_power(Host)</tspan> + </text> + </g> + <g + id="Layer1025" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5390"> + <marker + id="endMarker3" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5393" /> + </marker> + </defs> + <path + d="m 249.6969,262.6299 205.4799,-0.7086" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker3)" + id="path5395" + inkscape:connector-curvature="0" /> + <text + x="303.06931" + y="251.6738" + id="text5397"> + <tspan + font-size="8.0000" + id="tspan5399" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">scale_freq_up(pcpu_mask)</tspan> + </text> + </g> + <g + id="Layer1026" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 464.1768,297.9213 0,-36 -9,0 0,36 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5402" + inkscape:connector-curvature="0" /> + <path + d="m 464.1768,297.9213 0,-36 -9,0 0,36 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5404" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1027" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5407"> + <marker + id="endMarker4" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5410" /> + </marker> + </defs> + <path + d="m 464.1768,279.9213 76.9925,0" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker4)" + id="path5412" + inkscape:connector-curvature="0" /> + <text + x="465.31769" + y="269.31949" + id="text5414"> + <tspan + font-size="8.0000" + id="tspan5416" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">rte_power_freq_up()</tspan> + </text> + </g> + <g + id="Layer1028" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 550.1693,297.9213 0,-18 -9,0 0,18 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5419" + inkscape:connector-curvature="0" /> + <path + d="m 550.1693,297.9213 0,-18 -9,0 0,18 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5421" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1029" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 132.0591,154.4882 0,-18 -9,0 0,18 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5424" + inkscape:connector-curvature="0" /> + <path + d="m 132.0591,154.4882 0,-18 -9,0 0,18 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5426" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1030" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5429"> + <marker + id="endMarker5" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5432" /> + </marker> + </defs> + <path + d="m 132.0591,154.4882 108.6378,0.1417" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker5)" + id="path5434" + inkscape:connector-curvature="0" /> + <text + x="135.90199" + y="143.95731" + id="text5436"> + <tspan + font-size="8.0000" + id="tspan5438" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">guest_channel_send_msg()</tspan> + </text> + </g> + <g + id="Layer1031" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5441"> + <marker + id="startMarker2" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="9" + refY="5"> + <polyline + points="10,0 0,5 10,10 9,5" + fill="solid" + id="polyline5444" /> + </marker> + </defs> + <path + d="m 464.1768,297.9213 76.9925,0" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44;marker-start:url(#startMarker2)" + id="path5446" + inkscape:connector-curvature="0" /> + <text + x="486.673" + y="287.31949" + id="text5448"> + <tspan + font-size="8.0000" + id="tspan5450" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">status</tspan> + </text> + </g> + <g + id="Layer1032" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5453"> + <marker + id="startMarker3" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="9" + refY="5"> + <polyline + points="10,0 0,5 10,10 9,5" + fill="solid" + id="polyline5456" /> + </marker> + </defs> + <path + d="m 249.6969,298.6299 205.4799,-0.7086" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44;marker-start:url(#startMarker3)" + id="path5458" + inkscape:connector-curvature="0" /> + <text + x="336.4306" + y="287.6738" + id="text5460"> + <tspan + font-size="8.0000" + id="tspan5462" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">status</tspan> + </text> + </g> + <g + id="Layer1033" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <path + d="m 55.4053,154.4882 0,-18 -9,0 0,18 9,0 z" + style="fill:#9ab3d1;fill-rule:evenodd;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round" + id="path5465" + inkscape:connector-curvature="0" /> + <path + d="m 55.4053,154.4882 0,-18 -9,0 0,18 9,0 z" + style="fill:none;stroke:#000000;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none" + id="path5467" + inkscape:connector-curvature="0" /> + </g> + <g + id="Layer1034" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5470"> + <marker + id="endMarker6" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="1" + refY="5"> + <polyline + points="0,0 10,5 0,10 1,5" + fill="solid" + id="polyline5473" /> + </marker> + </defs> + <path + d="m 55.4053,136.4882 67.6538,0" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;marker-end:url(#endMarker6)" + id="path5475" + inkscape:connector-curvature="0" /> + <text + x="51.876801" + y="125.8864" + id="text5477"> + <tspan + font-size="8.0000" + id="tspan5479" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">rte_power_freq_up()</tspan> + </text> + </g> + <g + id="Layer1035" + transform="matrix(1.6134241,0,0,1.6733416,-27.320753,68.655772)"> + <defs + id="defs5482"> + <marker + id="startMarker4" + markerUnits="strokeWidth" + orient="auto" + markerWidth="8" + markerHeight="6" + viewBox="0 0 10 10" + refX="9" + refY="5"> + <polyline + points="10,0 0,5 10,10 9,5" + fill="solid" + id="polyline5485" /> + </marker> + </defs> + <path + d="m 55.4053,154.4882 72.1538,0" + style="fill:none;stroke:#1f497d;stroke-width:0.47999999;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:2.88, 1.44, 2.88, 1.44;marker-start:url(#startMarker4)" + id="path5487" + inkscape:connector-curvature="0" /> + <text + x="75.482201" + y="143.8864" + id="text5489"> + <tspan + font-size="8.0000" + id="tspan5491" + style="font-size:8px;fill:#000000;fill-opacity:1;stroke-opacity:1;font-family:Arial">status</tspan> + </text> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/img/vmdq_dcb_example.svg b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vmdq_dcb_example.svg new file mode 100644 index 00000000..4d2882ab --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/img/vmdq_dcb_example.svg @@ -0,0 +1,764 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2011 Intel Corporation --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="854.83661" + height="504.1741" + id="svg2" + version="1.1"> + <defs + id="defs4" /> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + showgrid="false" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + units="cm" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-102.91177,-430.74991)"> + <g + id="1" + transform="translate(105.20508,428.83072)"> + <path + d="m 208.15,255.15 0,2.3 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z m 0,6.9 0,2.35 c 0,0.6 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.55 -1.15,-1.15 l 0,-2.35 c 0,-0.6 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.55 1.2,1.15 z m 0,6.95 0,2.3 c 0,0.6 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.55 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z m 0,6.9 0,2.3 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z m 0,6.9 0,1.15 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 l 0,-1.15 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z" + style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.80000001px;stroke-linecap:butt;stroke-linejoin:bevel;stroke-opacity:1;stroke-dasharray:none" + id="2" + inkscape:connector-curvature="0" /> + <path + d="m 55.3,197.6 0,143.9 123.45,0 0,-143.9 -123.45,0 z" + style="fill:#facca7;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="3" + inkscape:connector-curvature="0" /> + <path + d="m 55.3,341.5 123.45,0 0,-143.9 -123.45,0 0,143.9 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="4" + inkscape:connector-curvature="0" /> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="264" + x="72" + xml:space="preserve" + id="5">NIC Port </text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="264" + x="151.2" + xml:space="preserve" + id="6">0</text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="287.20001" + x="106.4" + xml:space="preserve" + id="7">RX</text> + <path + d="m 672.6,197.6 0,143.9 123.5,0 0,-143.9 -123.5,0 z" + style="fill:#facca7;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="8" + inkscape:connector-curvature="0" /> + <path + d="m 672.6,341.5 123.5,0 0,-143.9 -123.5,0 0,143.9 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="9" + inkscape:connector-curvature="0" /> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="264" + x="689.59998" + xml:space="preserve" + id="10">NIC Port </text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="264" + x="768.79999" + xml:space="preserve" + id="11">0 </text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="287.20001" + x="722.40002" + xml:space="preserve" + id="12">TX</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="14.4" + x="251.2" + xml:space="preserve" + id="13">HW RX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="14.4" + x="339.20001" + xml:space="preserve" + id="14">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="252" + xml:space="preserve" + id="15">(</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="256" + xml:space="preserve" + id="16">VLAN </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="293.60001" + xml:space="preserve" + id="17">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="300.79999" + xml:space="preserve" + id="18">, </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="307.20001" + xml:space="preserve" + id="19">Prio </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="333.60001" + xml:space="preserve" + id="20">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="30.4" + x="340.79999" + xml:space="preserve" + id="21">)</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="89.599998" + x="530.40002" + xml:space="preserve" + id="22">HW TX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="89.599998" + x="616.79999" + xml:space="preserve" + id="23">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="406.39999" + x="530.40002" + xml:space="preserve" + id="24">HW TX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="406.39999" + x="616.79999" + xml:space="preserve" + id="25">15</text> + <path + d="m 390.4,63.3 0,95.9 123.45,0 0,-95.9 -123.45,0 z" + style="fill:#729fdc;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="26" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,159.2 123.45,0 0,-95.9 -123.45,0 0,95.9 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="27" + inkscape:connector-curvature="0" /> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="105.6" + x="398.39999" + xml:space="preserve" + id="28">User Thread </text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="128.8" + x="446.39999" + xml:space="preserve" + id="29">0</text> + <path + d="m 513.85,111.25 45.4,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="30" + inkscape:connector-curvature="0" /> + <path + d="m 566.8,111.25 -10,5 c 1.55,-3.15 1.55,-6.85 0,-10 l 10,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="31" + inkscape:connector-curvature="0" /> + <path + d="m 310.45,88.85 0,2.3 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z m 0,6.9 0,2.35 c 0,0.6 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.55 -1.15,-1.15 l 0,-2.35 c 0,-0.6 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.55 1.2,1.15 z m 0,6.95 0,2.15 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.15 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 z" + style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.80000001px;stroke-linecap:butt;stroke-linejoin:bevel;stroke-opacity:1;stroke-dasharray:none" + id="32" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,133.65 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="33" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,133.65 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="34" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,133.65 0,31.95 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="35" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,85.65 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="36" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,85.65 -10,-5 c 1.6,3.15 1.6,6.85 0,10 l 0,0 10,-5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="37" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,85.65 0,-31.95 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="38" + inkscape:connector-curvature="0" /> + <path + d="m 644.4,245.55 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="39" + inkscape:connector-curvature="0" /> + <path + d="m 672.6,245.55 -9.95,-5 c 1.55,3.15 1.55,6.85 0,10 l 9.95,-5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="40" + inkscape:connector-curvature="0" /> + <path + d="m 644.4,245.55 0,-134.3 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="41" + inkscape:connector-curvature="0" /> + <path + d="m 270.85,165.6 -23.3,0 0,79.95 -68.8,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="42" + inkscape:connector-curvature="0" /> + <path + d="m 269.6,160.6 15,5 -15,5 0,-10 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="43" + inkscape:connector-curvature="0" /> + <path + d="m 178.75,220 26.45,0 0,-166.3 65.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="44" + inkscape:connector-curvature="0" /> + <path + d="m 269.6,48.7 15,5 -15,4.95 0,-9.95 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="45" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,379.9 0,95.95 123.45,0 0,-95.95 -123.45,0 z" + style="fill:#729fdc;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="46" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,379.9 123.45,0 0,95.95 -123.45,0 0,-95.95 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="47" + inkscape:connector-curvature="0" /> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="422.39999" + x="398.39999" + xml:space="preserve" + id="48">User Thread </text> + <text + style="font-size:19.20000076px;font-style:normal;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="444.79999" + x="441.60001" + xml:space="preserve" + id="49">15</text> + <path + d="m 513.85,427.85 45.4,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="50" + inkscape:connector-curvature="0" /> + <path + d="m 566.8,427.85 -10,-5 c 1.55,3.15 1.55,6.85 0,10 l 10,-5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="51" + inkscape:connector-curvature="0" /> + <path + d="m 308.1,421.45 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 l 0,2.3 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 z m 0,-6.9 0,-2.3 c 0,-0.65 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.5 1.2,1.15 l 0,2.3 c 0,0.65 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.5 -1.15,-1.15 z m 0,-6.9 0,-2.2 c 0,-0.6 0.55,-1.15 1.15,-1.15 0.65,0 1.2,0.55 1.2,1.15 l 0,2.2 c 0,0.6 -0.55,1.15 -1.2,1.15 -0.6,0 -1.15,-0.55 -1.15,-1.15 z" + style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.80000001px;stroke-linecap:butt;stroke-linejoin:bevel;stroke-opacity:1;stroke-dasharray:none" + id="52" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,405.45 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="53" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,405.45 -10,-4.95 c 1.6,3.1 1.6,6.8 0,9.95 l 0,0 10,-5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="54" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,405.45 0,-31.95 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="55" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,453.45 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="56" + inkscape:connector-curvature="0" /> + <path + d="m 390.4,453.45 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="57" + inkscape:connector-curvature="0" /> + <path + d="m 362.2,453.45 0,31.95 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="58" + inkscape:connector-curvature="0" /> + <path + d="m 644.4,293.55 20.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="59" + inkscape:connector-curvature="0" /> + <path + d="m 672.6,293.55 -9.95,5 c 1.55,-3.15 1.55,-6.85 0,-10 l 9.95,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="60" + inkscape:connector-curvature="0" /> + <path + d="m 644.4,293.55 0,134.3 -24.7,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="61" + inkscape:connector-curvature="0" /> + <path + d="m 270.85,373.5 -23.3,0 0,-79.95 -68.8,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="62" + inkscape:connector-curvature="0" /> + <path + d="m 269.6,378.5 15,-5 -15,-5 0,10 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="63" + inkscape:connector-curvature="0" /> + <path + d="m 178.75,319.1 26.45,0 0,166.3 65.65,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="64" + inkscape:connector-curvature="0" /> + <path + d="m 269.6,490.4 15,-5 -15,-4.95 0,9.95 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="65" + inkscape:connector-curvature="0" /> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="126.4" + x="251.2" + xml:space="preserve" + id="66">HW RX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="126.4" + x="339.20001" + xml:space="preserve" + id="67">7</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="252" + xml:space="preserve" + id="68">(</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="256" + xml:space="preserve" + id="69">VLAN </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="293.60001" + xml:space="preserve" + id="70">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="300.79999" + xml:space="preserve" + id="71">, </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="307.20001" + xml:space="preserve" + id="72">Prio </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="333.60001" + xml:space="preserve" + id="73">7</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="141.60001" + x="340.79999" + xml:space="preserve" + id="74">)</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="334.39999" + x="264.79999" + xml:space="preserve" + id="75">HW RX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="334.39999" + x="353.60001" + xml:space="preserve" + id="76">120</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="269.60001" + xml:space="preserve" + id="77">(</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="273.60001" + xml:space="preserve" + id="78">VLAN </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="311.20001" + xml:space="preserve" + id="79">15</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="324.79999" + xml:space="preserve" + id="80">, </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="332" + xml:space="preserve" + id="81">Prio </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="358.39999" + xml:space="preserve" + id="82">0</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="349.60001" + x="365.60001" + xml:space="preserve" + id="83">)</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="446.39999" + x="240" + xml:space="preserve" + id="84">HW RX Queue </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="446.39999" + x="328.79999" + xml:space="preserve" + id="85">127</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="244.8" + xml:space="preserve" + id="86">(</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="249.60001" + xml:space="preserve" + id="87">VLAN </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="286.39999" + xml:space="preserve" + id="88">15</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="300.79999" + xml:space="preserve" + id="89">, </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="307.20001" + xml:space="preserve" + id="90">Prio </text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="333.60001" + xml:space="preserve" + id="91">7</text> + <text + style="font-size:12.80000019px;font-style:italic;font-weight:normal;text-align:start;text-anchor:start;fill:#000000;font-family:Arial" + y="461.60001" + x="340.79999" + xml:space="preserve" + id="92">)</text> + <path + d="m 652.6,255.15 0,2.3 c 0,0.65 -0.5,1.15 -1.15,1.15 -0.65,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.5,-1.15 1.15,-1.15 0.65,0 1.15,0.5 1.15,1.15 z m 0,6.9 0,2.35 c 0,0.6 -0.5,1.15 -1.15,1.15 -0.65,0 -1.15,-0.55 -1.15,-1.15 l 0,-2.35 c 0,-0.6 0.5,-1.15 1.15,-1.15 0.65,0 1.15,0.55 1.15,1.15 z m 0,6.95 0,2.3 c 0,0.6 -0.5,1.15 -1.15,1.15 -0.65,0 -1.15,-0.55 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.5,-1.15 1.15,-1.15 0.65,0 1.15,0.5 1.15,1.15 z m 0,6.9 0,2.3 c 0,0.65 -0.5,1.15 -1.15,1.15 -0.65,0 -1.15,-0.5 -1.15,-1.15 l 0,-2.3 c 0,-0.65 0.5,-1.15 1.15,-1.15 0.65,0 1.15,0.5 1.15,1.15 z m 0,6.9 0,1.15 c 0,0.65 -0.5,1.15 -1.15,1.15 -0.65,0 -1.15,-0.5 -1.15,-1.15 l 0,-1.15 c 0,-0.65 0.5,-1.15 1.15,-1.15 0.65,0 1.15,0.5 1.15,1.15 z" + style="fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.80000001px;stroke-linecap:butt;stroke-linejoin:bevel;stroke-opacity:1;stroke-dasharray:none" + id="93" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,37.7 0,31.95 17.65,0 0,-31.95 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="94" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,69.65 17.65,0 0,-31.95 -17.65,0 0,31.95 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="95" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,37.7 0,31.95 17.65,0 0,-31.95 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="96" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,69.65 17.65,0 0,-31.95 -17.65,0 0,31.95 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="97" + inkscape:connector-curvature="0" /> + <path + d="m 284.6,69.65 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="98" + inkscape:connector-curvature="0" /> + <path + d="m 285.45,37.7 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="99" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,149.6 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="100" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,181.6 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="101" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,149.6 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="102" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,181.6 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="103" + inkscape:connector-curvature="0" /> + <path + d="m 284.6,181.6 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="104" + inkscape:connector-curvature="0" /> + <path + d="m 285.45,149.6 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="105" + inkscape:connector-curvature="0" /> + <path + d="m 584.4,95.25 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="106" + inkscape:connector-curvature="0" /> + <path + d="m 584.4,127.25 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="107" + inkscape:connector-curvature="0" /> + <path + d="m 602.05,95.25 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="108" + inkscape:connector-curvature="0" /> + <path + d="m 602.05,127.25 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="109" + inkscape:connector-curvature="0" /> + <path + d="m 566.8,127.25 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="110" + inkscape:connector-curvature="0" /> + <path + d="m 567.65,95.25 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="111" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,357.5 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="112" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,389.5 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="113" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,357.5 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="114" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,389.5 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="115" + inkscape:connector-curvature="0" /> + <path + d="m 284.6,389.5 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="116" + inkscape:connector-curvature="0" /> + <path + d="m 285.45,357.5 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="117" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,469.45 0,31.95 17.65,0 0,-31.95 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="118" + inkscape:connector-curvature="0" /> + <path + d="m 302.2,501.4 17.65,0 0,-31.95 -17.65,0 0,31.95 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="119" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,469.45 0,31.95 17.65,0 0,-31.95 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="120" + inkscape:connector-curvature="0" /> + <path + d="m 319.85,501.4 17.65,0 0,-31.95 -17.65,0 0,31.95 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="121" + inkscape:connector-curvature="0" /> + <path + d="m 284.6,501.4 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="122" + inkscape:connector-curvature="0" /> + <path + d="m 285.45,469.45 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="123" + inkscape:connector-curvature="0" /> + <path + d="m 584.4,411.85 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="124" + inkscape:connector-curvature="0" /> + <path + d="m 584.4,443.85 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="125" + inkscape:connector-curvature="0" /> + <path + d="m 602.05,411.85 0,32 17.65,0 0,-32 -17.65,0 z" + style="fill:#f28226;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="126" + inkscape:connector-curvature="0" /> + <path + d="m 602.05,443.85 17.65,0 0,-32 -17.65,0 0,32 z" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="127" + inkscape:connector-curvature="0" /> + <path + d="m 566.8,443.85 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="128" + inkscape:connector-curvature="0" /> + <path + d="m 567.65,411.85 16.75,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="129" + inkscape:connector-curvature="0" /> + <path + d="m 2.4,268.6 45.35,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="130" + inkscape:connector-curvature="0" /> + <path + d="m 55.3,268.6 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="131" + inkscape:connector-curvature="0" /> + <path + d="m 796.1,268.6 45.35,0" + style="fill:none;stroke:#000000;stroke-width:2.29999995px;stroke-linecap:round;stroke-linejoin:round;stroke-opacity:1;stroke-dasharray:none" + id="132" + inkscape:connector-curvature="0" /> + <path + d="m 849,268.6 -10,5 c 1.6,-3.15 1.6,-6.85 0,-10 l 0,0 10,5 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:none" + id="133" + inkscape:connector-curvature="0" /> + </g> + </g> +</svg> diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/index.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/index.rst new file mode 100644 index 00000000..5bedf4f6 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/index.rst @@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2015 Intel Corporation. + +Sample Applications User Guides +=============================== + +.. toctree:: + :maxdepth: 2 + :numbered: + + intro + compiling + cmd_line + ethtool + exception_path + hello_world + skeleton + rxtx_callbacks + flow_classify + flow_filtering + ip_frag + ipv4_multicast + ip_reassembly + kernel_nic_interface + keep_alive + l2_forward_crypto + l2_forward_job_stats + l2_forward_real_virtual + l2_forward_cat + l3_forward + l3_forward_power_man + l3_forward_access_ctrl + l3_forward_virtual + link_status_intr + load_balancer + server_node_efd + service_cores + multi_process + qos_metering + qos_scheduler + quota_watermark + timer + packet_ordering + vmdq_dcb_forwarding + vhost + vhost_scsi + vhost_crypto + netmap_compatibility + ip_pipeline + test_pipeline + eventdev_pipeline + dist_app + vm_power_management + tep_termination + ptpclient + performance_thread + ipsec_secgw + bbdev_app + +**Figures** + +:numref:`figure_exception_path_example` :ref:`figure_exception_path_example` + +:numref:`figure_kernel_nic` :ref:`figure_kernel_nic` + +:numref:`figure_l2_fwd_benchmark_setup_jobstats` :ref:`figure_l2_fwd_benchmark_setup_jobstats` + +:numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` :ref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` + +:numref:`figure_l2_fwd_benchmark_setup` :ref:`figure_l2_fwd_benchmark_setup` + +:numref:`figure_l2_fwd_virtenv_benchmark_setup` :ref:`figure_l2_fwd_virtenv_benchmark_setup` + +:numref:`figure_l2_fwd_encrypt_flow` :ref:`figure_l2_fwd_encrypt_flow` + +:numref:`figure_ipv4_acl_rule` :ref:`figure_ipv4_acl_rule` + +:numref:`figure_example_rules` :ref:`figure_example_rules` + +:numref:`figure_load_bal_app_arch` :ref:`figure_load_bal_app_arch` + +:numref:`figure_sym_multi_proc_app` :ref:`figure_sym_multi_proc_app` + +:numref:`figure_client_svr_sym_multi_proc_app` :ref:`figure_client_svr_sym_multi_proc_app` + +:numref:`figure_qos_sched_app_arch` :ref:`figure_qos_sched_app_arch` + +:numref:`figure_pipeline_overview` :ref:`figure_pipeline_overview` + +:numref:`figure_ring_pipeline_perf_setup` :ref:`figure_ring_pipeline_perf_setup` + +:numref:`figure_threads_pipelines` :ref:`figure_threads_pipelines` + +:numref:`figure_vmdq_dcb_example` :ref:`figure_vmdq_dcb_example` + +:numref:`figure_test_pipeline_app` :ref:`figure_test_pipeline_app` + +:numref:`figure_dist_perf` :ref:`figure_dist_perf` + +:numref:`figure_dist_app` :ref:`figure_dist_app` + +:numref:`figure_vm_power_mgr_highlevel` :ref:`figure_vm_power_mgr_highlevel` + +:numref:`figure_vm_power_mgr_vm_request_seq` :ref:`figure_vm_power_mgr_vm_request_seq` +:numref:`figure_overlay_networking` :ref:`figure_overlay_networking` +:numref:`figure_tep_termination_arch` :ref:`figure_tep_termination_arch` + +:numref:`figure_ptpclient_highlevel` :ref:`figure_ptpclient_highlevel` + +:numref:`figure_efd_sample_app_overview` :ref:`figure_efd_sample_app_overview` + +**Tables** + +:numref:`table_qos_metering_1` :ref:`table_qos_metering_1` + +:numref:`table_qos_scheduler_1` :ref:`table_qos_scheduler_1` + +:numref:`table_test_pipeline_1` :ref:`table_test_pipeline_1` diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/intro.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/intro.rst new file mode 100644 index 00000000..575995de --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/intro.rst @@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2017 Intel Corporation. + +Introduction to the DPDK Sample Applications +============================================ + +The DPDK Sample Applications are small standalone applications which +demonstrate various features of DPDK. They can be considered as a cookbook of +DPDK features. Users interested in getting started with DPDK can take the +applications, try out the features, and then extend them to fit their needs. + + +The DPDK Sample Applications +---------------------------- + +Table :numref:`table_sample_apps` shows a list of some of the main sample +applications that are available in the examples directory of DPDK: + + .. _table_sample_apps: + + .. table:: **Some of the DPDK Sample applications** + + +---------------------------------------+--------------------------------------+ + | Bonding | Netmap Compatibility | + +---------------------------------------+--------------------------------------+ + | Command Line | Packet Ordering | + +---------------------------------------+--------------------------------------+ + | Distributor | Performance Thread | + +---------------------------------------+--------------------------------------+ + | Ethtool | Precision Time Protocol (PTP) Client | + +---------------------------------------+--------------------------------------+ + | Exception Path | Quality of Service (QoS) Metering | + +---------------------------------------+--------------------------------------+ + | Hello World | QoS Scheduler | + +---------------------------------------+--------------------------------------+ + | Internet Protocol (IP) Fragmentation | Quota and Watermark | + +---------------------------------------+--------------------------------------+ + | IP Pipeline | RX/TX Callbacks | + +---------------------------------------+--------------------------------------+ + | IP Reassembly | Server node EFD | + +---------------------------------------+--------------------------------------+ + | IPsec Security Gateway | Basic Forwarding/Skeleton App | + +---------------------------------------+--------------------------------------+ + | IPv4 multicast | Tunnel End Point (TEP) termination | + +---------------------------------------+--------------------------------------+ + | Kernel NIC Interface | Timer | + +---------------------------------------+--------------------------------------+ + | Network Layer 2 Forwarding + variants | Vhost | + +---------------------------------------+--------------------------------------+ + | Network Layer 3 Forwarding + variants | Vhost Xen | + +---------------------------------------+--------------------------------------+ + | Link Status Interrupt | VMDQ Forwarding | + +---------------------------------------+--------------------------------------+ + | Load Balancer | VMDQ and DCB Forwarding | + +---------------------------------------+--------------------------------------+ + | Multi-process | VM Power Management | + +---------------------------------------+--------------------------------------+ + +These examples range from simple to reasonably complex but most are designed +to demonstrate one particular feature of DPDK. Some of the more interesting +examples are highlighted below. + + +* :doc:`Hello World<hello_world>`: As with most introductions to a + programming framework a good place to start is with the Hello World + application. The Hello World example sets up the DPDK Environment Abstraction + Layer (EAL), and prints a simple "Hello World" message to each of the DPDK + enabled cores. This application doesn't do any packet forwarding but it is a + good way to test if the DPDK environment is compiled and set up properly. + +* :doc:`Basic Forwarding/Skeleton Application<skeleton>`: The Basic + Forwarding/Skeleton contains the minimum amount of code required to enable + basic packet forwarding with DPDK. This allows you to test if your network + interfaces are working with DPDK. + +* :doc:`Network Layer 2 forwarding<l2_forward_real_virtual>`: The Network Layer 2 + forwarding, or ``l2fwd`` application does forwarding based on Ethernet MAC + addresses like a simple switch. + +* :doc:`Network Layer 3 forwarding<l3_forward>`: The Network Layer3 + forwarding, or ``l3fwd`` application does forwarding based on Internet + Protocol, IPv4 or IPv6 like a simple router. + +* :doc:`Packet Distributor<dist_app>`: The Packet Distributor + demonstrates how to distribute packets arriving on an Rx port to different + cores for processing and transmission. + +* :doc:`Multi-Process Application<multi_process>`: The + multi-process application shows how two DPDK processes can work together using + queues and memory pools to share information. + +* :doc:`RX/TX callbacks Application<rxtx_callbacks>`: The RX/TX + callbacks sample application is a packet forwarding application that + demonstrates the use of user defined callbacks on received and transmitted + packets. The application calculates the latency of a packet between RX + (packet arrival) and TX (packet transmission) by adding callbacks to the RX + and TX packet processing functions. + +* :doc:`IPSec Security Gateway<ipsec_secgw>`: The IPSec Security + Gateway application is minimal example of something closer to a real world + example. This is also a good example of an application using the DPDK + Cryptodev framework. + +* :doc:`Precision Time Protocol (PTP) client<ptpclient>`: The PTP + client is another minimal implementation of a real world application. + In this case the application is a PTP client that communicates with a PTP + master clock to synchronize time on a Network Interface Card (NIC) using the + IEEE1588 protocol. + +* :doc:`Quality of Service (QoS) Scheduler<qos_scheduler>`: The QoS + Scheduler application demonstrates the use of DPDK to provide QoS scheduling. + +There are many more examples shown in the following chapters. Each of the +documented sample applications show how to compile, configure and run the +application as well as explaining the main functionality of the code. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ip_frag.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_frag.rst new file mode 100644 index 00000000..7914a977 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_frag.rst @@ -0,0 +1,140 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +IP Fragmentation Sample Application +=================================== + +The IPv4 Fragmentation application is a simple example of packet processing +using the Data Plane Development Kit (DPDK). +The application does L3 forwarding with IPv4 and IPv6 packet fragmentation. + +Overview +-------- + +The application demonstrates the use of zero-copy buffers for packet fragmentation. +The initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +This guide highlights the differences between the two applications. + +There are three key differences from the L2 Forwarding sample application: + +* The first difference is that the IP Fragmentation sample application makes use of indirect buffers. + +* The second difference is that the forwarding decision is taken + based on information read from the input packet's IP header. + +* The third difference is that the application differentiates between + IP and non-IP traffic by means of offload flags. + +The Longest Prefix Match (LPM for IPv4, LPM6 for IPv6) table is used to store/lookup an outgoing port number, +associated with that IP address. +Any unmatched packets are forwarded to the originating port. + +By default, input frame sizes up to 9.5 KB are supported. +Before forwarding, the input IP packet is fragmented to fit into the "standard" Ethernet* v2 MTU (1500 bytes). + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ip_fragmentation`` sub-directory. + +Running the Application +----------------------- + +The LPM object is created and loaded with the pre-configured entries read from +global l3fwd_ipv4_route_array and l3fwd_ipv6_route_array tables. +For each input packet, the packet forwarding decision +(that is, the identification of the output interface for the packet) is taken as a result of LPM lookup. +If the IP packet size is greater than default output MTU, +then the input packet is fragmented and several fragments are sent via the output interface. + +Application usage: + +.. code-block:: console + + ./build/ip_fragmentation [EAL options] -- -p PORTMASK [-q NQ] + +where: + +* -p PORTMASK is a hexadecimal bitmask of ports to configure + +* -q NQ is the number of queue (=ports) per lcore (the default is 1) + +To run the example in linuxapp environment with 2 lcores (2,4) over 2 ports(0,2) with 1 RX queue per lcore: + +.. code-block:: console + + ./build/ip_fragmentation -l 2,4 -n 3 -- -p 5 + EAL: coremask set to 14 + EAL: Detected lcore 0 on socket 0 + EAL: Detected lcore 1 on socket 1 + EAL: Detected lcore 2 on socket 0 + EAL: Detected lcore 3 on socket 1 + EAL: Detected lcore 4 on socket 0 + ... + + Initializing port 0 on lcore 2... Address:00:1B:21:76:FA:2C, rxq=0 txq=2,0 txq=4,1 + done: Link Up - speed 10000 Mbps - full-duplex + Skipping disabled port 1 + Initializing port 2 on lcore 4... Address:00:1B:21:5C:FF:54, rxq=0 txq=2,0 txq=4,1 + done: Link Up - speed 10000 Mbps - full-duplex + Skipping disabled port 3IP_FRAG: Socket 0: adding route 100.10.0.0/16 (port 0) + IP_FRAG: Socket 0: adding route 100.20.0.0/16 (port 1) + ... + IP_FRAG: Socket 0: adding route 0101:0101:0101:0101:0101:0101:0101:0101/48 (port 0) + IP_FRAG: Socket 0: adding route 0201:0101:0101:0101:0101:0101:0101:0101/48 (port 1) + ... + IP_FRAG: entering main loop on lcore 4 + IP_FRAG: -- lcoreid=4 portid=2 + IP_FRAG: entering main loop on lcore 2 + IP_FRAG: -- lcoreid=2 portid=0 + +To run the example in linuxapp environment with 1 lcore (4) over 2 ports(0,2) with 2 RX queues per lcore: + +.. code-block:: console + + ./build/ip_fragmentation -l 4 -n 3 -- -p 5 -q 2 + +To test the application, flows should be set up in the flow generator that match the values in the +l3fwd_ipv4_route_array and/or l3fwd_ipv6_route_array table. + +The default l3fwd_ipv4_route_array table is: + +.. code-block:: c + + struct l3fwd_ipv4_route l3fwd_ipv4_route_array[] = { + {IPv4(100, 10, 0, 0), 16, 0}, + {IPv4(100, 20, 0, 0), 16, 1}, + {IPv4(100, 30, 0, 0), 16, 2}, + {IPv4(100, 40, 0, 0), 16, 3}, + {IPv4(100, 50, 0, 0), 16, 4}, + {IPv4(100, 60, 0, 0), 16, 5}, + {IPv4(100, 70, 0, 0), 16, 6}, + {IPv4(100, 80, 0, 0), 16, 7}, + }; + +The default l3fwd_ipv6_route_array table is: + +.. code-block:: c + + struct l3fwd_ipv6_route l3fwd_ipv6_route_array[] = { + {{1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 0}, + {{2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 1}, + {{3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 2}, + {{4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 3}, + {{5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 4}, + {{6, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 5}, + {{7, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 6}, + {{8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 7}, + }; + +For example, for the input IPv4 packet with destination address: 100.10.1.1 and packet length 9198 bytes, +seven IPv4 packets will be sent out from port #0 to the destination address 100.10.1.1: +six of those packets will have length 1500 bytes and one packet will have length 318 bytes. +IP Fragmentation sample application provides basic NUMA support +in that all the memory structures are allocated on all sockets that have active lcores on them. + + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ip_pipeline.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_pipeline.rst new file mode 100644 index 00000000..b75509a0 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_pipeline.rst @@ -0,0 +1,530 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015-2018 Intel Corporation. + +Internet Protocol (IP) Pipeline Application +=========================================== + +Application overview +-------------------- + +The *Internet Protocol (IP) Pipeline* application is intended to be a vehicle for rapid development of packet processing +applications on multi-core CPUs. + +Following OpenFlow and P4 design principles, the application can be used to create functional blocks called pipelines out +of input/output ports, tables and actions in a modular way. Multiple pipelines can be inter-connected through packet queues +to create complete applications (super-pipelines). + +The pipelines are mapped to application threads, with each pipeline executed by a single thread and each thread able to run +one or several pipelines. The possibilities of creating pipelines out of ports, tables and actions, connecting multiple +pipelines together and mapping the pipelines to execution threads are endless, therefore this application can be seen as +a true application generator. + +Pipelines are created and managed through Command Line Interface (CLI): + + * Any standard TCP client (e.g. telnet, netcat, custom script, etc) is typically able to connect to the application, send + commands through the network and wait for the response before pushing the next command. + + * All the application objects are created and managed through CLI commands: + * 'Primitive' objects used to create pipeline ports: memory pools, links (i.e. network interfaces), SW queues, traffic managers, etc. + * Action profiles: used to define the actions to be executed by pipeline input/output ports and tables. + * Pipeline components: input/output ports, tables, pipelines, mapping of pipelines to execution threads. + +Running the application +----------------------- + +The application startup command line is:: + + ip_pipeline [EAL_ARGS] -- [-s SCRIPT_FILE] [-h HOST] [-p PORT] + +The application startup arguments are: + +``-s SCRIPT_FILE`` + + * Optional: Yes + + * Default: Not present + + * Argument: Path to the CLI script file to be run at application startup. + No CLI script file will run at startup if this argument is not present. + +``-h HOST`` + + * Optional: Yes + + * Default: ``0.0.0.0`` + + * Argument: IP Address of the host running ip pipeline application to be used by + remote TCP based client (telnet, netcat, etc.) for connection. + +``-p PORT`` + + * Optional: Yes + + * Default: ``8086`` + + * Argument: TCP port number at which the ip pipeline is running. + This port number should be used by remote TCP client (such as telnet, netcat, etc.) to connect to host application. + +Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. + +The following is an example command to run ip pipeline application configured for layer 2 forwarding: + +.. code-block:: console + + $ ./build/ip_pipeline -c 0x3 -- -s examples/route_ecmp.cli + +The application should start successfully and display as follows: + +.. code-block:: console + + EAL: Detected 40 lcore(s) + EAL: Detected 2 NUMA nodes + EAL: Multi-process socket /var/run/.rte_unix + EAL: Probing VFIO support... + EAL: PCI device 0000:02:00.0 on NUMA socket 0 + EAL: probe driver: 8086:10fb net_ixgbe + ... + +To run remote client (e.g. telnet) to communicate with the ip pipeline application: + +.. code-block:: console + + $ telnet 127.0.0.1 8086 + +When running a telnet client as above, command prompt is displayed: + +.. code-block:: console + + Trying 127.0.0.1... + Connected to 127.0.0.1. + Escape character is '^]'. + + Welcome to IP Pipeline! + + pipeline> + +Once application and telnet client start running, messages can be sent from client to application. +At any stage, telnet client can be terminated using the quit command. + + +Application stages +------------------ + +Initialization +~~~~~~~~~~~~~~ + +During this stage, EAL layer is initialised and application specific arguments are parsed. Furthermore, the data strcutures +(i.e. linked lists) for application objects are initialized. In case of any initialization error, an error message +is displayed and the application is terminated. + +.. _ip_pipeline_runtime: + +Run-time +~~~~~~~~ + +The master thread is creating and managing all the application objects based on CLI input. + +Each data plane thread runs one or several pipelines previously assigned to it in round-robin order. Each data plane thread +executes two tasks in time-sharing mode: + +1. *Packet processing task*: Process bursts of input packets read from the pipeline input ports. + +2. *Message handling task*: Periodically, the data plane thread pauses the packet processing task and polls for request + messages send by the master thread. Examples: add/remove pipeline to/from current data plane thread, add/delete rules + to/from given table of a specific pipeline owned by the current data plane thread, read statistics, etc. + +Examples +-------- + +.. _table_examples: + +.. tabularcolumns:: |p{3cm}|p{5cm}|p{4cm}|p{4cm}| + +.. table:: Pipeline examples provided with the application + + +-----------------------+----------------------+----------------+------------------------------------+ + | Name | Table(s) | Actions | Messages | + +=======================+======================+================+====================================+ + | L2fwd | Stub | Forward | 1. Mempool create | + | | | | 2. Link create | + | Note: Implemented | | | 3. Pipeline create | + | using pipeline with | | | 4. Pipeline port in/out | + | a simple pass-through | | | 5. Pipeline table | + | connection between | | | 6. Pipeline port in table | + | input and output | | | 7. Pipeline enable | + | ports. | | | 8. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Flow classification | Exact match | Forward | 1. Mempool create | + | | | | 2. Link create | + | | * Key = byte array | | 3. Pipeline create | + | | (16 bytes) | | 4. Pipeline port in/out | + | | * Offset = 278 | | 5. Pipeline table | + | | * Table size = 64K | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | KNI | Stub | Forward | 1. Mempool create | + | | | | 2. Link create | + | | | | 3. Pipeline create | + | | | | 4. Pipeline port in/out | + | | | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Firewall | ACL | Allow/Drop | 1. Mempool create | + | | | | 2. Link create | + | | * Key = n-tuple | | 3. Pipeline create | + | | * Offset = 270 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | IP routing | LPM (IPv4) | Forward | 1. Mempool Create | + | | | | 2. Link create | + | | * Key = IP dest addr | | 3. Pipeline creat | + | | * Offset = 286 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Equal-cost multi-path | LPM (IPv4) | Forward, | 1. Mempool Create | + | routing (ECMP) | | load balance, | 2. Link create | + | | * Key = IP dest addr | encap ether | 3. Pipeline create | + | | * Offset = 286 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table (LPM) | + | | | | 6. Pipeline table (Array) | + | | | | 7. Pipeline port in table (LPM) | + | | Array | | 8. Pipeline enable | + | | | | 9. Pipeline table rule add default | + | | * Key = Array index | | 10. Pipeline table rule add(LPM) | + | | * Offset = 256 | | 11. Pipeline table rule add(Array) | + | | * Size = 64K | | | + | | | | | + +-----------------------+----------------------+----------------+------------------------------------+ + +Command Line Interface (CLI) +---------------------------- + +Link +~~~~ + + Link configuration :: + + link <link_name> + dev <device_name>|port <port_id> + rxq <n_queues> <queue_size> <mempool_name> + txq <n_queues> <queue_size> promiscuous on | off + [rss <qid_0> ... <qid_n>] + + Note: The PCI device name must be specified in the Domain:Bus:Device.Function format. + + +Mempool +~~~~~~~ + + Mempool create :: + + mempool <mempool_name> buffer <buffer_size> + pool <pool_size> cache <cache_size> cpu <cpu_id> + + +Software queue +~~~~~~~~~~~~~~ + + Create software queue :: + + swq <swq_name> size <size> cpu <cpu_id> + + +Traffic manager +~~~~~~~~~~~~~~~ + + Add traffic manager subport profile :: + + tmgr subport profile + <tb_rate> <tb_size> + <tc0_rate> <tc1_rate> <tc2_rate> <tc3_rate> + <tc_period> + + + Add traffic manager pipe profile :: + + tmgr pipe profile + <tb_rate> <tb_size> + <tc0_rate> <tc1_rate> <tc2_rate> <tc3_rate> + <tc_period> + <tc_ov_weight> <wrr_weight0..15> + + Create traffic manager port :: + + tmgr <tmgr_name> + rate <rate> + spp <n_subports_per_port> + pps <n_pipes_per_subport> + qsize <qsize_tc0> + <qsize_tc1> <qsize_tc2> <qsize_tc3> + fo <frame_overhead> mtu <mtu> cpu <cpu_id> + + Configure traffic manager subport :: + + tmgr <tmgr_name> + subport <subport_id> + profile <subport_profile_id> + + Configure traffic manager pipe :: + + tmgr <tmgr_name> + subport <subport_id> + pipe from <pipe_id_first> to <pipe_id_last> + profile <pipe_profile_id> + + +Tap +~~~ + + Create tap port :: + + tap <name> + + +Kni +~~~ + + Create kni port :: + + kni <kni_name> + link <link_name> + mempool <mempool_name> + [thread <thread_id>] + + +Action profile +~~~~~~~~~~~~~~ + + Create action profile for pipeline input port :: + + port in action profile <profile_name> + [filter match | mismatch offset <key_offset> mask <key_mask> key <key_value> port <port_id>] + [balance offset <key_offset> mask <key_mask> port <port_id0> ... <port_id15>] + + Create action profile for the pipeline table :: + + table action profile <profile_name> + ipv4 | ipv6 + offset <ip_offset> + fwd + [balance offset <key_offset> mask <key_mask> outoffset <out_offset>] + [meter srtcm | trtcm + tc <n_tc> + stats none | pkts | bytes | both] + [tm spp <n_subports_per_port> pps <n_pipes_per_subport>] + [encap ether | vlan | qinq | mpls | pppoe] + [nat src | dst + proto udp | tcp] + [ttl drop | fwd + stats none | pkts] + [stats pkts | bytes | both] + [time] + + +Pipeline +~~~~~~~~ + +Create pipeline :: + + pipeline <pipeline_name> + period <timer_period_ms> + offset_port_id <offset_port_id> + cpu <cpu_id> + +Create pipeline input port :: + + pipeline <pipeline_name> port in + bsz <burst_size> + link <link_name> rxq <queue_id> + | swq <swq_name> + | tmgr <tmgr_name> + | tap <tap_name> mempool <mempool_name> mtu <mtu> + | kni <kni_name> + | source mempool <mempool_name> file <file_name> bpp <n_bytes_per_pkt> + [action <port_in_action_profile_name>] + [disabled] + +Create pipeline output port :: + + pipeline <pipeline_name> port out + bsz <burst_size> + link <link_name> txq <txq_id> + | swq <swq_name> + | tmgr <tmgr_name> + | tap <tap_name> + | kni <kni_name> + | sink [file <file_name> pkts <max_n_pkts>] + +Create pipeline table :: + + pipeline <pipeline_name> table + match + acl + ipv4 | ipv6 + offset <ip_header_offset> + size <n_rules> + | array + offset <key_offset> + size <n_keys> + | hash + ext | lru + key <key_size> + mask <key_mask> + offset <key_offset> + buckets <n_buckets> + size <n_keys> + | lpm + ipv4 | ipv6 + offset <ip_header_offset> + size <n_rules> + | stub + [action <table_action_profile_name>] + +Connect pipeline input port to table :: + + pipeline <pipeline_name> port in <port_id> table <table_id> + +Display statistics for specific pipeline input port, output port +or table :: + + pipeline <pipeline_name> port in <port_id> stats read [clear] + pipeline <pipeline_name> port out <port_id> stats read [clear] + pipeline <pipeline_name> table <table_id> stats read [clear] + +Enable given input port for specific pipeline instance :: + + pipeline <pipeline_name> port out <port_id> disable + +Disable given input port for specific pipeline instance :: + + pipeline <pipeline_name> port out <port_id> disable + +Add default rule to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add + match + default + action + fwd + drop + | port <port_id> + | meta + | table <table_id> + +Add rule to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add + + match + acl + priority <priority> + ipv4 | ipv6 <sa> <sa_depth> <da> <da_depth> + <sp0> <sp1> <dp0> <dp1> <proto> + | array <pos> + | hash + raw <key> + | ipv4_5tuple <sa> <da> <sp> <dp> <proto> + | ipv6_5tuple <sa> <da> <sp> <dp> <proto> + | ipv4_addr <addr> + | ipv6_addr <addr> + | qinq <svlan> <cvlan> + | lpm + ipv4 | ipv6 <addr> <depth> + + action + fwd + drop + | port <port_id> + | meta + | table <table_id> + [balance <out0> ... <out7>] + [meter + tc0 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + [tc1 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + tc2 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + tc3 meter <meter_profile_id> policer g <pa> y <pa> r <pa>]] + [tm subport <subport_id> pipe <pipe_id>] + [encap + ether <da> <sa> + | vlan <da> <sa> <pcp> <dei> <vid> + | qinq <da> <sa> <pcp> <dei> <vid> <pcp> <dei> <vid> + | mpls unicast | multicast + <da> <sa> + label0 <label> <tc> <ttl> + [label1 <label> <tc> <ttl> + [label2 <label> <tc> <ttl> + [label3 <label> <tc> <ttl>]]] + | pppoe <da> <sa> <session_id>] + [nat ipv4 | ipv6 <addr> <port>] + [ttl dec | keep] + [stats] + [time] + + where: + <pa> ::= g | y | r | drop + +Add bulk rules to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add bulk <file_name> <n_rules> + + Where: + - file_name = path to file + - File line format = match <match> action <action> + +Delete table rule for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule delete + match <match> + +Delete default table rule for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule delete + match + default + +Add meter profile to the table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> meter profile <meter_profile_id> + add srtcm cir <cir> cbs <cbs> ebs <ebs> + | trtcm cir <cir> pir <pir> cbs <cbs> pbs <pbs> + +Delete meter profile from the table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> + meter profile <meter_profile_id> delete + + +Update the dscp table for meter or traffic manager action for specific +pipeline instance :: + + pipeline <pipeline_name> table <table_id> dscp <file_name> + + Where: + - file_name = path to file + - exactly 64 lines + - File line format = <tc_id> <tc_queue_id> <color>, with <color> as: g | y | r + + +Pipeline enable/disable +~~~~~~~~~~~~~~~~~~~~~~~ + + Enable given pipeline instance for specific data plane thread :: + + thread <thread_id> pipeline <pipeline_name> enable + + + Disable given pipeline instance for specific data plane thread :: + + thread <thread_id> pipeline <pipeline_name> disable diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ip_reassembly.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_reassembly.rst new file mode 100644 index 00000000..18912cd2 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ip_reassembly.rst @@ -0,0 +1,238 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +IP Reassembly Sample Application +================================ + +The L3 Forwarding application is a simple example of packet processing using the DPDK. +The application performs L3 forwarding with reassembly for fragmented IPv4 and IPv6 packets. + +Overview +-------- + +The application demonstrates the use of the DPDK libraries to implement packet forwarding +with reassembly for IPv4 and IPv6 fragmented packets. +The initialization and run- time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +The main difference from the L2 Forwarding sample application is that +it reassembles fragmented IPv4 and IPv6 packets before forwarding. +The maximum allowed size of reassembled packet is 9.5 KB. + +There are two key differences from the L2 Forwarding sample application: + +* The first difference is that the forwarding decision is taken based on information read from the input packet's IP header. + +* The second difference is that the application differentiates between IP and non-IP traffic by means of offload flags. + +The Longest Prefix Match (LPM for IPv4, LPM6 for IPv6) table is used to store/lookup an outgoing port number, associated with that IPv4 address. Any unmatched packets are forwarded to the originating port.Compiling the Application +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ip_reassembly`` sub-directory. + + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/ip_reassembly [EAL options] -- -p PORTMASK [-q NQ] [--maxflows=FLOWS>] [--flowttl=TTL[(s|ms)]] + +where: + +* -p PORTMASK: Hexadecimal bitmask of ports to configure + +* -q NQ: Number of RX queues per lcore + +* --maxflows=FLOWS: determines maximum number of active fragmented flows (1-65535). Default value: 4096. + +* --flowttl=TTL[(s|ms)]: determines maximum Time To Live for fragmented packet. + If all fragments of the packet wouldn't appear within given time-out, + then they are considered as invalid and will be dropped. + Valid range is 1ms - 3600s. Default value: 1s. + +To run the example in linuxapp environment with 2 lcores (2,4) over 2 ports(0,2) with 1 RX queue per lcore: + +.. code-block:: console + + ./build/ip_reassembly -l 2,4 -n 3 -- -p 5 + EAL: coremask set to 14 + EAL: Detected lcore 0 on socket 0 + EAL: Detected lcore 1 on socket 1 + EAL: Detected lcore 2 on socket 0 + EAL: Detected lcore 3 on socket 1 + EAL: Detected lcore 4 on socket 0 + ... + + Initializing port 0 on lcore 2... Address:00:1B:21:76:FA:2C, rxq=0 txq=2,0 txq=4,1 + done: Link Up - speed 10000 Mbps - full-duplex + Skipping disabled port 1 + Initializing port 2 on lcore 4... Address:00:1B:21:5C:FF:54, rxq=0 txq=2,0 txq=4,1 + done: Link Up - speed 10000 Mbps - full-duplex + Skipping disabled port 3IP_FRAG: Socket 0: adding route 100.10.0.0/16 (port 0) + IP_RSMBL: Socket 0: adding route 100.20.0.0/16 (port 1) + ... + + IP_RSMBL: Socket 0: adding route 0101:0101:0101:0101:0101:0101:0101:0101/48 (port 0) + IP_RSMBL: Socket 0: adding route 0201:0101:0101:0101:0101:0101:0101:0101/48 (port 1) + ... + + IP_RSMBL: entering main loop on lcore 4 + IP_RSMBL: -- lcoreid=4 portid=2 + IP_RSMBL: entering main loop on lcore 2 + IP_RSMBL: -- lcoreid=2 portid=0 + +To run the example in linuxapp environment with 1 lcore (4) over 2 ports(0,2) with 2 RX queues per lcore: + +.. code-block:: console + + ./build/ip_reassembly -l 4 -n 3 -- -p 5 -q 2 + +To test the application, flows should be set up in the flow generator that match the values in the +l3fwd_ipv4_route_array and/or l3fwd_ipv6_route_array table. + +Please note that in order to test this application, +the traffic generator should be generating valid fragmented IP packets. +For IPv6, the only supported case is when no other extension headers other than +fragment extension header are present in the packet. + +The default l3fwd_ipv4_route_array table is: + +.. code-block:: c + + struct l3fwd_ipv4_route l3fwd_ipv4_route_array[] = { + {IPv4(100, 10, 0, 0), 16, 0}, + {IPv4(100, 20, 0, 0), 16, 1}, + {IPv4(100, 30, 0, 0), 16, 2}, + {IPv4(100, 40, 0, 0), 16, 3}, + {IPv4(100, 50, 0, 0), 16, 4}, + {IPv4(100, 60, 0, 0), 16, 5}, + {IPv4(100, 70, 0, 0), 16, 6}, + {IPv4(100, 80, 0, 0), 16, 7}, + }; + +The default l3fwd_ipv6_route_array table is: + +.. code-block:: c + + struct l3fwd_ipv6_route l3fwd_ipv6_route_array[] = { + {{1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 0}, + {{2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 1}, + {{3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 2}, + {{4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 3}, + {{5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 4}, + {{6, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 5}, + {{7, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 6}, + {{8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, 48, 7}, + }; + +For example, for the fragmented input IPv4 packet with destination address: 100.10.1.1, +a reassembled IPv4 packet be sent out from port #0 to the destination address 100.10.1.1 +once all the fragments are collected. + +Explanation +----------- + +The following sections provide some explanation of the sample application code. +As mentioned in the overview section, the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +The following sections describe aspects that are specific to the IP reassemble sample application. + +IPv4 Fragment Table Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This application uses the rte_ip_frag library. Please refer to Programmer's Guide for more detailed explanation of how to use this library. +Fragment table maintains information about already received fragments of the packet. +Each IP packet is uniquely identified by triple <Source IP address>, <Destination IP address>, <ID>. +To avoid lock contention, each RX queue has its own Fragment Table, +e.g. the application can't handle the situation when different fragments of the same packet arrive through different RX queues. +Each table entry can hold information about packet consisting of up to RTE_LIBRTE_IP_FRAG_MAX_FRAGS fragments. + +.. code-block:: c + + frag_cycles = (rte_get_tsc_hz() + MS_PER_S - 1) / MS_PER_S * max_flow_ttl; + + if ((qconf->frag_tbl[queue] = rte_ip_frag_tbl_create(max_flow_num, IPV4_FRAG_TBL_BUCKET_ENTRIES, max_flow_num, frag_cycles, socket)) == NULL) + { + RTE_LOG(ERR, IP_RSMBL, "ip_frag_tbl_create(%u) on " "lcore: %u for queue: %u failed\n", max_flow_num, lcore, queue); + return -1; + } + +Mempools Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +The reassembly application demands a lot of mbuf's to be allocated. +At any given time up to (2 \* max_flow_num \* RTE_LIBRTE_IP_FRAG_MAX_FRAGS \* <maximum number of mbufs per packet>) +can be stored inside Fragment Table waiting for remaining fragments. +To keep mempool size under reasonable limits and to avoid situation when one RX queue can starve other queues, +each RX queue uses its own mempool. + +.. code-block:: c + + nb_mbuf = RTE_MAX(max_flow_num, 2UL * MAX_PKT_BURST) * RTE_LIBRTE_IP_FRAG_MAX_FRAGS; + nb_mbuf *= (port_conf.rxmode.max_rx_pkt_len + BUF_SIZE - 1) / BUF_SIZE; + nb_mbuf *= 2; /* ipv4 and ipv6 */ + nb_mbuf += RTE_TEST_RX_DESC_DEFAULT + RTE_TEST_TX_DESC_DEFAULT; + nb_mbuf = RTE_MAX(nb_mbuf, (uint32_t)NB_MBUF); + + snprintf(buf, sizeof(buf), "mbuf_pool_%u_%u", lcore, queue); + + if ((rxq->pool = rte_mempool_create(buf, nb_mbuf, MBUF_SIZE, 0, sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, socket, MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET)) == NULL) { + + RTE_LOG(ERR, IP_RSMBL, "mempool_create(%s) failed", buf); + return -1; + } + +Packet Reassembly and Forwarding +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For each input packet, the packet forwarding operation is done by the l3fwd_simple_forward() function. +If the packet is an IPv4 or IPv6 fragment, then it calls rte_ipv4_reassemble_packet() for IPv4 packets, +or rte_ipv6_reassemble_packet() for IPv6 packets. +These functions either return a pointer to valid mbuf that contains reassembled packet, +or NULL (if the packet can't be reassembled for some reason). +Then l3fwd_simple_forward() continues with the code for the packet forwarding decision +(that is, the identification of the output interface for the packet) and +actual transmit of the packet. + +The rte_ipv4_reassemble_packet() or rte_ipv6_reassemble_packet() are responsible for: + +#. Searching the Fragment Table for entry with packet's <IP Source Address, IP Destination Address, Packet ID> + +#. If the entry is found, then check if that entry already timed-out. + If yes, then free all previously received fragments, + and remove information about them from the entry. + +#. If no entry with such key is found, then try to create a new one by one of two ways: + + #. Use as empty entry + + #. Delete a timed-out entry, free mbufs associated with it mbufs and store a new entry with specified key in it. + +#. Update the entry with new fragment information and check + if a packet can be reassembled (the packet's entry contains all fragments). + + #. If yes, then, reassemble the packet, mark table's entry as empty and return the reassembled mbuf to the caller. + + #. If no, then just return a NULL to the caller. + +If at any stage of packet processing a reassembly function encounters an error +(can't insert new entry into the Fragment table, or invalid/timed-out fragment), +then it will free all associated with the packet fragments, +mark the table entry as invalid and return NULL to the caller. + +Debug logging and Statistics Collection +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The RTE_LIBRTE_IP_FRAG_TBL_STAT controls statistics collection for the IP Fragment Table. +This macro is disabled by default. +To make ip_reassembly print the statistics to the standard output, +the user must send either an USR1, INT or TERM signal to the process. +For all of these signals, the ip_reassembly process prints Fragment table statistics for each RX queue, +plus the INT and TERM will cause process termination as usual. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ipsec_secgw.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ipsec_secgw.rst new file mode 100644 index 00000000..46696f2a --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ipsec_secgw.rst @@ -0,0 +1,620 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016-2017 Intel Corporation. + +IPsec Security Gateway Sample Application +========================================= + +The IPsec Security Gateway application is an example of a "real world" +application using DPDK cryptodev framework. + +Overview +-------- + +The application demonstrates the implementation of a Security Gateway +(not IPsec compliant, see the Constraints section below) using DPDK based on RFC4301, +RFC4303, RFC3602 and RFC2404. + +Internet Key Exchange (IKE) is not implemented, so only manual setting of +Security Policies and Security Associations is supported. + +The Security Policies (SP) are implemented as ACL rules, the Security +Associations (SA) are stored in a table and the routing is implemented +using LPM. + +The application classifies the ports as *Protected* and *Unprotected*. +Thus, traffic received on an Unprotected or Protected port is consider +Inbound or Outbound respectively. + +The application also supports complete IPSec protocol offload to hardware +(Look aside crypto accelarator or using ethernet device). It also support +inline ipsec processing by the supported ethernet device during transmission. +These modes can be selected during the SA creation configuration. + +In case of complete protocol offload, the processing of headers(ESP and outer +IP header) is done by the hardware and the application does not need to +add/remove them during outbound/inbound processing. + +For inline offloaded outbound traffic, the application will not do the LPM +lookup for routing, as the port on which the packet has to be forwarded will be +part of the SA. Security parameters will be configured on that port only, and +sending the packet on other ports could result in unencrypted packets being +sent out. + +The Path for IPsec Inbound traffic is: + +* Read packets from the port. +* Classify packets between IPv4 and ESP. +* Perform Inbound SA lookup for ESP packets based on their SPI. +* Perform Verification/Decryption (Not needed in case of inline ipsec). +* Remove ESP and outer IP header (Not needed in case of protocol offload). +* Inbound SP check using ACL of decrypted packets and any other IPv4 packets. +* Routing. +* Write packet to port. + +The Path for the IPsec Outbound traffic is: + +* Read packets from the port. +* Perform Outbound SP check using ACL of all IPv4 traffic. +* Perform Outbound SA lookup for packets that need IPsec protection. +* Add ESP and outer IP header (Not needed in case protocol offload). +* Perform Encryption/Digest (Not needed in case of inline ipsec). +* Routing. +* Write packet to port. + + +Constraints +----------- + +* No IPv6 options headers. +* No AH mode. +* Supported algorithms: AES-CBC, AES-CTR, AES-GCM, HMAC-SHA1 and NULL. +* Each SA must be handle by a unique lcore (*1 RX queue per port*). +* No chained mbufs. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``rpsec-secgw`` sub-directory. + +#. [Optional] Build the application for debugging: + This option adds some extra flags, disables compiler optimizations and + is verbose:: + + make DEBUG=1 + + +Running the Application +----------------------- + +The application has a number of command line options:: + + + ./build/ipsec-secgw [EAL options] -- + -p PORTMASK -P -u PORTMASK -j FRAMESIZE + --config (port,queue,lcore)[,(port,queue,lcore] + --single-sa SAIDX + -f CONFIG_FILE_PATH + +Where: + +* ``-p PORTMASK``: Hexadecimal bitmask of ports to configure. + +* ``-P``: *optional*. Sets all ports to promiscuous mode so that packets are + accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address + set to the Ethernet address of the port are accepted (default is enabled). + +* ``-u PORTMASK``: hexadecimal bitmask of unprotected ports + +* ``-j FRAMESIZE``: *optional*. Enables jumbo frames with the maximum size + specified as FRAMESIZE. If an invalid value is provided as FRAMESIZE + then the default value 9000 is used. + +* ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues + from which ports are mapped to which cores. + +* ``--single-sa SAIDX``: use a single SA for outbound traffic, bypassing the SP + on both Inbound and Outbound. This option is meant for debugging/performance + purposes. + +* ``-f CONFIG_FILE_PATH``: the full path of text-based file containing all + configuration items for running the application (See Configuration file + syntax section below). ``-f CONFIG_FILE_PATH`` **must** be specified. + **ONLY** the UNIX format configuration file is accepted. + + +The mapping of lcores to port/queues is similar to other l3fwd applications. + +For example, given the following command line:: + + ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \ + --vdev "crypto_null" -- -p 0xf -P -u 0x3 \ + --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \ + -f /path/to/config_file \ + +where each options means: + +* The ``-l`` option enables cores 20 and 21. + +* The ``-n`` option sets memory 4 channels. + +* The ``--socket-mem`` to use 2GB on socket 1. + +* The ``--vdev "crypto_null"`` option creates virtual NULL cryptodev PMD. + +* The ``-p`` option enables ports (detected) 0, 1, 2 and 3. + +* The ``-P`` option enables promiscuous mode. + +* The ``-u`` option sets ports 1 and 2 as unprotected, leaving 2 and 3 as protected. + +* The ``--config`` option enables one queue per port with the following mapping: + + +----------+-----------+-----------+---------------------------------------+ + | **Port** | **Queue** | **lcore** | **Description** | + | | | | | + +----------+-----------+-----------+---------------------------------------+ + | 0 | 0 | 20 | Map queue 0 from port 0 to lcore 20. | + | | | | | + +----------+-----------+-----------+---------------------------------------+ + | 1 | 0 | 20 | Map queue 0 from port 1 to lcore 20. | + | | | | | + +----------+-----------+-----------+---------------------------------------+ + | 2 | 0 | 21 | Map queue 0 from port 2 to lcore 21. | + | | | | | + +----------+-----------+-----------+---------------------------------------+ + | 3 | 0 | 21 | Map queue 0 from port 3 to lcore 21. | + | | | | | + +----------+-----------+-----------+---------------------------------------+ + +* The ``-f /path/to/config_file`` option enables the application read and + parse the configuration file specified, and configures the application + with a given set of SP, SA and Routing entries accordingly. The syntax of + the configuration file will be explained below in more detail. Please + **note** the parser only accepts UNIX format text file. Other formats + such as DOS/MAC format will cause a parse error. + +Refer to the *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + +The application would do a best effort to "map" crypto devices to cores, with +hardware devices having priority. Basically, hardware devices if present would +be assigned to a core before software ones. +This means that if the application is using a single core and both hardware +and software crypto devices are detected, hardware devices will be used. + +A way to achieve the case where you want to force the use of virtual crypto +devices is to whitelist the Ethernet devices needed and therefore implicitly +blacklisting all hardware crypto devices. + +For example, something like the following command line: + +.. code-block:: console + + ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \ + -w 81:00.0 -w 81:00.1 -w 81:00.2 -w 81:00.3 \ + --vdev "crypto_aesni_mb" --vdev "crypto_null" \ + -- \ + -p 0xf -P -u 0x3 --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \ + -f sample.cfg + + +Configurations +-------------- + +The following sections provide the syntax of configurations to initialize +your SP, SA and Routing tables. +Configurations shall be specified in the configuration file to be passed to +the application. The file is then parsed by the application. The successful +parsing will result in the appropriate rules being applied to the tables +accordingly. + + +Configuration File Syntax +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As mention in the overview, the Security Policies are ACL rules. +The application parsers the rules specified in the configuration file and +passes them to the ACL table, and replicates them per socket in use. + +Following are the configuration file syntax. + +General rule syntax +^^^^^^^^^^^^^^^^^^^ + +The parse treats one line in the configuration file as one configuration +item (unless the line concatenation symbol exists). Every configuration +item shall follow the syntax of either SP, SA, or Routing rules specified +below. + +The configuration parser supports the following special symbols: + + * Comment symbol **#**. Any character from this symbol to the end of + line is treated as comment and will not be parsed. + + * Line concatenation symbol **\\**. This symbol shall be placed in the end + of the line to be concatenated to the line below. Multiple lines' + concatenation is supported. + + +SP rule syntax +^^^^^^^^^^^^^^ + +The SP rule syntax is shown as follows: + +.. code-block:: console + + sp <ip_ver> <dir> esp <action> <priority> <src_ip> <dst_ip> + <proto> <sport> <dport> + + +where each options means: + +``<ip_ver>`` + + * IP protocol version + + * Optional: No + + * Available options: + + * *ipv4*: IP protocol version 4 + * *ipv6*: IP protocol version 6 + +``<dir>`` + + * The traffic direction + + * Optional: No + + * Available options: + + * *in*: inbound traffic + * *out*: outbound traffic + +``<action>`` + + * IPsec action + + * Optional: No + + * Available options: + + * *protect <SA_idx>*: the specified traffic is protected by SA rule + with id SA_idx + * *bypass*: the specified traffic traffic is bypassed + * *discard*: the specified traffic is discarded + +``<priority>`` + + * Rule priority + + * Optional: Yes, default priority 0 will be used + + * Syntax: *pri <id>* + +``<src_ip>`` + + * The source IP address and mask + + * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used + + * Syntax: + + * *src X.X.X.X/Y* for IPv4 + * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6 + +``<dst_ip>`` + + * The destination IP address and mask + + * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used + + * Syntax: + + * *dst X.X.X.X/Y* for IPv4 + * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6 + +``<proto>`` + + * The protocol start and end range + + * Optional: yes, default range of 0 to 0 will be used + + * Syntax: *proto X:Y* + +``<sport>`` + + * The source port start and end range + + * Optional: yes, default range of 0 to 0 will be used + + * Syntax: *sport X:Y* + +``<dport>`` + + * The destination port start and end range + + * Optional: yes, default range of 0 to 0 will be used + + * Syntax: *dport X:Y* + +Example SP rules: + +.. code-block:: console + + sp ipv4 out esp protect 105 pri 1 dst 192.168.115.0/24 sport 0:65535 \ + dport 0:65535 + + sp ipv6 in esp bypass pri 1 dst 0000:0000:0000:0000:5555:5555:\ + 0000:0000/96 sport 0:65535 dport 0:65535 + + +SA rule syntax +^^^^^^^^^^^^^^ + +The successfully parsed SA rules will be stored in an array table. + +The SA rule syntax is shown as follows: + +.. code-block:: console + + sa <dir> <spi> <cipher_algo> <cipher_key> <auth_algo> <auth_key> + <mode> <src_ip> <dst_ip> <action_type> <port_id> + +where each options means: + +``<dir>`` + + * The traffic direction + + * Optional: No + + * Available options: + + * *in*: inbound traffic + * *out*: outbound traffic + +``<spi>`` + + * The SPI number + + * Optional: No + + * Syntax: unsigned integer number + +``<cipher_algo>`` + + * Cipher algorithm + + * Optional: Yes, unless <aead_algo> is not used + + * Available options: + + * *null*: NULL algorithm + * *aes-128-cbc*: AES-CBC 128-bit algorithm + * *aes-256-cbc*: AES-CBC 256-bit algorithm + * *aes-128-ctr*: AES-CTR 128-bit algorithm + + * Syntax: *cipher_algo <your algorithm>* + +``<cipher_key>`` + + * Cipher key, NOT available when 'null' algorithm is used + + * Optional: Yes, unless <aead_algo> is not used. + Must be followed by <cipher_algo> option + + * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. + The number of bytes should be as same as the specified cipher algorithm + key size. + + For example: *cipher_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4: + A1:B2:C3:D4* + +``<auth_algo>`` + + * Authentication algorithm + + * Optional: Yes, unless <aead_algo> is not used + + * Available options: + + * *null*: NULL algorithm + * *sha1-hmac*: HMAC SHA1 algorithm + +``<auth_key>`` + + * Authentication key, NOT available when 'null' or 'aes-128-gcm' algorithm + is used. + + * Optional: Yes, unless <aead_algo> is not used. + Must be followed by <auth_algo> option + + * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. + The number of bytes should be as same as the specified authentication + algorithm key size. + + For example: *auth_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4: + A1:B2:C3:D4* + +``<aead_algo>`` + + * AEAD algorithm + + * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used + + * Available options: + + * *aes-128-gcm*: AES-GCM 128-bit algorithm + + * Syntax: *cipher_algo <your algorithm>* + +``<aead_key>`` + + * Cipher key, NOT available when 'null' algorithm is used + + * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used. + Must be followed by <aead_algo> option + + * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. + The number of bytes should be as same as the specified AEAD algorithm + key size. + + For example: *aead_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4: + A1:B2:C3:D4* + +``<mode>`` + + * The operation mode + + * Optional: No + + * Available options: + + * *ipv4-tunnel*: Tunnel mode for IPv4 packets + * *ipv6-tunnel*: Tunnel mode for IPv6 packets + * *transport*: transport mode + + * Syntax: mode XXX + +``<src_ip>`` + + * The source IP address. This option is not available when + transport mode is used + + * Optional: Yes, default address 0.0.0.0 will be used + + * Syntax: + + * *src X.X.X.X* for IPv4 + * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6 + +``<dst_ip>`` + + * The destination IP address. This option is not available when + transport mode is used + + * Optional: Yes, default address 0.0.0.0 will be used + + * Syntax: + + * *dst X.X.X.X* for IPv4 + * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6 + +``<type>`` + + * Action type to specify the security action. This option specify + the SA to be performed with look aside protocol offload to HW + accelerator or protocol offload on ethernet device or inline + crypto processing on the ethernet device during transmission. + + * Optional: Yes, default type *no-offload* + + * Available options: + + * *lookaside-protocol-offload*: look aside protocol offload to HW accelerator + * *inline-protocol-offload*: inline protocol offload on ethernet device + * *inline-crypto-offload*: inline crypto processing on ethernet device + * *no-offload*: no offloading to hardware + + ``<port_id>`` + + * Port/device ID of the ethernet/crypto accelerator for which the SA is + configured. For *inline-crypto-offload* and *inline-protocol-offload*, this + port will be used for routing. The routing table will not be referred in + this case. + + * Optional: No, if *type* is not *no-offload* + + * Syntax: + + * *port_id X* X is a valid device number in decimal + + +Example SA rules: + +.. code-block:: console + + sa out 5 cipher_algo null auth_algo null mode ipv4-tunnel \ + src 172.16.1.5 dst 172.16.2.5 + + sa out 25 cipher_algo aes-128-cbc \ + cipher_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \ + auth_algo sha1-hmac \ + auth_key c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3:c3 \ + mode ipv6-tunnel \ + src 1111:1111:1111:1111:1111:1111:1111:5555 \ + dst 2222:2222:2222:2222:2222:2222:2222:5555 + + sa in 105 aead_algo aes-128-gcm \ + aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \ + mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 + + sa out 5 cipher_algo aes-128-cbc cipher_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ + auth_algo sha1-hmac auth_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ + mode ipv4-tunnel src 172.16.1.5 dst 172.16.2.5 \ + type lookaside-protocol-offload port_id 4 + +Routing rule syntax +^^^^^^^^^^^^^^^^^^^ + +The Routing rule syntax is shown as follows: + +.. code-block:: console + + rt <ip_ver> <src_ip> <dst_ip> <port> + + +where each options means: + +``<ip_ver>`` + + * IP protocol version + + * Optional: No + + * Available options: + + * *ipv4*: IP protocol version 4 + * *ipv6*: IP protocol version 6 + +``<src_ip>`` + + * The source IP address and mask + + * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used + + * Syntax: + + * *src X.X.X.X/Y* for IPv4 + * *src XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6 + +``<dst_ip>`` + + * The destination IP address and mask + + * Optional: Yes, default address 0.0.0.0 and mask of 0 will be used + + * Syntax: + + * *dst X.X.X.X/Y* for IPv4 + * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX/Y* for IPv6 + +``<port>`` + + * The traffic output port id + + * Optional: yes, default output port 0 will be used + + * Syntax: *port X* + +Example SP rules: + +.. code-block:: console + + rt ipv4 dst 172.16.1.5/32 port 0 + + rt ipv6 dst 1111:1111:1111:1111:1111:1111:1111:5555/116 port 0 diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ipv4_multicast.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ipv4_multicast.rst new file mode 100644 index 00000000..ce1474ec --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ipv4_multicast.rst @@ -0,0 +1,326 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +IPv4 Multicast Sample Application +================================= + +The IPv4 Multicast application is a simple example of packet processing +using the Data Plane Development Kit (DPDK). +The application performs L3 multicasting. + +Overview +-------- + +The application demonstrates the use of zero-copy buffers for packet forwarding. +The initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +This guide highlights the differences between the two applications. +There are two key differences from the L2 Forwarding sample application: + +* The IPv4 Multicast sample application makes use of indirect buffers. + +* The forwarding decision is taken based on information read from the input packet's IPv4 header. + +The lookup method is the Four-byte Key (FBK) hash-based method. +The lookup table is composed of pairs of destination IPv4 address (the FBK) +and a port mask associated with that IPv4 address. + +.. note:: + + The max port mask supported in the given hash table is 0xf, so only first + four ports can be supported. + If using non-consecutive ports, use the destination IPv4 address accordingly. + +For convenience and simplicity, this sample application does not take IANA-assigned multicast addresses into account, +but instead equates the last four bytes of the multicast group (that is, the last four bytes of the destination IP address) +with the mask of ports to multicast packets to. +Also, the application does not consider the Ethernet addresses; +it looks only at the IPv4 destination address for any given packet. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ipv4_multicast`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/ipv4_multicast [EAL options] -- -p PORTMASK [-q NQ] + +where, + +* -p PORTMASK: Hexadecimal bitmask of ports to configure + +* -q NQ: determines the number of queues per lcore + +.. note:: + + Unlike the basic L2/L3 Forwarding sample applications, + NUMA support is not provided in the IPv4 Multicast sample application. + +Typically, to run the IPv4 Multicast sample application, issue the following command (as root): + +.. code-block:: console + + ./build/ipv4_multicast -l 0-3 -n 3 -- -p 0x3 -q 1 + +In this command: + +* The -l option enables cores 0, 1, 2 and 3 + +* The -n option specifies 3 memory channels + +* The -p option enables ports 0 and 1 + +* The -q option assigns 1 queue to each lcore + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. +As mentioned in the overview section, +the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +The following sections describe aspects that are specific to the IPv4 Multicast sample application. + +Memory Pool Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The IPv4 Multicast sample application uses three memory pools. +Two of the pools are for indirect buffers used for packet duplication purposes. +Memory pools for indirect buffers are initialized differently from the memory pool for direct buffers: + +.. code-block:: c + + packet_pool = rte_pktmbuf_pool_create("packet_pool", NB_PKT_MBUF, 32, + 0, PKT_MBUF_DATA_SIZE, rte_socket_id()); + header_pool = rte_pktmbuf_pool_create("header_pool", NB_HDR_MBUF, 32, + 0, HDR_MBUF_DATA_SIZE, rte_socket_id()); + clone_pool = rte_pktmbuf_pool_create("clone_pool", NB_CLONE_MBUF, 32, + 0, 0, rte_socket_id()); + +The reason for this is because indirect buffers are not supposed to hold any packet data and +therefore can be initialized with lower amount of reserved memory for each buffer. + +Hash Initialization +~~~~~~~~~~~~~~~~~~~ + +The hash object is created and loaded with the pre-configured entries read from a global array: + +.. code-block:: c + + static int + + init_mcast_hash(void) + { + uint32_t i; + mcast_hash_params.socket_id = rte_socket_id(); + + mcast_hash = rte_fbk_hash_create(&mcast_hash_params); + if (mcast_hash == NULL){ + return -1; + } + + for (i = 0; i < N_MCAST_GROUPS; i ++){ + if (rte_fbk_hash_add_key(mcast_hash, mcast_group_table[i].ip, mcast_group_table[i].port_mask) < 0) { + return -1; + } + } + return 0; + } + +Forwarding +~~~~~~~~~~ + +All forwarding is done inside the mcast_forward() function. +Firstly, the Ethernet* header is removed from the packet and the IPv4 address is extracted from the IPv4 header: + +.. code-block:: c + + /* Remove the Ethernet header from the input packet */ + + iphdr = (struct ipv4_hdr *)rte_pktmbuf_adj(m, sizeof(struct ether_hdr)); + RTE_ASSERT(iphdr != NULL); + dest_addr = rte_be_to_cpu_32(iphdr->dst_addr); + +Then, the packet is checked to see if it has a multicast destination address and +if the routing table has any ports assigned to the destination address: + +.. code-block:: c + + if (!IS_IPV4_MCAST(dest_addr) || + (hash = rte_fbk_hash_lookup(mcast_hash, dest_addr)) <= 0 || + (port_mask = hash & enabled_port_mask) == 0) { + rte_pktmbuf_free(m); + return; + } + +Then, the number of ports in the destination portmask is calculated with the help of the bitcnt() function: + +.. code-block:: c + + /* Get number of bits set. */ + + static inline uint32_t bitcnt(uint32_t v) + { + uint32_t n; + + for (n = 0; v != 0; v &= v - 1, n++) + ; + return n; + } + +This is done to determine which forwarding algorithm to use. +This is explained in more detail in the next section. + +Thereafter, a destination Ethernet address is constructed: + +.. code-block:: c + + /* construct destination Ethernet address */ + + dst_eth_addr = ETHER_ADDR_FOR_IPV4_MCAST(dest_addr); + +Since Ethernet addresses are also part of the multicast process, each outgoing packet carries the same destination Ethernet address. +The destination Ethernet address is constructed from the lower 23 bits of the multicast group OR-ed +with the Ethernet address 01:00:5e:00:00:00, as per RFC 1112: + +.. code-block:: c + + #define ETHER_ADDR_FOR_IPV4_MCAST(x) \ + (rte_cpu_to_be_64(0x01005e000000ULL | ((x) & 0x7fffff)) >> 16) + +Then, packets are dispatched to the destination ports according to the portmask associated with a multicast group: + +.. code-block:: c + + for (port = 0; use_clone != port_mask; port_mask >>= 1, port++) { + /* Prepare output packet and send it out. */ + + if ((port_mask & 1) != 0) { + if (likely ((mc = mcast_out_pkt(m, use_clone)) != NULL)) + mcast_send_pkt(mc, &dst_eth_addr.as_addr, qconf, port); + else if (use_clone == 0) + rte_pktmbuf_free(m); + } + } + +The actual packet transmission is done in the mcast_send_pkt() function: + +.. code-block:: c + + static inline void mcast_send_pkt(struct rte_mbuf *pkt, struct ether_addr *dest_addr, struct lcore_queue_conf *qconf, uint16_t port) + { + struct ether_hdr *ethdr; + uint16_t len; + + /* Construct Ethernet header. */ + + ethdr = (struct ether_hdr *)rte_pktmbuf_prepend(pkt, (uint16_t) sizeof(*ethdr)); + + RTE_ASSERT(ethdr != NULL); + + ether_addr_copy(dest_addr, ðdr->d_addr); + ether_addr_copy(&ports_eth_addr[port], ðdr->s_addr); + ethdr->ether_type = rte_be_to_cpu_16(ETHER_TYPE_IPv4); + + /* Put new packet into the output queue */ + + len = qconf->tx_mbufs[port].len; + qconf->tx_mbufs[port].m_table[len] = pkt; + qconf->tx_mbufs[port].len = ++len; + + /* Transmit packets */ + + if (unlikely(MAX_PKT_BURST == len)) + send_burst(qconf, port); + } + +Buffer Cloning +~~~~~~~~~~~~~~ + +This is the most important part of the application since it demonstrates the use of zero- copy buffer cloning. +There are two approaches for creating the outgoing packet and although both are based on the data zero-copy idea, +there are some differences in the detail. + +The first approach creates a clone of the input packet, for example, +walk though all segments of the input packet and for each of segment, +create a new buffer and attach that new buffer to the segment +(refer to rte_pktmbuf_clone() in the rte_mbuf library for more details). +A new buffer is then allocated for the packet header and is prepended to the cloned buffer. + +The second approach does not make a clone, it just increments the reference counter for all input packet segment, +allocates a new buffer for the packet header and prepends it to the input packet. + +Basically, the first approach reuses only the input packet's data, but creates its own copy of packet's metadata. +The second approach reuses both input packet's data and metadata. + +The advantage of first approach is that each outgoing packet has its own copy of the metadata, +so we can safely modify the data pointer of the input packet. +That allows us to skip creation if the output packet is for the last destination port +and instead modify input packet's header in place. +For example, for N destination ports, we need to invoke mcast_out_pkt() (N-1) times. + +The advantage of the second approach is that there is less work to be done for each outgoing packet, +that is, the "clone" operation is skipped completely. +However, there is a price to pay. +The input packet's metadata must remain intact, so for N destination ports, +we need to invoke mcast_out_pkt() (N) times. + +Therefore, for a small number of outgoing ports (and segments in the input packet), +first approach is faster. +As the number of outgoing ports (and/or input segments) grows, the second approach becomes more preferable. + +Depending on the number of segments or the number of ports in the outgoing portmask, +either the first (with cloning) or the second (without cloning) approach is taken: + +.. code-block:: c + + use_clone = (port_num <= MCAST_CLONE_PORTS && m->pkt.nb_segs <= MCAST_CLONE_SEGS); + +It is the mcast_out_pkt() function that performs the packet duplication (either with or without actually cloning the buffers): + +.. code-block:: c + + static inline struct rte_mbuf *mcast_out_pkt(struct rte_mbuf *pkt, int use_clone) + { + struct rte_mbuf *hdr; + + /* Create new mbuf for the header. */ + + if (unlikely ((hdr = rte_pktmbuf_alloc(header_pool)) == NULL)) + return NULL; + + /* If requested, then make a new clone packet. */ + + if (use_clone != 0 && unlikely ((pkt = rte_pktmbuf_clone(pkt, clone_pool)) == NULL)) { + rte_pktmbuf_free(hdr); + return NULL; + } + + /* prepend new header */ + + hdr->pkt.next = pkt; + + /* update header's fields */ + + hdr->pkt.pkt_len = (uint16_t)(hdr->pkt.data_len + pkt->pkt.pkt_len); + hdr->pkt.nb_segs = pkt->pkt.nb_segs + 1; + + /* copy metadata from source packet */ + + hdr->pkt.in_port = pkt->pkt.in_port; + hdr->pkt.vlan_macip = pkt->pkt.vlan_macip; + hdr->pkt.hash = pkt->pkt.hash; + hdr->ol_flags = pkt->ol_flags; + rte_mbuf_sanity_check(hdr, RTE_MBUF_PKT, 1); + + return hdr; + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/keep_alive.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/keep_alive.rst new file mode 100644 index 00000000..5ceaa442 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/keep_alive.rst @@ -0,0 +1,144 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015-2016 Intel Corporation. + +Keep Alive Sample Application +============================= + +The Keep Alive application is a simple example of a +heartbeat/watchdog for packet processing cores. It demonstrates how +to detect 'failed' DPDK cores and notify a fault management entity +of this failure. Its purpose is to ensure the failure of the core +does not result in a fault that is not detectable by a management +entity. + + +Overview +-------- + +The application demonstrates how to protect against 'silent outages' +on packet processing cores. A Keep Alive Monitor Agent Core (master) +monitors the state of packet processing cores (worker cores) by +dispatching pings at a regular time interval (default is 5ms) and +monitoring the state of the cores. Cores states are: Alive, MIA, Dead +or Buried. MIA indicates a missed ping, and Dead indicates two missed +pings within the specified time interval. When a core is Dead, a +callback function is invoked to restart the packet processing core; +A real life application might use this callback function to notify a +higher level fault management entity of the core failure in order to +take the appropriate corrective action. + +Note: Only the worker cores are monitored. A local (on the host) mechanism +or agent to supervise the Keep Alive Monitor Agent Core DPDK core is required +to detect its failure. + +Note: This application is based on the :doc:`l2_forward_real_virtual`. As +such, the initialization and run-time paths are very similar to those +of the L2 forwarding application. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l2fwd_keep_alive`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/l2fwd-keepalive [EAL options] \ + -- -p PORTMASK [-q NQ] [-K PERIOD] [-T PERIOD] + +where, + +* ``p PORTMASK``: A hexadecimal bitmask of the ports to configure + +* ``q NQ``: A number of queues (=ports) per lcore (default is 1) + +* ``K PERIOD``: Heartbeat check period in ms(5ms default; 86400 max) + +* ``T PERIOD``: statistics will be refreshed each PERIOD seconds (0 to + disable, 10 default, 86400 maximum). + +To run the application in linuxapp environment with 4 lcores, 16 ports +8 RX queues per lcore and a ping interval of 10ms, issue the command: + +.. code-block:: console + + ./build/l2fwd-keepalive -l 0-3 -n 4 -- -q 8 -p ffff -K 10 + +Refer to the *DPDK Getting Started Guide* for general information on +running applications and the Environment Abstraction Layer (EAL) +options. + + +Explanation +----------- + +The following sections provide some explanation of the The +Keep-Alive/'Liveliness' conceptual scheme. As mentioned in the +overview section, the initialization and run-time paths are very +similar to those of the :doc:`l2_forward_real_virtual`. + +The Keep-Alive/'Liveliness' conceptual scheme: + +* A Keep- Alive Agent Runs every N Milliseconds. + +* DPDK Cores respond to the keep-alive agent. + +* If keep-alive agent detects time-outs, it notifies the + fault management entity through a callback function. + +The following sections provide some explanation of the code aspects +that are specific to the Keep Alive sample application. + +The keepalive functionality is initialized with a struct +rte_keepalive and the callback function to invoke in the +case of a timeout. + +.. code-block:: c + + rte_global_keepalive_info = rte_keepalive_create(&dead_core, NULL); + if (rte_global_keepalive_info == NULL) + rte_exit(EXIT_FAILURE, "keepalive_create() failed"); + +The function that issues the pings keepalive_dispatch_pings() +is configured to run every check_period milliseconds. + +.. code-block:: c + + if (rte_timer_reset(&hb_timer, + (check_period * rte_get_timer_hz()) / 1000, + PERIODICAL, + rte_lcore_id(), + &rte_keepalive_dispatch_pings, + rte_global_keepalive_info + ) != 0 ) + rte_exit(EXIT_FAILURE, "Keepalive setup failure.\n"); + +The rest of the initialization and run-time path follows +the same paths as the L2 forwarding application. The only +addition to the main processing loop is the mark alive +functionality and the example random failures. + +.. code-block:: c + + rte_keepalive_mark_alive(&rte_global_keepalive_info); + cur_tsc = rte_rdtsc(); + + /* Die randomly within 7 secs for demo purposes.. */ + if (cur_tsc - tsc_initial > tsc_lifetime) + break; + +The rte_keepalive_mark_alive function simply sets the core state to alive. + +.. code-block:: c + + static inline void + rte_keepalive_mark_alive(struct rte_keepalive *keepcfg) + { + keepcfg->live_data[rte_lcore_id()].core_state = RTE_KA_STATE_ALIVE; + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/kernel_nic_interface.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/kernel_nic_interface.rst new file mode 100644 index 00000000..1b3ee9a5 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -0,0 +1,250 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Kernel NIC Interface Sample Application +======================================= + +The Kernel NIC Interface (KNI) is a DPDK control plane solution that +allows userspace applications to exchange packets with the kernel networking stack. +To accomplish this, DPDK userspace applications use an IOCTL call +to request the creation of a KNI virtual device in the Linux* kernel. +The IOCTL call provides interface information and the DPDK's physical address space, +which is re-mapped into the kernel address space by the KNI kernel loadable module +that saves the information to a virtual device context. +The DPDK creates FIFO queues for packet ingress and egress +to the kernel module for each device allocated. + +The KNI kernel loadable module is a standard net driver, +which upon receiving the IOCTL call access the DPDK's FIFO queue to +receive/transmit packets from/to the DPDK userspace application. +The FIFO queues contain pointers to data packets in the DPDK. This: + +* Provides a faster mechanism to interface with the kernel net stack and eliminates system calls + +* Facilitates the DPDK using standard Linux* userspace net tools (tcpdump, ftp, and so on) + +* Eliminate the copy_to_user and copy_from_user operations on packets. + +The Kernel NIC Interface sample application is a simple example that demonstrates the use +of the DPDK to create a path for packets to go through the Linux* kernel. +This is done by creating one or more kernel net devices for each of the DPDK ports. +The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and +also the exchange of packets between the DPDK application and the Linux* kernel. + +Overview +-------- + +The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used, +and allocates one or more KNI device for each physical NIC port with kernel module's support. +For a physical NIC port, one thread reads from the port and writes to KNI devices, +and another thread reads from KNI devices and writes the data unmodified to the physical NIC port. +It is recommended to configure one KNI device for each physical NIC port. +If configured with more than one KNI devices for a physical NIC port, +it is just for performance testing, or it can work together with VMDq support in future. + +The packet flow through the Kernel NIC Interface application is as shown in the following figure. + +.. _figure_kernel_nic: + +.. figure:: img/kernel_nic.* + + Kernel NIC Application Packet Flow + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``kni`` sub-directory. + +.. note:: + + This application is intended as a linuxapp only. + +Loading the Kernel Module +------------------------- + +Loading the KNI kernel module without any parameter is the typical way a DPDK application +gets packets into and out of the kernel net stack. +This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side: + +.. code-block:: console + + #insmod rte_kni.ko + +Pinning the kernel thread to a specific core can be done using a taskset command such as following: + +.. code-block:: console + + #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'` + +This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0), +which means it needs to check if that lcore is available on the board. +This command must be sent after the application has been launched, as insmod does not start the kni thread. + +For optimum performance, +the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application. + +To provide flexibility of performance, the kernel module of the KNI, +located in the kmod sub-directory of the DPDK target directory, +can be loaded with parameter of kthread_mode as follows: + +* #insmod rte_kni.ko kthread_mode=single + + This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side. + By default, it is in this single kernel thread mode. + It can set core affinity for this kernel thread by using Linux command taskset. + +* #insmod rte_kni.ko kthread_mode =multiple + + This mode will create a kernel thread for each KNI device for packet receiving in kernel side. + The core affinity of each kernel thread is set when creating the KNI device. + The lcore ID for each kernel thread is provided in the command line of launching the application. + Multiple kernel thread mode can provide scalable higher performance. + +To measure the throughput in a loopback mode, the kernel module of the KNI, +located in the kmod sub-directory of the DPDK target directory, +can be loaded with parameters as follows: + +* #insmod rte_kni.ko lo_mode=lo_mode_fifo + + This loopback mode will involve ring enqueue/dequeue operations in kernel space. + +* #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb + + This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]" + +Where: + +* -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. + +* -p PORTMASK: Hexadecimal bitmask of ports to configure. + +* --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]": + Determines which lcores of RX, TX, kernel thread are mapped to which ports. + +Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. + +The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx, +but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on. +The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less. + +The lcore_kthread in --config can be configured none, one or more lcore IDs. +In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port, +while no specific lcore affinity will be set for its kernel thread. +If configured one or more lcore IDs, one or more KNI devices will be allocated for each port, +while specific lcore affinity will be set for its kernel thread. +In single kernel thread mode, if configured none, a KNI device will be allocated for each port. +If configured one or more lcore IDs, +one or more KNI devices will be allocated for each port while +no lcore affinity will be set as there is only one kernel thread for all KNI devices. + +For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX, +and one lcore of kernel thread for each port: + +.. code-block:: console + + ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" + +KNI Operations +-------------- + +Once the KNI application is started, one can use different Linux* commands to manage the net interfaces. +If more than one KNI devices configured for a physical port, +only the first KNI device will be paired to the physical device. +Operations on other KNI devices will not affect the physical port handled in user space application. + +Assigning an IP address: + +.. code-block:: console + + #ifconfig vEth0_0 192.168.0.1 + +Displaying the NIC registers: + +.. code-block:: console + + #ethtool -d vEth0_0 + +Dumping the network traffic: + +.. code-block:: console + + #tcpdump -i vEth0_0 + +Change the MAC address: + +.. code-block:: console + + #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 + +When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*. + +Explanation +----------- + +The following sections provide some explanation of code. + +Initialization +~~~~~~~~~~~~~~ + +Setup of mbuf pool, driver and queues is similar to the setup done in the :doc:`l2_forward_real_virtual`.. +In addition, one or more kernel NIC interfaces are allocated for each +of the configured ports according to the command line parameters. + +The code for allocating the kernel NIC interfaces for a specific port is +in the function ``kni_alloc``. + +The other step in the initialization process that is unique to this sample application +is the association of each port with lcores for RX, TX and kernel threads. + +* One lcore to read from the port and write to the associated one or more KNI devices + +* Another lcore to read from one or more KNI devices and write to the port + +* Other lcores for pinning the kernel threads on one by one + +This is done by using the ``kni_port_params_array[]`` array, which is indexed by the port ID. +The code is in the function ``parse_config``. + +Packet Forwarding +~~~~~~~~~~~~~~~~~ + +After the initialization steps are completed, the main_loop() function is run on each lcore. +This function first checks the lcore_id against the user provided lcore_rx and lcore_tx +to see if this lcore is reading from or writing to kernel NIC interfaces. + +For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``), +the packet reception is the same as in L2 Forwarding sample application +(see :ref:`l2_fwd_app_rx_tx_packets`). +The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst(). +The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs. + +For the other case that reads from kernel NIC interfaces +and writes to a physical NIC port (``kni_egress``), +packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``. +The packet transmission is the same as in the L2 Forwarding sample application +(see :ref:`l2_fwd_app_rx_tx_packets`). + +Callbacks for Kernel Requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To execute specific PMD operations in user space requested by some Linux* commands, +callbacks must be implemented and filled in the struct rte_kni_ops structure. +Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and +configuring the network interface(up/down) re supported. +Default implementation for following is available in rte_kni library. +Application may choose to not implement following callbacks: + +- ``config_mac_address`` +- ``config_promiscusity`` diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_cat.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_cat.rst new file mode 100644 index 00000000..4ee877b4 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_cat.rst @@ -0,0 +1,207 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016 Intel Corporation. + +L2 Forwarding Sample Application with Cache Allocation Technology (CAT) +======================================================================= + +Basic Forwarding sample application is a simple *skeleton* example of +a forwarding application. It has been extended to make use of CAT via extended +command line options and linking against the libpqos library. + +It is intended as a demonstration of the basic components of a DPDK forwarding +application and use of the libpqos library to program CAT. +For more detailed implementations see the L2 and L3 forwarding +sample applications. + +CAT and Code Data Prioritization (CDP) features allow management of the CPU's +last level cache. CAT introduces classes of service (COS) that are essentially +bitmasks. In current CAT implementations, a bit in a COS bitmask corresponds to +one cache way in last level cache. +A CPU core is always assigned to one of the CAT classes. +By programming CPU core assignment and COS bitmasks, applications can be given +exclusive, shared, or mixed access to the CPU's last level cache. +CDP extends CAT so that there are two bitmasks per COS, +one for data and one for code. +The number of classes and number of valid bits in a COS bitmask is CPU model +specific and COS bitmasks need to be contiguous. Sample code calls this bitmask +``cbm`` or capacity bitmask. +By default, after reset, all CPU cores are assigned to COS 0 and all classes +are programmed to allow fill into all cache ways. +CDP is off by default. + +For more information about CAT please see: + +* https://github.com/01org/intel-cmt-cat + +White paper demonstrating example use case: + +* `Increasing Platform Determinism with Platform Quality of Service for the Data Plane Development Kit <http://www.intel.com/content/www/us/en/communications/increasing-platform-determinism-pqos-dpdk-white-paper.html>`_ + +Compiling the Application +------------------------- +.. note:: + + Requires ``libpqos`` from Intel's + `intel-cmt-cat software package <https://github.com/01org/intel-cmt-cat>`_ + hosted on GitHub repository. For installation notes, please see ``README`` file. + + GIT: + + * https://github.com/01org/intel-cmt-cat + + +#. To compile the application export the path to PQoS lib + and the DPDK source tree and go to the example directory: + + .. code-block:: console + + export PQOS_INSTALL_PATH=/path/to/libpqos + + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l2fwd-cat`` sub-directory. + + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment and enable CAT on cpus 0-2: + +.. code-block:: console + + ./build/l2fwd-cat -l 1 -n 4 -- --l3ca="0x3@(0-2)" + +or to enable CAT and CDP on cpus 1,3: + +.. code-block:: console + + ./build/l2fwd-cat -l 1 -n 4 -- --l3ca="(0x00C00,0x00300)@(1,3)" + +If CDP is not supported it will fail with following error message: + +.. code-block:: console + + PQOS: CDP requested but not supported. + PQOS: Requested CAT configuration is not valid! + PQOS: Shutting down PQoS library... + EAL: Error - exiting with code: 1 + Cause: PQOS: L3CA init failed! + +The option to enable CAT is: + +* ``--l3ca='<common_cbm@cpus>[,<(code_cbm,data_cbm)@cpus>...]'``: + + where ``cbm`` stands for capacity bitmask and must be expressed in + hexadecimal form. + + ``common_cbm`` is a single mask, for a CDP enabled system, a group of two + masks (``code_cbm`` and ``data_cbm``) is used. + + ``(`` and ``)`` are necessary if it's a group. + + ``cpus`` could be a single digit/range or a group and must be expressed in + decimal form. + + ``(`` and ``)`` are necessary if it's a group. + + e.g. ``--l3ca='0x00F00@(1,3),0x0FF00@(4-6),0xF0000@7'`` + + * cpus 1 and 3 share its 4 ways with cpus 4, 5 and 6; + + * cpus 4, 5 and 6 share half (4 out of 8 ways) of its L3 with cpus 1 and 3; + + * cpus 4, 5 and 6 have exclusive access to 4 out of 8 ways; + + * cpu 7 has exclusive access to all of its 4 ways; + + e.g. ``--l3ca='(0x00C00,0x00300)@(1,3)'`` for CDP enabled system + + * cpus 1 and 3 have access to 2 ways for code and 2 ways for data, code and + data ways are not overlapping. + + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + + +To reset or list CAT configuration and control CDP please use ``pqos`` tool +from Intel's +`intel-cmt-cat software package <https://github.com/01org/intel-cmt-cat>`_. + +To enabled or disable CDP: + +.. code-block:: console + + sudo ./pqos -S cdp-on + + sudo ./pqos -S cdp-off + +to reset CAT configuration: + +.. code-block:: console + + sudo ./pqos -R + +to list CAT config: + +.. code-block:: console + + sudo ./pqos -s + +For more info about ``pqos`` tool please see its man page or +`intel-cmt-cat wiki <https://github.com/01org/intel-cmt-cat/wiki>`_. + + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). The +``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + +The next task is to initialize the PQoS library and configure CAT. The +``argc`` and ``argv`` arguments are provided to the ``cat_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = cat_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "PQOS: L3CA init failed!\n"); + +``cat_init()`` is a wrapper function which parses the command, validates +the requested parameters and configures CAT accordingly. + +Parsing of command line arguments is done in ``parse_args(...)``. +libpqos is then initialized with the ``pqos_init(...)`` call. Next, libpqos is +queried for system CPU information and L3CA capabilities via +``pqos_cap_get(...)`` and ``pqos_cap_get_type(..., PQOS_CAP_TYPE_L3CA, ...)`` +calls. When all capability and topology information is collected, the requested +CAT configuration is validated. A check is then performed (on per socket basis) +for a sufficient number of un-associated COS. COS are selected and +configured via the ``pqos_l3ca_set(...)`` call. Finally, COS are associated to +relevant CPUs via ``pqos_l3ca_assoc_set(...)`` calls. + +``atexit(...)`` is used to register ``cat_exit(...)`` to be called on +a clean exit. ``cat_exit(...)`` performs a simple CAT clean-up, by associating +COS 0 to all involved CPUs via ``pqos_l3ca_assoc_set(...)`` calls. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_crypto.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_crypto.rst new file mode 100644 index 00000000..855afd8f --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_crypto.rst @@ -0,0 +1,484 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016-2017 Intel Corporation. + +.. _l2_fwd_crypto_app: + +L2 Forwarding with Crypto Sample Application +============================================ + +The L2 Forwarding with Crypto (l2fwd-crypto) sample application is a simple example of packet processing using +the Data Plane Development Kit (DPDK), in conjunction with the Cryptodev library. + +Overview +-------- + +The L2 Forwarding with Crypto sample application performs a crypto operation (cipher/hash) +specified by the user from command line (or using the default values), +with a crypto device capable of doing that operation, +for each packet that is received on a RX_PORT and performs L2 forwarding. +The destination port is the adjacent port from the enabled portmask, that is, +if the first four ports are enabled (portmask 0xf), +ports 0 and 1 forward into each other, and ports 2 and 3 forward into each other. +Also, if MAC addresses updating is enabled, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX_PORT MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l2fwd-crypt`` sub-directory. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + ./build/l2fwd-crypto [EAL options] -- [-p PORTMASK] [-q NQ] [-s] [-T PERIOD] / + [--cdev_type HW/SW/ANY] [--chain HASH_CIPHER/CIPHER_HASH/CIPHER_ONLY/HASH_ONLY/AEAD] / + [--cipher_algo ALGO] [--cipher_op ENCRYPT/DECRYPT] [--cipher_key KEY] / + [--cipher_key_random_size SIZE] [--cipher_iv IV] [--cipher_iv_random_size SIZE] / + [--auth_algo ALGO] [--auth_op GENERATE/VERIFY] [--auth_key KEY] / + [--auth_key_random_size SIZE] [--auth_iv IV] [--auth_iv_random_size SIZE] / + [--aead_algo ALGO] [--aead_op ENCRYPT/DECRYPT] [--aead_key KEY] / + [--aead_key_random_size SIZE] [--aead_iv] [--aead_iv_random_size SIZE] / + [--aad AAD] [--aad_random_size SIZE] / + [--digest size SIZE] [--sessionless] [--cryptodev_mask MASK] / + [--mac-updating] [--no-mac-updating] + +where, + +* p PORTMASK: A hexadecimal bitmask of the ports to configure (default is all the ports) + +* q NQ: A number of queues (=ports) per lcore (default is 1) + +* s: manage all ports from single core + +* T PERIOD: statistics will be refreshed each PERIOD seconds + + (0 to disable, 10 default, 86400 maximum) + +* cdev_type: select preferred crypto device type: HW, SW or anything (ANY) + + (default is ANY) + +* chain: select the operation chaining to perform: Cipher->Hash (CIPHER_HASH), + + Hash->Cipher (HASH_CIPHER), Cipher (CIPHER_ONLY), Hash (HASH_ONLY) + + or AEAD (AEAD) + + (default is Cipher->Hash) + +* cipher_algo: select the ciphering algorithm (default is aes-cbc) + +* cipher_op: select the ciphering operation to perform: ENCRYPT or DECRYPT + + (default is ENCRYPT) + +* cipher_key: set the ciphering key to be used. Bytes has to be separated with ":" + +* cipher_key_random_size: set the size of the ciphering key, + + which will be generated randomly. + + Note that if --cipher_key is used, this will be ignored. + +* cipher_iv: set the cipher IV to be used. Bytes has to be separated with ":" + +* cipher_iv_random_size: set the size of the cipher IV, which will be generated randomly. + + Note that if --cipher_iv is used, this will be ignored. + +* auth_algo: select the authentication algorithm (default is sha1-hmac) + +* auth_op: select the authentication operation to perform: GENERATE or VERIFY + + (default is GENERATE) + +* auth_key: set the authentication key to be used. Bytes has to be separated with ":" + +* auth_key_random_size: set the size of the authentication key, + + which will be generated randomly. + + Note that if --auth_key is used, this will be ignored. + +* auth_iv: set the auth IV to be used. Bytes has to be separated with ":" + +* auth_iv_random_size: set the size of the auth IV, which will be generated randomly. + + Note that if --auth_iv is used, this will be ignored. + +* aead_algo: select the AEAD algorithm (default is aes-gcm) + +* aead_op: select the AEAD operation to perform: ENCRYPT or DECRYPT + + (default is ENCRYPT) + +* aead_key: set the AEAD key to be used. Bytes has to be separated with ":" + +* aead_key_random_size: set the size of the AEAD key, + + which will be generated randomly. + + Note that if --aead_key is used, this will be ignored. + +* aead_iv: set the AEAD IV to be used. Bytes has to be separated with ":" + +* aead_iv_random_size: set the size of the AEAD IV, which will be generated randomly. + + Note that if --aead_iv is used, this will be ignored. + +* aad: set the AAD to be used. Bytes has to be separated with ":" + +* aad_random_size: set the size of the AAD, which will be generated randomly. + + Note that if --aad is used, this will be ignored. + +* digest_size: set the size of the digest to be generated/verified. + +* sessionless: no crypto session will be created. + +* cryptodev_mask: A hexadecimal bitmask of the cryptodevs to be used by the + application. + + (default is all cryptodevs). + +* [no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). + + +The application requires that crypto devices capable of performing +the specified crypto operation are available on application initialization. +This means that HW crypto device/s must be bound to a DPDK driver or +a SW crypto device/s (virtual crypto PMD) must be created (using --vdev). + +To run the application in linuxapp environment with 2 lcores, 2 ports and 2 crypto devices, issue the command: + +.. code-block:: console + + $ ./build/l2fwd-crypto -l 0-1 -n 4 --vdev "crypto_aesni_mb0" \ + --vdev "crypto_aesni_mb1" -- -p 0x3 --chain CIPHER_HASH \ + --cipher_op ENCRYPT --cipher_algo aes-cbc \ + --cipher_key 00:01:02:03:04:05:06:07:08:09:0a:0b:0c:0d:0e:0f \ + --auth_op GENERATE --auth_algo aes-xcbc-mac \ + --auth_key 10:11:12:13:14:15:16:17:18:19:1a:1b:1c:1d:1e:1f + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +.. Note:: + + * The ``l2fwd-crypto`` sample application requires IPv4 packets for crypto operation. + + * If multiple Ethernet ports is passed, then equal number of crypto devices are to be passed. + + * All crypto devices shall use the same session. + +Explanation +----------- + +The L2 forward with Crypto application demonstrates the performance of a crypto operation +on a packet received on a RX PORT before forwarding it to a TX PORT. + +The following figure illustrates a sample flow of a packet in the application, +from reception until transmission. + +.. _figure_l2_fwd_encrypt_flow: + +.. figure:: img/l2_fwd_encrypt_flow.* + + Encryption flow Through the L2 Forwarding with Crypto Application + + +The following sections provide some explanation of the application. + +Crypto operation specification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All the packets received in all the ports get transformed by the crypto device/s +(ciphering and/or authentication). +The crypto operation to be performed on the packet is parsed from the command line +(go to "Running the Application section for all the options). + +If no parameter is passed, the default crypto operation is: + +* Encryption with AES-CBC with 128 bit key. + +* Authentication with SHA1-HMAC (generation). + +* Keys, IV and AAD are generated randomly. + +There are two methods to pass keys, IV and ADD from the command line: + +* Passing the full key, separated bytes by ":":: + + --cipher_key 00:11:22:33:44 + +* Passing the size, so key is generated randomly:: + + --cipher_key_random_size 16 + +**Note**: + If full key is passed (first method) and the size is passed as well (second method), + the latter will be ignored. + +Size of these keys are checked (regardless the method), before starting the app, +to make sure that it is supported by the crypto devices. + +Crypto device initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the encryption operation is defined, crypto devices are initialized. +The crypto devices must be either bound to a DPDK driver (if they are physical devices) +or created using the EAL option --vdev (if they are virtual devices), +when running the application. + +The initialize_cryptodevs() function performs the device initialization. +It iterates through the list of the available crypto devices and +check which ones are capable of performing the operation. +Each device has a set of capabilities associated with it, +which are stored in the device info structure, so the function checks if the operation +is within the structure of each device. + +The following code checks if the device supports the specified cipher algorithm +(similar for the authentication algorithm): + +.. code-block:: c + + /* Check if device supports cipher algo */ + i = 0; + opt_cipher_algo = options->cipher_xform.cipher.algo; + cap = &dev_info.capabilities[i]; + while (cap->op != RTE_CRYPTO_OP_TYPE_UNDEFINED) { + cap_cipher_algo = cap->sym.cipher.algo; + if (cap->sym.xform_type == + RTE_CRYPTO_SYM_XFORM_CIPHER) { + if (cap_cipher_algo == opt_cipher_algo) { + if (check_type(options, &dev_info) == 0) + break; + } + } + cap = &dev_info.capabilities[++i]; + } + +If a capable crypto device is found, key sizes are checked to see if they are supported +(cipher key and IV for the ciphering): + +.. code-block:: c + + /* + * Check if length of provided cipher key is supported + * by the algorithm chosen. + */ + if (options->ckey_param) { + if (check_supported_size( + options->cipher_xform.cipher.key.length, + cap->sym.cipher.key_size.min, + cap->sym.cipher.key_size.max, + cap->sym.cipher.key_size.increment) + != 0) { + printf("Unsupported cipher key length\n"); + return -1; + } + /* + * Check if length of the cipher key to be randomly generated + * is supported by the algorithm chosen. + */ + } else if (options->ckey_random_size != -1) { + if (check_supported_size(options->ckey_random_size, + cap->sym.cipher.key_size.min, + cap->sym.cipher.key_size.max, + cap->sym.cipher.key_size.increment) + != 0) { + printf("Unsupported cipher key length\n"); + return -1; + } + options->cipher_xform.cipher.key.length = + options->ckey_random_size; + /* No size provided, use minimum size. */ + } else + options->cipher_xform.cipher.key.length = + cap->sym.cipher.key_size.min; + +After all the checks, the device is configured and it is added to the +crypto device list. + +**Note**: + The number of crypto devices that supports the specified crypto operation + must be at least the number of ports to be used. + +Session creation +~~~~~~~~~~~~~~~~ + +The crypto operation has a crypto session associated to it, which contains +information such as the transform chain to perform (e.g. ciphering then hashing), +pointers to the keys, lengths... etc. + +This session is created and is later attached to the crypto operation: + +.. code-block:: c + + static struct rte_cryptodev_sym_session * + initialize_crypto_session(struct l2fwd_crypto_options *options, + uint8_t cdev_id) + { + struct rte_crypto_sym_xform *first_xform; + struct rte_cryptodev_sym_session *session; + uint8_t socket_id = rte_cryptodev_socket_id(cdev_id); + struct rte_mempool *sess_mp = session_pool_socket[socket_id]; + + + if (options->xform_chain == L2FWD_CRYPTO_AEAD) { + first_xform = &options->aead_xform; + } else if (options->xform_chain == L2FWD_CRYPTO_CIPHER_HASH) { + first_xform = &options->cipher_xform; + first_xform->next = &options->auth_xform; + } else if (options->xform_chain == L2FWD_CRYPTO_HASH_CIPHER) { + first_xform = &options->auth_xform; + first_xform->next = &options->cipher_xform; + } else if (options->xform_chain == L2FWD_CRYPTO_CIPHER_ONLY) { + first_xform = &options->cipher_xform; + } else { + first_xform = &options->auth_xform; + } + + session = rte_cryptodev_sym_session_create(sess_mp); + + if (session == NULL) + return NULL; + + if (rte_cryptodev_sym_session_init(cdev_id, session, + first_xform, sess_mp) < 0) + return NULL; + + return session; + } + + ... + + port_cparams[i].session = initialize_crypto_session(options, + port_cparams[i].dev_id); + +Crypto operation creation +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Given N packets received from a RX PORT, N crypto operations are allocated +and filled: + +.. code-block:: c + + if (nb_rx) { + /* + * If we can't allocate a crypto_ops, then drop + * the rest of the burst and dequeue and + * process the packets to free offload structs + */ + if (rte_crypto_op_bulk_alloc( + l2fwd_crypto_op_pool, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + ops_burst, nb_rx) != + nb_rx) { + for (j = 0; j < nb_rx; j++) + rte_pktmbuf_free(pkts_burst[i]); + + nb_rx = 0; + } + +After filling the crypto operation (including session attachment), +the mbuf which will be transformed is attached to it:: + + op->sym->m_src = m; + +Since no destination mbuf is set, the source mbuf will be overwritten +after the operation is done (in-place). + +Crypto operation enqueuing/dequeuing +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the operation has been created, it has to be enqueued in one of the crypto devices. +Before doing so, for performance reasons, the operation stays in a buffer. +When the buffer has enough operations (MAX_PKT_BURST), they are enqueued in the device, +which will perform the operation at that moment: + +.. code-block:: c + + static int + l2fwd_crypto_enqueue(struct rte_crypto_op *op, + struct l2fwd_crypto_params *cparams) + { + unsigned lcore_id, len; + struct lcore_queue_conf *qconf; + + lcore_id = rte_lcore_id(); + + qconf = &lcore_queue_conf[lcore_id]; + len = qconf->op_buf[cparams->dev_id].len; + qconf->op_buf[cparams->dev_id].buffer[len] = op; + len++; + + /* enough ops to be sent */ + if (len == MAX_PKT_BURST) { + l2fwd_crypto_send_burst(qconf, MAX_PKT_BURST, cparams); + len = 0; + } + + qconf->op_buf[cparams->dev_id].len = len; + return 0; + } + + ... + + static int + l2fwd_crypto_send_burst(struct lcore_queue_conf *qconf, unsigned n, + struct l2fwd_crypto_params *cparams) + { + struct rte_crypto_op **op_buffer; + unsigned ret; + + op_buffer = (struct rte_crypto_op **) + qconf->op_buf[cparams->dev_id].buffer; + + ret = rte_cryptodev_enqueue_burst(cparams->dev_id, + cparams->qp_id, op_buffer, (uint16_t) n); + + crypto_statistics[cparams->dev_id].enqueued += ret; + if (unlikely(ret < n)) { + crypto_statistics[cparams->dev_id].errors += (n - ret); + do { + rte_pktmbuf_free(op_buffer[ret]->sym->m_src); + rte_crypto_op_free(op_buffer[ret]); + } while (++ret < n); + } + + return 0; + } + +After this, the operations are dequeued from the device, and the transformed mbuf +is extracted from the operation. Then, the operation is freed and the mbuf is +forwarded as it is done in the L2 forwarding application. + +.. code-block:: c + + /* Dequeue packets from Crypto device */ + do { + nb_rx = rte_cryptodev_dequeue_burst( + cparams->dev_id, cparams->qp_id, + ops_burst, MAX_PKT_BURST); + + crypto_statistics[cparams->dev_id].dequeued += + nb_rx; + + /* Forward crypto'd packets */ + for (j = 0; j < nb_rx; j++) { + m = ops_burst[j]->sym->m_src; + + rte_crypto_op_free(ops_burst[j]); + l2fwd_simple_forward(m, portid); + } + } while (nb_rx == MAX_PKT_BURST); diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_job_stats.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_job_stats.rst new file mode 100644 index 00000000..ba73d855 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_job_stats.rst @@ -0,0 +1,550 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2015 Intel Corporation. + +L2 Forwarding Sample Application (in Real and Virtualized Environments) with core load statistics. +================================================================================================== + +The L2 Forwarding sample application is a simple example of packet processing using +the Data Plane Development Kit (DPDK) which +also takes advantage of Single Root I/O Virtualization (SR-IOV) features in a virtualized environment. + +.. note:: + + This application is a variation of L2 Forwarding sample application. It demonstrate possible + scheme of job stats library usage therefore some parts of this document is identical with original + L2 forwarding application. + +Overview +-------- + +The L2 Forwarding sample application, which can operate in real and virtualized environments, +performs L2 forwarding for each packet that is received. +The destination port is the adjacent port from the enabled portmask, that is, +if the first four ports are enabled (portmask 0xf), +ports 1 and 2 forward into each other, and ports 3 and 4 forward into each other. +Also, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX port MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup_jobstats`. + +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats`. + +The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. + +.. _figure_l2_fwd_benchmark_setup_jobstats: + +.. figure:: img/l2_fwd_benchmark_setup.* + + Performance Benchmark Setup (Basic Environment) + +.. _figure_l2_fwd_virtenv_benchmark_setup_jobstats: + +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* + + Performance Benchmark Setup (Virtualized Environment) + + +Virtual Function Setup Instructions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This application can use the virtual function available in the system and +therefore can be used in a virtual machine without passing through +the whole Network Device into a guest machine in a virtualized scenario. +The virtual functions can be enabled in the host machine or the hypervisor with the respective physical function driver. + +For example, in a Linux* host machine, it is possible to enable a virtual function using the following command: + +.. code-block:: console + + modprobe ixgbe max_vfs=2,2 + +This command enables two Virtual Functions on each of Physical Function of the NIC, +with two physical ports in the PCI configuration space. +It is important to note that enabled Virtual Function 0 and 2 would belong to Physical Function 0 +and Virtual Function 1 and 3 would belong to Physical Function 1, +in this case enabling a total of four Virtual Functions. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l2fwd-jobstats`` sub-directory. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + ./build/l2fwd-jobstats [EAL options] -- -p PORTMASK [-q NQ] [-l] + +where, + +* p PORTMASK: A hexadecimal bitmask of the ports to configure + +* q NQ: A number of queues (=ports) per lcore (default is 1) + +* l: Use locale thousands separator when formatting big numbers. + +To run the application in linuxapp environment with 4 lcores, 16 ports, 8 RX queues per lcore and +thousands separator printing, issue the command: + +.. code-block:: console + + $ ./build/l2fwd-jobstats -l 0-3 -n 4 -- -q 8 -p ffff -l + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +Command Line Arguments +~~~~~~~~~~~~~~~~~~~~~~ + +The L2 Forwarding sample application takes specific parameters, +in addition to Environment Abstraction Layer (EAL) arguments +(see `Running the Application`_). +The preferred way to parse parameters is to use the getopt() function, +since it is part of a well-defined and portable library. + +The parsing of arguments is done in the l2fwd_parse_args() function. +The method of argument parsing is not described here. +Refer to the *glibc getopt(3)* man page for details. + +EAL arguments are parsed first, then application-specific arguments. +This is done at the beginning of the main() function: + +.. code-block:: c + + /* init EAL */ + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + argc -= ret; + argv += ret; + + /* parse application arguments (after the EAL ones) */ + + ret = l2fwd_parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n"); + +Mbuf Pool Initialization +~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the arguments are parsed, the mbuf pool is created. +The mbuf pool contains a set of mbuf objects that will be used by the driver +and the application to store network packet data: + +.. code-block:: c + + /* create the mbuf pool */ + l2fwd_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + + if (l2fwd_pktmbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n"); + +The rte_mempool is a generic structure used to handle pools of objects. +In this case, it is necessary to create a pool that will be used by the driver. +The number of allocated pkt mbufs is NB_MBUF, with a data room size of +RTE_MBUF_DEFAULT_BUF_SIZE each. +A per-lcore cache of MEMPOOL_CACHE_SIZE mbufs is kept. +The memory is allocated in rte_socket_id() socket, +but it is possible to extend this code to allocate one mbuf pool per socket. + +The rte_pktmbuf_pool_create() function uses the default mbuf pool and mbuf +initializers, respectively rte_pktmbuf_pool_init() and rte_pktmbuf_init(). +An advanced application may want to use the mempool API to create the +mbuf pool with more control. + +Driver Initialization +~~~~~~~~~~~~~~~~~~~~~ + +The main part of the code in the main() function relates to the initialization of the driver. +To fully understand this code, it is recommended to study the chapters that related to the Poll Mode Driver +in the *DPDK Programmer's Guide* and the *DPDK API Reference*. + +.. code-block:: c + + /* reset l2fwd_dst_ports */ + + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) + l2fwd_dst_ports[portid] = 0; + + last_port = 0; + + /* + * Each logical core is assigned a dedicated TX queue on each port. + */ + RTE_ETH_FOREACH_DEV(portid) { + /* skip ports that are not enabled */ + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) + continue; + + if (nb_ports_in_mask % 2) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + } + else + last_port = portid; + + nb_ports_in_mask++; + + rte_eth_dev_info_get((uint8_t) portid, &dev_info); + } + +The next step is to configure the RX and TX queues. +For each port, there is only one RX queue (only one lcore is able to poll a given port). +The number of TX queues depends on the number of available lcores. +The rte_eth_dev_configure() function is used to configure the number of queues for a port: + +.. code-block:: c + + ret = rte_eth_dev_configure((uint8_t)portid, 1, 1, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device: " + "err=%d, port=%u\n", + ret, portid); + +RX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +The application uses one lcore to poll one or several ports, depending on the -q option, +which specifies the number of queues per lcore. + +For example, if the user specifies -q 4, the application is able to poll four ports with one lcore. +If there are 16 ports on the target (and if the portmask argument is -p ffff ), +the application will need four lcores to poll all the ports. + +.. code-block:: c + + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, + rte_eth_dev_socket_id(portid), + NULL, + l2fwd_pktmbuf_pool); + + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, port=%u\n", + ret, (unsigned) portid); + +The list of queues that must be polled for a given lcore is stored in a private structure called struct lcore_queue_conf. + +.. code-block:: c + + struct lcore_queue_conf { + unsigned n_rx_port; + unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE]; + truct mbuf_table tx_mbufs[RTE_MAX_ETHPORTS]; + + struct rte_timer rx_timers[MAX_RX_QUEUE_PER_LCORE]; + struct rte_jobstats port_fwd_jobs[MAX_RX_QUEUE_PER_LCORE]; + + struct rte_timer flush_timer; + struct rte_jobstats flush_job; + struct rte_jobstats idle_job; + struct rte_jobstats_context jobs_context; + + rte_atomic16_t stats_read_pending; + rte_spinlock_t lock; + } __rte_cache_aligned; + +Values of struct lcore_queue_conf: + +* n_rx_port and rx_port_list[] are used in the main packet processing loop + (see Section `Receive, Process and Transmit Packets`_ later in this chapter). + +* rx_timers and flush_timer are used to ensure forced TX on low packet rate. + +* flush_job, idle_job and jobs_context are librte_jobstats objects used for managing l2fwd jobs. + +* stats_read_pending and lock are used during job stats read phase. + +TX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +Each lcore should be able to transmit on any port. For every port, a single TX queue is initialized. + +.. code-block:: c + + /* init one TX queue on each port */ + + fflush(stdout); + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, + rte_eth_dev_socket_id(portid), + NULL); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n", + ret, (unsigned) portid); + +Jobs statistics initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +There are several statistics objects available: + +* Flush job statistics + +.. code-block:: c + + rte_jobstats_init(&qconf->flush_job, "flush", drain_tsc, drain_tsc, + drain_tsc, 0); + + rte_timer_init(&qconf->flush_timer); + ret = rte_timer_reset(&qconf->flush_timer, drain_tsc, PERIODICAL, + lcore_id, &l2fwd_flush_job, NULL); + + if (ret < 0) { + rte_exit(1, "Failed to reset flush job timer for lcore %u: %s", + lcore_id, rte_strerror(-ret)); + } + +* Statistics per RX port + +.. code-block:: c + + rte_jobstats_init(job, name, 0, drain_tsc, 0, MAX_PKT_BURST); + rte_jobstats_set_update_period_function(job, l2fwd_job_update_cb); + + rte_timer_init(&qconf->rx_timers[i]); + ret = rte_timer_reset(&qconf->rx_timers[i], 0, PERIODICAL, lcore_id, + l2fwd_fwd_job, (void *)(uintptr_t)i); + + if (ret < 0) { + rte_exit(1, "Failed to reset lcore %u port %u job timer: %s", + lcore_id, qconf->rx_port_list[i], rte_strerror(-ret)); + } + +Following parameters are passed to rte_jobstats_init(): + +* 0 as minimal poll period + +* drain_tsc as maximum poll period + +* MAX_PKT_BURST as desired target value (RX burst size) + +Main loop +~~~~~~~~~ + +The forwarding path is reworked comparing to original L2 Forwarding application. +In the l2fwd_main_loop() function three loops are placed. + +.. code-block:: c + + for (;;) { + rte_spinlock_lock(&qconf->lock); + + do { + rte_jobstats_context_start(&qconf->jobs_context); + + /* Do the Idle job: + * - Read stats_read_pending flag + * - check if some real job need to be executed + */ + rte_jobstats_start(&qconf->jobs_context, &qconf->idle_job); + + do { + uint8_t i; + uint64_t now = rte_get_timer_cycles(); + + need_manage = qconf->flush_timer.expire < now; + /* Check if we was esked to give a stats. */ + stats_read_pending = + rte_atomic16_read(&qconf->stats_read_pending); + need_manage |= stats_read_pending; + + for (i = 0; i < qconf->n_rx_port && !need_manage; i++) + need_manage = qconf->rx_timers[i].expire < now; + + } while (!need_manage); + rte_jobstats_finish(&qconf->idle_job, qconf->idle_job.target); + + rte_timer_manage(); + rte_jobstats_context_finish(&qconf->jobs_context); + } while (likely(stats_read_pending == 0)); + + rte_spinlock_unlock(&qconf->lock); + rte_pause(); + } + +First infinite for loop is to minimize impact of stats reading. Lock is only locked/unlocked when asked. + +Second inner while loop do the whole jobs management. When any job is ready, the use rte_timer_manage() is used to call the job handler. +In this place functions l2fwd_fwd_job() and l2fwd_flush_job() are called when needed. +Then rte_jobstats_context_finish() is called to mark loop end - no other jobs are ready to execute. By this time stats are ready to be read +and if stats_read_pending is set, loop breaks allowing stats to be read. + +Third do-while loop is the idle job (idle stats counter). Its only purpose is monitoring if any job is ready or stats job read is pending +for this lcore. Statistics from this part of code is considered as the headroom available for additional processing. + +Receive, Process and Transmit Packets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main task of l2fwd_fwd_job() function is to read ingress packets from the RX queue of particular port and forward it. +This is done using the following code: + +.. code-block:: c + + total_nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, pkts_burst, + MAX_PKT_BURST); + + for (j = 0; j < total_nb_rx; j++) { + m = pkts_burst[j]; + rte_prefetch0(rte_pktmbuf_mtod(m, void *)); + l2fwd_simple_forward(m, portid); + } + +Packets are read in a burst of size MAX_PKT_BURST. +Then, each mbuf in the table is processed by the l2fwd_simple_forward() function. +The processing is very simple: process the TX port from the RX port, then replace the source and destination MAC addresses. + +The rte_eth_rx_burst() function writes the mbuf pointers in a local table and returns the number of available mbufs in the table. + +After first read second try is issued. + +.. code-block:: c + + if (total_nb_rx == MAX_PKT_BURST) { + const uint16_t nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, pkts_burst, + MAX_PKT_BURST); + + total_nb_rx += nb_rx; + for (j = 0; j < nb_rx; j++) { + m = pkts_burst[j]; + rte_prefetch0(rte_pktmbuf_mtod(m, void *)); + l2fwd_simple_forward(m, portid); + } + } + +This second read is important to give job stats library a feedback how many packets was processed. + +.. code-block:: c + + /* Adjust period time in which we are running here. */ + if (rte_jobstats_finish(job, total_nb_rx) != 0) { + rte_timer_reset(&qconf->rx_timers[port_idx], job->period, PERIODICAL, + lcore_id, l2fwd_fwd_job, arg); + } + +To maximize performance exactly MAX_PKT_BURST is expected (the target value) to be read for each l2fwd_fwd_job() call. +If total_nb_rx is smaller than target value job->period will be increased. If it is greater the period will be decreased. + +.. note:: + + In the following code, one line for getting the output port requires some explanation. + +During the initialization process, a static array of destination ports (l2fwd_dst_ports[]) is filled such that for each source port, +a destination port is assigned that is either the next or previous enabled port from the portmask. +Naturally, the number of ports in the portmask must be even, otherwise, the application exits. + +.. code-block:: c + + static void + l2fwd_simple_forward(struct rte_mbuf *m, unsigned portid) + { + struct ether_hdr *eth; + void *tmp; + unsigned dst_port; + + dst_port = l2fwd_dst_ports[portid]; + + eth = rte_pktmbuf_mtod(m, struct ether_hdr *); + + /* 02:00:00:00:00:xx */ + + tmp = ð->d_addr.addr_bytes[0]; + + *((uint64_t *)tmp) = 0x000000000002 + ((uint64_t) dst_port << 40); + + /* src addr */ + + ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr); + + l2fwd_send_packet(m, (uint8_t) dst_port); + } + +Then, the packet is sent using the l2fwd_send_packet (m, dst_port) function. +For this test application, the processing is exactly the same for all packets arriving on the same RX port. +Therefore, it would have been possible to call the l2fwd_send_burst() function directly from the main loop +to send all the received packets on the same TX port, +using the burst-oriented send function, which is more efficient. + +However, in real-life applications (such as, L3 routing), +packet N is not necessarily forwarded on the same port as packet N-1. +The application is implemented to illustrate that, so the same approach can be reused in a more complex application. + +The l2fwd_send_packet() function stores the packet in a per-lcore and per-txport table. +If the table is full, the whole packets table is transmitted using the l2fwd_send_burst() function: + +.. code-block:: c + + /* Send the packet on an output interface */ + + static int + l2fwd_send_packet(struct rte_mbuf *m, uint16_t port) + { + unsigned lcore_id, len; + struct lcore_queue_conf *qconf; + + lcore_id = rte_lcore_id(); + qconf = &lcore_queue_conf[lcore_id]; + len = qconf->tx_mbufs[port].len; + qconf->tx_mbufs[port].m_table[len] = m; + len++; + + /* enough pkts to be sent */ + + if (unlikely(len == MAX_PKT_BURST)) { + l2fwd_send_burst(qconf, MAX_PKT_BURST, port); + len = 0; + } + + qconf->tx_mbufs[port].len = len; return 0; + } + +To ensure that no packets remain in the tables, the flush job exists. The l2fwd_flush_job() +is called periodically to for each lcore draining TX queue of each port. +This technique introduces some latency when there are not many packets to send, +however it improves performance: + +.. code-block:: c + + static void + l2fwd_flush_job(__rte_unused struct rte_timer *timer, __rte_unused void *arg) + { + uint64_t now; + unsigned lcore_id; + struct lcore_queue_conf *qconf; + struct mbuf_table *m_table; + uint16_t portid; + + lcore_id = rte_lcore_id(); + qconf = &lcore_queue_conf[lcore_id]; + + rte_jobstats_start(&qconf->jobs_context, &qconf->flush_job); + + now = rte_get_timer_cycles(); + lcore_id = rte_lcore_id(); + qconf = &lcore_queue_conf[lcore_id]; + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) { + m_table = &qconf->tx_mbufs[portid]; + if (m_table->len == 0 || m_table->next_flush_time <= now) + continue; + + l2fwd_send_burst(qconf, portid); + } + + + /* Pass target to indicate that this job is happy of time interval + * in which it was called. */ + rte_jobstats_finish(&qconf->flush_job, qconf->flush_job.target); + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_real_virtual.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_real_virtual.rst new file mode 100644 index 00000000..2b2d5afa --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l2_forward_real_virtual.rst @@ -0,0 +1,476 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +.. _l2_fwd_app_real_and_virtual: + +L2 Forwarding Sample Application (in Real and Virtualized Environments) +======================================================================= + +The L2 Forwarding sample application is a simple example of packet processing using +the Data Plane Development Kit (DPDK) which +also takes advantage of Single Root I/O Virtualization (SR-IOV) features in a virtualized environment. + +.. note:: + + Please note that previously a separate L2 Forwarding in Virtualized Environments sample application was used, + however, in later DPDK versions these sample applications have been merged. + +Overview +-------- + +The L2 Forwarding sample application, which can operate in real and virtualized environments, +performs L2 forwarding for each packet that is received on an RX_PORT. +The destination port is the adjacent port from the enabled portmask, that is, +if the first four ports are enabled (portmask 0xf), +ports 1 and 2 forward into each other, and ports 3 and 4 forward into each other. +Also, if MAC addresses updating is enabled, the MAC addresses are affected as follows: + +* The source MAC address is replaced by the TX_PORT MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup`, +or in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup`. + +.. _figure_l2_fwd_benchmark_setup: + +.. figure:: img/l2_fwd_benchmark_setup.* + + Performance Benchmark Setup (Basic Environment) + +.. _figure_l2_fwd_virtenv_benchmark_setup: + +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* + + Performance Benchmark Setup (Virtualized Environment) + +This application may be used for basic VM to VM communication as shown in :numref:`figure_l2_fwd_vm2vm`, +when MAC addresses updating is disabled. + +.. _figure_l2_fwd_vm2vm: + +.. figure:: img/l2_fwd_vm2vm.* + + Virtual Machine to Virtual Machine communication. + +The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. + +.. _l2_fwd_vf_setup: + +Virtual Function Setup Instructions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This application can use the virtual function available in the system and +therefore can be used in a virtual machine without passing through +the whole Network Device into a guest machine in a virtualized scenario. +The virtual functions can be enabled in the host machine or the hypervisor with the respective physical function driver. + +For example, in a Linux* host machine, it is possible to enable a virtual function using the following command: + +.. code-block:: console + + modprobe ixgbe max_vfs=2,2 + +This command enables two Virtual Functions on each of Physical Function of the NIC, +with two physical ports in the PCI configuration space. +It is important to note that enabled Virtual Function 0 and 2 would belong to Physical Function 0 +and Virtual Function 1 and 3 would belong to Physical Function 1, +in this case enabling a total of four Virtual Functions. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l2fwd`` sub-directory. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + +where, + +* p PORTMASK: A hexadecimal bitmask of the ports to configure + +* q NQ: A number of queues (=ports) per lcore (default is 1) + +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). + +To run the application in linuxapp environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address +updating enabled, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +.. _l2_fwd_app_cmd_arguments: + +Command Line Arguments +~~~~~~~~~~~~~~~~~~~~~~ + +The L2 Forwarding sample application takes specific parameters, +in addition to Environment Abstraction Layer (EAL) arguments. +The preferred way to parse parameters is to use the getopt() function, +since it is part of a well-defined and portable library. + +The parsing of arguments is done in the l2fwd_parse_args() function. +The method of argument parsing is not described here. +Refer to the *glibc getopt(3)* man page for details. + +EAL arguments are parsed first, then application-specific arguments. +This is done at the beginning of the main() function: + +.. code-block:: c + + /* init EAL */ + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + argc -= ret; + argv += ret; + + /* parse application arguments (after the EAL ones) */ + + ret = l2fwd_parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n"); + +.. _l2_fwd_app_mbuf_init: + +Mbuf Pool Initialization +~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the arguments are parsed, the mbuf pool is created. +The mbuf pool contains a set of mbuf objects that will be used by the driver +and the application to store network packet data: + +.. code-block:: c + + /* create the mbuf pool */ + + l2fwd_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, + MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + + if (l2fwd_pktmbuf_pool == NULL) + rte_panic("Cannot init mbuf pool\n"); + +The rte_mempool is a generic structure used to handle pools of objects. +In this case, it is necessary to create a pool that will be used by the driver. +The number of allocated pkt mbufs is NB_MBUF, with a data room size of +RTE_MBUF_DEFAULT_BUF_SIZE each. +A per-lcore cache of 32 mbufs is kept. +The memory is allocated in NUMA socket 0, +but it is possible to extend this code to allocate one mbuf pool per socket. + +The rte_pktmbuf_pool_create() function uses the default mbuf pool and mbuf +initializers, respectively rte_pktmbuf_pool_init() and rte_pktmbuf_init(). +An advanced application may want to use the mempool API to create the +mbuf pool with more control. + +.. _l2_fwd_app_dvr_init: + +Driver Initialization +~~~~~~~~~~~~~~~~~~~~~ + +The main part of the code in the main() function relates to the initialization of the driver. +To fully understand this code, it is recommended to study the chapters that related to the Poll Mode Driver +in the *DPDK Programmer's Guide* - Rel 1.4 EAR and the *DPDK API Reference*. + +.. code-block:: c + + if (rte_pci_probe() < 0) + rte_exit(EXIT_FAILURE, "Cannot probe PCI\n"); + + /* reset l2fwd_dst_ports */ + + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) + l2fwd_dst_ports[portid] = 0; + + last_port = 0; + + /* + * Each logical core is assigned a dedicated TX queue on each port. + */ + + RTE_ETH_FOREACH_DEV(portid) { + /* skip ports that are not enabled */ + + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) + continue; + + if (nb_ports_in_mask % 2) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + } + else + last_port = portid; + + nb_ports_in_mask++; + + rte_eth_dev_info_get((uint8_t) portid, &dev_info); + } + +Observe that: + +* rte_igb_pmd_init_all() simultaneously registers the driver as a PCI driver and as an Ethernet* Poll Mode Driver. + +* rte_pci_probe() parses the devices on the PCI bus and initializes recognized devices. + +The next step is to configure the RX and TX queues. +For each port, there is only one RX queue (only one lcore is able to poll a given port). +The number of TX queues depends on the number of available lcores. +The rte_eth_dev_configure() function is used to configure the number of queues for a port: + +.. code-block:: c + + ret = rte_eth_dev_configure((uint8_t)portid, 1, 1, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device: " + "err=%d, port=%u\n", + ret, portid); + +.. _l2_fwd_app_rx_init: + +RX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +The application uses one lcore to poll one or several ports, depending on the -q option, +which specifies the number of queues per lcore. + +For example, if the user specifies -q 4, the application is able to poll four ports with one lcore. +If there are 16 ports on the target (and if the portmask argument is -p ffff ), +the application will need four lcores to poll all the ports. + +.. code-block:: c + + ret = rte_eth_rx_queue_setup((uint8_t) portid, 0, nb_rxd, SOCKET0, &rx_conf, l2fwd_pktmbuf_pool); + if (ret < 0) + + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: " + "err=%d, port=%u\n", + ret, portid); + +The list of queues that must be polled for a given lcore is stored in a private structure called struct lcore_queue_conf. + +.. code-block:: c + + struct lcore_queue_conf { + unsigned n_rx_port; + unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE]; + struct mbuf_table tx_mbufs[L2FWD_MAX_PORTS]; + } rte_cache_aligned; + + struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE]; + +The values n_rx_port and rx_port_list[] are used in the main packet processing loop +(see :ref:`l2_fwd_app_rx_tx_packets`). + +The global configuration for the RX queues is stored in a static structure: + +.. code-block:: c + + static const struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + }; + +.. _l2_fwd_app_tx_init: + +TX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +Each lcore should be able to transmit on any port. For every port, a single TX queue is initialized. + +.. code-block:: c + + /* init one TX queue on each port */ + + fflush(stdout); + + ret = rte_eth_tx_queue_setup((uint8_t) portid, 0, nb_txd, rte_eth_dev_socket_id(portid), &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup:err=%d, port=%u\n", ret, (unsigned) portid); + +The global configuration for TX queues is stored in a static structure: + +.. code-block:: c + + static const struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = RTE_TEST_TX_DESC_DEFAULT + 1, /* disable feature */ + }; + +.. _l2_fwd_app_rx_tx_packets: + +Receive, Process and Transmit Packets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the l2fwd_main_loop() function, the main task is to read ingress packets from the RX queues. +This is done using the following code: + +.. code-block:: c + + /* + * Read packet from RX queues + */ + + for (i = 0; i < qconf->n_rx_port; i++) { + portid = qconf->rx_port_list[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, pkts_burst, MAX_PKT_BURST); + + for (j = 0; j < nb_rx; j++) { + m = pkts_burst[j]; + rte_prefetch0[rte_pktmbuf_mtod(m, void *)); l2fwd_simple_forward(m, portid); + } + } + +Packets are read in a burst of size MAX_PKT_BURST. +The rte_eth_rx_burst() function writes the mbuf pointers in a local table and returns the number of available mbufs in the table. + +Then, each mbuf in the table is processed by the l2fwd_simple_forward() function. +The processing is very simple: process the TX port from the RX port, then replace the source and destination MAC addresses if MAC +addresses updating is enabled. + +.. note:: + + In the following code, one line for getting the output port requires some explanation. + +During the initialization process, a static array of destination ports (l2fwd_dst_ports[]) is filled such that for each source port, +a destination port is assigned that is either the next or previous enabled port from the portmask. +Naturally, the number of ports in the portmask must be even, otherwise, the application exits. + +.. code-block:: c + + static void + l2fwd_simple_forward(struct rte_mbuf *m, unsigned portid) + { + struct ether_hdr *eth; + void *tmp; + unsigned dst_port; + + dst_port = l2fwd_dst_ports[portid]; + + eth = rte_pktmbuf_mtod(m, struct ether_hdr *); + + /* 02:00:00:00:00:xx */ + + tmp = ð->d_addr.addr_bytes[0]; + + *((uint64_t *)tmp) = 0x000000000002 + ((uint64_t) dst_port << 40); + + /* src addr */ + + ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr); + + l2fwd_send_packet(m, (uint8_t) dst_port); + } + +Then, the packet is sent using the l2fwd_send_packet (m, dst_port) function. +For this test application, the processing is exactly the same for all packets arriving on the same RX port. +Therefore, it would have been possible to call the l2fwd_send_burst() function directly from the main loop +to send all the received packets on the same TX port, +using the burst-oriented send function, which is more efficient. + +However, in real-life applications (such as, L3 routing), +packet N is not necessarily forwarded on the same port as packet N-1. +The application is implemented to illustrate that, so the same approach can be reused in a more complex application. + +The l2fwd_send_packet() function stores the packet in a per-lcore and per-txport table. +If the table is full, the whole packets table is transmitted using the l2fwd_send_burst() function: + +.. code-block:: c + + /* Send the packet on an output interface */ + + static int + l2fwd_send_packet(struct rte_mbuf *m, uint16_t port) + { + unsigned lcore_id, len; + struct lcore_queue_conf *qconf; + + lcore_id = rte_lcore_id(); + qconf = &lcore_queue_conf[lcore_id]; + len = qconf->tx_mbufs[port].len; + qconf->tx_mbufs[port].m_table[len] = m; + len++; + + /* enough pkts to be sent */ + + if (unlikely(len == MAX_PKT_BURST)) { + l2fwd_send_burst(qconf, MAX_PKT_BURST, port); + len = 0; + } + + qconf->tx_mbufs[port].len = len; return 0; + } + +To ensure that no packets remain in the tables, each lcore does a draining of TX queue in its main loop. +This technique introduces some latency when there are not many packets to send, +however it improves performance: + +.. code-block:: c + + cur_tsc = rte_rdtsc(); + + /* + * TX burst queue drain + */ + + diff_tsc = cur_tsc - prev_tsc; + + if (unlikely(diff_tsc > drain_tsc)) { + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) { + if (qconf->tx_mbufs[portid].len == 0) + continue; + + l2fwd_send_burst(&lcore_queue_conf[lcore_id], qconf->tx_mbufs[portid].len, (uint8_t) portid); + + qconf->tx_mbufs[portid].len = 0; + } + + /* if timer is enabled */ + + if (timer_period > 0) { + /* advance the timer */ + + timer_tsc += diff_tsc; + + /* if timer has reached its timeout */ + + if (unlikely(timer_tsc >= (uint64_t) timer_period)) { + /* do this only on master core */ + + if (lcore_id == rte_get_master_lcore()) { + print_stats(); + + /* reset the timer */ + timer_tsc = 0; + } + } + } + + prev_tsc = cur_tsc; + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward.rst new file mode 100644 index 00000000..ddd0f9a8 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward.rst @@ -0,0 +1,314 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +L3 Forwarding Sample Application +================================ + +The L3 Forwarding application is a simple example of packet processing using the DPDK. +The application performs L3 forwarding. + +Overview +-------- + +The application demonstrates the use of the hash and LPM libraries in the DPDK to implement packet forwarding. +The initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +The main difference from the L2 Forwarding sample application is that the forwarding decision +is made based on information read from the input packet. + +The lookup method is either hash-based or LPM-based and is selected at run time. When the selected lookup method is hash-based, +a hash object is used to emulate the flow classification stage. +The hash object is used in correlation with a flow table to map each input packet to its flow at runtime. + +The hash lookup key is represented by a DiffServ 5-tuple composed of the following fields read from the input packet: +Source IP Address, Destination IP Address, Protocol, Source Port and Destination Port. +The ID of the output interface for the input packet is read from the identified flow table entry. +The set of flows used by the application is statically configured and loaded into the hash at initialization time. +When the selected lookup method is LPM based, an LPM object is used to emulate the forwarding stage for IPv4 packets. +The LPM object is used as the routing table to identify the next hop for each input packet at runtime. + +The LPM lookup key is represented by the Destination IP Address field read from the input packet. +The ID of the output interface for the input packet is the next hop returned by the LPM lookup. +The set of LPM rules used by the application is statically configured and loaded into the LPM object at initialization time. + +In the sample application, hash-based forwarding supports IPv4 and IPv6. LPM-based forwarding supports IPv4 only. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l3fwd`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options:: + + ./l3fwd [EAL options] -- -p PORTMASK + [-P] + [-E] + [-L] + --config(port,queue,lcore)[,(port,queue,lcore)] + [--eth-dest=X,MM:MM:MM:MM:MM:MM] + [--enable-jumbo [--max-pkt-len PKTLEN]] + [--no-numa] + [--hash-entry-num] + [--ipv6] + [--parse-ptype] + +Where, + +* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure + +* ``-P:`` Optional, sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. + +* ``-E:`` Optional, enable exact match. + +* ``-L:`` Optional, enable longest prefix match. + +* ``--config (port,queue,lcore)[,(port,queue,lcore)]:`` Determines which queues from which ports are mapped to which cores. + +* ``--eth-dest=X,MM:MM:MM:MM:MM:MM:`` Optional, ethernet destination for port X. + +* ``--enable-jumbo:`` Optional, enables jumbo frames. + +* ``--max-pkt-len:`` Optional, under the premise of enabling jumbo, maximum packet length in decimal (64-9600). + +* ``--no-numa:`` Optional, disables numa awareness. + +* ``--hash-entry-num:`` Optional, specifies the hash entry number in hexadecimal to be setup. + +* ``--ipv6:`` Optional, set if running ipv6 packets. + +* ``--parse-ptype:`` Optional, set to use software to analyze packet type. Without this option, hardware will check the packet type. + +For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, +while cores 8-15 and 24-31 appear on socket 1. + +To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2, +(which are in the same socket too), use the following command: + +.. code-block:: console + + ./build/l3fwd -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)" + +In this command: + +* The -l option enables cores 1, 2 + +* The -p option enables ports 0 and 1 + +* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core. + The following table shows the mapping in this example: + ++----------+-----------+-----------+-------------------------------------+ +| **Port** | **Queue** | **lcore** | **Description** | +| | | | | ++----------+-----------+-----------+-------------------------------------+ +| 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. | +| | | | | ++----------+-----------+-----------+-------------------------------------+ +| 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. | +| | | | | ++----------+-----------+-----------+-------------------------------------+ + +Refer to the *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +.. _l3_fwd_explanation: + +Explanation +----------- + +The following sections provide some explanation of the sample application code. As mentioned in the overview section, +the initialization and run-time paths are very similar to those of the :doc:`l2_forward_real_virtual`. +The following sections describe aspects that are specific to the L3 Forwarding sample application. + +Hash Initialization +~~~~~~~~~~~~~~~~~~~ + +The hash object is created and loaded with the pre-configured entries read from a global array, +and then generate the expected 5-tuple as key to keep consistence with those of real flow +for the convenience to execute hash performance test on 4M/8M/16M flows. + +.. note:: + + The Hash initialization will setup both ipv4 and ipv6 hash table, + and populate the either table depending on the value of variable ipv6. + To support the hash performance test with up to 8M single direction flows/16M bi-direction flows, + populate_ipv4_many_flow_into_table() function will populate the hash table with specified hash table entry number(default 4M). + +.. note:: + + Value of global variable ipv6 can be specified with --ipv6 in the command line. + Value of global variable hash_entry_number, + which is used to specify the total hash entry number for all used ports in hash performance test, + can be specified with --hash-entry-num VALUE in command line, being its default value 4. + +.. code-block:: c + + #if (APP_LOOKUP_METHOD == APP_LOOKUP_EXACT_MATCH) + + static void + setup_hash(int socketid) + { + // ... + + if (hash_entry_number != HASH_ENTRY_NUMBER_DEFAULT) { + if (ipv6 == 0) { + /* populate the ipv4 hash */ + populate_ipv4_many_flow_into_table(ipv4_l3fwd_lookup_struct[socketid], hash_entry_number); + } else { + /* populate the ipv6 hash */ + populate_ipv6_many_flow_into_table( ipv6_l3fwd_lookup_struct[socketid], hash_entry_number); + } + } else + if (ipv6 == 0) { + /* populate the ipv4 hash */ + populate_ipv4_few_flow_into_table(ipv4_l3fwd_lookup_struct[socketid]); + } else { + /* populate the ipv6 hash */ + populate_ipv6_few_flow_into_table(ipv6_l3fwd_lookup_struct[socketid]); + } + } + } + #endif + +LPM Initialization +~~~~~~~~~~~~~~~~~~ + +The LPM object is created and loaded with the pre-configured entries read from a global array. + +.. code-block:: c + + #if (APP_LOOKUP_METHOD == APP_LOOKUP_LPM) + + static void + setup_lpm(int socketid) + { + unsigned i; + int ret; + char s[64]; + + /* create the LPM table */ + + snprintf(s, sizeof(s), "IPV4_L3FWD_LPM_%d", socketid); + + ipv4_l3fwd_lookup_struct[socketid] = rte_lpm_create(s, socketid, IPV4_L3FWD_LPM_MAX_RULES, 0); + + if (ipv4_l3fwd_lookup_struct[socketid] == NULL) + rte_exit(EXIT_FAILURE, "Unable to create the l3fwd LPM table" + " on socket %d\n", socketid); + + /* populate the LPM table */ + + for (i = 0; i < IPV4_L3FWD_NUM_ROUTES; i++) { + /* skip unused ports */ + + if ((1 << ipv4_l3fwd_route_array[i].if_out & enabled_port_mask) == 0) + continue; + + ret = rte_lpm_add(ipv4_l3fwd_lookup_struct[socketid], ipv4_l3fwd_route_array[i].ip, + ipv4_l3fwd_route_array[i].depth, ipv4_l3fwd_route_array[i].if_out); + + if (ret < 0) { + rte_exit(EXIT_FAILURE, "Unable to add entry %u to the " + "l3fwd LPM table on socket %d\n", i, socketid); + } + + printf("LPM: Adding route 0x%08x / %d (%d)\n", + (unsigned)ipv4_l3fwd_route_array[i].ip, ipv4_l3fwd_route_array[i].depth, ipv4_l3fwd_route_array[i].if_out); + } + } + #endif + +Packet Forwarding for Hash-based Lookups +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For each input packet, the packet forwarding operation is done by the l3fwd_simple_forward() +or simple_ipv4_fwd_4pkts() function for IPv4 packets or the simple_ipv6_fwd_4pkts() function for IPv6 packets. +The l3fwd_simple_forward() function provides the basic functionality for both IPv4 and IPv6 packet forwarding +for any number of burst packets received, +and the packet forwarding decision (that is, the identification of the output interface for the packet) +for hash-based lookups is done by the get_ipv4_dst_port() or get_ipv6_dst_port() function. +The get_ipv4_dst_port() function is shown below: + +.. code-block:: c + + static inline uint8_t + get_ipv4_dst_port(void *ipv4_hdr, uint16_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) + { + int ret = 0; + union ipv4_5tuple_host key; + + ipv4_hdr = (uint8_t *)ipv4_hdr + offsetof(struct ipv4_hdr, time_to_live); + + m128i data = _mm_loadu_si128(( m128i*)(ipv4_hdr)); + + /* Get 5 tuple: dst port, src port, dst IP address, src IP address and protocol */ + + key.xmm = _mm_and_si128(data, mask0); + + /* Find destination port */ + + ret = rte_hash_lookup(ipv4_l3fwd_lookup_struct, (const void *)&key); + + return (uint8_t)((ret < 0)? portid : ipv4_l3fwd_out_if[ret]); + } + +The get_ipv6_dst_port() function is similar to the get_ipv4_dst_port() function. + +The simple_ipv4_fwd_4pkts() and simple_ipv6_fwd_4pkts() function are optimized for continuous 4 valid ipv4 and ipv6 packets, +they leverage the multiple buffer optimization to boost the performance of forwarding packets with the exact match on hash table. +The key code snippet of simple_ipv4_fwd_4pkts() is shown below: + +.. code-block:: c + + static inline void + simple_ipv4_fwd_4pkts(struct rte_mbuf* m[4], uint16_t portid, struct lcore_conf *qconf) + { + // ... + + data[0] = _mm_loadu_si128(( m128i*)(rte_pktmbuf_mtod(m[0], unsigned char *) + sizeof(struct ether_hdr) + offsetof(struct ipv4_hdr, time_to_live))); + data[1] = _mm_loadu_si128(( m128i*)(rte_pktmbuf_mtod(m[1], unsigned char *) + sizeof(struct ether_hdr) + offsetof(struct ipv4_hdr, time_to_live))); + data[2] = _mm_loadu_si128(( m128i*)(rte_pktmbuf_mtod(m[2], unsigned char *) + sizeof(struct ether_hdr) + offsetof(struct ipv4_hdr, time_to_live))); + data[3] = _mm_loadu_si128(( m128i*)(rte_pktmbuf_mtod(m[3], unsigned char *) + sizeof(struct ether_hdr) + offsetof(struct ipv4_hdr, time_to_live))); + + key[0].xmm = _mm_and_si128(data[0], mask0); + key[1].xmm = _mm_and_si128(data[1], mask0); + key[2].xmm = _mm_and_si128(data[2], mask0); + key[3].xmm = _mm_and_si128(data[3], mask0); + + const void *key_array[4] = {&key[0], &key[1], &key[2],&key[3]}; + + rte_hash_lookup_bulk(qconf->ipv4_lookup_struct, &key_array[0], 4, ret); + + dst_port[0] = (ret[0] < 0)? portid:ipv4_l3fwd_out_if[ret[0]]; + dst_port[1] = (ret[1] < 0)? portid:ipv4_l3fwd_out_if[ret[1]]; + dst_port[2] = (ret[2] < 0)? portid:ipv4_l3fwd_out_if[ret[2]]; + dst_port[3] = (ret[3] < 0)? portid:ipv4_l3fwd_out_if[ret[3]]; + + // ... + } + +The simple_ipv6_fwd_4pkts() function is similar to the simple_ipv4_fwd_4pkts() function. + +Known issue: IP packets with extensions or IP packets which are not TCP/UDP cannot work well at this mode. + +Packet Forwarding for LPM-based Lookups +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For each input packet, the packet forwarding operation is done by the l3fwd_simple_forward() function, +but the packet forwarding decision (that is, the identification of the output interface for the packet) +for LPM-based lookups is done by the get_ipv4_dst_port() function below: + +.. code-block:: c + + static inline uint16_t + get_ipv4_dst_port(struct ipv4_hdr *ipv4_hdr, uint16_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) + { + uint8_t next_hop; + + return ((rte_lpm_lookup(ipv4_l3fwd_lookup_struct, rte_be_to_cpu_32(ipv4_hdr->dst_addr), &next_hop) == 0)? next_hop : portid); + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst new file mode 100644 index 00000000..a44fbcd5 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst @@ -0,0 +1,339 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +L3 Forwarding with Access Control Sample Application +==================================================== + +The L3 Forwarding with Access Control application is a simple example of packet processing using the DPDK. +The application performs a security check on received packets. +Packets that are in the Access Control List (ACL), which is loaded during initialization, are dropped. +Others are forwarded to the correct port. + +Overview +-------- + +The application demonstrates the use of the ACL library in the DPDK to implement access control +and packet L3 forwarding. +The application loads two types of rules at initialization: + +* Route information rules, which are used for L3 forwarding + +* Access Control List (ACL) rules that blacklist (or block) packets with a specific characteristic + +When packets are received from a port, +the application extracts the necessary information from the TCP/IP header of the received packet and +performs a lookup in the rule database to figure out whether the packets should be dropped (in the ACL range) +or forwarded to desired ports. +The initialization and run-time paths are similar to those of the :doc:`l3_forward`. +However, there are significant differences in the two applications. +For example, the original L3 forwarding application uses either LPM or +an exact match algorithm to perform forwarding port lookup, +while this application uses the ACL library to perform both ACL and route entry lookup. +The following sections provide more detail. + +Classification for both IPv4 and IPv6 packets is supported in this application. +The application also assumes that all the packets it processes are TCP/UDP packets and +always extracts source/destination port information from the packets. + +Tuple Packet Syntax +~~~~~~~~~~~~~~~~~~~ + +The application implements packet classification for the IPv4/IPv6 5-tuple syntax specifically. +The 5-tuple syntax consist of a source IP address, a destination IP address, +a source port, a destination port and a protocol identifier. +The fields in the 5-tuple syntax have the following formats: + +* **Source IP address and destination IP address** + : Each is either a 32-bit field (for IPv4), or a set of 4 32-bit fields (for IPv6) represented by a value and a mask length. + For example, an IPv4 range of 192.168.1.0 to 192.168.1.255 could be represented by a value = [192, 168, 1, 0] and a mask length = 24. + +* **Source port and destination port** + : Each is a 16-bit field, represented by a lower start and a higher end. + For example, a range of ports 0 to 8192 could be represented by lower = 0 and higher = 8192. + +* **Protocol identifier** + : An 8-bit field, represented by a value and a mask, that covers a range of values. + To verify that a value is in the range, use the following expression: "(VAL & mask) == value" + +The trick in how to represent a range with a mask and value is as follows. +A range can be enumerated in binary numbers with some bits that are never changed and some bits that are dynamically changed. +Set those bits that dynamically changed in mask and value with 0. +Set those bits that never changed in the mask with 1, in value with number expected. +For example, a range of 6 to 7 is enumerated as 0b110 and 0b111. +Bit 1-7 are bits never changed and bit 0 is the bit dynamically changed. +Therefore, set bit 0 in mask and value with 0, set bits 1-7 in mask with 1, and bits 1-7 in value with number 0b11. +So, mask is 0xfe, value is 0x6. + +.. note:: + + The library assumes that each field in the rule is in LSB or Little Endian order when creating the database. + It internally converts them to MSB or Big Endian order. + When performing a lookup, the library assumes the input is in MSB or Big Endian order. + +Access Rule Syntax +~~~~~~~~~~~~~~~~~~ + +In this sample application, each rule is a combination of the following: + +* 5-tuple field: This field has a format described in Section. + +* priority field: A weight to measure the priority of the rules. + The rule with the higher priority will ALWAYS be returned if the specific input has multiple matches in the rule database. + Rules with lower priority will NEVER be returned in any cases. + +* userdata field: A user-defined field that could be any value. + It can be the forwarding port number if the rule is a route table entry or it can be a pointer to a mapping address + if the rule is used for address mapping in the NAT application. + The key point is that it is a useful reserved field for user convenience. + +ACL and Route Rules +~~~~~~~~~~~~~~~~~~~ + +The application needs to acquire ACL and route rules before it runs. +Route rules are mandatory, while ACL rules are optional. +To simplify the complexity of the priority field for each rule, all ACL and route entries are assumed to be in the same file. +To read data from the specified file successfully, the application assumes the following: + +* Each rule occupies a single line. + +* Only the following four rule line types are valid in this application: + +* ACL rule line, which starts with a leading character '@' + +* Route rule line, which starts with a leading character 'R' + +* Comment line, which starts with a leading character '#' + +* Empty line, which consists of a space, form-feed ('\f'), newline ('\n'), + carriage return ('\r'), horizontal tab ('\t'), or vertical tab ('\v'). + +Other lines types are considered invalid. + +* Rules are organized in descending order of priority, + which means rules at the head of the file always have a higher priority than those further down in the file. + +* A typical IPv4 ACL rule line should have a format as shown below: + + +.. _figure_ipv4_acl_rule: + +.. figure:: img/ipv4_acl_rule.* + + A typical IPv4 ACL rule + + +IPv4 addresses are specified in CIDR format as specified in RFC 4632. +They consist of the dot notation for the address and a prefix length separated by '/'. +For example, 192.168.0.34/32, where the address is 192.168.0.34 and the prefix length is 32. + +Ports are specified as a range of 16-bit numbers in the format MIN:MAX, +where MIN and MAX are the inclusive minimum and maximum values of the range. +The range 0:65535 represents all possible ports in a range. +When MIN and MAX are the same value, a single port is represented, for example, 20:20. + +The protocol identifier is an 8-bit value and a mask separated by '/'. +For example: 6/0xfe matches protocol values 6 and 7. + +* Route rules start with a leading character 'R' and have the same format as ACL rules except an extra field at the tail + that indicates the forwarding port number. + +Rules File Example +~~~~~~~~~~~~~~~~~~ + +.. _figure_example_rules: + +.. figure:: img/example_rules.* + + Rules example + + +Each rule is explained as follows: + +* Rule 1 (the first line) tells the application to drop those packets with source IP address = [1.2.3.*], + destination IP address = [192.168.0.36], protocol = [6]/[7] + +* Rule 2 (the second line) is similar to Rule 1, except the source IP address is ignored. + It tells the application to forward packets with destination IP address = [192.168.0.36], + protocol = [6]/[7], destined to port 1. + +* Rule 3 (the third line) tells the application to forward all packets to port 0. + This is something like a default route entry. + +As described earlier, the application assume rules are listed in descending order of priority, +therefore Rule 1 has the highest priority, then Rule 2, and finally, +Rule 3 has the lowest priority. + +Consider the arrival of the following three packets: + +* Packet 1 has source IP address = [1.2.3.4], destination IP address = [192.168.0.36], and protocol = [6] + +* Packet 2 has source IP address = [1.2.4.4], destination IP address = [192.168.0.36], and protocol = [6] + +* Packet 3 has source IP address = [1.2.3.4], destination IP address = [192.168.0.36], and protocol = [8] + +Observe that: + +* Packet 1 matches all of the rules + +* Packet 2 matches Rule 2 and Rule 3 + +* Packet 3 only matches Rule 3 + +For priority reasons, Packet 1 matches Rule 1 and is dropped. +Packet 2 matches Rule 2 and is forwarded to port 1. +Packet 3 matches Rule 3 and is forwarded to port 0. + +For more details on the rule file format, +please refer to rule_ipv4.db and rule_ipv6.db files (inside <RTE_SDK>/examples/l3fwd-acl/). + +Application Phases +~~~~~~~~~~~~~~~~~~ + +Once the application starts, it transitions through three phases: + +* **Initialization Phase** + - Perform the following tasks: + +* Parse command parameters. Check the validity of rule file(s) name(s), number of logical cores, receive and transmit queues. + Bind ports, queues and logical cores. Check ACL search options, and so on. + +* Call Environmental Abstraction Layer (EAL) and Poll Mode Driver (PMD) functions to initialize the environment and detect possible NICs. + The EAL creates several threads and sets affinity to a specific hardware thread CPU based on the configuration specified + by the command line arguments. + +* Read the rule files and format the rules into the representation that the ACL library can recognize. + Call the ACL library function to add the rules into the database and compile them as a trie of pattern sets. + Note that application maintains a separate AC contexts for IPv4 and IPv6 rules. + +* **Runtime Phase** + - Process the incoming packets from a port. Packets are processed in three steps: + + * Retrieval: Gets a packet from the receive queue. Each logical core may process several queues for different ports. + This depends on the configuration specified by command line arguments. + + * Lookup: Checks that the packet type is supported (IPv4/IPv6) and performs a 5-tuple lookup over corresponding AC context. + If an ACL rule is matched, the packets will be dropped and return back to step 1. + If a route rule is matched, it indicates the packet is not in the ACL list and should be forwarded. + If there is no matches for the packet, then the packet is dropped. + + * Forwarding: Forwards the packet to the corresponding port. + +* **Final Phase** - Perform the following tasks: + + Calls the EAL, PMD driver and ACL library to free resource, then quits. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l3fwd-acl`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/l3fwd-acl [EAL options] -- -p PORTMASK [-P] --config(port,queue,lcore)[,(port,queue,lcore)] --rule_ipv4 FILENAME rule_ipv6 FILENAME [--scalar] [--enable-jumbo [--max-pkt-len PKTLEN]] [--no-numa] + + +where, + +* -p PORTMASK: Hexadecimal bitmask of ports to configure + +* -P: Sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. + +* --config (port,queue,lcore)[,(port,queue,lcore)]: determines which queues from which ports are mapped to which cores + +* --rule_ipv4 FILENAME: Specifies the IPv4 ACL and route rules file + +* --rule_ipv6 FILENAME: Specifies the IPv6 ACL and route rules file + +* --scalar: Use a scalar function to perform rule lookup + +* --enable-jumbo: optional, enables jumbo frames + +* --max-pkt-len: optional, maximum packet length in decimal (64-9600) + +* --no-numa: optional, disables numa awareness + +For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, +while cores 8-15 and 24-31 appear on socket 1. + +To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2, +(which are in the same socket too), use the following command: + +.. code-block:: console + + ./build/l3fwd-acl -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)" --rule_ipv4="./rule_ipv4.db" -- rule_ipv6="./rule_ipv6.db" --scalar + +In this command: + +* The -l option enables cores 1, 2 + +* The -p option enables ports 0 and 1 + +* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core. + The following table shows the mapping in this example: + + +----------+------------+-----------+-------------------------------------+ + | **Port** | **Queue** | **lcore** | **Description** | + | | | | | + +==========+============+===========+=====================================+ + | 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. | + | | | | | + +----------+------------+-----------+-------------------------------------+ + | 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. | + | | | | | + +----------+------------+-----------+-------------------------------------+ + +* The --rule_ipv4 option specifies the reading of IPv4 rules sets from the ./ rule_ipv4.db file. + +* The --rule_ipv6 option specifies the reading of IPv6 rules sets from the ./ rule_ipv6.db file. + +* The --scalar option specifies the performing of rule lookup with a scalar function. + +Explanation +----------- + +The following sections provide some explanation of the sample application code. +The aspects of port, device and CPU configuration are similar to those of the :doc:`l3_forward`. +The following sections describe aspects that are specific to L3 forwarding with access control. + +Parse Rules from File +~~~~~~~~~~~~~~~~~~~~~ + +As described earlier, both ACL and route rules are assumed to be saved in the same file. +The application parses the rules from the file and adds them to the database by calling the ACL library function. +It ignores empty and comment lines, and parses and validates the rules it reads. +If errors are detected, the application exits with messages to identify the errors encountered. + +The application needs to consider the userdata and priority fields. +The ACL rules save the index to the specific rules in the userdata field, +while route rules save the forwarding port number. +In order to differentiate the two types of rules, ACL rules add a signature in the userdata field. +As for the priority field, the application assumes rules are organized in descending order of priority. +Therefore, the code only decreases the priority number with each rule it parses. + +Setting Up the ACL Context +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For each supported AC rule format (IPv4 5-tuple, IPv6 6-tuple) application creates a separate context handler +from the ACL library for each CPU socket on the board and adds parsed rules into that context. + +Note, that for each supported rule type, +application needs to calculate the expected offset of the fields from the start of the packet. +That's why only packets with fixed IPv4/ IPv6 header are supported. +That allows to perform ACL classify straight over incoming packet buffer - +no extra protocol field retrieval need to be performed. + +Subsequently, the application checks whether NUMA is enabled. +If it is, the application records the socket IDs of the CPU cores involved in the task. + +Finally, the application creates contexts handler from the ACL library, +adds rules parsed from the file into the database and build an ACL trie. +It is important to note that the application creates an independent copy of each database for each socket CPU +involved in the task to reduce the time for remote memory access. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_power_man.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_power_man.rst new file mode 100644 index 00000000..795a570b --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_power_man.rst @@ -0,0 +1,364 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +L3 Forwarding with Power Management Sample Application +====================================================== + +Introduction +------------ + +The L3 Forwarding with Power Management application is an example of power-aware packet processing using the DPDK. +The application is based on existing L3 Forwarding sample application, +with the power management algorithms to control the P-states and +C-states of the Intel processor via a power management library. + +Overview +-------- + +The application demonstrates the use of the Power libraries in the DPDK to implement packet forwarding. +The initialization and run-time paths are very similar to those of the :doc:`l3_forward`. +The main difference from the L3 Forwarding sample application is that this application introduces power-aware optimization algorithms +by leveraging the Power library to control P-state and C-state of processor based on packet load. + +The DPDK includes poll-mode drivers to configure Intel NIC devices and their receive (Rx) and transmit (Tx) queues. +The design principle of this PMD is to access the Rx and Tx descriptors directly without any interrupts to quickly receive, +process and deliver packets in the user space. + +In general, the DPDK executes an endless packet processing loop on dedicated IA cores that include the following steps: + +* Retrieve input packets through the PMD to poll Rx queue + +* Process each received packet or provide received packets to other processing cores through software queues + +* Send pending output packets to Tx queue through the PMD + +In this way, the PMD achieves better performance than a traditional interrupt-mode driver, +at the cost of keeping cores active and running at the highest frequency, +hence consuming the maximum power all the time. +However, during the period of processing light network traffic, +which happens regularly in communication infrastructure systems due to well-known "tidal effect", +the PMD is still busy waiting for network packets, which wastes a lot of power. + +Processor performance states (P-states) are the capability of an Intel processor +to switch between different supported operating frequencies and voltages. +If configured correctly, according to system workload, this feature provides power savings. +CPUFreq is the infrastructure provided by the Linux* kernel to control the processor performance state capability. +CPUFreq supports a user space governor that enables setting frequency via manipulating the virtual file device from a user space application. +The Power library in the DPDK provides a set of APIs for manipulating a virtual file device to allow user space application +to set the CPUFreq governor and set the frequency of specific cores. + +This application includes a P-state power management algorithm to generate a frequency hint to be sent to CPUFreq. +The algorithm uses the number of received and available Rx packets on recent polls to make a heuristic decision to scale frequency up/down. +Specifically, some thresholds are checked to see whether a specific core running an DPDK polling thread needs to increase frequency +a step up based on the near to full trend of polled Rx queues. +Also, it decreases frequency a step if packet processed per loop is far less than the expected threshold +or the thread's sleeping time exceeds a threshold. + +C-States are also known as sleep states. +They allow software to put an Intel core into a low power idle state from which it is possible to exit via an event, such as an interrupt. +However, there is a tradeoff between the power consumed in the idle state and the time required to wake up from the idle state (exit latency). +Therefore, as you go into deeper C-states, the power consumed is lower but the exit latency is increased. Each C-state has a target residency. +It is essential that when entering into a C-state, the core remains in this C-state for at least as long as the target residency in order +to fully realize the benefits of entering the C-state. +CPUIdle is the infrastructure provide by the Linux kernel to control the processor C-state capability. +Unlike CPUFreq, CPUIdle does not provide a mechanism that allows the application to change C-state. +It actually has its own heuristic algorithms in kernel space to select target C-state to enter by executing privileged instructions like HLT and MWAIT, +based on the speculative sleep duration of the core. +In this application, we introduce a heuristic algorithm that allows packet processing cores to sleep for a short period +if there is no Rx packet received on recent polls. +In this way, CPUIdle automatically forces the corresponding cores to enter deeper C-states +instead of always running to the C0 state waiting for packets. + +.. note:: + + To fully demonstrate the power saving capability of using C-states, + it is recommended to enable deeper C3 and C6 states in the BIOS during system boot up. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l3fwd-power`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/l3fwd_power [EAL options] -- -p PORTMASK [-P] --config(port,queue,lcore)[,(port,queue,lcore)] [--enable-jumbo [--max-pkt-len PKTLEN]] [--no-numa] + +where, + +* -p PORTMASK: Hexadecimal bitmask of ports to configure + +* -P: Sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. + +* --config (port,queue,lcore)[,(port,queue,lcore)]: determines which queues from which ports are mapped to which cores. + +* --enable-jumbo: optional, enables jumbo frames + +* --max-pkt-len: optional, maximum packet length in decimal (64-9600) + +* --no-numa: optional, disables numa awareness + +See :doc:`l3_forward` for details. +The L3fwd-power example reuses the L3fwd command line options. + +Explanation +----------- + +The following sections provide some explanation of the sample application code. +As mentioned in the overview section, +the initialization and run-time paths are identical to those of the L3 forwarding application. +The following sections describe aspects that are specific to the L3 Forwarding with Power Management sample application. + +Power Library Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Power library is initialized in the main routine. +It changes the P-state governor to userspace for specific cores that are under control. +The Timer library is also initialized and several timers are created later on, +responsible for checking if it needs to scale down frequency at run time by checking CPU utilization statistics. + +.. note:: + + Only the power management related initialization is shown. + +.. code-block:: c + + int main(int argc, char **argv) + { + struct lcore_conf *qconf; + int ret; + unsigned nb_ports; + uint16_t queueid, portid; + unsigned lcore_id; + uint64_t hz; + uint32_t n_tx_queue, nb_lcores; + uint8_t nb_rx_queue, queue, socketid; + + // ... + + /* init RTE timer library to be used to initialize per-core timers */ + + rte_timer_subsystem_init(); + + // ... + + + /* per-core initialization */ + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + /* init power management library for a specified core */ + + ret = rte_power_init(lcore_id); + if (ret) + rte_exit(EXIT_FAILURE, "Power management library " + "initialization failed on core%d\n", lcore_id); + + /* init timer structures for each enabled lcore */ + + rte_timer_init(&power_timers[lcore_id]); + + hz = rte_get_hpet_hz(); + + rte_timer_reset(&power_timers[lcore_id], hz/TIMER_NUMBER_PER_SECOND, SINGLE, lcore_id, power_timer_cb, NULL); + + // ... + } + + // ... + } + +Monitoring Loads of Rx Queues +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In general, the polling nature of the DPDK prevents the OS power management subsystem from knowing +if the network load is actually heavy or light. +In this sample, sampling network load work is done by monitoring received and +available descriptors on NIC Rx queues in recent polls. +Based on the number of returned and available Rx descriptors, +this example implements algorithms to generate frequency scaling hints and speculative sleep duration, +and use them to control P-state and C-state of processors via the power management library. +Frequency (P-state) control and sleep state (C-state) control work individually for each logical core, +and the combination of them contributes to a power efficient packet processing solution when serving light network loads. + +The rte_eth_rx_burst() function and the newly-added rte_eth_rx_queue_count() function are used in the endless packet processing loop +to return the number of received and available Rx descriptors. +And those numbers of specific queue are passed to P-state and C-state heuristic algorithms +to generate hints based on recent network load trends. + +.. note:: + + Only power control related code is shown. + +.. code-block:: c + + static + attribute ((noreturn)) int main_loop( attribute ((unused)) void *dummy) + { + // ... + + while (1) { + // ... + + /** + * Read packet from RX queues + */ + + lcore_scaleup_hint = FREQ_CURRENT; + lcore_rx_idle_count = 0; + + for (i = 0; i < qconf->n_rx_queue; ++i) + { + rx_queue = &(qconf->rx_queue_list[i]); + rx_queue->idle_hint = 0; + portid = rx_queue->port_id; + queueid = rx_queue->queue_id; + + nb_rx = rte_eth_rx_burst(portid, queueid, pkts_burst, MAX_PKT_BURST); + stats[lcore_id].nb_rx_processed += nb_rx; + + if (unlikely(nb_rx == 0)) { + /** + * no packet received from rx queue, try to + * sleep for a while forcing CPU enter deeper + * C states. + */ + + rx_queue->zero_rx_packet_count++; + + if (rx_queue->zero_rx_packet_count <= MIN_ZERO_POLL_COUNT) + continue; + + rx_queue->idle_hint = power_idle_heuristic(rx_queue->zero_rx_packet_count); + lcore_rx_idle_count++; + } else { + rx_ring_length = rte_eth_rx_queue_count(portid, queueid); + + rx_queue->zero_rx_packet_count = 0; + + /** + * do not scale up frequency immediately as + * user to kernel space communication is costly + * which might impact packet I/O for received + * packets. + */ + + rx_queue->freq_up_hint = power_freq_scaleup_heuristic(lcore_id, rx_ring_length); + } + + /* Prefetch and forward packets */ + + // ... + } + + if (likely(lcore_rx_idle_count != qconf->n_rx_queue)) { + for (i = 1, lcore_scaleup_hint = qconf->rx_queue_list[0].freq_up_hint; i < qconf->n_rx_queue; ++i) { + x_queue = &(qconf->rx_queue_list[i]); + + if (rx_queue->freq_up_hint > lcore_scaleup_hint) + + lcore_scaleup_hint = rx_queue->freq_up_hint; + } + + if (lcore_scaleup_hint == FREQ_HIGHEST) + + rte_power_freq_max(lcore_id); + + else if (lcore_scaleup_hint == FREQ_HIGHER) + rte_power_freq_up(lcore_id); + } else { + /** + * All Rx queues empty in recent consecutive polls, + * sleep in a conservative manner, meaning sleep as + * less as possible. + */ + + for (i = 1, lcore_idle_hint = qconf->rx_queue_list[0].idle_hint; i < qconf->n_rx_queue; ++i) { + rx_queue = &(qconf->rx_queue_list[i]); + if (rx_queue->idle_hint < lcore_idle_hint) + lcore_idle_hint = rx_queue->idle_hint; + } + + if ( lcore_idle_hint < SLEEP_GEAR1_THRESHOLD) + /** + * execute "pause" instruction to avoid context + * switch for short sleep. + */ + rte_delay_us(lcore_idle_hint); + else + /* long sleep force ruining thread to suspend */ + usleep(lcore_idle_hint); + + stats[lcore_id].sleep_time += lcore_idle_hint; + } + } + } + +P-State Heuristic Algorithm +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The power_freq_scaleup_heuristic() function is responsible for generating a frequency hint for the specified logical core +according to available descriptor number returned from rte_eth_rx_queue_count(). +On every poll for new packets, the length of available descriptor on an Rx queue is evaluated, +and the algorithm used for frequency hinting is as follows: + +* If the size of available descriptors exceeds 96, the maximum frequency is hinted. + +* If the size of available descriptors exceeds 64, a trend counter is incremented by 100. + +* If the length of the ring exceeds 32, the trend counter is incremented by 1. + +* When the trend counter reached 10000 the frequency hint is changed to the next higher frequency. + +.. note:: + + The assumption is that the Rx queue size is 128 and the thresholds specified above + must be adjusted accordingly based on actual hardware Rx queue size, + which are configured via the rte_eth_rx_queue_setup() function. + +In general, a thread needs to poll packets from multiple Rx queues. +Most likely, different queue have different load, so they would return different frequency hints. +The algorithm evaluates all the hints and then scales up frequency in an aggressive manner +by scaling up to highest frequency as long as one Rx queue requires. +In this way, we can minimize any negative performance impact. + +On the other hand, frequency scaling down is controlled in the timer callback function. +Specifically, if the sleep times of a logical core indicate that it is sleeping more than 25% of the sampling period, +or if the average packet per iteration is less than expectation, the frequency is decreased by one step. + +C-State Heuristic Algorithm +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Whenever recent rte_eth_rx_burst() polls return 5 consecutive zero packets, +an idle counter begins incrementing for each successive zero poll. +At the same time, the function power_idle_heuristic() is called to generate speculative sleep duration +in order to force logical to enter deeper sleeping C-state. +There is no way to control C- state directly, and the CPUIdle subsystem in OS is intelligent enough +to select C-state to enter based on actual sleep period time of giving logical core. +The algorithm has the following sleeping behavior depending on the idle counter: + +* If idle count less than 100, the counter value is used as a microsecond sleep value through rte_delay_us() + which execute pause instructions to avoid costly context switch but saving power at the same time. + +* If idle count is between 100 and 999, a fixed sleep interval of 100 μs is used. + A 100 μs sleep interval allows the core to enter the C1 state while keeping a fast response time in case new traffic arrives. + +* If idle count is greater than 1000, a fixed sleep value of 1 ms is used until the next timer expiration is used. + This allows the core to enter the C3/C6 states. + +.. note:: + + The thresholds specified above need to be adjusted for different Intel processors and traffic profiles. + +If a thread polls multiple Rx queues and different queue returns different sleep duration values, +the algorithm controls the sleep time in a conservative manner by sleeping for the least possible time +in order to avoid a potential performance impact. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_virtual.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_virtual.rst new file mode 100644 index 00000000..21eab8da --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/l3_forward_virtual.rst @@ -0,0 +1,98 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +L3 Forwarding in a Virtualization Environment Sample Application +================================================================ + +The L3 Forwarding in a Virtualization Environment sample application is a simple example of packet processing using the DPDK. +The application performs L3 forwarding that takes advantage of Single Root I/O Virtualization (SR-IOV) features +in a virtualized environment. + +Overview +-------- + +The application demonstrates the use of the hash and LPM libraries in the DPDK to implement packet forwarding. +The initialization and run-time paths are very similar to those of the :doc:`l3_forward`. +The forwarding decision is taken based on information read from the input packet. + +The lookup method is either hash-based or LPM-based and is selected at compile time. +When the selected lookup method is hash-based, a hash object is used to emulate the flow classification stage. +The hash object is used in correlation with the flow table to map each input packet to its flow at runtime. + +The hash lookup key is represented by the DiffServ 5-tuple composed of the following fields read from the input packet: +Source IP Address, Destination IP Address, Protocol, Source Port and Destination Port. +The ID of the output interface for the input packet is read from the identified flow table entry. +The set of flows used by the application is statically configured and loaded into the hash at initialization time. +When the selected lookup method is LPM based, an LPM object is used to emulate the forwarding stage for IPv4 packets. +The LPM object is used as the routing table to identify the next hop for each input packet at runtime. + +The LPM lookup key is represented by the Destination IP Address field read from the input packet. +The ID of the output interface for the input packet is the next hop returned by the LPM lookup. +The set of LPM rules used by the application is statically configured and loaded into the LPM object at the initialization time. + +.. note:: + + Please refer to :ref:`l2_fwd_vf_setup` for virtualized test case setup. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``l3fwd-vf`` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + ./build/l3fwd-vf [EAL options] -- -p PORTMASK --config(port,queue,lcore)[,(port,queue,lcore)] [--no-numa] + +where, + +* --p PORTMASK: Hexadecimal bitmask of ports to configure + +* --config (port,queue,lcore)[,(port,queue,lcore]: determines which queues from which ports are mapped to which cores + +* --no-numa: optional, disables numa awareness + +For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, +while cores 8-15 and 24-31 appear on socket 1. + +To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2, +(which are in the same socket too), use the following command: + +.. code-block:: console + + ./build/l3fwd-vf -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)" + +In this command: + +* The -l option enables cores 1 and 2 + +* The -p option enables ports 0 and 1 + +* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core. + The following table shows the mapping in this example: + + +----------+-----------+-----------+------------------------------------+ + | **Port** | **Queue** | **lcore** | **Description** | + | | | | | + +==========+===========+===========+====================================+ + | 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1 | + | | | | | + +----------+-----------+-----------+------------------------------------+ + | 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2 | + | | | | | + +----------+-----------+-----------+------------------------------------+ + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The operation of this application is similar to that of the basic L3 Forwarding Sample Application. +See :ref:`l3_fwd_explanation` for more information. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/link_status_intr.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/link_status_intr.rst new file mode 100644 index 00000000..c7665fe5 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/link_status_intr.rst @@ -0,0 +1,420 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Link Status Interrupt Sample Application +======================================== + +The Link Status Interrupt sample application is a simple example of packet processing using +the Data Plane Development Kit (DPDK) that +demonstrates how network link status changes for a network port can be captured and +used by a DPDK application. + +Overview +-------- + +The Link Status Interrupt sample application registers a user space callback for the link status interrupt of each port +and performs L2 forwarding for each packet that is received on an RX_PORT. +The following operations are performed: + +* RX_PORT and TX_PORT are paired with available ports one-by-one according to the core mask + +* The source MAC address is replaced by the TX_PORT MAC address + +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID + +This application can be used to demonstrate the usage of link status interrupt and its user space callbacks +and the behavior of L2 forwarding each time the link status changes. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``link_status_interrupt`` sub-directory. + +Running the Application +----------------------- + +The application requires a number of command line options: + +.. code-block:: console + + ./build/link_status_interrupt [EAL options] -- -p PORTMASK [-q NQ][-T PERIOD] + +where, + +* -p PORTMASK: A hexadecimal bitmask of the ports to configure + +* -q NQ: A number of queues (=ports) per lcore (default is 1) + +* -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to disable, 10 default) + +To run the application in a linuxapp environment with 4 lcores, 4 memory channels, 16 ports and 8 RX queues per lcore, +issue the command: + +.. code-block:: console + + $ ./build/link_status_interrupt -l 0-3 -n 4-- -q 8 -p ffff + +Refer to the *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +Command Line Arguments +~~~~~~~~~~~~~~~~~~~~~~ + +The Link Status Interrupt sample application takes specific parameters, +in addition to Environment Abstraction Layer (EAL) arguments (see Section `Running the Application`_). + +Command line parsing is done in the same way as it is done in the L2 Forwarding Sample Application. +See :ref:`l2_fwd_app_cmd_arguments` for more information. + +Mbuf Pool Initialization +~~~~~~~~~~~~~~~~~~~~~~~~ + +Mbuf pool initialization is done in the same way as it is done in the L2 Forwarding Sample Application. +See :ref:`l2_fwd_app_mbuf_init` for more information. + +Driver Initialization +~~~~~~~~~~~~~~~~~~~~~ + +The main part of the code in the main() function relates to the initialization of the driver. +To fully understand this code, it is recommended to study the chapters that related to the Poll Mode Driver in the +*DPDK Programmer's Guide and the DPDK API Reference*. + +.. code-block:: c + + if (rte_pci_probe() < 0) + rte_exit(EXIT_FAILURE, "Cannot probe PCI\n"); + + /* + * Each logical core is assigned a dedicated TX queue on each port. + */ + + RTE_ETH_FOREACH_DEV(portid) { + /* skip ports that are not enabled */ + + if ((lsi_enabled_port_mask & (1 << portid)) == 0) + continue; + + /* save the destination port id */ + + if (nb_ports_in_mask % 2) { + lsi_dst_ports[portid] = portid_last; + lsi_dst_ports[portid_last] = portid; + } + else + portid_last = portid; + + nb_ports_in_mask++; + + rte_eth_dev_info_get((uint8_t) portid, &dev_info); + } + +Observe that: + +* rte_pci_probe() parses the devices on the PCI bus and initializes recognized devices. + +The next step is to configure the RX and TX queues. +For each port, there is only one RX queue (only one lcore is able to poll a given port). +The number of TX queues depends on the number of available lcores. +The rte_eth_dev_configure() function is used to configure the number of queues for a port: + +.. code-block:: c + + ret = rte_eth_dev_configure((uint8_t) portid, 1, 1, &port_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot configure device: err=%d, port=%u\n", ret, portid); + +The global configuration is stored in a static structure: + +.. code-block:: c + + static const struct rte_eth_conf port_conf = { + .rxmode = { + .split_hdr_size = 0, + .offloads = DEV_RX_OFFLOAD_CRC_STRIP, + }, + .txmode = {}, + .intr_conf = { + .lsc = 1, /**< link status interrupt feature enabled */ + }, + }; + +Configuring lsc to 0 (the default) disables the generation of any link status change interrupts in kernel space +and no user space interrupt event is received. +The public interface rte_eth_link_get() accesses the NIC registers directly to update the link status. +Configuring lsc to non-zero enables the generation of link status change interrupts in kernel space +when a link status change is present and calls the user space callbacks registered by the application. +The public interface rte_eth_link_get() just reads the link status in a global structure +that would be updated in the interrupt host thread only. + +Interrupt Callback Registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application can register one or more callbacks to a specific port and interrupt event. +An example callback function that has been written as indicated below. + +.. code-block:: c + + static void + lsi_event_callback(uint16_t port_id, enum rte_eth_event_type type, void *param) + { + struct rte_eth_link link; + + RTE_SET_USED(param); + + printf("\n\nIn registered callback...\n"); + + printf("Event type: %s\n", type == RTE_ETH_EVENT_INTR_LSC ? "LSC interrupt" : "unknown event"); + + rte_eth_link_get_nowait(port_id, &link); + + if (link.link_status) { + printf("Port %d Link Up - speed %u Mbps - %s\n\n", port_id, (unsigned)link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? ("full-duplex") : ("half-duplex")); + } else + printf("Port %d Link Down\n\n", port_id); + } + +This function is called when a link status interrupt is present for the right port. +The port_id indicates which port the interrupt applies to. +The type parameter identifies the interrupt event type, +which currently can be RTE_ETH_EVENT_INTR_LSC only, but other types can be added in the future. +The param parameter is the address of the parameter for the callback. +This function should be implemented with care since it will be called in the interrupt host thread, +which is different from the main thread of its caller. + +The application registers the lsi_event_callback and a NULL parameter to the link status interrupt event on each port: + +.. code-block:: c + + rte_eth_dev_callback_register((uint8_t)portid, RTE_ETH_EVENT_INTR_LSC, lsi_event_callback, NULL); + +This registration can be done only after calling the rte_eth_dev_configure() function and before calling any other function. +If lsc is initialized with 0, the callback is never called since no interrupt event would ever be present. + +RX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +The application uses one lcore to poll one or several ports, depending on the -q option, +which specifies the number of queues per lcore. + +For example, if the user specifies -q 4, the application is able to poll four ports with one lcore. +If there are 16 ports on the target (and if the portmask argument is -p ffff), +the application will need four lcores to poll all the ports. + +.. code-block:: c + + ret = rte_eth_rx_queue_setup((uint8_t) portid, 0, nb_rxd, SOCKET0, &rx_conf, lsi_pktmbuf_pool); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup: err=%d, port=%u\n", ret, portid); + +The list of queues that must be polled for a given lcore is stored in a private structure called struct lcore_queue_conf. + +.. code-block:: c + + struct lcore_queue_conf { + unsigned n_rx_port; + unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE]; unsigned tx_queue_id; + struct mbuf_table tx_mbufs[LSI_MAX_PORTS]; + } rte_cache_aligned; + + struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE]; + +The n_rx_port and rx_port_list[] fields are used in the main packet processing loop +(see `Receive, Process and Transmit Packets`_). + +The global configuration for the RX queues is stored in a static structure: + +.. code-block:: c + + static const struct rte_eth_rxconf rx_conf = { + .rx_thresh = { + .pthresh = RX_PTHRESH, + .hthresh = RX_HTHRESH, + .wthresh = RX_WTHRESH, + }, + }; + +TX Queue Initialization +~~~~~~~~~~~~~~~~~~~~~~~ + +Each lcore should be able to transmit on any port. +For every port, a single TX queue is initialized. + +.. code-block:: c + + /* init one TX queue logical core on each port */ + + fflush(stdout); + + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd, rte_eth_dev_socket_id(portid), &tx_conf); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_eth_tx_queue_setup: err=%d,port=%u\n", ret, (unsigned) portid); + +The global configuration for TX queues is stored in a static structure: + +.. code-block:: c + + static const struct rte_eth_txconf tx_conf = { + .tx_thresh = { + .pthresh = TX_PTHRESH, + .hthresh = TX_HTHRESH, + .wthresh = TX_WTHRESH, + }, + .tx_free_thresh = RTE_TEST_TX_DESC_DEFAULT + 1, /* disable feature */ + }; + +Receive, Process and Transmit Packets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the lsi_main_loop() function, the main task is to read ingress packets from the RX queues. +This is done using the following code: + +.. code-block:: c + + /* + * Read packet from RX queues + */ + + for (i = 0; i < qconf->n_rx_port; i++) { + portid = qconf->rx_port_list[i]; + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0, pkts_burst, MAX_PKT_BURST); + port_statistics[portid].rx += nb_rx; + + for (j = 0; j < nb_rx; j++) { + m = pkts_burst[j]; + rte_prefetch0(rte_pktmbuf_mtod(m, void *)); + lsi_simple_forward(m, portid); + } + } + +Packets are read in a burst of size MAX_PKT_BURST. +The rte_eth_rx_burst() function writes the mbuf pointers in a local table and returns the number of available mbufs in the table. + +Then, each mbuf in the table is processed by the lsi_simple_forward() function. +The processing is very simple: processes the TX port from the RX port and then replaces the source and destination MAC addresses. + +.. note:: + + In the following code, the two lines for calculating the output port require some explanation. + If portId is even, the first line does nothing (as portid & 1 will be 0), and the second line adds 1. + If portId is odd, the first line subtracts one and the second line does nothing. + Therefore, 0 goes to 1, and 1 to 0, 2 goes to 3 and 3 to 2, and so on. + +.. code-block:: c + + static void + lsi_simple_forward(struct rte_mbuf *m, unsigned portid) + { + struct ether_hdr *eth; + void *tmp; + unsigned dst_port = lsi_dst_ports[portid]; + + eth = rte_pktmbuf_mtod(m, struct ether_hdr *); + + /* 02:00:00:00:00:xx */ + + tmp = ð->d_addr.addr_bytes[0]; + + *((uint64_t *)tmp) = 0x000000000002 + (dst_port << 40); + + /* src addr */ + ether_addr_copy(&lsi_ports_eth_addr[dst_port], ð->s_addr); + + lsi_send_packet(m, dst_port); + } + +Then, the packet is sent using the lsi_send_packet(m, dst_port) function. +For this test application, the processing is exactly the same for all packets arriving on the same RX port. +Therefore, it would have been possible to call the lsi_send_burst() function directly from the main loop +to send all the received packets on the same TX port using +the burst-oriented send function, which is more efficient. + +However, in real-life applications (such as, L3 routing), +packet N is not necessarily forwarded on the same port as packet N-1. +The application is implemented to illustrate that so the same approach can be reused in a more complex application. + +The lsi_send_packet() function stores the packet in a per-lcore and per-txport table. +If the table is full, the whole packets table is transmitted using the lsi_send_burst() function: + +.. code-block:: c + + /* Send the packet on an output interface */ + + static int + lsi_send_packet(struct rte_mbuf *m, uint16_t port) + { + unsigned lcore_id, len; + struct lcore_queue_conf *qconf; + + lcore_id = rte_lcore_id(); + qconf = &lcore_queue_conf[lcore_id]; + len = qconf->tx_mbufs[port].len; + qconf->tx_mbufs[port].m_table[len] = m; + len++; + + /* enough pkts to be sent */ + + if (unlikely(len == MAX_PKT_BURST)) { + lsi_send_burst(qconf, MAX_PKT_BURST, port); + len = 0; + } + qconf->tx_mbufs[port].len = len; + + return 0; + } + +To ensure that no packets remain in the tables, each lcore does a draining of the TX queue in its main loop. +This technique introduces some latency when there are not many packets to send. +However, it improves performance: + +.. code-block:: c + + cur_tsc = rte_rdtsc(); + + /* + * TX burst queue drain + */ + + diff_tsc = cur_tsc - prev_tsc; + + if (unlikely(diff_tsc > drain_tsc)) { + /* this could be optimized (use queueid instead of * portid), but it is not called so often */ + + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) { + if (qconf->tx_mbufs[portid].len == 0) + continue; + + lsi_send_burst(&lcore_queue_conf[lcore_id], + qconf->tx_mbufs[portid].len, (uint8_t) portid); + qconf->tx_mbufs[portid].len = 0; + } + + /* if timer is enabled */ + + if (timer_period > 0) { + /* advance the timer */ + + timer_tsc += diff_tsc; + + /* if timer has reached its timeout */ + + if (unlikely(timer_tsc >= (uint64_t) timer_period)) { + /* do this only on master core */ + + if (lcore_id == rte_get_master_lcore()) { + print_stats(); + + /* reset the timer */ + timer_tsc = 0; + } + } + } + prev_tsc = cur_tsc; + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/load_balancer.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/load_balancer.rst new file mode 100644 index 00000000..8f2abdfb --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/load_balancer.rst @@ -0,0 +1,201 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Load Balancer Sample Application +================================ + +The Load Balancer sample application demonstrates the concept of isolating the packet I/O task +from the application-specific workload. +Depending on the performance target, +a number of logical cores (lcores) are dedicated to handle the interaction with the NIC ports (I/O lcores), +while the rest of the lcores are dedicated to performing the application processing (worker lcores). +The worker lcores are totally oblivious to the intricacies of the packet I/O activity and +use the NIC-agnostic interface provided by software rings to exchange packets with the I/O cores. + +Overview +-------- + +The architecture of the Load Balance application is presented in the following figure. + +.. _figure_load_bal_app_arch: + +.. figure:: img/load_bal_app_arch.* + + Load Balancer Application Architecture + + +For the sake of simplicity, the diagram illustrates a specific case of two I/O RX and two I/O TX lcores off loading the packet I/O +overhead incurred by four NIC ports from four worker cores, with each I/O lcore handling RX/TX for two NIC ports. + +I/O RX Logical Cores +~~~~~~~~~~~~~~~~~~~~ + +Each I/O RX lcore performs packet RX from its assigned NIC RX rings and then distributes the received packets to the worker threads. +The application allows each I/O RX lcore to communicate with any of the worker threads, +therefore each (I/O RX lcore, worker lcore) pair is connected through a dedicated single producer - single consumer software ring. + +The worker lcore to handle the current packet is determined by reading a predefined 1-byte field from the input packet: + +worker_id = packet[load_balancing_field] % n_workers + +Since all the packets that are part of the same traffic flow are expected to have the same value for the load balancing field, +this scheme also ensures that all the packets that are part of the same traffic flow are directed to the same worker lcore (flow affinity) +in the same order they enter the system (packet ordering). + +I/O TX Logical Cores +~~~~~~~~~~~~~~~~~~~~ + +Each I/O lcore owns the packet TX for a predefined set of NIC ports. To enable each worker thread to send packets to any NIC TX port, +the application creates a software ring for each (worker lcore, NIC TX port) pair, +with each I/O TX core handling those software rings that are associated with NIC ports that it handles. + +Worker Logical Cores +~~~~~~~~~~~~~~~~~~~~ + +Each worker lcore reads packets from its set of input software rings and +routes them to the NIC ports for transmission by dispatching them to output software rings. +The routing logic is LPM based, with all the worker threads sharing the same LPM rules. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``load_balancer`` sub-directory. + +Running the Application +----------------------- + +To successfully run the application, +the command line used to start the application has to be in sync with the traffic flows configured on the traffic generator side. + +For examples of application command lines and traffic generator flows, please refer to the DPDK Test Report. +For more details on how to set up and run the sample applications provided with DPDK package, +please refer to the *DPDK Getting Started Guide*. + +Explanation +----------- + +Application Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application run-time configuration is done through the application command line parameters. +Any parameter that is not specified as mandatory is optional, +with the default value hard-coded in the main.h header file from the application folder. + +The list of application command line parameters is listed below: + +#. --rx "(PORT, QUEUE, LCORE), ...": The list of NIC RX ports and queues handled by the I/O RX lcores. + This parameter also implicitly defines the list of I/O RX lcores. This is a mandatory parameter. + +#. --tx "(PORT, LCORE), ... ": The list of NIC TX ports handled by the I/O TX lcores. + This parameter also implicitly defines the list of I/O TX lcores. + This is a mandatory parameter. + +#. --w "LCORE, ...": The list of the worker lcores. This is a mandatory parameter. + +#. --lpm "IP / PREFIX => PORT; ...": The list of LPM rules used by the worker lcores for packet forwarding. + This is a mandatory parameter. + +#. --rsz "A, B, C, D": Ring sizes: + + #. A = The size (in number of buffer descriptors) of each of the NIC RX rings read by the I/O RX lcores. + + #. B = The size (in number of elements) of each of the software rings used by the I/O RX lcores to send packets to worker lcores. + + #. C = The size (in number of elements) of each of the software rings used by the worker lcores to send packets to I/O TX lcores. + + #. D = The size (in number of buffer descriptors) of each of the NIC TX rings written by I/O TX lcores. + +#. --bsz "(A, B), (C, D), (E, F)": Burst sizes: + + #. A = The I/O RX lcore read burst size from NIC RX. + + #. B = The I/O RX lcore write burst size to the output software rings. + + #. C = The worker lcore read burst size from the input software rings. + + #. D = The worker lcore write burst size to the output software rings. + + #. E = The I/O TX lcore read burst size from the input software rings. + + #. F = The I/O TX lcore write burst size to the NIC TX. + +#. --pos-lb POS: The position of the 1-byte field within the input packet used by the I/O RX lcores + to identify the worker lcore for the current packet. + This field needs to be within the first 64 bytes of the input packet. + +The infrastructure of software rings connecting I/O lcores and worker lcores is built by the application +as a result of the application configuration provided by the user through the application command line parameters. + +A specific lcore performing the I/O RX role for a specific set of NIC ports can also perform the I/O TX role +for the same or a different set of NIC ports. +A specific lcore cannot perform both the I/O role (either RX or TX) and the worker role during the same session. + +Example: + +.. code-block:: console + + ./load_balancer -l 3-7 -n 4 -- --rx "(0,0,3),(1,0,3)" --tx "(0,3),(1,3)" --w "4,5,6,7" --lpm "1.0.0.0/24=>0; 1.0.1.0/24=>1;" --pos-lb 29 + +There is a single I/O lcore (lcore 3) that handles RX and TX for two NIC ports (ports 0 and 1) that +handles packets to/from four worker lcores (lcores 4, 5, 6 and 7) that +are assigned worker IDs 0 to 3 (worker ID for lcore 4 is 0, for lcore 5 is 1, for lcore 6 is 2 and for lcore 7 is 3). + +Assuming that all the input packets are IPv4 packets with no VLAN label and the source IP address of the current packet is A.B.C.D, +the worker lcore for the current packet is determined by byte D (which is byte 29). +There are two LPM rules that are used by each worker lcore to route packets to the output NIC ports. + +The following table illustrates the packet flow through the system for several possible traffic flows: + ++------------+----------------+-----------------+------------------------------+--------------+ +| **Flow #** | **Source** | **Destination** | **Worker ID (Worker lcore)** | **Output** | +| | **IP Address** | **IP Address** | | **NIC Port** | +| | | | | | ++============+================+=================+==============================+==============+ +| 1 | 0.0.0.0 | 1.0.0.1 | 0 (4) | 0 | +| | | | | | ++------------+----------------+-----------------+------------------------------+--------------+ +| 2 | 0.0.0.1 | 1.0.1.2 | 1 (5) | 1 | +| | | | | | ++------------+----------------+-----------------+------------------------------+--------------+ +| 3 | 0.0.0.14 | 1.0.0.3 | 2 (6) | 0 | +| | | | | | ++------------+----------------+-----------------+------------------------------+--------------+ +| 4 | 0.0.0.15 | 1.0.1.4 | 3 (7) | 1 | +| | | | | | ++------------+----------------+-----------------+------------------------------+--------------+ + +NUMA Support +~~~~~~~~~~~~ + +The application has built-in performance enhancements for the NUMA case: + +#. One buffer pool per each CPU socket. + +#. One LPM table per each CPU socket. + +#. Memory for the NIC RX or TX rings is allocated on the same socket with the lcore handling the respective ring. + +In the case where multiple CPU sockets are used in the system, +it is recommended to enable at least one lcore to fulfill the I/O role for the NIC ports that +are directly attached to that CPU socket through the PCI Express* bus. +It is always recommended to handle the packet I/O with lcores from the same CPU socket as the NICs. + +Depending on whether the I/O RX lcore (same CPU socket as NIC RX), +the worker lcore and the I/O TX lcore (same CPU socket as NIC TX) handling a specific input packet, +are on the same or different CPU sockets, the following run-time scenarios are possible: + +#. AAA: The packet is received, processed and transmitted without going across CPU sockets. + +#. AAB: The packet is received and processed on socket A, + but as it has to be transmitted on a NIC port connected to socket B, + the packet is sent to socket B through software rings. + +#. ABB: The packet is received on socket A, but as it has to be processed by a worker lcore on socket B, + the packet is sent to socket B through software rings. + The packet is transmitted by a NIC port connected to the same CPU socket as the worker lcore that processed it. + +#. ABC: The packet is received on socket A, it is processed by an lcore on socket B, + then it has to be transmitted out by a NIC connected to socket C. + The performance price for crossing the CPU socket boundary is paid twice for this packet. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/multi_process.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/multi_process.rst new file mode 100644 index 00000000..9c374da6 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/multi_process.rst @@ -0,0 +1,323 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +.. _multi_process_app: + +Multi-process Sample Application +================================ + +This chapter describes the example applications for multi-processing that are included in the DPDK. + +Example Applications +-------------------- + +Building the Sample Applications +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The multi-process example applications are built in the same way as other sample applications, +and as documented in the *DPDK Getting Started Guide*. + + +To compile the sample application see :doc:`compiling`. + +The applications are located in the ``multi_process`` sub-directory. + +.. note:: + + If just a specific multi-process application needs to be built, + the final make command can be run just in that application's directory, + rather than at the top-level multi-process directory. + +Basic Multi-process Example +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The examples/simple_mp folder in the DPDK release contains a basic example application to demonstrate how +two DPDK processes can work together using queues and memory pools to share information. + +Running the Application +^^^^^^^^^^^^^^^^^^^^^^^ + +To run the application, start one copy of the simple_mp binary in one terminal, +passing at least two cores in the coremask/corelist, as follows: + +.. code-block:: console + + ./build/simple_mp -l 0-1 -n 4 --proc-type=primary + +For the first DPDK process run, the proc-type flag can be omitted or set to auto, +since all DPDK processes will default to being a primary instance, +meaning they have control over the hugepage shared memory regions. +The process should start successfully and display a command prompt as follows: + +.. code-block:: console + + $ ./build/simple_mp -l 0-1 -n 4 --proc-type=primary + EAL: coremask set to 3 + EAL: Detected lcore 0 on socket 0 + EAL: Detected lcore 1 on socket 0 + EAL: Detected lcore 2 on socket 0 + EAL: Detected lcore 3 on socket 0 + ... + + EAL: Requesting 2 pages of size 1073741824 + EAL: Requesting 768 pages of size 2097152 + EAL: Ask a virtual area of 0x40000000 bytes + EAL: Virtual area found at 0x7ff200000000 (size = 0x40000000) + ... + + EAL: check igb_uio module + EAL: check module finished + EAL: Master core 0 is ready (tid=54e41820) + EAL: Core 1 is ready (tid=53b32700) + + Starting core 1 + + simple_mp > + +To run the secondary process to communicate with the primary process, +again run the same binary setting at least two cores in the coremask/corelist: + +.. code-block:: console + + ./build/simple_mp -l 2-3 -n 4 --proc-type=secondary + +When running a secondary process such as that shown above, the proc-type parameter can again be specified as auto. +However, omitting the parameter altogether will cause the process to try and start as a primary rather than secondary process. + +Once the process type is specified correctly, +the process starts up, displaying largely similar status messages to the primary instance as it initializes. +Once again, you will be presented with a command prompt. + +Once both processes are running, messages can be sent between them using the send command. +At any stage, either process can be terminated using the quit command. + +.. code-block:: console + + EAL: Master core 10 is ready (tid=b5f89820) EAL: Master core 8 is ready (tid=864a3820) + EAL: Core 11 is ready (tid=84ffe700) EAL: Core 9 is ready (tid=85995700) + Starting core 11 Starting core 9 + simple_mp > send hello_secondary simple_mp > core 9: Received 'hello_secondary' + simple_mp > core 11: Received 'hello_primary' simple_mp > send hello_primary + simple_mp > quit simple_mp > quit + +.. note:: + + If the primary instance is terminated, the secondary instance must also be shut-down and restarted after the primary. + This is necessary because the primary instance will clear and reset the shared memory regions on startup, + invalidating the secondary process's pointers. + The secondary process can be stopped and restarted without affecting the primary process. + +How the Application Works +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The core of this example application is based on using two queues and a single memory pool in shared memory. +These three objects are created at startup by the primary process, +since the secondary process cannot create objects in memory as it cannot reserve memory zones, +and the secondary process then uses lookup functions to attach to these objects as it starts up. + +.. code-block:: c + + if (rte_eal_process_type() == RTE_PROC_PRIMARY){ + send_ring = rte_ring_create(_PRI_2_SEC, ring_size, SOCKET0, flags); + recv_ring = rte_ring_create(_SEC_2_PRI, ring_size, SOCKET0, flags); + message_pool = rte_mempool_create(_MSG_POOL, pool_size, string_size, pool_cache, priv_data_sz, NULL, NULL, NULL, NULL, SOCKET0, flags); + } else { + recv_ring = rte_ring_lookup(_PRI_2_SEC); + send_ring = rte_ring_lookup(_SEC_2_PRI); + message_pool = rte_mempool_lookup(_MSG_POOL); + } + +Note, however, that the named ring structure used as send_ring in the primary process is the recv_ring in the secondary process. + +Once the rings and memory pools are all available in both the primary and secondary processes, +the application simply dedicates two threads to sending and receiving messages respectively. +The receive thread simply dequeues any messages on the receive ring, prints them, +and frees the buffer space used by the messages back to the memory pool. +The send thread makes use of the command-prompt library to interactively request user input for messages to send. +Once a send command is issued by the user, a buffer is allocated from the memory pool, filled in with the message contents, +then enqueued on the appropriate rte_ring. + +Symmetric Multi-process Example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The second example of DPDK multi-process support demonstrates how a set of processes can run in parallel, +with each process performing the same set of packet- processing operations. +(Since each process is identical in functionality to the others, +we refer to this as symmetric multi-processing, to differentiate it from asymmetric multi- processing - +such as a client-server mode of operation seen in the next example, +where different processes perform different tasks, yet co-operate to form a packet-processing system.) +The following diagram shows the data-flow through the application, using two processes. + +.. _figure_sym_multi_proc_app: + +.. figure:: img/sym_multi_proc_app.* + + Example Data Flow in a Symmetric Multi-process Application + + +As the diagram shows, each process reads packets from each of the network ports in use. +RSS is used to distribute incoming packets on each port to different hardware RX queues. +Each process reads a different RX queue on each port and so does not contend with any other process for that queue access. +Similarly, each process writes outgoing packets to a different TX queue on each port. + +Running the Application +^^^^^^^^^^^^^^^^^^^^^^^ + +As with the simple_mp example, the first instance of the symmetric_mp process must be run as the primary instance, +though with a number of other application- specific parameters also provided after the EAL arguments. +These additional parameters are: + +* -p <portmask>, where portmask is a hexadecimal bitmask of what ports on the system are to be used. + For example: -p 3 to use ports 0 and 1 only. + +* --num-procs <N>, where N is the total number of symmetric_mp instances that will be run side-by-side to perform packet processing. + This parameter is used to configure the appropriate number of receive queues on each network port. + +* --proc-id <n>, where n is a numeric value in the range 0 <= n < N (number of processes, specified above). + This identifies which symmetric_mp instance is being run, so that each process can read a unique receive queue on each network port. + +The secondary symmetric_mp instances must also have these parameters specified, +and the first two must be the same as those passed to the primary instance, or errors result. + +For example, to run a set of four symmetric_mp instances, running on lcores 1-4, +all performing level-2 forwarding of packets between ports 0 and 1, +the following commands can be used (assuming run as root): + +.. code-block:: console + + # ./build/symmetric_mp -l 1 -n 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=0 + # ./build/symmetric_mp -l 2 -n 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=1 + # ./build/symmetric_mp -l 3 -n 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=2 + # ./build/symmetric_mp -l 4 -n 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=3 + +.. note:: + + In the above example, the process type can be explicitly specified as primary or secondary, rather than auto. + When using auto, the first process run creates all the memory structures needed for all processes - + irrespective of whether it has a proc-id of 0, 1, 2 or 3. + +.. note:: + + For the symmetric multi-process example, since all processes work in the same manner, + once the hugepage shared memory and the network ports are initialized, + it is not necessary to restart all processes if the primary instance dies. + Instead, that process can be restarted as a secondary, + by explicitly setting the proc-type to secondary on the command line. + (All subsequent instances launched will also need this explicitly specified, + as auto-detection will detect no primary processes running and therefore attempt to re-initialize shared memory.) + +How the Application Works +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The initialization calls in both the primary and secondary instances are the same for the most part, +calling the rte_eal_init(), 1 G and 10 G driver initialization and then rte_pci_probe() functions. +Thereafter, the initialization done depends on whether the process is configured as a primary or secondary instance. + +In the primary instance, a memory pool is created for the packet mbufs and the network ports to be used are initialized - +the number of RX and TX queues per port being determined by the num-procs parameter passed on the command-line. +The structures for the initialized network ports are stored in shared memory and +therefore will be accessible by the secondary process as it initializes. + +.. code-block:: c + + if (num_ports & 1) + rte_exit(EXIT_FAILURE, "Application must use an even number of ports\n"); + + for(i = 0; i < num_ports; i++){ + if(proc_type == RTE_PROC_PRIMARY) + if (smp_port_init(ports[i], mp, (uint16_t)num_procs) < 0) + rte_exit(EXIT_FAILURE, "Error initializing ports\n"); + } + +In the secondary instance, rather than initializing the network ports, the port information exported by the primary process is used, +giving the secondary process access to the hardware and software rings for each network port. +Similarly, the memory pool of mbufs is accessed by doing a lookup for it by name: + +.. code-block:: c + + mp = (proc_type == RTE_PROC_SECONDARY) ? rte_mempool_lookup(_SMP_MBUF_POOL) : rte_mempool_create(_SMP_MBUF_POOL, NB_MBUFS, MBUF_SIZE, ... ) + +Once this initialization is complete, the main loop of each process, both primary and secondary, +is exactly the same - each process reads from each port using the queue corresponding to its proc-id parameter, +and writes to the corresponding transmit queue on the output port. + +Client-Server Multi-process Example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The third example multi-process application included with the DPDK shows how one can +use a client-server type multi-process design to do packet processing. +In this example, a single server process performs the packet reception from the ports being used and +distributes these packets using round-robin ordering among a set of client processes, +which perform the actual packet processing. +In this case, the client applications just perform level-2 forwarding of packets by sending each packet out on a different network port. + +The following diagram shows the data-flow through the application, using two client processes. + +.. _figure_client_svr_sym_multi_proc_app: + +.. figure:: img/client_svr_sym_multi_proc_app.* + + Example Data Flow in a Client-Server Symmetric Multi-process Application + + +Running the Application +^^^^^^^^^^^^^^^^^^^^^^^ + +The server process must be run initially as the primary process to set up all memory structures for use by the clients. +In addition to the EAL parameters, the application- specific parameters are: + +* -p <portmask >, where portmask is a hexadecimal bitmask of what ports on the system are to be used. + For example: -p 3 to use ports 0 and 1 only. + +* -n <num-clients>, where the num-clients parameter is the number of client processes that will process the packets received + by the server application. + +.. note:: + + In the server process, a single thread, the master thread, that is, the lowest numbered lcore in the coremask/corelist, performs all packet I/O. + If a coremask/corelist is specified with more than a single lcore bit set in it, + an additional lcore will be used for a thread to periodically print packet count statistics. + +Since the server application stores configuration data in shared memory, including the network ports to be used, +the only application parameter needed by a client process is its client instance ID. +Therefore, to run a server application on lcore 1 (with lcore 2 printing statistics) along with two client processes running on lcores 3 and 4, +the following commands could be used: + +.. code-block:: console + + # ./mp_server/build/mp_server -l 1-2 -n 4 -- -p 3 -n 2 + # ./mp_client/build/mp_client -l 3 -n 4 --proc-type=auto -- -n 0 + # ./mp_client/build/mp_client -l 4 -n 4 --proc-type=auto -- -n 1 + +.. note:: + + If the server application dies and needs to be restarted, all client applications also need to be restarted, + as there is no support in the server application for it to run as a secondary process. + Any client processes that need restarting can be restarted without affecting the server process. + +How the Application Works +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The server process performs the network port and data structure initialization much as the symmetric multi-process application does when run as primary. +One additional enhancement in this sample application is that the server process stores its port configuration data in a memory zone in hugepage shared memory. +This eliminates the need for the client processes to have the portmask parameter passed into them on the command line, +as is done for the symmetric multi-process application, and therefore eliminates mismatched parameters as a potential source of errors. + +In the same way that the server process is designed to be run as a primary process instance only, +the client processes are designed to be run as secondary instances only. +They have no code to attempt to create shared memory objects. +Instead, handles to all needed rings and memory pools are obtained via calls to rte_ring_lookup() and rte_mempool_lookup(). +The network ports for use by the processes are obtained by loading the network port drivers and probing the PCI bus, +which will, as in the symmetric multi-process example, +automatically get access to the network ports using the settings already configured by the primary/server process. + +Once all applications are initialized, the server operates by reading packets from each network port in turn and +distributing those packets to the client queues (software rings, one for each client process) in round-robin order. +On the client side, the packets are read from the rings in as big of bursts as possible, then routed out to a different network port. +The routing used is very simple. All packets received on the first NIC port are transmitted back out on the second port and vice versa. +Similarly, packets are routed between the 3rd and 4th network ports and so on. +The sending of packets is done by writing the packets directly to the network ports; they are not transferred back via the server process. + +In both the server and the client processes, outgoing packets are buffered before being sent, +so as to allow the sending of multiple packets in a single burst to improve efficiency. +For example, the client process will buffer packets to send, +until either the buffer is full or until we receive no further packets from the server. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/netmap_compatibility.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/netmap_compatibility.rst new file mode 100644 index 00000000..33b1cc21 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/netmap_compatibility.rst @@ -0,0 +1,130 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Netmap Compatibility Sample Application +======================================= + +Introduction +------------ + +The Netmap compatibility library provides a minimal set of APIs to give programs written against the Netmap APIs +the ability to be run, with minimal changes to their source code, using the DPDK to perform the actual packet I/O. + +Since Netmap applications use regular system calls, like ``open()``, ``ioctl()`` and +``mmap()`` to communicate with the Netmap kernel module performing the packet I/O, +the ``compat_netmap`` library provides a set of similar APIs to use in place of those system calls, +effectively turning a Netmap application into a DPDK application. + +The provided library is currently minimal and doesn't support all the features that Netmap supports, +but is enough to run simple applications, such as the bridge example detailed below. + +Knowledge of Netmap is required to understand the rest of this section. +Please refer to the Netmap distribution for details about Netmap. + +Available APIs +-------------- + +The library provides the following drop-in replacements for system calls usually used in Netmap applications: + +* ``rte_netmap_close()`` + +* ``rte_netmap_ioctl()`` + +* ``rte_netmap_open()`` + +* ``rte_netmap_mmap()`` + +* ``rte_netmap_poll()`` + +They use the same signature as their libc counterparts, and can be used as drop-in replacements in most cases. + +Caveats +------- + +Given the difference between the way Netmap and the DPDK approach packet I/O, +there are caveats and limitations to be aware of when trying to use the ``compat_netmap`` library, the most important of these are listed below. +These may change as the library is updated: + +* Any system call that can potentially affect file descriptors cannot be used with a descriptor returned by the ``rte_netmap_open()`` function. + +Note that: + +* The ``rte_netmap_mmap()`` function merely returns the address of a DPDK memzone. + The address, length, flags, offset, and other arguments are ignored. + +* The ``rte_netmap_poll()`` function only supports infinite (negative) or zero time outs. + It effectively turns calls to the ``poll()`` system call made in a Netmap application into polling of the DPDK ports, + changing the semantics of the usual POSIX defined poll. + +* Not all of Netmap's features are supported: host rings, + slot flags and so on are not supported or are simply not relevant in the DPDK model. + +* The Netmap manual page states that "*a device obtained through /dev/netmap also supports the ioctl supported by network devices*". + This is not the case with this compatibility layer. + +* The Netmap kernel module exposes a sysfs interface to change some internal parameters, such as the size of the shared memory region. + This interface is not available when using this compatibility layer. + +Porting Netmap Applications +--------------------------- + +Porting Netmap applications typically involves two major steps: + +* Changing the system calls to use their ``compat_netmap`` library counterparts. + +* Adding further DPDK initialization code. + +Since the ``compat_netmap`` functions have the same signature as the usual libc calls, the change is trivial in most cases. + +The usual DPDK initialization code involving ``rte_eal_init()`` and ``rte_pci_probe()`` +has to be added to the Netmap application in the same way it is used in all other DPDK sample applications. +Please refer to the *DPDK Programmer's Guide* and example source code for details about initialization. + +In addition of the regular DPDK initialization code, +the ported application needs to call initialization functions for the ``compat_netmap`` library, +namely ``rte_netmap_init()`` and ``rte_netmap_init_port()``. + +These two initialization functions take ``compat_netmap`` specific data structures as parameters: +``struct rte_netmap_conf`` and ``struct rte_netmap_port_conf``. +The structures' fields are Netmap related and are self-explanatory for developers familiar with Netmap. +They are defined in ``$RTE_SDK/examples/netmap_compat/lib/compat_netmap.h``. + +The bridge application is an example largely based on the bridge example shipped with the Netmap distribution. +It shows how a minimal Netmap application with minimal and straightforward source code changes can be run on top of the DPDK. +Please refer to ``$RTE_SDK/examples/netmap_compat/bridge/bridge.c`` for an example of a ported application. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``netmap_compat`` sub-directory. + +Running the "bridge" Sample Application +--------------------------------------- + +The application requires a single command line option: + +.. code-block:: console + + ./build/bridge [EAL options] -- -i INTERFACE_A [-i INTERFACE_B] + +where, + +* ``-i INTERFACE``: Interface (DPDK port number) to use. + + If a single ``-i`` parameter is given, the interface will send back all the traffic it receives. + If two ``-i`` parameters are given, the two interfaces form a bridge, + where traffic received on one interface is replicated and sent to the other interface. + +For example, to run the application in a linuxapp environment using port 0 and 2: + +.. code-block:: console + + ./build/bridge [EAL options] -- -i 0 -i 2 + +Refer to the *DPDK Getting Started Guide for Linux* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +Note that unlike a traditional bridge or the ``l2fwd`` sample application, no MAC address changes are done on the frames. +Do not forget to take this into account when configuring a traffic generators and testing this sample application. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/packet_ordering.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/packet_ordering.rst new file mode 100644 index 00000000..7cfcf3f3 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/packet_ordering.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +Packet Ordering Application +============================ + +The Packet Ordering sample app simply shows the impact of reordering a stream. +It's meant to stress the library with different configurations for performance. + +Overview +-------- + +The application uses at least three CPU cores: + +* RX core (maser core) receives traffic from the NIC ports and feeds Worker + cores with traffic through SW queues. + +* Worker core (slave core) basically do some light work on the packet. + Currently it modifies the output port of the packet for configurations with + more than one port enabled. + +* TX Core (slave core) receives traffic from Worker cores through software queues, + inserts out-of-order packets into reorder buffer, extracts ordered packets + from the reorder buffer and sends them to the NIC ports for transmission. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``packet_ordering`` sub-directory. + +Running the Application +----------------------- + +Refer to *DPDK Getting Started Guide* for general information on running applications +and the Environment Abstraction Layer (EAL) options. + +Application Command Line +~~~~~~~~~~~~~~~~~~~~~~~~ + +The application execution command line is: + +.. code-block:: console + + ./test-pipeline [EAL options] -- -p PORTMASK [--disable-reorder] + +The -c EAL CPU_COREMASK option has to contain at least 3 CPU cores. +The first CPU core in the core mask is the master core and would be assigned to +RX core, the last to TX core and the rest to Worker cores. + +The PORTMASK parameter must contain either 1 or even enabled port numbers. +When setting more than 1 port, traffic would be forwarded in pairs. +For example, if we enable 4 ports, traffic from port 0 to 1 and from 1 to 0, +then the other pair from 2 to 3 and from 3 to 2, having [0,1] and [2,3] pairs. + +The disable-reorder long option does, as its name implies, disable the reordering +of traffic, which should help evaluate reordering performance impact. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/performance_thread.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/performance_thread.rst new file mode 100644 index 00000000..fbd30a5e --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/performance_thread.rst @@ -0,0 +1,1221 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +Performance Thread Sample Application +===================================== + +The performance thread sample application is a derivative of the standard L3 +forwarding application that demonstrates different threading models. + +Overview +-------- +For a general description of the L3 forwarding applications capabilities +please refer to the documentation of the standard application in +:doc:`l3_forward`. + +The performance thread sample application differs from the standard L3 +forwarding example in that it divides the TX and RX processing between +different threads, and makes it possible to assign individual threads to +different cores. + +Three threading models are considered: + +#. When there is one EAL thread per physical core. +#. When there are multiple EAL threads per physical core. +#. When there are multiple lightweight threads per EAL thread. + +Since DPDK release 2.0 it is possible to launch applications using the +``--lcores`` EAL parameter, specifying cpu-sets for a physical core. With the +performance thread sample application its is now also possible to assign +individual RX and TX functions to different cores. + +As an alternative to dividing the L3 forwarding work between different EAL +threads the performance thread sample introduces the possibility to run the +application threads as lightweight threads (L-threads) within one or +more EAL threads. + +In order to facilitate this threading model the example includes a primitive +cooperative scheduler (L-thread) subsystem. More details of the L-thread +subsystem can be found in :ref:`lthread_subsystem`. + +**Note:** Whilst theoretically possible it is not anticipated that multiple +L-thread schedulers would be run on the same physical core, this mode of +operation should not be expected to yield useful performance and is considered +invalid. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the `performance-thread/l3fwd-thread` sub-directory. + +Running the Application +----------------------- + +The application has a number of command line options:: + + ./build/l3fwd-thread [EAL options] -- + -p PORTMASK [-P] + --rx(port,queue,lcore,thread)[,(port,queue,lcore,thread)] + --tx(lcore,thread)[,(lcore,thread)] + [--enable-jumbo] [--max-pkt-len PKTLEN]] [--no-numa] + [--hash-entry-num] [--ipv6] [--no-lthreads] [--stat-lcore lcore] + [--parse-ptype] + +Where: + +* ``-p PORTMASK``: Hexadecimal bitmask of ports to configure. + +* ``-P``: optional, sets all ports to promiscuous mode so that packets are + accepted regardless of the packet's Ethernet MAC destination address. + Without this option, only packets with the Ethernet MAC destination address + set to the Ethernet address of the port are accepted. + +* ``--rx (port,queue,lcore,thread)[,(port,queue,lcore,thread)]``: the list of + NIC RX ports and queues handled by the RX lcores and threads. The parameters + are explained below. + +* ``--tx (lcore,thread)[,(lcore,thread)]``: the list of TX threads identifying + the lcore the thread runs on, and the id of RX thread with which it is + associated. The parameters are explained below. + +* ``--enable-jumbo``: optional, enables jumbo frames. + +* ``--max-pkt-len``: optional, maximum packet length in decimal (64-9600). + +* ``--no-numa``: optional, disables numa awareness. + +* ``--hash-entry-num``: optional, specifies the hash entry number in hex to be + setup. + +* ``--ipv6``: optional, set it if running ipv6 packets. + +* ``--no-lthreads``: optional, disables l-thread model and uses EAL threading + model. See below. + +* ``--stat-lcore``: optional, run CPU load stats collector on the specified + lcore. + +* ``--parse-ptype:`` optional, set to use software to analyze packet type. + Without this option, hardware will check the packet type. + +The parameters of the ``--rx`` and ``--tx`` options are: + +* ``--rx`` parameters + + .. _table_l3fwd_rx_parameters: + + +--------+------------------------------------------------------+ + | port | RX port | + +--------+------------------------------------------------------+ + | queue | RX queue that will be read on the specified RX port | + +--------+------------------------------------------------------+ + | lcore | Core to use for the thread | + +--------+------------------------------------------------------+ + | thread | Thread id (continuously from 0 to N) | + +--------+------------------------------------------------------+ + + +* ``--tx`` parameters + + .. _table_l3fwd_tx_parameters: + + +--------+------------------------------------------------------+ + | lcore | Core to use for L3 route match and transmit | + +--------+------------------------------------------------------+ + | thread | Id of RX thread to be associated with this TX thread | + +--------+------------------------------------------------------+ + +The ``l3fwd-thread`` application allows you to start packet processing in two +threading models: L-Threads (default) and EAL Threads (when the +``--no-lthreads`` parameter is used). For consistency all parameters are used +in the same way for both models. + + +Running with L-threads +~~~~~~~~~~~~~~~~~~~~~~ + +When the L-thread model is used (default option), lcore and thread parameters +in ``--rx/--tx`` are used to affinitize threads to the selected scheduler. + +For example, the following places every l-thread on different lcores:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" + +The following places RX l-threads on lcore 0 and TX l-threads on lcore 1 and 2 +and so on:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,0,1)" \ + --tx="(1,0)(2,1)" + + +Running with EAL threads +~~~~~~~~~~~~~~~~~~~~~~~~ + +When the ``--no-lthreads`` parameter is used, the L-threading model is turned +off and EAL threads are used for all processing. EAL threads are enumerated in +the same way as L-threads, but the ``--lcores`` EAL parameter is used to +affinitize threads to the selected cpu-set (scheduler). Thus it is possible to +place every RX and TX thread on different lcores. + +For example, the following places every EAL thread on different lcores:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" \ + --no-lthreads + + +To affinitize two or more EAL threads to one cpu-set, the EAL ``--lcores`` +parameter is used. + +The following places RX EAL threads on lcore 0 and TX EAL threads on lcore 1 +and 2 and so on:: + + l3fwd-thread -l 0-7 -n 2 --lcores="(0,1)@0,(2,3)@1" -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" \ + --no-lthreads + + +Examples +~~~~~~~~ + +For selected scenarios the command line configuration of the application for L-threads +and its corresponding EAL threads command line can be realized as follows: + +a) Start every thread on different scheduler (1:1):: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" + + EAL thread equivalent:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" \ + --no-lthreads + +b) Start all threads on one core (N:1). + + Start 4 L-threads on lcore 0:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,0,1)" \ + --tx="(0,0)(0,1)" + + Start 4 EAL threads on cpu-set 0:: + + l3fwd-thread -l 0-7 -n 2 --lcores="(0-3)@0" -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,0,1)" \ + --tx="(2,0)(3,1)" \ + --no-lthreads + +c) Start threads on different cores (N:M). + + Start 2 L-threads for RX on lcore 0, and 2 L-threads for TX on lcore 1:: + + l3fwd-thread -l 0-7 -n 2 -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,0,1)" \ + --tx="(1,0)(1,1)" + + Start 2 EAL threads for RX on cpu-set 0, and 2 EAL threads for TX on + cpu-set 1:: + + l3fwd-thread -l 0-7 -n 2 --lcores="(0-1)@0,(2-3)@1" -- -P -p 3 \ + --rx="(0,0,0,0)(1,0,1,1)" \ + --tx="(2,0)(3,1)" \ + --no-lthreads + +Explanation +----------- + +To a great extent the sample application differs little from the standard L3 +forwarding application, and readers are advised to familiarize themselves with +the material covered in the :doc:`l3_forward` documentation before proceeding. + +The following explanation is focused on the way threading is handled in the +performance thread example. + + +Mode of operation with EAL threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The performance thread sample application has split the RX and TX functionality +into two different threads, and the RX and TX threads are +interconnected via software rings. With respect to these rings the RX threads +are producers and the TX threads are consumers. + +On initialization the TX and RX threads are started according to the command +line parameters. + +The RX threads poll the network interface queues and post received packets to a +TX thread via a corresponding software ring. + +The TX threads poll software rings, perform the L3 forwarding hash/LPM match, +and assemble packet bursts before performing burst transmit on the network +interface. + +As with the standard L3 forward application, burst draining of residual packets +is performed periodically with the period calculated from elapsed time using +the timestamps counter. + +The diagram below illustrates a case with two RX threads and three TX threads. + +.. _figure_performance_thread_1: + +.. figure:: img/performance_thread_1.* + + +Mode of operation with L-threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Like the EAL thread configuration the application has split the RX and TX +functionality into different threads, and the pairs of RX and TX threads are +interconnected via software rings. + +On initialization an L-thread scheduler is started on every EAL thread. On all +but the master EAL thread only a a dummy L-thread is initially started. +The L-thread started on the master EAL thread then spawns other L-threads on +different L-thread schedulers according the command line parameters. + +The RX threads poll the network interface queues and post received packets +to a TX thread via the corresponding software ring. + +The ring interface is augmented by means of an L-thread condition variable that +enables the TX thread to be suspended when the TX ring is empty. The RX thread +signals the condition whenever it posts to the TX ring, causing the TX thread +to be resumed. + +Additionally the TX L-thread spawns a worker L-thread to take care of +polling the software rings, whilst it handles burst draining of the transmit +buffer. + +The worker threads poll the software rings, perform L3 route lookup and +assemble packet bursts. If the TX ring is empty the worker thread suspends +itself by waiting on the condition variable associated with the ring. + +Burst draining of residual packets, less than the burst size, is performed by +the TX thread which sleeps (using an L-thread sleep function) and resumes +periodically to flush the TX buffer. + +This design means that L-threads that have no work, can yield the CPU to other +L-threads and avoid having to constantly poll the software rings. + +The diagram below illustrates a case with two RX threads and three TX functions +(each comprising a thread that processes forwarding and a thread that +periodically drains the output buffer of residual packets). + +.. _figure_performance_thread_2: + +.. figure:: img/performance_thread_2.* + + +CPU load statistics +~~~~~~~~~~~~~~~~~~~ + +It is possible to display statistics showing estimated CPU load on each core. +The statistics indicate the percentage of CPU time spent: processing +received packets (forwarding), polling queues/rings (waiting for work), +and doing any other processing (context switch and other overhead). + +When enabled statistics are gathered by having the application threads set and +clear flags when they enter and exit pertinent code sections. The flags are +then sampled in real time by a statistics collector thread running on another +core. This thread displays the data in real time on the console. + +This feature is enabled by designating a statistics collector core, using the +``--stat-lcore`` parameter. + + +.. _lthread_subsystem: + +The L-thread subsystem +---------------------- + +The L-thread subsystem resides in the examples/performance-thread/common +directory and is built and linked automatically when building the +``l3fwd-thread`` example. + +The subsystem provides a simple cooperative scheduler to enable arbitrary +functions to run as cooperative threads within a single EAL thread. +The subsystem provides a pthread like API that is intended to assist in +reuse of legacy code written for POSIX pthreads. + +The following sections provide some detail on the features, constraints, +performance and porting considerations when using L-threads. + + +.. _comparison_between_lthreads_and_pthreads: + +Comparison between L-threads and POSIX pthreads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The fundamental difference between the L-thread and pthread models is the +way in which threads are scheduled. The simplest way to think about this is to +consider the case of a processor with a single CPU. To run multiple threads +on a single CPU, the scheduler must frequently switch between the threads, +in order that each thread is able to make timely progress. +This is the basis of any multitasking operating system. + +This section explores the differences between the pthread model and the +L-thread model as implemented in the provided L-thread subsystem. If needed a +theoretical discussion of preemptive vs cooperative multi-threading can be +found in any good text on operating system design. + + +Scheduling and context switching +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The POSIX pthread library provides an application programming interface to +create and synchronize threads. Scheduling policy is determined by the host OS, +and may be configurable. The OS may use sophisticated rules to determine which +thread should be run next, threads may suspend themselves or make other threads +ready, and the scheduler may employ a time slice giving each thread a maximum +time quantum after which it will be preempted in favor of another thread that +is ready to run. To complicate matters further threads may be assigned +different scheduling priorities. + +By contrast the L-thread subsystem is considerably simpler. Logically the +L-thread scheduler performs the same multiplexing function for L-threads +within a single pthread as the OS scheduler does for pthreads within an +application process. The L-thread scheduler is simply the main loop of a +pthread, and in so far as the host OS is concerned it is a regular pthread +just like any other. The host OS is oblivious about the existence of and +not at all involved in the scheduling of L-threads. + +The other and most significant difference between the two models is that +L-threads are scheduled cooperatively. L-threads cannot not preempt each +other, nor can the L-thread scheduler preempt a running L-thread (i.e. +there is no time slicing). The consequence is that programs implemented with +L-threads must possess frequent rescheduling points, meaning that they must +explicitly and of their own volition return to the scheduler at frequent +intervals, in order to allow other L-threads an opportunity to proceed. + +In both models switching between threads requires that the current CPU +context is saved and a new context (belonging to the next thread ready to run) +is restored. With pthreads this context switching is handled transparently +and the set of CPU registers that must be preserved between context switches +is as per an interrupt handler. + +An L-thread context switch is achieved by the thread itself making a function +call to the L-thread scheduler. Thus it is only necessary to preserve the +callee registers. The caller is responsible to save and restore any other +registers it is using before a function call, and restore them on return, +and this is handled by the compiler. For ``X86_64`` on both Linux and BSD the +System V calling convention is used, this defines registers RSP, RBP, and +R12-R15 as callee-save registers (for more detailed discussion a good reference +is `X86 Calling Conventions <https://en.wikipedia.org/wiki/X86_calling_conventions>`_). + +Taking advantage of this, and due to the absence of preemption, an L-thread +context switch is achieved with less than 20 load/store instructions. + +The scheduling policy for L-threads is fixed, there is no prioritization of +L-threads, all L-threads are equal and scheduling is based on a FIFO +ready queue. + +An L-thread is a struct containing the CPU context of the thread +(saved on context switch) and other useful items. The ready queue contains +pointers to threads that are ready to run. The L-thread scheduler is a simple +loop that polls the ready queue, reads from it the next thread ready to run, +which it resumes by saving the current context (the current position in the +scheduler loop) and restoring the context of the next thread from its thread +struct. Thus an L-thread is always resumed at the last place it yielded. + +A well behaved L-thread will call the context switch regularly (at least once +in its main loop) thus returning to the scheduler's own main loop. Yielding +inserts the current thread at the back of the ready queue, and the process of +servicing the ready queue is repeated, thus the system runs by flipping back +and forth the between L-threads and scheduler loop. + +In the case of pthreads, the preemptive scheduling, time slicing, and support +for thread prioritization means that progress is normally possible for any +thread that is ready to run. This comes at the price of a relatively heavier +context switch and scheduling overhead. + +With L-threads the progress of any particular thread is determined by the +frequency of rescheduling opportunities in the other L-threads. This means that +an errant L-thread monopolizing the CPU might cause scheduling of other threads +to be stalled. Due to the lower cost of context switching, however, voluntary +rescheduling to ensure progress of other threads, if managed sensibly, is not +a prohibitive overhead, and overall performance can exceed that of an +application using pthreads. + + +Mutual exclusion +^^^^^^^^^^^^^^^^ + +With pthreads preemption means that threads that share data must observe +some form of mutual exclusion protocol. + +The fact that L-threads cannot preempt each other means that in many cases +mutual exclusion devices can be completely avoided. + +Locking to protect shared data can be a significant bottleneck in +multi-threaded applications so a carefully designed cooperatively scheduled +program can enjoy significant performance advantages. + +So far we have considered only the simplistic case of a single core CPU, +when multiple CPUs are considered things are somewhat more complex. + +First of all it is inevitable that there must be multiple L-thread schedulers, +one running on each EAL thread. So long as these schedulers remain isolated +from each other the above assertions about the potential advantages of +cooperative scheduling hold true. + +A configuration with isolated cooperative schedulers is less flexible than the +pthread model where threads can be affinitized to run on any CPU. With isolated +schedulers scaling of applications to utilize fewer or more CPUs according to +system demand is very difficult to achieve. + +The L-thread subsystem makes it possible for L-threads to migrate between +schedulers running on different CPUs. Needless to say if the migration means +that threads that share data end up running on different CPUs then this will +introduce the need for some kind of mutual exclusion system. + +Of course ``rte_ring`` software rings can always be used to interconnect +threads running on different cores, however to protect other kinds of shared +data structures, lock free constructs or else explicit locking will be +required. This is a consideration for the application design. + +In support of this extended functionality, the L-thread subsystem implements +thread safe mutexes and condition variables. + +The cost of affinitizing and of condition variable signaling is significantly +lower than the equivalent pthread operations, and so applications using these +features will see a performance benefit. + + +Thread local storage +^^^^^^^^^^^^^^^^^^^^ + +As with applications written for pthreads an application written for L-threads +can take advantage of thread local storage, in this case local to an L-thread. +An application may save and retrieve a single pointer to application data in +the L-thread struct. + +For legacy and backward compatibility reasons two alternative methods are also +offered, the first is modelled directly on the pthread get/set specific APIs, +the second approach is modelled on the ``RTE_PER_LCORE`` macros, whereby +``PER_LTHREAD`` macros are introduced, in both cases the storage is local to +the L-thread. + + +.. _constraints_and_performance_implications: + +Constraints and performance implications when using L-threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +.. _API_compatibility: + +API compatibility +^^^^^^^^^^^^^^^^^ + +The L-thread subsystem provides a set of functions that are logically equivalent +to the corresponding functions offered by the POSIX pthread library, however not +all pthread functions have a corresponding L-thread equivalent, and not all +features available to pthreads are implemented for L-threads. + +The pthread library offers considerable flexibility via programmable attributes +that can be associated with threads, mutexes, and condition variables. + +By contrast the L-thread subsystem has fixed functionality, the scheduler policy +cannot be varied, and L-threads cannot be prioritized. There are no variable +attributes associated with any L-thread objects. L-threads, mutexes and +conditional variables, all have fixed functionality. (Note: reserved parameters +are included in the APIs to facilitate possible future support for attributes). + +The table below lists the pthread and equivalent L-thread APIs with notes on +differences and/or constraints. Where there is no L-thread entry in the table, +then the L-thread subsystem provides no equivalent function. + +.. _table_lthread_pthread: + +.. table:: Pthread and equivalent L-thread APIs. + + +----------------------------+------------------------+-------------------+ + | **Pthread function** | **L-thread function** | **Notes** | + +============================+========================+===================+ + | pthread_barrier_destroy | | | + +----------------------------+------------------------+-------------------+ + | pthread_barrier_init | | | + +----------------------------+------------------------+-------------------+ + | pthread_barrier_wait | | | + +----------------------------+------------------------+-------------------+ + | pthread_cond_broadcast | lthread_cond_broadcast | See note 1 | + +----------------------------+------------------------+-------------------+ + | pthread_cond_destroy | lthread_cond_destroy | | + +----------------------------+------------------------+-------------------+ + | pthread_cond_init | lthread_cond_init | | + +----------------------------+------------------------+-------------------+ + | pthread_cond_signal | lthread_cond_signal | See note 1 | + +----------------------------+------------------------+-------------------+ + | pthread_cond_timedwait | | | + +----------------------------+------------------------+-------------------+ + | pthread_cond_wait | lthread_cond_wait | See note 5 | + +----------------------------+------------------------+-------------------+ + | pthread_create | lthread_create | See notes 2, 3 | + +----------------------------+------------------------+-------------------+ + | pthread_detach | lthread_detach | See note 4 | + +----------------------------+------------------------+-------------------+ + | pthread_equal | | | + +----------------------------+------------------------+-------------------+ + | pthread_exit | lthread_exit | | + +----------------------------+------------------------+-------------------+ + | pthread_getspecific | lthread_getspecific | | + +----------------------------+------------------------+-------------------+ + | pthread_getcpuclockid | | | + +----------------------------+------------------------+-------------------+ + | pthread_join | lthread_join | | + +----------------------------+------------------------+-------------------+ + | pthread_key_create | lthread_key_create | | + +----------------------------+------------------------+-------------------+ + | pthread_key_delete | lthread_key_delete | | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_destroy | lthread_mutex_destroy | | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_init | lthread_mutex_init | | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_lock | lthread_mutex_lock | See note 6 | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_trylock | lthread_mutex_trylock | See note 6 | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_timedlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_mutex_unlock | lthread_mutex_unlock | | + +----------------------------+------------------------+-------------------+ + | pthread_once | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_destroy | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_init | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_rdlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_timedrdlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_timedwrlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_tryrdlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_trywrlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_unlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_rwlock_wrlock | | | + +----------------------------+------------------------+-------------------+ + | pthread_self | lthread_current | | + +----------------------------+------------------------+-------------------+ + | pthread_setspecific | lthread_setspecific | | + +----------------------------+------------------------+-------------------+ + | pthread_spin_init | | See note 10 | + +----------------------------+------------------------+-------------------+ + | pthread_spin_destroy | | See note 10 | + +----------------------------+------------------------+-------------------+ + | pthread_spin_lock | | See note 10 | + +----------------------------+------------------------+-------------------+ + | pthread_spin_trylock | | See note 10 | + +----------------------------+------------------------+-------------------+ + | pthread_spin_unlock | | See note 10 | + +----------------------------+------------------------+-------------------+ + | pthread_cancel | lthread_cancel | | + +----------------------------+------------------------+-------------------+ + | pthread_setcancelstate | | | + +----------------------------+------------------------+-------------------+ + | pthread_setcanceltype | | | + +----------------------------+------------------------+-------------------+ + | pthread_testcancel | | | + +----------------------------+------------------------+-------------------+ + | pthread_getschedparam | | | + +----------------------------+------------------------+-------------------+ + | pthread_setschedparam | | | + +----------------------------+------------------------+-------------------+ + | pthread_yield | lthread_yield | See note 7 | + +----------------------------+------------------------+-------------------+ + | pthread_setaffinity_np | lthread_set_affinity | See notes 2, 3, 8 | + +----------------------------+------------------------+-------------------+ + | | lthread_sleep | See note 9 | + +----------------------------+------------------------+-------------------+ + | | lthread_sleep_clks | See note 9 | + +----------------------------+------------------------+-------------------+ + + +**Note 1**: + +Neither lthread signal nor broadcast may be called concurrently by L-threads +running on different schedulers, although multiple L-threads running in the +same scheduler may freely perform signal or broadcast operations. L-threads +running on the same or different schedulers may always safely wait on a +condition variable. + + +**Note 2**: + +Pthread attributes may be used to affinitize a pthread with a cpu-set. The +L-thread subsystem does not support a cpu-set. An L-thread may be affinitized +only with a single CPU at any time. + + +**Note 3**: + +If an L-thread is intended to run on a different NUMA node than the node that +creates the thread then, when calling ``lthread_create()`` it is advantageous +to specify the destination core as a parameter of ``lthread_create()``. See +:ref:`memory_allocation_and_NUMA_awareness` for details. + + +**Note 4**: + +An L-thread can only detach itself, and cannot detach other L-threads. + + +**Note 5**: + +A wait operation on a pthread condition variable is always associated with and +protected by a mutex which must be owned by the thread at the time it invokes +``pthread_wait()``. By contrast L-thread condition variables are thread safe +(for waiters) and do not use an associated mutex. Multiple L-threads (including +L-threads running on other schedulers) can safely wait on a L-thread condition +variable. As a consequence the performance of an L-thread condition variables +is typically an order of magnitude faster than its pthread counterpart. + + +**Note 6**: + +Recursive locking is not supported with L-threads, attempts to take a lock +recursively will be detected and rejected. + + +**Note 7**: + +``lthread_yield()`` will save the current context, insert the current thread +to the back of the ready queue, and resume the next ready thread. Yielding +increases ready queue backlog, see :ref:`ready_queue_backlog` for more details +about the implications of this. + + +N.B. The context switch time as measured from immediately before the call to +``lthread_yield()`` to the point at which the next ready thread is resumed, +can be an order of magnitude faster that the same measurement for +pthread_yield. + + +**Note 8**: + +``lthread_set_affinity()`` is similar to a yield apart from the fact that the +yielding thread is inserted into a peer ready queue of another scheduler. +The peer ready queue is actually a separate thread safe queue, which means that +threads appearing in the peer ready queue can jump any backlog in the local +ready queue on the destination scheduler. + +The context switch time as measured from the time just before the call to +``lthread_set_affinity()`` to just after the same thread is resumed on the new +scheduler can be orders of magnitude faster than the same measurement for +``pthread_setaffinity_np()``. + + +**Note 9**: + +Although there is no ``pthread_sleep()`` function, ``lthread_sleep()`` and +``lthread_sleep_clks()`` can be used wherever ``sleep()``, ``usleep()`` or +``nanosleep()`` might ordinarily be used. The L-thread sleep functions suspend +the current thread, start an ``rte_timer`` and resume the thread when the +timer matures. The ``rte_timer_manage()`` entry point is called on every pass +of the scheduler loop. This means that the worst case jitter on timer expiry +is determined by the longest period between context switches of any running +L-threads. + +In a synthetic test with many threads sleeping and resuming then the measured +jitter is typically orders of magnitude lower than the same measurement made +for ``nanosleep()``. + + +**Note 10**: + +Spin locks are not provided because they are problematical in a cooperative +environment, see :ref:`porting_locks_and_spinlocks` for a more detailed +discussion on how to avoid spin locks. + + +.. _Thread_local_storage_performance: + +Thread local storage +^^^^^^^^^^^^^^^^^^^^ + +Of the three L-thread local storage options the simplest and most efficient is +storing a single application data pointer in the L-thread struct. + +The ``PER_LTHREAD`` macros involve a run time computation to obtain the address +of the variable being saved/retrieved and also require that the accesses are +de-referenced via a pointer. This means that code that has used +``RTE_PER_LCORE`` macros being ported to L-threads might need some slight +adjustment (see :ref:`porting_thread_local_storage` for hints about porting +code that makes use of thread local storage). + +The get/set specific APIs are consistent with their pthread counterparts both +in use and in performance. + + +.. _memory_allocation_and_NUMA_awareness: + +Memory allocation and NUMA awareness +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All memory allocation is from DPDK huge pages, and is NUMA aware. Each +scheduler maintains its own caches of objects: lthreads, their stacks, TLS, +mutexes and condition variables. These caches are implemented as unbounded lock +free MPSC queues. When objects are created they are always allocated from the +caches on the local core (current EAL thread). + +If an L-thread has been affinitized to a different scheduler, then it can +always safely free resources to the caches from which they originated (because +the caches are MPSC queues). + +If the L-thread has been affinitized to a different NUMA node then the memory +resources associated with it may incur longer access latency. + +The commonly used pattern of setting affinity on entry to a thread after it has +started, means that memory allocation for both the stack and TLS will have been +made from caches on the NUMA node on which the threads creator is running. +This has the side effect that access latency will be sub-optimal after +affinitizing. + +This side effect can be mitigated to some extent (although not completely) by +specifying the destination CPU as a parameter of ``lthread_create()`` this +causes the L-thread's stack and TLS to be allocated when it is first scheduled +on the destination scheduler, if the destination is a on another NUMA node it +results in a more optimal memory allocation. + +Note that the lthread struct itself remains allocated from memory on the +creating node, this is unavoidable because an L-thread is known everywhere by +the address of this struct. + + +.. _object_cache_sizing: + +Object cache sizing +^^^^^^^^^^^^^^^^^^^ + +The per lcore object caches pre-allocate objects in bulk whenever a request to +allocate an object finds a cache empty. By default 100 objects are +pre-allocated, this is defined by ``LTHREAD_PREALLOC`` in the public API +header file lthread_api.h. This means that the caches constantly grow to meet +system demand. + +In the present implementation there is no mechanism to reduce the cache sizes +if system demand reduces. Thus the caches will remain at their maximum extent +indefinitely. + +A consequence of the bulk pre-allocation of objects is that every 100 (default +value) additional new object create operations results in a call to +``rte_malloc()``. For creation of objects such as L-threads, which trigger the +allocation of even more objects (i.e. their stacks and TLS) then this can +cause outliers in scheduling performance. + +If this is a problem the simplest mitigation strategy is to dimension the +system, by setting the bulk object pre-allocation size to some large number +that you do not expect to be exceeded. This means the caches will be populated +once only, the very first time a thread is created. + + +.. _Ready_queue_backlog: + +Ready queue backlog +^^^^^^^^^^^^^^^^^^^ + +One of the more subtle performance considerations is managing the ready queue +backlog. The fewer threads that are waiting in the ready queue then the faster +any particular thread will get serviced. + +In a naive L-thread application with N L-threads simply looping and yielding, +this backlog will always be equal to the number of L-threads, thus the cost of +a yield to a particular L-thread will be N times the context switch time. + +This side effect can be mitigated by arranging for threads to be suspended and +wait to be resumed, rather than polling for work by constantly yielding. +Blocking on a mutex or condition variable or even more obviously having a +thread sleep if it has a low frequency workload are all mechanisms by which a +thread can be excluded from the ready queue until it really does need to be +run. This can have a significant positive impact on performance. + + +.. _Initialization_and_shutdown_dependencies: + +Initialization, shutdown and dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The L-thread subsystem depends on DPDK for huge page allocation and depends on +the ``rte_timer subsystem``. The DPDK EAL initialization and +``rte_timer_subsystem_init()`` **MUST** be completed before the L-thread sub +system can be used. + +Thereafter initialization of the L-thread subsystem is largely transparent to +the application. Constructor functions ensure that global variables are properly +initialized. Other than global variables each scheduler is initialized +independently the first time that an L-thread is created by a particular EAL +thread. + +If the schedulers are to be run as isolated and independent schedulers, with +no intention that L-threads running on different schedulers will migrate between +schedulers or synchronize with L-threads running on other schedulers, then +initialization consists simply of creating an L-thread, and then running the +L-thread scheduler. + +If there will be interaction between L-threads running on different schedulers, +then it is important that the starting of schedulers on different EAL threads +is synchronized. + +To achieve this an additional initialization step is necessary, this is simply +to set the number of schedulers by calling the API function +``lthread_num_schedulers_set(n)``, where ``n`` is the number of EAL threads +that will run L-thread schedulers. Setting the number of schedulers to a +number greater than 0 will cause all schedulers to wait until the others have +started before beginning to schedule L-threads. + +The L-thread scheduler is started by calling the function ``lthread_run()`` +and should be called from the EAL thread and thus become the main loop of the +EAL thread. + +The function ``lthread_run()``, will not return until all threads running on +the scheduler have exited, and the scheduler has been explicitly stopped by +calling ``lthread_scheduler_shutdown(lcore)`` or +``lthread_scheduler_shutdown_all()``. + +All these function do is tell the scheduler that it can exit when there are no +longer any running L-threads, neither function forces any running L-thread to +terminate. Any desired application shutdown behavior must be designed and +built into the application to ensure that L-threads complete in a timely +manner. + +**Important Note:** It is assumed when the scheduler exits that the application +is terminating for good, the scheduler does not free resources before exiting +and running the scheduler a subsequent time will result in undefined behavior. + + +.. _porting_legacy_code_to_run_on_lthreads: + +Porting legacy code to run on L-threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Legacy code originally written for a pthread environment may be ported to +L-threads if the considerations about differences in scheduling policy, and +constraints discussed in the previous sections can be accommodated. + +This section looks in more detail at some of the issues that may have to be +resolved when porting code. + + +.. _pthread_API_compatibility: + +pthread API compatibility +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The first step is to establish exactly which pthread APIs the legacy +application uses, and to understand the requirements of those APIs. If there +are corresponding L-lthread APIs, and where the default pthread functionality +is used by the application then, notwithstanding the other issues discussed +here, it should be feasible to run the application with L-threads. If the +legacy code modifies the default behavior using attributes then if may be +necessary to make some adjustments to eliminate those requirements. + + +.. _blocking_system_calls: + +Blocking system API calls +^^^^^^^^^^^^^^^^^^^^^^^^^ + +It is important to understand what other system services the application may be +using, bearing in mind that in a cooperatively scheduled environment a thread +cannot block without stalling the scheduler and with it all other cooperative +threads. Any kind of blocking system call, for example file or socket IO, is a +potential problem, a good tool to analyze the application for this purpose is +the ``strace`` utility. + +There are many strategies to resolve these kind of issues, each with it +merits. Possible solutions include: + +* Adopting a polled mode of the system API concerned (if available). + +* Arranging for another core to perform the function and synchronizing with + that core via constructs that will not block the L-thread. + +* Affinitizing the thread to another scheduler devoted (as a matter of policy) + to handling threads wishing to make blocking calls, and then back again when + finished. + + +.. _porting_locks_and_spinlocks: + +Locks and spinlocks +^^^^^^^^^^^^^^^^^^^ + +Locks and spinlocks are another source of blocking behavior that for the same +reasons as system calls will need to be addressed. + +If the application design ensures that the contending L-threads will always +run on the same scheduler then it its probably safe to remove locks and spin +locks completely. + +The only exception to the above rule is if for some reason the +code performs any kind of context switch whilst holding the lock +(e.g. yield, sleep, or block on a different lock, or on a condition variable). +This will need to determined before deciding to eliminate a lock. + +If a lock cannot be eliminated then an L-thread mutex can be substituted for +either kind of lock. + +An L-thread blocking on an L-thread mutex will be suspended and will cause +another ready L-thread to be resumed, thus not blocking the scheduler. When +default behavior is required, it can be used as a direct replacement for a +pthread mutex lock. + +Spin locks are typically used when lock contention is likely to be rare and +where the period during which the lock may be held is relatively short. +When the contending L-threads are running on the same scheduler then an +L-thread blocking on a spin lock will enter an infinite loop stopping the +scheduler completely (see :ref:`porting_infinite_loops` below). + +If the application design ensures that contending L-threads will always run +on different schedulers then it might be reasonable to leave a short spin lock +that rarely experiences contention in place. + +If after all considerations it appears that a spin lock can neither be +eliminated completely, replaced with an L-thread mutex, or left in place as +is, then an alternative is to loop on a flag, with a call to +``lthread_yield()`` inside the loop (n.b. if the contending L-threads might +ever run on different schedulers the flag will need to be manipulated +atomically). + +Spinning and yielding is the least preferred solution since it introduces +ready queue backlog (see also :ref:`ready_queue_backlog`). + + +.. _porting_sleeps_and_delays: + +Sleeps and delays +^^^^^^^^^^^^^^^^^ + +Yet another kind of blocking behavior (albeit momentary) are delay functions +like ``sleep()``, ``usleep()``, ``nanosleep()`` etc. All will have the +consequence of stalling the L-thread scheduler and unless the delay is very +short (e.g. a very short nanosleep) calls to these functions will need to be +eliminated. + +The simplest mitigation strategy is to use the L-thread sleep API functions, +of which two variants exist, ``lthread_sleep()`` and ``lthread_sleep_clks()``. +These functions start an rte_timer against the L-thread, suspend the L-thread +and cause another ready L-thread to be resumed. The suspended L-thread is +resumed when the rte_timer matures. + + +.. _porting_infinite_loops: + +Infinite loops +^^^^^^^^^^^^^^ + +Some applications have threads with loops that contain no inherent +rescheduling opportunity, and rely solely on the OS time slicing to share +the CPU. In a cooperative environment this will stop everything dead. These +kind of loops are not hard to identify, in a debug session you will find the +debugger is always stopping in the same loop. + +The simplest solution to this kind of problem is to insert an explicit +``lthread_yield()`` or ``lthread_sleep()`` into the loop. Another solution +might be to include the function performed by the loop into the execution path +of some other loop that does in fact yield, if this is possible. + + +.. _porting_thread_local_storage: + +Thread local storage +^^^^^^^^^^^^^^^^^^^^ + +If the application uses thread local storage, the use case should be +studied carefully. + +In a legacy pthread application either or both the ``__thread`` prefix, or the +pthread set/get specific APIs may have been used to define storage local to a +pthread. + +In some applications it may be a reasonable assumption that the data could +or in fact most likely should be placed in L-thread local storage. + +If the application (like many DPDK applications) has assumed a certain +relationship between a pthread and the CPU to which it is affinitized, there +is a risk that thread local storage may have been used to save some data items +that are correctly logically associated with the CPU, and others items which +relate to application context for the thread. Only a good understanding of the +application will reveal such cases. + +If the application requires an that an L-thread is to be able to move between +schedulers then care should be taken to separate these kinds of data, into per +lcore, and per L-thread storage. In this way a migrating thread will bring with +it the local data it needs, and pick up the new logical core specific values +from pthread local storage at its new home. + + +.. _pthread_shim: + +Pthread shim +~~~~~~~~~~~~ + +A convenient way to get something working with legacy code can be to use a +shim that adapts pthread API calls to the corresponding L-thread ones. +This approach will not mitigate any of the porting considerations mentioned +in the previous sections, but it will reduce the amount of code churn that +would otherwise been involved. It is a reasonable approach to evaluate +L-threads, before investing effort in porting to the native L-thread APIs. + + +Overview +^^^^^^^^ +The L-thread subsystem includes an example pthread shim. This is a partial +implementation but does contain the API stubs needed to get basic applications +running. There is a simple "hello world" application that demonstrates the +use of the pthread shim. + +A subtlety of working with a shim is that the application will still need +to make use of the genuine pthread library functions, at the very least in +order to create the EAL threads in which the L-thread schedulers will run. +This is the case with DPDK initialization, and exit. + +To deal with the initialization and shutdown scenarios, the shim is capable of +switching on or off its adaptor functionality, an application can control this +behavior by the calling the function ``pt_override_set()``. The default state +is disabled. + +The pthread shim uses the dynamic linker loader and saves the loaded addresses +of the genuine pthread API functions in an internal table, when the shim +functionality is enabled it performs the adaptor function, when disabled it +invokes the genuine pthread function. + +The function ``pthread_exit()`` has additional special handling. The standard +system header file pthread.h declares ``pthread_exit()`` with +``__attribute__((noreturn))`` this is an optimization that is possible because +the pthread is terminating and this enables the compiler to omit the normal +handling of stack and protection of registers since the function is not +expected to return, and in fact the thread is being destroyed. These +optimizations are applied in both the callee and the caller of the +``pthread_exit()`` function. + +In our cooperative scheduling environment this behavior is inadmissible. The +pthread is the L-thread scheduler thread, and, although an L-thread is +terminating, there must be a return to the scheduler in order that the system +can continue to run. Further, returning from a function with attribute +``noreturn`` is invalid and may result in undefined behavior. + +The solution is to redefine the ``pthread_exit`` function with a macro, +causing it to be mapped to a stub function in the shim that does not have the +``noreturn`` attribute. This macro is defined in the file +``pthread_shim.h``. The stub function is otherwise no different than any of +the other stub functions in the shim, and will switch between the real +``pthread_exit()`` function or the ``lthread_exit()`` function as +required. The only difference is that the mapping to the stub by macro +substitution. + +A consequence of this is that the file ``pthread_shim.h`` must be included in +legacy code wishing to make use of the shim. It also means that dynamic +linkage of a pre-compiled binary that did not include pthread_shim.h is not be +supported. + +Given the requirements for porting legacy code outlined in +:ref:`porting_legacy_code_to_run_on_lthreads` most applications will require at +least some minimal adjustment and recompilation to run on L-threads so +pre-compiled binaries are unlikely to be met in practice. + +In summary the shim approach adds some overhead but can be a useful tool to help +establish the feasibility of a code reuse project. It is also a fairly +straightforward task to extend the shim if necessary. + +**Note:** Bearing in mind the preceding discussions about the impact of making +blocking calls then switching the shim in and out on the fly to invoke any +pthread API this might block is something that should typically be avoided. + + +Building and running the pthread shim +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The shim example application is located in the sample application +in the performance-thread folder + +To build and run the pthread shim example + +#. Go to the example applications folder + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/performance-thread/pthread_shim + + +#. Set the target (a default target is used if not specified). For example: + + .. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + + See the DPDK Getting Started Guide for possible RTE_TARGET values. + +#. Build the application: + + .. code-block:: console + + make + +#. To run the pthread_shim example + + .. code-block:: console + + lthread-pthread-shim -c core_mask -n number_of_channels + +.. _lthread_diagnostics: + +L-thread Diagnostics +~~~~~~~~~~~~~~~~~~~~ + +When debugging you must take account of the fact that the L-threads are run in +a single pthread. The current scheduler is defined by +``RTE_PER_LCORE(this_sched)``, and the current lthread is stored at +``RTE_PER_LCORE(this_sched)->current_lthread``. Thus on a breakpoint in a GDB +session the current lthread can be obtained by displaying the pthread local +variable ``per_lcore_this_sched->current_lthread``. + +Another useful diagnostic feature is the possibility to trace significant +events in the life of an L-thread, this feature is enabled by changing the +value of LTHREAD_DIAG from 0 to 1 in the file ``lthread_diag_api.h``. + +Tracing of events can be individually masked, and the mask may be programmed +at run time. An unmasked event results in a callback that provides information +about the event. The default callback simply prints trace information. The +default mask is 0 (all events off) the mask can be modified by calling the +function ``lthread_diagniostic_set_mask()``. + +It is possible register a user callback function to implement more +sophisticated diagnostic functions. +Object creation events (lthread, mutex, and condition variable) accept, and +store in the created object, a user supplied reference value returned by the +callback function. + +The lthread reference value is passed back in all subsequent event callbacks, +the mutex and APIs are provided to retrieve the reference value from +mutexes and condition variables. This enables a user to monitor, count, or +filter for specific events, on specific objects, for example to monitor for a +specific thread signaling a specific condition variable, or to monitor +on all timer events, the possibilities and combinations are endless. + +The callback function can be set by calling the function +``lthread_diagnostic_enable()`` supplying a callback function pointer and an +event mask. + +Setting ``LTHREAD_DIAG`` also enables counting of statistics about cache and +queue usage, and these statistics can be displayed by calling the function +``lthread_diag_stats_display()``. This function also performs a consistency +check on the caches and queues. The function should only be called from the +master EAL thread after all slave threads have stopped and returned to the C +main program, otherwise the consistency check will fail. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/ptpclient.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/ptpclient.rst new file mode 100644 index 00000000..53de5037 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/ptpclient.rst @@ -0,0 +1,252 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +PTP Client Sample Application +============================= + +The PTP (Precision Time Protocol) client sample application is a simple +example of using the DPDK IEEE1588 API to communicate with a PTP master clock +to synchronize the time on the NIC and, optionally, on the Linux system. + +Note, PTP is a time syncing protocol and cannot be used within DPDK as a +time-stamping mechanism. See the following for an explanation of the protocol: +`Precision Time Protocol +<https://en.wikipedia.org/wiki/Precision_Time_Protocol>`_. + + +Limitations +----------- + +The PTP sample application is intended as a simple reference implementation of +a PTP client using the DPDK IEEE1588 API. +In order to keep the application simple the following assumptions are made: + +* The first discovered master is the master for the session. +* Only L2 PTP packets are supported. +* Only the PTP v2 protocol is supported. +* Only the slave clock is implemented. + + +How the Application Works +------------------------- + +.. _figure_ptpclient_highlevel: + +.. figure:: img/ptpclient.* + + PTP Synchronization Protocol + +The PTP synchronization in the sample application works as follows: + +* Master sends *Sync* message - the slave saves it as T2. +* Master sends *Follow Up* message and sends time of T1. +* Slave sends *Delay Request* frame to PTP Master and stores T3. +* Master sends *Delay Response* T4 time which is time of received T3. + +The adjustment for slave can be represented as: + + adj = -[(T2-T1)-(T4 - T3)]/2 + +If the command line parameter ``-T 1`` is used the application also +synchronizes the PTP PHC clock with the Linux kernel clock. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``ptpclient`` sub-directory. + +.. note:: + To compile the application edit the ``config/common_linuxapp`` configuration file to enable IEEE1588 + and then recompile DPDK: + + .. code-block:: console + + CONFIG_RTE_LIBRTE_IEEE1588=y + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + ./build/ptpclient -l 1 -n 4 -- -p 0x1 -T 0 + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + +* ``-p portmask``: Hexadecimal portmask. +* ``-T 0``: Update only the PTP slave clock. +* ``-T 1``: Update the PTP slave clock and synchronize the Linux Kernel to the PTP clock. + + +Code Explanation +---------------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). The +``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + +And than we parse application specific arguments + +.. code-block:: c + + argc -= ret; + argv += ret; + + ret = ptp_parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with PTP initialization\n"); + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUFS * nb_ports, + MBUF_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``port_init()`` function with portmask provided by user: + +.. code-block:: c + + for (portid = 0; portid < nb_ports; portid++) + if ((ptp_enabled_port_mask & (1 << portid)) != 0) { + + if (port_init(portid, mbuf_pool) == 0) { + ptp_enabled_ports[ptp_enabled_port_nb] = portid; + ptp_enabled_port_nb++; + } else { + rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8 "\n", + portid); + } + } + + +Once the initialization is complete, the application is ready to launch a +function on an lcore. In this example ``lcore_main()`` is called on a single +lcore. + +.. code-block:: c + + lcore_main(); + +The ``lcore_main()`` function is explained below. + + +The Lcores Main +~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function on the +available lcores. + +The main work of the application is done within the loop: + +.. code-block:: c + + for (portid = 0; portid < ptp_enabled_port_nb; portid++) { + + portid = ptp_enabled_ports[portid]; + nb_rx = rte_eth_rx_burst(portid, 0, &m, 1); + + if (likely(nb_rx == 0)) + continue; + + if (m->ol_flags & PKT_RX_IEEE1588_PTP) + parse_ptp_frames(portid, m); + + rte_pktmbuf_free(m); + } + +Packets are received one by one on the RX ports and, if required, PTP response +packets are transmitted on the TX ports. + +If the offload flags in the mbuf indicate that the packet is a PTP packet then +the packet is parsed to determine which type: + +.. code-block:: c + + if (m->ol_flags & PKT_RX_IEEE1588_PTP) + parse_ptp_frames(portid, m); + + +All packets are freed explicitly using ``rte_pktmbuf_free()``. + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. + + +PTP parsing +~~~~~~~~~~~ + +The ``parse_ptp_frames()`` function processes PTP packets, implementing slave +PTP IEEE1588 L2 functionality. + +.. code-block:: c + + void + parse_ptp_frames(uint16_t portid, struct rte_mbuf *m) { + struct ptp_header *ptp_hdr; + struct ether_hdr *eth_hdr; + uint16_t eth_type; + + eth_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); + eth_type = rte_be_to_cpu_16(eth_hdr->ether_type); + + if (eth_type == PTP_PROTOCOL) { + ptp_data.m = m; + ptp_data.portid = portid; + ptp_hdr = (struct ptp_header *)(rte_pktmbuf_mtod(m, char *) + + sizeof(struct ether_hdr)); + + switch (ptp_hdr->msgtype) { + case SYNC: + parse_sync(&ptp_data); + break; + case FOLLOW_UP: + parse_fup(&ptp_data); + break; + case DELAY_RESP: + parse_drsp(&ptp_data); + print_clock_info(&ptp_data); + break; + default: + break; + } + } + } + +There are 3 types of packets on the RX path which we must parse to create a minimal +implementation of the PTP slave client: + +* SYNC packet. +* FOLLOW UP packet +* DELAY RESPONSE packet. + +When we parse the *FOLLOW UP* packet we also create and send a *DELAY_REQUEST* packet. +Also when we parse the *DELAY RESPONSE* packet, and all conditions are met we adjust the PTP slave clock. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/qos_metering.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/qos_metering.rst new file mode 100644 index 00000000..6391841c --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/qos_metering.rst @@ -0,0 +1,151 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +QoS Metering Sample Application +=============================== + +The QoS meter sample application is an example that demonstrates the use of DPDK to provide QoS marking and metering, +as defined by RFC2697 for Single Rate Three Color Marker (srTCM) and RFC 2698 for Two Rate Three Color Marker (trTCM) algorithm. + +Overview +-------- + +The application uses a single thread for reading the packets from the RX port, +metering, marking them with the appropriate color (green, yellow or red) and writing them to the TX port. + +A policing scheme can be applied before writing the packets to the TX port by dropping or +changing the color of the packet in a static manner depending on both the input and output colors of the packets that are processed by the meter. + +The operation mode can be selected as compile time out of the following options: + +* Simple forwarding + +* srTCM color blind + +* srTCM color aware + +* srTCM color blind + +* srTCM color aware + +Please refer to RFC2697 and RFC2698 for details about the srTCM and trTCM configurable parameters +(CIR, CBS and EBS for srTCM; CIR, PIR, CBS and PBS for trTCM). + +The color blind modes are functionally equivalent with the color-aware modes when +all the incoming packets are colored as green. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``qos_meter`` sub-directory. + +Running the Application +----------------------- + +The application execution command line is as below: + +.. code-block:: console + + ./qos_meter [EAL options] -- -p PORTMASK + +The application is constrained to use a single core in the EAL core mask and 2 ports only in the application port mask +(first port from the port mask is used for RX and the other port in the core mask is used for TX). + +Refer to *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +Selecting one of the metering modes is done with these defines: + +.. code-block:: c + + #define APP_MODE_FWD 0 + #define APP_MODE_SRTCM_COLOR_BLIND 1 + #define APP_MODE_SRTCM_COLOR_AWARE 2 + #define APP_MODE_TRTCM_COLOR_BLIND 3 + #define APP_MODE_TRTCM_COLOR_AWARE 4 + + #define APP_MODE APP_MODE_SRTCM_COLOR_BLIND + +To simplify debugging (for example, by using the traffic generator RX side MAC address based packet filtering feature), +the color is defined as the LSB byte of the destination MAC address. + +The traffic meter parameters are configured in the application source code with following default values: + +.. code-block:: c + + struct rte_meter_srtcm_params app_srtcm_params[] = { + + {.cir = 1000000 * 46, .cbs = 2048, .ebs = 2048}, + + }; + + struct rte_meter_trtcm_params app_trtcm_params[] = { + + {.cir = 1000000 * 46, .pir = 1500000 * 46, .cbs = 2048, .pbs = 2048}, + + }; + +Assuming the input traffic is generated at line rate and all packets are 64 bytes Ethernet frames (IPv4 packet size of 46 bytes) +and green, the expected output traffic should be marked as shown in the following table: + +.. _table_qos_metering_1: + +.. table:: Output Traffic Marking + + +-------------+------------------+-------------------+----------------+ + | **Mode** | **Green (Mpps)** | **Yellow (Mpps)** | **Red (Mpps)** | + | | | | | + +=============+==================+===================+================+ + | srTCM blind | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | srTCM color | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM blind | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM color | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | FWD | 14.88 | 0 | 0 | + | | | | | + +-------------+------------------+-------------------+----------------+ + +To set up the policing scheme as desired, it is necessary to modify the main.h source file, +where this policy is implemented as a static structure, as follows: + +.. code-block:: c + + int policer_table[e_RTE_METER_COLORS][e_RTE_METER_COLORS] = + { + { GREEN, RED, RED}, + { DROP, YELLOW, RED}, + { DROP, DROP, RED} + }; + +Where rows indicate the input color, columns indicate the output color, +and the value that is stored in the table indicates the action to be taken for that particular case. + +There are four different actions: + +* GREEN: The packet's color is changed to green. + +* YELLOW: The packet's color is changed to yellow. + +* RED: The packet's color is changed to red. + +* DROP: The packet is dropped. + +In this particular case: + +* Every packet which input and output color are the same, keeps the same color. + +* Every packet which color has improved is dropped (this particular case can't happen, so these values will not be used). + +* For the rest of the cases, the color is changed to red. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/qos_scheduler.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/qos_scheduler.rst new file mode 100644 index 00000000..d6c3bc6c --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/qos_scheduler.rst @@ -0,0 +1,305 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +QoS Scheduler Sample Application +================================ + +The QoS sample application demonstrates the use of the DPDK to provide QoS scheduling. + +Overview +-------- + +The architecture of the QoS scheduler application is shown in the following figure. + +.. _figure_qos_sched_app_arch: + +.. figure:: img/qos_sched_app_arch.* + + QoS Scheduler Application Architecture + + +There are two flavors of the runtime execution for this application, +with two or three threads per each packet flow configuration being used. +The RX thread reads packets from the RX port, +classifies the packets based on the double VLAN (outer and inner) and +the lower two bytes of the IP destination address and puts them into the ring queue. +The worker thread dequeues the packets from the ring and calls the QoS scheduler enqueue/dequeue functions. +If a separate TX core is used, these are sent to the TX ring. +Otherwise, they are sent directly to the TX port. +The TX thread, if present, reads from the TX ring and write the packets to the TX port. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``qos_sched`` sub-directory. + + .. note:: + + This application is intended as a linuxapp only. + +.. note:: + + To get statistics on the sample app using the command line interface as described in the next section, + DPDK must be compiled defining *CONFIG_RTE_SCHED_COLLECT_STATS*, + which can be done by changing the configuration file for the specific target to be compiled. + +Running the Application +----------------------- + +.. note:: + + In order to run the application, a total of at least 4 + G of huge pages must be set up for each of the used sockets (depending on the cores in use). + +The application has a number of command line options: + +.. code-block:: console + + ./qos_sched [EAL options] -- <APP PARAMS> + +Mandatory application parameters include: + +* --pfc "RX PORT, TX PORT, RX LCORE, WT LCORE, TX CORE": Packet flow configuration. + Multiple pfc entities can be configured in the command line, + having 4 or 5 items (if TX core defined or not). + +Optional application parameters include: + +* -i: It makes the application to start in the interactive mode. + In this mode, the application shows a command line that can be used for obtaining statistics while + scheduling is taking place (see interactive mode below for more information). + +* --mst n: Master core index (the default value is 1). + +* --rsz "A, B, C": Ring sizes: + +* A = Size (in number of buffer descriptors) of each of the NIC RX rings read + by the I/O RX lcores (the default value is 128). + +* B = Size (in number of elements) of each of the software rings used + by the I/O RX lcores to send packets to worker lcores (the default value is 8192). + +* C = Size (in number of buffer descriptors) of each of the NIC TX rings written + by worker lcores (the default value is 256) + +* --bsz "A, B, C, D": Burst sizes + +* A = I/O RX lcore read burst size from the NIC RX (the default value is 64) + +* B = I/O RX lcore write burst size to the output software rings, + worker lcore read burst size from input software rings,QoS enqueue size (the default value is 64) + +* C = QoS dequeue size (the default value is 32) + +* D = Worker lcore write burst size to the NIC TX (the default value is 64) + +* --msz M: Mempool size (in number of mbufs) for each pfc (default 2097152) + +* --rth "A, B, C": The RX queue threshold parameters + +* A = RX prefetch threshold (the default value is 8) + +* B = RX host threshold (the default value is 8) + +* C = RX write-back threshold (the default value is 4) + +* --tth "A, B, C": TX queue threshold parameters + +* A = TX prefetch threshold (the default value is 36) + +* B = TX host threshold (the default value is 0) + +* C = TX write-back threshold (the default value is 0) + +* --cfg FILE: Profile configuration to load + +Refer to *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +The profile configuration file defines all the port/subport/pipe/traffic class/queue parameters +needed for the QoS scheduler configuration. + +The profile file has the following format: + +:: + + ; port configuration [port] + + frame overhead = 24 + number of subports per port = 1 + number of pipes per subport = 4096 + queue sizes = 64 64 64 64 + + ; Subport configuration + + [subport 0] + tb rate = 1250000000; Bytes per second + tb size = 1000000; Bytes + tc 0 rate = 1250000000; Bytes per second + tc 1 rate = 1250000000; Bytes per second + tc 2 rate = 1250000000; Bytes per second + tc 3 rate = 1250000000; Bytes per second + tc period = 10; Milliseconds + tc oversubscription period = 10; Milliseconds + + pipe 0-4095 = 0; These pipes are configured with pipe profile 0 + + ; Pipe configuration + + [pipe profile 0] + tb rate = 305175; Bytes per second + tb size = 1000000; Bytes + + tc 0 rate = 305175; Bytes per second + tc 1 rate = 305175; Bytes per second + tc 2 rate = 305175; Bytes per second + tc 3 rate = 305175; Bytes per second + tc period = 40; Milliseconds + + tc 0 oversubscription weight = 1 + tc 1 oversubscription weight = 1 + tc 2 oversubscription weight = 1 + tc 3 oversubscription weight = 1 + + tc 0 wrr weights = 1 1 1 1 + tc 1 wrr weights = 1 1 1 1 + tc 2 wrr weights = 1 1 1 1 + tc 3 wrr weights = 1 1 1 1 + + ; RED params per traffic class and color (Green / Yellow / Red) + + [red] + tc 0 wred min = 48 40 32 + tc 0 wred max = 64 64 64 + tc 0 wred inv prob = 10 10 10 + tc 0 wred weight = 9 9 9 + + tc 1 wred min = 48 40 32 + tc 1 wred max = 64 64 64 + tc 1 wred inv prob = 10 10 10 + tc 1 wred weight = 9 9 9 + + tc 2 wred min = 48 40 32 + tc 2 wred max = 64 64 64 + tc 2 wred inv prob = 10 10 10 + tc 2 wred weight = 9 9 9 + + tc 3 wred min = 48 40 32 + tc 3 wred max = 64 64 64 + tc 3 wred inv prob = 10 10 10 + tc 3 wred weight = 9 9 9 + +Interactive mode +~~~~~~~~~~~~~~~~ + +These are the commands that are currently working under the command line interface: + +* Control Commands + +* --quit: Quits the application. + +* General Statistics + + * stats app: Shows a table with in-app calculated statistics. + + * stats port X subport Y: For a specific subport, it shows the number of packets that + went through the scheduler properly and the number of packets that were dropped. + The same information is shown in bytes. + The information is displayed in a table separating it in different traffic classes. + + * stats port X subport Y pipe Z: For a specific pipe, it shows the number of packets that + went through the scheduler properly and the number of packets that were dropped. + The same information is shown in bytes. + This information is displayed in a table separating it in individual queues. + +* Average queue size + +All of these commands work the same way, averaging the number of packets throughout a specific subset of queues. + +Two parameters can be configured for this prior to calling any of these commands: + + * qavg n X: n is the number of times that the calculation will take place. + Bigger numbers provide higher accuracy. The default value is 10. + + * qavg period X: period is the number of microseconds that will be allowed between each calculation. + The default value is 100. + +The commands that can be used for measuring average queue size are: + +* qavg port X subport Y: Show average queue size per subport. + +* qavg port X subport Y tc Z: Show average queue size per subport for a specific traffic class. + +* qavg port X subport Y pipe Z: Show average queue size per pipe. + +* qavg port X subport Y pipe Z tc A: Show average queue size per pipe for a specific traffic class. + +* qavg port X subport Y pipe Z tc A q B: Show average queue size of a specific queue. + +Example +~~~~~~~ + +The following is an example command with a single packet flow configuration: + +.. code-block:: console + + ./qos_sched -l 1,5,7 -n 4 -- --pfc "3,2,5,7" --cfg ./profile.cfg + +This example uses a single packet flow configuration which creates one RX thread on lcore 5 reading +from port 3 and a worker thread on lcore 7 writing to port 2. + +Another example with 2 packet flow configurations using different ports but sharing the same core for QoS scheduler is given below: + +.. code-block:: console + + ./qos_sched -l 1,2,6,7 -n 4 -- --pfc "3,2,2,6,7" --pfc "1,0,2,6,7" --cfg ./profile.cfg + +Note that independent cores for the packet flow configurations for each of the RX, WT and TX thread are also supported, +providing flexibility to balance the work. + +The EAL coremask/corelist is constrained to contain the default mastercore 1 and the RX, WT and TX cores only. + +Explanation +----------- + +The Port/Subport/Pipe/Traffic Class/Queue are the hierarchical entities in a typical QoS application: + +* A subport represents a predefined group of users. + +* A pipe represents an individual user/subscriber. + +* A traffic class is the representation of a different traffic type with a specific loss rate, + delay and jitter requirements; such as data voice, video or data transfers. + +* A queue hosts packets from one or multiple connections of the same type belonging to the same user. + +The traffic flows that need to be configured are application dependent. +This application classifies based on the QinQ double VLAN tags and the IP destination address as indicated in the following table. + +.. _table_qos_scheduler_1: + +.. table:: Entity Types + + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | + | | | | | + +================+=========================+==================================================+==================================+ + | Port | - | Ethernet port | Physical port | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + +Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/quota_watermark.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/quota_watermark.rst new file mode 100644 index 00000000..67200e15 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/quota_watermark.rst @@ -0,0 +1,465 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2017 Intel Corporation. + +Quota and Watermark Sample Application +====================================== + +The Quota and Watermark sample application is a simple example of packet +processing using Data Plane Development Kit (DPDK) that showcases the use +of a quota as the maximum number of packets enqueue/dequeue at a time and +low and high thresholds, or watermarks, to signal low and high ring usage +respectively. + +Additionally, it shows how the thresholds can be used to feedback congestion notifications to data producers by +temporarily stopping processing overloaded rings and sending Ethernet flow control frames. + +This sample application is split in two parts: + +* qw - The core quota and watermark sample application + +* qwctl - A command line tool to alter quota and watermarks while qw is running + +Overview +-------- + +The Quota and Watermark sample application performs forwarding for each packet that is received on a given port. +The destination port is the adjacent port from the enabled port mask, that is, +if the first four ports are enabled (port mask 0xf), ports 0 and 1 forward into each other, +and ports 2 and 3 forward into each other. +The MAC addresses of the forwarded Ethernet frames are not affected. + +Internally, packets are pulled from the ports by the master logical core and put on a variable length processing pipeline, +each stage of which being connected by rings, as shown in :numref:`figure_pipeline_overview`. + +.. _figure_pipeline_overview: + +.. figure:: img/pipeline_overview.* + + Pipeline Overview + + +An adjustable quota value controls how many packets are being moved through the pipeline per enqueue and dequeue. +Adjustable threshold values associated with the rings control a back-off mechanism that +tries to prevent the pipeline from being overloaded by: + +* Stopping enqueuing on rings for which the usage has crossed the high watermark threshold + +* Sending Ethernet pause frames + +* Only resuming enqueuing on a ring once its usage goes below a global low watermark threshold + +This mechanism allows congestion notifications to go up the ring pipeline and +eventually lead to an Ethernet flow control frame being send to the source. + +On top of serving as an example of quota and watermark usage, +this application can be used to benchmark ring based processing pipelines performance using a traffic- generator, +as shown in :numref:`figure_ring_pipeline_perf_setup`. + +.. _figure_ring_pipeline_perf_setup: + +.. figure:: img/ring_pipeline_perf_setup.* + + Ring-based Processing Pipeline Performance Setup + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``quota_watermark`` sub-directory. + +Running the Application +----------------------- + +The core application, qw, has to be started first. + +Once it is up and running, one can alter quota and watermarks while it runs using the control application, qwctl. + +Running the Core Application +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application requires a single command line option: + +.. code-block:: console + + ./qw/build/qw [EAL options] -- -p PORTMASK + +where, + +-p PORTMASK: A hexadecimal bitmask of the ports to configure + +To run the application in a linuxapp environment with four logical cores and ports 0 and 2, +issue the following command: + +.. code-block:: console + + ./qw/build/qw -l 0-3 -n 4 -- -p 5 + +Refer to the *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +Running the Control Application +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The control application requires a number of command line options: + +.. code-block:: console + + ./qwctl/build/qwctl [EAL options] --proc-type=secondary + +The --proc-type=secondary option is necessary for the EAL to properly initialize the control application to +use the same huge pages as the core application and thus be able to access its rings. + +To run the application in a linuxapp environment on logical core 0, issue the following command: + +.. code-block:: console + + ./qwctl/build/qwctl -l 0 -n 4 --proc-type=secondary + +Refer to the *DPDK Getting Started* Guide for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +qwctl is an interactive command line that let the user change variables in a running instance of qw. +The help command gives a list of available commands: + +.. code-block:: console + + $ qwctl > help + +Code Overview +------------- + +The following sections provide a quick guide to the application's source code. + +Core Application - qw +~~~~~~~~~~~~~~~~~~~~~ + +EAL and Drivers Setup +^^^^^^^^^^^^^^^^^^^^^ + +The EAL arguments are parsed at the beginning of the main() function: + +.. code-block:: c + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Cannot initialize EAL\n"); + + argc -= ret; + argv += ret; + +Then, a call to init_dpdk(), defined in init.c, is made to initialize the poll mode drivers: + +.. code-block:: c + + void + init_dpdk(void) + { + int ret; + + /* Bind the drivers to usable devices */ + + ret = rte_pci_probe(); + if (ret < 0) + rte_exit(EXIT_FAILURE, "rte_pci_probe(): error %d\n", ret); + + if (rte_eth_dev_count_avail() < 2) + rte_exit(EXIT_FAILURE, "Not enough Ethernet port available\n"); + } + +To fully understand this code, it is recommended to study the chapters that relate to the *Poll Mode Driver* +in the *DPDK Getting Started Guide* and the *DPDK API Reference*. + +Shared Variables Setup +^^^^^^^^^^^^^^^^^^^^^^ + +The quota and high and low watermark shared variables are put into an rte_memzone using a call to setup_shared_variables(): + +.. code-block:: c + + void + setup_shared_variables(void) + { + const struct rte_memzone *qw_memzone; + + qw_memzone = rte_memzone_reserve(QUOTA_WATERMARK_MEMZONE_NAME, + 3 * sizeof(int), rte_socket_id(), 0); + if (qw_memzone == NULL) + rte_exit(EXIT_FAILURE, "%s\n", rte_strerror(rte_errno)); + + quota = qw_memzone->addr; + low_watermark = (unsigned int *) qw_memzone->addr + 1; + high_watermark = (unsigned int *) qw_memzone->addr + 2; + } + +These three variables are initialized to a default value in main() and +can be changed while qw is running using the qwctl control program. + +Application Arguments +^^^^^^^^^^^^^^^^^^^^^ + +The qw application only takes one argument: a port mask that specifies which ports should be used by the application. +At least two ports are needed to run the application and there should be an even number of ports given in the port mask. + +The port mask parsing is done in parse_qw_args(), defined in args.c. + +Mbuf Pool Initialization +^^^^^^^^^^^^^^^^^^^^^^^^ + +Once the application's arguments are parsed, an mbuf pool is created. +It contains a set of mbuf objects that are used by the driver and the application to store network packets: + +.. code-block:: c + + /* Create a pool of mbuf to store packets */ + mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", MBUF_PER_POOL, 32, 0, + MBUF_DATA_SIZE, rte_socket_id()); + + if (mbuf_pool == NULL) + rte_panic("%s\n", rte_strerror(rte_errno)); + +The rte_mempool is a generic structure used to handle pools of objects. +In this case, it is necessary to create a pool that will be used by the driver. + +The number of allocated pkt mbufs is MBUF_PER_POOL, with a data room size +of MBUF_DATA_SIZE each. +A per-lcore cache of 32 mbufs is kept. +The memory is allocated in on the master lcore's socket, but it is possible to extend this code to allocate one mbuf pool per socket. + +The rte_pktmbuf_pool_create() function uses the default mbuf pool and mbuf +initializers, respectively rte_pktmbuf_pool_init() and rte_pktmbuf_init(). +An advanced application may want to use the mempool API to create the +mbuf pool with more control. + +Ports Configuration and Pairing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each port in the port mask is configured and a corresponding ring is created in the master lcore's array of rings. +This ring is the first in the pipeline and will hold the packets directly coming from the port. + +.. code-block:: c + + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) + if (is_bit_set(port_id, portmask)) { + configure_eth_port(port_id); + init_ring(master_lcore_id, port_id); + } + + pair_ports(); + +The configure_eth_port() and init_ring() functions are used to configure a port and a ring respectively and are defined in init.c. +They make use of the DPDK APIs defined in rte_eth.h and rte_ring.h. + +pair_ports() builds the port_pairs[] array so that its key-value pairs are a mapping between reception and transmission ports. +It is defined in init.c. + +Logical Cores Assignment +^^^^^^^^^^^^^^^^^^^^^^^^ + +The application uses the master logical core to poll all the ports for new packets and enqueue them on a ring associated with the port. + +Each logical core except the last runs pipeline_stage() after a ring for each used port is initialized on that core. +pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See :numref:`figure_threads_pipelines`. + +.. code-block:: c + + /* Start pipeline_stage() on all the available slave lcore but the last */ + + for (lcore_id = 0 ; lcore_id < last_lcore_id; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) && lcore_id != master_lcore_id) { + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) + if (is_bit_set(port_id, portmask)) + init_ring(lcore_id, port_id); + + rte_eal_remote_launch(pipeline_stage, NULL, lcore_id); + } + } + +The last available logical core runs send_stage(), +which is the last stage of the pipeline dequeuing packets from the last ring in the pipeline and +sending them out on the destination port setup by pair_ports(). + +.. code-block:: c + + /* Start send_stage() on the last slave core */ + + rte_eal_remote_launch(send_stage, NULL, last_lcore_id); + +Receive, Process and Transmit Packets +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. _figure_threads_pipelines: + +.. figure:: img/threads_pipelines.* + + Threads and Pipelines + + +In the receive_stage() function running on the master logical core, +the main task is to read ingress packets from the RX ports and enqueue them +on the port's corresponding first ring in the pipeline. +This is done using the following code: + +.. code-block:: c + + lcore_id = rte_lcore_id(); + + /* Process each port round robin style */ + + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (!is_bit_set(port_id, portmask)) + continue; + + ring = rings[lcore_id][port_id]; + + if (ring_state[port_id] != RING_READY) { + if (rte_ring_count(ring) > *low_watermark) + continue; + else + ring_state[port_id] = RING_READY; + } + + /* Enqueue received packets on the RX ring */ + nb_rx_pkts = rte_eth_rx_burst(port_id, 0, pkts, + (uint16_t) *quota); + ret = rte_ring_enqueue_bulk(ring, (void *) pkts, + nb_rx_pkts, &free); + if (RING_SIZE - free > *high_watermark) { + ring_state[port_id] = RING_OVERLOADED; + send_pause_frame(port_id, 1337); + } + + if (ret == 0) { + + /* + * Return mbufs to the pool, + * effectively dropping packets + */ + for (i = 0; i < nb_rx_pkts; i++) + rte_pktmbuf_free(pkts[i]); + } + } + +For each port in the port mask, the corresponding ring's pointer is fetched into ring and that ring's state is checked: + +* If it is in the RING_READY state, \*quota packets are grabbed from the port and put on the ring. + Should this operation make the ring's usage cross its high watermark, + the ring is marked as overloaded and an Ethernet flow control frame is sent to the source. + +* If it is not in the RING_READY state, this port is ignored until the ring's usage crosses the \*low_watermark value. + +The pipeline_stage() function's task is to process and move packets from the preceding pipeline stage. +This thread is running on most of the logical cores to create and arbitrarily long pipeline. + +.. code-block:: c + + lcore_id = rte_lcore_id(); + + previous_lcore_id = get_previous_lcore_id(lcore_id); + + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (!is_bit_set(port_id, portmask)) + continue; + + tx = rings[lcore_id][port_id]; + rx = rings[previous_lcore_id][port_id]; + + if (ring_state[port_id] != RING_READY) { + if (rte_ring_count(tx) > *low_watermark) + continue; + else + ring_state[port_id] = RING_READY; + } + + /* Dequeue up to quota mbuf from rx */ + nb_dq_pkts = rte_ring_dequeue_burst(rx, pkts, + *quota, NULL); + if (unlikely(nb_dq_pkts < 0)) + continue; + + /* Enqueue them on tx */ + ret = rte_ring_enqueue_bulk(tx, pkts, + nb_dq_pkts, &free); + if (RING_SIZE - free > *high_watermark) + ring_state[port_id] = RING_OVERLOADED; + + if (ret == 0) { + + /* + * Return mbufs to the pool, + * effectively dropping packets + */ + for (i = 0; i < nb_dq_pkts; i++) + rte_pktmbuf_free(pkts[i]); + } + } + +The thread's logic works mostly like receive_stage(), +except that packets are moved from ring to ring instead of port to ring. + +In this example, no actual processing is done on the packets, +but pipeline_stage() is an ideal place to perform any processing required by the application. + +Finally, the send_stage() function's task is to read packets from the last ring in a pipeline and +send them on the destination port defined in the port_pairs[] array. +It is running on the last available logical core only. + +.. code-block:: c + + lcore_id = rte_lcore_id(); + + previous_lcore_id = get_previous_lcore_id(lcore_id); + + for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) { + if (!is_bit_set(port_id, portmask)) continue; + + dest_port_id = port_pairs[port_id]; + tx = rings[previous_lcore_id][port_id]; + + if (rte_ring_empty(tx)) continue; + + /* Dequeue packets from tx and send them */ + + nb_dq_pkts = rte_ring_dequeue_burst(tx, (void *) tx_pkts, *quota); + nb_tx_pkts = rte_eth_tx_burst(dest_port_id, 0, tx_pkts, nb_dq_pkts); + } + +For each port in the port mask, up to \*quota packets are pulled from the last ring in its pipeline and +sent on the destination port paired with the current port. + +Control Application - qwctl +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The qwctl application uses the rte_cmdline library to provide the user with an interactive command line that +can be used to modify and inspect parameters in a running qw application. +Those parameters are the global quota and low_watermark value as well as each ring's built-in high watermark. + +Command Definitions +^^^^^^^^^^^^^^^^^^^ + +The available commands are defined in commands.c. + +It is advised to use the cmdline sample application user guide as a reference for everything related to the rte_cmdline library. + +Accessing Shared Variables +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The setup_shared_variables() function retrieves the shared variables quota and +low_watermark from the rte_memzone previously created by qw. + +.. code-block:: c + + static void + setup_shared_variables(void) + { + const struct rte_memzone *qw_memzone; + + qw_memzone = rte_memzone_lookup(QUOTA_WATERMARK_MEMZONE_NAME); + if (qw_memzone == NULL) + rte_exit(EXIT_FAILURE, "Couldn't find memzone\n"); + + quota = qw_memzone->addr; + + low_watermark = (unsigned int *) qw_memzone->addr + 1; + high_watermark = (unsigned int *) qw_memzone->addr + 2; + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/rxtx_callbacks.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/rxtx_callbacks.rst new file mode 100644 index 00000000..85d96d8a --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/rxtx_callbacks.rst @@ -0,0 +1,199 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +RX/TX Callbacks Sample Application +================================== + +The RX/TX Callbacks sample application is a packet forwarding application that +demonstrates the use of user defined callbacks on received and transmitted +packets. The application performs a simple latency check, using callbacks, to +determine the time packets spend within the application. + +In the sample application a user defined callback is applied to all received +packets to add a timestamp. A separate callback is applied to all packets +prior to transmission to calculate the elapsed time, in CPU cycles. + + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``rxtx_callbacks`` sub-directory. + +The callbacks feature requires that the ``CONFIG_RTE_ETHDEV_RXTX_CALLBACKS`` +setting is on in the ``config/common_`` config file that applies to the +target. This is generally on by default: + +.. code-block:: console + + CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + ./build/rxtx_callbacks -l 1 -n 4 + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + + + +Explanation +----------- + +The ``rxtx_callbacks`` application is mainly a simple forwarding application +based on the :doc:`skeleton`. See that section of the documentation for more +details of the forwarding part of the application. + +The sections below explain the additional RX/TX callback code. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the application initialization and calls the +execution threads for each lcore. This function is effectively identical to +the ``main()`` function explained in :doc:`skeleton`. + +The ``lcore_main()`` function is also identical. + +The main difference is in the user defined ``port_init()`` function where the +callbacks are added. This is explained in the next section: + + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization is shown below with +comments: + +.. code-block:: c + + static inline int + port_init(uint16_t port, struct rte_mempool *mbuf_pool) + { + struct rte_eth_conf port_conf = port_conf_default; + const uint16_t rx_rings = 1, tx_rings = 1; + struct ether_addr addr; + int retval; + uint16_t q; + + /* Configure the Ethernet device. */ + retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf); + if (retval != 0) + return retval; + + /* Allocate and set up 1 RX queue per Ethernet port. */ + for (q = 0; q < rx_rings; q++) { + retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL, mbuf_pool); + if (retval < 0) + return retval; + } + + /* Allocate and set up 1 TX queue per Ethernet port. */ + for (q = 0; q < tx_rings; q++) { + retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL); + if (retval < 0) + return retval; + } + + /* Start the Ethernet port. */ + retval = rte_eth_dev_start(port); + if (retval < 0) + return retval; + + /* Enable RX in promiscuous mode for the Ethernet device. */ + rte_eth_promiscuous_enable(port); + + + /* Add the callbacks for RX and TX.*/ + rte_eth_add_rx_callback(port, 0, add_timestamps, NULL); + rte_eth_add_tx_callback(port, 0, calc_latency, NULL); + + return 0; + } + + +The RX and TX callbacks are added to the ports/queues as function pointers: + +.. code-block:: c + + rte_eth_add_rx_callback(port, 0, add_timestamps, NULL); + rte_eth_add_tx_callback(port, 0, calc_latency, NULL); + +More than one callback can be added and additional information can be passed +to callback function pointers as a ``void*``. In the examples above ``NULL`` +is used. + +The ``add_timestamps()`` and ``calc_latency()`` functions are explained below. + + +The add_timestamps() Callback +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``add_timestamps()`` callback is added to the RX port and is applied to +all packets received: + +.. code-block:: c + + static uint16_t + add_timestamps(uint16_t port __rte_unused, uint16_t qidx __rte_unused, + struct rte_mbuf **pkts, uint16_t nb_pkts, void *_ __rte_unused) + { + unsigned i; + uint64_t now = rte_rdtsc(); + + for (i = 0; i < nb_pkts; i++) + pkts[i]->udata64 = now; + + return nb_pkts; + } + +The DPDK function ``rte_rdtsc()`` is used to add a cycle count timestamp to +each packet (see the *cycles* section of the *DPDK API Documentation* for +details). + + +The calc_latency() Callback +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``calc_latency()`` callback is added to the TX port and is applied to all +packets prior to transmission: + +.. code-block:: c + + static uint16_t + calc_latency(uint16_t port __rte_unused, uint16_t qidx __rte_unused, + struct rte_mbuf **pkts, uint16_t nb_pkts, void *_ __rte_unused) + { + uint64_t cycles = 0; + uint64_t now = rte_rdtsc(); + unsigned i; + + for (i = 0; i < nb_pkts; i++) + cycles += now - pkts[i]->udata64; + + latency_numbers.total_cycles += cycles; + latency_numbers.total_pkts += nb_pkts; + + if (latency_numbers.total_pkts > (100 * 1000 * 1000ULL)) { + printf("Latency = %"PRIu64" cycles\n", + latency_numbers.total_cycles / latency_numbers.total_pkts); + + latency_numbers.total_cycles = latency_numbers.total_pkts = 0; + } + + return nb_pkts; + } + +The ``calc_latency()`` function accumulates the total number of packets and +the total number of cycles used. Once more than 100 million packets have been +transmitted the average cycle count per packet is printed out and the counters +are reset. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/server_node_efd.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/server_node_efd.rst new file mode 100644 index 00000000..adf258c0 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/server_node_efd.rst @@ -0,0 +1,450 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016-2017 Intel Corporation. + +Server-Node EFD Sample Application +================================== + +This sample application demonstrates the use of EFD library as a flow-level +load balancer, for more information about the EFD Library please refer to the +DPDK programmer's guide. + +This sample application is a variant of the +:ref:`client-server sample application <multi_process_app>` +where a specific target node is specified for every and each flow +(not in a round-robin fashion as the original load balancing sample application). + +Overview +-------- + +The architecture of the EFD flow-based load balancer sample application is +presented in the following figure. + +.. _figure_efd_sample_app_overview: + +.. figure:: img/server_node_efd.* + + Using EFD as a Flow-Level Load Balancer + +As shown in :numref:`figure_efd_sample_app_overview`, +the sample application consists of a front-end node (server) +using the EFD library to create a load-balancing table for flows, +for each flow a target backend worker node is specified. The EFD table does not +store the flow key (unlike a regular hash table), and hence, it can +individually load-balance millions of flows (number of targets * maximum number +of flows fit in a flow table per target) while still fitting in CPU cache. + +It should be noted that although they are referred to as nodes, the frontend +server and worker nodes are processes running on the same platform. + +Front-end Server +~~~~~~~~~~~~~~~~ + +Upon initializing, the frontend server node (process) creates a flow +distributor table (based on the EFD library) which is populated with flow +information and its intended target node. + +The sample application assigns a specific target node_id (process) for each of +the IP destination addresses as follows: + +.. code-block:: c + + node_id = i % num_nodes; /* Target node id is generated */ + ip_dst = rte_cpu_to_be_32(i); /* Specific ip destination address is + assigned to this target node */ + +then the pair of <key,target> is inserted into the flow distribution table. + +The main loop of the server process receives a burst of packets, then for +each packet, a flow key (IP destination address) is extracted. The flow +distributor table is looked up and the target node id is returned. Packets are +then enqueued to the specified target node id. + +It should be noted that flow distributor table is not a membership test table. +I.e. if the key has already been inserted the target node id will be correct, +but for new keys the flow distributor table will return a value (which can be +valid). + +Backend Worker Nodes +~~~~~~~~~~~~~~~~~~~~ + +Upon initializing, the worker node (process) creates a flow table (a regular +hash table that stores the key default size 1M flows) which is populated with +only the flow information that is serviced at this node. This flow key is +essential to point out new keys that have not been inserted before. + +The worker node's main loop is simply receiving packets then doing a hash table +lookup. If a match occurs then statistics are updated for flows serviced by +this node. If no match is found in the local hash table then this indicates +that this is a new flow, which is dropped. + + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``server_node_efd`` sub-directory. + +Running the Application +----------------------- + +The application has two binaries to be run: the front-end server +and the back-end node. + +The frontend server (server) has the following command line options:: + + ./server [EAL options] -- -p PORTMASK -n NUM_NODES -f NUM_FLOWS + +Where, + +* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure +* ``-n NUM_NODES:`` Number of back-end nodes that will be used +* ``-f NUM_FLOWS:`` Number of flows to be added in the EFD table (1 million, by default) + +The back-end node (node) has the following command line options:: + + ./node [EAL options] -- -n NODE_ID + +Where, + +* ``-n NODE_ID:`` Node ID, which cannot be equal or higher than NUM_MODES + + +First, the server app must be launched, with the number of nodes that will be run. +Once it has been started, the node instances can be run, with different NODE_ID. +These instances have to be run as secondary processes, with ``--proc-type=secondary`` +in the EAL options, which will attach to the primary process memory, and therefore, +they can access the queues created by the primary process to distribute packets. + +To successfully run the application, the command line used to start the +application has to be in sync with the traffic flows configured on the traffic +generator side. + +For examples of application command lines and traffic generator flows, please +refer to the DPDK Test Report. For more details on how to set up and run the +sample applications provided with DPDK package, please refer to the +:ref:`DPDK Getting Started Guide for Linux <linux_gsg>` and +:ref:`DPDK Getting Started Guide for FreeBSD <freebsd_gsg>`. + + +Explanation +----------- + +As described in previous sections, there are two processes in this example. + +The first process, the front-end server, creates and populates the EFD table, +which is used to distribute packets to nodes, which the number of flows +specified in the command line (1 million, by default). + + +.. code-block:: c + + static void + create_efd_table(void) + { + uint8_t socket_id = rte_socket_id(); + + /* create table */ + efd_table = rte_efd_create("flow table", num_flows * 2, sizeof(uint32_t), + 1 << socket_id, socket_id); + + if (efd_table == NULL) + rte_exit(EXIT_FAILURE, "Problem creating the flow table\n"); + } + + static void + populate_efd_table(void) + { + unsigned int i; + int32_t ret; + uint32_t ip_dst; + uint8_t socket_id = rte_socket_id(); + uint64_t node_id; + + /* Add flows in table */ + for (i = 0; i < num_flows; i++) { + node_id = i % num_nodes; + + ip_dst = rte_cpu_to_be_32(i); + ret = rte_efd_update(efd_table, socket_id, + (void *)&ip_dst, (efd_value_t)node_id); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Unable to add entry %u in " + "EFD table\n", i); + } + + printf("EFD table: Adding 0x%x keys\n", num_flows); + } + +After initialization, packets are received from the enabled ports, and the IPv4 +address from the packets is used as a key to look up in the EFD table, +which tells the node where the packet has to be distributed. + +.. code-block:: c + + static void + process_packets(uint32_t port_num __rte_unused, struct rte_mbuf *pkts[], + uint16_t rx_count, unsigned int socket_id) + { + uint16_t i; + uint8_t node; + efd_value_t data[EFD_BURST_MAX]; + const void *key_ptrs[EFD_BURST_MAX]; + + struct ipv4_hdr *ipv4_hdr; + uint32_t ipv4_dst_ip[EFD_BURST_MAX]; + + for (i = 0; i < rx_count; i++) { + /* Handle IPv4 header.*/ + ipv4_hdr = rte_pktmbuf_mtod_offset(pkts[i], struct ipv4_hdr *, + sizeof(struct ether_hdr)); + ipv4_dst_ip[i] = ipv4_hdr->dst_addr; + key_ptrs[i] = (void *)&ipv4_dst_ip[i]; + } + + rte_efd_lookup_bulk(efd_table, socket_id, rx_count, + (const void **) key_ptrs, data); + for (i = 0; i < rx_count; i++) { + node = (uint8_t) ((uintptr_t)data[i]); + + if (node >= num_nodes) { + /* + * Node is out of range, which means that + * flow has not been inserted + */ + flow_dist_stats.drop++; + rte_pktmbuf_free(pkts[i]); + } else { + flow_dist_stats.distributed++; + enqueue_rx_packet(node, pkts[i]); + } + } + + for (i = 0; i < num_nodes; i++) + flush_rx_queue(i); + } + +The burst of packets received is enqueued in temporary buffers (per node), +and enqueued in the shared ring between the server and the node. +After this, a new burst of packets is received and this process is +repeated infinitely. + +.. code-block:: c + + static void + flush_rx_queue(uint16_t node) + { + uint16_t j; + struct node *cl; + + if (cl_rx_buf[node].count == 0) + return; + + cl = &nodes[node]; + if (rte_ring_enqueue_bulk(cl->rx_q, (void **)cl_rx_buf[node].buffer, + cl_rx_buf[node].count, NULL) != cl_rx_buf[node].count){ + for (j = 0; j < cl_rx_buf[node].count; j++) + rte_pktmbuf_free(cl_rx_buf[node].buffer[j]); + cl->stats.rx_drop += cl_rx_buf[node].count; + } else + cl->stats.rx += cl_rx_buf[node].count; + + cl_rx_buf[node].count = 0; + } + +The second process, the back-end node, receives the packets from the shared +ring with the server and send them out, if they belong to the node. + +At initialization, it attaches to the server process memory, to have +access to the shared ring, parameters and statistics. + +.. code-block:: c + + rx_ring = rte_ring_lookup(get_rx_queue_name(node_id)); + if (rx_ring == NULL) + rte_exit(EXIT_FAILURE, "Cannot get RX ring - " + "is server process running?\n"); + + mp = rte_mempool_lookup(PKTMBUF_POOL_NAME); + if (mp == NULL) + rte_exit(EXIT_FAILURE, "Cannot get mempool for mbufs\n"); + + mz = rte_memzone_lookup(MZ_SHARED_INFO); + if (mz == NULL) + rte_exit(EXIT_FAILURE, "Cannot get port info structure\n"); + info = mz->addr; + tx_stats = &(info->tx_stats[node_id]); + filter_stats = &(info->filter_stats[node_id]); + +Then, the hash table that contains the flows that will be handled +by the node is created and populated. + +.. code-block:: c + + static struct rte_hash * + create_hash_table(const struct shared_info *info) + { + uint32_t num_flows_node = info->num_flows / info->num_nodes; + char name[RTE_HASH_NAMESIZE]; + struct rte_hash *h; + + /* create table */ + struct rte_hash_parameters hash_params = { + .entries = num_flows_node * 2, /* table load = 50% */ + .key_len = sizeof(uint32_t), /* Store IPv4 dest IP address */ + .socket_id = rte_socket_id(), + .hash_func_init_val = 0, + }; + + snprintf(name, sizeof(name), "hash_table_%d", node_id); + hash_params.name = name; + h = rte_hash_create(&hash_params); + + if (h == NULL) + rte_exit(EXIT_FAILURE, + "Problem creating the hash table for node %d\n", + node_id); + return h; + } + + static void + populate_hash_table(const struct rte_hash *h, const struct shared_info *info) + { + unsigned int i; + int32_t ret; + uint32_t ip_dst; + uint32_t num_flows_node = 0; + uint64_t target_node; + + /* Add flows in table */ + for (i = 0; i < info->num_flows; i++) { + target_node = i % info->num_nodes; + if (target_node != node_id) + continue; + + ip_dst = rte_cpu_to_be_32(i); + + ret = rte_hash_add_key(h, (void *) &ip_dst); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Unable to add entry %u " + "in hash table\n", i); + else + num_flows_node++; + + } + + printf("Hash table: Adding 0x%x keys\n", num_flows_node); + } + +After initialization, packets are dequeued from the shared ring +(from the server) and, like in the server process, +the IPv4 address from the packets is used as a key to look up in the hash table. +If there is a hit, packet is stored in a buffer, to be eventually transmitted +in one of the enabled ports. If key is not there, packet is dropped, since the +flow is not handled by the node. + +.. code-block:: c + + static inline void + handle_packets(struct rte_hash *h, struct rte_mbuf **bufs, uint16_t num_packets) + { + struct ipv4_hdr *ipv4_hdr; + uint32_t ipv4_dst_ip[PKT_READ_SIZE]; + const void *key_ptrs[PKT_READ_SIZE]; + unsigned int i; + int32_t positions[PKT_READ_SIZE] = {0}; + + for (i = 0; i < num_packets; i++) { + /* Handle IPv4 header.*/ + ipv4_hdr = rte_pktmbuf_mtod_offset(bufs[i], struct ipv4_hdr *, + sizeof(struct ether_hdr)); + ipv4_dst_ip[i] = ipv4_hdr->dst_addr; + key_ptrs[i] = &ipv4_dst_ip[i]; + } + /* Check if packets belongs to any flows handled by this node */ + rte_hash_lookup_bulk(h, key_ptrs, num_packets, positions); + + for (i = 0; i < num_packets; i++) { + if (likely(positions[i] >= 0)) { + filter_stats->passed++; + transmit_packet(bufs[i]); + } else { + filter_stats->drop++; + /* Drop packet, as flow is not handled by this node */ + rte_pktmbuf_free(bufs[i]); + } + } + } + +Finally, note that both processes updates statistics, such as transmitted, received +and dropped packets, which are shown and refreshed by the server app. + +.. code-block:: c + + static void + do_stats_display(void) + { + unsigned int i, j; + const char clr[] = {27, '[', '2', 'J', '\0'}; + const char topLeft[] = {27, '[', '1', ';', '1', 'H', '\0'}; + uint64_t port_tx[RTE_MAX_ETHPORTS], port_tx_drop[RTE_MAX_ETHPORTS]; + uint64_t node_tx[MAX_NODES], node_tx_drop[MAX_NODES]; + + /* to get TX stats, we need to do some summing calculations */ + memset(port_tx, 0, sizeof(port_tx)); + memset(port_tx_drop, 0, sizeof(port_tx_drop)); + memset(node_tx, 0, sizeof(node_tx)); + memset(node_tx_drop, 0, sizeof(node_tx_drop)); + + for (i = 0; i < num_nodes; i++) { + const struct tx_stats *tx = &info->tx_stats[i]; + + for (j = 0; j < info->num_ports; j++) { + const uint64_t tx_val = tx->tx[info->id[j]]; + const uint64_t drop_val = tx->tx_drop[info->id[j]]; + + port_tx[j] += tx_val; + port_tx_drop[j] += drop_val; + node_tx[i] += tx_val; + node_tx_drop[i] += drop_val; + } + } + + /* Clear screen and move to top left */ + printf("%s%s", clr, topLeft); + + printf("PORTS\n"); + printf("-----\n"); + for (i = 0; i < info->num_ports; i++) + printf("Port %u: '%s'\t", (unsigned int)info->id[i], + get_printable_mac_addr(info->id[i])); + printf("\n\n"); + for (i = 0; i < info->num_ports; i++) { + printf("Port %u - rx: %9"PRIu64"\t" + "tx: %9"PRIu64"\n", + (unsigned int)info->id[i], info->rx_stats.rx[i], + port_tx[i]); + } + + printf("\nSERVER\n"); + printf("-----\n"); + printf("distributed: %9"PRIu64", drop: %9"PRIu64"\n", + flow_dist_stats.distributed, flow_dist_stats.drop); + + printf("\nNODES\n"); + printf("-------\n"); + for (i = 0; i < num_nodes; i++) { + const unsigned long long rx = nodes[i].stats.rx; + const unsigned long long rx_drop = nodes[i].stats.rx_drop; + const struct filter_stats *filter = &info->filter_stats[i]; + + printf("Node %2u - rx: %9llu, rx_drop: %9llu\n" + " tx: %9"PRIu64", tx_drop: %9"PRIu64"\n" + " filter_passed: %9"PRIu64", " + "filter_drop: %9"PRIu64"\n", + i, rx, rx_drop, node_tx[i], node_tx_drop[i], + filter->passed, filter->drop); + } + + printf("\n"); + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/service_cores.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/service_cores.rst new file mode 100644 index 00000000..b8339e70 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/service_cores.rst @@ -0,0 +1,145 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2017 Intel Corporation. + +Service Cores Sample Application +================================ + +The service cores sample application demonstrates the service cores capabilities +of DPDK. The service cores infrastructure is part of the DPDK EAL, and allows +any DPDK component to register a service. A service is a work item or task, that +requires CPU time to perform its duty. + +This sample application registers 5 dummy services. These 5 services are used +to show how the service_cores API can be used to orchestrate these services to +run on different service lcores. This orchestration is done by calling the +service cores APIs, however the sample application introduces a "profile" +concept to contain the service mapping details. Note that the profile concept +is application specific, and not a part of the service cores API. + + +Compiling the Application +------------------------- + +#. Go to the example directory: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/service_cores + +#. Set the target (a default target is used if not specified). For example: + + .. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + + See the *DPDK Getting Started* Guide for possible RTE_TARGET values. + +#. Build the application: + + .. code-block:: console + + make + +Running the Application +----------------------- + +To run the example, just execute the binary. Since the application dynamically +adds service cores in the application code itself, there is no requirement to +pass a service core-mask as an EAL argument at startup time. + +.. code-block:: console + + $ ./build/service_cores + + +Explanation +----------- + +The following sections provide some explanation of code focusing on +registering applications from an applications point of view, and modifying the +service core counts and mappings at runtime. + + +Registering a Service +~~~~~~~~~~~~~~~~~~~~~ + +The following code section shows how to register a service as an application. +Note that the service component header must be included by the application in +order to register services: ``rte_service_component.h``, in addition +to the ordinary service cores header ``rte_service.h`` which provides +the runtime functions to add, remove and remap service cores. + +.. code-block:: c + + struct rte_service_spec service = { + .name = "service_name", + }; + int ret = rte_service_component_register(services, &id); + if (ret) + return -1; + + /* set the service itself to be ready to run. In the case of + * ethdev, eventdev etc PMDs, this will be set when the + * appropriate configure or setup function is called. + */ + rte_service_component_runstate_set(id, 1); + + /* Collect statistics for the service */ + rte_service_set_stats_enable(id, 1); + + /* The application sets the service to running state. Note that this + * function enables the service to run - while the 'component' version + * of this function (as above) marks the service itself as ready */ + ret = rte_service_runstate_set(id, 1); + + +Controlling A Service Core +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This section demonstrates how to add a service core. The ``rte_service.h`` +header file provides the functions for dynamically adding and removing cores. +The APIs to add and remove cores use lcore IDs similar to existing DPDK +functions. + +These are the functions to start a service core, and have it run a service: + +.. code-block:: c + + /* the lcore ID to use as a service core */ + uint32_t service_core_id = 7; + ret = rte_service_lcore_add(service_core_id); + if(ret) + return -1; + + /* service cores are in "stopped" state when added, so start it */ + ret = rte_service_lcore_start(service_core_id); + if(ret) + return -1; + + /* map a service to the service core, causing it to run the service */ + uint32_t service_id; /* ID of a registered service */ + uint32_t enable = 1; /* 1 maps the service, 0 unmaps */ + ret = rte_service_map_lcore_set(service_id, service_core_id, enable); + if(ret) + return -1; + + +Removing A Service Core +~~~~~~~~~~~~~~~~~~~~~~~ + +To remove a service core, the steps are similar to adding but in reverse order. +Note that it is not allowed to remove a service core if the service is running, +and the service-core is the only core running that service (see documentation +for ``rte_service_lcore_stop`` function for details). + + +Conclusion +~~~~~~~~~~ + +The service cores infrastructure provides DPDK with two main features. The first +is to abstract away hardware differences: the service core can CPU cycles to +a software fallback implementation, allowing the application to be abstracted +from the difference in HW / SW availability. The second feature is a flexible +method of registering functions to be run, allowing the running of the +functions to be scaled across multiple CPUs. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/skeleton.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/skeleton.rst new file mode 100644 index 00000000..95956171 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/skeleton.rst @@ -0,0 +1,288 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2015 Intel Corporation. + +Basic Forwarding Sample Application +=================================== + +The Basic Forwarding sample application is a simple *skeleton* example of a +forwarding application. + +It is intended as a demonstration of the basic components of a DPDK forwarding +application. For more detailed implementations see the L2 and L3 forwarding +sample applications. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``skeleton`` sub-directory. + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + ./build/basicfwd -l 1 -n 4 + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). The +``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + mbuf_pool = rte_mempool_create("MBUF_POOL", + NUM_MBUFS * nb_ports, + MBUF_SIZE, + MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + rte_socket_id(), + 0); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``port_init()`` function which is explained in the next section: + +.. code-block:: c + + RTE_ETH_FOREACH_DEV(portid) { + if (port_init(portid, mbuf_pool) != 0) { + rte_exit(EXIT_FAILURE, + "Cannot init port %" PRIu8 "\n", portid); + } + } + + +Once the initialization is complete, the application is ready to launch a +function on an lcore. In this example ``lcore_main()`` is called on a single +lcore. + + +.. code-block:: c + + lcore_main(); + +The ``lcore_main()`` function is explained below. + + + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization used in the Basic +Forwarding application is shown below: + +.. code-block:: c + + static inline int + port_init(uint16_t port, struct rte_mempool *mbuf_pool) + { + struct rte_eth_conf port_conf = port_conf_default; + const uint16_t rx_rings = 1, tx_rings = 1; + struct ether_addr addr; + int retval; + uint16_t q; + + if (!rte_eth_dev_is_valid_port(port)) + return -1; + + /* Configure the Ethernet device. */ + retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf); + if (retval != 0) + return retval; + + /* Allocate and set up 1 RX queue per Ethernet port. */ + for (q = 0; q < rx_rings; q++) { + retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL, mbuf_pool); + if (retval < 0) + return retval; + } + + /* Allocate and set up 1 TX queue per Ethernet port. */ + for (q = 0; q < tx_rings; q++) { + retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL); + if (retval < 0) + return retval; + } + + /* Start the Ethernet port. */ + retval = rte_eth_dev_start(port); + if (retval < 0) + return retval; + + /* Enable RX in promiscuous mode for the Ethernet device. */ + rte_eth_promiscuous_enable(port); + + return 0; + } + +The Ethernet ports are configured with default settings using the +``rte_eth_dev_configure()`` function and the ``port_conf_default`` struct: + +.. code-block:: c + + static const struct rte_eth_conf port_conf_default = { + .rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN } + }; + +For this example the ports are set up with 1 RX and 1 TX queue using the +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + retval = rte_eth_dev_start(port); + + +Finally the RX port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(port); + + +The Lcores Main +~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function on the +available lcores. For the Basic Forwarding application the lcore function +looks like the following: + +.. code-block:: c + + static __attribute__((noreturn)) void + lcore_main(void) + { + uint16_t port; + + /* + * Check that the port is on the same NUMA node as the polling thread + * for best performance. + */ + RTE_ETH_FOREACH_DEV(port) + if (rte_eth_dev_socket_id(port) > 0 && + rte_eth_dev_socket_id(port) != + (int)rte_socket_id()) + printf("WARNING, port %u is on remote NUMA node to " + "polling thread.\n\tPerformance will " + "not be optimal.\n", port); + + printf("\nCore %u forwarding packets. [Ctrl+C to quit]\n", + rte_lcore_id()); + + /* Run until the application is quit or killed. */ + for (;;) { + /* + * Receive packets on a port and forward them on the paired + * port. The mapping is 0 -> 1, 1 -> 0, 2 -> 3, 3 -> 2, etc. + */ + RTE_ETH_FOREACH_DEV(port) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + } + + +The main work of the application is done within the loop: + +.. code-block:: c + + for (;;) { + RTE_ETH_FOREACH_DEV(port) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + +Packets are received in bursts on the RX ports and transmitted in bursts on +the TX ports. The ports are grouped in pairs with a simple mapping scheme +using the an XOR on the port number:: + + 0 -> 1 + 1 -> 0 + + 2 -> 3 + 3 -> 2 + + etc. + +The ``rte_eth_tx_burst()`` function frees the memory buffers of packets that +are transmitted. If packets fail to transmit, ``(nb_tx < nb_rx)``, then they +must be freed explicitly using ``rte_pktmbuf_free()``. + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/tep_termination.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/tep_termination.rst new file mode 100644 index 00000000..b3332977 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/tep_termination.rst @@ -0,0 +1,233 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2015 Intel Corporation. + +TEP termination Sample Application +================================== + +The TEP (Tunnel End point) termination sample application simulates a VXLAN +Tunnel Endpoint (VTEP) termination in DPDK, which is used to demonstrate +the offload and filtering capabilities of Intel® XL710 10/40 Gigabit Ethernet +Controller for VXLAN packet. +This sample uses the basic virtio devices management mechanism from vhost example, +and also uses the us-vHost interface and tunnel filtering mechanism to direct +a specified traffic to a specific VM. +In addition, this sample is also designed to show how tunneling protocols can be handled. + +Background +---------- + +With virtualization, overlay networks allow a network structure to be built +or imposed across physical nodes which is abstracted away from the actual +underlining physical network connections. +This allows network isolation, QOS, etc to be provided on a per client basis. + +.. _figure_overlay_networking: + +.. figure:: img/overlay_networking.* + + Overlay Networking. + +In a typical setup, the network overlay tunnel is terminated at the Virtual/Tunnel End Point (VEP/TEP). +The TEP is normally located at the physical host level ideally in the software switch. +Due to processing constraints and the inevitable bottleneck that the switch +becomes, the ability to offload overlay support features becomes an important requirement. +Intel® XL710 10/40 Gigabit Ethernet network card provides hardware filtering +and offload capabilities to support overlay networks implementations such as MAC in UDP and MAC in GRE. + +Sample Code Overview +-------------------- + +The DPDK TEP termination sample code demonstrates the offload and filtering +capabilities of Intel® XL710 10/40 Gigabit Ethernet Controller for VXLAN packet. + +The sample code is based on vhost library. +The vhost library is developed for user space Ethernet switch to easily integrate with vhost functionality. + +The sample will support the followings: + +* Tunneling packet recognition. + +* The port of UDP tunneling is configurable + +* Directing incoming traffic to the correct queue based on the tunnel filter type. + The supported filter type are listed below. + + * Inner MAC and VLAN and tenant ID + + * Inner MAC and tenant ID, and Outer MAC + + * Inner MAC and tenant ID + + The tenant ID will be assigned from a static internal table based on the us-vhost device ID. + Each device will receive a unique device ID. + The inner MAC will be learned by the first packet transmitted from a device. + +* Decapsulation of RX VXLAN traffic. This is a software only operation. + +* Encapsulation of TX VXLAN traffic. This is a software only operation. + +* Inner IP and inner L4 checksum offload. + +* TSO offload support for tunneling packet. + +The following figure shows the framework of the TEP termination sample +application based on DPDK vhost lib. + +.. _figure_tep_termination_arch: + +.. figure:: img/tep_termination_arch.* + + TEP termination Framework Overview + +Supported Distributions +----------------------- + +The example in this section have been validated with the following distributions: + +* Fedora* 18 + +* Fedora* 19 + +* Fedora* 20 + +Compiling the Sample Code +------------------------- + +To enable vhost, turn on vhost library in the configure file +``config/common_linuxapp``. + + .. code-block:: console + + CONFIG_RTE_LIBRTE_VHOST=y + +Then following the to compile the sample application shown in +:doc:`compiling`. + +Running the Sample Code +----------------------- + +#. Go to the examples directory: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/tep_termination + +#. Run the tep_termination sample code: + + .. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + -p 0x1 --dev-basename tep-termination --nb-devices 4 + --udp-port 4789 --filter-type 1 + +.. note:: + + Please note the huge-dir parameter instructs the DPDK to allocate its memory from the 2 MB page hugetlbfs. + +Parameters +~~~~~~~~~~ + +**The same parameters with the vhost sample.** + +Refer to :ref:`vhost_app_parameters` for detailed explanation. + +**Number of Devices.** + +The nb-devices option specifies the number of virtIO device. +The default value is 2. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 2 + +**Tunneling UDP port.** + +The udp-port option is used to specify the destination UDP number for UDP tunneling packet. +The default value is 4789. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 2 --udp-port 4789 + +**Filter Type.** + +The filter-type option is used to specify which filter type is used to +filter UDP tunneling packet to a specified queue. +The default value is 1, which means the filter type of inner MAC and tenant ID is used. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 2 --udp-port 4789 --filter-type 1 + +**TX Checksum.** + +The tx-checksum option is used to enable or disable the inner header checksum offload. +The default value is 0, which means the checksum offload is disabled. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 2 --tx-checksum + +**TCP segment size.** + +The tso-segsz option specifies the TCP segment size for TSO offload for tunneling packet. +The default value is 0, which means TSO offload is disabled. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --tx-checksum --tso-segsz 800 + +**Decapsulation option.** + +The decap option is used to enable or disable decapsulation operation for received VXLAN packet. +The default value is 1. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 4 --udp-port 4789 --decap 1 + +**Encapsulation option.** + +The encap option is used to enable or disable encapsulation operation for transmitted packet. +The default value is 1. + +.. code-block:: console + + user@target:~$ ./build/app/tep_termination -l 0-3 -n 4 --huge-dir /mnt/huge -- + --nb-devices 4 --udp-port 4789 --encap 1 + + +Running the Virtual Machine (QEMU) +---------------------------------- + +Refer to :ref:`vhost_app_run_vm`. + +Running DPDK in the Virtual Machine +----------------------------------- + +Refer to :ref:`vhost_app_run_dpdk_inside_guest`. + +Passing Traffic to the Virtual Machine Device +--------------------------------------------- + +For a virtio-net device to receive traffic, the traffic's Layer 2 header must include +both the virtio-net device's MAC address. +The DPDK sample code behaves in a similar manner to a learning switch in that +it learns the MAC address of the virtio-net devices from the first transmitted packet. +On learning the MAC address, +the DPDK vhost sample code prints a message with the MAC address and tenant ID virtio-net device. +For example: + +.. code-block:: console + + DATA: (0) MAC_ADDRESS cc:bb:bb:bb:bb:bb and VNI 1000 registered + +The above message indicates that device 0 has been registered with MAC address cc:bb:bb:bb:bb:bb and VNI 1000. +Any packets received on the NIC with these values are placed on the devices receive queue. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/test_pipeline.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/test_pipeline.rst new file mode 100644 index 00000000..a9370c80 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/test_pipeline.rst @@ -0,0 +1,239 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Test Pipeline Application +========================= + +The Test Pipeline application illustrates the use of the DPDK Packet Framework tool suite. +Its purpose is to demonstrate the performance of single-table DPDK pipelines. + +Overview +-------- + +The application uses three CPU cores: + +* Core A ("RX core") receives traffic from the NIC ports and feeds core B with traffic through SW queues. + +* Core B ("Pipeline core") implements a single-table DPDK pipeline + whose type is selectable through specific command line parameter. + Core B receives traffic from core A through software queues, + processes it according to the actions configured in the table entries that + are hit by the input packets and feeds it to core C through another set of software queues. + +* Core C ("TX core") receives traffic from core B through software queues and sends it to the NIC ports for transmission. + +.. _figure_test_pipeline_app: + +.. figure:: img/test_pipeline_app.* + + Test Pipeline Application + +Compiling the Application +------------------------- +To compile the sample application see :doc:`compiling` + +The application is located in the ``$RTE_SDK/test/test-pipline`` directory. + + +Running the Application +----------------------- + +Application Command Line +~~~~~~~~~~~~~~~~~~~~~~~~ + +The application execution command line is: + +.. code-block:: console + + ./test-pipeline [EAL options] -- -p PORTMASK --TABLE_TYPE + +The -c or -l EAL CPU coremask/corelist option has to contain exactly 3 CPU cores. +The first CPU core in the core mask is assigned for core A, the second for core B and the third for core C. + +The PORTMASK parameter must contain 2 or 4 ports. + +Table Types and Behavior +~~~~~~~~~~~~~~~~~~~~~~~~ + +:numref:`table_test_pipeline_1` describes the table types used and how they are populated. + +The hash tables are pre-populated with 16 million keys. +For hash tables, the following parameters can be selected: + +* **Configurable key size implementation or fixed (specialized) key size implementation (e.g. hash-8-ext or hash-spec-8-ext).** + The key size specialized implementations are expected to provide better performance for 8-byte and 16-byte key sizes, + while the key-size-non-specialized implementation is expected to provide better performance for larger key sizes; + +* **Key size (e.g. hash-spec-8-ext or hash-spec-16-ext).** + The available options are 8, 16 and 32 bytes; + +* **Table type (e.g. hash-spec-16-ext or hash-spec-16-lru).** + The available options are ext (extendable bucket) or lru (least recently used). + +.. _table_test_pipeline_1: + +.. table:: Table Types + + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | **#** | **TABLE_TYPE** | **Description of Core B Table** | **Pre-added Table Entries** | + | | | | | + +=======+========================+==========================================================+=======================================================+ + | 1 | none | Core B is not implementing a DPDK pipeline. | N/A | + | | | Core B is implementing a pass-through from its input set | | + | | | of software queues to its output set of software queues. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 2 | stub | Stub table. Core B is implementing the same pass-through | N/A | + | | | functionality as described for the "none" option by | | + | | | using the DPDK Packet Framework by using one | | + | | | stub table for each input NIC port. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 3 | hash-[spec]-8-lru | LRU hash table with 8-byte key size and 16 million | 16 million entries are successfully added to the | + | | | entries. | hash table with the following key format: | + | | | | | + | | | | [4-byte index, 4 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Sendto output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | core B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 4 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 4 | hash-[spec]-8-ext | Extendable bucket hash table with 8-byte key size | Same as hash-[spec]-8-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 5 | hash-[spec]-16-lru | LRU hash table with 16-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 12 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for core | + | | | | B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 12 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 6 | hash-[spec]-16-ext | Extendable bucket hash table with 16-byte key size | Same as hash-[spec]-16-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 7 | hash-[spec]-32-lru | LRU hash table with 32-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 28 bytes of 0]. | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | Lpmcore B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 28 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 8 | hash-[spec]-32-ext | Extendable bucket hash table with 32-byte key size | Same as hash-[spec]-32-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 9 | lpm | Longest Prefix Match (LPM) IPv4 table. | In the case of two ports, two routes | + | | | | are added to the table: | + | | | | | + | | | | [0.0.0.0/9 => send to output port 0] | + | | | | | + | | | | [0.128.0.0/9 => send to output port 1] | + | | | | | + | | | | In case of four ports, four entries are added to the | + | | | | table: | + | | | | | + | | | | [0.0.0.0/10 => send to output port 0] | + | | | | | + | | | | [0.64.0.0/10 => send to output port 1] | + | | | | | + | | | | [0.128.0.0/10 => send to output port 2] | + | | | | | + | | | | [0.192.0.0/10 => send to output port 3] | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is storing the IPv4 destination | + | | | | within the packet meta data to be later used by core | + | | | | B as the lookup key. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 10 | acl | Access Control List (ACL) table | In the case of two ports, two ACL rules are added to | + | | | | the table: | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.0.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0] | + | | | | | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.128.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0]. | + | | | | | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + +Input Traffic +~~~~~~~~~~~~~ + +Regardless of the table type used for the core B pipeline, +the same input traffic can be used to hit all table entries with uniform distribution, +which results in uniform distribution of packets sent out on the set of output NIC ports. +The profile for input traffic is TCP/IPv4 packets with: + +* destination IP address as A.B.C.D with A fixed to 0 and B, C,D random + +* source IP address fixed to 0.0.0.0 + +* destination TCP port fixed to 0 + +* source TCP port fixed to 0 diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/timer.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/timer.rst new file mode 100644 index 00000000..d87a7abc --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/timer.rst @@ -0,0 +1,176 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +Timer Sample Application +======================== + +The Timer sample application is a simple application that demonstrates the use of a timer in a DPDK application. +This application prints some messages from different lcores regularly, demonstrating the use of timers. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``timer`` sub-directory. + +Running the Application +----------------------- + +To run the example in linuxapp environment: + +.. code-block:: console + + $ ./build/timer -l 0-3 -n 4 + +Refer to the *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +Initialization and Main Loop +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In addition to EAL initialization, the timer subsystem must be initialized, by calling the rte_timer_subsystem_init() function. + +.. code-block:: c + + /* init EAL */ + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_panic("Cannot init EAL\n"); + + /* init RTE timer library */ + + rte_timer_subsystem_init(); + +After timer creation (see the next paragraph), +the main loop is executed on each slave lcore using the well-known rte_eal_remote_launch() and also on the master. + +.. code-block:: c + + /* call lcore_mainloop() on every slave lcore */ + + RTE_LCORE_FOREACH_SLAVE(lcore_id) { + rte_eal_remote_launch(lcore_mainloop, NULL, lcore_id); + } + + /* call it on master lcore too */ + + (void) lcore_mainloop(NULL); + +The main loop is very simple in this example: + +.. code-block:: c + + while (1) { + /* + * Call the timer handler on each core: as we don't + * need a very precise timer, so only call + * rte_timer_manage() every ~10ms (at 2 GHz). In a real + * application, this will enhance performances as + * reading the HPET timer is not efficient. + */ + + cur_tsc = rte_rdtsc(); + + diff_tsc = cur_tsc - prev_tsc; + + if (diff_tsc > TIMER_RESOLUTION_CYCLES) { + rte_timer_manage(); + prev_tsc = cur_tsc; + } + } + +As explained in the comment, it is better to use the TSC register (as it is a per-lcore register) to check if the +rte_timer_manage() function must be called or not. +In this example, the resolution of the timer is 10 milliseconds. + +Managing Timers +~~~~~~~~~~~~~~~ + +In the main() function, the two timers are initialized. +This call to rte_timer_init() is necessary before doing any other operation on the timer structure. + +.. code-block:: c + + /* init timer structures */ + + rte_timer_init(&timer0); + rte_timer_init(&timer1); + +Then, the two timers are configured: + +* The first timer (timer0) is loaded on the master lcore and expires every second. + Since the PERIODICAL flag is provided, the timer is reloaded automatically by the timer subsystem. + The callback function is timer0_cb(). + +* The second timer (timer1) is loaded on the next available lcore every 333 ms. + The SINGLE flag means that the timer expires only once and must be reloaded manually if required. + The callback function is timer1_cb(). + +.. code-block:: c + + /* load timer0, every second, on master lcore, reloaded automatically */ + + hz = rte_get_hpet_hz(); + + lcore_id = rte_lcore_id(); + + rte_timer_reset(&timer0, hz, PERIODICAL, lcore_id, timer0_cb, NULL); + + /* load timer1, every second/3, on next lcore, reloaded manually */ + + lcore_id = rte_get_next_lcore(lcore_id, 0, 1); + + rte_timer_reset(&timer1, hz/3, SINGLE, lcore_id, timer1_cb, NULL); + +The callback for the first timer (timer0) only displays a message until a global counter reaches 20 (after 20 seconds). +In this case, the timer is stopped using the rte_timer_stop() function. + +.. code-block:: c + + /* timer0 callback */ + + static void + timer0_cb( attribute ((unused)) struct rte_timer *tim, __attribute ((unused)) void *arg) + { + static unsigned counter = 0; + + unsigned lcore_id = rte_lcore_id(); + + printf("%s() on lcore %u\n", FUNCTION , lcore_id); + + /* this timer is automatically reloaded until we decide to stop it, when counter reaches 20. */ + + if ((counter ++) == 20) + rte_timer_stop(tim); + } + +The callback for the second timer (timer1) displays a message and reloads the timer on the next lcore, using the +rte_timer_reset() function: + +.. code-block:: c + + /* timer1 callback */ + + static void + timer1_cb( attribute ((unused)) struct rte_timer *tim, _attribute ((unused)) void *arg) + { + unsigned lcore_id = rte_lcore_id(); + uint64_t hz; + + printf("%s() on lcore %u\\n", FUNCTION , lcore_id); + + /* reload it on another lcore */ + + hz = rte_get_hpet_hz(); + + lcore_id = rte_get_next_lcore(lcore_id, 0, 1); + + rte_timer_reset(&timer1, hz/3, SINGLE, lcore_id, timer1_cb, NULL); + } diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/vhost.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost.rst new file mode 100644 index 00000000..fd42cb3f --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost.rst @@ -0,0 +1,203 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2016 Intel Corporation. + +Vhost Sample Application +======================== + +The vhost sample application demonstrates integration of the Data Plane +Development Kit (DPDK) with the Linux* KVM hypervisor by implementing the +vhost-net offload API. The sample application performs simple packet +switching between virtual machines based on Media Access Control (MAC) +address or Virtual Local Area Network (VLAN) tag. The splitting of Ethernet +traffic from an external switch is performed in hardware by the Virtual +Machine Device Queues (VMDQ) and Data Center Bridging (DCB) features of +the Intel® 82599 10 Gigabit Ethernet Controller. + +Testing steps +------------- + +This section shows the steps how to test a typical PVP case with this +vhost-switch sample, whereas packets are received from the physical NIC +port first and enqueued to the VM's Rx queue. Through the guest testpmd's +default forwarding mode (io forward), those packets will be put into +the Tx queue. The vhost-switch example, in turn, gets the packets and +puts back to the same physical NIC port. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vhost`` sub-directory. + +.. note:: + In this example, you need build DPDK both on the host and inside guest. + +Start the vswitch example +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vhost-switch -l 0-3 -n 4 --socket-mem 1024 \ + -- --socket-file /tmp/sock0 --client \ + ... + +Check the `Parameters`_ section for the explanations on what do those +parameters mean. + +.. _vhost_app_run_vm: + +Start the VM +~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -machine accel=kvm -cpu host \ + -m $mem -object memory-backend-file,id=mem,size=$mem,mem-path=/dev/hugepages,share=on \ + -mem-prealloc -numa node,memdev=mem \ + \ + -chardev socket,id=char1,path=/tmp/sock0,server \ + -netdev type=vhost-user,id=hostnet1,chardev=char1 \ + -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:00:00:14 \ + ... + +.. note:: + For basic vhost-user support, QEMU 2.2 (or above) is required. For + some specific features, a higher version might be need. Such as + QEMU 2.7 (or above) for the reconnect feature. + +.. _vhost_app_run_dpdk_inside_guest: + +Run testpmd inside guest +~~~~~~~~~~~~~~~~~~~~~~~~ + +Make sure you have DPDK built inside the guest. Also make sure the +corresponding virtio-net PCI device is bond to a uio driver, which +could be done by: + +.. code-block:: console + + modprobe uio_pci_generic + $RTE_SDK/usertools/dpdk-devbind.py -b=uio_pci_generic 0000:00:04.0 + +Then start testpmd for packet forwarding testing. + +.. code-block:: console + + ./x86_64-native-gcc/app/testpmd -l 0-1 -- -i + > start tx_first + +Inject packets +-------------- + +While a virtio-net is connected to vhost-switch, a VLAN tag starts with +1000 is assigned to it. So make sure configure your packet generator +with the right MAC and VLAN tag, you should be able to see following +log from the vhost-switch console. It means you get it work:: + + VHOST_DATA: (0) mac 52:54:00:00:00:14 and vlan 1000 registered + + +.. _vhost_app_parameters: + +Parameters +---------- + +**--socket-file path** +Specifies the vhost-user socket file path. + +**--client** +DPDK vhost-user will act as the client mode when such option is given. +In the client mode, QEMU will create the socket file. Otherwise, DPDK +will create it. Put simply, it's the server to create the socket file. + + +**--vm2vm mode** +The vm2vm parameter sets the mode of packet switching between guests in +the host. + +- 0 disables vm2vm, impling that VM's packets will always go to the NIC port. +- 1 means a normal mac lookup packet routing. +- 2 means hardware mode packet forwarding between guests, it allows packets + go to the NIC port, hardware L2 switch will determine which guest the + packet should forward to or need send to external, which bases on the + packet destination MAC address and VLAN tag. + +**--mergeable 0|1** +Set 0/1 to disable/enable the mergeable Rx feature. It's disabled by default. + +**--stats interval** +The stats parameter controls the printing of virtio-net device statistics. +The parameter specifies an interval (in unit of seconds) to print statistics, +with an interval of 0 seconds disabling statistics. + +**--rx-retry 0|1** +The rx-retry option enables/disables enqueue retries when the guests Rx queue +is full. This feature resolves a packet loss that is observed at high data +rates, by allowing it to delay and retry in the receive path. This option is +enabled by default. + +**--rx-retry-num num** +The rx-retry-num option specifies the number of retries on an Rx burst, it +takes effect only when rx retry is enabled. The default value is 4. + +**--rx-retry-delay msec** +The rx-retry-delay option specifies the timeout (in micro seconds) between +retries on an RX burst, it takes effect only when rx retry is enabled. The +default value is 15. + +**--dequeue-zero-copy** +Dequeue zero copy will be enabled when this option is given. it is worth to +note that if NIC is binded to driver with iommu enabled, dequeue zero copy +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu +dma mapping for guest memory. + +**--vlan-strip 0|1** +VLAN strip option is removed, because different NICs have different behaviors +when disabling VLAN strip. Such feature, which heavily depends on hardware, +should be removed from this example to reduce confusion. Now, VLAN strip is +enabled and cannot be disabled. + +**--builtin-net-driver** +A very simple vhost-user net driver which demonstrates how to use the generic +vhost APIs will be used when this option is given. It is disabled by default. + +Common Issues +------------- + +* QEMU fails to allocate memory on hugetlbfs, with an error like the + following:: + + file_ram_alloc: can't mmap RAM pages: Cannot allocate memory + + When running QEMU the above error indicates that it has failed to allocate + memory for the Virtual Machine on the hugetlbfs. This is typically due to + insufficient hugepages being free to support the allocation request. The + number of free hugepages can be checked as follows: + + .. code-block:: console + + cat /sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages + + The command above indicates how many hugepages are free to support QEMU's + allocation request. + +* Failed to build DPDK in VM + + Make sure "-cpu host" QEMU option is given. + +* Device start fails if NIC's max queues > the default number of 128 + + mbuf pool size is dependent on the MAX_QUEUES configuration, if NIC's + max queue number is larger than 128, device start will fail due to + insufficient mbuf. + + Change the default number to make it work as below, just set the number + according to the NIC's property. :: + + make EXTRA_CFLAGS="-DMAX_QUEUES=320" + +* Option "builtin-net-driver" is incompatible with QEMU + + QEMU vhost net device start will fail if protocol feature is not negotiated. + DPDK virtio-user pmd can be the replacement of QEMU. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_crypto.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_crypto.rst new file mode 100644 index 00000000..65c86a53 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_crypto.rst @@ -0,0 +1,82 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2017-2018 Intel Corporation. + +Vhost_Crypto Sample Application +=============================== + +The vhost_crypto sample application implemented a simple Crypto device, +which used as the backend of Qemu vhost-user-crypto device. Similar with +vhost-user-net and vhost-user-scsi device, the sample application used +domain socket to communicate with Qemu, and the virtio ring was processed +by vhost_crypto sample application. + +Testing steps +------------- + +This section shows the steps how to start a VM with the crypto device as +fast data path for critical application. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``examples`` sub-directory. + +Start the vhost_crypto example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vhost_crypto [EAL options] -- [--socket-file PATH] + [--cdev-id ID] [--cdev-queue-id ID] [--zero-copy] [--guest-polling] + +where, + +* socket-file PATH: the path of UNIX socket file to be created, multiple + instances of this config item is supported. Upon absence of this item, + the default socket-file `/tmp/vhost_crypto1.socket` is used. + +* cdev-id ID: the target DPDK Cryptodev's ID to process the actual crypto + workload. Upon absence of this item the default value of `0` will be used. + For details of DPDK Cryptodev, please refer to DPDK Cryptodev Library + Programmers' Guide. + +* cdev-queue-id ID: the target DPDK Cryptodev's queue ID to process the + actual crypto workload. Upon absence of this item the default value of `0` + will be used. For details of DPDK Cryptodev, please refer to DPDK Cryptodev + Library Programmers' Guide. + +* zero-copy: the presence of this item means the ZERO-COPY feature will be + enabled. Otherwise it is disabled. PLEASE NOTE the ZERO-COPY feature is still + in experimental stage and may cause the problem like segmentation fault. If + the user wants to use LKCF in the guest, this feature shall be turned off. + +* guest-polling: the presence of this item means the application assumes the + guest works in polling mode, thus will NOT notify the guest completion of + processing. + +The application requires that crypto devices capable of performing +the specified crypto operation are available on application initialization. +This means that HW crypto device/s must be bound to a DPDK driver or +a SW crypto device/s (virtual crypto PMD) must be created (using --vdev). + +.. _vhost_crypto_app_run_vm: + +Start the VM +~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -machine accel=kvm \ + -m $mem -object memory-backend-file,id=mem,size=$mem,\ + mem-path=/dev/hugepages,share=on -numa node,memdev=mem \ + -drive file=os.img,if=none,id=disk \ + -device ide-hd,drive=disk,bootindex=0 \ + -chardev socket,id={chardev_id},path={PATH} \ + -object cryptodev-vhost-user,id={obj_id},chardev={chardev_id} \ + -device virtio-crypto-pci,id={dev_id},cryptodev={obj_id} \ + ... + +.. note:: + You must check whether your Qemu can support "vhost-user-crypto" or not. diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_scsi.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_scsi.rst new file mode 100644 index 00000000..7ab7d0d5 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/vhost_scsi.rst @@ -0,0 +1,77 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2017 Intel Corporation. + +Vhost_scsi Sample Application +============================= + +The vhost_scsi sample application implemented a simple SCSI block device, +which used as the backend of Qemu vhost-user-scsi device. Users can extend +the exist example to use other type of block device(e.g. AIO) besides +memory based block device. Similar with vhost-user-net device, the sample +application used domain socket to communicate with Qemu, and the virtio +ring was processed by vhost_scsi sample application. + +The sample application reuse lots codes from SPDK(Storage Performance +Development Kit, https://github.com/spdk/spdk) vhost-user-scsi target, +for DPDK vhost library used in storage area, user can take SPDK as +reference as well. + +Testing steps +------------- + +This section shows the steps how to start a VM with the block device as +fast data path for critical application. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``examples`` sub-directory. + +You will also need to build DPDK both on the host and inside the guest + +Start the vhost_scsi example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vhost_scsi -m 1024 + +.. _vhost_scsi_app_run_vm: + +Start the VM +~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -machine accel=kvm \ + -m $mem -object memory-backend-file,id=mem,size=$mem,\ + mem-path=/dev/hugepages,share=on -numa node,memdev=mem \ + -drive file=os.img,if=none,id=disk \ + -device ide-hd,drive=disk,bootindex=0 \ + -chardev socket,id=char0,path=/tmp/vhost.socket \ + -device vhost-user-scsi-pci,chardev=char0,bootindex=2 \ + ... + +.. note:: + You must check whether your Qemu can support "vhost-user-scsi" or not, + Qemu v2.10 or newer version is required. + +Vhost_scsi Common Issues +------------------------ + +* vhost_scsi can not start with block size 512 Bytes: + + Currently DPDK vhost library was designed for NET device(althrough the APIs + are generic now), for 512 Bytes block device, Qemu BIOS(x86 BIOS Enhanced + Disk Device) will enumerate all block device and do some IOs to those block + devices with 512 Bytes sector size. DPDK vhost library can not process such + scenarios(both BIOS and OS will enumerate the block device), so as a + workaround, the vhost_scsi example application hardcoded the block size + with 4096 Bytes. + +* vhost_scsi can only support the block device as fast data disk(non OS image): + + Make sure ``bootindex=2`` Qemu option is given to vhost-user-scsi-pci device. + diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/vm_power_management.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/vm_power_management.rst new file mode 100644 index 00000000..855570d6 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/vm_power_management.rst @@ -0,0 +1,470 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +VM Power Management Application +=============================== + +Introduction +------------ + +Applications running in Virtual Environments have an abstract view of +the underlying hardware on the Host, in particular applications cannot see +the binding of virtual to physical hardware. +When looking at CPU resourcing, the pinning of Virtual CPUs(vCPUs) to +Host Physical CPUs(pCPUS) is not apparent to an application +and this pinning may change over time. +Furthermore, Operating Systems on virtual machines do not have the ability +to govern their own power policy; the Machine Specific Registers (MSRs) +for enabling P-State transitions are not exposed to Operating Systems +running on Virtual Machines(VMs). + +The Virtual Machine Power Management solution shows an example of +how a DPDK application can indicate its processing requirements using VM local +only information(vCPU/lcore, etc.) to a Host based Monitor which is responsible +for accepting requests for frequency changes for a vCPU, translating the vCPU +to a pCPU via libvirt and affecting the change in frequency. + +The solution is comprised of two high-level components: + +#. Example Host Application + + Using a Command Line Interface(CLI) for VM->Host communication channel management + allows adding channels to the Monitor, setting and querying the vCPU to pCPU pinning, + inspecting and manually changing the frequency for each CPU. + The CLI runs on a single lcore while the thread responsible for managing + VM requests runs on a second lcore. + + VM requests arriving on a channel for frequency changes are passed + to the librte_power ACPI cpufreq sysfs based library. + The Host Application relies on both qemu-kvm and libvirt to function. + + This monitoring application is responsible for: + + - Accepting requests from client applications: Client applications can + request frequency changes for a vCPU, translating + the vCPU to a pCPU via libvirt and affecting the change in frequency. + + - Accepting policies from client applications: Client application can + send a policy to the host application. The + host application will then apply the rules of the policy independent + of the application. For example, the policy can contain time-of-day + information for busy/quiet periods, and the host application can scale + up/down the relevant cores when required. See the details of the guest + application below for more information on setting the policy values. + + - Out-of-band monitoring of workloads via cores hardware event counters: + The host application can manage power for an application in a virtualised + OR non-virtualised environment by looking at the event counters of the + cores and taking action based on the branch hit/miss ratio. See the host + application '--core-list' command line parameter below. + +#. librte_power for Virtual Machines + + Using an alternate implementation for the librte_power API, requests for + frequency changes are forwarded to the host monitor rather than + the APCI cpufreq sysfs interface used on the host. + + The l3fwd-power application will use this implementation when deployed on a VM + (see :doc:`l3_forward_power_man`). + +.. _figure_vm_power_mgr_highlevel: + +.. figure:: img/vm_power_mgr_highlevel.* + + Highlevel Solution + + +Overview +-------- + +VM Power Management employs qemu-kvm to provide communications channels +between the host and VMs in the form of Virtio-Serial which appears as +a paravirtualized serial device on a VM and can be configured to use +various backends on the host. For this example each Virtio-Serial endpoint +on the host is configured as AF_UNIX file socket, supporting poll/select +and epoll for event notification. +In this example each channel endpoint on the host is monitored via +epoll for EPOLLIN events. +Each channel is specified as qemu-kvm arguments or as libvirt XML for each VM, +where each VM can have a number of channels up to a maximum of 64 per VM, +in this example each DPDK lcore on a VM has exclusive access to a channel. + +To enable frequency changes from within a VM, a request via the librte_power interface +is forwarded via Virtio-Serial to the host, each request contains the vCPU +and power command(scale up/down/min/max). +The API for host and guest librte_power is consistent across environments, +with the selection of VM or Host Implementation determined at automatically +at runtime based on the environment. + +Upon receiving a request, the host translates the vCPU to a pCPU via +the libvirt API before forwarding to the host librte_power. + +.. _figure_vm_power_mgr_vm_request_seq: + +.. figure:: img/vm_power_mgr_vm_request_seq.* + + VM request to scale frequency + + +Performance Considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +While Haswell Microarchitecture allows for independent power control for each core, +earlier Microarchtectures do not offer such fine grained control. +When deployed on pre-Haswell platforms greater care must be taken in selecting +which cores are assigned to a VM, for instance a core will not scale down +until its sibling is similarly scaled. + +Configuration +------------- + +BIOS +~~~~ + +Enhanced Intel SpeedStep® Technology must be enabled in the platform BIOS +if the power management feature of DPDK is to be used. +Otherwise, the sys file folder /sys/devices/system/cpu/cpu0/cpufreq will not exist, +and the CPU frequency-based power management cannot be used. +Consult the relevant BIOS documentation to determine how these settings +can be accessed. + +Host Operating System +~~~~~~~~~~~~~~~~~~~~~ + +The Host OS must also have the *apci_cpufreq* module installed, in some cases +the *intel_pstate* driver may be the default Power Management environment. +To enable *acpi_cpufreq* and disable *intel_pstate*, add the following +to the grub Linux command line: + +.. code-block:: console + + intel_pstate=disable + +Upon rebooting, load the *acpi_cpufreq* module: + +.. code-block:: console + + modprobe acpi_cpufreq + +Hypervisor Channel Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Virtio-Serial channels are configured via libvirt XML: + + +.. code-block:: xml + + <name>{vm_name}</name> + <controller type='virtio-serial' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> + </controller> + <channel type='unix'> + <source mode='bind' path='/tmp/powermonitor/{vm_name}.{channel_num}'/> + <target type='virtio' name='virtio.serial.port.poweragent.{vm_channel_num}'/> + <address type='virtio-serial' controller='0' bus='0' port='{N}'/> + </channel> + + +Where a single controller of type *virtio-serial* is created and up to 32 channels +can be associated with a single controller and multiple controllers can be specified. +The convention is to use the name of the VM in the host path *{vm_name}* and +to increment *{channel_num}* for each channel, likewise the port value *{N}* +must be incremented for each channel. + +Each channel on the host will appear in *path*, the directory */tmp/powermonitor/* +must first be created and given qemu permissions + +.. code-block:: console + + mkdir /tmp/powermonitor/ + chown qemu:qemu /tmp/powermonitor + +Note that files and directories within /tmp are generally removed upon +rebooting the host and the above steps may need to be carried out after each reboot. + +The serial device as it appears on a VM is configured with the *target* element attribute *name* +and must be in the form of *virtio.serial.port.poweragent.{vm_channel_num}*, +where *vm_channel_num* is typically the lcore channel to be used in DPDK VM applications. + +Each channel on a VM will be present at */dev/virtio-ports/virtio.serial.port.poweragent.{vm_channel_num}* + +Compiling and Running the Host Application +------------------------------------------ + +Compiling +~~~~~~~~~ + +For information on compiling DPDK and the sample applications +see :doc:`compiling`. + +The application is located in the ``vm_power_manager`` sub-directory. + +To build just the ``vm_power_manager`` application: + +.. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + export RTE_TARGET=build + cd ${RTE_SDK}/examples/vm_power_manager/ + make + +Running +~~~~~~~ + +The application does not have any specific command line options other than *EAL*: + +.. code-block:: console + + ./build/vm_power_mgr [EAL options] + +The application requires exactly two cores to run, one core is dedicated to the CLI, +while the other is dedicated to the channel endpoint monitor, for example to run +on cores 0 & 1 on a system with 4 memory channels: + +.. code-block:: console + + ./build/vm_power_mgr -l 0-1 -n 4 + +After successful initialization the user is presented with VM Power Manager CLI: + +.. code-block:: console + + vm_power> + +Virtual Machines can now be added to the VM Power Manager: + +.. code-block:: console + + vm_power> add_vm {vm_name} + +When a {vm_name} is specified with the *add_vm* command a lookup is performed +with libvirt to ensure that the VM exists, {vm_name} is used as an unique identifier +to associate channels with a particular VM and for executing operations on a VM within the CLI. +VMs do not have to be running in order to add them. + +A number of commands can be issued via the CLI in relation to VMs: + + Remove a Virtual Machine identified by {vm_name} from the VM Power Manager. + + .. code-block:: console + + rm_vm {vm_name} + + Add communication channels for the specified VM, the virtio channels must be enabled + in the VM configuration(qemu/libvirt) and the associated VM must be active. + {list} is a comma-separated list of channel numbers to add, using the keyword 'all' + will attempt to add all channels for the VM: + + .. code-block:: console + + add_channels {vm_name} {list}|all + + Enable or disable the communication channels in {list}(comma-separated) + for the specified VM, alternatively list can be replaced with keyword 'all'. + Disabled channels will still receive packets on the host, however the commands + they specify will be ignored. Set status to 'enabled' to begin processing requests again: + + .. code-block:: console + + set_channel_status {vm_name} {list}|all enabled|disabled + + Print to the CLI the information on the specified VM, the information + lists the number of vCPUS, the pinning to pCPU(s) as a bit mask, along with + any communication channels associated with each VM, along with the status of each channel: + + .. code-block:: console + + show_vm {vm_name} + + Set the binding of Virtual CPU on VM with name {vm_name} to the Physical CPU mask: + + .. code-block:: console + + set_pcpu_mask {vm_name} {vcpu} {pcpu} + + Set the binding of Virtual CPU on VM to the Physical CPU: + + .. code-block:: console + + set_pcpu {vm_name} {vcpu} {pcpu} + +Manual control and inspection can also be carried in relation CPU frequency scaling: + + Get the current frequency for each core specified in the mask: + + .. code-block:: console + + show_cpu_freq_mask {mask} + + Set the current frequency for the cores specified in {core_mask} by scaling each up/down/min/max: + + .. code-block:: console + + set_cpu_freq {core_mask} up|down|min|max + + Get the current frequency for the specified core: + + .. code-block:: console + + show_cpu_freq {core_num} + + Set the current frequency for the specified core by scaling up/down/min/max: + + .. code-block:: console + + set_cpu_freq {core_num} up|down|min|max + +There are also some command line parameters for enabling the out-of-band +monitoring of branch ratio on cores doing busy polling via PMDs. + + .. code-block:: console + + --core-list {list of cores} + + When this parameter is used, the list of cores specified will monitor the ratio + between branch hits and branch misses. A tightly polling PMD thread will have a + very low branch ratio, so the core frequency will be scaled down to the minimim + allowed value. When packets are received, the code path will alter, causing the + branch ratio to increase. When the ratio goes above the ratio threshold, the + core frequency will be scaled up to the maximum allowed value. + + .. code-block:: console + + --branch-ratio {ratio} + + The branch ratio is a floating point number that specifies the threshold at which + to scale up or down for the given workload. The default branch ratio is 0.01, + and will need to be adjusted for different workloads. + + +Compiling and Running the Guest Applications +-------------------------------------------- + +l3fwd-power is one sample application that can be used with vm_power_manager. + +A guest CLI is also provided for validating the setup. + +For both l3fwd-power and guest CLI, the channels for the VM must be monitored by the +host application using the *add_channels* command on the host. This typically uses +the following commands in the host application: + +.. code-block:: console + + vm_power> add_vm vmname + vm_power> add_channels vmname all + vm_power> set_channel_status vmname all enabled + vm_power> show_vm vmname + + +Compiling +~~~~~~~~~ + +For information on compiling DPDK and the sample applications +see :doc:`compiling`. + +For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`. + +The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``. + +To build just the ``guest_vm_power_manager`` application: + +.. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + export RTE_TARGET=build + cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/ + make + +Running +~~~~~~~ + +The standard *EAL* command line parameters are required: + +.. code-block:: console + + ./build/guest_vm_power_mgr [EAL options] -- [guest options] + +The guest example uses a channel for each lcore enabled. For example, +to run on cores 0,1,2,3: + +.. code-block:: console + + ./build/guest_vm_power_mgr -l 0-3 + +Optionally, there is a list of command line parameter should the user wish to send a power +policy down to the host application. These parameters are as follows: + + .. code-block:: console + + --vm-name {name of guest vm} + + This parameter allows the user to change the Virtual Machine name passed down to the + host application via the power policy. The default is "ubuntu2" + + .. code-block:: console + + --vcpu-list {list vm cores} + + A comma-separated list of cores in the VM that the user wants the host application to + monitor. The list of cores in any vm starts at zero, and these are mapped to the + physical cores by the host application once the policy is passed down. + Valid syntax includes individial cores '2,3,4', or a range of cores '2-4', or a + combination of both '1,3,5-7' + + .. code-block:: console + + --busy-hours {list of busy hours} + + A comma-separated list of hours within which to set the core frequency to maximum. + Valid syntax includes individial hours '2,3,4', or a range of hours '2-4', or a + combination of both '1,3,5-7'. Valid hours are 0 to 23. + + .. code-block:: console + + --quiet-hours {list of quiet hours} + + A comma-separated list of hours within which to set the core frequency to minimum. + Valid syntax includes individial hours '2,3,4', or a range of hours '2-4', or a + combination of both '1,3,5-7'. Valid hours are 0 to 23. + + .. code-block:: console + + --policy {policy type} + + The type of policy. This can be one of the following values: + TRAFFIC - based on incoming traffic rates on the NIC. + TIME - busy/quiet hours policy. + BRANCH_RATIO - uses branch ratio counters to determine core busyness. + Not all parameters are needed for all policy types. For example, BRANCH_RATIO + only needs the vcpu-list parameter, not any of the hours. + + +After successful initialization the user is presented with VM Power Manager Guest CLI: + +.. code-block:: console + + vm_power(guest)> + +To change the frequency of a lcore, use the set_cpu_freq command. +Where {core_num} is the lcore and channel to change frequency by scaling up/down/min/max. + +.. code-block:: console + + set_cpu_freq {core_num} up|down|min|max + +To start the application and configure the power policy, and send it to the host: + +.. code-block:: console + + ./build/guest_vm_power_mgr -l 0-3 -n 4 -- --vm-name=ubuntu --policy=BRANCH_RATIO --vcpu-list=2-4 + +Once the VM Power Manager Guest CLI appears, issuing the 'send_policy now' command +will send the policy to the host: + +.. code-block:: console + + send_policy now + +Once the policy is sent to the host, the host application takes over the power monitoring +of the specified cores in the policy. + diff --git a/src/spdk/dpdk/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst b/src/spdk/dpdk/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst new file mode 100644 index 00000000..0e9da970 --- /dev/null +++ b/src/spdk/dpdk/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst @@ -0,0 +1,266 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation. + +VMDQ and DCB Forwarding Sample Application +========================================== + +The VMDQ and DCB Forwarding sample application is a simple example of packet processing using the DPDK. +The application performs L2 forwarding using VMDQ and DCB to divide the incoming traffic into queues. +The traffic splitting is performed in hardware by the VMDQ and DCB features of the Intel® 82599 and X710/XL710 Ethernet Controllers. + +Overview +-------- + +This sample application can be used as a starting point for developing a new application that is based on the DPDK and +uses VMDQ and DCB for traffic partitioning. + +The VMDQ and DCB filters work on MAC and VLAN traffic to divide the traffic into input queues on the basis of the Destination MAC +address, VLAN ID and VLAN user priority fields. +VMDQ filters split the traffic into 16 or 32 groups based on the Destination MAC and VLAN ID. +Then, DCB places each packet into one of queues within that group, based upon the VLAN user priority field. + +All traffic is read from a single incoming port (port 0) and output on port 1, without any processing being performed. +With Intel® 82599 NIC, for example, the traffic is split into 128 queues on input, where each thread of the application reads from +multiple queues. When run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. + +As supplied, the sample application configures the VMDQ feature to have 32 pools with 4 queues each as indicated in :numref:`figure_vmdq_dcb_example`. +The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 16 pools of 8 queues. While the +Intel® X710 or XL710 Ethernet Controller NICs support many configurations of VMDQ pools of 4 or 8 queues each. For simplicity, only 16 +or 32 pools is supported in this sample. And queues numbers for each VMDQ pool can be changed by setting CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM +in config/common_* file. +The nb-pools, nb-tcs and enable-rss parameters can be passed on the command line, after the EAL parameters: + +.. code-block:: console + + ./build/vmdq_dcb [EAL options] -- -p PORTMASK --nb-pools NP --nb-tcs TC --enable-rss + +where, NP can be 16 or 32, TC can be 4 or 8, rss is disabled by default. + +.. _figure_vmdq_dcb_example: + +.. figure:: img/vmdq_dcb_example.* + + Packet Flow Through the VMDQ and DCB Sample Application + + +In Linux* user space, the application can display statistics with the number of packets received on each queue. +To have the application display the statistics, send a SIGHUP signal to the running application process. + +The VMDQ and DCB Forwarding sample application is in many ways simpler than the L2 Forwarding application +(see :doc:`l2_forward_real_virtual`) +as it performs unidirectional L2 forwarding of packets from one port to a second port. +No command-line options are taken by this application apart from the standard EAL command-line options. + +.. note:: + + Since VMD queues are being used for VMM, this application works correctly + when VTd is disabled in the BIOS or Linux* kernel (intel_iommu=off). + +Compiling the Application +------------------------- + + + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vmdq_dcb`` sub-directory. + +Running the Application +----------------------- + +To run the example in a linuxapp environment: + +.. code-block:: console + + user@target:~$ ./build/vmdq_dcb -l 0-3 -n 4 -- -p 0x3 --nb-pools 32 --nb-tcs 4 + +Refer to the *DPDK Getting Started Guide* for general information on running applications and +the Environment Abstraction Layer (EAL) options. + +Explanation +----------- + +The following sections provide some explanation of the code. + +Initialization +~~~~~~~~~~~~~~ + +The EAL, driver and PCI configuration is performed largely as in the L2 Forwarding sample application, +as is the creation of the mbuf pool. +See :doc:`l2_forward_real_virtual`. +Where this example application differs is in the configuration of the NIC port for RX. + +The VMDQ and DCB hardware feature is configured at port initialization time by setting the appropriate values in the +rte_eth_conf structure passed to the rte_eth_dev_configure() API. +Initially in the application, +a default structure is provided for VMDQ and DCB configuration to be filled in later by the application. + +.. code-block:: c + + /* empty vmdq+dcb configuration structure. Filled in programmatically */ + static const struct rte_eth_conf vmdq_dcb_conf_default = { + .rxmode = { + .mq_mode = ETH_MQ_RX_VMDQ_DCB, + .split_hdr_size = 0, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_VMDQ_DCB, + }, + /* + * should be overridden separately in code with + * appropriate values + */ + .rx_adv_conf = { + .vmdq_dcb_conf = { + .nb_queue_pools = ETH_32_POOLS, + .enable_default_pool = 0, + .default_pool = 0, + .nb_pool_maps = 0, + .pool_map = {{0, 0},}, + .dcb_tc = {0}, + }, + .dcb_rx_conf = { + .nb_tcs = ETH_4_TCS, + /** Traffic class each UP mapped to. */ + .dcb_tc = {0}, + }, + .vmdq_rx_conf = { + .nb_queue_pools = ETH_32_POOLS, + .enable_default_pool = 0, + .default_pool = 0, + .nb_pool_maps = 0, + .pool_map = {{0, 0},}, + }, + }, + .tx_adv_conf = { + .vmdq_dcb_tx_conf = { + .nb_queue_pools = ETH_32_POOLS, + .dcb_tc = {0}, + }, + }, + }; + +The get_eth_conf() function fills in an rte_eth_conf structure with the appropriate values, +based on the global vlan_tags array, +and dividing up the possible user priority values equally among the individual queues +(also referred to as traffic classes) within each pool. With Intel® 82599 NIC, +if the number of pools is 32, then the user priority fields are allocated 2 to a queue. +If 16 pools are used, then each of the 8 user priority fields is allocated to its own queue within the pool. +With Intel® X710/XL710 NICs, if number of tcs is 4, and number of queues in pool is 8, +then the user priority fields are allocated 2 to one tc, and a tc has 2 queues mapping to it, then +RSS will determine the destination queue in 2. +For the VLAN IDs, each one can be allocated to possibly multiple pools of queues, +so the pools parameter in the rte_eth_vmdq_dcb_conf structure is specified as a bitmask value. +For destination MAC, each VMDQ pool will be assigned with a MAC address. In this sample, each VMDQ pool +is assigned to the MAC like 52:54:00:12:<port_id>:<pool_id>, that is, +the MAC of VMDQ pool 2 on port 1 is 52:54:00:12:01:02. + +.. code-block:: c + + const uint16_t vlan_tags[] = { + 0, 1, 2, 3, 4, 5, 6, 7, + 8, 9, 10, 11, 12, 13, 14, 15, + 16, 17, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 30, 31 + }; + + /* pool mac addr template, pool mac addr is like: 52 54 00 12 port# pool# */ + static struct ether_addr pool_addr_template = { + .addr_bytes = {0x52, 0x54, 0x00, 0x12, 0x00, 0x00} + }; + + /* Builds up the correct configuration for vmdq+dcb based on the vlan tags array + * given above, and the number of traffic classes available for use. */ + static inline int + get_eth_conf(struct rte_eth_conf *eth_conf) + { + struct rte_eth_vmdq_dcb_conf conf; + struct rte_eth_vmdq_rx_conf vmdq_conf; + struct rte_eth_dcb_rx_conf dcb_conf; + struct rte_eth_vmdq_dcb_tx_conf tx_conf; + uint8_t i; + + conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; + vmdq_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; + tx_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; + conf.nb_pool_maps = num_pools; + vmdq_conf.nb_pool_maps = num_pools; + conf.enable_default_pool = 0; + vmdq_conf.enable_default_pool = 0; + conf.default_pool = 0; /* set explicit value, even if not used */ + vmdq_conf.default_pool = 0; + + for (i = 0; i < conf.nb_pool_maps; i++) { + conf.pool_map[i].vlan_id = vlan_tags[i]; + vmdq_conf.pool_map[i].vlan_id = vlan_tags[i]; + conf.pool_map[i].pools = 1UL << i ; + vmdq_conf.pool_map[i].pools = 1UL << i; + } + for (i = 0; i < ETH_DCB_NUM_USER_PRIORITIES; i++){ + conf.dcb_tc[i] = i % num_tcs; + dcb_conf.dcb_tc[i] = i % num_tcs; + tx_conf.dcb_tc[i] = i % num_tcs; + } + dcb_conf.nb_tcs = (enum rte_eth_nb_tcs)num_tcs; + (void)(rte_memcpy(eth_conf, &vmdq_dcb_conf_default, sizeof(*eth_conf))); + (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_dcb_conf, &conf, + sizeof(conf))); + (void)(rte_memcpy(ð_conf->rx_adv_conf.dcb_rx_conf, &dcb_conf, + sizeof(dcb_conf))); + (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_rx_conf, &vmdq_conf, + sizeof(vmdq_conf))); + (void)(rte_memcpy(ð_conf->tx_adv_conf.vmdq_dcb_tx_conf, &tx_conf, + sizeof(tx_conf))); + if (rss_enable) { + eth_conf->rxmode.mq_mode= ETH_MQ_RX_VMDQ_DCB_RSS; + eth_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP | + ETH_RSS_UDP | + ETH_RSS_TCP | + ETH_RSS_SCTP; + } + return 0; + } + + ...... + + /* Set mac for each pool.*/ + for (q = 0; q < num_pools; q++) { + struct ether_addr mac; + mac = pool_addr_template; + mac.addr_bytes[4] = port; + mac.addr_bytes[5] = q; + printf("Port %u vmdq pool %u set mac %02x:%02x:%02x:%02x:%02x:%02x\n", + port, q, + mac.addr_bytes[0], mac.addr_bytes[1], + mac.addr_bytes[2], mac.addr_bytes[3], + mac.addr_bytes[4], mac.addr_bytes[5]); + retval = rte_eth_dev_mac_addr_add(port, &mac, + q + vmdq_pool_base); + if (retval) { + printf("mac addr add failed at pool %d\n", q); + return retval; + } + } + +Once the network port has been initialized using the correct VMDQ and DCB values, +the initialization of the port's RX and TX hardware rings is performed similarly to that +in the L2 Forwarding sample application. +See :doc:`l2_forward_real_virtual` for more information. + +Statistics Display +~~~~~~~~~~~~~~~~~~ + +When run in a linuxapp environment, +the VMDQ and DCB Forwarding sample application can display statistics showing the number of packets read from each RX queue. +This is provided by way of a signal handler for the SIGHUP signal, +which simply prints to standard output the packet counts in grid form. +Each row of the output is a single pool with the columns being the queue number within that pool. + +To generate the statistics output, use the following command: + +.. code-block:: console + + user@host$ sudo killall -HUP vmdq_dcb_app + +Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, +rather than the terminal from which the HUP signal was sent. |
