diff options
Diffstat (limited to 'doc')
279 files changed, 76506 insertions, 0 deletions
diff --git a/doc/.gitignore b/doc/.gitignore new file mode 100644 index 0000000..fa2b508 --- /dev/null +++ b/doc/.gitignore @@ -0,0 +1,21 @@ +mdate-sh +draft-zebra-00.txt +*.pdf +*.eps +frr.ps +frr.dvi +stamp-vti +*.aux +*.cp +*.cps +*.fn +*.fns +*.ky +*.kys +*.log +*.op +*.pg +*.toc +*.tp +*.vr +refix diff --git a/doc/Makefile b/doc/Makefile new file mode 100644 index 0000000..3b4d3d2 --- /dev/null +++ b/doc/Makefile @@ -0,0 +1,18 @@ +all: ALWAYS + @$(MAKE) -s -C .. doc +%: ALWAYS + @$(MAKE) -s -C .. doc/$@ +html: + @$(MAKE) -s -C .. doc/user/_build/html/.buildinfo +info: + @$(MAKE) -s -C .. doc/user/_build/texinfo/frr.info +pdf: + @$(MAKE) -s -C .. doc/user/_build/latexpdf +frr.info: info +frr.pdf: pdf + +Makefile: + #nothing +ALWAYS: +.PHONY: ALWAYS makefiles html info frr.info pdf frr.pdf +.SUFFIXES: diff --git a/doc/accords/README.md b/doc/accords/README.md new file mode 100644 index 0000000..0461b04 --- /dev/null +++ b/doc/accords/README.md @@ -0,0 +1,32 @@ +FRR accords +=========== + + +This directory contains some text documents with "accords" agreed upon by the +FRR community. The idea here is that by passing text documents through the +PR/review process, consensus for work items, design decisions, etc. can be +established and documented. They can also be changed later by followup PRs if +consensus shifts. This is intended to reduce friction, and provide more +transparency for newcomers & less frequent contributors. + +Examples of things that could go here: + +- agreement on how to fix some deeper-seated existing shortcoming in FRR that + might take some time to fix, to get consensus before putting time into it. + +- larger design (especially system/package integration) decisions that are not + immediately tangible to the code. + +- scoping decisions, particularly negative (i.e. we decided at some point that + FRR is not the right place for something) - these are otherwise lost in some + ancient closed PR, and some new contributor might be unaware and waste time. + +Files in this directory are not formatted in any specific way and not rendered +into documentation. They're intended to be read with your code editor of +choice. + +To avoid misunderstandings, there is one "rule" about wording: the consensus +actual is worded with "will", "going to", "is" - this reflects the idea that +when the PR is merged, it *is* community consensus. Words like "should", +"would" or "might" should be limited to context and reference that is provided +as rationale for the consensus. diff --git a/doc/accords/cli-colors b/doc/accords/cli-colors new file mode 100644 index 0000000..04bdfc7 --- /dev/null +++ b/doc/accords/cli-colors @@ -0,0 +1,44 @@ +Adding colors to FRR CLI output +=============================== + + +There were multiple approaches/attempts to get colored output for the CLI into +FRR, most recently End of 2022 in PR #12497. After some discussion, some items +crystallized out: + +First, generally speaking, colors (or other rich output formatting) must be +used sparingly. In particular, e.g. "every IP address" is not something to +color. The output formatting needs to have an actual purpose to improve UX, +not turn it into a christmas tree. + +In the long run, the CLI will hopefully become a YANG frontend. In that case, +the CLI frontend component is a great place to apply all kinds of output/UI/UX +features. However, this is a long way off. + +That said, an implementation in the current vtysh+daemon ecosystem is not out +of the question, especially if the use of colors/formatting is limited to +important places (which is desirable anyway - see general statement above.) +We don't want to litter formatting all over every single vty_out call. + +A color option on a per-command/DEFUN level (i.e. the way `[json]` is done) was +rejected. The decision to color output must take information from vtysh's +environment into account, notably the TERM environment variable, the NO_COLOR +environment variable, and whether stdout is a terminal or not. An explicit +`--color` switch (or `terminal color` vtysh command, or other similar things) +is needed too. To be clear, the switch must not be on individual commands, it +needs to be on the vtysh session level. + +Lastly, the output pager needs to work with this. + + +Suggested implementation +------------------------ + +(not part of the consensus / accord, only to record discussion) + +As far as discussion went, the most promising approach to actually implement +this is to put some type of unconditional formatting tag into daemon's vty_out +calls. This would be some escape-like sequence - an actual ANSI color code +itself is not particularly readable or pretty, though that would work as well. +vtysh would then, while passing through the output from the daemons, replace or +remove these tags according to terminal/user settings. diff --git a/doc/accords/frr-service-is-watchfrr b/doc/accords/frr-service-is-watchfrr new file mode 100644 index 0000000..2301c83 --- /dev/null +++ b/doc/accords/frr-service-is-watchfrr @@ -0,0 +1,16 @@ +The "FRR" service unit is watchfrr +================================== + + +"FRR" on the distribution/OS level is one service (generally called "frr"). +Exposing individual daemons (zebra, staticd, bgpd, ...) as service units does +not match FRR's internal expectations. + +At some future point, watchfrr will add functionality to receive "router bgp", +"router ospf" etc. commands (or their YANG variants) and automatically start +the required daemons. In particular with multi-instance setups, this will +simplify config (no more mucking around /etc/frr/daemons - if watchfrr +understands which daemons are needed by a given config, the daemons file is +pointless.) + +This to some degree assumes an "integrated-config world". diff --git a/doc/accords/integrated-config-wins b/doc/accords/integrated-config-wins new file mode 100644 index 0000000..5a02b99 --- /dev/null +++ b/doc/accords/integrated-config-wins @@ -0,0 +1,10 @@ +Integrated config wins +====================== + + +The use of split-configuration setups (zebra.conf, staticd.conf, bgpd.conf, +etc.) in FRR is considered deprecated and will go away at some point. + +At this point there is no timeline yet on removing split-config support, and +this needs to go through an extensive deprecation period with increasingly +loud user warnings. diff --git a/doc/developer/.gitignore b/doc/developer/.gitignore new file mode 100644 index 0000000..81c60dc --- /dev/null +++ b/doc/developer/.gitignore @@ -0,0 +1,2 @@ +/_templates +/_build diff --git a/doc/developer/.readthedocs.yaml b/doc/developer/.readthedocs.yaml new file mode 100644 index 0000000..113672f --- /dev/null +++ b/doc/developer/.readthedocs.yaml @@ -0,0 +1,12 @@ +# Required +version: 2 + +# Set the version of Python and other tools you might need +build: + os: ubuntu-22.04 + tools: + python: "3.11" + +# Build documentation in the docs/ directory with Sphinx +sphinx: + configuration: doc/developer/conf.py diff --git a/doc/developer/MLD-and-PIMv6-Design.png b/doc/developer/MLD-and-PIMv6-Design.png Binary files differnew file mode 100644 index 0000000..b5066de --- /dev/null +++ b/doc/developer/MLD-and-PIMv6-Design.png diff --git a/doc/developer/Makefile b/doc/developer/Makefile new file mode 100644 index 0000000..38afb43 --- /dev/null +++ b/doc/developer/Makefile @@ -0,0 +1,16 @@ +all: ALWAYS + @$(MAKE) -s -C ../.. developer-html +help: ALWAYS + @$(MAKE) -s -C ../.. doc/help +pdf: ALWAYS + @$(MAKE) -s -C ../.. doc/developer/_build/latexpdf +info: ALWAYS + @$(MAKE) -s -C ../.. doc/developer/_build/texinfo/frr.info +%: ALWAYS + @$(MAKE) -s -C ../.. doc/developer/_build/$@ + +Makefile: + #nothing +ALWAYS: +.PHONY: ALWAYS makefiles +.SUFFIXES: diff --git a/doc/developer/PIMv6-Design.pptx b/doc/developer/PIMv6-Design.pptx Binary files differnew file mode 100644 index 0000000..fc17059 --- /dev/null +++ b/doc/developer/PIMv6-Design.pptx diff --git a/doc/developer/_static/overrides.css b/doc/developer/_static/overrides.css new file mode 100644 index 0000000..302b8d6 --- /dev/null +++ b/doc/developer/_static/overrides.css @@ -0,0 +1,255 @@ +/* remove max-width restriction */ +div.body { + max-width: none; +} + +/* Palette URL: http://paletton.com/#uid=70p0p0kt6uvcDRAlhBavokxLJ6w */ + +:root { +--primary-0: #F36F16; /* Main Primary color */ +--primary-1: #FFC39A; +--primary-2: #FF9A55; +--primary-3: #A34403; +--primary-4: #341500; +--primary-9: #FFF3EB; + +--secondary-1-0: #F39C16; /* Main Secondary color (1) */ +--secondary-1-1: #FFD79A; +--secondary-1-2: #FFBC55; +--secondary-1-3: #A36403; +--secondary-1-4: #341F00; +--secondary-1-9: #FFF7EB; + +--secondary-2-0: #1A599F; /* Main Secondary color (2) */ +--secondary-2-1: #92B9E5; +--secondary-2-2: #477CB8; +--secondary-2-3: #0A386B; +--secondary-2-4: #011122; +--secondary-2-9: #E3EBF4; + +--complement-0: #0E9A83; /* Main Complement color */ +--complement-1: #8AE4D4; +--complement-2: #3CB4A0; +--complement-3: #026857; +--complement-4: #00211B; +--complement-9: #E0F4F0; +} + +/* new */ + +body { + font-family: "Fira Sans", Helvetica, Arial, sans-serif; + font-weight:400; +} +h1, h2, h3, h4, h5, h6 { + font-family: "Fira Sans", Helvetica, Arial, sans-serif; + font-weight:500; +} +code, pre, tt { + font-family: "Fira Mono"; +} +h1 { + background-color:var(--secondary-1-1); + border-bottom:1px solid var(--secondary-1-0); + font-weight:300; +} +h2 { + margin-top:36pt; +} + +a, +a:hover, +a:visited, +.code-block-caption a.headerlink:hover, +.rst-content dl:not(.docutils) dt .headerlink { + color: var(--complement-0); +} +.code-block-caption a.headerlink { + visibility:hidden; +} + +/* admonitions */ + +.admonition.warning { + border:1px dashed var(--primary-2); +} +.admonition.warning .admonition-title { + color: var(--primary-3); + background-color: var(--primary-1); +} +.admonition.note, +.admonition.hint { + border:1px dashed var(--complement-2); +} +.admonition.note .admonition-title, +.admonition.hint .admonition-title { + color: var(--complement-3); + background-color: var(--complement-1); +} +.admonition.seealso, +div.seealso { + background-color:var(--complement-9); +} +.admonition.seealso .admonition-title { + color: var(--complement-3); + background-color:var(--complement-1); + border-bottom:1px solid var(--complement-2); +} +.admonition.admonition-todo .admonition-title { + background-image: repeating-linear-gradient( + 135deg, + #ffa, + #ffa 14.14213452px, + #bbb 14.14213452px, + #bbb 28.28427124px + ); + color:#000; +} +.admonition.admonition-todo { + background-image: repeating-linear-gradient( + 135deg, + #ffd, + #ffd 14.14213452px, + #eed 14.14213452px, + #eed 28.28427124px + ); +} + +.rst-content dl .admonition p.last { + margin-bottom:0 !important; +} + +/* file block */ + +.code-block-caption { +/* border-radius: 4px; */ + font-style:italic; + font-weight:300; + border-bottom: 1px solid var(--secondary-2-1); + background-color: var(--secondary-2-9); + padding:2px 8px; +} + +/* navbar */ + +.wy-nav-side { + background-color: var(--secondary-1-4); + border-right:2px solid var(--primary-3); +} +.wy-menu-vertical a, +.wy-menu-vertical a:visited, +.wy-menu-vertical a:hover, +.wy-side-nav-search>a, +.wy-side-nav-search .wy-dropdown>a { + color: var(--primary-0); +} + +nav div.wy-side-nav-search { + background-color: #eee; +} +nav div.wy-side-scroll { + background-color: var(--secondary-1-4); +} +nav .wy-menu-vertical a:hover { + background-color:var(--primary-0); + color:var(--primary-4); +} +nav .wy-menu-vertical li.current ul a:hover { + background-color:var(--secondary-1-2); + color:var(--primary-4); +} +nav .wy-menu-vertical li.current ul a { + background-color:var(--secondary-1-1); + color:var(--primary-3); +} +nav .wy-menu-vertical li.on a:hover, +nav .wy-menu-vertical li.current>a:hover { + background-color:#fcfcfc; +} +.wy-side-nav-search input[type=text] { + border-color:var(--primary-2); +} +.wy-menu-vertical li.toctree-l1.current>a { + border-top:1px solid var(--secondary-1-3); + border-bottom:1px solid var(--secondary-1-3); +} +.wy-menu-vertical li.toctree-l2.current>a { + background-color:var(--secondary-1-2); +} +.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a { + background-color:var(--secondary-1-9); +} + +.wy-nav-content { + padding: 25pt 40pt; +} +div[role=navigation] > hr { + display:none; +} +div[role=navigation] { + margin-bottom:15pt; +} +h1 { + margin-left:-40pt; + margin-right:-40pt; + padding:5pt 40pt 5pt 40pt; +} + +.rst-content pre.literal-block, .rst-content div[class^='highlight'] { + border-color:var(--secondary-1-1); +} + +span.pre { + color: var(--complement-3); +} +pre { + background-color: var(--secondary-1-9); + border-color: var(--secondary-1-1); +} +.highlight .p { color: var(--secondary-2-3); } +.highlight .k { color: var(--secondary-2-0); } +.highlight .kt { color: var(--complement-0); } +.highlight .cm { color: var(--primary-3); } +.highlight .ow { color: var(--primary-3); } +.highlight .na { color: var(--primary-2); } +.highlight .nv { color: var(--complement-0); } + +.rst-content code.frrfmtout { + background-color: var(--secondary-1-9); + border-color: var(--secondary-1-1); + font-size:100%; +} +.rst-content code.frrfmtout::before { + content: "⇒ \""; +} +.rst-content code.frrfmtout::after { + content: "\""; +} +.rst-content code.frrfmtout span { + color: var(--secondary-1-4); + font-size:100%; +} + +strong { + font-weight:500; +} +.rst-content dl:not(.docutils) dt { + font-family:Fira Mono; + font-weight:600; + background-color:var(--secondary-2-9); + color:var(--secondary-2-3); + border-top:2px solid var(--secondary-2-2); +} +dt code.descname { + color: var(--secondary-2-4); +} + +@media (min-width: 1200px) { + .container { width: auto; } +} +@media (min-width: 992px) { + .container { width: auto; } +} +@media (min-width: 768px) { + .container { width: auto; } +} diff --git a/doc/developer/bgp-typecodes.rst b/doc/developer/bgp-typecodes.rst new file mode 100644 index 0000000..c7921a7 --- /dev/null +++ b/doc/developer/bgp-typecodes.rst @@ -0,0 +1,25 @@ +BGP-4[+] UPDATE Attribute Preprocessor Constants +================================================ + +This is a list of preprocessor constants that map to BGP attributes defined by +various BGP RFCs. In the code these are defined as BGP_ATTR_<ATTR>. + ++-------+------------------+------------------------------------------+ +| Value | Attribute | References | ++=======+==================+==========================================+ +| 1 | ORIGIN | [RFC 4271] | +| 2 | AS_PATH | [RFC 4271] | +| 3 | NEXT_HOP | [RFC 4271] | +| 4 | MULTI_EXIT_DISC | [RFC 4271] | +| 5 | LOCAL_PREF | [RFC 4271] | +| 6 | ATOMIC_AGGREGATE | [RFC 4271] | +| 7 | AGGREGATOR | [RFC 4271] | +| 8 | COMMUNITIES | [RFC 1997] | +| 9 | ORIGINATOR_ID | [RFC 4456] | +| 10 | CLUSTER_LIST | [RFC 4456] | +| 14 | MP_REACH_NLRI | [RFC 4760] | +| 15 | MP_UNREACH_NLRI | [RFC 4760] | +| 16 | EXT_COMMUNITIES | [RFC 4360] | +| 17 | AS4_PATH | [RFC 4893] | +| 18 | AS4_AGGREGATOR | [RFC 4893] | ++-------+------------------+------------------------------------------+ diff --git a/doc/developer/bgpd.rst b/doc/developer/bgpd.rst new file mode 100644 index 0000000..a35fa61 --- /dev/null +++ b/doc/developer/bgpd.rst @@ -0,0 +1,11 @@ +.. _bgpd: + +**** +BGPD +**** + +.. toctree:: + :maxdepth: 2 + + next-hop-tracking + bgp-typecodes diff --git a/doc/developer/building-docker.rst b/doc/developer/building-docker.rst new file mode 100644 index 0000000..9d42784 --- /dev/null +++ b/doc/developer/building-docker.rst @@ -0,0 +1,204 @@ +Docker +====== + +This page covers how to build FRR Docker images. + +Images +"""""" +FRR has Docker build infrastructure to produce Docker images containing +source-built FRR on the following base platforms: + +* Alpine +* Centos 7 +* Centos 8 + +The following platform images are used to support Travis CI and can also +be used to reproduce topotest failures when the docker host is Ubuntu +(tested on 18.04 and 20.04): + +* Ubuntu 20.04 +* Ubuntu 22.04 + +The following platform images may also be built, but these simply install a +binary package from an existing repository and do not perform source builds: + +* Debian 10 + +Some of these are available on `DockerHub +<https://hub.docker.com/repository/docker/frrouting/frr/tags?page=1>`_. + +There is no guarantee on what is and is not available from DockerHub at time of +writing. + +Scripts +""""""" + +Some platforms contain an included build script that may be run from the host. +This will set appropriate packaging environment variables and clean up +intermediate build images. + +These scripts serve another purpose. They allow building platform packages +without needing the platform. For example, the Centos 8 docker image can also +be leveraged to build Centos 8 RPMs that can then be used separately from +Docker. + +If you are only interested in the Docker images and don't want the cleanup +functionality of the scripts you can ignore them and perform a normal Docker +build. If you want to build multi-arch docker images this is required as the +scripts do not support using Buildkit for multi-arch builds. + +Building Alpine Image +--------------------- + +Script:: + + ./docker/alpine/build.sh + +No script:: + + docker build -f docker/alpine/Dockerfile . + +No script, multi-arch (ex. amd64, arm64, armv7):: + + docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -f docker/alpine/Dockerfile -t frr:latest . + + +Building Debian Image +--------------------- + +:: + + cd docker/debian + docker build . + +Multi-arch (ex. amd64, arm64, armv7):: + + cd docker/debian + docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t frr-debian:latest . + +Building Centos 7 Image +----------------------- + +Script:: + + ./docker/centos-7/build.sh + +No script:: + + docker build -f docker/centos-7/Dockerfile . + +No script, multi-arch (ex. amd64, arm64):: + + docker buildx build --platform linux/amd64,linux/arm64 -f docker/centos-7/Dockerfile -t frr-centos7:latest . + + +Building Centos 8 Image +----------------------- + +Script:: + + ./docker/centos-8/build.sh + +No script:: + + docker build -f docker/centos-8/Dockerfile . + +No script, multi-arch (ex. amd64, arm64):: + + docker buildx build --platform linux/amd64,linux/arm64 -f docker/centos-8/Dockerfile -t frr-centos8:latest . + + + +Building ubi 8 Image +----------------------- + +Script:: + + ./docker/ubi-8/build.sh + +Script with params, an example could be this (all that info will go to docker label) :: + + ./docker/ubi-8/build.sh frr:ubi-8-my-test "$(git rev-parse --short=10 HEAD)" my_release my_name my_vendor + +No script:: + + docker build -f docker/ubi-8/Dockerfile . + +No script, multi-arch (ex. amd64, arm64):: + + docker buildx build --platform linux/amd64,linux/arm64 -f docker/ubi-8/Dockerfile -t frr-ubi-8:latest . + + + +Building Ubuntu 20.04 Image +--------------------------- + +Build image (from project root directory):: + + docker build -t frr-ubuntu20:latest --build-arg=UBUNTU_VERSION=20.04 -f docker/ubuntu-ci/Dockerfile . + +Running Full Topotest:: + + docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \ + frr-ubuntu20:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile' + +Extract results from the above run into `run-results` dir and analyze:: + + tests/topotest/analyze.py -C frr -Ar run-results + +Start the container:: + + docker run -d --init --privileged --name frr-ubuntu20 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu20:latest + +Running a topotest (when the docker host is Ubuntu):: + + docker exec frr-ubuntu20 bash -c 'cd ~/frr/tests/topotests/ospf_topo1 ; sudo pytest test_ospf_topo1.py' + +Starting an interactive bash session:: + + docker exec -it frr-ubuntu20 bash + +Stopping an removing a container:: + + docker stop frr-ubuntu20 ; docker rm frr-ubuntu20 + +Removing the built image:: + + docker rmi frr-ubuntu20:latest + + +Building Ubuntu 22.04 Image +--------------------------- + +Build image (from project root directory):: + + docker build -t frr-ubuntu22:latest -f docker/ubuntu-ci/Dockerfile . + +Running Full Topotest:: + + docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \ + frr-ubuntu22:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile' + +Extract results from the above run into `run-results` dir and analyze:: + + tests/topotest/analyze.py -C frr -Ar run-results + +Start the container:: + + docker run -d --init --privileged --name frr-ubuntu22 --mount type=bind,source=/lib/modules,target=/lib/modules frr-ubuntu22:latest + +Running a topotest (when the docker host is Ubuntu):: + + docker exec frr-ubuntu22 bash -c 'cd ~/frr/tests/topotests/ospf_topo1 ; sudo pytest test_ospf_topo1.py' + +Starting an interactive bash session:: + + docker exec -it frr-ubuntu22 bash + +Stopping an removing a container:: + + docker stop frr-ubuntu22 ; docker rm frr-ubuntu22 + +Removing the built image:: + + docker rmi frr-ubuntu22:latest diff --git a/doc/developer/building-frr-for-alpine.rst b/doc/developer/building-frr-for-alpine.rst new file mode 100644 index 0000000..68e58c9 --- /dev/null +++ b/doc/developer/building-frr-for-alpine.rst @@ -0,0 +1,109 @@ +Alpine Linux 3.7+ +========================================================= + +For building Alpine Linux dev packages, we use docker. + +Install docker 17.05 or later +----------------------------- + +Depending on your host, there are different ways of installing docker. Refer +to the documentation here for instructions on how to install a free version of +docker: https://www.docker.com/community-edition + +Pre-built packages and docker images +------------------------------------ + +The master branch of https://github.com/frrouting/frr.git has a +continuous delivery of docker images to docker hub at: +https://hub.docker.com/r/ajones17/frr/. These images have the frr packages +in /pkgs/apk and have the frr package pre-installed. To copy Alpine +packages out of these images: + +:: + + id=`docker create ajones17/frr:latest` + docker cp ${id}:/pkgs _some_directory_ + docker rm $id + +To run the frr daemons (see below for how to configure them): + +:: + + docker run -it --rm --name frr ajones17/frr:latest + docker exec -it frr /bin/sh + +Work with sources +----------------- + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + +Build apk packages +------------------ + +:: + + ./docker/alpine/build.sh + +This will put the apk packages in: + +:: + + ./docker/pkgs/apk/x86_64/ + +Usage +----- + +To create a base image with the frr packages installed: + +:: + + docker build --rm -f docker/alpine/Dockerfile -t frr:latest . + +Or, if you don't have a git checkout of the sources, you can build a base +image directly off the github account: + +:: + + docker build --rm -f docker/alpine/Dockerfile -t frr:latest \ + https://github.com/frrouting/frr.git + +And to run the image: + +:: + + docker run -it --rm --name frr frr:latest + +In the default configuration, none of the frr daemons will be running. +To configure the daemons, exec into the container and edit the configuration +files or mount a volume with configuration files into the container on +startup. To configure by hand: + +:: + + docker exec -it frr /bin/sh + vi /etc/frr/daemons + /etc/init.d/frr start + +Or, to configure the daemons using /etc/frr from a host volume, put the +config files in, say, ./docker/etc and bind mount that into the +container: + +:: + + docker run -it --rm -v `pwd`/docker/etc:/etc/frr frr:latest + +We can also build the base image directly from docker-compose, with a +docker-compose.yml file like this one: + +:: + + version: '2.2' + + services: + frr: + build: + context: https://github.com/frrouting/frr.git + dockerfile: docker/alpine/Dockerfile diff --git a/doc/developer/building-frr-for-archlinux.rst b/doc/developer/building-frr-for-archlinux.rst new file mode 100644 index 0000000..406d22d --- /dev/null +++ b/doc/developer/building-frr-for-archlinux.rst @@ -0,0 +1,131 @@ +Arch Linux +================ + +Installing Dependencies +----------------------- + +.. code-block:: console + + sudo pacman -Syu + sudo pacman -S \ + git autoconf automake libtool make cmake pcre readline texinfo \ + pkg-config pam json-c bison flex python-pytest \ + c-ares python python2-ipaddress python-sphinx \ + net-snmp perl libcap libelf libunwind + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Protobuf +^^^^^^^^ + +.. code-block:: console + + sudo pacman -S protobuf-c + +ZeroMQ +^^^^^^ + +.. code-block:: console + + sudo pacman -S zeromq + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd --system -g frr --home-dir /var/run/frr/ \ + -c "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` [*Create the file if it doesn't exist*] and +append the following values (ignore the other settings): + +:: + + # Enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Enable packet forwarding for IPv6 + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install service files +^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service + sudo systemctl enable frr + +Start FRR +^^^^^^^^^ + +.. code-block:: shell + + systemctl start frr diff --git a/doc/developer/building-frr-for-centos6.rst b/doc/developer/building-frr-for-centos6.rst new file mode 100644 index 0000000..233d089 --- /dev/null +++ b/doc/developer/building-frr-for-centos6.rst @@ -0,0 +1,276 @@ +.. _building-centos6: + +CentOS 6 +======================================== + +This document describes installation from source. If you want to build an RPM, +see :ref:`packaging-redhat`. + +Instructions are tested with ``CentOS 6.8`` on ``x86_64`` platform + +Warning: +-------- +``CentOS 6`` is very old and not fully supported by the FRR community +anymore. Building FRR takes multiple manual steps to update the build +system with newer packages than what's available from the archives. +However, the built packages can still be installed afterwards on +a standard ``CentOS 6`` without any special packages. + +Support for CentOS 6 is now on a best-effort base by the community. + +CentOS 6 restrictions: +---------------------- + +- PIMd is not supported on ``CentOS 6``. Upgrade to ``CentOS 7`` if + PIMd is needed +- MPLS is not supported on ``CentOS 6``. MPLS requires Linux Kernel 4.5 + or higher (LDP can be built, but may have limited use without MPLS) +- Zebra is unable to detect what bridge/vrf an interface is associated + with (IFLA\_INFO\_SLAVE\_KIND does not exist in the kernel headers, + you can use a newer kernel + headers to get this functionality) +- frr\_reload.py will not work, as this requires Python 2.7, and CentOS + 6 only has 2.6. You can install Python 2.7 via IUS, but it won't work + properly unless you compile and install the ipaddr package for it. +- Building the package requires Sphinx >= 1.1. Only a non-standard + package provides a newer sphinx and requires manual installation + (see below) + + +Install required packages +------------------------- + +Add packages: + +.. code-block:: shell + + sudo yum install git autoconf automake libtool make \ + readline-devel texinfo net-snmp-devel groff pkgconfig \ + json-c-devel pam-devel flex epel-release c-ares-devel libcap-devel \ + elfutils-libelf-devel protobuf-c-devel + +Install newer version of bison (CentOS 6 package source is too old) from CentOS +7: + +.. code-block:: shell + + sudo yum install rpm-build + curl -O http://vault.centos.org/7.0.1406/os/Source/SPackages/bison-2.7-4.el7.src.rpm + rpmbuild --rebuild ./bison-2.7-4.el7.src.rpm + sudo yum install ./rpmbuild/RPMS/x86_64/bison-2.7-4.el6.x86_64.rpm + rm -rf rpmbuild + +Install newer version of autoconf and automake (Package versions are too old): + +.. code-block:: shell + + curl -O http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz + tar xvf autoconf-2.69.tar.gz + cd autoconf-2.69 + ./configure --prefix=/usr + make + sudo make install + cd .. + + curl -O http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz + tar xvf automake-1.15.tar.gz + cd automake-1.15 + ./configure --prefix=/usr + make + sudo make install + cd .. + +Install ``Python 2.7`` in parallel to default 2.6. Make sure you've install +EPEL (``epel-release`` as above). Then install current ``python27``: +``python27-devel`` and ``pytest`` + +.. code-block:: shell + + sudo rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm + sudo rpm -ivh https://centos6.iuscommunity.org/ius-release.rpm + sudo yum install python27 python27-pip python27-devel + sudo pip2.7 install pytest + +Please note that ``CentOS 6`` needs to keep python pointing to version 2.6 for +``yum`` to keep working, so don't create a symlink for python2.7 to python. + +Install newer ``Sphinx-Build`` based on ``Python 2.7``. + +Create a new repo ``/etc/yum.repos.d/puias6.repo`` with the following contents: + +:: + + ### Name: RPM Repository for RHEL 6 - PUIAS (used for Sphinx-Build) + ### URL: http://springdale.math.ias.edu/data/puias/computational + [puias-computational] + name = RPM Repository for RHEL 6 - Sphinx-Build + baseurl = http://springdale.math.ias.edu/data/puias/computational/$releasever/$basearch + #mirrorlist = + enabled = 1 + protect = 0 + gpgkey = + gpgcheck = 0 + +Update rpm database & Install newer sphinx + +.. code-block:: shell + + sudo yum update + sudo yum install python27-sphinx + +Install libyang and its dependencies: + +.. code-block:: shell + + sudo yum install pcre-devel doxygen cmake + git clone https://github.com/CESNET/libyang.git + cd libyang + git checkout 090926a89d59a3c4000719505d563aaf6ac60f2 + mkdir build ; cd build + cmake -DENABLE_LYD_PRIV=ON -DCMAKE_INSTALL_PREFIX:PATH=/usr -D CMAKE_BUILD_TYPE:String="Release" .. + make build-rpm + sudo yum install ./rpms/RPMS/x86_64/libyang-0.16.111-0.x86_64.rpm ./rpms/RPMS/x86_64/libyang-devel-0.16.111-0.x86_64.rpm + cd ../.. + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not using any +packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo groupadd -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd -u 92 -g 92 -M -r -G frrvty -s /sbin/nologin \ + -c "FRR FRRouting suite" -d /var/run/frr frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +.. code-block:: shell + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --bindir=/usr/bin \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --libdir=/usr/lib/frr \ + --libexecdir=/usr/lib/frr \ + --localstatedir=/var/run/frr \ + --with-moduledir=/usr/lib/frr/modules \ + --disable-pimd \ + --enable-snmp=agentx \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --disable-ldpd \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /var/log/frr + sudo mkdir /etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /etc/frr/babeld.conf + sudo touch /etc/frr/bfdd.conf + sudo touch /etc/frr/bgpd.conf + sudo touch /etc/frr/eigrpd.conf + sudo touch /etc/frr/isisd.conf + sudo touch /etc/frr/ldpd.conf + sudo touch /etc/frr/nhrpd.conf + sudo touch /etc/frr/ospf6d.conf + sudo touch /etc/frr/ospfd.conf + sudo touch /etc/frr/pbrd.conf + sudo touch /etc/frr/pimd.conf + sudo touch /etc/frr/ripd.conf + sudo touch /etc/frr/ripngd.conf + sudo touch /etc/frr/staticd.conf + sudo touch /etc/frr/zebra.conf + sudo chown -R frr:frr /etc/frr/ + sudo touch /etc/frr/vtysh.conf + sudo chown frr:frrvty /etc/frr/vtysh.conf + sudo chmod 640 /etc/frr/*.conf + +Install daemon config file +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo install -p -m 644 tools/etc/frr/daemons /etc/frr/ + sudo chown frr:frr /etc/frr/daemons + +Edit /etc/frr/daemons as needed to select the required daemons +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc. +Enable the daemons as required by changing the value to ``yes`` + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Edit :file:`/etc/sysctl.conf` and set the following values (ignore the other +settings):: + + # Controls IP packet forwarding + net.ipv4.ip_forward = 1 + net.ipv6.conf.all.forwarding=1 + + # Controls source route verification + net.ipv4.conf.default.rp_filter = 0 + +Load the modified sysctl's on the system: + +.. code-block:: shell + + sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf + +Add init.d startup file +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo install -p -m 755 tools/frr /etc/init.d/frr + sudo chkconfig --add frr + +Enable FRR daemon at startup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo chkconfig frr on + +Start FRR manually (or reboot) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo /etc/init.d/frr start diff --git a/doc/developer/building-frr-for-centos7.rst b/doc/developer/building-frr-for-centos7.rst new file mode 100644 index 0000000..e6da830 --- /dev/null +++ b/doc/developer/building-frr-for-centos7.rst @@ -0,0 +1,163 @@ +CentOS 7 +======================================== + +This document describes installation from source. If you want to build an RPM, +see :ref:`packaging-redhat`. + +CentOS 7 restrictions: +---------------------- + +- MPLS is not supported on ``CentOS 7`` with default kernel. MPLS + requires Linux Kernel 4.5 or higher (LDP can be built, but may have + limited use without MPLS) + +Install required packages +------------------------- + +Add packages: + +:: + + sudo yum install git autoconf automake libtool make \ + readline-devel texinfo net-snmp-devel groff pkgconfig \ + json-c-devel pam-devel bison flex pytest c-ares-devel \ + python-devel python-sphinx libcap-devel \ + elfutils-libelf-devel libunwind-devel protobuf-c-devel + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo groupadd -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd -u 92 -g 92 -M -r -G frrvty -s /sbin/nologin \ + -c "FRR FRRouting suite" -d /var/run/frr frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --bindir=/usr/bin \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --libdir=/usr/lib/frr \ + --libexecdir=/usr/lib/frr \ + --localstatedir=/var/run/frr \ + --with-moduledir=/usr/lib/frr/modules \ + --enable-snmp=agentx \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --disable-ldpd \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion \ + SPHINXBUILD=/usr/bin/sphinx-build + make + make check + sudo make install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo mkdir /var/log/frr + sudo mkdir /etc/frr + sudo touch /etc/frr/zebra.conf + sudo touch /etc/frr/bgpd.conf + sudo touch /etc/frr/ospfd.conf + sudo touch /etc/frr/ospf6d.conf + sudo touch /etc/frr/isisd.conf + sudo touch /etc/frr/ripd.conf + sudo touch /etc/frr/ripngd.conf + sudo touch /etc/frr/pimd.conf + sudo touch /etc/frr/nhrpd.conf + sudo touch /etc/frr/eigrpd.conf + sudo touch /etc/frr/babeld.conf + sudo chown -R frr:frr /etc/frr/ + sudo touch /etc/frr/vtysh.conf + sudo chown frr:frrvty /etc/frr/vtysh.conf + sudo chmod 640 /etc/frr/*.conf + +Install daemon config file +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -p -m 644 tools/etc/frr/daemons /etc/frr/ + sudo chown frr:frr /etc/frr/daemons + +Edit /etc/frr/daemons as needed to select the required daemons +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc. +Enable the daemons as required by changing the value to ``yes`` + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the +following content: + +:: + + # Sysctl for routing + # + # Routing: We need to forward packets + net.ipv4.conf.all.forwarding=1 + net.ipv6.conf.all.forwarding=1 + +Load the modified sysctl's on the system: + +:: + + sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf + +Install frr Service +^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -p -m 644 tools/frr.service /usr/lib/systemd/system/frr.service + +Register the systemd files +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl preset frr.service + +Enable required frr at startup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl enable frr + +Reboot or start FRR manually +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl start frr diff --git a/doc/developer/building-frr-for-centos8.rst b/doc/developer/building-frr-for-centos8.rst new file mode 100644 index 0000000..6d18e7b --- /dev/null +++ b/doc/developer/building-frr-for-centos8.rst @@ -0,0 +1,157 @@ +CentOS 8 +======== + +This document describes installation from source. If you want to build an RPM, +see :ref:`packaging-redhat`. + +Install required packages +------------------------- + +Add packages: + +:: + + sudo dnf install --enablerepo=PowerTools git autoconf pcre-devel \ + automake libtool make readline-devel texinfo net-snmp-devel pkgconfig \ + groff pkgconfig json-c-devel pam-devel bison flex python2-pytest \ + c-ares-devel python2-devel libcap-devel \ + elfutils-libelf-devel libunwind-devel \ + protobuf-c-devel + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo groupadd -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd -u 92 -g 92 -M -r -G frrvty -s /sbin/nologin \ + -c "FRR FRRouting suite" -d /var/run/frr frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --bindir=/usr/bin \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --libdir=/usr/lib/frr \ + --libexecdir=/usr/lib/frr \ + --localstatedir=/var/run/frr \ + --with-moduledir=/usr/lib/frr/modules \ + --enable-snmp=agentx \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --disable-ldpd \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion \ + SPHINXBUILD=/usr/bin/sphinx-build + make + make check + sudo make install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo mkdir /var/log/frr + sudo mkdir /etc/frr + sudo touch /etc/frr/zebra.conf + sudo touch /etc/frr/bgpd.conf + sudo touch /etc/frr/ospfd.conf + sudo touch /etc/frr/ospf6d.conf + sudo touch /etc/frr/isisd.conf + sudo touch /etc/frr/ripd.conf + sudo touch /etc/frr/ripngd.conf + sudo touch /etc/frr/pimd.conf + sudo touch /etc/frr/nhrpd.conf + sudo touch /etc/frr/eigrpd.conf + sudo touch /etc/frr/babeld.conf + sudo chown -R frr:frr /etc/frr/ + sudo touch /etc/frr/vtysh.conf + sudo chown frr:frrvty /etc/frr/vtysh.conf + sudo chmod 640 /etc/frr/*.conf + +Install daemon config file +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -p -m 644 tools/etc/frr/daemons /etc/frr/ + sudo chown frr:frr /etc/frr/daemons + +Edit /etc/frr/daemons as needed to select the required daemons +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc. +Enable the daemons as required by changing the value to ``yes`` + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the +following content: + +:: + + # Sysctl for routing + # + # Routing: We need to forward packets + net.ipv4.conf.all.forwarding=1 + net.ipv6.conf.all.forwarding=1 + +Load the modified sysctl's on the system: + +:: + + sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf + +Install frr Service +^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -p -m 644 tools/frr.service /usr/lib/systemd/system/frr.service + +Register the systemd files +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl preset frr.service + +Enable required frr at startup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl enable frr + +Reboot or start FRR manually +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo systemctl start frr diff --git a/doc/developer/building-frr-for-debian12.rst b/doc/developer/building-frr-for-debian12.rst new file mode 100644 index 0000000..ca882ee --- /dev/null +++ b/doc/developer/building-frr-for-debian12.rst @@ -0,0 +1,119 @@ +Debian 12 +========= + +Install required packages +------------------------- + +Add packages: + +:: + + sudo apt-get install git autoconf automake libtool make \ + libprotobuf-c-dev protobuf-c-compiler build-essential \ + python3-dev python3-pytest python3-sphinx libjson-c-dev \ + libelf-dev libreadline-dev cmake libcap-dev bison flex \ + pkg-config texinfo gdb libgrpc-dev python3-grpc-tools + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo addgroup --system --gid 92 frr + sudo addgroup --system --gid 85 frrvty + sudo adduser --system --ingroup frr --home /var/opt/frr/ \ + --gecos "FRR suite" --shell /bin/false frr + sudo usermod -a -G frrvty frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --localstatedir=/var/opt/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + +For more compile options, see ``./configure --help`` + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Edit ``/etc/frr/daemons`` and enable the FRR daemons for the protocols you need + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the +other settings) + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +**Reboot** or use ``sysctl -p`` to apply the same config to the running +system + +Troubleshooting +--------------- + +Shared library error +^^^^^^^^^^^^^^^^^^^^ + +If you try and start any of the frrouting daemons you may see the below +error due to the frrouting shared library directory not being found: + +:: + + ./zebra: error while loading shared libraries: libfrr.so.0: cannot open + shared object file: No such file or directory + +The fix is to add the following line to /etc/ld.so.conf which will +continue to reference the library directory after the system reboots. To +load the library directory path immediately run the ldconfig command +after adding the line to the file eg: + +:: + + echo include /usr/local/lib >> /etc/ld.so.conf + ldconfig diff --git a/doc/developer/building-frr-for-debian8.rst b/doc/developer/building-frr-for-debian8.rst new file mode 100644 index 0000000..7071cb6 --- /dev/null +++ b/doc/developer/building-frr-for-debian8.rst @@ -0,0 +1,150 @@ +Debian 8 +======================================== + +Debian 8 restrictions: +---------------------- + +- MPLS is not supported on ``Debian 8`` with default kernel. MPLS + requires Linux Kernel 4.5 or higher (LDP can be built, but may have + limited use without MPLS) + +Install required packages +------------------------- + +Add packages: + +:: + + sudo apt-get install git autoconf automake libtool make \ + libreadline-dev texinfo libjson-c-dev pkg-config bison flex python3-pip \ + libc-ares-dev python3-dev python3-sphinx build-essential \ + libsnmp-dev libcap-dev libelf-dev libprotobuf-c-dev protobuf-c-compiler + +Install newer pytest (>3.0) from pip + +:: + + sudo pip3 install pytest + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo addgroup --system --gid 92 frr + sudo addgroup --system --gid 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /bin/false frr + sudo usermod -a -G frrvty frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --localstatedir=/var/run/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -m 755 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf + sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the +other settings) + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +**Reboot** or use ``sysctl -p`` to apply the same config to the running +system + +Troubleshooting +^^^^^^^^^^^^^^^ + +**Local state directory** + +The local state directory must exist and have the correct permissions +applied for the frrouting daemons to start. In the above ./configure +example the local state directory is set to /var/run/frr +(--localstatedir=/var/run/frr) Debian considers /var/run/frr to be +temporary and this is removed after a reboot. + +When using a different local state directory you need to create the new +directory and change the ownership to the frr user, for example: + +:: + + mkdir /var/opt/frr + chown frr /var/opt/frr + +**Shared library error** + +If you try and start any of the frrouting daemons you may see the below +error due to the frrouting shared library directory not being found: + +:: + + ./zebra: error while loading shared libraries: libfrr.so.0: cannot open shared object file: No such file or directory + +The fix is to add the following line to /etc/ld.so.conf which will +continue to reference the library directory after the system reboots. To +load the library directory path immediately run the ldconfig command +after adding the line to the file eg: + +:: + + echo include /usr/local/lib >> /etc/ld.so.conf + ldconfig diff --git a/doc/developer/building-frr-for-debian9.rst b/doc/developer/building-frr-for-debian9.rst new file mode 100644 index 0000000..1b2f1b9 --- /dev/null +++ b/doc/developer/building-frr-for-debian9.rst @@ -0,0 +1,127 @@ +Debian 9 +======================================== + +Install required packages +------------------------- + +Add packages: + +:: + + sudo apt-get install git autoconf automake libtool make \ + libreadline-dev texinfo libjson-c-dev pkg-config bison flex \ + libc-ares-dev python3-dev python3-pytest python3-sphinx build-essential \ + libsnmp-dev libcap-dev libelf-dev libunwind-dev \ + libprotobuf-c-dev protobuf-c-compiler + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo addgroup --system --gid 92 frr + sudo addgroup --system --gid 85 frrvty + sudo adduser --system --ingroup frr --home /var/opt/frr/ \ + --gecos "FRR suite" --shell /bin/false frr + sudo usermod -a -G frrvty frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example.) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --localstatedir=/var/opt/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo install -m 755 -o frr -g frr -d /var/log/frr + sudo install -m 755 -o frr -g frr -d /var/opt/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf + sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf + sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the +other settings) + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +**Reboot** or use ``sysctl -p`` to apply the same config to the running +system + +Troubleshooting +--------------- + +Shared library error +^^^^^^^^^^^^^^^^^^^^ + +If you try and start any of the frrouting daemons you may see the below +error due to the frrouting shared library directory not being found: + +:: + + ./zebra: error while loading shared libraries: libfrr.so.0: cannot open + shared object file: No such file or directory + +The fix is to add the following line to /etc/ld.so.conf which will +continue to reference the library directory after the system reboots. To +load the library directory path immediately run the ldconfig command +after adding the line to the file eg: + +:: + + echo include /usr/local/lib >> /etc/ld.so.conf + ldconfig diff --git a/doc/developer/building-frr-for-fedora.rst b/doc/developer/building-frr-for-fedora.rst new file mode 100644 index 0000000..35a24b2 --- /dev/null +++ b/doc/developer/building-frr-for-fedora.rst @@ -0,0 +1,136 @@ +Fedora 24+ +========== + +This document describes installation from source. If you want to build an RPM, +see :ref:`packaging-redhat`. + +These instructions have been tested on Fedora 24+. + +Installing Dependencies +----------------------- + +.. code-block:: console + + sudo dnf install git autoconf automake libtool make \ + readline-devel texinfo net-snmp-devel groff pkgconfig json-c-devel \ + pam-devel python3-pytest bison flex c-ares-devel python3-devel \ + python3-sphinx perl-core patch libcap-devel \ + elfutils-libelf-devel libunwind-devel protobuf-c-devel + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd -u 92 -g 92 -M -r -G frrvty -s /sbin/nologin \ + -c "FRR FRRouting suite" -d /var/run/frr frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the following +content: + +:: + + # + # Enable packet forwarding + # + net.ipv4.conf.all.forwarding=1 + net.ipv6.conf.all.forwarding=1 + # + # Enable MPLS Label processing on all interfaces + # + #net.mpls.conf.eth0.input=1 + #net.mpls.conf.eth1.input=1 + #net.mpls.conf.eth2.input=1 + #net.mpls.platform_labels=100000 + +.. note:: + + MPLS must be invidividually enabled on each interface that requires it. See + the example in the config block above. + +Load the modified sysctls on the system: + +.. code-block:: console + + sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf + +Create a new file ``/etc/modules-load.d/mpls.conf`` with the following content: + +:: + + # Load MPLS Kernel Modules + mpls-router + mpls-iptunnel + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + + +.. note:: + Fedora ships with the ``firewalld`` service enabled. You may run into some + issues with the iptables rules it installs by default. If you wish to just + stop the service and clear `ALL` rules do these commands: + + .. code-block:: console + + sudo systemctl disable firewalld.service + sudo systemctl stop firewalld.service + sudo iptables -F + +Install frr Service +^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -p -m 644 tools/frr.service /usr/lib/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: frr + + sudo systemctl start frr diff --git a/doc/developer/building-frr-for-freebsd10.rst b/doc/developer/building-frr-for-freebsd10.rst new file mode 100644 index 0000000..707f1e7 --- /dev/null +++ b/doc/developer/building-frr-for-freebsd10.rst @@ -0,0 +1,130 @@ +FreeBSD 10 +========================================== + +FreeBSD 10 restrictions: +------------------------ + +- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS + +Install required packages +------------------------- + +Add packages: (Allow the install of the package management tool if this +is first package install and asked) + +:: + + pkg install git autoconf automake libtool gmake json-c pkgconf \ + bison flex py36-pytest c-ares python3.6 py36-sphinx libunwind \ + protobuf-c + +.. include:: building-libunwind-note.rst + +Make sure there is no /usr/bin/flex preinstalled (and use the newly +installed in /usr/local/bin): (FreeBSD frequently provides a older flex +as part of the base OS which takes preference in path) + +.. include:: building-libyang.rst + +:: + + rm -f /usr/bin/flex + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +:: + + pw groupadd frr -g 101 + pw groupadd frrvty -g 102 + pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \ + -d /usr/local/etc/frr -s /usr/sbin/nologin + +(You may prefer different options on configure statement. These are just +an example) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + export MAKE=gmake + export LDFLAGS="-L/usr/local/lib" + export CPPFLAGS="-I/usr/local/include" + ./configure \ + --sysconfdir=/usr/local/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --prefix=/usr/local \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /usr/local/etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/babeld.conf + sudo touch /usr/local/etc/frr/bfdd.conf + sudo touch /usr/local/etc/frr/bgpd.conf + sudo touch /usr/local/etc/frr/eigrpd.conf + sudo touch /usr/local/etc/frr/isisd.conf + sudo touch /usr/local/etc/frr/ldpd.conf + sudo touch /usr/local/etc/frr/nhrpd.conf + sudo touch /usr/local/etc/frr/ospf6d.conf + sudo touch /usr/local/etc/frr/ospfd.conf + sudo touch /usr/local/etc/frr/pbrd.conf + sudo touch /usr/local/etc/frr/pimd.conf + sudo touch /usr/local/etc/frr/ripd.conf + sudo touch /usr/local/etc/frr/ripngd.conf + sudo touch /usr/local/etc/frr/staticd.conf + sudo touch /usr/local/etc/frr/zebra.conf + sudo chown -R frr:frr /usr/local/etc/frr/ + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf + sudo chmod 640 /usr/local/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running system. diff --git a/doc/developer/building-frr-for-freebsd11.rst b/doc/developer/building-frr-for-freebsd11.rst new file mode 100644 index 0000000..af0b72b --- /dev/null +++ b/doc/developer/building-frr-for-freebsd11.rst @@ -0,0 +1,135 @@ +FreeBSD 11 +========== + +FreeBSD 11 restrictions: +------------------------ + +- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS + +Install required packages +------------------------- + +Add packages: (Allow the install of the package management tool if this +is first package install and asked) + +.. code-block:: shell + + pkg install git autoconf automake libtool gmake json-c pkgconf \ + bison flex py36-pytest c-ares python3.6 py36-sphinx texinfo libunwind \ + protobuf-c + +.. include:: building-libunwind-note.rst + +Make sure there is no /usr/bin/flex preinstalled (and use the newly +installed in /usr/local/bin): (FreeBSD frequently provides a older flex +as part of the base OS which takes preference in path) + +.. include:: building-libyang.rst + +.. code-block:: shell + + rm -f /usr/bin/flex + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not using any +packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + pw groupadd frr -g 101 + pw groupadd frrvty -g 102 + pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \ + -d /usr/local/etc/frr -s /usr/sbin/nologin + + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +.. code-block:: shell + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + setenv MAKE gmake + setenv LDFLAGS -L/usr/local/lib + setenv CPPFLAGS -I/usr/local/include + ln -s /usr/local/bin/sphinx-build-3.6 /usr/local/bin/sphinx-build + ./configure \ + --sysconfdir=/usr/local/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --prefix=/usr/local \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /usr/local/etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/babeld.conf + sudo touch /usr/local/etc/frr/bfdd.conf + sudo touch /usr/local/etc/frr/bgpd.conf + sudo touch /usr/local/etc/frr/eigrpd.conf + sudo touch /usr/local/etc/frr/isisd.conf + sudo touch /usr/local/etc/frr/ldpd.conf + sudo touch /usr/local/etc/frr/nhrpd.conf + sudo touch /usr/local/etc/frr/ospf6d.conf + sudo touch /usr/local/etc/frr/ospfd.conf + sudo touch /usr/local/etc/frr/pbrd.conf + sudo touch /usr/local/etc/frr/pimd.conf + sudo touch /usr/local/etc/frr/ripd.conf + sudo touch /usr/local/etc/frr/ripngd.conf + sudo touch /usr/local/etc/frr/staticd.conf + sudo touch /usr/local/etc/frr/zebra.conf + sudo chown -R frr:frr /usr/local/etc/frr/ + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf + sudo chmod 640 /usr/local/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running system. diff --git a/doc/developer/building-frr-for-freebsd13.rst b/doc/developer/building-frr-for-freebsd13.rst new file mode 100644 index 0000000..0bc8277 --- /dev/null +++ b/doc/developer/building-frr-for-freebsd13.rst @@ -0,0 +1,122 @@ +FreeBSD 13 +========== + +FreeBSD 13 restrictions: +------------------------ + +- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS +- PIM for IPv6 is not currently supported on ``FreeBSD``. + +Install required packages +------------------------- + +Add packages: (Allow the install of the package management tool if this +is first package install and asked) + +.. code-block:: shell + + pkg install git autoconf automake libtool gmake json-c pkgconf \ + bison py39-pytest c-ares py39-sphinx texinfo libunwind libyang2 + +.. include:: building-libunwind-note.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not using any +packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + pw groupadd frr -g 101 + pw groupadd frrvty -g 102 + pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \ + -d /usr/local/etc/frr -s /usr/sbin/nologin + + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +.. code-block:: shell + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + export MAKE=gmake LDFLAGS=-L/usr/local/lib CPPFLAGS=-I/usr/local/include + ./configure \ + --sysconfdir=/usr/local/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --prefix=/usr/local \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /usr/local/etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/babeld.conf + sudo touch /usr/local/etc/frr/bfdd.conf + sudo touch /usr/local/etc/frr/bgpd.conf + sudo touch /usr/local/etc/frr/eigrpd.conf + sudo touch /usr/local/etc/frr/isisd.conf + sudo touch /usr/local/etc/frr/ldpd.conf + sudo touch /usr/local/etc/frr/nhrpd.conf + sudo touch /usr/local/etc/frr/ospf6d.conf + sudo touch /usr/local/etc/frr/ospfd.conf + sudo touch /usr/local/etc/frr/pbrd.conf + sudo touch /usr/local/etc/frr/pimd.conf + sudo touch /usr/local/etc/frr/ripd.conf + sudo touch /usr/local/etc/frr/ripngd.conf + sudo touch /usr/local/etc/frr/staticd.conf + sudo touch /usr/local/etc/frr/zebra.conf + sudo chown -R frr:frr /usr/local/etc/frr/ + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf + sudo chmod 640 /usr/local/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running system. diff --git a/doc/developer/building-frr-for-freebsd9.rst b/doc/developer/building-frr-for-freebsd9.rst new file mode 100644 index 0000000..3033287 --- /dev/null +++ b/doc/developer/building-frr-for-freebsd9.rst @@ -0,0 +1,140 @@ +FreeBSD 9 +========================================= + +FreeBSD 9 restrictions: +----------------------- + +- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS + +Install required packages +------------------------- + +Add packages: (Allow the install of the package management tool if this +is first package install and asked) + +:: + + pkg install -y git autoconf automake libtool gmake \ + pkgconf texinfo json-c bison flex py36-pytest c-ares \ + python3 py36-sphinx libexecinfo protobuf-c + +Make sure there is no /usr/bin/flex preinstalled (and use the newly +installed in /usr/local/bin): (FreeBSD frequently provides a older flex +as part of the base OS which takes preference in path) + +:: + + rm -f /usr/bin/flex + +For building with clang (instead of gcc), upgrade clang from 3.4 default +to 3.6 *This is needed to build FreeBSD packages as well - for packages +clang is default* (Clang 3.4 as shipped with FreeBSD 9 crashes during +compile) + +:: + + pkg install clang36 + pkg delete clang34 + mv /usr/bin/clang /usr/bin/clang34 + ln -s /usr/local/bin/clang36 /usr/bin/clang + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +:: + + pw groupadd frr -g 101 + pw groupadd frrvty -g 102 + pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \ + -d /usr/local/etc/frr -s /usr/sbin/nologin + +(You may prefer different options on configure statement. These are just +an example) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + export MAKE=gmake + export LDFLAGS="-L/usr/local/lib" + export CPPFLAGS="-I/usr/local/include" + ./configure \ + --sysconfdir=/usr/local/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --prefix=/usr/local \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /usr/local/etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/babeld.conf + sudo touch /usr/local/etc/frr/bfdd.conf + sudo touch /usr/local/etc/frr/bgpd.conf + sudo touch /usr/local/etc/frr/eigrpd.conf + sudo touch /usr/local/etc/frr/isisd.conf + sudo touch /usr/local/etc/frr/ldpd.conf + sudo touch /usr/local/etc/frr/nhrpd.conf + sudo touch /usr/local/etc/frr/ospf6d.conf + sudo touch /usr/local/etc/frr/ospfd.conf + sudo touch /usr/local/etc/frr/pbrd.conf + sudo touch /usr/local/etc/frr/pimd.conf + sudo touch /usr/local/etc/frr/ripd.conf + sudo touch /usr/local/etc/frr/ripngd.conf + sudo touch /usr/local/etc/frr/staticd.conf + sudo touch /usr/local/etc/frr/zebra.conf + sudo chown -R frr:frr /usr/local/etc/frr/ + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf + sudo chmod 640 /usr/local/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running system. diff --git a/doc/developer/building-frr-for-netbsd6.rst b/doc/developer/building-frr-for-netbsd6.rst new file mode 100644 index 0000000..8958862 --- /dev/null +++ b/doc/developer/building-frr-for-netbsd6.rst @@ -0,0 +1,139 @@ +NetBSD 6 +======================================== + +NetBSD 6 restrictions: +---------------------- + +- MPLS is not supported on ``NetBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS + +Install required packages +------------------------- + +Configure Package location: + +:: + + PKG_PATH="ftp://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/`uname -m`/`uname -r`/All" + export PKG_PATH + +Add packages: + +:: + + sudo pkg_add git autoconf automake libtool gmake openssl \ + pkg-config json-c py36-test python36 py36-sphinx \ + protobuf-c + +Install SSL Root Certificates (for git https access): + +:: + + sudo pkg_add mozilla-rootcerts + sudo touch /etc/openssl/openssl.cnf + sudo mozilla-rootcerts install + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo groupadd -g 92 frr + sudo groupadd -g 93 frrvty + sudo useradd -g 92 -u 92 -G frrvty -c "FRR suite" \ + -d /nonexistent -s /sbin/nologin frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + MAKE=gmake + export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib" + export CPPFLAGS="-I/usr/pkg/include" + ./configure \ + --sysconfdir=/usr/pkg/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo mkdir /var/log/frr + sudo mkdir /usr/pkg/etc/frr + sudo touch /usr/pkg/etc/frr/zebra.conf + sudo touch /usr/pkg/etc/frr/bgpd.conf + sudo touch /usr/pkg/etc/frr/ospfd.conf + sudo touch /usr/pkg/etc/frr/ospf6d.conf + sudo touch /usr/pkg/etc/frr/isisd.conf + sudo touch /usr/pkg/etc/frr/ripd.conf + sudo touch /usr/pkg/etc/frr/ripngd.conf + sudo touch /usr/pkg/etc/frr/pimd.conf + sudo chown -R frr:frr /usr/pkg/etc/frr + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/pkg/etc/frr/*.conf + sudo chmod 640 /usr/pkg/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running +system + +Install rc.d init files +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + cp pkgsrc/*.sh /etc/rc.d/ + chmod 555 /etc/rc.d/*.sh + +Enable FRR processes +^^^^^^^^^^^^^^^^^^^^ + +(Enable the required processes only) + +:: + + echo "zebra=YES" >> /etc/rc.conf + echo "bgpd=YES" >> /etc/rc.conf + echo "ospfd=YES" >> /etc/rc.conf + echo "ospf6d=YES" >> /etc/rc.conf + echo "isisd=YES" >> /etc/rc.conf + echo "ripngd=YES" >> /etc/rc.conf + echo "ripd=YES" >> /etc/rc.conf + echo "pimd=YES" >> /etc/rc.conf diff --git a/doc/developer/building-frr-for-netbsd7.rst b/doc/developer/building-frr-for-netbsd7.rst new file mode 100644 index 0000000..e751ba3 --- /dev/null +++ b/doc/developer/building-frr-for-netbsd7.rst @@ -0,0 +1,129 @@ +NetBSD 7 +======================================== + +NetBSD 7 restrictions: +---------------------- + +- MPLS is not supported on ``NetBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS + +Install required packages +------------------------- + +:: + + sudo pkgin install git autoconf automake libtool gmake openssl \ + pkg-config json-c python36 py36-test py36-sphinx \ + protobuf-c + +Install SSL Root Certificates (for git https access): + +:: + + sudo pkgin install mozilla-rootcerts + sudo touch /etc/openssl/openssl.cnf + sudo mozilla-rootcerts install + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +Add frr groups and user +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo groupadd -g 92 frr + sudo groupadd -g 93 frrvty + sudo useradd -g 92 -u 92 -G frrvty -c "FRR suite" \ + -d /nonexistent -s /sbin/nologin frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + MAKE=gmake + export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib" + export CPPFLAGS="-I/usr/pkg/include" + ./configure \ + --sysconfdir=/usr/pkg/etc/frr \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --localstatedir=/var/run/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sudo mkdir /usr/pkg/etc/frr + sudo touch /usr/pkg/etc/frr/zebra.conf + sudo touch /usr/pkg/etc/frr/bgpd.conf + sudo touch /usr/pkg/etc/frr/ospfd.conf + sudo touch /usr/pkg/etc/frr/ospf6d.conf + sudo touch /usr/pkg/etc/frr/isisd.conf + sudo touch /usr/pkg/etc/frr/ripd.conf + sudo touch /usr/pkg/etc/frr/ripngd.conf + sudo touch /usr/pkg/etc/frr/pimd.conf + sudo chown -R frr:frr /usr/pkg/etc/frr + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/pkg/etc/frr/*.conf + sudo chmod 640 /usr/pkg/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running +system + +Install rc.d init files +^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + cp pkgsrc/*.sh /etc/rc.d/ + chmod 555 /etc/rc.d/*.sh + +Enable FRR processes +^^^^^^^^^^^^^^^^^^^^ + +(Enable the required processes only) + +:: + + echo "zebra=YES" >> /etc/rc.conf + echo "bgpd=YES" >> /etc/rc.conf + echo "ospfd=YES" >> /etc/rc.conf + echo "ospf6d=YES" >> /etc/rc.conf + echo "isisd=YES" >> /etc/rc.conf + echo "ripngd=YES" >> /etc/rc.conf + echo "ripd=YES" >> /etc/rc.conf + echo "pimd=YES" >> /etc/rc.conf diff --git a/doc/developer/building-frr-for-openbsd6.rst b/doc/developer/building-frr-for-openbsd6.rst new file mode 100644 index 0000000..00bc2e5 --- /dev/null +++ b/doc/developer/building-frr-for-openbsd6.rst @@ -0,0 +1,182 @@ +OpenBSD 6 +========================================= + +Install required packages +------------------------- + +Configure PKG\_PATH + +:: + + export PKG_PATH=http://ftp5.usa.openbsd.org/pub/OpenBSD/$(uname -r)/packages/$(machine -a)/ + +Add packages: + +:: + + pkg_add clang libcares python3 + pkg_add git autoconf-2.69p2 automake-1.15.1 libtool bison + pkg_add gmake json-c py-test py-sphinx libexecinfo protobuf-c + +Select Python2.7 as default (required for pytest) + +:: + + ln -s /usr/local/bin/python2.7 /usr/local/bin/python + +.. include:: building-libyang.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not +using any packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +:: + + groupadd -g 525 _frr + groupadd -g 526 _frrvty + useradd -g 525 -u 525 -c "FRR suite" -G _frrvty \ + -d /nonexistent -s /sbin/nologin _frr + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +.. warning:: + + In openbsd the proper links for the libyang library may not have been created. + +:: + + ln -s /usr/lib/libyang.so.1.10.17 /usr/lib/libyang.so + +.. warning:: + + ``openbsd`` since version 6.2 has ``clang`` as the default compiler so to + build frr, clang must be used (the included gcc version is very old). + +:: + + git clone https://github.com/frrouting/frr.git frr + cd frr + export AUTOCONF_VERSION="2.69" + export AUTOMAKE_VERSION="1.15" + ./bootstrap.sh + export LDFLAGS="-L/usr/local/lib" + export CPPFLAGS="-I/usr/local/include" + ./configure \ + --sysconfdir=/etc/frr \ + --localstatedir=/var/frr \ + --enable-multipath=64 \ + --enable-user=_frr \ + --enable-group=_frr \ + --enable-vty-group=_frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion \ + CC=clang + gmake + gmake check + doas gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + doas mkdir /var/frr + doas chown _frr:_frr /var/frr + doas chmod 755 /var/frr + doas mkdir /etc/frr + doas touch /etc/frr/zebra.conf + doas touch /etc/frr/bgpd.conf + doas touch /etc/frr/ospfd.conf + doas touch /etc/frr/ospf6d.conf + doas touch /etc/frr/isisd.conf + doas touch /etc/frr/ripd.conf + doas touch /etc/frr/ripngd.conf + doas touch /etc/frr/pimd.conf + doas touch /etc/frr/ldpd.conf + doas touch /etc/frr/nhrpd.conf + doas chown -R _frr:_frr /etc/frr + doas touch /etc/frr/vtysh.conf + doas chown -R _frr:_frrvty /etc/frr/vtysh.conf + doas chmod 750 /etc/frr + doas chmod 640 /etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/rc.conf``: + +:: + + net.inet6.ip6.forwarding=1 # 1=Permit forwarding of IPv6 packets + net.inet6.ip6.mforwarding=1 # 1=Permit forwarding of IPv6 multicast packets + net.inet6.ip6.multipath=1 # 1=Enable IPv6 multipath routing + +**Reboot** to apply the config to the system + +Enable MPLS Forwarding +^^^^^^^^^^^^^^^^^^^^^^ + +To enable MPLS forwarding on a given interface, use the following +command: + +:: + + doas ifconfig em0 mpls + +Alternatively, to make MPLS forwarding persistent across reboots, add +the "mpls" keyword in the hostname.\* files of the desired interfaces. +Example: + +:: + + cat /etc/hostname.em0 + inet 10.0.1.1 255.255.255.0 mpls + +Install rc.d init files +^^^^^^^^^^^^^^^^^^^^^^^ + +(create them in /etc/rc.d - no example are included at this time with +FRR source) + +Example (for zebra - store as ``/etc/rc.d/frr_zebra.sh``) + +:: + + #!/bin/sh + # + # $OpenBSD: frr_zebra.rc,v 1.1 2013/04/18 20:29:08 sthen Exp $ + + daemon="/usr/local/sbin/zebra -d" + + . /etc/rc.d/rc.subr + + rc_cmd $1 + +Enable FRR processes +^^^^^^^^^^^^^^^^^^^^ + +(Enable the required processes only) + +:: + + echo "frr_zebra=YES" >> /etc/rc.conf + echo "frr_bgpd=YES" >> /etc/rc.conf + echo "frr_ospfd=YES" >> /etc/rc.conf + echo "frr_ospf6d=YES" >> /etc/rc.conf + echo "frr_isisd=YES" >> /etc/rc.conf + echo "frr_ripngd=YES" >> /etc/rc.conf + echo "frr_ripd=YES" >> /etc/rc.conf + echo "frr_pimd=YES" >> /etc/rc.conf + echo "frr_ldpd=YES" >> /etc/rc.conf diff --git a/doc/developer/building-frr-for-opensuse.rst b/doc/developer/building-frr-for-opensuse.rst new file mode 100644 index 0000000..3ff445b --- /dev/null +++ b/doc/developer/building-frr-for-opensuse.rst @@ -0,0 +1,146 @@ +openSUSE +======== + +This document describes installation from source. + +These instructions have been tested on openSUSE Tumbleweed in a Raspberry Pi 400. + +Installing Dependencies +----------------------- + +.. code-block:: console + + zypper in git autoconf automake libtool make \ + readline-devel texinfo net-snmp-devel groff pkgconfig libjson-c-devel\ + pam-devel python3-pytest bison flex c-ares-devel python3-devel\ + python3-Sphinx perl patch libcap-devel libyang-devel \ + libelf-devel libunwind-devel protobuf-c + +.. include:: building-libunwind-note.rst + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo useradd -u 92 -g 92 -M -r -G frrvty -s /sbin/nologin \ + -c "FRR FRRouting suite" -d /var/run/frr frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +.. note:: + + In some platforms like raspberry for performance reasons + some directories are in file systems (/var/run, ...) mounted with tempfs + so will disapear after every reboot. + In frr the /var/run/frr is used to store pid files for every daemon. + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the following +content: + +:: + + # + # Enable packet forwarding + # + net.ipv4.conf.all.forwarding=1 + net.ipv6.conf.all.forwarding=1 + # + # Enable MPLS Label processing on all interfaces + # + #net.mpls.conf.eth0.input=1 + #net.mpls.conf.eth1.input=1 + #net.mpls.conf.eth2.input=1 + #net.mpls.platform_labels=100000 + +.. note:: + + MPLS must be invidividually enabled on each interface that requires it. See + the example in the config block above. + +Load the modified sysctls on the system: + +.. code-block:: console + + sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf + +Create a new file ``/etc/modules-load.d/mpls.conf`` with the following content: + +:: + + # Load MPLS Kernel Modules + mpls-router + mpls-iptunnel + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + + +.. note:: + The ``firewalld`` service could be enabled. You may run into some + issues with the iptables rules it installs by default. If you wish to just + stop the service and clear `ALL` rules do these commands: + + .. code-block:: console + + sudo systemctl disable firewalld.service + sudo systemctl stop firewalld.service + sudo iptables -F + +Install frr Service +^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -p -m 644 tools/frr.service /usr/lib/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``bgpd=no`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: console + + sudo systemctl start frr + +Check the starting messages of frr with + +.. code-block:: console + + journalctl -u frr --follow diff --git a/doc/developer/building-frr-for-openwrt.rst b/doc/developer/building-frr-for-openwrt.rst new file mode 100644 index 0000000..47cf2cb --- /dev/null +++ b/doc/developer/building-frr-for-openwrt.rst @@ -0,0 +1,79 @@ +OpenWrt +======= + +General info about OpenWrt buildsystem: `link <https://openwrt.org/docs/guide-developer/build-system/start>`_. + +Prepare build environment +------------------------- + +For Debian based distributions, run: + +:: + + sudo apt-get install git build-essential libssl-dev libncurses5-dev \ + unzip zlib1g-dev subversion mercurial + +For other environments, instructions can be found in the +`official documentation +<https://openwrt.org/docs/guide-developer/build-system/install-buildsystem#examples_of_package_installations>`_. + + +Get OpenWrt Sources (from Git) +------------------------------ + +.. note:: + The OpenWrt build will fail if you run it as root. So take care to run it as a nonprivileged user. + +Clone the OpenWrt sources and retrieve the package feeds + +:: + + git clone https://github.com/openwrt/openwrt.git + cd openwrt + ./scripts/feeds update -a + ./scripts/feeds install -a + +Configure OpenWrt for your target and select the needed FRR packages in Network -> Routing and Redirection -> frr, +exit and save + +:: + + make menuconfig + +Then, to compile either a complete OpenWrt image, or the FRR packages, run: + +:: + + make or make package/frr/compile + +It may be possible that on first build ``make package/frr/compile`` not +to work and it may be needed to run a ``make`` for the entire build +environment. Add ``V=s`` to get more debugging output. + +More information about OpenWrt buildsystem can be found `here +<https://openwrt.org/docs/guide-developer/build-system/use-buildsystem>`__. + +Work with sources +----------------- + +To update to a newer version, or change other options, you need to edit the ``feeds/packages/frr/Makefile``. + +More information about working with patches in OpenWrt buildsystem can be found `here +<https://openwrt.org/docs/guide-developer/build-system/use-patches-with-buildsystem>`__. + +Usage +----- + +Edit ``/usr/sbin/frr.init`` and add/remove the daemons name in section +``DAEMONS=`` or don't install unneeded packages For example: zebra bgpd ldpd +isisd nhrpd ospfd ospf6d pimd ripd ripngd + +Enable the service +^^^^^^^^^^^^^^^^^^ + +- ``service frr enable`` + +Start the service +^^^^^^^^^^^^^^^^^ + +- ``service frr start`` diff --git a/doc/developer/building-frr-for-ubuntu1404.rst b/doc/developer/building-frr-for-ubuntu1404.rst new file mode 100644 index 0000000..cc6c3c0 --- /dev/null +++ b/doc/developer/building-frr-for-ubuntu1404.rst @@ -0,0 +1,142 @@ +Ubuntu 14.04 LTS +================ + +This document describes installation from source. If you want to build a +``deb``, see :ref:`packaging-debian`. + +Installing Dependencies +----------------------- + +.. code-block:: console + + apt-get update + apt-get install \ + git autoconf automake libtool make libreadline-dev texinfo \ + pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ + libc-ares-dev python3-dev python3-sphinx install-info build-essential \ + libsnmp-dev perl libcap-dev libelf-dev + +.. include:: building-libyang.rst + +Protobuf +^^^^^^^^ + +.. code-block:: console + + sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` and uncomment the following values (ignore the +other settings): + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +.. warning:: + + MPLS is not supported on Ubuntu 14.04 with the default kernel. MPLS requires + kernel 4.5 or higher. LDPD can be built, but may have limited use without + MPLS. For an updated Ubuntu Kernel, see + http://kernel.ubuntu.com/~kernel-ppa/mainline/ + +Ubuntu 18.04 ships with kernel 4.15. MPLS modules are present by default. To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install the init.d service +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 755 tools/frr /etc/init.d/frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start the init.d service +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + /etc/init.d/frr start + +Use ``/etc/init.d/frr status`` to check its status. diff --git a/doc/developer/building-frr-for-ubuntu1604.rst b/doc/developer/building-frr-for-ubuntu1604.rst new file mode 100644 index 0000000..e5c2389 --- /dev/null +++ b/doc/developer/building-frr-for-ubuntu1604.rst @@ -0,0 +1,142 @@ +Ubuntu 16.04 LTS +================ + +This document describes installation from source. If you want to build a +``deb``, see :ref:`packaging-debian`. + +Installing Dependencies +----------------------- + +.. code-block:: console + + apt-get update + apt-get install \ + git autoconf automake libtool make libreadline-dev texinfo \ + pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ + libc-ares-dev python3-dev python-ipaddress python3-sphinx \ + install-info build-essential libsnmp-dev perl libcap-dev \ + libelf-dev libprotobuf-c-dev protobuf-c-compiler + +.. include:: building-libyang.rst + +Protobuf +^^^^^^^^ + +.. code-block:: console + + sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` and uncomment the following values (ignore the +other settings): + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +.. warning:: + + MPLS is not supported on Ubuntu 16.04 with the default kernel. MPLS requires + kernel 4.5 or higher. LDPD can be built, but may have limited use without + MPLS. For an updated Ubuntu Kernel, see + http://kernel.ubuntu.com/~kernel-ppa/mainline/ + +Ubuntu 18.04 ships with kernel 4.15. MPLS modules are present by default. To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install service files +^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: console + + systemctl start frr diff --git a/doc/developer/building-frr-for-ubuntu1804.rst b/doc/developer/building-frr-for-ubuntu1804.rst new file mode 100644 index 0000000..fcfd94e --- /dev/null +++ b/doc/developer/building-frr-for-ubuntu1804.rst @@ -0,0 +1,148 @@ +Ubuntu 18.04 LTS +================ + +This document describes installation from source. If you want to build a +``deb``, see :ref:`packaging-debian`. + +Installing Dependencies +----------------------- + +.. code-block:: console + + sudo apt update + sudo apt-get install \ + git autoconf automake libtool make libreadline-dev texinfo \ + pkg-config libpam0g-dev libjson-c-dev bison flex \ + libc-ares-dev python3-dev python3-sphinx \ + install-info build-essential libsnmp-dev perl libcap-dev \ + libelf-dev libunwind-dev + +.. include:: building-libunwind-note.rst + +.. include:: building-libyang.rst + +Protobuf +^^^^^^^^ + +.. code-block:: console + + sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + +ZeroMQ +^^^^^^ + +.. code-block:: console + + sudo apt-get install libzmq5 libzmq3-dev + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` and uncomment the following values (ignore the +other settings): + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +Ubuntu 18.04 ships with kernel 4.15. MPLS modules are present by default. To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +If the above command returns an error, you may need to install the appropriate +or latest linux-modules-extra-<kernel-version>-generic package. For example +``apt-get install linux-modules-extra-`uname -r`-generic`` + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install service files +^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: shell + + systemctl start frr diff --git a/doc/developer/building-frr-for-ubuntu2004.rst b/doc/developer/building-frr-for-ubuntu2004.rst new file mode 100644 index 0000000..fdfc25d --- /dev/null +++ b/doc/developer/building-frr-for-ubuntu2004.rst @@ -0,0 +1,164 @@ +Ubuntu 20.04 LTS +================ + +This document describes installation from source. If you want to build a +``deb``, see :ref:`packaging-debian`. + +Installing Dependencies +----------------------- + +.. code-block:: console + + sudo apt update + sudo apt-get install \ + git autoconf automake libtool make libreadline-dev texinfo \ + pkg-config libpam0g-dev libjson-c-dev bison flex \ + libc-ares-dev python3-dev python3-sphinx \ + install-info build-essential libsnmp-dev perl \ + libcap-dev python2 libelf-dev libunwind-dev + +.. include:: building-libunwind-note.rst + +Note that Ubuntu 20 no longer installs python 2.x, so it must be +installed explicitly. Ensure that your system has a symlink named +``/usr/bin/python`` pointing at ``/usr/bin/python3``. + +In addition, ``pip`` for python2 must be installed if you wish to run +the FRR topotests. That version of ``pip`` is not available from the +ubuntu apt repositories; in order to install it: + +.. code-block:: shell + + curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py + sudo python2 ./get-pip.py + + # And verify the installation + pip2 --version + +.. include:: building-libyang.rst + +Protobuf +^^^^^^^^ + +.. code-block:: console + + sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + +ZeroMQ +^^^^^^ + +.. code-block:: console + + sudo apt-get install libzmq5 libzmq3-dev + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` and uncomment the following values (ignore the +other settings): + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +Ubuntu 20.04 ships with kernel 5.4; MPLS modules are present by default. To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +If the above command returns an error, you may need to install the appropriate +or latest linux-modules-extra-<kernel-version>-generic package. For example +``apt-get install linux-modules-extra-`uname -r`-generic`` + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install service files +^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: shell + + systemctl start frr diff --git a/doc/developer/building-frr-for-ubuntu2204.rst b/doc/developer/building-frr-for-ubuntu2204.rst new file mode 100644 index 0000000..97bdf88 --- /dev/null +++ b/doc/developer/building-frr-for-ubuntu2204.rst @@ -0,0 +1,183 @@ +Ubuntu 22.04 LTS +================ + +This document describes installation from source. If you want to build a +``deb``, see :ref:`packaging-debian`. + +Installing Dependencies +----------------------- + +.. code-block:: console + + sudo apt update + sudo apt-get install \ + git autoconf automake libtool make libreadline-dev texinfo \ + pkg-config libpam0g-dev libjson-c-dev bison flex \ + libc-ares-dev python3-dev python3-sphinx \ + install-info build-essential libsnmp-dev perl \ + libcap-dev python2 libelf-dev libunwind-dev \ + libyang2 libyang2-dev + +.. include:: building-libunwind-note.rst + +Note that Ubuntu >= 20 no longer installs python 2.x, so it must be +installed explicitly. Ensure that your system has a symlink named +``/usr/bin/python`` pointing at ``/usr/bin/python3``. + +.. code-block:: shell + + sudo ln -s /usr/bin/python3 /usr/bin/python + python --version + +In addition, ``pip`` for python2 must be installed if you wish to run +the FRR topotests. That version of ``pip`` is not available from the +ubuntu apt repositories; in order to install it: + +.. code-block:: shell + + curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py + sudo python2 ./get-pip.py + + # And verify the installation + pip2 --version + + +Protobuf +^^^^^^^^ +This is optional + +.. code-block:: console + + sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + + +Config Rollbacks +^^^^^^^^^^^^^^^^ + +If config rollbacks are enabled using ``--enable-config-rollbacks`` +the sqlite3 developer package also should be installed. + +.. code-block:: console + + sudo apt install libsqlite3-dev + + +ZeroMQ +^^^^^^ +This is optional + +.. code-block:: console + + sudo apt-get install libzmq5 libzmq3-dev + +Building & Installing FRR +------------------------- + +Add FRR user and groups +^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo groupadd -r -g 92 frr + sudo groupadd -r -g 85 frrvty + sudo adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRR suite" --shell /sbin/nologin frr + sudo usermod -a -G frrvty frr + +Compile +^^^^^^^ + +.. include:: include-compile.rst + +Install FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 775 -o frr -g frr -d /var/log/frr + sudo install -m 775 -o frr -g frrvty -d /etc/frr + sudo install -m 640 -o frr -g frrvty tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/frr.conf /etc/frr/frr.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons.conf /etc/frr/daemons.conf + sudo install -m 640 -o frr -g frr tools/etc/frr/daemons /etc/frr/daemons + +Tweak sysctls +^^^^^^^^^^^^^ + +Some sysctls need to be changed in order to enable IPv4/IPv6 forwarding and +MPLS (if supported by your platform). If your platform does not support MPLS, +skip the MPLS related configuration in this section. + +Edit :file:`/etc/sysctl.conf` and uncomment the following values (ignore the +other settings): + +:: + + # Uncomment the next line to enable packet forwarding for IPv4 + net.ipv4.ip_forward=1 + + # Uncomment the next line to enable packet forwarding for IPv6 + # Enabling this option disables Stateless Address Autoconfiguration + # based on Router Advertisements for this host + net.ipv6.conf.all.forwarding=1 + +Reboot or use ``sysctl -p`` to apply the same config to the running system. + +Add MPLS kernel modules +""""""""""""""""""""""" + +Ubuntu 20.04 ships with kernel 5.4; MPLS modules are present by default. To +enable, add the following lines to :file:`/etc/modules-load.d/modules.conf`: + +:: + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + +And load the kernel modules on the running system: + +.. code-block:: console + + sudo modprobe mpls-router mpls-iptunnel + +If the above command returns an error, you may need to install the appropriate +or latest linux-modules-extra-<kernel-version>-generic package. For example +``apt-get install linux-modules-extra-`uname -r`-generic`` + +Enable MPLS Forwarding +"""""""""""""""""""""" + +Edit :file:`/etc/sysctl.conf` and the following lines. Make sure to add a line +equal to :file:`net.mpls.conf.eth0.input` for each interface used with MPLS. + +:: + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + +Install service files +^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service + sudo systemctl enable frr + +Enable daemons +^^^^^^^^^^^^^^ + +Open :file:`/etc/frr/daemons` with your text editor of choice. Look for the +section with ``watchfrr_enable=...`` and ``zebra=...`` etc. Enable the daemons +as required by changing the value to ``yes``. + +Start FRR +^^^^^^^^^ + +.. code-block:: shell + + systemctl start frr diff --git a/doc/developer/building-libunwind-note.rst b/doc/developer/building-libunwind-note.rst new file mode 100644 index 0000000..0beb1f8 --- /dev/null +++ b/doc/developer/building-libunwind-note.rst @@ -0,0 +1,6 @@ +.. note:: + + The ``libunwind`` library is optional but highly recommended, as it improves + backtraces printed for crashes and debugging. However, if it is not + available for some reason, it can simply be left out without any loss of + functionality. diff --git a/doc/developer/building-libyang.rst b/doc/developer/building-libyang.rst new file mode 100644 index 0000000..c36cd34 --- /dev/null +++ b/doc/developer/building-libyang.rst @@ -0,0 +1,47 @@ +FRR depends on the relatively new ``libyang`` library to provide YANG/NETCONF +support. Unfortunately, most distributions do not yet offer a ``libyang`` +package from their repositories. Therefore we offer two options to install this +library. + +**Option 1: Binary Install** + +The FRR project builds some binary ``libyang`` packages. + +RPM packages are at our `RPM repository <https://rpm.frrouting.org>`_. + +DEB packages are available as CI artifacts `here +<https://ci1.netdef.org/browse/LIBYANG-LIBYANGV2/latestSuccessful/artifact>`_. + +.. warning:: + + ``libyang`` version 2.0.0 or newer is required to build FRR. + +.. note:: + + The ``libyang`` development packages need to be installed in addition to the + libyang core package in order to build FRR successfully. Make sure to + download and install those from the link above alongside the binary + packages. + + Depending on your platform, you may also need to install the PCRE + development package. Typically this is ``libpcre2-dev`` or ``pcre2-devel``. + +**Option 2: Source Install** + +.. note:: + + Ensure that the `libyang build requirements + <https://github.com/CESNET/libyang/#build-requirements>`_ + are met before continuing. Usually this entails installing ``cmake`` and + ``libpcre2-dev`` or ``pcre2-devel``. + +.. code-block:: console + + git clone https://github.com/CESNET/libyang.git + cd libyang + git checkout v2.0.0 + mkdir build; cd build + cmake -D CMAKE_INSTALL_PREFIX:PATH=/usr \ + -D CMAKE_BUILD_TYPE:String="Release" .. + make + sudo make install diff --git a/doc/developer/building.rst b/doc/developer/building.rst new file mode 100644 index 0000000..8ca0c13 --- /dev/null +++ b/doc/developer/building.rst @@ -0,0 +1,35 @@ +.. _building: + +************ +Building FRR +************ + +.. toctree:: + :maxdepth: 2 + + static-linking + building-frr-for-alpine + building-frr-for-archlinux + building-frr-for-centos6 + building-frr-for-centos7 + building-frr-for-centos8 + building-frr-for-debian8 + building-frr-for-debian9 + building-frr-for-debian12 + building-frr-for-fedora + building-frr-for-freebsd9 + building-frr-for-freebsd10 + building-frr-for-freebsd11 + building-frr-for-freebsd13 + building-frr-for-netbsd6 + building-frr-for-netbsd7 + building-frr-for-openbsd6 + building-frr-for-opensuse + building-frr-for-openwrt + building-frr-for-ubuntu1404 + building-frr-for-ubuntu1604 + building-frr-for-ubuntu1804 + building-frr-for-ubuntu2004 + building-frr-for-ubuntu2204 + building-docker + cross-compiling diff --git a/doc/developer/checkpatch.rst b/doc/developer/checkpatch.rst new file mode 100644 index 0000000..d8fe007 --- /dev/null +++ b/doc/developer/checkpatch.rst @@ -0,0 +1,1251 @@ +.. SPDX-License-Identifier: GPL-2.0-only + +.. _checkpatch: + +========== +Checkpatch +========== + +Checkpatch (scripts/checkpatch.pl) is a perl script which checks for trivial +style violations in patches and optionally corrects them. Checkpatch can +also be run on file contexts and without the kernel tree. + +Checkpatch is not always right. Your judgement takes precedence over checkpatch +messages. If your code looks better with the violations, then its probably +best left alone. + + +Options +======= + +This section will describe the options checkpatch can be run with. + +Usage:: + + ./scripts/checkpatch.pl [OPTION]... [FILE]... + +Available options: + + - -q, --quiet + + Enable quiet mode. + + - -v, --verbose + Enable verbose mode. Additional verbose test descriptions are output + so as to provide information on why that particular message is shown. + + - --no-tree + + Run checkpatch without the kernel tree. + + - --no-signoff + + Disable the 'Signed-off-by' line check. The sign-off is a simple line at + the end of the explanation for the patch, which certifies that you wrote it + or otherwise have the right to pass it on as an open-source patch. + + Example:: + + Signed-off-by: Random J Developer <random@developer.example.org> + + Setting this flag effectively stops a message for a missing signed-off-by + line in a patch context. + + - --patch + + Treat FILE as a patch. This is the default option and need not be + explicitly specified. + + - --emacs + + Set output to emacs compile window format. This allows emacs users to jump + from the error in the compile window directly to the offending line in the + patch. + + - --terse + + Output only one line per report. + + - --showfile + + Show the diffed file position instead of the input file position. + + - -g, --git + + Treat FILE as a single commit or a git revision range. + + Single commit with: + + - <rev> + - <rev>^ + - <rev>~n + + Multiple commits with: + + - <rev1>..<rev2> + - <rev1>...<rev2> + - <rev>-<count> + + - -f, --file + + Treat FILE as a regular source file. This option must be used when running + checkpatch on source files in the kernel. + + - --subjective, --strict + + Enable stricter tests in checkpatch. By default the tests emitted as CHECK + do not activate by default. Use this flag to activate the CHECK tests. + + - --list-types + + Every message emitted by checkpatch has an associated TYPE. Add this flag + to display all the types in checkpatch. + + Note that when this flag is active, checkpatch does not read the input FILE, + and no message is emitted. Only a list of types in checkpatch is output. + + - --types TYPE(,TYPE2...) + + Only display messages with the given types. + + Example:: + + ./scripts/checkpatch.pl mypatch.patch --types EMAIL_SUBJECT,BRACES + + - --ignore TYPE(,TYPE2...) + + Checkpatch will not emit messages for the specified types. + + Example:: + + ./scripts/checkpatch.pl mypatch.patch --ignore EMAIL_SUBJECT,BRACES + + - --show-types + + By default checkpatch doesn't display the type associated with the messages. + Set this flag to show the message type in the output. + + - --max-line-length=n + + Set the max line length (default 100). If a line exceeds the specified + length, a LONG_LINE message is emitted. + + + The message level is different for patch and file contexts. For patches, + a WARNING is emitted. While a milder CHECK is emitted for files. So for + file contexts, the --strict flag must also be enabled. + + - --min-conf-desc-length=n + + Set the Kconfig entry minimum description length, if shorter, warn. + + - --tab-size=n + + Set the number of spaces for tab (default 8). + + - --root=PATH + + PATH to the kernel tree root. + + This option must be specified when invoking checkpatch from outside + the kernel root. + + - --no-summary + + Suppress the per file summary. + + - --mailback + + Only produce a report in case of Warnings or Errors. Milder Checks are + excluded from this. + + - --summary-file + + Include the filename in summary. + + - --debug KEY=[0|1] + + Turn on/off debugging of KEY, where KEY is one of 'values', 'possible', + 'type', and 'attr' (default is all off). + + - --fix + + This is an EXPERIMENTAL feature. If correctable errors exists, a file + <inputfile>.EXPERIMENTAL-checkpatch-fixes is created which has the + automatically fixable errors corrected. + + - --fix-inplace + + EXPERIMENTAL - Similar to --fix but input file is overwritten with fixes. + + DO NOT USE this flag unless you are absolutely sure and you have a backup + in place. + + - --ignore-perl-version + + Override checking of perl version. Runtime errors maybe encountered after + enabling this flag if the perl version does not meet the minimum specified. + + - --codespell + + Use the codespell dictionary for checking spelling errors. + + - --codespellfile + + Use the specified codespell file. + Default is '/usr/share/codespell/dictionary.txt'. + + - --typedefsfile + + Read additional types from this file. + + - --color[=WHEN] + + Use colors 'always', 'never', or only when output is a terminal ('auto'). + Default is 'auto'. + + - --kconfig-prefix=WORD + + Use WORD as a prefix for Kconfig symbols (default is `CONFIG_`). + + - -h, --help, --version + + Display the help text. + +Message Levels +============== + +Messages in checkpatch are divided into three levels. The levels of messages +in checkpatch denote the severity of the error. They are: + + - ERROR + + This is the most strict level. Messages of type ERROR must be taken + seriously as they denote things that are very likely to be wrong. + + - WARNING + + This is the next stricter level. Messages of type WARNING requires a + more careful review. But it is milder than an ERROR. + + - CHECK + + This is the mildest level. These are things which may require some thought. + +Type Descriptions +================= + +This section contains a description of all the message types in checkpatch. + +.. Types in this section are also parsed by checkpatch. +.. The types are grouped into subsections based on use. + + +Allocation style +---------------- + + **ALLOC_ARRAY_ARGS** + The first argument for kcalloc or kmalloc_array should be the + number of elements. sizeof() as the first argument is generally + wrong. + + See: https://www.kernel.org/doc/html/latest/core-api/memory-allocation.html + + **ALLOC_SIZEOF_STRUCT** + The allocation style is bad. In general for family of + allocation functions using sizeof() to get memory size, + constructs like:: + + p = alloc(sizeof(struct foo), ...) + + should be:: + + p = alloc(sizeof(*p), ...) + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#allocating-memory + + **ALLOC_WITH_MULTIPLY** + Prefer kmalloc_array/kcalloc over kmalloc/kzalloc with a + sizeof multiply. + + See: https://www.kernel.org/doc/html/latest/core-api/memory-allocation.html + + +API usage +--------- + + **ARCH_DEFINES** + Architecture specific defines should be avoided wherever + possible. + + **ARCH_INCLUDE_LINUX** + Whenever asm/file.h is included and linux/file.h exists, a + conversion can be made when linux/file.h includes asm/file.h. + However this is not always the case (See signal.h). + This message type is emitted only for includes from arch/. + + **AVOID_BUG** + BUG() or BUG_ON() should be avoided totally. + Use WARN() and WARN_ON() instead, and handle the "impossible" + error condition as gracefully as possible. + + See: https://www.kernel.org/doc/html/latest/process/deprecated.html#bug-and-bug-on + + **CONSIDER_KSTRTO** + The simple_strtol(), simple_strtoll(), simple_strtoul(), and + simple_strtoull() functions explicitly ignore overflows, which + may lead to unexpected results in callers. The respective kstrtol(), + kstrtoll(), kstrtoul(), and kstrtoull() functions tend to be the + correct replacements. + + See: https://www.kernel.org/doc/html/latest/process/deprecated.html#simple-strtol-simple-strtoll-simple-strtoul-simple-strtoull + + **CONSTANT_CONVERSION** + Use of __constant_<foo> form is discouraged for the following functions:: + + __constant_cpu_to_be[x] + __constant_cpu_to_le[x] + __constant_be[x]_to_cpu + __constant_le[x]_to_cpu + __constant_htons + __constant_ntohs + + Using any of these outside of include/uapi/ is not preferred as using the + function without __constant_ is identical when the argument is a + constant. + + In big endian systems, the macros like __constant_cpu_to_be32(x) and + cpu_to_be32(x) expand to the same expression:: + + #define __constant_cpu_to_be32(x) ((__force __be32)(__u32)(x)) + #define __cpu_to_be32(x) ((__force __be32)(__u32)(x)) + + In little endian systems, the macros __constant_cpu_to_be32(x) and + cpu_to_be32(x) expand to __constant_swab32 and __swab32. __swab32 + has a __builtin_constant_p check:: + + #define __swab32(x) \ + (__builtin_constant_p((__u32)(x)) ? \ + ___constant_swab32(x) : \ + __fswab32(x)) + + So ultimately they have a special case for constants. + Similar is the case with all of the macros in the list. Thus + using the __constant_... forms are unnecessarily verbose and + not preferred outside of include/uapi. + + See: https://lore.kernel.org/lkml/1400106425.12666.6.camel@joe-AO725/ + + **DEPRECATED_API** + Usage of a deprecated RCU API is detected. It is recommended to replace + old flavourful RCU APIs by their new vanilla-RCU counterparts. + + The full list of available RCU APIs can be viewed from the kernel docs. + + See: https://www.kernel.org/doc/html/latest/RCU/whatisRCU.html#full-list-of-rcu-apis + + **DEPRECATED_VARIABLE** + EXTRA_{A,C,CPP,LD}FLAGS are deprecated and should be replaced by the new + flags added via commit f77bf01425b1 ("kbuild: introduce ccflags-y, + asflags-y and ldflags-y"). + + The following conversion scheme maybe used:: + + EXTRA_AFLAGS -> asflags-y + EXTRA_CFLAGS -> ccflags-y + EXTRA_CPPFLAGS -> cppflags-y + EXTRA_LDFLAGS -> ldflags-y + + See: + + 1. https://lore.kernel.org/lkml/20070930191054.GA15876@uranus.ravnborg.org/ + 2. https://lore.kernel.org/lkml/1313384834-24433-12-git-send-email-lacombar@gmail.com/ + 3. https://www.kernel.org/doc/html/latest/kbuild/makefiles.html#compilation-flags + + **DEVICE_ATTR_FUNCTIONS** + The function names used in DEVICE_ATTR is unusual. + Typically, the store and show functions are used with <attr>_store and + <attr>_show, where <attr> is a named attribute variable of the device. + + Consider the following examples:: + + static DEVICE_ATTR(type, 0444, type_show, NULL); + static DEVICE_ATTR(power, 0644, power_show, power_store); + + The function names should preferably follow the above pattern. + + See: https://www.kernel.org/doc/html/latest/driver-api/driver-model/device.html#attributes + + **DEVICE_ATTR_RO** + The DEVICE_ATTR_RO(name) helper macro can be used instead of + DEVICE_ATTR(name, 0444, name_show, NULL); + + Note that the macro automatically appends _show to the named + attribute variable of the device for the show method. + + See: https://www.kernel.org/doc/html/latest/driver-api/driver-model/device.html#attributes + + **DEVICE_ATTR_RW** + The DEVICE_ATTR_RW(name) helper macro can be used instead of + DEVICE_ATTR(name, 0644, name_show, name_store); + + Note that the macro automatically appends _show and _store to the + named attribute variable of the device for the show and store methods. + + See: https://www.kernel.org/doc/html/latest/driver-api/driver-model/device.html#attributes + + **DEVICE_ATTR_WO** + The DEVICE_AATR_WO(name) helper macro can be used instead of + DEVICE_ATTR(name, 0200, NULL, name_store); + + Note that the macro automatically appends _store to the + named attribute variable of the device for the store method. + + See: https://www.kernel.org/doc/html/latest/driver-api/driver-model/device.html#attributes + + **DUPLICATED_SYSCTL_CONST** + Commit d91bff3011cf ("proc/sysctl: add shared variables for range + check") added some shared const variables to be used instead of a local + copy in each source file. + + Consider replacing the sysctl range checking value with the shared + one in include/linux/sysctl.h. The following conversion scheme may + be used:: + + &zero -> SYSCTL_ZERO + &one -> SYSCTL_ONE + &int_max -> SYSCTL_INT_MAX + + See: + + 1. https://lore.kernel.org/lkml/20190430180111.10688-1-mcroce@redhat.com/ + 2. https://lore.kernel.org/lkml/20190531131422.14970-1-mcroce@redhat.com/ + + **ENOSYS** + ENOSYS means that a nonexistent system call was called. + Earlier, it was wrongly used for things like invalid operations on + otherwise valid syscalls. This should be avoided in new code. + + See: https://lore.kernel.org/lkml/5eb299021dec23c1a48fa7d9f2c8b794e967766d.1408730669.git.luto@amacapital.net/ + + **ENOTSUPP** + ENOTSUPP is not a standard error code and should be avoided in new patches. + EOPNOTSUPP should be used instead. + + See: https://lore.kernel.org/netdev/20200510182252.GA411829@lunn.ch/ + + **EXPORT_SYMBOL** + EXPORT_SYMBOL should immediately follow the symbol to be exported. + + **IN_ATOMIC** + in_atomic() is not for driver use so any such use is reported as an ERROR. + Also in_atomic() is often used to determine if sleeping is permitted, + but it is not reliable in this use model. Therefore its use is + strongly discouraged. + + However, in_atomic() is ok for core kernel use. + + See: https://lore.kernel.org/lkml/20080320201723.b87b3732.akpm@linux-foundation.org/ + + **LOCKDEP** + The lockdep_no_validate class was added as a temporary measure to + prevent warnings on conversion of device->sem to device->mutex. + It should not be used for any other purpose. + + See: https://lore.kernel.org/lkml/1268959062.9440.467.camel@laptop/ + + **MALFORMED_INCLUDE** + The #include statement has a malformed path. This has happened + because the author has included a double slash "//" in the pathname + accidentally. + + **USE_LOCKDEP** + lockdep_assert_held() annotations should be preferred over + assertions based on spin_is_locked() + + See: https://www.kernel.org/doc/html/latest/locking/lockdep-design.html#annotations + + **UAPI_INCLUDE** + No #include statements in include/uapi should use a uapi/ path. + + **USLEEP_RANGE** + usleep_range() should be preferred over udelay(). The proper way of + using usleep_range() is mentioned in the kernel docs. + + See: https://www.kernel.org/doc/html/latest/timers/timers-howto.html#delays-information-on-the-various-kernel-delay-sleep-mechanisms + + +Comments +-------- + + **BLOCK_COMMENT_STYLE** + The comment style is incorrect. The preferred style for multi- + line comments is:: + + /* + * This is the preferred style + * for multi line comments. + */ + + The networking comment style is a bit different, with the first line + not empty like the former:: + + /* This is the preferred comment style + * for files in net/ and drivers/net/ + */ + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#commenting + + **C99_COMMENTS** + C99 style single line comments (//) should not be used. + Prefer the block comment style instead. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#commenting + + **DATA_RACE** + Applications of data_race() should have a comment so as to document the + reasoning behind why it was deemed safe. + + See: https://lore.kernel.org/lkml/20200401101714.44781-1-elver@google.com/ + + **FSF_MAILING_ADDRESS** + Kernel maintainers reject new instances of the GPL boilerplate paragraph + directing people to write to the FSF for a copy of the GPL, since the + FSF has moved in the past and may do so again. + So do not write paragraphs about writing to the Free Software Foundation's + mailing address. + + See: https://lore.kernel.org/lkml/20131006222342.GT19510@leaf/ + + +Commit message +-------------- + + **BAD_SIGN_OFF** + The signed-off-by line does not fall in line with the standards + specified by the community. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#developer-s-certificate-of-origin-1-1 + + **BAD_STABLE_ADDRESS_STYLE** + The email format for stable is incorrect. + Some valid options for stable address are:: + + 1. stable@vger.kernel.org + 2. stable@kernel.org + + For adding version info, the following comment style should be used:: + + stable@vger.kernel.org # version info + + **COMMIT_COMMENT_SYMBOL** + Commit log lines starting with a '#' are ignored by git as + comments. To solve this problem addition of a single space + infront of the log line is enough. + + **COMMIT_MESSAGE** + The patch is missing a commit description. A brief + description of the changes made by the patch should be added. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#describe-your-changes + + **EMAIL_SUBJECT** + Naming the tool that found the issue is not very useful in the + subject line. A good subject line summarizes the change that + the patch brings. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#describe-your-changes + + **FROM_SIGN_OFF_MISMATCH** + The author's email does not match with that in the Signed-off-by: + line(s). This can be sometimes caused due to an improperly configured + email client. + + This message is emitted due to any of the following reasons:: + + - The email names do not match. + - The email addresses do not match. + - The email subaddresses do not match. + - The email comments do not match. + + **MISSING_SIGN_OFF** + The patch is missing a Signed-off-by line. A signed-off-by + line should be added according to Developer's certificate of + Origin. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin + + **NO_AUTHOR_SIGN_OFF** + The author of the patch has not signed off the patch. It is + required that a simple sign off line should be present at the + end of explanation of the patch to denote that the author has + written it or otherwise has the rights to pass it on as an open + source patch. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin + + **DIFF_IN_COMMIT_MSG** + Avoid having diff content in commit message. + This causes problems when one tries to apply a file containing both + the changelog and the diff because patch(1) tries to apply the diff + which it found in the changelog. + + See: https://lore.kernel.org/lkml/20150611134006.9df79a893e3636019ad2759e@linux-foundation.org/ + + **GERRIT_CHANGE_ID** + To be picked up by gerrit, the footer of the commit message might + have a Change-Id like:: + + Change-Id: Ic8aaa0728a43936cd4c6e1ed590e01ba8f0fbf5b + Signed-off-by: A. U. Thor <author@example.com> + + The Change-Id line must be removed before submitting. + + **GIT_COMMIT_ID** + The proper way to reference a commit id is: + commit <12+ chars of sha1> ("<title line>") + + An example may be:: + + Commit e21d2170f36602ae2708 ("video: remove unnecessary + platform_set_drvdata()") removed the unnecessary + platform_set_drvdata(), but left the variable "dev" unused, + delete it. + + See: https://www.kernel.org/doc/html/latest/process/submitting-patches.html#describe-your-changes + + +Comparison style +---------------- + + **ASSIGN_IN_IF** + Do not use assignments in if condition. + Example:: + + if ((foo = bar(...)) < BAZ) { + + should be written as:: + + foo = bar(...); + if (foo < BAZ) { + + **BOOL_COMPARISON** + Comparisons of A to true and false are better written + as A and !A. + + See: https://lore.kernel.org/lkml/1365563834.27174.12.camel@joe-AO722/ + + **COMPARISON_TO_NULL** + Comparisons to NULL in the form (foo == NULL) or (foo != NULL) + are better written as (!foo) and (foo). + + **CONSTANT_COMPARISON** + Comparisons with a constant or upper case identifier on the left + side of the test should be avoided. + + +Indentation and Line Breaks +--------------------------- + + **CODE_INDENT** + Code indent should use tabs instead of spaces. + Outside of comments, documentation and Kconfig, + spaces are never used for indentation. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#indentation + + **DEEP_INDENTATION** + Indentation with 6 or more tabs usually indicate overly indented + code. + + It is suggested to refactor excessive indentation of + if/else/for/do/while/switch statements. + + See: https://lore.kernel.org/lkml/1328311239.21255.24.camel@joe2Laptop/ + + **SWITCH_CASE_INDENT_LEVEL** + switch should be at the same indent as case. + Example:: + + switch (suffix) { + case 'G': + case 'g': + mem <<= 30; + break; + case 'M': + case 'm': + mem <<= 20; + break; + case 'K': + case 'k': + mem <<= 10; + fallthrough; + default: + break; + } + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#indentation + + **LONG_LINE** + The line has exceeded the specified maximum length. + To use a different maximum line length, the --max-line-length=n option + may be added while invoking checkpatch. + + Earlier, the default line length was 80 columns. Commit bdc48fa11e46 + ("checkpatch/coding-style: deprecate 80-column warning") increased the + limit to 100 columns. This is not a hard limit either and it's + preferable to stay within 80 columns whenever possible. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#breaking-long-lines-and-strings + + **LONG_LINE_STRING** + A string starts before but extends beyond the maximum line length. + To use a different maximum line length, the --max-line-length=n option + may be added while invoking checkpatch. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#breaking-long-lines-and-strings + + **LONG_LINE_COMMENT** + A comment starts before but extends beyond the maximum line length. + To use a different maximum line length, the --max-line-length=n option + may be added while invoking checkpatch. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#breaking-long-lines-and-strings + + **SPLIT_STRING** + Quoted strings that appear as messages in userspace and can be + grepped, should not be split across multiple lines. + + See: https://lore.kernel.org/lkml/20120203052727.GA15035@leaf/ + + **MULTILINE_DEREFERENCE** + A single dereferencing identifier spanned on multiple lines like:: + + struct_identifier->member[index]. + member = <foo>; + + is generally hard to follow. It can easily lead to typos and so makes + the code vulnerable to bugs. + + If fixing the multiple line dereferencing leads to an 80 column + violation, then either rewrite the code in a more simple way or if the + starting part of the dereferencing identifier is the same and used at + multiple places then store it in a temporary variable, and use that + temporary variable only at all the places. For example, if there are + two dereferencing identifiers:: + + member1->member2->member3.foo1; + member1->member2->member3.foo2; + + then store the member1->member2->member3 part in a temporary variable. + It not only helps to avoid the 80 column violation but also reduces + the program size by removing the unnecessary dereferences. + + But if none of the above methods work then ignore the 80 column + violation because it is much easier to read a dereferencing identifier + on a single line. + + **TRAILING_STATEMENTS** + Trailing statements (for example after any conditional) should be + on the next line. + Statements, such as:: + + if (x == y) break; + + should be:: + + if (x == y) + break; + + +Macros, Attributes and Symbols +------------------------------ + + **ARRAY_SIZE** + The ARRAY_SIZE(foo) macro should be preferred over + sizeof(foo)/sizeof(foo[0]) for finding number of elements in an + array. + + The macro is defined in include/linux/kernel.h:: + + #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) + + **AVOID_EXTERNS** + Function prototypes don't need to be declared extern in .h + files. It's assumed by the compiler and is unnecessary. + + **AVOID_L_PREFIX** + Local symbol names that are prefixed with `.L` should be avoided, + as this has special meaning for the assembler; a symbol entry will + not be emitted into the symbol table. This can prevent `objtool` + from generating correct unwind info. + + Symbols with STB_LOCAL binding may still be used, and `.L` prefixed + local symbol names are still generally usable within a function, + but `.L` prefixed local symbol names should not be used to denote + the beginning or end of code regions via + `SYM_CODE_START_LOCAL`/`SYM_CODE_END` + + **BIT_MACRO** + Defines like: 1 << <digit> could be BIT(digit). + The BIT() macro is defined via include/linux/bits.h:: + + #define BIT(nr) (1UL << (nr)) + + **CONST_READ_MOSTLY** + When a variable is tagged with the __read_mostly annotation, it is a + signal to the compiler that accesses to the variable will be mostly + reads and rarely(but NOT never) a write. + + const __read_mostly does not make any sense as const data is already + read-only. The __read_mostly annotation thus should be removed. + + **DATE_TIME** + It is generally desirable that building the same source code with + the same set of tools is reproducible, i.e. the output is always + exactly the same. + + The kernel does *not* use the ``__DATE__`` and ``__TIME__`` macros, + and enables warnings if they are used as they can lead to + non-deterministic builds. + + See: https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html#timestamps + + **DEFINE_ARCH_HAS** + The ARCH_HAS_xyz and ARCH_HAVE_xyz patterns are wrong. + + For big conceptual features use Kconfig symbols instead. And for + smaller things where we have compatibility fallback functions but + want architectures able to override them with optimized ones, we + should either use weak functions (appropriate for some cases), or + the symbol that protects them should be the same symbol we use. + + See: https://lore.kernel.org/lkml/CA+55aFycQ9XJvEOsiM3txHL5bjUc8CeKWJNR_H+MiicaddB42Q@mail.gmail.com/ + + **DO_WHILE_MACRO_WITH_TRAILING_SEMICOLON** + do {} while(0) macros should not have a trailing semicolon. + + **INIT_ATTRIBUTE** + Const init definitions should use __initconst instead of + __initdata. + + Similarly init definitions without const require a separate + use of const. + + **INLINE_LOCATION** + The inline keyword should sit between storage class and type. + + For example, the following segment:: + + inline static int example_function(void) + { + ... + } + + should be:: + + static inline int example_function(void) + { + ... + } + + **MISPLACED_INIT** + It is possible to use section markers on variables in a way + which gcc doesn't understand (or at least not the way the + developer intended):: + + static struct __initdata samsung_pll_clock exynos4_plls[nr_plls] = { + + does not put exynos4_plls in the .initdata section. The __initdata + marker can be virtually anywhere on the line, except right after + "struct". The preferred location is before the "=" sign if there is + one, or before the trailing ";" otherwise. + + See: https://lore.kernel.org/lkml/1377655732.3619.19.camel@joe-AO722/ + + **MULTISTATEMENT_MACRO_USE_DO_WHILE** + Macros with multiple statements should be enclosed in a + do - while block. Same should also be the case for macros + starting with `if` to avoid logic defects:: + + #define macrofun(a, b, c) \ + do { \ + if (a == 5) \ + do_this(b, c); \ + } while (0) + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#macros-enums-and-rtl + + **PREFER_FALLTHROUGH** + Use the `fallthrough;` pseudo keyword instead of + `/* fallthrough */` like comments. + + **TRAILING_SEMICOLON** + Macro definition should not end with a semicolon. The macro + invocation style should be consistent with function calls. + This can prevent any unexpected code paths:: + + #define MAC do_something; + + If this macro is used within a if else statement, like:: + + if (some_condition) + MAC; + + else + do_something; + + Then there would be a compilation error, because when the macro is + expanded there are two trailing semicolons, so the else branch gets + orphaned. + + See: https://lore.kernel.org/lkml/1399671106.2912.21.camel@joe-AO725/ + + **SINGLE_STATEMENT_DO_WHILE_MACRO** + For the multi-statement macros, it is necessary to use the do-while + loop to avoid unpredictable code paths. The do-while loop helps to + group the multiple statements into a single one so that a + function-like macro can be used as a function only. + + But for the single statement macros, it is unnecessary to use the + do-while loop. Although the code is syntactically correct but using + the do-while loop is redundant. So remove the do-while loop for single + statement macros. + + **WEAK_DECLARATION** + Using weak declarations like __attribute__((weak)) or __weak + can have unintended link defects. Avoid using them. + + +Functions and Variables +----------------------- + + **CAMELCASE** + Avoid CamelCase Identifiers. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#naming + + **CONST_CONST** + Using `const <type> const *` is generally meant to be + written `const <type> * const`. + + **CONST_STRUCT** + Using const is generally a good idea. Checkpatch reads + a list of frequently used structs that are always or + almost always constant. + + The existing structs list can be viewed from + `scripts/const_structs.checkpatch`. + + See: https://lore.kernel.org/lkml/alpine.DEB.2.10.1608281509480.3321@hadrien/ + + **EMBEDDED_FUNCTION_NAME** + Embedded function names are less appropriate to use as + refactoring can cause function renaming. Prefer the use of + "%s", __func__ to embedded function names. + + Note that this does not work with -f (--file) checkpatch option + as it depends on patch context providing the function name. + + **FUNCTION_ARGUMENTS** + This warning is emitted due to any of the following reasons: + + 1. Arguments for the function declaration do not follow + the identifier name. Example:: + + void foo + (int bar, int baz) + + This should be corrected to:: + + void foo(int bar, int baz) + + 2. Some arguments for the function definition do not + have an identifier name. Example:: + + void foo(int) + + All arguments should have identifier names. + + **FUNCTION_WITHOUT_ARGS** + Function declarations without arguments like:: + + int foo() + + should be:: + + int foo(void) + + **GLOBAL_INITIALISERS** + Global variables should not be initialized explicitly to + 0 (or NULL, false, etc.). Your compiler (or rather your + loader, which is responsible for zeroing out the relevant + sections) automatically does it for you. + + **INITIALISED_STATIC** + Static variables should not be initialized explicitly to zero. + Your compiler (or rather your loader) automatically does + it for you. + + **MULTIPLE_ASSIGNMENTS** + Multiple assignments on a single line makes the code unnecessarily + complicated. So on a single line assign value to a single variable + only, this makes the code more readable and helps avoid typos. + + **RETURN_PARENTHESES** + return is not a function and as such doesn't need parentheses:: + + return (bar); + + can simply be:: + + return bar; + + +Permissions +----------- + + **DEVICE_ATTR_PERMS** + The permissions used in DEVICE_ATTR are unusual. + Typically only three permissions are used - 0644 (RW), 0444 (RO) + and 0200 (WO). + + See: https://www.kernel.org/doc/html/latest/filesystems/sysfs.html#attributes + + **EXECUTE_PERMISSIONS** + There is no reason for source files to be executable. The executable + bit can be removed safely. + + **EXPORTED_WORLD_WRITABLE** + Exporting world writable sysfs/debugfs files is usually a bad thing. + When done arbitrarily they can introduce serious security bugs. + In the past, some of the debugfs vulnerabilities would seemingly allow + any local user to write arbitrary values into device registers - a + situation from which little good can be expected to emerge. + + See: https://lore.kernel.org/linux-arm-kernel/cover.1296818921.git.segoon@openwall.com/ + + **NON_OCTAL_PERMISSIONS** + Permission bits should use 4 digit octal permissions (like 0700 or 0444). + Avoid using any other base like decimal. + + **SYMBOLIC_PERMS** + Permission bits in the octal form are more readable and easier to + understand than their symbolic counterparts because many command-line + tools use this notation. Experienced kernel developers have been using + these traditional Unix permission bits for decades and so they find it + easier to understand the octal notation than the symbolic macros. + For example, it is harder to read S_IWUSR|S_IRUGO than 0644, which + obscures the developer's intent rather than clarifying it. + + See: https://lore.kernel.org/lkml/CA+55aFw5v23T-zvDZp-MmD_EYxF8WbafwwB59934FV7g21uMGQ@mail.gmail.com/ + + +Spacing and Brackets +-------------------- + + **ASSIGNMENT_CONTINUATIONS** + Assignment operators should not be written at the start of a + line but should follow the operand at the previous line. + + **BRACES** + The placement of braces is stylistically incorrect. + The preferred way is to put the opening brace last on the line, + and put the closing brace first:: + + if (x is true) { + we do y + } + + This applies for all non-functional blocks. + However, there is one special case, namely functions: they have the + opening brace at the beginning of the next line, thus:: + + int function(int x) + { + body of function + } + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#placing-braces-and-spaces + + **BRACKET_SPACE** + Whitespace before opening bracket '[' is prohibited. + There are some exceptions: + + 1. With a type on the left:: + + int [] a; + + 2. At the beginning of a line for slice initialisers:: + + [0...10] = 5, + + 3. Inside a curly brace:: + + = { [0...10] = 5 } + + **CONCATENATED_STRING** + Concatenated elements should have a space in between. + Example:: + + printk(KERN_INFO"bar"); + + should be:: + + printk(KERN_INFO "bar"); + + **ELSE_AFTER_BRACE** + `else {` should follow the closing block `}` on the same line. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#placing-braces-and-spaces + + **LINE_SPACING** + Vertical space is wasted given the limited number of lines an + editor window can display when multiple blank lines are used. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#spaces + + **OPEN_BRACE** + The opening brace should be following the function definitions on the + next line. For any non-functional block it should be on the same line + as the last construct. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#placing-braces-and-spaces + + **POINTER_LOCATION** + When using pointer data or a function that returns a pointer type, + the preferred use of * is adjacent to the data name or function name + and not adjacent to the type name. + Examples:: + + char *linux_banner; + unsigned long long memparse(char *ptr, char **retptr); + char *match_strdup(substring_t *s); + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#spaces + + **SPACING** + Whitespace style used in the kernel sources is described in kernel docs. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#spaces + + **TRAILING_WHITESPACE** + Trailing whitespace should always be removed. + Some editors highlight the trailing whitespace and cause visual + distractions when editing files. + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#spaces + + **UNNECESSARY_PARENTHESES** + Parentheses are not required in the following cases: + + 1. Function pointer uses:: + + (foo->bar)(); + + could be:: + + foo->bar(); + + 2. Comparisons in if:: + + if ((foo->bar) && (foo->baz)) + if ((foo == bar)) + + could be:: + + if (foo->bar && foo->baz) + if (foo == bar) + + 3. addressof/dereference single Lvalues:: + + &(foo->bar) + *(foo->bar) + + could be:: + + &foo->bar + *foo->bar + + **WHILE_AFTER_BRACE** + while should follow the closing bracket on the same line:: + + do { + ... + } while(something); + + See: https://www.kernel.org/doc/html/latest/process/coding-style.html#placing-braces-and-spaces + + +Others +------ + + **CONFIG_DESCRIPTION** + Kconfig symbols should have a help text which fully describes + it. + + **CORRUPTED_PATCH** + The patch seems to be corrupted or lines are wrapped. + Please regenerate the patch file before sending it to the maintainer. + + **CVS_KEYWORD** + Since linux moved to git, the CVS markers are no longer used. + So, CVS style keywords ($Id$, $Revision$, $Log$) should not be + added. + + **DEFAULT_NO_BREAK** + switch default case is sometimes written as "default:;". This can + cause new cases added below default to be defective. + + A "break;" should be added after empty default statement to avoid + unwanted fallthrough. + + **DOS_LINE_ENDINGS** + For DOS-formatted patches, there are extra ^M symbols at the end of + the line. These should be removed. + + **DT_SCHEMA_BINDING_PATCH** + DT bindings moved to a json-schema based format instead of + freeform text. + + See: https://www.kernel.org/doc/html/latest/devicetree/bindings/writing-schema.html + + **DT_SPLIT_BINDING_PATCH** + Devicetree bindings should be their own patch. This is because + bindings are logically independent from a driver implementation, + they have a different maintainer (even though they often + are applied via the same tree), and it makes for a cleaner history in the + DT only tree created with git-filter-branch. + + See: https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters + + **EMBEDDED_FILENAME** + Embedding the complete filename path inside the file isn't particularly + useful as often the path is moved around and becomes incorrect. + + **FILE_PATH_CHANGES** + Whenever files are added, moved, or deleted, the MAINTAINERS file + patterns can be out of sync or outdated. + + So MAINTAINERS might need updating in these cases. + + **MEMSET** + The memset use appears to be incorrect. This may be caused due to + badly ordered parameters. Please recheck the usage. + + **NOT_UNIFIED_DIFF** + The patch file does not appear to be in unified-diff format. Please + regenerate the patch file before sending it to the maintainer. + + **PRINTF_0XDECIMAL** + Prefixing 0x with decimal output is defective and should be corrected. + + **SPDX_LICENSE_TAG** + The source file is missing or has an improper SPDX identifier tag. + The Linux kernel requires the precise SPDX identifier in all source files, + and it is thoroughly documented in the kernel docs. + + See: https://www.kernel.org/doc/html/latest/process/license-rules.html + + **TYPO_SPELLING** + Some words may have been misspelled. Consider reviewing them. diff --git a/doc/developer/cli.rst b/doc/developer/cli.rst new file mode 100644 index 0000000..59073b3 --- /dev/null +++ b/doc/developer/cli.rst @@ -0,0 +1,1007 @@ +.. _command-line-interface: + +Command Line Interface +====================== + +FRR features a flexible modal command line interface. Often when adding new +features or modifying existing code it is necessary to create or modify CLI +commands. FRR has a powerful internal CLI system that does most of the heavy +lifting for you. + +Modes +----- +FRR's CLI is organized by modes. Each mode is associated with some set of +functionality, e.g. EVPN, or some underlying object such as an interface. Each +mode contains a set of commands that control the associated functionality or +object. Users move between the modes by entering a command, which is usually +different for each source and destination mode. + +A summary of the modes is given in the following figure. + +.. graphviz:: ../figures/nodes.dot + +.. seealso:: :ref:`cli-data-structures` + +Walkup +^^^^^^ +FRR exhibits, for historical reasons, a peculiar behavior called 'walkup'. +Suppose a user is in ``OSPF_NODE``, which contains only OSPF-specific commands, +and enters the following command: :: + + ip route 192.168.100.0/24 10.0.2.2 + +This command is not defined in ``OSPF_NODE``, so the matcher will fail to match +the command in that node. The matcher will then check "parent" nodes of +``OSPF_NODE``. In this case the direct parent of ``OSPF_NODE`` is +``CONFIG_NODE``, so the current node switches to ``CONFIG_NODE`` and the command +is tried in that node. Since static route commands are defined in +``CONFIG_NODE`` the command succeeds. The procedure of attempting to execute +unmatched commands by sequentially "walking up" to parent nodes only happens in +children (direct and indirect) below ``CONFIG_NODE`` and stops at +``CONFIG_NODE``. + +Unfortunately, the internal representation of the various modes is not actually +a graph. Instead, there is an array. The parent-child relationships are not +explicitly defined in any datastructure but instead are hard-coded into the +specific commands that switch nodes. For walkup, there is a function that takes +a node and returns the parent of the node. This interface causes all manner of +insidious problems, even for experienced developers, and needs to be fixed at +some point in the future. + +Deprecation of old style of commands +------------------------------------ + +There are currently 2 styles of defining commands within a FRR source file. +``DEFUN`` and ``DEFPY``. ``DEFPY`` should be used for all new commands that +a developer is writing. This is because it allows for much better handling +of command line arguments as well as ensuring that input is correct. ``DEFUN`` +is listed here for historical reasons as well as for ensuring that existing +code can be understood by new developers. + +Defining Commands +----------------- +All definitions for the CLI system are exposed in ``lib/command.h``. In this +header there are a set of macros used to define commands. These macros are +collectively referred to as "DEFUNs", because of their syntax: + +:: + + DEFUN(command_name, + command_name_cmd, + "example command FOO...", + "Examples\n" + "CLI command\n" + "Argument\n") + { + // ...command handler... + } + +DEFUNs generally take four arguments which are expanded into the appropriate +constructs for hooking into the CLI. In order these are: + +- **Function name** - the name of the handler function for the command +- **Command name** - the identifier of the ``struct cmd_element`` for the + command. By convention this should be the function name with ``_cmd`` + appended. +- **Command definition** - an expression in FRR's CLI grammar that defines the + form of the command and its arguments, if any +- **Doc string** - a newline-delimited string that documents each element in + the command definition + +In the above example, ``command_name`` is the function name, +``command_name_cmd`` is the command name, ``"example..."`` is the definition and +the last argument is the doc string. The block following the macro is the body +of the handler function, details on which are presented later in this section. + +In order to make the command show up to the user it must be installed into the +CLI graph. To do this, call: + +``install_element(NODE, &command_name_cmd);`` + +This will install the command into the specified CLI node. Usually these calls +are grouped together in a CLI initialization function for a set of commands, and +the DEFUNs themselves are grouped into the same source file to avoid cluttering +the codebase. The names of these files follow the form ``*_vty.[ch]`` by +convention. Please do not scatter individual CLI commands in the middle of +source files; instead expose the necessary functions in a header and place the +command definition in a ``*_vty.[ch]`` file. + +.. note:: + + Please see :ref:`cli-workflow` for requirements when creating CLI commands + (e.g., JSON structure and formatting). + +Definition Grammar +^^^^^^^^^^^^^^^^^^ +FRR uses its own grammar for defining CLI commands. The grammar draws from +syntax commonly seen in \*nix manpages and should be fairly intuitive. The +parser is implemented in Bison and the lexer in Flex. These may be found in +``lib/command_parse.y`` and ``lib/command_lex.l``, respectively. + + **ProTip**: if you define a new command and find that the parser is + throwing syntax or other errors, the parser is the last place you want + to look. Bison is very stable and if it detects a syntax error, 99% of + the time it will be a syntax error in your definition. + +The formal grammar in BNF is given below. This is the grammar implemented in the +Bison parser. At runtime, the Bison parser reads all of the CLI strings and +builds a combined directed graph that is used to match and interpret user input. + +Human-friendly explanations of how to use this grammar are given a bit later in +this section alongside information on the :ref:`cli-data-structures` constructed +by the parser. + +.. productionlist:: + command: `cmd_token_seq` + : `cmd_token_seq` `placeholder_token` "..." + cmd_token_seq: *empty* + : `cmd_token_seq` `cmd_token` + cmd_token: `simple_token` + : `selector` + simple_token: `literal_token` + : `placeholder_token` + literal_token: WORD `varname_token` + varname_token: "$" WORD + placeholder_token: `placeholder_token_real` `varname_token` + placeholder_token_real: IPV4 + : IPV4_PREFIX + : IPV6 + : IPV6_PREFIX + : VARIABLE + : RANGE + : MAC + : MAC_PREFIX + : ASNUM + selector: "<" `selector_seq_seq` ">" `varname_token` + : "{" `selector_seq_seq` "}" `varname_token` + : "[" `selector_seq_seq` "]" `varname_token` + : "![" `selector_seq_seq` "]" `varname_token` + selector_seq_seq: `selector_seq_seq` "|" `selector_token_seq` + : `selector_token_seq` + selector_token_seq: `selector_token_seq` `selector_token` + : `selector_token` + selector_token: `selector` + : `simple_token` + +Tokens +^^^^^^ +The various capitalized tokens in the BNF above are in fact themselves +placeholders, but not defined as such in the formal grammar; the grammar +provides the structure, and the tokens are actually more like a type system for +the strings you write in your CLI definitions. A CLI definition string is broken +apart and each piece is assigned a type by the lexer based on a set of regular +expressions. The parser uses the type information to verify the string and +determine the structure of the CLI graph; additional metadata (such as the raw +text of each token) is encoded into the graph as it is constructed by the +parser, but this is merely a dumb copy job. + +Here is a brief summary of the various token types along with examples. + ++-----------------+--------------------------+-------------------------------------------------------+ +| Token type | Syntax | Description | ++=================+==========================+=======================================================+ +| ``WORD`` | ``show ip bgp`` | Matches itself. In the example every token is a WORD. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``IPV4`` | ``A.B.C.D`` | Matches an IPv4 address. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``IPV6`` | ``X:X::X:X`` | Matches an IPv6 address. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``IPV4_PREFIX`` | ``A.B.C.D/M`` | Matches an IPv4 prefix in CIDR notation. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``IPV6_PREFIX`` | ``X:X::X:X/M`` | Matches an IPv6 prefix in CIDR notation. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``MAC`` | ``X:X:X:X:X:X`` | Matches a 48-bit mac address. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``MAC_PREFIX`` | ``X:X:X:X:X:X/M`` | Matches a 48-bit mac address with a mask. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``VARIABLE`` | ``FOOBAR`` | Matches anything. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``RANGE`` | ``(X-Y)`` | Matches numbers in the range X..Y inclusive. | ++-----------------+--------------------------+-------------------------------------------------------+ +| ``ASNUM`` | ``<A.B|(1-4294967295)>`` | Matches an AS in plain or dot format. | ++-----------------+--------------------------+-------------------------------------------------------+ + +When presented with user input, the parser will search over all defined +commands in the current context to find a match. It is aware of the various +types of user input and has a ranking system to help disambiguate commands. For +instance, suppose the following commands are defined in the user's current +context: + +:: + + example command FOO + example command (22-49) + example command A.B.C.D/X + +The following table demonstrates the matcher's choice for a selection of +possible user input. + ++---------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+ +| Input | Matched command | Reason | ++=================================+===========================+==============================================================================================================+ +| ``example command eLi7eH4xx0r`` | example command FOO | ``eLi7eH4xx0r`` is not an integer or IPv4 prefix, | +| | | but FOO is a variable and matches all input. | ++---------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+ +| ``example command 42`` | example command (22-49) | ``42`` is not an IPv4 prefix. It does match both | +| | | ``(22-49)`` and ``FOO``, but RANGE tokens are more specific and have a higher priority than VARIABLE tokens. | ++---------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+ +| ``example command 10.3.3.0/24`` | example command A.B.C.D/X | The user entered an IPv4 prefix, which is best matched by the last command. | ++---------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+ + +Rules +^^^^^ +There are also constructs which allow optional tokens, mutual exclusion, +one-or-more selection and repetition. + +- ``<angle|brackets>`` -- Contain sequences of tokens separated by pipes and + provide mutual exclusion. User input matches at most one option. +- ``[square brackets]`` -- Contains sequences of tokens that can be omitted. + ``[<a|b>]`` can be shortened to ``[a|b]``. +- ``![exclamation square brackets]`` -- same as ``[square brackets]``, but + only allow skipping the contents if the command input starts with ``no``. + (For cases where the positive command needs a parameter, but the parameter + is optional for the negative case.) +- ``{curly|braces}`` -- similar to angle brackets, but instead of mutual + exclusion, curly braces indicate that one or more of the pipe-separated + sequences may be provided in any order. +- ``VARIADICS...`` -- Any token which accepts input (anything except WORD) + which occurs as the last token of a line may be followed by an ellipsis, + which indicates that input matching the token may be repeated an unlimited + number of times. +- ``$name`` -- Specify a variable name for the preceding token. See + "Variable Names" below. + +Some general notes: + +- Options are allowed at the beginning of the command. The developer is + entreated to use these extremely sparingly. They are most useful for + implementing the 'no' form of configuration commands. Please think carefully + before using them for anything else. There is usually a better solution, even + if it is just separating out the command definition into separate ones. +- The developer should judiciously apply separation of concerns when defining + commands. CLI definitions for two unrelated or vaguely related commands or + configuration items should be defined in separate commands. Clarity is + preferred over LOC (within reason). +- The maximum number of space-separated tokens that can be entered is + presently limited to 256. Please keep this limit in mind when + implementing new CLI. + +Variable Names +^^^^^^^^^^^^^^ +The parser tries to fill the "varname" field on each token. This can happen +either manually or automatically. Manual specifications work by appending +``$name`` after the input specifier: + +:: + + foo bar$cmd WORD$name A.B.C.D$ip + +Note that you can also assign variable names to fixed input tokens, this can be +useful if multiple commands share code. You can also use "$name" after a +multiple-choice option: + +:: + + foo bar <A.B.C.D|X:X::X:X>$addr [optionA|optionB]$mode + +The variable name is in this case assigned to the last token in each of the +branches. + +Automatic assignment of variable names works by applying the following rules: + +- manual names always have priority +- a ``[no]`` at the beginning receives ``no`` as varname on the ``no`` token +- ``VARIABLE`` tokens whose text is not ``WORD`` or ``NAME`` receive a cleaned + lowercase version of the token text as varname, e.g. ``ROUTE-MAP`` becomes + ``route_map``. +- other variable tokens (i.e. everything except "fixed") receive the text of + the preceding fixed token as varname, if one can be found. E.g. + ``ip route A.B.C.D/M INTERFACE`` assigns "route" to the ``A.B.C.D/M`` token. + +These rules should make it possible to avoid manual varname assignment in 90% of +the cases. + +Doc Strings +^^^^^^^^^^^ +Each token in a command definition should be documented with a brief doc string +that informs a user of the meaning and/or purpose of the subsequent command +tree. These strings are provided as the last parameter to DEFUN macros, +concatenated together and separated by an escaped newline (``\n``). These are +best explained by example. + +:: + + DEFUN (config_terminal, + config_terminal_cmd, + "configure terminal", + "Configuration from vty interface\n" + "Configuration terminal\n") + +The last parameter is split into two lines for readability. Two newline +delimited doc strings are present, one for each token in the command. The second +string documents the functionality of the ``terminal`` command in the +``configure`` subtree. + +Note that the first string, for ``configure`` does not contain documentation for +'terminal'. This is because the CLI is best envisioned as a tree, with tokens +defining branches. An imaginary ``start`` token is the root of every command in +a CLI node. Each subsequent written token descends into a subtree, so the +documentation for that token ideally summarizes all the functionality contained +in the subtree. + +A consequence of this structure is that the developer must be careful to use the +same doc strings when defining multiple commands that are part of the same tree. +Commands which share prefixes must share the same doc strings for those +prefixes. On startup the parser will generate warnings if it notices +inconsistent doc strings. Behavior is undefined; the same token may show up +twice in completions, with different doc strings, or it may show up once with a +random doc string. Parser warnings should be heeded and fixed to avoid confusing +users. + +The number of doc strings provided must be equal to the amount of tokens present +in the command definition, read left to right, ignoring any special constructs. + +In the examples below, each arrowed token needs a doc string. + +:: + + "show ip bgp" + ^ ^ ^ + + "command <foo|bar> [example]" + ^ ^ ^ ^ + +DEFPY +^^^^^ +``DEFPY(...)`` is an enhanced version of ``DEFUN()`` which is preprocessed by +:file:`python/clidef.py`. The python script parses the command definition +string, extracts variable names and types, and generates a C wrapper function +that parses the variables and passes them on. This means that in the CLI +function body, you will receive additional parameters with appropriate types. + +This is best explained by an example. Invoking ``DEFPY`` like this: + +.. code-block:: c + + DEFPY(func, func_cmd, "[no] foo bar A.B.C.D (0-99)$num", "...help...") + +defines the handler function like this: + +.. code-block:: c + + func(self, vty, argc, argv, /* standard CLI arguments */ + const char *no, /* unparsed "no" */ + struct in_addr bar, /* parsed IP address */ + const char *bar_str, /* unparsed IP address */ + long num, /* parsed num */ + const char *num_str) /* unparsed num */ + +Note that as documented in the previous section, ``bar`` is automatically +applied as variable name for ``A.B.C.D``. The Python script then detects this as +an IP address argument and generates code to parse it into a ``struct in_addr``, +passing it in ``bar``. The raw value is passed in ``bar_str``. The range/number +argument works in the same way with the explicitly given variable name. + +Type rules +"""""""""" + ++----------------------------+--------------------------------+--------------------------+ +| Token(s) | Type | Value if omitted by user | ++============================+================================+==========================+ +| ``A.B.C.D`` | ``struct in_addr`` | ``0.0.0.0`` | ++----------------------------+--------------------------------+--------------------------+ +| ``X:X::X:X`` | ``struct in6_addr`` | ``::`` | ++----------------------------+--------------------------------+--------------------------+ +| ``A.B.C.D + X:X::X:X`` | ``const union sockunion *`` | ``NULL`` | ++----------------------------+--------------------------------+--------------------------+ +| ``A.B.C.D/M`` | ``const struct prefix_ipv4 *`` | ``all-zeroes struct`` | ++----------------------------+--------------------------------+--------------------------+ +| ``X:X::X:X/M`` | ``const struct prefix_ipv6 *`` | ``all-zeroes struct`` | ++----------------------------+--------------------------------+--------------------------+ +| ``A.B.C.D/M + X:X::X:X/M`` | ``const struct prefix *`` | ``all-zeroes struct`` | ++----------------------------+--------------------------------+--------------------------+ +| ``(0-9)`` | ``long`` | ``0`` | ++----------------------------+--------------------------------+--------------------------+ +| ``VARIABLE`` | ``const char *`` | ``NULL`` | ++----------------------------+--------------------------------+--------------------------+ +| ``word`` | ``const char *`` | ``NULL`` | ++----------------------------+--------------------------------+--------------------------+ +| *all other* | ``const char *`` | ``NULL`` | ++----------------------------+--------------------------------+--------------------------+ + +Note the following details: + +- Not all parameters are pointers, some are passed as values. +- When the type is not ``const char *``, there will be an extra ``_str`` + argument with type ``const char *``. +- You can give a variable name not only to ``VARIABLE`` tokens but also to + ``word`` tokens (e.g. constant words). This is useful if some parts of a + command are optional. The type will be ``const char *``. +- ``[no]`` will be passed as ``const char *no``. +- Most pointers will be ``NULL`` when the argument is optional and the + user did not supply it. As noted in the table above, some prefix + struct type arguments are passed as pointers to all-zeroes structs, + not as ``NULL`` pointers. +- If a parameter is not a pointer, but is optional and the user didn't use it, + the default value will be passed. Check the ``_str`` argument if you need to + determine whether the parameter was omitted. +- If the definition contains multiple parameters with the same variable name, + they will be collapsed into a single function parameter. The python code will + detect if the types are compatible (i.e. IPv4 + IPv6 variants) and choose a + corresponding C type. +- The standard DEFUN parameters (``self, vty, argc, argv``) are still present + and can be used. A DEFUN can simply be **edited into a DEFPY without further + changes and it will still work**; this allows easy forward migration. +- A file may contain both ``DEFUN`` and ``DEFPY`` statements. + +Getting a parameter dump +"""""""""""""""""""""""" +The clidef.py script can be called to get a list of DEFUNs/DEFPYs with the +parameter name/type list: + +:: + + lib/clippy python/clidef.py --all-defun --show lib/plist.c > /dev/null + +The generated code is printed to stdout, the info dump to stderr. The +``--all-defun`` argument will make it process DEFUN blocks as well as DEFPYs, +which is useful prior to converting some DEFUNs. **The dump does not list the +``_str`` arguments** to keep the output shorter. + +Note that the ``clidef.py`` script cannot be run with python directly, it needs +to be run with *clippy* since the latter makes the CLI parser available. + +Include & Makefile requirements +""""""""""""""""""""""""""""""" +A source file that uses DEFPY needs to include the ``*_clippy.c`` file **before +all DEFPY statements**: + +.. code-block:: c + + /* GPL header */ + #include ... + ... + #include "daemon/filename_clippy.c" + + DEFPY(...) + DEFPY(...) + + install_element(...) + +This dependency needs to be marked in ``Makefile.am`` or ``subdir.am``: (there +is no ordering requirement) + +.. code-block:: make + + # ... + + # if linked into a LTLIBRARY (.la/.so): + filename.lo: filename_clippy.c + + # if linked into an executable or static library (.a): + filename.o: filename_clippy.c + +Handlers +^^^^^^^^ +The block that follows a CLI definition is executed when a user enters input +that matches the definition. Its function signature looks like this: + +.. code-block:: c + + int (*func) (const struct cmd_element *, struct vty *, int, struct cmd_token *[]); + +The first argument is the command definition struct. The last argument is an +ordered array of tokens that correspond to the path taken through the graph, and +the argument just prior to that is the length of the array. + +The arrangement of the token array has changed from Quagga's CLI implementation. +In the old system, missing arguments were padded with ``NULL`` so that the same +parts of a command would show up at the same indices regardless of what was +entered. The new system does not perform such padding and therefore it is +generally *incorrect* to assume consistent indices in this array. As a simple +example: + +Command definition: + +:: + + command [foo] <bar|baz> + +User enters: + +:: + + command foo bar + +Array: + +:: + + [0] -> command + [1] -> foo + [2] -> bar + +User enters: + +:: + + command baz + +Array: + +:: + + [0] -> command + [1] -> baz + + +.. _cli-data-structures: + +Data Structures +--------------- +On startup, the CLI parser sequentially parses each command string definition +and constructs a directed graph with each token forming a node. This graph is +the basis of the entire CLI system. It is used to match user input in order to +generate command completions and match commands to functions. + +There is one graph per CLI node (not the same as a graph node in the CLI graph). +The CLI node struct keeps a reference to its graph (see :file:`lib/command.h`). + +While most of the graph maintains the form of a tree, special constructs +outlined in the Rules section introduce some quirks. ``<>``, ``[]`` and ``{}`` +form self-contained 'subgraphs'. Each subgraph is a tree except that all of the +'leaves' actually share a child node. This helps with minimizing graph size and +debugging. + +As a working example, here is the graph of the following command: :: + + show [ip] bgp neighbors [<A.B.C.D|X:X::X:X|WORD>] [json] + +.. figure:: ../figures/cligraph.png + :align: center + + Graph of example CLI command + + +``FORK`` and ``JOIN`` nodes are plumbing nodes that don't correspond to user +input. They're necessary in order to deduplicate these constructs where +applicable. + +Options follow the same form, except that there is an edge from the ``FORK`` +node to the ``JOIN`` node. Since all of the subgraphs in the example command are +optional, all of them have this edge. + +Keywords follow the same form, except that there is an edge from ``JOIN`` to +``FORK``. Because of this the CLI graph cannot be called acyclic. There is +special logic in the input matching code that keeps a stack of paths already +taken through the node in order to disallow following the same path more than +once. + +Variadics are a bit special; they have an edge back to themselves, which allows +repeating the same input indefinitely. + +The leaves of the graph are nodes that have no out edges. These nodes are +special; their data section does not contain a token, as most nodes do, or +``NULL``, as in ``FORK``/``JOIN`` nodes, but instead has a pointer to a +``cmd_element``. All paths through the graph that terminate on a leaf are +guaranteed to be defined by that command. When a user enters a complete command, +the command matcher tokenizes the input and executes a DFS on the CLI graph. If +it is simultaneously able to exhaust all input (one input token per graph node), +and then find exactly one leaf connected to the last node it reaches, then the +input has matched the corresponding command and the command is executed. If it +finds more than one node, then the command is ambiguous (more on this in +deduplication). If it cannot exhaust all input, the command is unknown. If it +exhausts all input but does not find an edge node, the command is incomplete. + +The parser uses an incremental strategy to build the CLI graph for a node. Each +command is parsed into its own graph, and then this graph is merged into the +overall graph. During this merge step, the parser makes a best-effort attempt to +remove duplicate nodes. If it finds a node in the overall graph that is equal to +a node in the corresponding position in the command graph, it will intelligently +merge the properties from the node in the command graph into the +already-existing node. Subgraphs are also checked for isomorphism and merged +where possible. The definition of whether two nodes are 'equal' is based on the +equality of some set of token properties; read the parser source for the most +up-to-date definition of equality. + +When the parser is unable to deduplicate some complicated constructs, this can +result in two identical paths through separate parts of the graph. If this +occurs and the user enters input that matches these paths, they will receive an +'ambiguous command' error and will be unable to execute the command. Most of the +time the parser can detect and warn about duplicate commands, but it will not +always be able to do this. Hence care should be taken before defining a new +command to ensure it is not defined elsewhere. + +struct cmd\_token +^^^^^^^^^^^^^^^^^ + +.. code-block:: c + + /* Command token struct. */ + struct cmd_token + { + enum cmd_token_type type; // token type + uint8_t attr; // token attributes + bool allowrepeat; // matcher can match token repetitively? + + char *text; // token text + char *desc; // token description + long long min, max; // for ranges + char *arg; // user input that matches this token + char *varname; // variable name + }; + +This struct is used in the CLI graph to match input against. It is also used to +pass user input to command handler functions, as it is frequently useful for +handlers to have access to that information. When a command is matched, the +sequence of ``cmd_tokens`` that form the matching path are duplicated and placed +in order into ``*argv[]``. Before this happens the ``->arg`` field is set to +point at the snippet of user input that matched it. + +For most nontrivial commands the handler function will need to determine which +of the possible matching inputs was entered. Previously this was done by +looking at the first few characters of input. This is now considered an +anti-pattern and should be avoided. Instead, use the ``->type`` or ``->text`` +fields for this logic. The ``->type`` field can be used when the possible +inputs differ in type. When the possible types are the same, use the ``->text`` +field. This field has the full text of the corresponding token in the +definition string and using it makes for much more readable code. An example is +helpful. + +Command definition: + +:: + + command <(1-10)|foo|BAR> + +In this example, the user may enter any one of: + +* an integer between 1 and 10 +* "foo" +* anything at all + +If the user enters "command f", then: + +:: + + argv[1]->type == WORD_TKN + argv[1]->arg == "f" + argv[1]->text == "foo" + +Range tokens have some special treatment; a token with ``->type == RANGE_TKN`` +will have the ``->min`` and ``->max`` fields set to the bounding values of the +range. + +struct cmd\_element +^^^^^^^^^^^^^^^^^^^ + +.. code-block:: c + + struct cmd_node { + /* Node index. */ + enum node_type node; + + /* Prompt character at vty interface. */ + const char *prompt; + + /* Is this node's configuration goes to vtysh ? */ + int vtysh; + + /* Node's configuration write function */ + int (*func)(struct vty *); + + /* Node's command graph */ + struct graph *cmdgraph; + + /* Vector of this node's command list. */ + vector cmd_vector; + + /* Hashed index of command node list, for de-dupping primarily */ + struct hash *cmd_hash; + }; + +This struct corresponds to a CLI mode. The last three fields are most relevant +here. + +cmdgraph + This is a pointer to the command graph that was described in the first part + of this section. It is the datastructure used for matching user input to + commands. + +cmd_vector + This is a list of all the ``struct cmd_element`` defined in the mode. + +cmd_hash + This is a hash table of all the ``struct cmd_element`` defined in the mode. + When ``install_element`` is called, it checks that the element it is given is + not already present in the hash table as a safeguard against duplicate calls + resulting in a command being defined twice, which renders the command + ambiguous. + +All ``struct cmd_node`` are themselves held in a static vector defined in +:file:`lib/command.c` that defines the global CLI space. + +Command Abbreviation & Matching Priority +---------------------------------------- +It is possible for users to elide parts of tokens when the CLI matcher does not +need them to make an unambiguous match. This is best explained by example. + +Command definitions: + +:: + + command dog cow + command dog crow + +User input: + +:: + + c d c -> ambiguous command + c d co -> match "command dog cow" + + +The parser will look ahead and attempt to disambiguate the input based on tokens +later on in the input string. + +Command definitions: + +:: + + show ip bgp A.B.C.D + show ipv6 bgp X:X::X:X + +User enters: + +:: + + s i b 4.3.2.1 -> match "show ip bgp A.B.C.D" + s i b ::e0 -> match "show ipv6 bgp X:X::X:X" + +Reading left to right, both of these commands would be ambiguous since 'i' does +not explicitly select either 'ip' or 'ipv6'. However, since the user later +provides a token that matches only one of the commands (an IPv4 or IPv6 address) +the parser is able to look ahead and select the appropriate command. This has +some implications for parsing the ``*argv[]`` that is passed to the command +handler. + +Now consider a command definition such as: + +:: + + command <foo|VAR> + +'foo' only matches the string 'foo', but 'VAR' matches any input, including +'foo'. Who wins? In situations like this the matcher will always choose the +'better' match, so 'foo' will win. + +Consider also: + +:: + + show <ip|ipv6> foo + +User input: + +:: + + show ip foo + +``ip`` partially matches ``ipv6`` but exactly matches ``ip``, so ``ip`` will +win. + +Adding a CLI Node +----------------- + +To add a new CLI node, you should: + +#. define a new numerical node constant +#. define a node structure in the relevant daemon +#. call ``install_node()`` in the relevant daemon +#. define and install the new node in vtysh +#. define corresponding node entry commands in daemon and vtysh +#. add a new entry to the ``ctx_keywords`` dictionary in ``tools/frr-reload.py`` + +Defining the numerical node constant +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Add your new node value to the enum before ``NODE_TYPE_MAX`` in +``lib/command.h``: + +.. code-block:: c + + enum node_type { + AUTH_NODE, // Authentication mode of vty interface. + VIEW_NODE, // View node. Default mode of vty interface. + [...] + MY_NEW_NODE, + NODE_TYPE_MAX, // maximum + }; + +Defining a node structure +^^^^^^^^^^^^^^^^^^^^^^^^^ +In your daemon-specific code where you define your new commands that +attach to the new node, add a node definition: + +.. code-block:: c + + static struct cmd_node my_new_node = { + .name = "my new node name", + .node = MY_NEW_NODE, // enum node_type lib/command.h + .parent_node = CONFIG_NODE, + .prompt = "%s(my-new-node-prompt)# ", + .config_write = my_new_node_config_write, + }; + +You will need to define ``my_new_node_config_write(struct vty \*vty)`` +(or omit this field if you have no relevant configuration to save). + +Calling ``install_node()`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ +In the daemon's initialization function, before installing your new commands +with ``install_element()``, add a call ``install_node(&my_new_node)``. + +Defining and installing the new node in vtysh +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The build tools automatically collect command definitions for vtysh. +However, new nodes must be coded in vtysh specifically. + +In ``vtysh/vtysh.c``, define a stripped-down node structure and +call ``install_node()``: + +.. code-block:: c + + static struct cmd_node my_new_node = { + .name = "my new node name", + .node = MY_NEW_NODE, /* enum node_type lib/command.h */ + .parent_node = CONFIG_NODE, + .prompt = "%s(my-new-node-prompt)# ", + }; + [...] + void vtysh_init_vty(void) + { + [...] + install_node(&my_new_node) + [...] + } + +Defining corresponding node entry commands in daemon and vtysh +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The command that descends into the new node is typically programmed +with ``VTY_PUSH_CONTEXT`` or equivalent in the daemon's CLI handler function. +(If the CLI has been updated to use the new northbound architecture, +``VTY_PUSH_XPATH`` is used instead.) + +In vtysh, you must implement a corresponding node change so that vtysh +tracks the daemon's movement through the node tree. + +Although the build tools typically scan daemon code for CLI definitions +to replicate their parsing in vtysh, the node-descent function in the +daemon must be blocked from this replication so that a hand-coded +skeleton can be written in ``vtysh.c``. + +Accordingly, use one of the ``*_NOSH`` macros such as ``DEFUN_NOSH``, +``DEFPY_NOSH``, or ``DEFUN_YANG_NOSH`` for the daemon's node-descent +CLI definition, and use ``DEFUNSH`` in ``vtysh.c`` for the vtysh equivalent. + +.. seealso:: :ref:`vtysh-special-defuns` + +Examples: + +``zebra_whatever.c`` + +.. code-block:: c + + DEFPY_NOSH(my_new_node, + my_new_node_cmd, + "my-new-node foo", + "New Thing\n" + "A foo\n") + { + [...] + VTY_PUSH_CONTEXT(MY_NEW_NODE, bar); + [...] + } + + +``ripd_whatever.c`` + +.. code-block:: c + + DEFPY_YANG_NOSH(my_new_node, + my_new_node_cmd, + "my-new-node foo", + "New Thing\n" + "A foo\n") + { + [...] + VTY_PUSH_XPATH(MY_NEW_NODE, xbar); + [...] + } + + +``vtysh.c`` + +.. code-block:: c + + DEFUNSH(VTYSH_ZEBRA, my_new_node, + my_new_node_cmd, + "my-new-node foo", + "New Thing\n" + "A foo\n") + { + vty->node = MY_NEW_NODE; + return CMD_SUCCESS; + } + [...] + install_element(CONFIG_NODE, &my_new_node_cmd); + + +Adding a new entry to the ``ctx_keywords`` dictionary +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In file ``tools/frr-reload.py``, the ``ctx_keywords`` dictionary +describes the various node relationships. +Add a new node entry at the appropriate level in this dictionary. + +.. code-block:: python + + ctx_keywords = { + [...] + "key chain ": { + "key ": {} + }, + [...] + "my-new-node": {}, + [...] + } + + + +Inspection & Debugging +---------------------- + +Permutations +^^^^^^^^^^^^ +It is sometimes useful to check all the possible combinations of input that +would match an arbitrary definition string. There is a tool in +:file:`tools/permutations` that reads CLI definition strings on ``stdin`` and +prints out all matching input permutations. It also dumps a text representation +of the graph, which is more useful for debugging than anything else. It looks +like this: + +.. code-block:: shell + + $ ./permutations "show [ip] bgp [<view|vrf> WORD]" + + show ip bgp view WORD + show ip bgp vrf WORD + show ip bgp + show bgp view WORD + show bgp vrf WORD + show bgp + +This functionality is also built into VTY/VTYSH; :clicmd:`list permutations` +will list all possible matching input permutations in the current CLI node. + +Graph Inspection +^^^^^^^^^^^^^^^^ +When in the Telnet or VTYSH console, :clicmd:`show cli graph` will dump the +entire command space of the current mode in the DOT graph language. This can be +fed into one of the various GraphViz layout engines, such as ``dot``, +``neato``, etc. + +For example, to generate an image of the entire command space for the top-level +mode (``ENABLE_NODE``): + +.. code-block:: shell + + sudo vtysh -c 'show cli graph' | dot -Tjpg -Grankdir=LR > graph.jpg + +To do the same for the BGP mode: + +.. code-block:: shell + + sudo vtysh -c 'conf t' -c 'router bgp' -c 'show cli graph' | dot -Tjpg -Grankdir=LR > bgpgraph.jpg + +This information is very helpful when debugging command resolution, tracking +down duplicate / ambiguous commands, and debugging patches to the CLI graph +builder. diff --git a/doc/developer/conf.py b/doc/developer/conf.py new file mode 100644 index 0000000..495c604 --- /dev/null +++ b/doc/developer/conf.py @@ -0,0 +1,406 @@ +# -*- coding: utf-8 -*- +# +# FRR documentation build configuration file, created by +# sphinx-quickstart on Tue Jan 31 16:00:52 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys +import os +import re +import pygments +from sphinx.highlighting import lexers +from sphinx.util import logging +logger = logging.getLogger(__name__) + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +needs_sphinx = "1.0" + +# prolog for various variable substitutions +rst_prolog = "" + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = ["sphinx.ext.todo", "sphinx.ext.graphviz"] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ["_templates"] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst'] +source_suffix = ".rst" + +# The encoding of source files. +# source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = "index" + +# General information about the project. +project = u"FRR" +copyright = u"2017, FRR" +author = u"FRR authors" + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. + +# The short X.Y version. +version = u"?.?" +# The full version, including alpha/beta/rc tags. +release = u"?.?-?" + + +# ----------------------------------------------------------------------------- +# Extract values from codebase for substitution into docs. +# ----------------------------------------------------------------------------- + +# Various installation prefixes. Values are extracted from config.status. +# Reasonable defaults are set in case that file does not exist. +replace_vars = { + "AUTHORS": author, + "COPYRIGHT_YEAR": "1999-2005", + "COPYRIGHT_STR": "Copyright (c) 1999-2005", + "PACKAGE_NAME": project.lower(), + "PACKAGE_TARNAME": project.lower(), + "PACKAGE_STRING": project.lower() + " latest", + "PACKAGE_URL": "https://frrouting.org/", + "PACKAGE_VERSION": "latest", + "INSTALL_PREFIX_ETC": "/etc/frr", + "INSTALL_PREFIX_SBIN": "/usr/lib/frr", + "INSTALL_PREFIX_STATE": "/var/run/frr", + "INSTALL_PREFIX_MODULES": "/usr/lib/frr/modules", + "INSTALL_USER": "frr", + "INSTALL_GROUP": "frr", + "INSTALL_VTY_GROUP": "frrvty", + "GROUP": "frr", + "USER": "frr", +} + +# extract version information, installation location, other stuff we need to +# use when building final documents +val = re.compile('^S\["([^"]+)"\]="(.*)"$') +try: + with open("../../config.status", "r") as cfgstatus: + for ln in cfgstatus.readlines(): + m = val.match(ln) + if not m or m.group(1) not in replace_vars.keys(): + continue + replace_vars[m.group(1)] = m.group(2) +except IOError: + # if config.status doesn't exist, just ignore it + pass + +# manually fill out some of these we can't get from config.status +replace_vars["COPYRIGHT_STR"] = "Copyright (c)" +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["COPYRIGHT_YEAR"]) +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["AUTHORS"]) +release = replace_vars["PACKAGE_VERSION"] +version = release.split("-")[0] + +# add substitutions to prolog +for key, value in replace_vars.items(): + rst_prolog += ".. |{0}| replace:: {1}\n".format(key, value) + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +# today = '' +# Else, today_fmt is used as the format for a strftime call. +# today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = [ + "_build", + "building-libunwind-note.rst", + "building-libyang.rst", + "topotests-snippets.rst", + "topotests-markers.rst", + "include-compile.rst", +] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +# default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +# add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +# add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +# show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = "sphinx" + +# A list of ignored prefixes for module index sorting. +# modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +# keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = "default" + +try: + import sphinx_rtd_theme + + html_theme = "sphinx_rtd_theme" +except ImportError: + pass + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# html_theme_options = { +# 'sidebarbgcolor': '#374249' +# } + +# Add any paths that contain custom themes here, relative to this directory. +# html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# "<project> v<release> documentation". +# html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +# html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +html_logo = "../figures/frr-icon.svg" + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +html_favicon = "../figures/frr-logo-icon.png" + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ["_static"] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +# html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +# html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +# html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +# html_additional_pages = {} + +# If false, no module index is generated. +# html_domain_indices = True + +# If false, no index is generated. +# html_use_index = True + +# If true, the index is split into individual pages for each letter. +# html_split_index = False + +# If true, links to the reST sources are added to the pages. +# html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +# html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +# html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a <link> tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +# html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +# html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr' +# html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +# html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +# html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = "FRRdoc" + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + #'papersize': 'letterpaper', + # The font size ('10pt', '11pt' or '12pt'). + #'pointsize': '10pt', + # Additional stuff for the LaTeX preamble. + #'preamble': '', + # Latex figure (float) alignment + #'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, "FRR.tex", u"FRR Developer's Manual", u"FRR", "manual"), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +latex_logo = "../figures/frr-logo-medium.png" + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +# latex_use_parts = False + +# If true, show page references after internal links. +# latex_show_pagerefs = False + +# If true, show URL addresses after external links. +# latex_show_urls = False + +# Documents to append as an appendix to all manuals. +# latex_appendices = [] + +# If false, no module index is generated. +# latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [(master_doc, "frr", u"FRR Developer's Manual", [author], 1)] + +# If true, show URL addresses after external links. +# man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ( + master_doc, + "frr", + u"FRR Developer's Manual", + author, + "FRR", + "One line description of project.", + "Miscellaneous", + ), +] + +# Documents to append as an appendix to all manuals. +# texinfo_appendices = [] + +# If false, no module index is generated. +# texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +# texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +# texinfo_no_detailmenu = False + +# contents of ../extra/frrlexer.py. +# This is read here to support VPATH build. Since this section is execfile()'d +# with the file location, we can safely use a relative path here to save the +# contents of the lexer file for later use even if our relative path changes +# due to VPATH. +with open("../extra/frrlexer.py", "rb") as lex: + frrlexerpy = lex.read() + +frrfmt_re = re.compile(r'^\s*%(?P<spec>[^\s]+)\s+\((?P<types>.*)\)\s*$') + +def parse_frrfmt(env, text, node): + from sphinx import addnodes + + m = frrfmt_re.match(text) + if not m: + logger.warning('could not parse frrfmt:: %r' % (text), location=node) + node += addnodes.desc_name(text, text) + return text + + spec, types = m.group('spec'), m.group('types') + + node += addnodes.desc_sig_operator('%', '%') + node += addnodes.desc_name(spec + ' ', spec + ' ') + plist = addnodes.desc_parameterlist() + for typ in types.split(','): + typ = typ.strip() + plist += addnodes.desc_parameter(typ, typ) + node += plist + return '%' + spec + +# custom extensions here +def setup(app): + # object type for FRR CLI commands, can be extended to document parent CLI + # node later on + app.add_object_type("clicmd", "clicmd") + + # printfrr extensions + app.add_object_type("frrfmt", "frrfmt", parse_node=parse_frrfmt) + + if "add_css_file" in dir(app): + app.add_css_file("overrides.css") + else: + app.add_stylesheet("overrides.css") + + # load Pygments lexer for FRR config syntax + # + # NB: in Pygments 2.2+ this can be done with `load_lexer_from_file`, but we + # do it manually since not all of our supported build platforms have 2.2 + # yet. + # + # frrlexer = pygments.lexers.load_lexer_from_file('../extra/frrlexer.py', lexername="FRRLexer") + custom_namespace = {} + exec(frrlexerpy, custom_namespace) + lexers["frr"] = custom_namespace["FRRLexer"]() diff --git a/doc/developer/cross-compiling.rst b/doc/developer/cross-compiling.rst new file mode 100644 index 0000000..3bf78f7 --- /dev/null +++ b/doc/developer/cross-compiling.rst @@ -0,0 +1,326 @@ +Cross-Compiling +=============== + +FRR is capable of being cross-compiled to a number of different architectures. +With an adequate toolchain this process is fairly straightforward, though one +must exercise caution to validate this toolchain's correctness before attempting +to compile FRR or its dependencies; small oversights in the construction of the +build tools may lead to problems which quickly become difficult to diagnose. + +Toolchain Preliminary +--------------------- + +The first step to cross-compiling any program is to identify the system which +the program (FRR) will run on. From here on this will be called the "host" +machine, following autotools' convention, while the machine building FRR will be +called the "build" machine. The toolchain will of course be installed onto the +build machine and be leveraged to build FRR for the host machine to run. + +.. note:: + + The build machine used while writing this guide was ``x86_64-pc-linux-gnu`` + and the target machine was ``arm-linux-gnueabihf`` (a Raspberry Pi 3B+). + Replace this with your targeted tuple below if you plan on running the + commands from this guide: + + .. code-block:: shell + + export HOST_ARCH="arm-linux-gnueabihf" + + For your given target, the build system's OS may have some support for + building cross compilers natively, or may even offer binary toolchains built + upstream for the target architecture. Check your package manager or OS + documentation before committing to building a toolchain from scratch. + +This guide will not detail *how* to build a cross-compiling toolchain but +will instead assume one already exists and is installed on the build system. +The methods for building the toolchain itself may differ between operating +systems so consult the OS documentation for any particulars regarding +cross-compilers. The OSDev wiki has a `pleasant tutorial`_ on cross-compiling in +the context of operating system development which bootstraps from only the +native GCC and binutils on the build machine. This may be useful if the build +machine's OS does not offer existing tools to build a cross-compiler targeting +the host. + +.. _pleasant tutorial: https://wiki.osdev.org/GCC_Cross-Compiler + +This guide will also not demonstrate how to build all of FRR's dependencies for the +target architecture. Instead, general instructions for using a cross-compiling +toolchain to compile packages using CMake, Autotools, and Makefiles are +provided; these three cases apply to almost all FRR dependencies. + +.. _glibc mismatch: + +.. warning:: + + Ensure the versions and implementations of the C standard library (glibc or + what have you) match on the host and the build toolchain. ``ldd --version`` + will help you here. Upgrade one or the other if the they do not match. + +Testing the Toolchain +--------------------- + +Before any cross-compilation begins it would be prudent to test the new +toolchain by writing, compiling and linking a simple program. + +.. code-block:: shell + + # A small program + cat > nothing.c <<EOF + int main() { return 0; } + EOF + + # Build and link with the cross-compiler + ${HOST_ARCH}-gcc -o nothing nothing.c + + # Inspect the resulting binary, results may vary + file ./nothing + + # nothing: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), + # dynamically linked, interpreter /lib/ld-linux-armhf.so.3, + # for GNU/Linux 3.2.0, not stripped + +If this produced no errors then the installed toolchain is probably ready to +start compiling the build dependencies and eventually FRR itself. There still +may be lurking issues but fundamentally the toolchain can produce binaries and +that's good enough to start working with it. + +.. warning:: + + If any errors occurred during the previous functional test please look back + and address them before moving on; this indicates your cross-compiling + toolchain is *not* in a position to build FRR or its dependencies. Even if + everything was fine, keep in mind that many errors from here on *may still + be related* to your toolchain (e.g. libstdc++.so or other components) and this + small test is not a guarantee of complete toolchain coherence. + +Cross-compiling Dependencies +---------------------------- + +When compiling FRR it is necessary to compile some of its dependencies alongside +it on the build machine. This is so symbols from the shared libraries (which +will be loaded at run-time on the host machine) can be linked to the FRR +binaries at compile time; additionally, headers for these libraries are needed +during the compile stage for a successful build. + +Sysroot Overview +^^^^^^^^^^^^^^^^ + +All build dependencies should be installed into a "root" directory on the build +computer, hereafter called the "sysroot". This directory will be prefixed to +paths while searching for requisite libraries and headers during the build +process. Often this may be set via a ``--prefix`` flag when building the +dependent packages, meaning a ``make install`` will copy compiled libraries into +(e.g.) ``/usr/${HOST_ARCH}/usr``. + +If the toolchain was built on the build machine then there is likely already a +sysroot where those tools and standard libraries were installed; it may be +helpful to use that directory as the sysroot for this build as well. + +Basic Workflow +^^^^^^^^^^^^^^ + +Before compiling or building any dependencies, make note of which daemons are +being targeted and which libraries will be needed. Not all dependencies are +necessary if only building with a subset of the daemons. + +The following workflow will compile and install any libraries which can be built +with Autotools. The resultant library will be installed into the sysroot +``/usr/${HOST_ARCH}``. + +.. code-block:: shell + + ./configure \ + CC=${HOST_ARCH}-gcc \ + CXX=${HOST_ARCH}-g++ \ + --build=${HOST_ARCH} \ + --prefix=/usr/${HOST_ARCH} + make + make install + +Some libraries like ``json-c`` and ``libyang`` are packaged with CMake and can +be built and installed generally like: + +.. code-block:: shell + + mkdir build + cd build + CC=${HOST_ARCH}-gcc \ + CXX=${HOST_ARCH}-g++ \ + cmake \ + -DCMAKE_INSTALL_PREFIX=/usr/${HOST_ARCH} \ + .. + make + make install + +For programs with only a Makefile (e.g. ``libcap``) the process may look still a +little different: + +.. code-block:: shell + + CC=${HOST_ARCH}-gcc make + make install DESTDIR=/usr/${HOST_ARCH} + +These three workflows should handle the bulk of building and installing the +build-time dependencies for FRR. Verify that the installed files are being +placed correctly into the sysroot and were actually built using the +cross-compile toolchain, not by the native toolchain by accident. + +Dependency Notes +^^^^^^^^^^^^^^^^ + +There are a lot of things that can go wrong during a cross-compilation. Some of +the more common errors and a few special considerations are collected below for +reference. + +libyang +""""""" + +``-DENABLE_LYD_PRIV=ON`` should be provided during the CMake step. + +Ensure also that the version of ``libyang`` being installed corresponds to the +version required by the targeted FRR version. + +gRPC +"""" + +This piece is requisite only if the ``--enable-grpc`` flag will be passed +later on to FRR. One may get burned when compiling gRPC if the ``protoc`` +version on the build machine differs from the version of ``protoc`` being linked +to during a gRPC build. The error messages from this defect look like: + +.. code-block:: shell + + gens/src/proto/grpc/channelz/channelz.pb.h: In member function ‘void grpc::channelz::v1::ServerRef::set_name(const char*, size_t)’: + gens/src/proto/grpc/channelz/channelz.pb.h:9127:64: error: ‘EmptyDefault’ is not a member of ‘google::protobuf::internal::ArenaStringPtr’ + 9127 | name_.Set(::PROTOBUF_NAMESPACE_ID::internal::ArenaStringPtr::EmptyDefault{}, ::std::string( + +This happens because protocol buffer code generation uses ``protoc`` to create +classes with different getters and setters corresponding to the protobuf data +defined by the source tree's ``.proto`` files. Clearly the cross-compiled +``protoc`` cannot be used for this code generation because that binary is built +for a different CPU. + +The solution is to install matching versions of native and cross-compiled +protocol buffers; this way the native binary will generate code and the +cross-compiled library will be linked to by gRPC and these versions will not +disagree. + +---- + +The ``-latomic`` linker flag may also be necessary here if using ``libstdc++`` +since GCC's C++11 implementation makes library calls in certain cases for +``<atomic>`` so ``-latomic`` cannot be assumed. + +Cross-compiling FRR Itself +-------------------------- + +With all the necessary libraries cross-compiled and installed into the sysroot, +the last thing to actually build is FRR itself: + +.. code-block:: shell + + # Clone and bootstrap the build + git clone 'https://github.com/FRRouting/frr.git' + # (e.g.) git checkout stable/7.5 + ./bootstrap.sh + + # Build clippy using the native toolchain + mkdir build-clippy + cd build-clippy + ../configure --enable-clippy-only + make clippy-only + cd .. + + # Next, configure FRR and use the clippy we just built + ./configure \ + CC=${HOST_ARCH}-gcc \ + CXX=${HOST_ARCH}-g++ \ + --host=${HOST_ARCH} \ + --with-sysroot=/usr/${HOST_ARCH} \ + --with-clippy=./build-clippy/lib/clippy \ + --sysconfdir=/etc/frr \ + --sbindir="\${prefix}/lib/frr" \ + --localstatedir=/var/run/frr \ + --prefix=/usr \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --disable-doc \ + --enable-grpc + + # Send it + make + +Installation to Host Machine +---------------------------- + +If no errors were observed during the previous steps it is safe to ``make +install`` FRR into its own directory. + +.. code-block:: shell + + # Install FRR its own "sysroot" + make install DESTDIR=/some/path/to/sysroot + +After running the above command, FRR binaries, modules and example configuration +files will be installed into some path on the build machine. The directory +will have folders like ``/usr`` and ``/etc``; this "root" should now be copied +to the host and installed on top of the root directory there. + +.. code-block:: shell + + # Tar this sysroot (preserving permissions) + tar -C /some/path/to/sysroot -cpvf frr-${HOST_ARCH}.tar . + + # Transfer tar file to host machine + scp frr-${HOST_ARCH}.tar me@host-machine: + + # Overlay the tarred sysroot on top of the host machine's root + ssh me@host-machine <<-EOF + # May need to elevate permissions here + tar -C / -xpvf frr-${HOST_ARCH}.tar.gz . + EOF + +Now FRR should be installed just as if ``make install`` had been run on the host +machine. Create configuration files and assign permissions as needed. Lastly, +ensure the correct users and groups exist for FRR on the host machine. + +Troubleshooting +--------------- + +Even when every precaution has been taken some things may still go wrong! This +section details some common runtime problems. + +Mismatched Libraries +^^^^^^^^^^^^^^^^^^^^ + +If you see something like this after installing on the host: + +.. code-block:: console + + /usr/lib/frr/zebra: error while loading shared libraries: libyang.so.1: cannot open shared object file: No such file or directory + +... at least one of FRR's dependencies which was linked to the binary earlier is +not available on the host OS. Even if it has been installed the host +repository's version may lag what is needed for more recent FRR builds (this is +likely to happen with YANG at the moment). + +If the matching library is not available from the host OS package manager it may +be possible to compile them using the same toolchain used to compile FRR. The +library may have already been built earlier when compiling FRR on the build +machine, in which case it may be as simple as following the same workflow laid +out during the `Installation to Host Machine`_. + +Mismatched Glibc Versions +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The version and implementation of the C standard library must match on both the +host and build toolchain. The error corresponding to this misconfiguration will +look like: + +.. code-block:: console + + /usr/lib/frr/zebra: /lib/${HOST_ARCH}/libc.so.6: version `GLIBC_2.32' not found (required by /usr/lib/libfrr.so.0) + +See the earlier warning about preventing a `glibc mismatch`_. diff --git a/doc/developer/cspf.rst b/doc/developer/cspf.rst new file mode 100644 index 0000000..7a5a55e --- /dev/null +++ b/doc/developer/cspf.rst @@ -0,0 +1,197 @@ +Path Computation Algorithms +=========================== + +Introduction +------------ + +Both RSVP-TE and Segment Routing Flex Algo need to compute end to end path +with other constraints as the standard IGP metric. Based on Shortest Path First +(SPF) algorithms, a new class of Constrained SPF (CSPF) is provided by the FRR +library. + +Supported constraints are as follow: +- Standard IGP metric (here, CSPF provides the same result as a normal SPF) +- Traffic Engineering (TE) IGP metric +- Delay from the IGP Extended Metrics +- Bandwidth for a given Class of Service (CoS) for bandwidth reservation + +Algorithm +--------- + +The CSPF algorithm is based on a Priority Queue which store the on-going +possible path sorted by their respective weights. This weight corresponds +to the cost of the cuurent path from the source up to the current node. + +The algorithm is as followed: + +.. code-block:: c + + cost = MAX_COST; + Priority_Queue.empty(); + Visited_Node.empty(); + Processed_Path.empty(); + src = new_path(source_address); + src.cost = 0; + dst = new_destinatio(destination_address); + dst.cost = MAX_COST; + Processed_Path.add(src); + Processed_Path.add(dst); + while (Priority_Queue.count != 0) { + current_path = Priority_Queue.pop(); + current_node = next_path.destination; + Visited_Node.add(current_node); + for (current_node.edges: edge) { + if (prune_edge(current_path, edge) + continue; + if (relax(current_path) && cost > current_path.cost) { + optim_path = current_path; + cost = current_path.cost; + } + } + } + + prune_edge(path, edge) { + // check that path + edge meet constraints e.g. + if (current_path.cost + edge.cost > constrained_cost) + return false; + else + return true; + } + + relax_edge(current_path, edge) { + next_node = edge.destination; + if (Visited_Node.get(next_node)) + return false; + next_path = Processed_Path.get(edge.destination); + if (!next_path) { + next_path = new path(edge.destination); + Processed_Path.add(next_path); + } + total_cost = current_path.cost + edge.cost; + if (total_cost < next_path.cost) { + next_path = current_path; + next_path.add_edge(edge); + next_path.cost = total_cost; + Priority_Queue.add(next_path); + } + return (next_path.destination == destination); + } + + +Definition +---------- + +.. c:struct:: constraints + +This is the constraints structure that contains: + +- cost: the total cost that the path must respect +- ctype: type of constraints: + + - CSPF_METRIC for standard metric + - CSPF_TE_METRIC for TE metric + - CSPF_DELAY for delay metric + +- bw: bandwidth that the path must respect +- cos: Class of Service (COS) for the bandwidth +- family: AF_INET or AF_INET6 +- type: RSVP_TE, SR_TE or SRV6_TE + +.. c:struct:: c_path + +This is the Constraint Path structure that contains: + +- edges: List of Edges that compose the path +- status: FAILED, IN_PROGRESS, SUCCESS, NO_SOURCE, NO_DESTINATION, SAME_SRC_DST +- weight: the cost from source to the destination of the path +- dst: key of the destination vertex + +.. c:struct:: cspf + +This is the main structure for path computation. Even if it is public, you +don't need to set manually the internal field of the structure. Instead, use +the following functions: + +.. c:function:: struct cspf *cspf_new(void); + +Function to create an empty cspf for future call of path computation + +.. c:function:: struct cspf *cspf_init(struct cspf *algo, const struct ls_vertex *src, const struct ls_vertex *dst, struct constraints *csts); + +This function initialize the cspf with source and destination vertex and +constraints and return pointer to the cspf structure. If input cspf structure +is NULL, a new cspf structure is allocated and initialize. + +.. c:function:: struct cspf *cspf_init_v4(struct cspf *algo, struct ls_ted *ted, const struct in_addr src, const struct in_addr dst, struct constraints *csts); + +Same as cspf_init, but here, source and destination vertex are extract from +the TED data base based on respective IPv4 source and destination addresses. + +.. c:function:: struct cspf *cspf_init_v6(struct cspf *algo, struct ls_ted *ted, const struct in6_addr src, const struct in6_addr dst, struct constraints *csts); + +Same as cspf_init_v4 but with IPv6 source and destination addresses. + +.. c:function:: void cspf_clean(struct cspf *algo); + +Clean internal structure of cspf in order to reuse it for another path +computation. + +.. c:function:: void cspf_del(struct cspf *algo); + +Delete cspf structure. A call to cspf_clean() function is perform prior to +free allocated memeory. + +.. c:function:: struct c_path *compute_p2p_path(struct ls_ted *ted, struct cspf *algo); + +Compute point to point path from the ted and cspf. +The function always return a constraints path. The status of the path gives +indication about the success or failure of the algorithm. If cspf structure has +not been initialize with a call to `cspf_init() or cspf_init_XX()`, the +algorithm returns a constraints path with status set to FAILED. +Note that a call to `cspf_clean()` is performed at the end of this function, +thus it is mandatory to initialize the cspf structure again prior to call again +the path computation algorithm. + + +Usage +----- + +Of course, CSPF algorithm needs a network topology that contains the +various metrics. Link State provides such Traffic Engineering Database. + +To perform a Path Computation with given constraints, proceed as follow: + +.. code-block:: c + + struct cspf *algo; + struct ls_ted *ted; + struct in_addr src; + struct in_addr dst; + struct constraints csts; + struct c_path *path; + + // Create a new CSPF structure + algo = cspf_new(); + + // Initialize constraints + csts.cost = 100; + csts.ctype = CSPF_TE_METRIC; + csts.family = AF_INET; + csts.type = SR_TE; + csts.bw = 1000000; + csts.cos = 3; + + // Then, initialise th CSPF with source, destination and constraints + cspf_init_v4(algo, ted, src, dst, &csts); + + // Finally, got the Computed Path; + path = compute_p2p_path(ted, algo); + + if (path.status == SUCCESS) + zlog_info("Got a valid constraints path"); + else + zlog_info("Unable to compute constraints path. Got %d status", path->status); + + +If you would compute another path, you must call `cspf_init()` prior to +`compute_p2p_path()` to change source, destination and/or constraints. diff --git a/doc/developer/draft-zebra-00.ms b/doc/developer/draft-zebra-00.ms new file mode 100644 index 0000000..b5d6924 --- /dev/null +++ b/doc/developer/draft-zebra-00.ms @@ -0,0 +1,209 @@ +.pl 10.0i +.po 0 +.ll 7.2i +.lt 7.2i +.nr LL 7.2i +.nr LT 7.2i +.ds LF Ishiguro +.ds RF FORMFEED[Page %] +.ds CF +.ds LH RFC DRAFT +.ds RH March 1998 +.ds CH +.hy 0 +.ad l +Network Working Group K. Ishiguro +Request for Comments: DRAFT Digital Magic Labs, Inc. + March 1998 +.sp 2 +.ce +Zebra Protocol Draft +.sp 2 +.fi +.ne 4 +Status of this Memo +.sp +.in 3 +This draft is very eary beta version. +.sp +.in 0 +.ne 4 +Introduction +.sp +.in 3 +The zebra protocol is a communication protocol between kernel +routing table manager and routing protocol daemon. It is built over +TCP/IP protocol suite. +.sp +.in 0 +.ne 4 +Request message formats +.sp +.in 3 +zebra is TCP-based protocol. +.sp +Below is request packet format. +.sp +.in 0 +.DS +0 1 2 3 +0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Length (2) | Command (1) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +.DE +.sp +.in 3 +Length is total packet length. +.sp +Here is summary of command list. +.sp +.in 0 +.DS +1 - ZEBRA_IPV4_ROUTE_ADD +2 - ZEBRA_IPV4_ROUTE_DELETE +3 - ZEBRA_IPV6_ROUTE_ADD +4 - ZEBRA_IPV6_ROUTE_DELETE +5 - ZEBRA_GET_ONE_INTERFACE +6 - ZEBRA_GET_ALL_INTERFACE +7 - ZEBRA_GET_HOSTINFO +.DE +.sp +.in 0 +.ne 4 +IPv4 reply message formats +.sp +.in 0 +.DS +0 1 2 3 +0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+ +| Type (1) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Gateway (4) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +.DE +.sp +.in 3 +Type field specify route's origin type. +.sp +.in 0 +.DS +1 - ZEBRA_ROUTE_RESERVE +2 - ZEBRA_ROUTE_CONNECT +3 - ZEBRA_ROUTE_STATIC +4 - ZEBRA_ROUTE_RIP +5 - ZEBRA_ROUTE_RIPNG +6 - ZEBRA_ROUTE_BGP +7 - ZEBRA_ROUTE_RADIX +.DE +.sp +.in 3 +After above message there can be variale length IPv4 prefix data. +Each IPv4 prefix is encoded as a two tuple of the form <masklength, +prefix> +.sp +.in 0 +.DS ++----------------------+ +|Subnet mask (1 octet) | ++----------------------+ +|IPv4 prefix (variable)| ++----------------------+ +.DE +.sp +.in 0 +.ne 4 +IPv6 reply message formats +.sp +.in 0 +.DS +0 1 2 3 +0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+ +| Type (1) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Gateway (16) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +.DE +.sp +.in 3 +Type field specify route's origin type. +.sp +.in 0 +.DS +1 - ZEBRA_ROUTE_RESERVE +2 - ZEBRA_ROUTE_CONNECT +3 - ZEBRA_ROUTE_STATIC +4 - ZEBRA_ROUTE_RIP +5 - ZEBRA_ROUTE_RIPNG +6 - ZEBRA_ROUTE_BGP +7 - ZEBRA_ROUTE_RADIX +.DE +.sp +.in 0 +.DS ++----------------------+ +| ifindex (4 octet) | ++----------------------+ +| prefixlen (1 octet)| ++----------------------+ +|IPv6 prefix (variable)| ++----------------------+ +.DE +.sp +.in 3 +I am not sure but it seems some operation systems IPv6 +implementation may need interface index when add and delete +linklocal routes. +.sp +I have added ifindex field to specify IPv6 routes interface +index. If this index is value zero, it will ignored. +.sp +.in 0 +.ne 4 +Interface information message format. +.sp +.in 0 +.DS +0 1 2 3 +0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Interface name (20) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Index (1) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Interface flag (4) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Interface metric (4) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Interface MTU (4) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| Interface Address count (4) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +.DE +.sp +.in 3 +Address message format. +.sp +.in 0 +.ne 4 +Host inforamtion message format. +.sp +.in 0 +.DS +0 1 2 3 +0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +|IPv4 forwarding|IPv6 forwarding| ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +.DE +.sp +.in 3 +Host information contain IPv4/IPv6 forwarding information. diff --git a/doc/developer/fpm.rst b/doc/developer/fpm.rst new file mode 100644 index 0000000..56d3367 --- /dev/null +++ b/doc/developer/fpm.rst @@ -0,0 +1,119 @@ +FPM +=== + +FPM stands for Forwarding Plane Manager and it's a module for use with Zebra. + +The encapsulation header for the messages exchanged with the FPM is +defined by the file :file:`fpm/fpm.h` in the frr tree. The routes +themselves are encoded in Netlink or protobuf format, with Netlink +being the default. + +Netlink is standard format for encoding messages to talk with kernel space +in Linux and it is also the name of the socket type used by it. +The FPM netlink usage differs from Linux's in: + +- Linux netlink sockets use datagrams in a multicast fashion, FPM uses + as a stream and it is unicast. +- FPM netlink messages might have more or less information than a normal + Linux netlink socket message (example: RTM_NEWROUTE might add an extra + route attribute to signalize VxLAN encapsulation). + +Protobuf is one of a number of new serialization formats wherein the +message schema is expressed in a purpose-built language. Code for +encoding/decoding to/from the wire format is generated from the +schema. Protobuf messages can be extended easily while maintaining +backward-compatibility with older code. Protobuf has the following +advantages over Netlink: + +- Code for serialization/deserialization is generated automatically. This + reduces the likelihood of bugs, allows third-party programs to be integrated + quickly, and makes it easy to add fields. +- The message format is not tied to an OS (Linux), and can be evolved + independently. + +.. note:: + + Currently there are two FPM modules in ``zebra``: + + * ``fpm`` + * ``dplane_fpm_nl`` + +fpm +^^^ + +The first FPM implementation that was built using hooks in ``zebra`` route +handling functions. It uses its own netlink/protobuf encoding functions to +translate ``zebra`` route data structures into formatted binary data. + + +dplane_fpm_nl +^^^^^^^^^^^^^ + +The newer FPM implementation that was built using ``zebra``'s data plane +framework as a plugin. It only supports netlink and it shares ``zebra``'s +netlink functions to translate route event snapshots into formatted binary +data. + + +Protocol Specification +---------------------- + +FPM (in any mode) uses a TCP connection to talk with external applications. +It operates as TCP client and uses the CLI configured address/port to connect +to the FPM server (defaults to port ``2620``). + +FPM frames all data with a header to help the external reader figure how +many bytes it has to read in order to read the full message (this helps +simulates datagrams like in the original netlink Linux kernel usage). + +Frame header: + +:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +---------------+---------------+-------------------------------+ + | Version | Message type | Message length | + +---------------+---------------+-------------------------------+ + | Data... | + +---------------------------------------------------------------+ + + +Version +^^^^^^^ + +Currently there is only one version, so it should be always ``1``. + + +Message Type +^^^^^^^^^^^^ + +Defines what underlining protocol we are using: netlink (``1``) or protobuf (``2``). + + +Message Length +^^^^^^^^^^^^^^ + +Amount of data in this frame in network byte order. + + +Data +^^^^ + +The netlink or protobuf message payload. + + +Route Status Notification from ASIC +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The dplane_fpm_nl has the ability to read route netlink messages +from the underlying fpm implementation that can tell zebra +whether or not the route has been Offloaded/Failed or Trapped. +The end developer must send the data up the same socket that has +been created to listen for FPM messages from Zebra. The data sent +must have a Frame Header with Version set to 1, Message Type set to 1 +and an appropriate message Length. The message data must contain +a RTM_NEWROUTE netlink message that sends the prefix and nexthops +associated with the route. Finally rtm_flags must contain +RTM_F_OFFLOAD, RTM_F_TRAP and or RTM_F_OFFLOAD_FAILED to signify +what has happened to the route in the ASIC. diff --git a/doc/developer/frr-release-procedure.rst b/doc/developer/frr-release-procedure.rst new file mode 100644 index 0000000..9378637 --- /dev/null +++ b/doc/developer/frr-release-procedure.rst @@ -0,0 +1,267 @@ +.. _frr-release-procedure: + +FRR Release Procedure +===================== + +``<version>`` - version to be released, e.g. 7.3 +``origin`` - FRR upstream repository + +Stage 1 - Preparation +--------------------- + +#. Prepare changelog for the new release + + Note: use ``tools/release_notes.py`` to help draft release notes changelog + +#. Checkout the existing ``dev/<version>`` branch. + + .. code-block:: console + + git checkout dev/<version> + +#. Create and push a new branch called ``stable/<version>`` based on the + ``dev/<version>`` branch. + + .. code-block:: console + + git checkout -b stable/<version> + +#. Remove the development branch called ``dev/<version>`` + + .. code-block:: console + + git push origin --delete dev/<version> + +#. Update Changelog for Red Hat Packages: + + Edit :file:`redhat/frr.spec.in` and look for the ``%changelog`` section: + + - Change last (top of list) entry from ``%{version}`` to the **last** + released version number. For example, if ``<version>`` is ``7.3`` and the + last public release was ``7.2``, you would use ``7.2``, changing the file + like so:: + + * Tue Nov 7 2017 Martin Winter <mwinter@opensourcerouting.org> - %{version} + + to:: + + * Tue Nov 7 2017 Martin Winter <mwinter@opensourcerouting.org> - 7.2 + + - Add new entry to the top of the list with ``%{version}`` tag. Make sure + to watch the format, i.e. the day is always 2 characters, with the 1st + character being a space if the day is one digit. + + - Add the changelog text below this entry. + +#. Update Changelog for Debian Packages: + + Update :file:`debian/changelog`: + + - Run following with **last** release version number and debian revision + (usually -1) as argument to ``dch --newversion VERSION``. For example, if + ``<version>`` is ``7.3`` then you will run ``dch --newversion 7.3-1``. + + - The ``dch`` will run an editor, and you should add the changelog text below + this entry, usually that would be: **New upstream version**. + + - Verify the changelog format using ``dpkg-parsechangelog``. In the + repository root: + + .. code-block:: console + + dpkg-parsechangelog + + You should see output like this:: + + vagrant@local ~/frr> dpkg-parsechangelog + Source: frr + Version: 7.3-dev-0 + Distribution: UNRELEASED + Urgency: medium + Maintainer: FRRouting-Dev <dev@lists.frrouting.org> + Timestamp: 1540478210 + Date: Thu, 25 Oct 2018 16:36:50 +0200 + Changes: + frr (7.3-dev-0) RELEASED; urgency=medium + . + * Your Changes Here + +#. Commit the changes, adding the changelog to the commit message. Follow all + existing commit guidelines. The commit message should be akin to:: + + debian, redhat: updating changelog for new release + +#. Change main version number: + + - Edit :file:`configure.ac` and change version in the ``AC_INIT`` command + to ``<version>`` + + Add and commit this change. This commit should be separate from the commit + containing the changelog. The commit message should be:: + + FRR Release <version> + + The version field should be complete; i.e. for ``8.0.0``, the version should + be ``8.0.0`` and not ``8.0`` or ``8``. + + +Stage 2 - Staging +----------------- + +#. Push the stable branch to a new remote branch prefixed with ``rc``:: + + git push origin stable/<version>:rc/version + + This will trigger the NetDEF CI, which serve as a sanity check on the + release branch. Verify that all tests pass and that all package builds are + successful. To do this, go to the NetDEF CI located here: + + https://ci1.netdef.org/browse/FRR-FRR + + In the top left, look for ``rc-<version>`` in the "Plan branch" dropdown. + Select this version. Note that it may take a few minutes for the CI to kick + in on this new branch and appear in the list. + +#. Push the stable branch: + + .. code-block:: console + + git push origin stable/<version>:refs/heads/stable/<version> + +#. Create and push a git tag for the version: + + .. code-block:: console + + git tag -a frr-<version> -m "FRRouting Release <version>" + git push origin frr-<version> + +#. Create a new branch based on ``master``, cherry-pick the commit made earlier + that added the changelogs, and use it to create a PR against ``master``. + This way ``master`` has the latest changelog for the next cycle. + +#. Kick off the "Release" build plan on the CI system for the correct release. + Contact Martin Winter for this step. Ensure all release packages build + successfully. + +#. Kick off the Snapcraft build plan for the release. + +#. Build Docker images + + 1. Log into the Netdef Docker build VM + 2. ``sudo -su builduser`` + 3. Suppose we are releasing 8.5.0, then ``X.Y.Z`` is ``8.5.0``. Run this: + + .. code-block:: console + + cd /home/builduser/frr + TAG=X.Y.Z + git fetch --all + git checkout frr-$TAG + docker buildx build --platform linux/amd64,linux/arm64,linux/ppc64le,linux/s390x,linux/arm/v7,linux/arm/v6 -f docker/alpine/Dockerfile -t quay.io/frrouting/frr:$TAG --push . + git tag docker/$TAG + git push origin docker/$TAG + + This will build a multi-arch image and upload it to Quay, as well as + create a git tag corresponding to the commit that the image was built + from and upload that to Github. It's important that the git tag point to + the exact codebase that was used to build the docker image, so if any + changes need to be made on top of the ``frr-$TAG`` release tag, make + sure these changes are committed and pointed at by the ``docker/X.Y.Z`` + tag. + + +Stage 3 - Publish +----------------- + +#. Upload both the Debian and RPM packages to their respective repositories. + +#. Coordinate with the maintainer of FRR's RPM repository to publish the RPM + packages on that repository. Update the repository webpage. Verify that the + instructions on the webpage work and that FRR is installable from the + repository on a Red Hat system. + + Current maintainer: *Martin Winter* + +#. Coordinate with the maintainer of FRR Debian package to publish the Debian + packages on that repository. Update the repository webpage. Verify that the + instructions on the webpage work and that FRR is installable from the + repository on a Debian system. + + Current maintainer: *Jafar Al-Gharaibeh* + +#. Log in to the Read The Docs instance. in the "FRRouting" project, navigate + to the "Overview" tab. Ensure there is a ``stable-<version>`` version listed + and that it is enabled. Go to "Admin" and then "Advanced Settings". Change + "Default version" to the new version. This ensures that the documentation + shown to visitors is that of the latest release by default. + + This step must be performed by someone with administrative access to the + Read the Docs instance. + +#. On GitHub, go to the <https://github.com/FRRouting/frr/releases>_ and click + "Draft a new release". Write a release announcement. The release + announcement should follow the template in + ``release-announcement-template.md``, located next to this document. Check + for spelling errors, and optionally (but preferably) have other maintainers + proofread the announcement text. + + Do not attach any packages or source tarballs to the GitHub release. + + Publish the release once it is reviewed. + +#. Deploy Snapcraft release. Remember that this will automatically upgrade Snap + users. + + Current maintainer: *Martin Winter* + +#. Build and publish the Docker containers. + + Current maintainer: *Quentin Young* + +#. Clone the ``frr-www`` repository: + + .. code-block:: console + + git clone https://github.com/FRRouting/frr-www.git + +#. Add a new release announcement, using a previous announcement as template: + + .. code-block:: console + + cp content/release/<old-version>.md content/release/<new-version>.md + + Paste the GitHub release announcement text into this document, and **remove + line breaks**. In other words, this:: + + This is one continuous + sentence that should be + rendered on one line + + Needs to be changed to this:: + + This is one continuous sentence that should be rendered on one line + + This is very important otherwise the announcement will be unreadable on the + website. + + To get the number of commiters and commits, here is a couple of handy commands: + + .. code-block:: console + + # The number of commits + % git log --oneline --no-merges base_8.2...base_8.1 | wc -l + + # The number of commiters + % git shortlog --summary --no-merges base_8.2...base_8.1 | wc -l + + Make sure to add a link to the GitHub releases page at the top. + +#. Deploy the updated ``frr-www`` on the frrouting.org web server and verify + that the announcement text is visible. + +#. Update readthedocs.org (Default Version) for https://docs.frrouting.org to + be the version of this latest release. + +#. Send an email to ``announce@lists.frrouting.org``. The text of this email + should include text as appropriate from the GitHub release and a link to the + GitHub release, Debian repository, and RPM repository. diff --git a/doc/developer/fuzzing.rst b/doc/developer/fuzzing.rst new file mode 100644 index 0000000..8a33187 --- /dev/null +++ b/doc/developer/fuzzing.rst @@ -0,0 +1,164 @@ +.. _fuzzing: + +Fuzzing +======= + +This page describes the fuzzing targets and supported fuzzers available in FRR +and how to use them. Familiarity with fuzzing techniques and tools is assumed. + +Overview +-------- + +It is well known that networked applications tend to be difficult to fuzz on +their network-facing attack surfaces. Approaches involving actual network +transmission tend to be slow and are subject to intermediate devices and +networking stacks which tend to drop fuzzed packets, especially if the fuzzing +surface covers IP itself. Some time was spent on fuzzing FRR this way with some +mediocre results but attention quickly turned towards skipping the actual +networking and instead adding fuzzing targets directly in the packet processing +code for use by more traditional in- and out-of-process fuzzers. Results from +this approach have been very fruitful. + +The patches to add fuzzing targets are kept in a separate git branch. Typically +it is better to keep them in the main branch so they are kept up to date and do +not need to be constantly synchronized with the main codebase. Unfortunately, +changes to FRR to support fuzzing necessarily extend far beyond the +entrypoints. Checksums must be disarmed, interactions with the kernel must be +skipped, sockets and files must be avoided, desired under/overflows must be +marked, etc. There are the usual ``LD_PRELOAD`` libraries to emulate these +things (preeny et al) but FRR is a very kernel-reliant program and these +libraries tend to create annoying problems when used with FRR for whatever +reason. Keeping this code in the main codebase is cluttering, difficult to work +with / around, and runs the risk of accidentally introducing bugs even if +``#ifdef``'d out. Consequently it's in a separate branch that is rebased on +``master`` from time to time. + + +Code +---- + +The git branch with fuzzing targets is located here: + +https://github.com/FRRouting/frr/tree/fuzz + +To build libFuzzer targets, pass ``--enable-libfuzzer`` to ``configure``. +To build AFL targets, compile with ``afl-clang`` as usual. + +Fuzzing with sanitizers is strongly recommended, especially ASAN, which you can +enable by passing ``--enable-address-sanitizer`` to ``configure``. + +Suggested UBSAN flags: ``-fsanitize-recover=unsigned-integer-overflow,implicit-conversion -fsanitize=unsigned-integer-overflow,implicit-conversion,nullability-arg,nullability-assign,nullability-return`` +Recommended cflags: ``-Wno-all -g3 -O3 -funroll-loops`` + +Design +------ + +All fuzzing targets have support for libFuzzer and AFL. This is done by writing +the target as a libFuzzer entrypoint (``LLVMFuzzerTestOneInput()``) and calling +it from the AFL entrypoint in ``main()``. New targets should use this rule. + +When adding AFL entrypoints, it's a good idea to use AFL persistent mode for +better performance. Grep ``bgpd/bgp_main.c`` for ``__AFL_INIT()`` for an +example of how to do this in FRR. Typically it involves moving all internal +daemon setup into a setup function. Then this setup function is called exactly +once for the lifetime of the process. In ``LLVMFuzzerTestOneInput()`` this +means you need to call it at the start of the function protected by a static +boolean that is set to true, since that function is your entrypoint. You also +need to call it prior to ``__AFL_INIT()`` in ``main()`` because ``main()`` is +your entrypoint in the AFL case. + +Adding support to daemons +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This section describes how to add entrypoints to daemons that do not have any +yet. + +Because libFuzzer has its own ``main()`` function, when adding fuzzing support +to a daemon that doesn't have any targets already, ``main()`` needs to be +``#ifdef``'d out like so: + +.. code:: c + + #ifndef FUZZING_LIBFUZZER + + int main(int argc, char **argv) + { + ... + } + + #endif /* FUZZING_LIBFUZZER */ + + +The ``FUZZING_LIBFUZZER`` macro is set by ``--enable-libfuzzer``. + +Because libFuzzer can only be linked into daemons that have +``LLVMFuzzerTestOneInput()`` implemented, we can't pass ``-fsanitize=fuzzer`` +to all daemons in ``AM_CFLAGS``. It needs to go into a variable specific to +each daemon. Since it can be thought of as a kind of sanitizer, for daemons +that have libFuzzer support there are now individual flags variables for those +daemons named ``DAEMON_SAN_FLAGS`` (e.g. ``BGPD_SAN_FLAGS``, +``ZEBRA_SAN_FLAGS``). This variable has the contents of the generic +``SAN_FLAGS`` plus any fuzzing-related flags. It is used in daemons' +``subdir.am`` in place of ``SAN_FLAGS``. Daemons that don't support libFuzzer +still use ``SAN_FLAGS``. If you want to add fuzzing support to a daemon you +need to do this flag variable conversion; look at ``configure.ac`` for +examples, it is fairly straightforward. Remember to update ``subdir.am`` to use +the new variable. + +Do note that when fuzzing is enabled, ``SAN_FLAGS`` gains +``-fsanitize=fuzzer-no-link``; the result is that all daemons are instrumented +for fuzzing but only the ones with ``LLVMFuzzerTestOneInput()`` actually get +linked with libFuzzer. + + +Targets +------- + +A given daemon can have lots of different paths that are interesting to fuzz. +There's not really a great way to handle this, most fuzzers assume the program +has one entrypoint. The approach taken in FRR for multiple entrypoints is to +control which path is taken within ``LLVMFuzzerTestOneInput()`` using +``#ifdef`` and passing whatever controlling macro definition you want. Take a +look at that function for the daemon you're interested in fuzzing, pick the +target, add ``#define MY_TARGET 1`` somewhere before the ``#ifdef`` switch, +recompile. + +.. list-table:: Fuzzing Targets + + * - Daemon + - Target + - Fuzzers + * - bgpd + - packet parser + - libfuzzer, afl + * - ospfd + - packet parser + - libfuzzer, afl + * - pimd + - packet parser + - libfuzzer, afl + * - vrrpd + - packet parser + - libfuzzer, afl + * - vrrpd + - zapi parser + - libfuzzer, afl + * - zebra + - netlink + - libfuzzer, afl + * - zebra + - zserv / zapi + - libfuzzer, afl + + +Fuzzer Notes +------------ + +Some interesting seed corpuses for various daemons are available `here +<https://github.com/qlyoung/frr-fuzz/tree/master/samples>`_. + +For libFuzzer, you need to pass ``-rss_limit_mb=0`` if you are fuzzing with +ASAN enabled, as you should. + +For AFL, afl++ is strongly recommended; afl proper isn't really maintained +anymore. diff --git a/doc/developer/grpc.rst b/doc/developer/grpc.rst new file mode 100644 index 0000000..4e81adf --- /dev/null +++ b/doc/developer/grpc.rst @@ -0,0 +1,524 @@ +.. _grpc-dev: + +*************** +Northbound gRPC +*************** + +To enable gRPC support one needs to add `--enable-grpc` when running +`configure`. Additionally, when launching each daemon one needs to request +the gRPC module be loaded and which port to bind to. This can be done by adding +`-M grpc:<port>` to the daemon's CLI arguments. + +Currently there is no gRPC "routing" so you will need to bind your gRPC +`channel` to the particular daemon's gRPC port to interact with that daemon's +gRPC northbound interface. + +The minimum version of gRPC known to work is 1.16.1. + +.. _grpc-languages-bindings: + +Programming Language Bindings +============================= + +The gRPC supported programming language bindings can be found here: +https://grpc.io/docs/languages/ + +After picking a programming language that supports gRPC bindings, the +next step is to generate the FRR northbound bindings. To generate the +northbound bindings you'll need the programming language binding +generator tools and those are language specific. + +C++ Example +----------- + +The next sections will use C++ as an example for accessing FRR +northbound through gRPC. + +.. _grpc-c++-generate: + +Generating C++ FRR Bindings +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generating FRR northbound bindings for C++ example: + +:: + + # Install gRPC (e.g., on Ubuntu 20.04) + sudo apt-get install libgrpc++-dev libgrpc-dev + + mkdir /tmp/frr-cpp + cd grpc + + protoc --cpp_out=/tmp/frr-cpp \ + --grpc_out=/tmp/frr-cpp \ + -I $(pwd) \ + --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` \ + frr-northbound.proto + + +.. _grpc-c++-if-sample: + +Using C++ To Get Version and Interfaces State +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Below is a sample program to print all interfaces discovered. + +:: + + # test.cpp + #include <string> + #include <sstream> + #include <grpc/grpc.h> + #include <grpcpp/create_channel.h> + #include "frr-northbound.pb.h" + #include "frr-northbound.grpc.pb.h" + + int main() { + frr::GetRequest request; + frr::GetResponse reply; + grpc::ClientContext context; + grpc::Status status; + + auto channel = grpc::CreateChannel("localhost:50051", + grpc::InsecureChannelCredentials()); + auto stub = frr::Northbound::NewStub(channel); + + request.set_type(frr::GetRequest::ALL); + request.set_encoding(frr::JSON); + request.set_with_defaults(true); + request.add_path("/frr-interface:lib"); + auto stream = stub->Get(&context, request); + + std::ostringstream ss; + while (stream->Read(&reply)) + ss << reply.data().data() << std::endl; + + status = stream->Finish(); + assert(status.ok()); + std::cout << "Interface Info:\n" << ss.str() << std::endl; + } + +Below is how to compile and run the program, with the example output: + +:: + + $ g++ -o test test.cpp frr-northbound.grpc.pb.cc frr-northbound.pb.cc -lgrpc++ -lprotobuf + $ ./test + Interface Info: + { + "frr-interface:lib": { + "interface": [ + { + "name": "lo", + "vrf": "default", + "state": { + "if-index": 1, + "mtu": 0, + "mtu6": 65536, + "speed": 0, + "metric": 0, + "phy-address": "00:00:00:00:00:00" + }, + "frr-zebra:zebra": { + "state": { + "up-count": 0, + "down-count": 0, + "ptm-status": "disabled" + } + } + }, + { + "name": "r1-eth0", + "vrf": "default", + "state": { + "if-index": 2, + "mtu": 1500, + "mtu6": 1500, + "speed": 10000, + "metric": 0, + "phy-address": "02:37:ac:63:59:b9" + }, + "frr-zebra:zebra": { + "state": { + "up-count": 0, + "down-count": 0, + "ptm-status": "disabled" + } + } + } + ] + }, + "frr-zebra:zebra": { + "mcast-rpf-lookup": "mrib-then-urib", + "workqueue-hold-timer": 10, + "zapi-packets": 1000, + "import-kernel-table": { + "distance": 15 + }, + "dplane-queue-limit": 200 + } + } + + + +.. _grpc-python-example: + +Python Example +-------------- + +The next sections will use Python as an example for writing scripts to use +the northbound. + +.. _grpc-python-generate: + +Generating Python FRR Bindings +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generating FRR northbound bindings for Python example: + +:: + + # Install python3 virtual environment capability e.g., + sudo apt-get install python3-venv + + # Create a virtual environment for python grpc and activate + python3 -m venv venv-grpc + source venv-grpc/bin/activate + + # Install grpc requirements + pip install grpcio grpcio-tools + + mkdir /tmp/frr-python + cd grpc + + python3 -m grpc_tools.protoc \ + --python_out=/tmp/frr-python \ + --grpc_python_out=/tmp/frr-python \ + -I $(pwd) \ + frr-northbound.proto + +.. _grpc-python-if-sample: + +Using Python To Get Capabilities and Interfaces State +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Below is a sample script to print capabilities and all interfaces Python +discovered. This demostrates the 2 different RPC results one gets from gRPC, +Unary (`GetCapabilities`) and Streaming (`Get`) for the interface state. + +:: + + import grpc + import frr_northbound_pb2 + import frr_northbound_pb2_grpc + + channel = grpc.insecure_channel('localhost:50051') + stub = frr_northbound_pb2_grpc.NorthboundStub(channel) + + # Print Capabilities + request = frr_northbound_pb2.GetCapabilitiesRequest() + response = stub.GetCapabilities(request) + print(response) + + # Print Interface State and Config + request = frr_northbound_pb2.GetRequest() + request.path.append("/frr-interface:lib") + request.type=frr_northbound_pb2.GetRequest.ALL + request.encoding=frr_northbound_pb2.XML + + for r in stub.Get(request): + print(r.data.data) + +The previous script will output something like: + +:: + + frr_version: "7.7-dev-my-manual-build" + rollback_support: true + supported_modules { + name: "frr-filter" + organization: "FRRouting" + revision: "2019-07-04" + } + supported_modules { + name: "frr-interface" + organization: "FRRouting" + revision: "2020-02-05" + } + [...] + supported_encodings: JSON + supported_encodings: XML + + <lib xmlns="http://frrouting.org/yang/interface"> + <interface> + <name>lo</name> + <vrf>default</vrf> + <state> + <if-index>1</if-index> + <mtu>0</mtu> + <mtu6>65536</mtu6> + <speed>0</speed> + <metric>0</metric> + <phy-address>00:00:00:00:00:00</phy-address> + </state> + <zebra xmlns="http://frrouting.org/yang/zebra"> + <state> + <up-count>0</up-count> + <down-count>0</down-count> + </state> + </zebra> + </interface> + <interface> + <name>r1-eth0</name> + <vrf>default</vrf> + <state> + <if-index>2</if-index> + <mtu>1500</mtu> + <mtu6>1500</mtu6> + <speed>10000</speed> + <metric>0</metric> + <phy-address>f2:62:2e:f3:4c:e4</phy-address> + </state> + <zebra xmlns="http://frrouting.org/yang/zebra"> + <state> + <up-count>0</up-count> + <down-count>0</down-count> + </state> + </zebra> + </interface> + </lib> + +.. _grpc-ruby-example: + +Ruby Example +------------ + +Next sections will use Ruby as an example for writing scripts to use +the northbound. + +.. _grpc-ruby-generate: + +Generating Ruby FRR Bindings +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generating FRR northbound bindings for Ruby example: + +:: + + # Install the required gems: + # - grpc: the gem that will talk with FRR's gRPC plugin. + # - grpc-tools: the gem that provides the code generator. + gem install grpc + gem install grpc-tools + + # Create your project/scripts directory: + mkdir /tmp/frr-ruby + + # Go to FRR's grpc directory: + cd grpc + + # Generate the ruby bindings: + grpc_tools_ruby_protoc \ + --ruby_out=/tmp/frr-ruby \ + --grpc_out=/tmp/frr-ruby \ + frr-northbound.proto + + +.. _grpc-ruby-if-sample: + +Using Ruby To Get Interfaces State +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Here is a sample script to print all interfaces FRR discovered: + +:: + + require 'frr-northbound_services_pb' + + # Create the connection with FRR's gRPC: + stub = Frr::Northbound::Stub.new('localhost:50051', :this_channel_is_insecure) + + # Create a new state request to get interface state: + request = Frr::GetRequest.new + request.type = :STATE + request.path.push('/frr-interface:lib') + + # Ask FRR. + response = stub.get(request) + + # Print the response. + response.each do |result| + result.data.data.each_line do |line| + puts line + end + end + + +.. note:: + + The generated files will assume that they are in the search path (e.g. + inside gem) so you'll need to either edit it to use ``require_relative`` or + tell Ruby where to look for them. For simplicity we'll use ``-I .`` to tell + it is in the current directory. + + +The previous script will output something like this: + +:: + + $ cd /tmp/frr-ruby + # Add `-I.` so ruby finds the FRR generated file locally. + $ ruby -I. interface.rb + { + "frr-interface:lib": { + "interface": [ + { + "name": "eth0", + "vrf": "default", + "state": { + "if-index": 2, + "mtu": 1500, + "mtu6": 1500, + "speed": 1000, + "metric": 0, + "phy-address": "11:22:33:44:55:66" + }, + "frr-zebra:zebra": { + "state": { + "up-count": 0, + "down-count": 0 + } + } + }, + { + "name": "lo", + "vrf": "default", + "state": { + "if-index": 1, + "mtu": 0, + "mtu6": 65536, + "speed": 0, + "metric": 0, + "phy-address": "00:00:00:00:00:00" + }, + "frr-zebra:zebra": { + "state": { + "up-count": 0, + "down-count": 0 + } + } + } + ] + } + } + + +.. _grpc-ruby-bfd-profile-sample: + +Using Ruby To Create BFD Profiles +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In this example you'll learn how to edit configuration using JSON +and programmatic (XPath) format. + +:: + + require 'frr-northbound_services_pb' + + # Create the connection with FRR's gRPC: + stub = Frr::Northbound::Stub.new('localhost:50051', :this_channel_is_insecure) + + # Create a new candidate configuration change. + new_candidate = stub.create_candidate(Frr::CreateCandidateRequest.new) + + # Use JSON to configure. + request = Frr::LoadToCandidateRequest.new + request.candidate_id = new_candidate.candidate_id + request.type = :MERGE + request.config = Frr::DataTree.new + request.config.encoding = :JSON + request.config.data = <<-EOJ + { + "frr-bfdd:bfdd": { + "bfd": { + "profile": [ + { + "name": "test-prof", + "detection-multiplier": 4, + "required-receive-interval": 800000 + } + ] + } + } + } + EOJ + + # Load configuration to candidate. + stub.load_to_candidate(request) + + # Commit candidate. + stub.commit( + Frr::CommitRequest.new( + candidate_id: new_candidate.candidate_id, + phase: :ALL, + comment: 'create test-prof' + ) + ) + + # + # Now lets delete the previous profile and create a new one. + # + + # Create a new candidate configuration change. + new_candidate = stub.create_candidate(Frr::CreateCandidateRequest.new) + + # Edit the configuration candidate. + request = Frr::EditCandidateRequest.new + request.candidate_id = new_candidate.candidate_id + + # Delete previously created profile. + request.delete.push( + Frr::PathValue.new( + path: "/frr-bfdd:bfdd/bfd/profile[name='test-prof']", + ) + ) + + # Add new profile with two configurations. + request.update.push( + Frr::PathValue.new( + path: "/frr-bfdd:bfdd/bfd/profile[name='test-prof-2']/detection-multiplier", + value: 5.to_s + ) + ) + request.update.push( + Frr::PathValue.new( + path: "/frr-bfdd:bfdd/bfd/profile[name='test-prof-2']/desired-transmission-interval", + value: 900_000.to_s + ) + ) + + # Modify the candidate. + stub.edit_candidate(request) + + # Commit the candidate configuration. + stub.commit( + Frr::CommitRequest.new( + candidate_id: new_candidate.candidate_id, + phase: :ALL, + comment: 'replace test-prof with test-prof-2' + ) + ) + + +And here is the new FRR configuration: + +:: + + $ sudo vtysh -c 'show running-config' + ... + bfd + profile test-prof-2 + detect-multiplier 5 + transmit-interval 900 + ! + ! diff --git a/doc/developer/hooks.rst b/doc/developer/hooks.rst new file mode 100644 index 0000000..b37a4ae --- /dev/null +++ b/doc/developer/hooks.rst @@ -0,0 +1,171 @@ +.. highlight:: c + +Hooks +===== + +Libfrr provides type-safe subscribable hook points where other pieces of +code can add one or more callback functions. "type-safe" in this case +applies to the function pointers used for subscriptions. The +implementations checks (at compile-time) whether a callback to be added has +the appropriate function signature (parameters) for the hook. + +Example: + +.. code-block:: c + :caption: mydaemon.h + + #include "hook.h" + DECLARE_HOOK(some_update_event, (struct eventinfo *info), (info)); + +.. code-block:: c + :caption: mydaemon.c + + #include "mydaemon.h" + DEFINE_HOOK(some_update_event, (struct eventinfo *info), (info)); + ... + hook_call(some_update_event, info); + +.. code-block:: c + :caption: mymodule.c + + #include "mydaemon.h" + static int event_handler(struct eventinfo *info); + ... + hook_register(some_update_event, event_handler); + +Do not use parameter names starting with "hook", these can collide with +names used by the hook code itself. + + +Return values +------------- + +Callbacks to be placed on hooks always return "int" for now; hook_call will +sum up the return values from each called function. (The default is 0 if no +callbacks are registered.) + +There are no pre-defined semantics for the value, in most cases it is +ignored. For success/failure indication, 0 should be success, and +handlers should make sure to only return 0 or 1 (not -1 or other values). + +There is no built-in way to abort executing a chain after a failure of one +of the callbacks. If this is needed, the hook can use an extra +``bool *aborted`` argument. + + +Priorities +---------- + +Hooks support a "priority" value for ordering registered calls +relative to each other. The priority is a signed integer where lower +values are called earlier. There are also "Koohs", which is hooks with +reverse priority ordering (for cleanup/deinit hooks, so you can use the +same priority value). + +Recommended priority value ranges are: + +======================== =================================================== +Range Usage +------------------------ --------------------------------------------------- + -999 ... 0 ... 999 main executable / daemon, or library + +-1999 ... -1000 modules registering calls that should run before + the daemon's bits + +1000 ... 1999 modules' calls that should run after daemon's + (includes default value: 1000) +======================== =================================================== + +Note: the default value is 1000, based on the following 2 expectations: + +- most hook_register() usage will be in loadable modules +- usage of hook_register() in the daemon itself may need relative ordering + to itself, making an explicit value the expected case + +The priority value is passed as extra argument on hook_register_prio() / +hook_register_arg_prio(). Whether a hook runs in reverse is determined +solely by the code defining / calling the hook. (DECLARE_KOOH is actually +the same thing as DECLARE_HOOK, it's just there to make it obvious.) + + +Definition +---------- + +.. c:macro:: DECLARE_HOOK(name, arglist, passlist) +.. c:macro:: DECLARE_KOOH(name, arglist, passlist) + + :param name: Name of the hook to be defined + :param arglist: Function definition style parameter list in braces. + :param passlist: List of the same parameters without their types. + + Note: the second and third macro args must be the hook function's + parameter list, with the same names for each parameter. The second + macro arg is with types (used for defining things), the third arg is + just the names (used for passing along parameters). + + This macro must be placed in a header file; this header file must be + included to register a callback on the hook. + + Examples: + + .. code-block:: c + + DECLARE_HOOK(foo, (), ()); + DECLARE_HOOK(bar, (int arg), (arg)); + DECLARE_HOOK(baz, (const void *x, in_addr_t y), (x, y)); + +.. c:macro:: DEFINE_HOOK(name, arglist, passlist) + + Implements an hook. Each ``DECLARE_HOOK`` must have be accompanied by + exactly one ``DEFINE_HOOK``, which needs to be placed in a source file. + **The hook can only be called from this source file.** This is intentional + to avoid overloading and/or misusing hooks for distinct purposes. + + The compiled source file will include a global symbol with the name of the + hook prefixed by `_hook_`. Trying to register a callback for a hook that + doesn't exist will therefore result in a linker error, or a module + load-time error for dynamic modules. + +.. c:macro:: DEFINE_KOOH(name, arglist, passlist) + + Same as ``DEFINE_HOOK``, but the sense of priorities / order of callbacks + is reversed. This should be used for cleanup hooks. + +.. c:function:: int hook_call(name, ...) + + Calls the specified named hook. Parameters to the hook are passed right + after the hook name, e.g.: + + .. code-block:: c + + hook_call(foo); + hook_call(bar, 0); + hook_call(baz, NULL, INADDR_ANY); + + Returns the sum of return values from all callbacks. The ``DEFINE_HOOK`` + statement for the hook must be placed in the file before any ``hook_call`` + use of the hook. + + +Callback registration +--------------------- + +.. c:function:: void hook_register(name, int (*callback)(...)) +.. c:function:: void hook_register_prio(name, int priority, int (*callback)(...)) +.. c:function:: void hook_register_arg(name, int (*callback)(void *arg, ...), void *arg) +.. c:function:: void hook_register_arg_prio(name, int priority, int (*callback)(void *arg, ...), void *arg) + + Register a callback with an hook. If the caller needs to pass an extra + argument to the callback, the _arg variant can be used and the extra + parameter will be passed as first argument to the callback. There is no + typechecking for this argument. + + The priority value is used as described above. The variants without a + priority parameter use 1000 as priority value. + +.. c:function:: void hook_unregister(name, int (*callback)(...)) +.. c:function:: void hook_unregister_arg(name, int (*callback)(void *arg, ...), void *arg) + + Removes a previously registered callback from a hook. Note that there + is no _prio variant of these calls. The priority value is only used during + registration. diff --git a/doc/developer/images/PCEPlib_design.jpg b/doc/developer/images/PCEPlib_design.jpg Binary files differnew file mode 100644 index 0000000..41aada3 --- /dev/null +++ b/doc/developer/images/PCEPlib_design.jpg diff --git a/doc/developer/images/PCEPlib_internal_deps.jpg b/doc/developer/images/PCEPlib_internal_deps.jpg Binary files differnew file mode 100644 index 0000000..8380021 --- /dev/null +++ b/doc/developer/images/PCEPlib_internal_deps.jpg diff --git a/doc/developer/images/PCEPlib_socket_comm.jpg b/doc/developer/images/PCEPlib_socket_comm.jpg Binary files differnew file mode 100644 index 0000000..3d62a46 --- /dev/null +++ b/doc/developer/images/PCEPlib_socket_comm.jpg diff --git a/doc/developer/images/PCEPlib_threading_model.jpg b/doc/developer/images/PCEPlib_threading_model.jpg Binary files differnew file mode 100644 index 0000000..afe91c2 --- /dev/null +++ b/doc/developer/images/PCEPlib_threading_model.jpg diff --git a/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg b/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg Binary files differnew file mode 100644 index 0000000..5648a9d --- /dev/null +++ b/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg diff --git a/doc/developer/images/PCEPlib_timers.jpg b/doc/developer/images/PCEPlib_timers.jpg Binary files differnew file mode 100644 index 0000000..a178ee9 --- /dev/null +++ b/doc/developer/images/PCEPlib_timers.jpg diff --git a/doc/developer/include-compile.rst b/doc/developer/include-compile.rst new file mode 100644 index 0000000..b98d237 --- /dev/null +++ b/doc/developer/include-compile.rst @@ -0,0 +1,30 @@ +Clone the FRR git repo and use the included ``configure`` script to configure +FRR's build time options to your liking. The full option listing can be +obtained by running ``./configure -h``. The options shown below are examples. + +.. code-block:: console + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + ./configure \ + --prefix=/usr \ + --includedir=\${prefix}/include \ + --bindir=\${prefix}/bin \ + --sbindir=\${prefix}/lib/frr \ + --libdir=\${prefix}/lib/frr \ + --libexecdir=\${prefix}/lib/frr \ + --localstatedir=/var/run/frr \ + --sysconfdir=/etc/frr \ + --with-moduledir=\${prefix}/lib/frr/modules \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-snmp=agentx \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + sudo make install diff --git a/doc/developer/index.rst b/doc/developer/index.rst new file mode 100644 index 0000000..bd794b1 --- /dev/null +++ b/doc/developer/index.rst @@ -0,0 +1,26 @@ +FRRouting Developer's Guide +=========================== + +.. toctree:: + :maxdepth: 2 + + workflow + checkpatch + building + packaging + process-architecture + library + fuzzing + tracing + testing + mgmtd-dev + bgpd + fpm + grpc + ospf + zebra + vtysh + path + pceplib + link-state + northbound/northbound diff --git a/doc/developer/ldpd-basic-test-setup.md b/doc/developer/ldpd-basic-test-setup.md new file mode 100644 index 0000000..b25a2b6 --- /dev/null +++ b/doc/developer/ldpd-basic-test-setup.md @@ -0,0 +1,681 @@ +## Topology + +The goal of this test is to verify that the all the basic functionality +of ldpd is working as expected, be it running on Linux or OpenBSD. In +addition to that, more advanced features are also tested, like LDP +sessions over IPv6, MD5 authentication and pseudowire signaling. + +In the topology below there are 3 PE routers, 3 CE routers and one P +router (not attached to any consumer site). + +All routers have IPv4 addresses and OSPF is used as the IGP. The +three routers from the bottom of the picture, P, PE2 and PE3, are also +configured for IPv6 (dual-stack) and static IPv6 routes are used to +provide connectivity among them. + +The three CEs share the same VPLS membership. LDP is used to set up the +LSPs among the PEs and to signal the pseudowires. MD5 authentication is +used to protect all LDP sessions. + +``` + CE1 172.16.1.1/24 + + + | + +---+---+ + | PE1 | + | IOS XE| + | | + +---+---+ + | + | 10.0.1.0/24 + | + +---+---+ + | P | + +------+ IOS XR+------+ + | | | | + | +-------+ | + 10.0.2.0/24 | | 10.0.3.0/24 +2001:db8:2::/64 | | 2001:db8:3::/64 + | | + +---+---+ +---+---+ + | PE2 | | PE3 | + |OpenBSD+-------------+ Linux | + | | | | + +---+---+ 10.0.4.0/24 +---+---+ + | 2001:db8:4::/64 | + + + + 172.16.1.2/24 CE2 CE3 172.16.1.3/24 +``` + +## Configuration + +#### Linux +1 - Enable IPv4/v6 forwarding: +``` +# sysctl -w net.ipv4.ip_forward=1 +# sysctl -w net.ipv6.conf.all.forwarding=1 +``` + +2 - Enable MPLS forwarding: +``` +# modprobe mpls-router +# modprobe mpls-iptunnel +# echo 100000 > /proc/sys/net/mpls/platform_labels +# echo 1 > /proc/sys/net/mpls/conf/eth1/input +# echo 1 > /proc/sys/net/mpls/conf/eth2/input +``` + +3 - Set up the interfaces: +``` +# ip link add name lo1 type dummy +# ip link set dev lo1 up +# ip addr add 4.4.4.4/32 dev lo1 +# ip -6 addr add 4:4:4::4/128 dev lo1 +# ip link set dev eth1 up +# ip addr add 10.0.4.4/24 dev eth1 +# ip -6 addr add 2001:db8:4::4/64 dev eth1 +# ip link set dev eth2 up +# ip addr add 10.0.3.4/24 dev eth2 +# ip -6 addr add 2001:db8:3::4/64 dev eth2 +``` + +4 - Set up the bridge and pseudowire interfaces: +``` +# ip link add type bridge +# ip link set dev bridge0 up +# ip link set dev eth0 up +# ip link set dev eth0 master bridge0 +# ip link add name mpw0 type dummy +# ip link set dev mpw0 up +# ip link set dev mpw0 master bridge0 +# ip link add name mpw1 type dummy +# ip link set dev mpw1 up +# ip link set dev mpw1 master bridge0 +``` + +> NOTE: MPLS support in the Linux kernel is very recent and it still +doesn't support pseudowire interfaces. We are using here dummy interfaces +just to show how the VPLS configuration should look like in the future. + +5 - Add static IPv6 routes for the remote loopbacks: +``` +# ip -6 route add 2:2:2::2/128 via 2001:db8:3::2 +# ip -6 route add 3:3:3::3/128 via 2001:db8:4::3 +``` + +6 - Edit /etc/frr/ospfd.conf: +``` +router ospf + network 4.4.4.4/32 area 0.0.0.0 + network 10.0.3.4/24 area 0.0.0.0 + network 10.0.4.4/24 area 0.0.0.0 +! +``` + +7 - Edit /etc/frr/ldpd.conf: +``` +debug mpls ldp messages recv +debug mpls ldp messages sent +debug mpls ldp zebra +! +mpls ldp + router-id 4.4.4.4 + dual-stack cisco-interop + neighbor 1.1.1.1 password opensourcerouting + neighbor 2.2.2.2 password opensourcerouting + neighbor 3.3.3.3 password opensourcerouting + ! + address-family ipv4 + discovery transport-address 4.4.4.4 + label local advertise explicit-null + ! + interface eth2 + ! + interface eth1 + ! + ! + address-family ipv6 + discovery transport-address 4:4:4::4 + ttl-security disable + ! + interface eth2 + ! + interface eth1 + ! + ! +! +l2vpn ENG type vpls + bridge br0 + member interface eth0 + ! + member pseudowire mpw0 + neighbor lsr-id 1.1.1.1 + pw-id 100 + ! + member pseudowire mpw1 + neighbor lsr-id 3.3.3.3 + neighbor address 3:3:3::3 + pw-id 100 + ! +! +``` + +> NOTE: We have to disable ttl-security under the ipv6 address-family +in order to interoperate with the IOS-XR router. GTSM is mandatory for +LDPv6 but the IOS-XR implementation is not RFC compliant in this regard. + +8 - Run zebra, ospfd and ldpd. + +#### OpenBSD +1 - Enable IPv4/v6 forwarding: +``` +# sysctl net.inet.ip.forwarding=1 +# sysctl net.inet6.ip6.forwarding=1 +``` + +2 - Enable MPLS forwarding: +``` +# ifconfig em2 10.0.2.3/24 mpls +# ifconfig em3 10.0.4.3/24 mpls +``` + +3 - Set up the interfaces: +``` +# ifconfig lo1 alias 3.3.3.3 netmask 255.255.255.255 +# ifconfig lo1 inet6 3:3:3::3/128 +# ifconfig em2 inet6 2001:db8:2::3/64 +# ifconfig em3 inet6 2001:db8:4::3/64 +``` + +4 - Set up the bridge and pseudowire interfaces: +``` +# ifconfig bridge0 create +# ifconfig bridge0 up +# ifconfig em1 up +# ifconfig bridge0 add em1 +# ifconfig mpw0 create +# ifconfig mpw0 up +# ifconfig bridge0 add mpw0 +# ifconfig mpw1 create +# ifconfig mpw1 up +# ifconfig bridge0 add mpw1 +``` + +5 - Add static IPv6 routes for the remote loopbacks: +``` +# route -n add 4:4:4::4/128 2001:db8:4::4 +# route -n add 2:2:2::2/128 2001:db8:2::2 +``` + +6 - Edit /etc/frr/ospfd.conf: +``` +router ospf + network 10.0.2.3/24 area 0 + network 10.0.4.3/24 area 0 + network 3.3.3.3/32 area 0 +! +``` + +7 - Edit /etc/frr/ldpd.conf: +``` +debug mpls ldp messages recv +debug mpls ldp messages sent +debug mpls ldp zebra +! +mpls ldp + router-id 3.3.3.3 + dual-stack cisco-interop + neighbor 1.1.1.1 password opensourcerouting + neighbor 2.2.2.2 password opensourcerouting + neighbor 4.4.4.4 password opensourcerouting + ! + address-family ipv4 + discovery transport-address 3.3.3.3 + label local advertise explicit-null + ! + interface em3 + ! + interface em2 + ! + ! + address-family ipv6 + discovery transport-address 3:3:3::3 + ttl-security disable + ! + interface em3 + ! + interface em2 + ! + ! +! +l2vpn ENG type vpls + bridge br0 + member interface em1 + ! + member pseudowire mpw0 + neighbor lsr-id 1.1.1.1 + pw-id 100 + ! + member pseudowire mpw1 + neighbor lsr-id 4.4.4.4 + neighbor address 4:4:4::4 + pw-id 100 + ! +! +``` + +8 - Run zebra, ospfd and ldpd. + +#### Cisco routers +CE1 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.1 255.255.255.0 + ! +! +``` + +CE2 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.2 255.255.255.0 + ! +! +``` + +CE3 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.3 255.255.255.0 + ! +! +``` + +PE1 - IOS-XE (1): +``` +mpls ldp neighbor 2.2.2.2 password opensourcerouting +mpls ldp neighbor 3.3.3.3 password opensourcerouting +mpls ldp neighbor 4.4.4.4 password opensourcerouting +! +l2vpn vfi context VFI + vpn id 1 + member pseudowire2 + member pseudowire1 +! +bridge-domain 1 + member GigabitEthernet1 service-instance 1 + member vfi VFI +! +interface Loopback1 + ip address 1.1.1.1 255.255.255.255 +! +interface pseudowire1 + encapsulation mpls + neighbor 3.3.3.3 100 +! +interface pseudowire2 + encapsulation mpls + neighbor 4.4.4.4 100 +! +interface GigabitEthernet3 + ip address 10.0.1.1 255.255.255.0 + mpls ip +! +router ospf 1 + network 0.0.0.0 255.255.255.255 area 0 +! +``` + +P - IOS-XR (2): +``` +interface Loopback1 + ipv4 address 2.2.2.2 255.255.255.255 + ipv6 address 2:2:2::2/128 +! +interface GigabitEthernet0/0/0/0 + ipv4 address 10.0.1.2 255.255.255.0 +! +interface GigabitEthernet0/0/0/1 + ipv4 address 10.0.2.2 255.255.255.0 + ipv6 address 2001:db8:2::2/64 + ipv6 enable +! +interface GigabitEthernet0/0/0/2 + ipv4 address 10.0.3.2 255.255.255.0 + ipv6 address 2001:db8:3::2/64 + ipv6 enable +! +router static + address-family ipv6 unicast + 3:3:3::3/128 2001:db8:2::3 + 4:4:4::4/128 2001:db8:3::4 + ! +! +router ospf 1 + router-id 2.2.2.2 + address-family ipv4 unicast + area 0 + interface Loopback1 + ! + interface GigabitEthernet0/0/0/0 + ! + interface GigabitEthernet0/0/0/1 + ! + interface GigabitEthernet0/0/0/2 + ! + ! +! +mpls ldp + router-id 2.2.2.2 + neighbor + 1.1.1.1:0 password clear opensourcerouting + 3.3.3.3:0 password clear opensourcerouting + 4.4.4.4:0 password clear opensourcerouting + ! + address-family ipv4 + ! + address-family ipv6 + discovery transport-address 2:2:2::2 + ! + interface GigabitEthernet0/0/0/0 + address-family ipv4 + ! + ! + interface GigabitEthernet0/0/0/1 + address-family ipv4 + ! + address-family ipv6 + ! + ! + interface GigabitEthernet0/0/0/2 + address-family ipv4 + ! + address-family ipv6 + ! + ! +! +``` + +## Verification - Control Plane + +Using the CLI on the Linux box, the goal is to ensure that everything +is working as expected. + +First, verify that all the required adjacencies and neighborships sessions +were established: + +``` +linux# show mpls ldp discovery +Local LDP Identifier: 4.4.4.4:0 +Discovery Sources: + Interfaces: + eth1: xmit/recv + LDP Id: 3.3.3.3:0, Transport address: 3.3.3.3 + Hold time: 15 sec + LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 + Hold time: 15 sec + eth2: xmit/recv + LDP Id: 2.2.2.2:0, Transport address: 2.2.2.2 + Hold time: 15 sec + LDP Id: 2.2.2.2:0, Transport address: 2:2:2::2 + Hold time: 15 sec + Targeted Hellos: + 4.4.4.4 -> 1.1.1.1: xmit/recv + LDP Id: 1.1.1.1:0, Transport address: 1.1.1.1 + Hold time: 45 sec + 4:4:4::4 -> 3:3:3::3: xmit/recv + LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 + Hold time: 45 sec + +linux# show mpls ldp neighbor +Peer LDP Identifier: 1.1.1.1:0 + TCP connection: 4.4.4.4:40921 - 1.1.1.1:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:06:02 + LDP Discovery Sources: + IPv4: + Targeted Hello: 1.1.1.1 + +Peer LDP Identifier: 2.2.2.2:0 + TCP connection: 4:4:4::4:52286 - 2:2:2::2:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:06:02 + LDP Discovery Sources: + IPv4: + Interface: eth2 + IPv6: + Interface: eth2 + +Peer LDP Identifier: 3.3.3.3:0 + TCP connection: 4:4:4::4:60575 - 3:3:3::3:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:05:57 + LDP Discovery Sources: + IPv4: + Interface: eth1 + IPv6: + Targeted Hello: 3:3:3::3 + Interface: eth1 +``` + +Note that the neighborships with the P and PE2 routers were established +over IPv6, since this is the default behavior for dual-stack LSRs, as +specified in RFC 7552. If desired, the **dual-stack transport-connection +prefer ipv4** command can be used to establish these sessions over IPv4 +(the command should be applied an all routers). + +Now, verify that there's a remote label for each PE address: +``` +linux# show mpls ldp binding +1.1.1.1/32 + Local binding: label: 20 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 imp-null + 2.2.2.2 24000 + 3.3.3.3 20 +2.2.2.2/32 + Local binding: label: 21 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 18 + 2.2.2.2 imp-null + 3.3.3.3 21 +3.3.3.3/32 + Local binding: label: 22 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 21 + 2.2.2.2 24003 + 3.3.3.3 imp-null +4.4.4.4/32 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 22 + 2.2.2.2 24001 + 3.3.3.3 22 +10.0.1.0/24 + Local binding: label: 23 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 imp-null + 2.2.2.2 imp-null + 3.3.3.3 23 +10.0.2.0/24 + Local binding: label: 24 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 20 + 2.2.2.2 imp-null + 3.3.3.3 imp-null +10.0.3.0/24 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 19 + 2.2.2.2 imp-null + 3.3.3.3 24 +10.0.4.0/24 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 23 + 2.2.2.2 24002 + 3.3.3.3 imp-null +2:2:2::2/128 + Local binding: label: 18 + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null + 3.3.3.3 18 +3:3:3::3/128 + Local binding: label: 19 + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 24007 +4:4:4::4/128 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 24006 + 3.3.3.3 19 +2001:db8:2::/64 + Local binding: label: - + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null + 3.3.3.3 imp-null +2001:db8:3::/64 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null +2001:db8:4::/64 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 3.3.3.3 imp-null +``` + +Check if the pseudowires are up: +``` +linux# show l2vpn atom vc +Interface Peer ID VC ID Name Status +--------- --------------- ---------- ---------------- ---------- +mpw1 3.3.3.3 100 ENG UP +mpw0 1.1.1.1 100 ENG UP +``` + +Check the label bindings of the pseudowires: +``` +linux# show l2vpn atom binding + Destination Address: 1.1.1.1, VC ID: 100 + Local Label: 25 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Remote Label: 16 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Destination Address: 3.3.3.3, VC ID: 100 + Local Label: 26 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Remote Label: 26 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 +``` + +## Verification - Data Plane + +Verify that all the exchanged label mappings were installed in zebra: +``` +linux# show mpls table + Inbound Outbound + Label Type Nexthop Label +-------- ------- --------------- -------- + 17 LDP 2001:db8:3::2 3 + 19 LDP 2001:db8:3::2 24005 + 20 LDP 10.0.3.2 24000 + 21 LDP 10.0.3.2 3 + 22 LDP 10.0.3.2 24001 + 23 LDP 10.0.3.2 3 + 24 LDP 10.0.3.2 3 + 25 LDP 10.0.3.2 3 + +linux# show ip route ldp +Codes: K - kernel route, C - connected, S - static, R - RIP, + O - OSPF, I - IS-IS, B - BGP, P - PIM, A - Babel, L - LDP, + > - selected route, * - FIB route + +L>* 1.1.1.1/32 [0/0] via 10.0.3.2, eth2 label 24000 +L>* 3.3.3.3/32 [0/0] via 10.0.3.2, eth2 label 24001 +``` + +Verify that all the exchanged label mappings were installed in the kernel: +``` +$ ip -M ro +17 via inet6 2001:db8:3::2 dev eth2 proto zebra +19 as to 24005 via inet6 2001:db8:3::2 dev eth2 proto zebra +20 as to 24000 via inet 10.0.3.2 dev eth2 proto zebra +21 via inet 10.0.3.2 dev eth2 proto zebra +22 as to 24001 via inet 10.0.3.2 dev eth2 proto zebra +23 via inet 10.0.3.2 dev eth2 proto zebra +24 via inet 10.0.3.2 dev eth2 proto zebra +25 via inet 10.0.3.2 dev eth2 proto zebra +$ +$ ip route | grep mpls +1.1.1.1 encap mpls 24000 via 10.0.3.2 dev eth2 proto zebra metric 20 +3.3.3.3 encap mpls 24001 via 10.0.3.2 dev eth2 proto zebra metric 20 +``` + +Now ping PE1's loopback using lo1's address as a source address: +``` +$ ping -c 5 -I 4.4.4.4 1.1.1.1 +PING 1.1.1.1 (1.1.1.1) from 4.4.4.4 : 56(84) bytes of data. +64 bytes from 1.1.1.1: icmp_seq=1 ttl=253 time=3.02 ms +64 bytes from 1.1.1.1: icmp_seq=2 ttl=253 time=3.13 ms +64 bytes from 1.1.1.1: icmp_seq=3 ttl=253 time=3.19 ms +64 bytes from 1.1.1.1: icmp_seq=4 ttl=253 time=3.07 ms +64 bytes from 1.1.1.1: icmp_seq=5 ttl=253 time=3.27 ms + +--- 1.1.1.1 ping statistics --- +5 packets transmitted, 5 received, 0% packet loss, time 4005ms +rtt min/avg/max/mdev = 3.022/3.140/3.278/0.096 ms +``` + +Verify that the ICMP echo request packets are leaving with the MPLS +label advertised by the P router. Also, verify that the ICMP echo reply +packets are arriving with an explicit-null MPLS label: +``` +# tcpdump -n -i eth2 mpls and icmp +tcpdump: verbose output suppressed, use -v or -vv for full protocol decode +listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes +10:01:40.758771 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 1, length 64 +10:01:40.761777 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 1, length 64 +10:01:41.760343 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 2, length 64 +10:01:41.763448 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 2, length 64 +10:01:42.761758 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 3, length 64 +10:01:42.764924 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 3, length 64 +10:01:43.763193 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 4, length 64 +10:01:43.766237 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 4, length 64 +10:01:44.764552 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 5, length 64 +10:01:44.767803 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 5, length 64 +``` diff --git a/doc/developer/library.rst b/doc/developer/library.rst new file mode 100644 index 0000000..2e36c25 --- /dev/null +++ b/doc/developer/library.rst @@ -0,0 +1,21 @@ +.. _libfrr: + +*************************** +Library Facilities (libfrr) +*************************** + +.. toctree:: + :maxdepth: 2 + + memtypes + rcu + lists + logging + xrefs + locking + hooks + cli + modules + scripting + + diff --git a/doc/developer/link-state.rst b/doc/developer/link-state.rst new file mode 100644 index 0000000..aaa253d --- /dev/null +++ b/doc/developer/link-state.rst @@ -0,0 +1,499 @@ +Link State API Documentation +============================ + +Introduction +------------ + +The Link State (LS) API aims to provide a set of structures and functions to +build and manage a Traffic Engineering Database for the various FRR daemons. +This API has been designed for several use cases: + +- BGP Link State (BGP-LS): where BGP protocol need to collect the link state + information from the routing daemons (IS-IS and/or OSPF) to implement RFC7752 +- Path Computation Element (PCE): where path computation algorithms are based + on Traffic Engineering Database +- ReSerVation Protocol (RSVP): where signaling need to know the Traffic + Engineering topology of the network in order to determine the path of + RSVP tunnels + +Architecture +------------ + +The main requirements from the various uses cases are as follow: + +- Provides a set of data model and function to ease Link State information + manipulation (storage, serialize, parse ...) +- Ease and normalize Link State information exchange between FRR daemons +- Provides database structure for Traffic Engineering Database (TED) + +To ease Link State understanding, FRR daemons have been classified into two +categories: + +- **Consumer**: Daemons that consume Link State information e.g. BGPd +- **Producer**: Daemons that are able to collect Link State information and + send them to consumer daemons e.g. OSPFd IS-ISd + +Zebra daemon, and more precisely, the ZAPI message is used to convey the Link +State information between *producer* and *consumer*, but, Zebra acts as a +simple pass through and does not store any Link State information. A new ZAPI +**Opaque** message has been design for that purpose. + +Each consumer and producer daemons are free to store or not Link State data and +organise the information following the Traffic Engineering Database model +provided by the API or any other data structure e.g. Hash, RB-tree ... + +Link State API +-------------- + +This is the low level API that allows any daemons manipulate the Link State +elements that are stored in the Link State Database. + +Data structures +^^^^^^^^^^^^^^^ + +3 types of Link State structure have been defined: + +.. c:struct:: ls_node + + that groups all information related to a node + +.. c:struct:: ls_attributes + + that groups all information related to a link + +.. c:struct:: ls_prefix + + that groups all information related to a prefix + +These 3 types of structures are those handled by BGP-LS (see RFC7752) and +suitable to describe a Traffic Engineering topology. + +Each structure, in addition to the specific parameters, embed the node +identifier which advertises the Link State and a bit mask as flags to +indicates which parameters are valid i.e. for which the value is valid and +corresponds to a Link State information conveyed by the routing protocol. + +.. c:struct:: ls_node_id + + defines the Node identifier as router ID IPv4 address plus the area ID for + OSPF or the ISO System ID plus the IS-IS level for IS-IS. + +Functions +^^^^^^^^^ + +A set of functions is provided to create, delete and compare Link State +Node, Atribute and Prefix: + +.. c:function:: struct ls_node *ls_node_new(struct ls_node_id adv, struct in_addr router_id, struct in6_addr router6_id) +.. c:function:: struct ls_attributes *ls_attributes_new(struct ls_node_id adv, struct in_addr local, struct in6_addr local6, uint32_t local_id) +.. c:function:: struct ls_prefix *ls_prefix_new(struct ls_node_id adv, struct prefix p) + + Create respectively a new Link State Node, Attribute or Prefix. + Structure is dynamically allocated. Link State Node ID (adv) is mandatory + and: + + - at least one of IPv4 or IPv6 must be provided for the router ID + (router_id or router6_id) for Node + - at least one of local, local6 or local_id must be provided for Attribute + - prefix is mandatory for Link State Prefix. + +.. c:function:: void ls_node_del(struct ls_node *node) +.. c:function:: void ls_attributes_del(struct ls_attributes *attr) +.. c:function:: void ls_prefix_del(struct ls_prefix *pref) + + Remove, respectively Link State Node, Attributes or Prefix. + Data structure is freed. + +.. c:function:: void ls_attributes_srlg_del(struct ls_attributes *attr) + + Remove SRLGs attribute if defined. Data structure is freed. + +.. c:function:: int ls_node_same(struct ls_node *n1, struct ls_node *n2) +.. c:function:: int ls_attributes_same(struct ls_attributes *a1, struct ls_attributes *a2) +.. c:function:: int ls_prefix_same(struct ls_prefix *p1, struct ls_prefix*p2) + + Check, respectively if two Link State Nodes, Attributes or Prefix are equal. + Note that these routines have the same return value sense as '==' (which is + different from a comparison). + + +Link State TED +-------------- + +This is the high level API that provides functions to create, update, delete a +Link State Database to build a Traffic Engineering Database (TED). + +Data Structures +^^^^^^^^^^^^^^^ + +The Traffic Engineering is modeled as a Graph in order to ease Path Computation +algorithm implementation. Denoted **G(V, E)**, a graph is composed by a list of +**Vertices (V)** which represents the network Node and a list of **Edges (E)** +which represents Link. An additional list of **prefixes (P)** is also added and +also attached to the *Vertex (V)* which advertise it. + +*Vertex (V)* contains the list of outgoing *Edges (E)* that connect this Vertex +with its direct neighbors and the list of incoming *Edges (E)* that connect +the direct neighbors to this Vertex. Indeed, the *Edge (E)* is unidirectional, +thus, it is necessary to add 2 Edges to model a bidirectional relation between +2 Vertices. Finally, the *Vertex (V)* contains a pointer to the corresponding +Link State Node. + +*Edge (E)* contains the source and destination Vertex that this Edge +is connecting and a pointer to the corresponding Link State Attributes. + +A unique Key is used to identify both Vertices and Edges within the Graph. + + +:: + + -------------- --------------------------- -------------- + | Connected |---->| Connected Edge Va to Vb |--->| Connected | + --->| Vertex | --------------------------- | Vertex |----> + | | | | + | - Key (Va) | | - Key (Vb) | + <---| - Vertex | --------------------------- | - Vertex |<---- + | |<----| Connected Edge Vb to Va |<---| | + -------------- --------------------------- -------------- + + +4 data structures have been defined to implement the Graph model: + +.. c:struct:: ls_vertex +.. c:struct:: ls_edge +.. c:struct:: ls_ted + + - :c:struct:`ls_prefix` + +TED stores Vertex, Edge and Subnet elements with a RB Tree structure. +The Vertex key corresponds to the Router ID for OSPF and ISO System ID for +IS-IS. The Edge key corresponds to the IPv4 address, the lowest 64 bits of +the IPv6 address or the combination of the local & remote ID of the interface. +The Subnet key corresponds to the Prefix address (v4 or v6). + +An additional status for Vertex, Edge and Subnet allows to determine the state +of the element in the TED: UNSET, NEW, UPDATE, DELETE, SYNC, ORPHAN. Normal +state is SYNC. NEW, UPDATE and DELETE are temporary state when element is +processed. UNSET is normally never used and ORPHAN serves to identify elements +that must be remove when TED is cleaning. + +Vertex, Edges and Subnets management functions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. c:function:: struct ls_vertex *ls_vertex_add(struct ls_ted *ted, struct ls_node *node) +.. c:function:: struct ls_edge *ls_edge_add(struct ls_ted *ted, struct ls_attributes *attributes) +.. c:function:: struct ls_subnet *ls_subnet_add(struct ls_ted *ted, struct ls_prefix *pref) + + Add, respectively new Vertex, Edge or Subnet to the Link State Datebase. + Vertex, Edge or Subnet are created from, respectively the Link State Node, + Attribute or Prefix structure. Data structure are dynamically allocated. + +.. c:function:: struct ls_vertex *ls_vertex_update(struct ls_ted *ted, struct ls_node *node) +.. c:function:: struct ls_edge *ls_edge_update(struct ls_ted *ted, struct ls_attributes *attributes) +.. c:function:: struct ls_subnet *ls_subnet_update(struct ls_ted *ted, struct ls_prefix *pref) + + Update, respectively Vertex, Edge or Subnet with, respectively the Link + State Node, Attribute or Prefix. A new data structure is created if no one + corresponds to the Link State Node, Attribute or Prefix. If element already + exists in the TED, its associated Link State information is replaced by the + new one if there are different and the old associated Link State information + is deleted and memory freed. + +.. c:function:: void ls_vertex_del(struct ls_ted *ted, struct ls_vertex *vertex) +.. c:function:: void ls_vertex_del_all(struct ls_ted *ted, struct ls_vertex *vertex) +.. c:function:: void ls_edge_del(struct ls_ted *ted, struct ls_edge *edge) +.. c:function:: void ls_edge_del_all(struct ls_ted *ted, struct ls_edge *edge) +.. c:function:: void ls_subnet_del(struct ls_ted *ted, struct ls_subnet *subnet) +.. c:function:: void ls_subnet_del_all(struct ls_ted *ted, struct ls_subnet *subnet) + + Delete, respectively Link State Vertex, Edge or Subnet. Data structure are + freed but not the associated Link State information with the simple `_del()` + form of the function while the `_del_all()` version freed also associated + Link State information. TED is not modified if Vertex, Edge or Subnet is + NULL or not found in the Data Base. Note that references between Vertices, + Edges and Subnets are removed first. + +.. c:function:: struct ls_vertex *ls_find_vertex_by_key(struct ls_ted *ted, const uint64_t key) +.. c:function:: struct ls_vertex *ls_find_vertex_by_id(struct ls_ted *ted, struct ls_node_id id) + + Find Vertex in the TED by its unique key or its Link State Node ID. + Return Vertex if found, NULL otherwise. + +.. c:function:: struct ls_edge *ls_find_edge_by_key(struct ls_ted *ted, const uint64_t key) +.. c:function:: struct ls_edge *ls_find_edge_by_source(struct ls_ted *ted, struct ls_attributes *attributes); +.. c:function:: struct ls_edge *ls_find_edge_by_destination(struct ls_ted *ted, struct ls_attributes *attributes); + + Find Edge in the Link State Data Base by its key, source or distination + (local IPv4 or IPv6 address or local ID) informations of the Link State + Attributes. Return Edge if found, NULL otherwise. + +.. c:function:: struct ls_subnet *ls_find_subnet(struct ls_ted *ted, const struct prefix prefix) + + Find Subnet in the Link State Data Base by its key, i.e. the associated + prefix. Return Subnet if found, NULL otherwise. + +.. c:function:: int ls_vertex_same(struct ls_vertex *v1, struct ls_vertex *v2) +.. c:function:: int ls_edge_same(struct ls_edge *e1, struct ls_edge *e2) +.. c:function:: int ls_subnet_same(struct ls_subnet *s1, struct ls_subnet *s2) + + Check, respectively if two Vertices, Edges or Subnets are equal. + Note that these routines has the same return value sense as '==' + (which is different from a comparison). + + +TED management functions +^^^^^^^^^^^^^^^^^^^^^^^^ + +Some helpers functions have been also provided to ease TED management: + +.. c:function:: struct ls_ted *ls_ted_new(const uint32_t key, char *name, uint32_t asn) + + Create a new Link State Data Base. Key must be different from 0. + Name could be NULL and AS number equal to 0 if unknown. + +.. c:function:: void ls_ted_del(struct ls_ted *ted) +.. c:function:: void ls_ted_del_all(struct ls_ted *ted) + + Delete existing Link State Data Base. Vertices, Edges, and Subnets are not + removed with ls_ted_del() function while they are with ls_ted_del_all(). + +.. c:function:: void ls_connect_vertices(struct ls_vertex *src, struct ls_vertex *dst, struct ls_edge *edge) + + Connect Source and Destination Vertices by given Edge. Only non NULL source + and destination vertices are connected. + +.. c:function:: void ls_connect(struct ls_vertex *vertex, struct ls_edge *edge, bool source) +.. c:function:: void ls_disconnect(struct ls_vertex *vertex, struct ls_edge *edge, bool source) + + Connect / Disconnect Link State Edge to the Link State Vertex which could be + a Source (source = true) or a Destination (source = false) Vertex. + +.. c:function:: void ls_disconnect_edge(struct ls_edge *edge) + + Disconnect Link State Edge from both Source and Destination Vertex. + Note that Edge is not removed but its status is marked as ORPHAN. + +.. c:function:: void ls_vertex_clean(struct ls_ted *ted, struct ls_vertex *vertex, struct zclient *zclient) + + Clean Vertex structure by removing all Edges and Subnets marked as ORPHAN + from this vertex. Corresponding Link State Update message is sent if zclient + parameter is not NULL. Note that associated Link State Attribute and Prefix + are also removed and memory freed. + +.. c:function:: void ls_ted_clean(struct ls_ted *ted) + + Clean Link State Data Base by removing all Vertices, Edges and SubNets + marked as ORPHAN. Note that associated Link State Node, Attributes and + Prefix are removed too. + +.. c:function:: void ls_show_vertex(struct ls_vertex *vertex, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_edge(struct ls_edeg *edge, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_subnet(struct ls_subnet *subnet, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_vertices(struct ls_ted *ted, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_edges(struct ls_ted *ted, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_subnets(struct ls_ted *ted, struct vty *vty, struct json_object *json, bool verbose) +.. c:function:: void ls_show_ted(struct ls_ted *ted, struct vty *vty, struct json_object *json, bool verbose) + + Respectively, show Vertex, Edge, Subnet provided as parameter, all Vertices, + all Edges, all Subnets and the whole TED if not specified. Output could be + more detailed with verbose parameter for VTY output. If both JSON and VTY + output are specified, JSON takes precedence over VTY. + +.. c:function:: void ls_dump_ted(struct ls_ted *ted) + + Dump TED information to the current logging output. + +Link State Messages +------------------- + +This part of the API provides functions and data structure to ease the +communication between the *Producer* and *Consumer* daemons. + +Communications principles +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Recent ZAPI Opaque Message is used to exchange Link State data between daemons. +For that purpose, Link State API provides new functions to serialize and parse +Link State information through the ZAPI Opaque message. A dedicated flag, +named ZAPI_OPAQUE_FLAG_UNICAST, allows daemons to send a unicast or a multicast +Opaque message and is used as follow for the Link State exchange: + +- Multicast: To send data update to all daemons that have subscribed to the + Link State Update message +- Unicast: To send initial Link State information from a particular daemon. All + data are send only to the daemon that request Link State Synchronisatio + +Figure 1 below, illustrates the ZAPI Opaque message exchange between a +*Producer* (an IGP like OSPF or IS-IS) and a *Consumer* (e.g. BGP). The +message sequences are as follows: + +- First, both *Producer* and *Consumer* must register to their respective ZAPI + Opaque Message: **Link State Sync** for the *Producer* in order to receive + Database synchronisation request from a *Consumer*, **Link State Update** for + the *Consumer* in order to received any Link State update from a *Producer*. + These register messages are stored by Zebra to determine to which daemon it + should redistribute the ZAPI messages it receives. +- Then, the *Consumer* sends a **Link State Synchronistation** request with the + Multicast method in order to receive the complete Link State Database from a + *Producer*. ZEBRA daemon forwards this message to any *Producer* daemons that + previously registered to this message. If no *Producer* has yet registered, + the request is lost. Thus, if the *Consumer* receives no response whithin a + given timer, it means that no *Producer* are available right now. So, the + *Consumer* must send the same request until it receives a Link State Database + Synchronistation message. This behaviour is necessary as we can't control in + which order daemons are started. It is up to the *Consumer* daemon to fix the + timeout and the number of retry. +- When a *Producer* receives a **Link State Synchronisation** request, it + starts sending all elements of its own Link State Database through the + **Link State Database Synchronisation** message. These messages are send with + the Unicast method to avoid flooding other daemons with these elements. ZEBRA + layer ensures to forward the message to the right daemon. +- When a *Producer* update its Link State Database, it automatically sends a + **Link State Update** message with the Multicast method. In turn, ZEBRA + daemon forwards the message to all *Consumer* daemons that previously + registered to this message. if no daemon is registered, the message is lost. +- A daemon could unregister from the ZAPI Opaque message registry at any time. + In this case, the ZEBRA daemon stops to forward any messages it receives to + this daemon, even if it was previously converns. + +:: + + IGP ZEBRA Consumer + (OSPF/IS-IS) (ZAPI Opaque Thread) (e.g. BGP) + | | | \ + | | Register LS Update | | + | |<----------------------------| Register Phase + | | | | + | | Request LS Sync | | + | |<----------------------------| | + : : : A | + | Register LS Sync | | | | + |----------------------------->| | | / + : : : |TimeOut + : : : | + | | | | + | | Request LS Sync | v \ + | Request LS Sync |<----------------------------| | + |<-----------------------------| | Synchronistation + | LS DB Update | | Phase + |----------------------------->| LS DB Update | | + | |---------------------------->| | + | LS DB Update (cont'd) | | | + |----------------------------->| LS DB Update (cont'd) | | + | . |---------------------------->| | + | . | . | | + | . | . | | + | LS DB Update (end) | . | | + |----------------------------->| LS DB Update (end) | | + | |---------------------------->| | + | | | / + : : : + : : : + | LS DB Update | | \ + |----------------------------->| LS DB Update | | + | |---------------------------->| Update Phase + | | | | + : : : / + : : : + | | | \ + | | Unregister LS Update | | + | |<----------------------------| Deregister Phase + | | | | + | LS DB Update | | | + |----------------------------->| | | + | | | / + | | | + + Figure 1: Link State messages exchange + + +Data Structures +^^^^^^^^^^^^^^^ + +The Link State Message is defined to convey Link State parameters from +the routing protocol (OSPF or IS-IS) to other daemons e.g. BGP. + +.. c:struct:: ls_message + +The structure is composed of: + +- Event of the message: + + - Sync: Send the whole LS DB following a request + - Add: Send the a new Link State element + - Update: Send an update of an existing Link State element + - Delete: Indicate that the given Link State element is removed + +- Type of Link State element: Node, Attribute or Prefix +- Remote node id when known +- Data: Node, Attributes or Prefix + +A Link State Message can carry only one Link State Element (Node, Attributes +of Prefix) at once, and only one Link State Message is sent through ZAPI +Opaque Link State type at once. + +Functions +^^^^^^^^^ + +.. c:function:: int ls_register(struct zclient *zclient, bool server) +.. c:function:: int ls_unregister(struct zclient *zclient, bool server) + + Register / Unregister daemon to received ZAPI Link State Opaque messages. + Server must be set to true for *Producer* and to false for *Consumer*. + +.. c:function:: int ls_request_sync(struct zclient *zclient) + + Request initial Synchronisation to collect the whole Link State Database. + +.. c:function:: struct ls_message *ls_parse_msg(struct stream *s) + + Parse Link State Message from stream. Used this function once receiving a + new ZAPI Opaque message of type Link State. + +.. c:function:: void ls_delete_msg(struct ls_message *msg) + + Delete existing message. Data structure is freed. + +.. c:function:: int ls_send_msg(struct zclient *zclient, struct ls_message *msg, struct zapi_opaque_reg_info *dst) + + Send Link State Message as new ZAPI Opaque message of type Link State. + If destination is not NULL, message is sent as Unicast otherwise it is + broadcast to all registered daemon. + +.. c:function:: struct ls_message *ls_vertex2msg(struct ls_message *msg, struct ls_vertex *vertex) +.. c:function:: struct ls_message *ls_edge2msg(struct ls_message *msg, struct ls_edge *edge) +.. c:function:: struct ls_message *ls_subnet2msg(struct ls_message *msg, struct ls_subnet *subnet) + + Create respectively a new Link State Message from a Link State Vertex, Edge + or Subnet. If Link State Message is NULL, a new data structure is + dynamically allocated. Note that the Vertex, Edge and Subnet status is used + to determine the corresponding Link State Message event: ADD, UPDATE, + DELETE, SYNC. + +.. c:function:: int ls_msg2vertex(struct ls_ted *ted, struct ls_message *msg) +.. c:function:: int ls_msg2edge(struct ls_ted *ted, struct ls_message *msg) +.. c:function:: int ls_msg2subnet(struct ls_ted *ted, struct ls_message *msg) + + Convert Link State Message respectively in Vertex, Edge or Subnet and + update the Link State Database accordingly to the message event: SYNC, ADD, + UPDATE or DELETE. + +.. c:function:: struct ls_element *ls_msg2ted(struct ls_ted *ted, struct ls_message *msg, bool delete) +.. c:function:: struct ls_element *ls_stream2ted(struct ls_ted *ted, struct ls_message *msg, bool delete) + + Convert Link State Message or Stream Buffer in a Link State element (Vertex, + Edge or Subnet) and update the Link State Database accordingly to the + message event: SYNC, ADD, UPDATE or DELETE. The function return the generic + structure ls_element that point to the Vertex, Edge or Subnet which has been + added, updated or synchronous in the database. Note that the delete boolean + parameter governs the action for the DELETE action: true, Link State Element + is removed from the database and NULL is return. If set to false, database + is not updated and the function sets the Link State Element status to + Delete and return the element for futur deletion by the calling function. + +.. c:function:: int ls_sync_ted(struct ls_ted *ted, struct zclient *zclient, struct zapi_opaque_reg_info *dst) + + Send all the content of the Link State Data Base to the given destination. + Link State content is sent is this order: Vertices, Edges then Subnet. + This function must be used when a daemon request a Link State Data Base + Synchronization. diff --git a/doc/developer/lists.rst b/doc/developer/lists.rst new file mode 100644 index 0000000..ccac10a --- /dev/null +++ b/doc/developer/lists.rst @@ -0,0 +1,777 @@ +.. _lists: + +Type-safe containers +==================== + +.. note:: + + This section previously used the term *list*; it was changed to *container* + to be more clear. + +Common container interface +-------------------------- + +FRR includes a set of container implementations with abstracted +common APIs. The purpose of this is easily allow swapping out one +data structure for another while also making the code easier to read and write. +There is one API for unsorted containers and a similar but not identical API +for sorted containers - and heaps use a middle ground of both. + +For unsorted containers, the following implementations exist: + +- single-linked list with tail pointer (e.g. STAILQ in BSD) + +- double-linked list + +- atomic single-linked list with tail pointer + + +Being partially sorted, the oddball structure: + +- an 8-ary heap + + +For sorted containers, these data structures are implemented: + +- single-linked list + +- atomic single-linked list + +- skiplist + +- red-black tree (based on OpenBSD RB_TREE) + +- hash table (note below) + +Except for hash tables, each of the sorted data structures has a variant with +unique and non-unique items. Hash tables always require unique items +and mostly follow the "sorted" API but use the hash value as sorting +key. Also, iterating while modifying does not work with hash tables. +Conversely, the heap always has non-unique items, but iterating while modifying +doesn't work either. + + +The following sorted structures are likely to be implemented at some point +in the future: + +- atomic skiplist + +- atomic hash table (note below) + + +The APIs are all designed to be as type-safe as possible. This means that +there will be a compiler warning when an item doesn't match the container, or +the return value has a different type, or other similar situations. **You +should never use casts with these APIs.** If a cast is necessary in relation +to these APIs, there is probably something wrong with the overall design. + +Only the following pieces use dynamically allocated memory: + +- the hash table itself is dynamically grown and shrunk + +- skiplists store up to 4 next pointers inline but will dynamically allocate + memory to hold an item's 5th up to 16th next pointer (if they exist) + +- the heap uses a dynamically grown and shrunk array of items + +Cheat sheet +----------- + +Available types: + +:: + + DECLARE_LIST + DECLARE_ATOMLIST + DECLARE_DLIST + + DECLARE_HEAP + + DECLARE_SORTLIST_UNIQ + DECLARE_SORTLIST_NONUNIQ + DECLARE_ATOMLIST_UNIQ + DECLARE_ATOMLIST_NONUNIQ + DECLARE_SKIPLIST_UNIQ + DECLARE_SKIPLIST_NONUNIQ + DECLARE_RBTREE_UNIQ + DECLARE_RBTREE_NONUNIQ + + DECLARE_HASH + +Functions provided: + ++------------------------------------+-------+------+------+---------+------------+ +| Function | LIST | HEAP | HASH | \*_UNIQ | \*_NONUNIQ | ++====================================+=======+======+======+=========+============+ +| _init, _fini | yes | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ +| _first, _next, _next_safe, | yes | yes | yes | yes | yes | +| | | | | | | +| _const_first, _const_next | | | | | | ++------------------------------------+-------+------+------+---------+------------+ +| _last, _prev, _prev_safe, | DLIST | -- | -- | RB only | RB only | +| | only | | | | | +| _const_last, _const_prev | | | | | | ++------------------------------------+-------+------+------+---------+------------+ +| _swap_all | yes | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ +| _anywhere | yes | -- | -- | -- | -- | ++------------------------------------+-------+------+------+---------+------------+ +| _add_head, _add_tail, _add_after | yes | -- | -- | -- | -- | ++------------------------------------+-------+------+------+---------+------------+ +| _add | -- | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ +| _member | yes | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ +| _del, _pop | yes | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ +| _find, _const_find | -- | -- | yes | yes | -- | ++------------------------------------+-------+------+------+---------+------------+ +| _find_lt, _find_gteq, | -- | -- | -- | yes | yes | +| | | | | | | +| _const_find_lt, _const_find_gteq | | | | | | ++------------------------------------+-------+------+------+---------+------------+ +| use with frr_each() macros | yes | yes | yes | yes | yes | ++------------------------------------+-------+------+------+---------+------------+ + + + +Datastructure type setup +------------------------ + +Each of the data structures has a ``PREDECL_*`` and a ``DECLARE_*`` macro to +set up an "instantiation" of the container. This works somewhat similar to C++ +templating, though much simpler. + +**In all following text, the Z prefix is replaced with a name chosen +for the instance of the datastructure.** + +The common setup pattern will look like this: + +.. code-block:: c + + #include <typesafe.h> + + PREDECL_XXX(Z); + struct item { + int otherdata; + struct Z_item mylistitem; + } + + struct Z_head mylisthead; + + /* unsorted: */ + DECLARE_XXX(Z, struct item, mylistitem); + + /* sorted, items that compare as equal cannot be added to list */ + int compare_func(const struct item *a, const struct item *b); + DECLARE_XXX_UNIQ(Z, struct item, mylistitem, compare_func); + + /* sorted, items that compare as equal can be added to list */ + int compare_func(const struct item *a, const struct item *b); + DECLARE_XXX_NONUNIQ(Z, struct item, mylistitem, compare_func); + + /* hash tables: */ + int compare_func(const struct item *a, const struct item *b); + uint32_t hash_func(const struct item *a); + DECLARE_XXX(Z, struct item, mylistitem, compare_func, hash_func); + +``XXX`` is replaced with the name of the data structure, e.g. ``SKIPLIST`` +or ``ATOMLIST``. The ``DECLARE_XXX`` invocation can either occur in a `.h` +file (if the container needs to be accessed from several C files) or it can be +placed in a `.c` file (if the container is only accessed from that file.) The +``PREDECL_XXX`` invocation defines the ``struct Z_item`` and ``struct +Z_head`` types and must therefore occur before these are used. + +To switch between compatible data structures, only these two lines need to be +changes. To switch to a data structure with a different API, some source +changes are necessary. + +Common iteration macros +----------------------- + +The following iteration macros work across all data structures: + +.. c:macro:: frr_each(Z, head, item) + + Equivalent to: + + .. code-block:: c + + for (item = Z_first(&head); item; item = Z_next(&head, item)) + + Note that this will fail if the container is modified while being iterated + over. + +.. c:macro:: frr_each_safe(Z, head, item) + + Same as the previous, but the next element is pre-loaded into a "hidden" + variable (named ``Z_safe``.) Equivalent to: + + .. code-block:: c + + for (item = Z_first(&head); item; item = next) { + next = Z_next_safe(&head, item); + ... + } + + .. warning:: + + Iterating over hash tables while adding or removing items is not + possible. The iteration position will be corrupted when the hash + tables is resized while iterating. This will cause items to be + skipped or iterated over twice. + +.. c:macro:: frr_each_from(Z, head, item, from) + + Iterates over the container, starting at item ``from``. This variant is + "safe" as in the previous macro. Equivalent to: + + .. code-block:: c + + for (item = from; item; item = from) { + from = Z_next_safe(&head, item); + ... + } + + .. note:: + + The ``from`` variable is written to. This is intentional - you can + resume iteration after breaking out of the loop by keeping the ``from`` + value persistent and reusing it for the next loop. + +.. c:macro:: frr_rev_each(Z, head, item) +.. c:macro:: frr_rev_each_safe(Z, head, item) +.. c:macro:: frr_rev_each_from(Z, head, item, from) + + Reverse direction variants of the above. Only supported on containers that + implement ``_last`` and ``_prev`` (i.e. ``RBTREE`` and ``DLIST``). + +To iterate over ``const`` pointers, add ``_const`` to the name of the +datastructure (``Z`` above), e.g. ``frr_each (mylist, head, item)`` becomes +``frr_each (mylist_const, head, item)``. + +Common API +---------- + +The following documentation assumes that a container has been defined using +``Z`` as the name, and ``itemtype`` being the type of the items (e.g. +``struct item``.) + +.. c:function:: void Z_init(struct Z_head *) + + Initializes the container for use. For most implementations, this just sets + some values. Hash tables are the only implementation that allocates + memory in this call. + +.. c:function:: void Z_fini(struct Z_head *) + + Reverse the effects of :c:func:`Z_init()`. The container must be empty + when this function is called. + + .. warning:: + + This function may ``assert()`` if the container is not empty. + +.. c:function:: size_t Z_count(const struct Z_head *) + + Returns the number of items in a structure. All structures store a + counter in their `Z_head` so that calling this function completes + in O(1). + + .. note:: + + For atomic containers with concurrent access, the value will already be + outdated by the time this function returns and can therefore only be + used as an estimate. + +.. c:function:: bool Z_member(const struct Z_head *, const itemtype *) + + Determines whether some item is a member of the given container. The + item must either be valid on some container, or set to all zeroes. + + On some containers, if no faster way to determine membership is possible, + this is simply ``item == Z_find(head, item)``. + + Not currently available for atomic containers. + +.. c:function:: const itemtype *Z_const_first(const struct Z_head *) +.. c:function:: itemtype *Z_first(struct Z_head *) + + Returns the first item in the structure, or ``NULL`` if the structure is + empty. This is O(1) for all data structures except red-black trees + where it is O(log n). + +.. c:function:: const itemtype *Z_const_last(const struct Z_head *) +.. c:function:: itemtype *Z_last(struct Z_head *) + + Last item in the structure, or ``NULL``. Only available on containers + that support reverse iteration (i.e. ``RBTREE`` and ``DLIST``). + +.. c:function:: itemtype *Z_pop(struct Z_head *) + + Remove and return the first item in the structure, or ``NULL`` if the + structure is empty. Like :c:func:`Z_first`, this is O(1) for all + data structures except red-black trees where it is O(log n) again. + + This function can be used to build queues (with unsorted structures) or + priority queues (with sorted structures.) + + Another common pattern is deleting all container items: + + .. code-block:: c + + while ((item = Z_pop(head))) + item_free(item); + + .. note:: + + This function can - and should - be used with hash tables. It is not + affected by the "modification while iterating" problem. To remove + all items from a hash table, use the loop demonstrated above. + +.. c:function:: const itemtype *Z_const_next(const struct Z_head *, const itemtype *prev) +.. c:function:: itemtype *Z_next(struct Z_head *, itemtype *prev) + + Return the item that follows after ``prev``, or ``NULL`` if ``prev`` is + the last item. + + .. warning:: + + ``prev`` must not be ``NULL``! Use :c:func:`Z_next_safe()` if + ``prev`` might be ``NULL``. + +.. c:function:: itemtype *Z_next_safe(struct Z_head *, itemtype *prev) + + Same as :c:func:`Z_next()`, except that ``NULL`` is returned if + ``prev`` is ``NULL``. + +.. c:function:: const itemtype *Z_const_prev(const struct Z_head *, const itemtype *next) +.. c:function:: itemtype *Z_prev(struct Z_head *, itemtype *next) +.. c:function:: itemtype *Z_prev_safe(struct Z_head *, itemtype *next) + + As above, but preceding item. Only available on structures that support + reverse iteration (i.e. ``RBTREE`` and ``DLIST``). + +.. c:function:: itemtype *Z_del(struct Z_head *, itemtype *item) + + Remove ``item`` from the container and return it. + + .. note:: + + This function's behaviour is undefined if ``item`` is not actually + on the container. Some structures return ``NULL`` in this case while + others return ``item``. The function may also call ``assert()`` (but + most don't.) + +.. c:function:: itemtype *Z_swap_all(struct Z_head *, struct Z_head *) + + Swap the contents of 2 containers (of identical type). This exchanges the + contents of the two head structures and updates pointers if necessary for + the particular data structure. Fast for all structures. + + (Not currently available on atomic containers.) + +.. todo:: + + ``Z_del_after()`` / ``Z_del_hint()``? + +API for unsorted structures +--------------------------- + +Since the insertion position is not pre-defined for unsorted data, there +are several functions exposed to insert data: + +.. note:: + + ``item`` must not be ``NULL`` for any of the following functions. + +.. c:macro:: DECLARE_XXX(Z, type, field) + + :param listtype XXX: ``LIST``, ``DLIST`` or ``ATOMLIST`` to select a data + structure implementation. + :param token Z: Gives the name prefix that is used for the functions + created for this instantiation. ``DECLARE_XXX(foo, ...)`` + gives ``struct foo_item``, ``foo_add_head()``, ``foo_count()``, etc. Note + that this must match the value given in ``PREDECL_XXX(foo)``. + :param typename type: Specifies the data type of the list items, e.g. + ``struct item``. Note that ``struct`` must be added here, it is not + automatically added. + :param token field: References a struct member of ``type`` that must be + typed as ``struct foo_item``. This struct member is used to + store "next" pointers or other data structure specific data. + +.. c:function:: void Z_add_head(struct Z_head *, itemtype *item) + + Insert an item at the beginning of the structure, before the first item. + This is an O(1) operation for non-atomic lists. + +.. c:function:: void Z_add_tail(struct Z_head *, itemtype *item) + + Insert an item at the end of the structure, after the last item. + This is also an O(1) operation for non-atomic lists. + +.. c:function:: void Z_add_after(struct Z_head *, itemtype *after, itemtype *item) + + Insert ``item`` behind ``after``. If ``after`` is ``NULL``, the item is + inserted at the beginning of the list as with :c:func:`Z_add_head`. + This is also an O(1) operation for non-atomic lists. + + A common pattern is to keep a "previous" pointer around while iterating: + + .. code-block:: c + + itemtype *prev = NULL, *item; + + frr_each_safe(Z, head, item) { + if (something) { + Z_add_after(head, prev, item); + break; + } + prev = item; + } + + .. todo:: + + maybe flip the order of ``item`` & ``after``? + ``Z_add_after(head, item, after)`` + +.. c:function:: bool Z_anywhere(const itemtype *) + + Returns whether an item is a member of *any* container of this type. + The item must either be valid on some container, or set to all zeroes. + + Guaranteed to be fast (pointer compare or similar.) + + Not currently available for sorted and atomic containers. Might be added + for sorted containers at some point (when needed.) + + +API for sorted structures +------------------------- + +Sorted data structures do not need to have an insertion position specified, +therefore the insertion calls are different from unsorted containers. Also, +sorted containers can be searched for a value. + +.. c:macro:: DECLARE_XXX_UNIQ(Z, type, field, compare_func) + + :param listtype XXX: One of the following: + ``SORTLIST`` (single-linked sorted list), ``SKIPLIST`` (skiplist), + ``RBTREE`` (RB-tree) or ``ATOMSORT`` (atomic single-linked list). + :param token Z: Gives the name prefix that is used for the functions + created for this instantiation. ``DECLARE_XXX(foo, ...)`` + gives ``struct foo_item``, ``foo_add()``, ``foo_count()``, etc. Note + that this must match the value given in ``PREDECL_XXX(foo)``. + :param typename type: Specifies the data type of the items, e.g. + ``struct item``. Note that ``struct`` must be added here, it is not + automatically added. + :param token field: References a struct member of ``type`` that must be + typed as ``struct foo_item``. This struct member is used to + store "next" pointers or other data structure specific data. + :param funcptr compare_func: Item comparison function, must have the + following function signature: + ``int function(const itemtype *, const itemtype*)``. This function + may be static if the container is only used in one file. + +.. c:macro:: DECLARE_XXX_NONUNIQ(Z, type, field, compare_func) + + Same as above, but allow adding multiple items to the container that compare + as equal in ``compare_func``. Ordering between these items is undefined + and depends on the container implementation. + +.. c:function:: itemtype *Z_add(struct Z_head *, itemtype *item) + + Insert an item at the appropriate sorted position. If another item exists + in the container that compares as equal (``compare_func()`` == 0), ``item`` + is not inserted and the already-existing item in the container is + returned. Otherwise, on successful insertion, ``NULL`` is returned. + + For ``_NONUNIQ`` containers, this function always returns NULL since + ``item`` can always be successfully added to the container. + +.. c:function:: const itemtype *Z_const_find(const struct Z_head *, const itemtype *ref) +.. c:function:: itemtype *Z_find(struct Z_head *, const itemtype *ref) + + Search the container for an item that compares equal to ``ref``. If no + equal item is found, return ``NULL``. + + This function is likely used with a temporary stack-allocated value for + ``ref`` like so: + + .. code-block:: c + + itemtype searchfor = { .foo = 123 }; + + itemtype *item = Z_find(head, &searchfor); + + .. note:: + + The ``Z_find()`` function is only available for containers that contain + unique items (i.e. ``DECLARE_XXX_UNIQ``.) This is because on a container + with non-unique items, more than one item may compare as equal to + the item that is searched for. + +.. c:function:: const itemtype *Z_const_find_gteq(const struct Z_head *, const itemtype *ref) +.. c:function:: itemtype *Z_find_gteq(struct Z_head *, const itemtype *ref) + + Search the container for an item that compares greater or equal to + ``ref``. See :c:func:`Z_find()` above. + +.. c:function:: const itemtype *Z_const_find_lt(const struct Z_head *, const itemtype *ref) +.. c:function:: itemtype *Z_find_lt(struct Z_head *, const itemtype *ref) + + Search the container for an item that compares less than + ``ref``. See :c:func:`Z_find()` above. + + +API for hash tables +------------------- + +.. c:macro:: DECLARE_HASH(Z, type, field, compare_func, hash_func) + + :param listtype HASH: Only ``HASH`` is currently available. + :param token Z: Gives the name prefix that is used for the functions + created for this instantiation. ``DECLARE_XXX(foo, ...)`` + gives ``struct foo_item``, ``foo_add()``, ``foo_count()``, etc. Note + that this must match the value given in ``PREDECL_XXX(foo)``. + :param typename type: Specifies the data type of the items, e.g. + ``struct item``. Note that ``struct`` must be added here, it is not + automatically added. + :param token field: References a struct member of ``type`` that must be + typed as ``struct foo_item``. This struct member is used to + store "next" pointers or other data structure specific data. + :param funcptr compare_func: Item comparison function, must have the + following function signature: + ``int function(const itemtype *, const itemtype*)``. This function + may be static if the container is only used in one file. For hash tables, + this function is only used to check for equality, the ordering is + ignored. + :param funcptr hash_func: Hash calculation function, must have the + following function signature: + ``uint32_t function(const itemtype *)``. The hash value for items + stored in a hash table is cached in each item, so this value need not + be cached by the user code. + + .. warning:: + + Items that compare as equal cannot be inserted. Refer to the notes + about sorted structures in the previous section. + + +.. c:function:: void Z_init_size(struct Z_head *, size_t size) + + Same as :c:func:`Z_init()` but preset the minimum hash table to + ``size``. + +Hash tables also support :c:func:`Z_add()` and :c:func:`Z_find()` with +the same semantics as noted above. :c:func:`Z_find_gteq()` and +:c:func:`Z_find_lt()` are **not** provided for hash tables. + +Hash table invariants +^^^^^^^^^^^^^^^^^^^^^ + +There are several ways to injure yourself using the hash table API. + +First, note that there are two functions related to computing uniqueness of +objects inserted into the hash table. There is a hash function and a comparison +function. The hash function computes the hash of the object. Our hash table +implementation uses `chaining +<https://en.wikipedia.org/wiki/Hash_table#Separate_chaining_with_linked_lists>`_. +This means that your hash function does not have to be perfect; multiple +objects having the same computed hash will be placed into a linked list +corresponding to that key. The closer to perfect the hash function, the better +performance, as items will be more evenly distributed and the chain length will +not be long on any given lookup, minimizing the number of list operations +required to find the correct item. However, the comparison function *must* be +perfect, in the sense that any two unique items inserted into the hash table +must compare not equal. At insertion time, if you try to insert an item that +compares equal to an existing item the insertion will not happen and +``hash_get()`` will return the existing item. However, this invariant *must* be +maintained while the object is in the hash table. Suppose you insert items +``A`` and ``B`` into the hash table which both hash to the same value ``1234`` +but do not compare equal. They will be placed in a chain like so:: + + 1234 : A -> B + +Now suppose you do something like this elsewhere in the code:: + + *A = *B + +I.e. you copy all fields of ``B`` into ``A``, such that the comparison function +now says that they are equal based on their contents. At this point when you +look up ``B`` in the hash table, ``hash_get()`` will search the chain for the +first item that compares equal to ``B``, which will be ``A``. This leads to +insidious bugs. + +.. warning:: + + Never modify the values looked at by the comparison or hash functions after + inserting an item into a hash table. + +A similar situation can occur with the hash allocation function. ``hash_get()`` +accepts a function pointer that it will call to get the item that should be +inserted into the list if the provided item is not already present. There is a +builtin function, ``hash_alloc_intern``, that will simply return the item you +provided; if you always want to store the value you pass to ``hash_get`` you +should use this one. If you choose to provide a different one, that function +*must* return a new item that hashes and compares equal to the one you provided +to ``hash_get()``. If it does not the behavior of the hash table is undefined. + +.. warning:: + + Always make sure your hash allocation function returns a value that hashes + and compares equal to the item you provided to ``hash_get()``. + +Finally, if you maintain pointers to items you have inserted into a hash table, +then before deallocating them you must release them from the hash table. This +is basic memory management but worth repeating as bugs have arisen from failure +to do this. + + +API for heaps +------------- + +Heaps provide the same API as the sorted data structures, except: + +* none of the find functions (:c:func:`Z_find()`, :c:func:`Z_find_gteq()` + or :c:func:`Z_find_lt()`) are available. +* iterating over the heap yields the items in semi-random order, only the + first item is guaranteed to be in order and actually the "lowest" item + on the heap. Being a heap, only the rebalancing performed on removing the + first item (either through :c:func:`Z_pop()` or :c:func:`Z_del()`) causes + the new lowest item to bubble up to the front. +* all heap modifications are O(log n). However, cacheline efficiency and + latency is likely quite a bit better than with other data structures. + +Atomic lists +------------ + +`atomlist.h` provides an unsorted and a sorted atomic single-linked list. +Since atomic memory accesses can be considerably slower than plain memory +accessses (depending on the CPU type), these lists should only be used where +necessary. + +The following guarantees are provided regarding concurrent access: + +- the operations are lock-free but not wait-free. + + Lock-free means that it is impossible for all threads to be blocked. Some + thread will always make progress, regardless of what other threads do. (This + even includes a random thread being stopped by a debugger in a random + location.) + + Wait-free implies that the time any single thread might spend in one of the + calls is bounded. This is not provided here since it is not normally + relevant to practical operations. What this means is that if some thread is + hammering a particular list with requests, it is possible that another + thread is blocked for an extended time. The lock-free guarantee still + applies since the hammering thread is making progress. + +- without a RCU mechanism in place, the point of contention for atomic lists + is memory deallocation. As it is, **a rwlock is required for correct + operation**. The *read* lock must be held for all accesses, including + reading the list, adding items to the list, and removing items from the + list. The *write* lock must be acquired and released before deallocating + any list element. If this is not followed, an use-after-free can occur + as a MT race condition when an element gets deallocated while another + thread is accessing the list. + + .. note:: + + The *write* lock does not need to be held for deleting items from the + list, and there should not be any instructions between the + ``pthread_rwlock_wrlock`` and ``pthread_rwlock_unlock``. The write lock + is used as a sequence point, not as an exclusion mechanism. + +- insertion operations are always safe to do with the read lock held. + Added items are immediately visible after the insertion call returns and + should not be touched anymore. + +- when removing a *particular* (pre-determined) item, the caller must ensure + that no other thread is attempting to remove that same item. If this cannot + be guaranteed by architecture, a separate lock might need to be added. + +- concurrent `pop` calls are always safe to do with only the read lock held. + This does not fall under the previous rule since the `pop` call will select + the next item if the first is already being removed by another thread. + + **Deallocation locking still applies.** Assume another thread starts + reading the list, but gets task-switched by the kernel while reading the + first item. `pop` will happily remove and return that item. If it is + deallocated without acquiring and releasing the write lock, the other thread + will later resume execution and try to access the now-deleted element. + +- the list count should be considered an estimate. Since there might be + concurrent insertions or removals in progress, it might already be outdated + by the time the call returns. No attempt is made to have it be correct even + for a nanosecond. + +Overall, atomic lists are well-suited for MT queues; concurrent insertion, +iteration and removal operations will work with the read lock held. + +Code snippets +^^^^^^^^^^^^^ + +Iteration: + +.. code-block:: c + + struct item *i; + + pthread_rwlock_rdlock(&itemhead_rwlock); + frr_each(itemlist, &itemhead, i) { + /* lock must remain held while iterating */ + ... + } + pthread_rwlock_unlock(&itemhead_rwlock); + +Head removal (pop) and deallocation: + +.. code-block:: c + + struct item *i; + + pthread_rwlock_rdlock(&itemhead_rwlock); + i = itemlist_pop(&itemhead); + pthread_rwlock_unlock(&itemhead_rwlock); + + /* i might still be visible for another thread doing an + * frr_each() (but won't be returned by another pop()) */ + ... + + pthread_rwlock_wrlock(&itemhead_rwlock); + pthread_rwlock_unlock(&itemhead_rwlock); + /* i now guaranteed to be gone from the list. + * note nothing between wrlock() and unlock() */ + XFREE(MTYPE_ITEM, i); + +FAQ +--- + +What are the semantics of ``const`` in the container APIs? + ``const`` pointers to list heads and/or items are interpreted to mean that + both the container itself as well as the data items are read-only. + +Why is it ``PREDECL`` + ``DECLARE`` instead of ``DECLARE`` + ``DEFINE``? + The rule is that a ``DEFINE`` must be in a ``.c`` file, and linked exactly + once because it defines some kind of global symbol. This is not the case + for the data structure macros; they only define ``static`` symbols and it + is perfectly fine to include both ``PREDECL`` and ``DECLARE`` in a header + file. It is also perfectly fine to have the same ``DECLARE`` statement in + 2 ``.c`` files, but only **if the macro arguments are identical.** Maybe + don't do that unless you really need it. + +FRR lists +--------- + +.. TODO:: + + document + +BSD lists +--------- + +.. TODO:: + + refer to external docs diff --git a/doc/developer/locking.rst b/doc/developer/locking.rst new file mode 100644 index 0000000..bce1311 --- /dev/null +++ b/doc/developer/locking.rst @@ -0,0 +1,79 @@ +.. _locking: + +Locking +======= + +FRR ships two small wrappers around ``pthread_mutex_lock()`` / +``pthread_mutex_unlock``. Use ``#include "frr_pthread.h"`` to get these +macros. + +.. c:macro:: frr_with_mutex (mutex) + + (With ``pthread_mutex_t *mutex``.) + + Begin a C statement block that is executed with the mutex locked. Any + exit from the block (``break``, ``return``, ``goto``, end of block) will + cause the mutex to be unlocked:: + + int somefunction(int option) + { + frr_with_mutex (&my_mutex) { + /* mutex will be locked */ + + if (!option) + /* mutex will be unlocked before return */ + return -1; + + if (something(option)) + /* mutex will be unlocked before goto */ + goto out_err; + + somethingelse(); + + /* mutex will be unlocked at end of block */ + } + + return 0; + + out_err: + somecleanup(); + return -1; + } + + This is a macro that internally uses a ``for`` loop. It is explicitly + acceptable to use ``break`` to get out of the block. Even though a single + statement works correctly, FRR coding style requires that this macro always + be used with a ``{ ... }`` block. + +.. c:macro:: frr_mutex_lock_autounlock(mutex) + + (With ``pthread_mutex_t *mutex``.) + + Lock mutex and unlock at the end of the current C statement block:: + + int somefunction(int option) + { + frr_mutex_lock_autounlock(&my_mutex); + /* mutex will be locked */ + + ... + if (error) + /* mutex will be unlocked before return */ + return -1; + ... + + /* mutex will be unlocked before return */ + return 0; + } + + This is a macro that internally creates a variable with a destructor. + When the variable goes out of scope (i.e. the block ends), the mutex is + released. + + .. warning:: + + This macro should only used when :c:func:`frr_with_mutex` would + result in excessively/weirdly nested code. This generally is an + indicator that the code might be trying to do too many things with + the lock held. Try any possible venues to reduce the amount of + code covered by the lock and move to :c:func:`frr_with_mutex`. diff --git a/doc/developer/logging.rst b/doc/developer/logging.rst new file mode 100644 index 0000000..52653d3 --- /dev/null +++ b/doc/developer/logging.rst @@ -0,0 +1,873 @@ +.. _logging: + +.. highlight:: c + +Logging +======= + +One of the most frequent decisions to make while writing code for FRR is what +to log, what level to log it at, and when to log it. Here is a list of +recommendations for these decisions. + + +printfrr() +---------- + +``printfrr()`` is FRR's modified version of ``printf()``, designed to make +life easier when printing nontrivial datastructures. The following variants +are available: + +.. c:function:: ssize_t snprintfrr(char *buf, size_t len, const char *fmt, ...) +.. c:function:: ssize_t vsnprintfrr(char *buf, size_t len, const char *fmt, va_list) + + These correspond to ``snprintf``/``vsnprintf``. If you pass NULL for buf + or 0 for len, no output is written but the return value is still calculated. + + The return value is always the full length of the output, unconstrained by + `len`. It does **not** include the terminating ``\0`` character. A + malformed format string can result in a ``-1`` return value. + +.. c:function:: ssize_t csnprintfrr(char *buf, size_t len, const char *fmt, ...) +.. c:function:: ssize_t vcsnprintfrr(char *buf, size_t len, const char *fmt, va_list) + + Same as above, but the ``c`` stands for "continue" or "concatenate". The + output is appended to the string instead of overwriting it. + +.. c:function:: char *asprintfrr(struct memtype *mt, const char *fmt, ...) +.. c:function:: char *vasprintfrr(struct memtype *mt, const char *fmt, va_list) + + These functions allocate a dynamic buffer (using MTYPE `mt`) and print to + that. If the format string is malformed, they return a copy of the format + string, so the return value is always non-NULL and always dynamically + allocated with `mt`. + +.. c:function:: char *asnprintfrr(struct memtype *mt, char *buf, size_t len, const char *fmt, ...) +.. c:function:: char *vasnprintfrr(struct memtype *mt, char *buf, size_t len, const char *fmt, va_list) + + This variant tries to use the static buffer provided, but falls back to + dynamic allocation if it is insufficient. + + The return value can be either `buf` or a newly allocated string using + `mt`. You MUST free it like this:: + + char *ret = asnprintfrr(MTYPE_FOO, buf, sizeof(buf), ...); + if (ret != buf) + XFREE(MTYPE_FOO, ret); + +.. c:function:: ssize_t bprintfrr(struct fbuf *fb, const char *fmt, ...) +.. c:function:: ssize_t vbprintfrr(struct fbuf *fb, const char *fmt, va_list) + + These are the "lowest level" functions, which the other variants listed + above use to implement their functionality on top. Mainly useful for + implementing printfrr extensions since those get a ``struct fbuf *`` to + write their output to. + +.. c:macro:: FMT_NSTD(expr) + + This macro turns off/on format warnings as needed when non-ISO-C + compatible printfrr extensions are used (e.g. ``%.*p`` or ``%Ld``.):: + + vty_out(vty, "standard compatible %pI4\n", &addr); + FMT_NSTD(vty_out(vty, "non-standard %-47.*pHX\n", (int)len, buf)); + + When the frr-format plugin is in use, this macro is a no-op since the + frr-format plugin supports all printfrr extensions. Since the FRR CI + includes a system with the plugin enabled, this means format errors will + not slip by undetected even with FMT_NSTD. + +.. note:: + + ``printfrr()`` does not support the ``%n`` format. + +AS-Safety +^^^^^^^^^ + +``printfrr()`` are AS-Safe under the following conditions: + +* the ``[v]as[n]printfrr`` variants are not AS-Safe (allocating memory) +* floating point specifiers are not AS-Safe (system printf is used for these) +* the positional ``%1$d`` syntax should not be used (8 arguments are supported + while AS-Safe) +* extensions are only AS-Safe if their printer is AS-Safe + +printfrr Extensions +------------------- + +``printfrr()`` format strings can be extended with suffixes after `%p` or `%d`. +Printf features like field lengths can be used normally with these extensions, +e.g. ``%-15pI4`` works correctly, **except if the extension consumes the +width or precision**. Extensions that do so are listed below as ``%*pXX`` +rather than ``%pXX``. + +The extension specifier after ``%p`` or ``%d`` is always an uppercase letter; +by means of established pattern uppercase letters and numbers form the type +identifier which may be followed by lowercase flags. + +You can grep the FRR source for ``printfrr_ext_autoreg`` to see all extended +printers and what exactly they do. More printers are likely to be added as +needed/useful, so the list here may be outdated. + +.. note:: + + The ``zlog_*``/``flog_*`` and ``vty_out`` functions all use printfrr + internally, so these extensions are available there. However, they are + **not** available when calling ``snprintf`` directly. You need to call + ``snprintfrr`` instead. + +Networking data types +^^^^^^^^^^^^^^^^^^^^^ + +.. role:: frrfmtout(code) + +.. frrfmt:: %pI4 (struct in_addr *, in_addr_t *) + + :frrfmtout:`1.2.3.4` + + ``%pI4s``: :frrfmtout:`*` — print star instead of ``0.0.0.0`` (for multicast) + +.. frrfmt:: %pI6 (struct in6_addr *) + + :frrfmtout:`fe80::1234` + + ``%pI6s``: :frrfmtout:`*` — print star instead of ``::`` (for multicast) + +.. frrfmt:: %pEA (struct ethaddr *) + + :frrfmtout:`01:23:45:67:89:ab` + +.. frrfmt:: %pIA (struct ipaddr *) + + :frrfmtout:`1.2.3.4` / :frrfmtout:`fe80::1234` + + ``%pIAs``: — print star instead of zero address (for multicast) + +.. frrfmt:: %pFX (struct prefix *) + + :frrfmtout:`1.2.3.0/24` / :frrfmtout:`fe80::1234/64` + + This accepts the following types: + + - :c:struct:`prefix` + - :c:struct:`prefix_ipv4` + - :c:struct:`prefix_ipv6` + - :c:struct:`prefix_eth` + - :c:struct:`prefix_evpn` + - :c:struct:`prefix_fs` + + It does **not** accept the following types: + + - :c:struct:`prefix_ls` + - :c:struct:`prefix_rd` + - :c:struct:`prefix_sg` (use :frrfmt:`%pPSG4`) + - :c:union:`prefixptr` (dereference to get :c:struct:`prefix`) + - :c:union:`prefixconstptr` (dereference to get :c:struct:`prefix`) + + Options: + + ``%pFXh``: (address only) :frrfmtout:`1.2.3.0` / :frrfmtout:`fe80::1234` + +.. frrfmt:: %pPSG4 (struct prefix_sg *) + + :frrfmtout:`(*,1.2.3.4)` + + This is *(S,G)* output for use in zebra. (Note prefix_sg is not a prefix + "subclass" like the other prefix_* structs.) + +.. frrfmt:: %pSU (union sockunion *) + + ``%pSU``: :frrfmtout:`1.2.3.4` / :frrfmtout:`fe80::1234` + + ``%pSUs``: :frrfmtout:`1.2.3.4` / :frrfmtout:`fe80::1234%89` + (adds IPv6 scope ID as integer) + + ``%pSUp``: :frrfmtout:`1.2.3.4:567` / :frrfmtout:`[fe80::1234]:567` + (adds port) + + ``%pSUps``: :frrfmtout:`1.2.3.4:567` / :frrfmtout:`[fe80::1234%89]:567` + (adds port and scope ID) + +.. frrfmt:: %pRN (struct route_node *, struct bgp_node *, struct agg_node *) + + :frrfmtout:`192.168.1.0/24` (dst-only node) + + :frrfmtout:`2001:db8::/32 from fe80::/64` (SADR node) + +.. frrfmt:: %pNH (struct nexthop *) + + ``%pNHvv``: :frrfmtout:`via 1.2.3.4, eth0` — verbose zebra format + + ``%pNHv``: :frrfmtout:`1.2.3.4, via eth0` — slightly less verbose zebra format + + ``%pNHs``: :frrfmtout:`1.2.3.4 if 15` — same as :c:func:`nexthop2str()` + + ``%pNHcg``: :frrfmtout:`1.2.3.4` — compact gateway only + + ``%pNHci``: :frrfmtout:`eth0` — compact interface only + +.. frrfmt:: %dPF (int) + + :frrfmtout:`AF_INET` + + Prints an `AF_*` / `PF_*` constant. ``PF`` is used here to avoid confusion + with `AFI` constants, even though the FRR codebase prefers `AF_INET` over + `PF_INET` & co. + +.. frrfmt:: %dSO (int) + + :frrfmtout:`SOCK_STREAM` + +Time/interval formats +^^^^^^^^^^^^^^^^^^^^^ + +.. frrfmt:: %pTS (struct timespec *) + +.. frrfmt:: %pTV (struct timeval *) + +.. frrfmt:: %pTT (time_t *) + + Above 3 options internally result in the same code being called, support + the same flags and produce equal output with one exception: ``%pTT`` + has no sub-second precision and the formatter will never print a + (nonsensical) ``.000``. + + Exactly one of ``I``, ``M`` or ``R`` must immediately follow after + ``TS``/``TV``/``TT`` to specify whether the input is an interval, monotonic + timestamp or realtime timestamp: + + ``%pTVI``: input is an interval, not a timestamp. Print interval. + + ``%pTVIs``: input is an interval, convert to wallclock by subtracting it + from current time (i.e. interval has passed **s**\ ince.) + + ``%pTVIu``: input is an interval, convert to wallclock by adding it to + current time (i.e. **u**\ ntil interval has passed.) + + ``%pTVM`` - input is a timestamp on CLOCK_MONOTONIC, convert to wallclock + time (by grabbing current CLOCK_MONOTONIC and CLOCK_REALTIME and doing the + math) and print calendaric date. + + ``%pTVMs`` - input is a timestamp on CLOCK_MONOTONIC, print interval + **s**\ ince that timestamp (elapsed.) + + ``%pTVMu`` - input is a timestamp on CLOCK_MONOTONIC, print interval + **u**\ ntil that timestamp (deadline.) + + ``%pTVR`` - input is a timestamp on CLOCK_REALTIME, print calendaric date. + + ``%pTVRs`` - input is a timestamp on CLOCK_REALTIME, print interval + **s**\ ince that timestamp. + + ``%pTVRu`` - input is a timestamp on CLOCK_REALTIME, print interval + **u**\ ntil that timestamp. + + ``%pTVA`` - reserved for CLOCK_TAI in case a PTP implementation is + interfaced to FRR. Not currently implemented. + + .. note:: + + If ``%pTVRs`` or ``%pTVRu`` are used, this is generally an indication + that a CLOCK_MONOTONIC timestamp should be used instead (or added in + parallel.) CLOCK_REALTIME might be adjusted by NTP, PTP or similar + procedures, causing bogus intervals to be printed. + + ``%pTVM`` on first look might be assumed to have the same problem, but + on closer thought the assumption is always that current system time is + correct. And since a CLOCK_MONOTONIC interval is also quite safe to + assume to be correct, the (past) absolute timestamp to be printed from + this can likely be correct even if it doesn't match what CLOCK_REALTIME + would have indicated at that point in the past. This logic does, + however, not quite work for *future* times. + + Generally speaking, almost all use cases in FRR should (and do) use + CLOCK_MONOTONIC (through :c:func:`monotime()`.) + + Flags common to printing calendar times and intervals: + + ``p``: include spaces in appropriate places (depends on selected format.) + + ``%p.3TV...``: specify sub-second resolution (use with ``FMT_NSTD`` to + suppress gcc warning.) As noted above, ``%pTT`` will never print sub-second + digits since there are none. Only some formats support printing sub-second + digits and the default may vary. + + The following flags are available for printing calendar times/dates: + + (no flag): :frrfmtout:`Sat Jan 1 00:00:00 2022` - print output from + ``ctime()``, in local time zone. Since FRR does not currently use/enable + locale support, this is always the C locale. (Locale support getting added + is unlikely for the time being and would likely break other things worse + than this.) + + ``i``: :frrfmtout:`2022-01-01T00:00:00.123` - ISO8601 timestamp in local + time zone (note there is no ``Z`` or ``+00:00`` suffix.) Defaults to + millisecond precision. + + ``ip``: :frrfmtout:`2022-01-01 00:00:00.123` - use readable form of ISO8601 + with space instead of ``T`` separator. + + The following flags are available for printing intervals: + + (no flag): :frrfmtout:`9w9d09:09:09.123` - does not match any + preexisting format; added because it does not lose precision (like ``t``) + for longer intervals without printing huge numbers (like ``h``/``m``). + Defaults to millisecond precision. The week/day fields are left off if + they're zero, ``p`` adds a space after the respective letter. + + ``t``: :frrfmtout:`9w9d09h`, :frrfmtout:`9d09h09m`, :frrfmtout:`09:09:09` - + this replaces :c:func:`frrtime_to_interval()`. ``p`` adds spaces after + week/day/hour letters. + + ``d``: print decimal number of seconds. Defaults to millisecond precision. + + ``x`` / ``tx`` / ``dx``: Like no flag / ``t`` / ``d``, but print + :frrfmtout:`-` for zero or negative intervals (for use with unset timers.) + + ``h``: :frrfmtout:`09:09:09` + + ``hx``: :frrfmtout:`09:09:09`, :frrfmtout:`--:--:--` - this replaces + :c:func:`pim_time_timer_to_hhmmss()`. + + ``m``: :frrfmtout:`09:09` + + ``mx``: :frrfmtout:`09:09`, :frrfmtout:`--:--` - this replaces + :c:func:`pim_time_timer_to_mmss()`. + +FRR library helper formats +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. frrfmt:: %pTH (struct event *) + + Print remaining time on timer event. Interval-printing flag characters + listed above for ``%pTV`` can be added, e.g. ``%pTHtx``. + + ``NULL`` pointers are printed as ``-``. + +.. frrfmt:: %pTHD (struct event *) + + Print debugging information for given event. Sample output: + + .. code-block:: none + + {(thread *)NULL} + {(thread *)0x55a3b5818910 arg=0x55a3b5827c50 timer r=7.824 mld_t_query() &mld_ifp->t_query from pimd/pim6_mld.c:1369} + {(thread *)0x55a3b5827230 arg=0x55a3b5827c50 read fd=16 mld_t_recv() &mld_ifp->t_recv from pimd/pim6_mld.c:1186} + + (The output is aligned to some degree.) + +FRR daemon specific formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following formats are only available in specific daemons, as the code +implementing them is part of the daemon, not the library. + +zebra +""""" + +.. frrfmt:: %pZN (struct route_node *) + + Print information for a RIB node, including zebra-specific data. + + :frrfmtout:`::/0 src fe80::/64 (MRIB)` (``%pZN``) + + :frrfmtout:`1234` (``%pZNt`` - table number) + +bgpd +"""" + +.. frrfmt:: %pBD (struct bgp_dest *) + + Print prefix for a BGP destination. + + :frrfmtout:`fe80::1234/64` + +.. frrfmt:: %pBP (struct peer *) + + :frrfmtout:`192.168.1.1(leaf1.frrouting.org)` + + Print BGP peer's IP and hostname together. + +pimd/pim6d +"""""""""" + +.. frrfmt:: %pPA (pim_addr *) + + Format IP address according to IP version (pimd vs. pim6d) being compiled. + + :frrfmtout:`fe80::1234` / :frrfmtout:`10.0.0.1` + + :frrfmtout:`*` (``%pPAs`` - replace 0.0.0.0/:: with star) + +.. frrfmt:: %pSG (pim_sgaddr *) + + Format S,G pair according to IP version (pimd vs. pim6d) being compiled. + Braces are included. + + :frrfmtout:`(*,224.0.0.0)` + + +General utility formats +^^^^^^^^^^^^^^^^^^^^^^^ + +.. frrfmt:: %m (no argument) + + :frrfmtout:`Permission denied` + + Prints ``strerror(errno)``. Does **not** consume any input argument, don't + pass ``errno``! + + (This is a GNU extension not specific to FRR. FRR guarantees it is + available on all systems in printfrr, though BSDs support it in printf too.) + +.. frrfmt:: %pSQ (char *) + + ([S]tring [Q]uote.) Like ``%s``, but produce a quoted string. Options: + + ``n`` - treat ``NULL`` as empty string instead. + + ``q`` - include ``""`` quotation marks. Note: ``NULL`` is printed as + ``(null)``, not ``"(null)"`` unless ``n`` is used too. This is + intentional. + + ``s`` - use escaping suitable for RFC5424 syslog. This means ``]`` is + escaped too. + + If a length is specified (``%*pSQ`` or ``%.*pSQ``), null bytes in the input + string do not end the string and are just printed as ``\x00``. + +.. frrfmt:: %pSE (char *) + + ([S]tring [E]scape.) Like ``%s``, but escape special characters. + Options: + + ``n`` - treat ``NULL`` as empty string instead. + + Unlike :frrfmt:`%pSQ`, this escapes many more characters that are fine for + a quoted string but not on their own. + + If a length is specified (``%*pSE`` or ``%.*pSE``), null bytes in the input + string do not end the string and are just printed as ``\x00``. + +.. frrfmt:: %pVA (struct va_format *) + + Recursively invoke printfrr, with arguments passed in through: + + .. c:struct:: va_format + + .. c:member:: const char *fmt + + Format string to use for the recursive printfrr call. + + .. c:member:: va_list *va + + Formatting arguments. Note this is passed as a pointer, not - as in + most other places - a direct struct reference. Internally uses + ``va_copy()`` so repeated calls can be made (e.g. for determining + output length.) + +.. frrfmt:: %pFB (struct fbuf *) + + Insert text from a ``struct fbuf *``, i.e. the output of a call to + :c:func:`bprintfrr()`. + +.. frrfmt:: %*pHX (void *, char *, unsigned char *) + + ``%pHX``: :frrfmtout:`12 34 56 78` + + ``%pHXc``: :frrfmtout:`12:34:56:78` (separate with [c]olon) + + ``%pHXn``: :frrfmtout:`12345678` (separate with [n]othing) + + Insert hexdump. This specifier requires a precision or width to be + specified. A precision (``%.*pHX``) takes precedence, but generates a + compiler warning since precisions are undefined for ``%p`` in ISO C. If + no precision is given, the width is used instead (and normal handling of + the width is suppressed). + + Note that width and precision are ``int`` arguments, not ``size_t``. Use + like:: + + char *buf; + size_t len; + + snprintfrr(out, sizeof(out), "... %*pHX ...", (int)len, buf); + + /* with padding to width - would generate a warning due to %.*p */ + FMT_NSTD(snprintfrr(out, sizeof(out), "... %-47.*pHX ...", (int)len, buf)); + +.. frrfmt:: %*pHS (void *, char *, unsigned char *) + + ``%pHS``: :frrfmtout:`hex.dump` + + This is a complementary format for :frrfmt:`%*pHX` to print the text + representation for a hexdump. Non-printable characters are replaced with + a dot. + +.. frrfmt:: %pIS (struct iso_address *) + + ([IS]o Network address) - Format ISO Network Address + + ``%pIS``: :frrfmtout:`01.0203.04O5` + ISO Network address is printed as separated byte. The number of byte of the + address is embeded in the `iso_net` structure. + + ``%pISl``: :frrfmtout:`01.0203.04O5.0607.0809.1011.1213.14` - long format to + print the long version of the ISO Network address which include the System + ID and the PSEUDO-ID of the IS-IS system + + Note that the `ISO_ADDR_STRLEN` define gives the total size of the string + that could be used in conjunction to snprintfrr. Use like:: + + char buf[ISO_ADDR_STRLEN]; + struct iso_address addr = {.addr_len = 4, .area_addr = {1, 2, 3, 4}}; + snprintfrr(buf, ISO_ADDR_STRLEN, "%pIS", &addr); + +.. frrfmt:: %pSY (uint8_t *) + + (IS-IS [SY]stem ID) - Format IS-IS System ID + + ``%pSY``: :frrfmtout:`0102.0304.0506` + +.. frrfmt:: %pPN (uint8_t *) + + (IS-IS [P]seudo [N]ode System ID) - Format IS-IS Pseudo Node System ID + + ``%pPN``: :frrfmtout:`0102.0304.0506.07` + +.. frrfmt:: %pLS (uint8_t *) + + (IS-IS [L]sp fragment [S]ystem ID) - Format IS-IS Pseudo System ID + + ``%pLS``: :frrfmtout:`0102.0304.0506.07-08` + + Note that the `ISO_SYSID_STRLEN` define gives the total size of the string + that could be used in conjunction to snprintfrr. Use like:: + + char buf[ISO_SYSID_STRLEN]; + uint8_t id[8] = {1, 2, 3, 4 , 5 , 6 , 7, 8}; + snprintfrr(buf, SYS_ID_SIZE, "%pSY", id); + + +Integer formats +^^^^^^^^^^^^^^^ + +.. note:: + + These formats currently only exist for advanced type checking with the + ``frr-format`` GCC plugin. They should not be used directly since they will + cause compiler warnings when used without the plugin. Use with + :c:macro:`FMT_NSTD` if necessary. + + It is possible ISO C23 may introduce another format for these, possibly + ``%w64d`` discussed in `JTC 1/SC 22/WG 14/N2680 <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2680.pdf>`_. + +.. frrfmt:: %Lu (uint64_t) + + :frrfmtout:`12345` + +.. frrfmt:: %Ld (int64_t) + + :frrfmtout:`-12345` + +Log levels +---------- + +Errors and warnings +^^^^^^^^^^^^^^^^^^^ + +If it is something that the user will want to look at and maybe do +something, it is either an **error** or a **warning**. + +We're expecting that warnings and errors are in some way visible to the +user (in the worst case by looking at the log after the network broke, but +maybe by a syslog collector from all routers.) Therefore, anything that +needs to get the user in the loop—and only these things—are warnings or +errors. + +Note that this doesn't necessarily mean the user needs to fix something in +the FRR instance. It also includes when we detect something else needs +fixing, for example another router, the system we're running on, or the +configuration. The common point is that the user should probably do +*something*. + +Deciding between a warning and an error is slightly less obvious; the rule +of thumb here is that an error will cause considerable fallout beyond its +direct effect. Closing a BGP session due to a malformed update is an error +since all routes from the peer are dropped; discarding one route because +its attributes don't make sense is a warning. + +This also loosely corresponds to the kind of reaction we're expecting from +the user. An error is likely to need immediate response while a warning +might be snoozed for a bit and addressed as part of general maintenance. +If a problem will self-repair (e.g. by retransmits), it should be a +warning—unless the impact until that self-repair is very harsh. + +Examples for warnings: + +* a BGP update, LSA or LSP could not be processed, but operation is + proceeding and the broken pieces are likely to self-fix later +* some kind of controller cannot be reached, but we can work without it +* another router is using some unknown or unsupported capability + +Examples for errors: + +* dropping a BGP session due to malformed data +* a socket for routing protocol operation cannot be opened +* desynchronization from network state because something went wrong +* *everything that we as developers would really like to be notified about, + i.e. some assumption in the code isn't holding up* + + +Informational messages +^^^^^^^^^^^^^^^^^^^^^^ + +Anything that provides introspection to the user during normal operation +is an **info** message. + +This includes all kinds of operational state transitions and events, +especially if they might be interesting to the user during the course of +figuring out a warning or an error. + +By itself, these messages should mostly be statements of fact. They might +indicate the order and relationship in which things happened. Also covered +are conditions that might be "operational issues" like a link failure due +to an unplugged cable. If it's pretty much the point of running a routing +daemon for, it's not a warning or an error, just business as usual. + +The user should be able to see the state of these bits from operational +state output, i.e. `show interface` or `show foobar neighbors`. The log +message indicating the change may have been printed weeks ago, but the +state can always be viewed. (If some state change has an info message but +no "show" command, maybe that command needs to be added.) + +Examples: + +* all kinds of up/down state changes + + * interface coming up or going down + * addresses being added or deleted + * peers and neighbors coming up or going down + +* rejection of some routes due to user-configured route maps +* backwards compatibility handling because another system on the network + has a different or smaller feature set + +.. note:: + The previously used **notify** priority is replaced with *info* in all + cases. We don't currently have a well-defined use case for it. + + +Debug messages and asserts +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Everything that is only interesting on-demand, or only while developing, +is a **debug** message. It might be interesting to the user for a +particularly evasive issue, but in general these are details that an +average user might not even be able to make sense of. + +Most (or all?) debug messages should be behind a `debug foobar` category +switch that controls which subset of these messages is currently +interesting and thus printed. If a debug message doesn't have such a +guard, there should be a good explanation as to why. + +Conversely, debug messages are the only thing that should be guarded by +these switches. Neither info nor warning or error messages should be +hidden in this way. + +**Asserts** should only be used as pretty crashes. We are expecting that +asserts remain enabled in production builds, but please try to not use +asserts in a way that would cause a security problem if the assert wasn't +there (i.e. don't use them for length checks.) + +The purpose of asserts is mainly to help development and bug hunting. If +the daemon crashes, then having some more information is nice, and the +assert can provide crucial hints that cut down on the time needed to track +an issue. That said, if the issue can be reasonably handled and/or isn't +going to crash the daemon, it shouldn't be an assert. + +For anything else where internal constraints are violated but we're not +breaking due to it, it's an error instead (not a debug.) These require +"user action" of notifying the developers. + +Examples: + +* mismatched :code:`prev`/:code:`next` pointers in lists +* some field that is absolutely needed is :code:`NULL` +* any other kind of data structure corruption that will cause the daemon + to crash sooner or later, one way or another + +Thread-local buffering +---------------------- + +The core logging code in :file:`lib/zlog.c` allows setting up per-thread log +message buffers in order to improve logging performance. The following rules +apply for this buffering: + +* Only messages of priority *DEBUG* or *INFO* are buffered. +* Any higher-priority message causes the thread's entire buffer to be flushed, + thus message ordering is preserved on a per-thread level. +* There is no guarantee on ordering between different threads; in most cases + this is arbitrary to begin with since the threads essentially race each + other in printing log messages. If an order is established with some + synchronization primitive, add calls to :c:func:`zlog_tls_buffer_flush()`. +* The buffers are only ever accessed by the thread they are created by. This + means no locking is necessary. + +Both the main/default thread and additional threads created by +:c:func:`frr_pthread_new()` with the default :c:func:`frr_run()` handler will +initialize thread-local buffering and call :c:func:`zlog_tls_buffer_flush()` +when idle. + +If some piece of code runs for an extended period, it may be useful to insert +calls to :c:func:`zlog_tls_buffer_flush()` in appropriate places: + +.. c:function:: void zlog_tls_buffer_flush(void) + + Write out any pending log messages that the calling thread may have in its + buffer. This function is safe to call regardless of the per-thread log + buffer being set up / in use or not. + +When working with threads that do not use the :c:struct:`thread_master` +event loop, per-thread buffers can be managed with: + +.. c:function:: void zlog_tls_buffer_init(void) + + Set up thread-local buffering for log messages. This function may be + called repeatedly without adverse effects, but remember to call + :c:func:`zlog_tls_buffer_fini()` at thread exit. + + .. warning:: + + If this function is called, but :c:func:`zlog_tls_buffer_flush()` is + not used, log message output will lag behind since messages will only be + written out when the buffer is full. + + Exiting the thread without calling :c:func:`zlog_tls_buffer_fini()` + will cause buffered log messages to be lost. + +.. c:function:: void zlog_tls_buffer_fini(void) + + Flush pending messages and tear down thread-local log message buffering. + This function may be called repeatedly regardless of whether + :c:func:`zlog_tls_buffer_init()` was ever called. + +Log targets +----------- + +The actual logging subsystem (in :file:`lib/zlog.c`) is heavily separated +from the actual log writers. It uses an atomic linked-list (`zlog_targets`) +with RCU to maintain the log targets to be called. This list is intended to +function as "backend" only, it **is not used for configuration**. + +Logging targets provide their configuration layer on top of this and maintain +their own capability to enumerate and store their configuration. Some targets +(e.g. syslog) are inherently single instance and just stuff their config in +global variables. Others (e.g. file/fd output) are multi-instance capable. +There is another layer boundary here between these and the VTY configuration +that they use. + +Basic internals +^^^^^^^^^^^^^^^ + +.. c:struct:: zlog_target + + This struct needs to be filled in by any log target and then passed to + :c:func:`zlog_target_replace()`. After it has been registered, + **RCU semantics apply**. Most changes to associated data should make a + copy, change that, and then replace the entire struct. + + Additional per-target data should be "appended" by embedding this struct + into a larger one, for use with `containerof()`, and + :c:func:`zlog_target_clone()` and :c:func:`zlog_target_free()` should be + used to allocate/free the entire container struct. + + Do not use this structure to maintain configuration. It should only + contain (a copy of) the data needed to perform the actual logging. For + example, the syslog target uses this: + + .. code-block:: c + + struct zlt_syslog { + struct zlog_target zt; + int syslog_facility; + }; + + static void zlog_syslog(struct zlog_target *zt, struct zlog_msg *msgs[], size_t nmsgs) + { + struct zlt_syslog *zte = container_of(zt, struct zlt_syslog, zt); + size_t i; + + for (i = 0; i < nmsgs; i++) + if (zlog_msg_prio(msgs[i]) <= zt->prio_min) + syslog(zlog_msg_prio(msgs[i]) | zte->syslog_facility, "%s", + zlog_msg_text(msgs[i], NULL)); + } + + +.. c:function:: struct zlog_target *zlog_target_clone(struct memtype *mt, struct zlog_target *oldzt, size_t size) + + Allocates a logging target struct. Note that the ``oldzt`` argument may be + ``NULL`` to allocate a "from scratch". If ``oldzt`` is not ``NULL``, the + generic bits in :c:struct:`zlog_target` are copied. **Target specific + bits are not copied.** + +.. c:function:: struct zlog_target *zlog_target_replace(struct zlog_target *oldzt, struct zlog_target *newzt) + + Adds, replaces or deletes a logging target (either ``oldzt`` or ``newzt`` may be ``NULL``.) + + Returns ``oldzt`` for freeing. The target remains possibly in use by + other threads until the RCU cycle ends. This implies you cannot release + resources (e.g. memory, file descriptors) immediately. + + The replace operation is not atomic; for a brief period it is possible that + messages are delivered on both ``oldzt`` and ``newzt``. + + .. warning:: + + ``oldzt`` must remain **functional** until the RCU cycle ends. + +.. c:function:: void zlog_target_free(struct memtype *mt, struct zlog_target *zt) + + Counterpart to :c:func:`zlog_target_clone()`, frees a target (using RCU.) + +.. c:member:: void (*zlog_target.logfn)(struct zlog_target *zt, struct zlog_msg *msgs[], size_t nmsg) + + Called on a target to deliver "normal" logging messages. ``msgs`` is an + array of opaque structs containing the actual message. Use ``zlog_msg_*`` + functions to access message data (this is done to allow some optimizations, + e.g. lazy formatting the message text and timestamp as needed.) + + .. note:: + + ``logfn()`` must check each individual message's priority value against + the configured ``prio_min``. While the ``prio_min`` field is common to + all targets and used by the core logging code to early-drop unneeded log + messages, the array is **not** filtered for each ``logfn()`` call. + +.. c:member:: void (*zlog_target.logfn_sigsafe)(struct zlog_target *zt, const char *text, size_t len) + + Called to deliver "exception" logging messages (i.e. SEGV messages.) + Must be Async-Signal-Safe (may not allocate memory or call "complicated" + libc functions.) May be ``NULL`` if the log target cannot handle this. + +Standard targets +^^^^^^^^^^^^^^^^ + +:file:`lib/zlog_targets.c` provides the standard file / fd / syslog targets. +The syslog target is single-instance while file / fd targets can be +instantiated as needed. There are 3 built-in targets that are fully +autonomous without any config: + +- startup logging to `stderr`, until either :c:func:`zlog_startup_end()` or + :c:func:`zlog_aux_init()` is called. +- stdout logging for non-daemon programs using :c:func:`zlog_aux_init()` +- crashlogs written to :file:`/var/tmp/frr.daemon.crashlog` + +The regular CLI/command-line logging setup is handled by :file:`lib/log_vty.c` +which makes the appropriate instantiations of syslog / file / fd targets. + +.. todo:: + + :c:func:`zlog_startup_end()` should do an explicit switchover from + startup stderr logging to configured logging. Currently, configured logging + starts in parallel as soon as the respective setup is executed. This results + in some duplicate logging. diff --git a/doc/developer/memtypes.rst b/doc/developer/memtypes.rst new file mode 100644 index 0000000..2e181c4 --- /dev/null +++ b/doc/developer/memtypes.rst @@ -0,0 +1,140 @@ +.. highlight:: c + +Memtypes +======== + +FRR includes wrappers around ``malloc()`` and ``free()`` that count the number +of objects currently allocated, for each of a defined ``MTYPE``. + +To this extent, there are *memory groups* and *memory types*. Each memory +type must belong to a memory group, this is used just to provide some basic +structure. + +Example: + +.. code-block:: c + :caption: mydaemon.h + + DECLARE_MGROUP(MYDAEMON); + DECLARE_MTYPE(MYNEIGHBOR); + +.. code-block:: c + :caption: mydaemon.c + + DEFINE_MGROUP( MYDAEMON, "My daemon's memory"); + DEFINE_MTYPE( MYDAEMON, MYNEIGHBOR, "Neighbor entry"); + DEFINE_MTYPE_STATIC(MYDAEMON, MYNEIGHBORNAME, "Neighbor name"); + + struct neigh *neighbor_new(const char *name) + { + struct neigh *n = XMALLOC(MYNEIGHBOR, sizeof(*n)); + n->name = XSTRDUP(MYNEIGHBORNAME, name); + return n; + } + + void neighbor_free(struct neigh *n) + { + XFREE(MYNEIGHBORNAME, n->name); + XFREE(MYNEIGHBOR, n); + } + + +Definition +---------- + +.. c:struct:: memtype + + This is the (internal) type used for MTYPE definitions. The macros below + should be used to create these, but in some cases it is useful to pass a + ``struct memtype *`` pointer to some helper function. + + The ``MTYPE_name`` created by the macros is declared as a pointer, i.e. + a function taking a ``struct memtype *`` argument can be called with an + ``MTYPE_name`` argument (as opposed to ``&MTYPE_name``.) + + .. note:: + + As ``MTYPE_name`` is a variable assigned from ``&_mt_name`` and not a + constant expression, it cannot be used as initializer for static + variables. In the case please fall back to ``&_mt_name``. + +.. c:macro:: DECLARE_MGROUP(name) + + This macro forward-declares a memory group and should be placed in a + ``.h`` file. It expands to an ``extern struct memgroup`` statement. + +.. c:macro:: DEFINE_MGROUP(mname, description) + + Defines/implements a memory group. Must be placed into exactly one ``.c`` + file (multiple inclusion will result in a link-time symbol conflict). + + Contains additional logic (constructor and destructor) to register the + memory group in a global list. + +.. c:macro:: DECLARE_MTYPE(name) + + Forward-declares a memory type and makes ``MTYPE_name`` available for use. + Note that the ``MTYPE_`` prefix must not be included in the name, it is + automatically prefixed. + + ``MTYPE_name`` is created as a `static const` symbol, i.e. a compile-time + constant. It refers to an ``extern struct memtype _mt_name``, where `name` + is replaced with the actual name. + +.. c:macro:: DEFINE_MTYPE(group, name, description) + + Define/implement a memory type, must be placed into exactly one ``.c`` + file (multiple inclusion will result in a link-time symbol conflict). + + Like ``DEFINE_MGROUP``, this contains actual code to register the MTYPE + under its group. + +.. c:macro:: DEFINE_MTYPE_STATIC(group, name, description) + + Same as ``DEFINE_MTYPE``, but the ``DEFINE_MTYPE_STATIC`` variant places + the C ``static`` keyword on the definition, restricting the MTYPE's + availability to the current source file. This should be appropriate in + >80% of cases. + + .. todo:: + + Daemons currently have ``daemon_memory.[ch]`` files listing all of + their MTYPEs. This is not how it should be, most of these types + should be moved into the appropriate files where they are used. + Only a few MTYPEs should remain non-static after that. + + +Usage +----- + +.. c:function:: void *XMALLOC(struct memtype *mtype, size_t size) + +.. c:function:: void *XCALLOC(struct memtype *mtype, size_t size) + +.. c:function:: void *XSTRDUP(struct memtype *mtype, const char *name) + + Allocation wrappers for malloc/calloc/realloc/strdup, taking an extra + mtype parameter. + +.. c:function:: void *XREALLOC(struct memtype *mtype, void *ptr, size_t size) + + Wrapper around realloc() with MTYPE tracking. Note that ``ptr`` may + be NULL, in which case the function does the same as XMALLOC (regardless + of whether the system realloc() supports this.) + +.. c:function:: void XFREE(struct memtype *mtype, void *ptr) + + Wrapper around free(), again taking an extra mtype parameter. This is + actually a macro, with the following additional properties: + + - the macro contains ``ptr = NULL`` + - if ptr is NULL, no operation is performed (as is guaranteed by system + implementations.) Do not surround XFREE with ``if (ptr != NULL)`` + checks. + +.. c:function:: void XCOUNTFREE(struct memtype *mtype, void *ptr) + + This macro is used to count the ``ptr`` as freed without actually freeing + it. This may be needed in some very specific cases, for example, when the + ``ptr`` was allocated using any of the above wrappers and will be freed + by some external library using simple ``free()``. diff --git a/doc/developer/mgmtd-dev.rst b/doc/developer/mgmtd-dev.rst new file mode 100644 index 0000000..9839aa8 --- /dev/null +++ b/doc/developer/mgmtd-dev.rst @@ -0,0 +1,222 @@ +.. +.. SPDX-License-Identifier: GPL-2.0-or-later +.. +.. June 19 2023, Christian Hopps <chopps@labn.net> +.. +.. Copyright (c) 2023, LabN Consulting, L.L.C. +.. + +.. _mgmtd_dev: + +MGMTD Development +================= + +Overview +^^^^^^^^ + +``mgmtd`` (Management Daemon) is a new centralized management daemon for FRR. + +Previously, ``vtysh`` was the only centralized management service provided. +Internally ``vtysh`` connects to each daemon and sends CLI commands (both +configuration and operational state queries) over a socket connection. This +service only supports CLI which is no longer sufficient. + +An important next step was made with the addition of YANG support. A YANG +infrastructure was added through a new development called *northbound*. This +*northbound* interface added the capability of daemons to be configured and +queried using YANG models. However, this interface was per daemon and not +centralized, which is not sufficient. + +``mgmtd`` harnesses this new *northbound* interface to provide a centralized +interface for all daemons. It utilizes the daemons YANG models to interact with +each daemon. ``mgmtd`` currently provides the CLI interface for each daemon that +has been converted to it, but in the future RESTCONF and NETCONF servers can +easily be added as *front-ends* to mgmtd to support those protocols as well. + + +Converting A Daemon to MGMTD +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A daemon must first be transitioned to the new *northbound* interface if that +has not already been done (see `this northbound conversion documentation +<https://github.com/opensourcerouting/frr/wiki/Retrofitting-Configuration-Commands>`_ +for how to do this). Once this is done a few simple steps are all that is +required move the daemon over to ``mgmtd`` control. + +Overview of Changes +------------------- + +Adding support for a *northbound* converted daemon involves very little work. It +requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support. +``mgmtd`` was designed to keep this as simple as possible. + +Front-End Interface: + +1. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.c``) +2. Add YANG module description into array defined in ``mgmtd/mgmt_main.c`` +3. Add CLI handler file[s] to ``mgmtd/subdir.am`` (e.g., ``staticd/static_vty.c``) +4. [if needed] Exclude (#ifndef) non-configuration CLI handlers from CLI source + file (e.g., inside ``staticd/static_vty.c``) + +Back-End Interface: + +5. Add XPATHs mappings to a couple arrays to direct ``mgmtd`` at your daemon in + ``mgmtd/mgmt_be_adapter.c`` + + +Add YANG and CLI into MGMTD +--------------------------- + +As an example here is the addition made to ``mgmtd/subdir.am`` for adding +``staticd`` support. + +.. code-block:: make + + if STATICD + nodist_mgmtd_mgmtd_SOURCES += \ + yang/frr-staticd.yang.c \ + yang/frr-bfdd.yang.c \ + # end + nodist_mgmtd_libmgmt_be_nb_la_SOURCES += staticd/static_vty.c + endif + +An here is the addition to the modules array in ``mgmtd/mgmt_main.c``: + +.. code-block:: c + + static const struct frr_yang_module_info *const mgmt_yang_modules[] = { + &frr_filter_info, + ... + #ifdef HAVE_STATICD + &(struct frr_yang_module_info){.name = "frr-staticd", + .ignore_cbs = true}, + #endif + } + + +CLI Handlers +------------ + +The daemon's CLI handlers for configuration (which having been converted to +*northbound* now simply generate YANG changes) will be linked directly into +``mgmtd``. + +If the operational and debug CLI commands are kept in files separate from the +daemon's configuration CLI commands then no extra work is required. Otherwise some +CPP #ifndef's will be required. + +Currently ``mgmtd`` supports configuration CLI but not operational +state so if both types of CLI handlers are present in a single file (e.g. a +``xxx_vty.c`` or ``xxx_cli.c`` file ) then #ifndef will be used to exclude these +non-configuration CLI handlers from ``mgmtd``. The same goes for *debug* CLI +handlers. For example: + +.. code-block:: c + + DEFPY(daemon_one_config, daemon_one_config_cmd, + "daemon one [optional-arg]" + ... + { + ... + } + + #ifndef INCLUDE_MGMTD_CMDDEFS_ONLY + DEFPY(daemon_show_oepr, daemon_show_oepr_cmd, + "show daemon oper [all]" + ... + { + ... + } + #endif /* ifndef INCLUDE_MGMTD_CMDDEFS_ONLY */ + + void daemon_vty_init(void) + { + install_element(CONFIG_NODE, &daemon_one_config_cmd); + ... + + #ifndef INCLUDE_MGMTD_CMDDEFS_ONLY + install_element(ENABLE_NODE, &daemon_show_oper_cmd); + #endif /* ifndef INCLUDE_MGMTD_CMDDEFS_ONLY */ + + } + + +Add Back-End XPATH mappings +--------------------------- + +In order for ``mgmtd`` to direct configuration to your daemon you need to add +some XPATH mappings to ``mgmtd/mgmt_be_adapter.c``. These XPATHs determine which +configuration changes get sent over the *back-end* interface to your daemon. + +Below are the strings added for staticd support: + +.. code-block:: c + + static const struct mgmt_be_xpath_map_init mgmt_xpath_map_init[] = { + { + .xpath_regexp = "/frr-vrf:lib/*", + .subscr_info = + { + #if HAVE_STATICD + [MGMTD_BE_CLIENT_ID_STATICD] = + MGMT_SUBSCR_VALIDATE_CFG | + MGMT_SUBSCR_NOTIFY_CFG, + #endif + }, + }, + ... + { + .xpath_regexp = + "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*", + .subscr_info = + { + #if HAVE_STATICD + [MGMTD_BE_CLIENT_ID_STATICD] = + MGMT_SUBSCR_VALIDATE_CFG | + MGMT_SUBSCR_NOTIFY_CFG, + #endif + }, + }, + }; + + #if HAVE_STATICD + static struct mgmt_be_client_xpath staticd_xpaths[] = { + { + .xpath = "/frr-vrf:lib/*", + .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG, + }, + ... + { + .xpath = + "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*", + .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG, + }, + }; + #endif + + static struct mgmt_be_client_xpath_map + mgmt_client_xpaths[MGMTD_BE_CLIENT_ID_MAX] = { + #ifdef HAVE_STATICD + [MGMTD_BE_CLIENT_ID_STATICD] = {staticd_xpaths, + array_size(staticd_xpaths)}, + #endif + }; + + +MGMTD Internals +^^^^^^^^^^^^^^^ + +This section will describe the internal functioning of ``mgmtd``, for now a +couple diagrams are included to aide in source code perusal. + + +The client side of a CLI change + +.. figure:: ../figures/cli-change-client.svg + :align: center + + +The server (mgmtd) side of a CLI change + +.. figure:: ../figures/cli-change-mgmtd.svg + :align: center diff --git a/doc/developer/modules.rst b/doc/developer/modules.rst new file mode 100644 index 0000000..0feac8e --- /dev/null +++ b/doc/developer/modules.rst @@ -0,0 +1,142 @@ +.. _modules: + +Modules +======= + +FRR has facilities to load DSOs at startup via ``dlopen()``. These are used to +implement modules, such as SNMP and FPM. + +Limitations +----------- + +- can't load, unload, or reload during runtime. This just needs some + work and can probably be done in the future. +- doesn't fix any of the "things need to be changed in the code in the + library" issues. Most prominently, you can't add a CLI node because + CLI nodes are listed in the library... +- if your module crashes, the daemon crashes. Should be obvious. +- **does not provide a stable API or ABI**. Your module must match a + version of FRR and you may have to update it frequently to match + changes. +- **does not create a license boundary**. Your module will need to link + libzebra and include header files from the daemons, meaning it will + be GPL-encumbered. + +Installation +------------ + +Look for ``moduledir`` in ``configure.ac``, default is normally +``/usr/lib64/frr/modules`` but depends on ``--libdir`` / ``--prefix``. + +The daemon's name is prepended when looking for a module, e.g. "snmp" +tries to find "zebra\_snmp" first when used in zebra. This is just to +make it nicer for the user, with the snmp module having the same name +everywhere. + +Modules can be packaged separately from FRR. The SNMP and FPM modules +are good candidates for this because they have dependencies (net-snmp / +protobuf) that are not FRR dependencies. However, any distro packages +should have an "exact-match" dependency onto the FRR package. Using a +module from a different FRR version will probably blow up nicely. + +For snapcraft (and during development), modules can be loaded with full +path (e.g. -M ``$SNAP/lib/frr/modules/zebra_snmp.so``). Note that +libtool puts output files in the .libs directory, so during development +you have to use ``./zebra -M .libs/zebra_snmp.so``. + +Creating a module +----------------- + +... best to look at the existing SNMP or FPM modules. + +Basic boilerplate: + +:: + + #include "hook.h" + #include "module.h" + #include "libfrr.h" + #include "frrevent.h" + + static int module_late_init(struct event_loop *master) + { + /* Do initialization stuff here */ + return 0; + } + + static int + module_init (void) + { + hook_register(frr_late_init, module_late_init); + return 0; + } + + FRR_MODULE_SETUP( + .name = "my module", + .version = "0.0", + .description = "my module", + .init = module_init, + ); + +The ``frr_late_init`` hook will be called after the daemon has finished +its other startup and is about to enter the main event loop; this is the +best place for most initialisation. + +Compiler & Linker magic +----------------------- + +There's a ``THIS_MODULE`` (like in the Linux kernel), which uses +``visibility`` attributes to restrict it to the current module. If you +get a linker error with ``_frrmod_this_module``, there is some linker +SNAFU. This shouldn't be possible, though one way to get it would be to +not include libzebra (which provides a fallback definition for the +symbol). + +libzebra and the daemons each have their own ``THIS_MODULE``, as do all +loadable modules. In any other libraries (e.g. ``libfrrsnmp``), +``THIS_MODULE`` will use the definition in libzebra; same applies if the +main executable doesn't use ``FRR_DAEMON_INFO`` (e.g. all testcases). + +The deciding factor here is "what dynamic linker unit are you using the +symbol from." If you're in a library function and want to know who +called you, you can't use ``THIS_MODULE`` (because that'll just tell you +you're in the library). Put a macro around your function that adds +``THIS_MODULE`` in the *caller's code calling your function*. + +The idea is to use this in the future for module unloading. Hooks +already remember which module they were installed by, as groundwork for +a function that removes all of a module's installed hooks. + +There's also the ``frr_module`` symbol in modules, pretty much a +standard entry point for loadable modules. + +Command line parameters +----------------------- + +Command line parameters can be passed directly to a module by appending a +colon to the module name when loading it, e.g. ``-M mymodule:myparameter``. +The text after the colon will be accessible in the module's code through +``THIS_MODULE->load_args``. For example, see how the format parameter is +configured in the ``zfpm_init()`` function inside ``zebra_fpm.c``. + +Hooks +----- + +Hooks are just points in the code where you can register your callback +to be called. The parameter list is specific to the hook point. Since +there is no stable API, the hook code has some extra type safety checks +making sure you get a compiler warning when the hook parameter list +doesn't match your callback. Don't ignore these warnings. + +Relation to MTYPE macros +------------------------ + +The MTYPE macros, while primarily designed to decouple MTYPEs from the +library and beautify the code, also work very nicely with loadable +modules -- both constructors and destructors are executed when +loading/unloading modules. + +This means there is absolutely no change required to MTYPEs, you can +just use them in a module and they will even clean up themselves when we +implement module unloading and an unload happens. In fact, it's +impossible to create a bug where unloading fails to de-register a MTYPE. diff --git a/doc/developer/next-hop-tracking.rst b/doc/developer/next-hop-tracking.rst new file mode 100644 index 0000000..99e1d65 --- /dev/null +++ b/doc/developer/next-hop-tracking.rst @@ -0,0 +1,350 @@ +Next Hop Tracking +================== + +Next hop tracking is an optimization feature that reduces the processing time +involved in the BGP bestpath algorithm by monitoring changes to the routing +table. + +Background +----------- + +Recursive routes are of the form: + +:: + + p/m --> n + [Ex: 1.1.0.0/16 --> 2.2.2.2] + +where 'n' itself is resolved through another route as follows: + +:: + + p2/m --> h, interface + [Ex: 2.2.2.0/24 --> 3.3.3.3, eth0] + +Usually, BGP routes are recursive in nature and BGP nexthops get resolved +through an IGP route. IGP usually adds its routes pointing to an interface +(these are called non-recursive routes). + +When BGP receives a recursive route from a peer, it needs to validate the +nexthop. The path is marked valid or invalid based on the reachability status +of the nexthop. Nexthop validation is also important for BGP decision process +as the metric to reach the nexthop is a parameter to best path selection +process. + +As it goes with routing, this is a dynamic process. Route to the nexthop can +change. The nexthop can become unreachable or reachable. In the current BGP +implementation, the nexthop validation is done periodically in the scanner run. +The default scanner run interval is one minute. Every minute, the scanner task +walks the entire BGP table. It checks the validity of each nexthop with Zebra +(the routing table manager) through a request and response message exchange +between BGP and Zebra process. BGP process is blocked for that duration. The +mechanism has two major drawbacks: + +- The scanner task runs to completion. That can potentially starve the other + tasks for long periods of time, based on the BGP table size and number of + nexthops. + +- Convergence around routing changes that affect the nexthops can be long + (around a minute with the default intervals). The interval can be shortened + to achieve faster reaction time, but it makes the first problem worse, with + the scanner task consuming most of the CPU resources. + +The next-hop tracking feature makes this process event-driven. It eliminates +periodic nexthop validation and introduces an asynchronous communication path +between BGP and Zebra for route change notifications that can then be acted +upon. + +Goal +---- + +Stating the obvious, the main goal is to remove the two limitations we +discussed in the previous section. The goals, in a constructive tone, +are the following: + +- **Fairness**: the scanner run should not consume an unjustly high amount of + CPU time. This should give an overall good performance and response time to + other events (route changes, session events, IO/user interface). + +- **Convergence**: BGP must react to nexthop changes instantly and provide + sub-second convergence. This may involve diverting the routes from one + nexthop to another. + +Overview of changes +------------------------ + +The changes are in both BGP and Zebra modules. The short summary is +the following: + +- Zebra implements a registration mechanism by which clients can + register for next hop notification. Consequently, it maintains a + separate table, per (VRF, AF) pair, of next hops and interested + client-list per next hop. + +- When the main routing table changes in Zebra, it evaluates the next + hop table: for each next hop, it checks if the route table + modifications have changed its state. If so, it notifies the + interested clients. + +- BGP is one such client. It registers the next hops corresponding to + all of its received routes/paths. It also threads the paths against + each nexthop structure. + +- When BGP receives a next hop notification from Zebra, it walks the + corresponding path list. It makes them valid or invalid depending + on the next hop notification. It then re-computes best path for the + corresponding destination. This may result in re-announcing those + destinations to peers. + +Design +------ + +Modules +^^^^^^^ + +The core design introduces an "nht" (next hop tracking) module in BGP +and "rnh" (recursive nexthop) module in Zebra. The "nht" module +provides the following APIs: + ++----------------------------+--------------------------------------------------+ +| Function | Action | ++============================+==================================================+ +| bgp_find_or_add_nexthop() | find or add a nexthop in BGP nexthop table | ++----------------------------+--------------------------------------------------+ +| bgp_parse_nexthop_update() | parse a nexthop update message coming from zebra | ++----------------------------+--------------------------------------------------+ + +The "rnh" module provides the following APIs: + ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| Function | Action | ++============================+==========================================================================================================+ +| zebra_add_rnh() | add a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_delete_rnh() | delete a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_lookup_rnh() | lookup a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_add_rnh_client() | register a client for nexthop notifications against a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_remove_rnh_client() | remove the client registration for a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_evaluate_rnh_table() | (re)evaluate the recursive nexthop table (most probably because the main routing table has changed). | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_cleanup_rnh_client() | Cleanup a client from the "rnh" module data structures (most probably because the client is going away). | ++----------------------------+----------------------------------------------------------------------------------------------------------+ + +4.2. Control flow + +The next hop registration control flow is the following: + +:: + + <==== BGP Process ====>|<==== Zebra Process ====> + | + receive module nht module | zserv module rnh module + ---------------------------------------------------------------------- + | | | + bgp_update_ | | | + main() | bgp_find_or_add_ | | + | nexthop() | | + | | | + | | zserv_nexthop_ | + | | register() | + | | | zebra_add_rnh() + | | | + + +The next hop notification control flow is the following: + +:: + + <==== Zebra Process ====>|<==== BGP Process ====> + | + rib module rnh module | zebra module nht module + ---------------------------------------------------------------------- + | | | + meta_queue_ | | | + process() | zebra_evaluate_ | | + | rnh_table() | | + | | | + | | bgp_read_nexthop_ | + | | update() | + | | | bgp_parse_ + | | | nexthop_update() + | | | + + +zclient message format +^^^^^^^^^^^^^^^^^^^^^^ + +ZEBRA_NEXTHOP_REGISTER and ZEBRA_NEXTHOP_UNREGISTER messages are +encoded in the following way: + +:: + + . 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +``ZEBRA_NEXTHOP_UPDATE`` message is encoded as follows: + +:: + + . 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix getting resolved . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | metric | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | #nexthops | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | nexthop type | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . resolving Nexthop details . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | nexthop type | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . resolving Nexthop details . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +BGP data structure +^^^^^^^^^^^^^^^^^^ +Legend: + +:: + + /\ struct bgp_node: a BGP destination/route/prefix + \/ + + [ ] struct bgp_path_info: a BGP path (e.g. route received from a peer) + + _ + (_) struct bgp_nexthop_cache: a BGP nexthop + + /\ NULL + \/--+ ^ + | : + +--[ ]--[ ]--[ ]--> NULL + /\ : + \/--+ : + | : + +--[ ]--[ ]--> NULL + : + _ : + (_)........... + + +Zebra data structure +^^^^^^^^^^^^^^^^^^^^ + +RNH table:: + + . O + / \ + O O + / \ + O O + + struct rnh + { + uint8_t flags; + struct route_entry *state; + struct list *client_list; + struct route_node *node; + }; + +User interface changes +^^^^^^^^^^^^^^^^^^^^^^ + +:: + + frr# show ip nht + 3.3.3.3 + resolved via kernel + via 11.0.0.6, swp1 + Client list: bgp(fd 12) + 11.0.0.10 + resolved via connected + is directly connected, swp2 + Client list: bgp(fd 12) + 11.0.0.18 + resolved via connected + is directly connected, swp4 + Client list: bgp(fd 12) + 11.11.11.11 + resolved via kernel + via 10.0.1.2, eth0 + Client list: bgp(fd 12) + + frr# show ip bgp nexthop + Current BGP nexthop cache: + 3.3.3.3 valid [IGP metric 0], #paths 3 + Last update: Wed Oct 16 04:43:49 2013 + + 11.0.0.10 valid [IGP metric 1], #paths 1 + Last update: Wed Oct 16 04:43:51 2013 + + 11.0.0.18 valid [IGP metric 1], #paths 2 + Last update: Wed Oct 16 04:43:47 2013 + + 11.11.11.11 valid [IGP metric 0], #paths 1 + Last update: Wed Oct 16 04:43:47 2013 + + frr# show ipv6 nht + frr# show ip bgp nexthop detail + + frr# debug bgp nht + frr# debug zebra nht + + 6. Sample test cases + + r2----r3 + / \ / + r1----r4 + + - Verify that a change in IGP cost triggers NHT + + shutdown the r1-r4 and r2-r4 links + + no shut the r1-r4 and r2-r4 links and wait for OSPF to come back + up + + We should be back to the original nexthop via r4 now + - Verify that a NH becoming unreachable triggers NHT + + Shutdown all links to r4 + - Verify that a NH becoming reachable triggers NHT + + no shut all links to r4 + +Future work +^^^^^^^^^^^ + +- route-policy for next hop validation (e.g. ignore default route) +- damping for rapid next hop changes +- prioritized handling of nexthop changes ((un)reachability vs. metric + changes) +- handling recursion loop, e.g:: + + 11.11.11.11/32 -> 12.12.12.12 + 12.12.12.12/32 -> 11.11.11.11 + 11.0.0.0/8 -> <interface> +- better statistics diff --git a/doc/developer/northbound/advanced-topics.rst b/doc/developer/northbound/advanced-topics.rst new file mode 100644 index 0000000..bee29a9 --- /dev/null +++ b/doc/developer/northbound/advanced-topics.rst @@ -0,0 +1,294 @@ +Auto-generated CLI commands +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to have less code to maintain, it should be possible to write a +tool that auto-generates CLI commands based on the FRR YANG models. As a +matter of fact, there are already a number of NETCONF-based CLIs that do +exactly that (e.g. `Clixon <https://github.com/clicon/clixon>`__, +ConfD’s CLI). + +The problem however is that there isn’t an exact one-to-one mapping +between the existing CLI commands and the corresponding YANG nodes from +the native models. As an example, ripd’s +``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` command +changes three YANG leaves at the same time. In order to auto-generate +CLI commands and retain their original form, it’s necessary to add +annotations in the YANG modules to specify how the commands should look +like. Without YANG annotations, the CLI auto-generator will generate a +command for each YANG leaf, (leaf-)list and presence-container. The +ripd’s ``timers basic`` command, for instance, would become three +different commands, which would be undesirable. + + This Tail-f’s® + `document <http://info.tail-f.com/hubfs/Whitepapers/Tail-f_ConfD-CLI__Cfg_Mode_App_Note_Rev%20C.pdf>`__ + shows how to customize ConfD auto-generated CLI commands using YANG + annotations. + +The good news is that *libyang* allows users to create plugins to +implement their own YANG extensions, which can be used to implement CLI +annotations. If done properly, a CLI generator can save FRR developers +from writing and maintaining hundreds if not thousands of DEFPYs! + +CLI on a separate program +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The flexible design of the northbound architecture opens the door to +move the CLI to a separate program in the long-term future. Some +advantages of doing so would be: \* Treat the CLI as just another +northbound client, instead of having CLI commands embedded in the +binaries of all FRR daemons. \* Improved robustness: bugs in CLI +commands (e.g. null-pointer dereferences) or in the CLI code itself +wouldn’t affect the FRR daemons. \* Foster innovation by allowing other +CLI programs to be implemented, possibly using higher level programming +languages. + +The problem, however, is that the northbound retrofitting process will +convert only the CLI configuration commands and EXEC commands in a first +moment. Retrofitting the “show” commands is a completely different story +and shouldn’t happen anytime soon. This should hinder progress towards +moving the CLI to a separate program. + +Proposed feature: confirmed commits +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Confirmed commits allow the user to request an automatic rollback to the +previous configuration if the commit operation is not confirmed within a +number of minutes. This is particularly useful when the user is +accessing the CLI through the network (e.g. using SSH) and any +configuration change might cause an unexpected loss of connectivity +between the user and the router (e.g. misconfiguration of a routing +protocol). By using a confirmed commit, the user can rest assured the +connectivity will be restored after the given timeout expires, avoiding +the need to access the router physically to fix the problem. + +Example of how this feature could be provided in the CLI: +``commit confirmed [minutes <1-60>]``. The ability to do confirmed +commits should also be exposed in the northbound API so that the +northbound plugins can also take advantage of it (in the case of the +Sysrepo and ConfD plugins, confirmed commits are implemented externally +in the *netopeer2-server* and *confd* daemons, respectively). + +Proposed feature: enable/disable configuration commands/sections +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since the ``lyd_node`` data structure from *libyang* can hold private +data, it should be possible to mark configuration commands or sections +as active or inactive. This would allow CLI users to leverage this +feature to disable parts of the running configuration without actually +removing the associated commands, and then re-enable the disabled +configuration commands or sections later when necessary. Example: + +:: + + ripd(config)# show configuration running + Configuration: + [snip] + ! + router rip + default-metric 2 + distance 80 + network eth0 + network eth1 + ! + end + ripd(config)# disable router rip + ripd(config)# commit + % Configuration committed successfully (Transaction ID #7). + + ripd(config)# show configuration running + Configuration: + [snip] + ! + !router rip + !default-metric 2 + !distance 80 + !network eth0 + !network eth1 + ! + end + ripd(config)# enable router rip + ripd(config)# commit + % Configuration committed successfully (Transaction ID #8). + + ripd(config)# show configuration running + [snip] + frr defaults traditional + ! + router rip + default-metric 2 + distance 80 + network eth0 + network eth1 + ! + end + +This capability could be useful in a number of occasions, like disabling +configuration commands that are no longer necessary (e.g. ACLs) but that +might be necessary at a later point in the future. Other example is +allowing users to disable a configuration section for testing purposes, +and then re-enable it easily without needing to copy and paste any +command. + +Configuration reloads +~~~~~~~~~~~~~~~~~~~~~ + +Given the limitations of the previous northbound architecture, the FRR +daemons didn’t have the ability to reload their configuration files by +themselves. The SIGHUP handler of most daemons would only re-read the +configuration file and merge it into the running configuration. In most +cases, however, what is desired is to replace the running configuration +by the updated configuration file. The *frr-reload.py* script was +written to work around this problem and it does it well to a certain +extent. The problem with the *frr-reload.py* script is that it’s full of +special cases here and there, which makes it fragile and unreliable. +Maintaining the script is also an additional burden for FRR developers, +few of whom are familiar with its code or know when it needs to be +updated to account for a new feature. + +In the new northbound architecture, reloading the configuration file can +be easily implemented using a configuration transaction. Once the FRR +northbound retrofitting process is complete, all daemons should have the +ability to reload their configuration files upon receiving the SIGHUP +signal, or when the ``configuration load [...] replace`` command is +used. Once that point is reached, the *frr-reload.py* script will no +longer be necessary and should be removed from the FRR repository. + +Configuration changes coming from the kernel +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This +`post <http://discuss.tail-f.com/t/who-should-not-set-configuration-once-a-system-is-up-and-running/111>`__ +from the Tail-f’s® forum describes the problem of letting systems +configure themselves behind the users back. Here are some selected +snippets from it: > Traditionally, northbound interface users are the +ones in charge of providing configuration data for systems. > > In some +systems, we see a deviation from this traditional practice; allowing +systems to configure “themselves” behind the scenes (or behind the users +back). > > While there might be a business case for such a practice, +this kind of configuration remains “dangerous” from northbound users +perspective and makes systems hard to predict and even harder to debug. +(…) > > With the advent of transactional Network configuration, this +practice can not work anymore. The fact that systems are given the right +to change configuration is a key here in breaking transactional +configuration in a Network. + +FRR is immune to some of the problems described in the aforementioned +post. Management clients can configure interfaces that don’t yet exist, +and once an interface is deleted from the kernel, its configuration is +retained in FRR. + +There are however some cases where information learned from the kernel +(e.g. using netlink) can affect the running configuration of all FRR +daemons. Examples: interface rename events, VRF rename events, interface +being moved to a different VRF, etc. In these cases, since these events +can’t be ignored, the best we can do is to send YANG notifications to +the management clients to inform about the configuration changes. The +management clients should then be prepared to handle such notifications +and react accordingly. + +Interfaces and VRFs +~~~~~~~~~~~~~~~~~~~ + +As of now zebra doesn’t have the ability to create VRFs or virtual +interfaces in the kernel. The ``vrf`` and ``interface`` commands only +create pre-provisioned VRFs and interfaces that are only activated when +the corresponding information is learned from the kernel. When +configuring FRR using an external management client, like a NETCONF +client, it might be desirable to actually create functional VRFs and +virtual interfaces (e.g. VLAN subinterfaces, bridges, etc) that are +installed in the kernel using OS-specific APIs (e.g. netlink, routing +socket, etc). Work needs to be done in this area to make this possible. + +Shared configuration objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +One of the existing problems in FRR is that it’s hard to ensure that all +daemons are in sync with respect to the shared configuration objects +(e.g. interfaces, VRFs, route-maps, ACLs, etc). When a route-map is +configured using *vtysh*, the same command is sent to all relevant +daemons (the daemons that implement route-maps), which ensures +synchronization among them. The problem is when a daemon starts after +the route-maps are created. In this case this daemon wouldn’t be aware +of the previously configured route-maps (unlike the other daemons), +which can lead to a lot of confusion and unexpected problems. + +With the new northbound architecture, configuration objects can be +manipulated using higher level abstractions, which opens more +possibilities to solve this decades-long problem. As an example, one +solution would be to make the FRR daemons fetch the shared configuration +objects from zebra using the ZAPI interface during initialization. The +shared configuration objects could be requested using a list of XPaths +expressions in the ``ZEBRA_HELLO`` message, which zebra would respond by +sending the shared configuration objects encoded in the JSON format. +This solution however doesn’t address the case where zebra starts or +restarts after the other FRR daemons. Other solution would be to store +the shared configuration objects in the northbound SQL database and make +all daemons fetch these objects from there. So far no work has been made +on this area as more investigation needs to be done. + +vtysh support +~~~~~~~~~~~~~ + +As explained in the [[Transactional CLI]] page, all commands introduced +by the transactional CLI are not yet available in *vtysh*. This needs to +be addressed in the short term future. Some challenges for doing that +work include: \* How to display configurations (running, candidates and +rollbacks) in a more clever way? The implementation of the +``show running-config`` command in *vtysh* is not something that should +be followed as an example. A better idea would be to fetch the desired +configuration from all daemons (encoded in JSON for example), merge them +all into a single ``lyd_node`` variable and then display the combined +configurations from this variable (the configuration merges would +transparently take care of combining the shared configuration objects). +In order to be able to manipulate the JSON configurations, *vtysh* will +need to load the YANG modules from all daemons at startup (this might +have a minimal impact on startup time). The only issue with this +approach is that the ``cli_show()`` callbacks from all daemons are +embedded in their binaries and thus not accessible externally. It might +be necessary to compile these callbacks on a separate shared library so +that they are accessible to *vtysh* too. Other than that, displaying the +combined configurations in the JSON/XML formats should be +straightforward. \* With the current design, transaction IDs are +per-daemon and not global across all FRR daemons. This means that the +same transaction ID can represent different transactions on different +daemons. Given this observation, how to implement the +``rollback configuration`` command in *vtysh*? The easy solution would +be to add a ``daemon WORD`` argument to specify the context of the +rollback, but per-daemon rollbacks would certainly be confusing and +convoluted to end users. A better idea would be to attack the root of +the problem: change configuration transactions to be global instead of +being per-daemon. This involves a bigger change in the northbound +architecture, and would have implications on how transactions are stored +in the SQL database (daemon-specific and shared configuration objects +would need to have their own tables or columns). \* Loading +configuration files in the JSON or XML formats will be tricky, as +*vtysh* will need to know which sections of the configuration should be +sent to which daemons. *vtysh* will either need to fetch the YANG +modules implemented by all daemons at runtime or obtain this information +at compile-time somehow. + +Detecting type mismatches at compile-time +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As described in the [[Retrofitting Configuration Commands]] page, the +northbound configuration callbacks detect type mismatches at runtime +when fetching data from the the ``dnode`` parameter (which represents +the configuration node being created, modified, deleted or moved). When +a type mismatch is detected, the program aborts and displays a backtrace +showing where the problem happened. It would be desirable to detect such +type mismatches at compile-time, the earlier the problems are detected +the sooner they are fixed. + +One possible solution to this problem would be to auto-generate C +structures from the YANG models and provide a function that converts a +libyang’s ``lyd_node`` variable to a C structure containing the same +information. The northbound callbacks could then fetch configuration +data from this C structure, which would naturally lead to type +mismatches being detected at compile time. One of the challenges of +doing this would be the handling of YANG lists and leaf-lists. It would +be necessary to use dynamic data structures like hashes or rb-trees to +hold all elements of the lists and leaf-lists, and the process of +converting a ``lyd_node`` to an auto-generated C-structure could be +expensive. At this point it’s unclear if it’s worth adding more +complexity in the northbound architecture to solve this specific +problem. diff --git a/doc/developer/northbound/architecture.rst b/doc/developer/northbound/architecture.rst new file mode 100644 index 0000000..e571971 --- /dev/null +++ b/doc/developer/northbound/architecture.rst @@ -0,0 +1,275 @@ +Introduction +------------ + +The goal of the new northbound API is to provide a better interface to +configure and monitor FRR programatically. The current design based on +CLI commands is no longer adequate in a world where computer networks +are becoming increasingly bigger, more diverse and more complex. Network +scripting using *expect* and screen scraping techniques is too primitive +and unreliable to be used in large-scale networks. What is proposed is +to modernize FRR to turn it into an API-first routing stack, and +reposition the CLI on top of this API. The most important change, +however, is not the API that will be provided to external users. In +fact, multiple APIs will be supported and users will have the ability to +write custom management APIs if necessary. The biggest change is the +introduction of a model-driven management architecture based on the +`YANG <https://tools.ietf.org/html/rfc7950>`__ modeling language. +Instead of writing code tied to any particular user interface +(e.g. DEFUNs), YANG allows us to write API-agnostic code (in the form of +callbacks) that can be used by any management interface. As an example, +it shouldn’t matter if a set of configuration changes is coming from a +`NETCONF <https://tools.ietf.org/html/rfc6241>`__ session or from a CLI +terminal, the same callbacks should be called to process the +configuration changes regardless of where they came from. This +model-driven design ensures feature parity across all management +interfaces supported by FRR. + +Quoting :rfc:`7950`: + + YANG is a language originally designed to model data for the NETCONF + protocol. A YANG module defines hierarchies of data that can be used for + NETCONF-based operations, including configuration, state data, RPCs, and + notifications. This allows a complete description of all data sent between a + NETCONF client and server. Although out of scope for this specification, + YANG can also be used with protocols other than NETCONF. + +While the YANG and NETCONF specifications are tightly coupled with one +another, both are independent to a certain extent and are evolving +separately. Examples of other management protocols that use YANG include +`RESTCONF <https://tools.ietf.org/html/rfc8040>`__, +`gNMI <https://github.com/openconfig/reference/tree/master/rpc/gnmi>`__ +and +`CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__. + +In addition to being management-protocol independent, some other +advantages of using YANG in FRR are listed below: \* Have a formal +contract between FRR and application developers (management clients). A +management client that has access to the FRR YANG models knows about all +existing configuration options available for use. This information can +be used to auto-generate user-friendly interfaces like Web-UIs, custom +CLIs and even code bindings for several different programming languages. +Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for +example, it’s possible to generate Python class hierarchies from YANG +models and use these classes to instantiate objects that mirror the +structure of the YANG modules and can be serialized/deserialized using +different encoding formats. \* Support different encoding formats for +instance data. Currently only JSON and XML are supported, but +`GPB <https://developers.google.com/protocol-buffers/>`__ and +`CBOR <http://cbor.io/>`__ are other viable options in the long term. +Additional encoding formats can be implemented in the *libyang* library +for optimal performance, or externally by translating data to/from one +of the supported formats (with a performance penalty). \* Have a formal +mechanism to introduce backward-incompatible changes based on `semantic +versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the +YANG standard, which allows backward-compatible module updates only). \* +Provide seamless support to the industry-standard NETCONF/RESTCONF +protocols as alternative management APIs. If FRR configuration/state +data is modeled using YANG, supporting YANG-based protocols like NETCONF +and RESTCONF is much easier. + +As important as shifting to a model-driven management paradigm, the new +northbound architecture also introduces the concept of configuration +transactions. Configuration transactions allow management clients to +commit multiple configuration changes at the same time and rest assured +that either all changes will be applied or none will (all-or-nothing). +Configuration transactions are implemented as pseudo-atomic operations +and facilitate automation by removing the burden of error recovery from +the management side. Another property of configuration transactions is +that the configuration changes are always processed in a pre-defined +order to ensure consistency. Configuration transactions that encompass +multiple network devices are called network-wide transactions and are +also supported by the new northbound architecture. When FRR is built +using the ``--enable-config-rollbacks`` option, all committed +transactions are recorded in the FRR rollback log, which can reside +either in memory (volatile) or on persistent storage. + + Network-wide Transactions is the most important leap in network + management technology since SNMP. The error recovery and sequencing + tasks are removed from the manager side. This is usually more than + half the cost in a mature system; more than the entire cost of the + managed devices. + `[source] <https://www.nanog.org/sites/default/files/tuesday_tutorial_moberg_netconf_35.pdf>`__. + +Figures 1 and 2 below illustrate the old and new northbound architecture +of FRR, respectively. As it can be seen, in the old architecture the CLI +was the only interface used to configure and monitor FRR (the SNMP +plugin was’t taken into account given the small number of implemented +MIBs). This means that the only way to automate FRR was by writing +scripts that send CLI commands and parse the text output (which usually +doesn’t have any structure) using screen scraping and regular +expressions. + +.. figure:: images/arch-before.png + :alt: diagram of northbound architecture prior to nbapi conversion + + Old northbound architecture + +The new northbound architectures, on the other hand, features a +multitude of different management APIs, all of them connected to the +northbound layer of the FRR daemons. By default, only the CLI interface +is compiled built-in in the FRR daemons. The other management interfaces +are provided as optional plugins and need to be loaded during the daemon +initialization (e.g. *zebra -M confd*). This design makes it possible to +integrate FRR with different NETCONF solutions without introducing +vendor lock-in. The [[Plugins - Writing Your Own]] page explains how to +write custom northbound plugins that can be tailored to all needs +(e.g. support custom transport protocols, different data encoding +formats, fine-grained access control, etc). + +.. figure:: images/arch-after.png + :alt: diagram of northbound architecture after nbapi conversion + + New northbound architecture + +Figure 3 shows the internal view of the FRR northbound architecture. In +this image we can see that northbound layer is an abstract entity +positioned between the northbound callbacks and the northbound clients. +The northbound layer is responsible to process the requests coming from +the northbound clients and call the appropriate callbacks to satisfy +these requests. The northbound plugins communicate with the northbound +layer through a public API, which allow users to write third-party +plugins that can be maintained separately. The northbound plugins, in +turn, have their own APIs to communicate with external management +clients. + +.. figure:: images/nb-layer.png + :alt: diagram of northbound architecture internals + + New northbound architecture - internal view + +Initially the CLI (and all of its commands) will be maintained inside +the FRR daemons. In the long term, however, the goal is to move the CLI +to a separate program just like any other management client. The +[[Advanced Topics]] page describes the motivations and challenges of +doing that. Last but not least, the *libyang* block inside the +northbound layer is the engine that makes everything possible. The +*libyang* library will be described in more detail in the following +sections. + +YANG models +----------- + +The main decision to be made when using YANG is which models to +implement. There’s a general consensus that using standard models is +preferable over using custom (native) models. The reasoning is that +applications based on standard models can be reused for all network +appliances that support those models, whereas the same doesn’t apply for +applications written based on custom models. + +That said, there are multiple standards bodies publishing YANG models +and unfortunately not all of them are converging (or at least not yet). +In the context of FRR, which is a routing stack, the two sets of YANG +models that would make sense to implement are the ones from IETF and +from the OpenConfig working group. The question that arises is: which +one of them should we commit to? Or should we try to support both +somehow, at the cost of extra development efforts? + +Another problem, from an implementation point of view, is that it’s +challenging to adapt the existing code base to match standard models. A +more reasonable solution, at least in a first moment, would be to use +YANG deviations and augmentations to do the opposite: adapt the standard +models to the existing code. In practice however this is not as simple +as it seems. There are cases where the differences are too substantial +to be worked around without restructuring the code by changing its data +structures and their relationships. As an example, the *ietf-rip* model +places per-interface RIP configuration parameters inside the +*control-plane-protocol* list (which is augmented by *ietf-rip*). This +means that it’s impossible to configure RIP interface parameters without +first configuring a RIP routing instance. The *ripd* daemon on the other +hand allows the operator to configure RIP interface parameters even if +``router rip`` is not configured. If we were to implement the *ietf-rip* +module natively, we’d need to change ripd’s CLI commands (and the +associated code) to reflect the new configuration hierarchy. + +Taking into account that FRR has a huge code base and that the +northbound retrofitting process per-se will cause a lot of impact, it +was decided to take a conservative approach and write custom YANG models +for FRR modeled after the existing CLI commands. Having YANG models that +closely mirror the CLI commands will allow the FRR developers to +retrofit the code base much more easily, without introducing +backward-incompatible changes in the CLI and reducing the likelihood of +introducing bugs. The [[Retrofitting Configuration Commands]] page +explains in detail how to convert configuration commands to the new +northbound model. + +Even though having native YANG models is not the ideal solution, it will +be already a big step forward for FRR to migrate to a model-driven +management architecture, with support for configuration transactions and +multiple management interfaces, including NETCONF and RESTCONF (through +the northbound plugins). + +The new northbound also features an experimental YANG module translator +that will allow users to translate to and from standard YANG models by +using translation tables. The [[YANG module translator]] page describes +this mechanism in more detail. At this point it’s unclear what can be +achieved through module translation and if that can be considered as a +definitive solution to support standard models or not. + +Northbound Architecture +----------------------- + +.. figure:: images/lys-node.png + :alt: diagram of libyanbg's lys_node data structure + + ``libyang's`` lys_node data structure + + +.. figure:: images/lyd-node.png + :alt: diagram of libyanbg's lyd_node data structure + + ``libyang's`` lyd_node data structure + + +.. figure:: images/ly-ctx.png + :alt: diagram of libyanbg's ly_ctx data structure + + ``libyang's`` ly_ctx data structure + + +.. figure:: images/transactions.png + :alt: diagram showing how configuration transactions work + + Configuration transactions + + +Testing +------- + +The new northbound adds the libyang library as a new mandatory +dependency for FRR. To obtain and install this library, follow the steps +below: + +.. code-block:: console + + git clone https://github.com/CESNET/libyang + cd libyang + git checkout devel + mkdir build ; cd build + cmake -DENABLE_LYD_PRIV=ON .. + make + sudo make install + + +.. note:: + + first make sure to install the libyang + `requirements <https://github.com/CESNET/libyang#build-requirements>`__. + + +FRR needs libyang from version 0.16.7 or newer, which is maintained in +the ``devel`` branch. libyang 0.15.x is maintained in the ``master`` +branch and doesn’t contain one small feature used by FRR (the +``LY_CTX_DISABLE_SEARCHDIR_CWD`` flag). FRR also makes use of the +libyang’s ``ENABLE_LYD_PRIV`` feature, which is disabled by default and +needs to be enabled at compile time. + +It’s advisable (but not required) to install sqlite3 and build FRR with +``--enable-config-rollbacks`` in order to have access to the +configuration rollback feature. + +To test the northbound, the suggested method is to use the +[[Transactional CLI]] with the *ripd* daemon and play with the new +commands. The ``debug northbound`` command can be used to see which +northbound callbacks are called in response to the ``commit`` command. +For reference, the [[Demos]] page shows a small demonstration of the +transactional CLI in action and what it’s capable of. diff --git a/doc/developer/northbound/demos.rst b/doc/developer/northbound/demos.rst new file mode 100644 index 0000000..876bd25 --- /dev/null +++ b/doc/developer/northbound/demos.rst @@ -0,0 +1,27 @@ +Transactional CLI +----------------- + +This short demo shows some of the capabilities of the new transactional +CLI: + +|asciicast1| + +ConfD + NETCONF + Cisco YDK +--------------------------- + +This is a very simple demo of *ripd* being configured by a python +script. The script uses NETCONF to communicate with *ripd*, which has +the ConfD plugin loaded. The most interesting part, however, is the fact +that the python script is not using handcrafted XML payloads to +configure *ripd*. Instead, the script is using python bindings generated +using Cisco’s YANG Development Kit (YDK). + +- Script used in the demo: + https://gist.github.com/rwestphal/defa9bd1ccf216ab082d4711ae402f95 + +|asciicast2| + +.. |asciicast1| image:: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1.png + :target: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1 +.. |asciicast2| image:: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv.png + :target: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv diff --git a/doc/developer/northbound/images/arch-after.png b/doc/developer/northbound/images/arch-after.png Binary files differnew file mode 100644 index 0000000..01e6ae6 --- /dev/null +++ b/doc/developer/northbound/images/arch-after.png diff --git a/doc/developer/northbound/images/arch-before.png b/doc/developer/northbound/images/arch-before.png Binary files differnew file mode 100644 index 0000000..ab2bb0d --- /dev/null +++ b/doc/developer/northbound/images/arch-before.png diff --git a/doc/developer/northbound/images/ly-ctx.png b/doc/developer/northbound/images/ly-ctx.png Binary files differnew file mode 100644 index 0000000..4d4e138 --- /dev/null +++ b/doc/developer/northbound/images/ly-ctx.png diff --git a/doc/developer/northbound/images/lyd-node.png b/doc/developer/northbound/images/lyd-node.png Binary files differnew file mode 100644 index 0000000..4ba2b48 --- /dev/null +++ b/doc/developer/northbound/images/lyd-node.png diff --git a/doc/developer/northbound/images/lys-node.png b/doc/developer/northbound/images/lys-node.png Binary files differnew file mode 100644 index 0000000..e9e46e7 --- /dev/null +++ b/doc/developer/northbound/images/lys-node.png diff --git a/doc/developer/northbound/images/nb-layer.png b/doc/developer/northbound/images/nb-layer.png Binary files differnew file mode 100644 index 0000000..4aa1fd6 --- /dev/null +++ b/doc/developer/northbound/images/nb-layer.png diff --git a/doc/developer/northbound/images/transactions.png b/doc/developer/northbound/images/transactions.png Binary files differnew file mode 100644 index 0000000..d18faf4 --- /dev/null +++ b/doc/developer/northbound/images/transactions.png diff --git a/doc/developer/northbound/links.rst b/doc/developer/northbound/links.rst new file mode 100644 index 0000000..e80374c --- /dev/null +++ b/doc/developer/northbound/links.rst @@ -0,0 +1,233 @@ +RFCs +~~~~ + +- `RFC 7950 - The YANG 1.1 Data Modeling + Language <https://tools.ietf.org/html/rfc7950>`__ +- `RFC 7951 - JSON Encoding of Data Modeled with + YANG <https://tools.ietf.org/html/rfc7951>`__ +- `RFC 8342 - Network Management Datastore Architecture + (NMDA) <https://tools.ietf.org/html/rfc8342>`__ +- `RFC 6087 - Guidelines for Authors and Reviewers of YANG Data Model + Documents <https://tools.ietf.org/html/rfc6087>`__ +- `RFC 8340 - YANG Tree + Diagrams <https://tools.ietf.org/html/rfc8340>`__ +- `RFC 6991 - Common YANG Data + Types <https://tools.ietf.org/html/rfc6991>`__ +- `RFC 6241 - Network Configuration Protocol + (NETCONF) <https://tools.ietf.org/html/rfc6241>`__ +- `RFC 8040 - RESTCONF + Protocol <https://tools.ietf.org/html/rfc8040>`__ + +YANG models +~~~~~~~~~~~ + +- Collection of several YANG models, including models from standards + organizations such as the IETF and vendor specific models: + https://github.com/YangModels/yang +- OpenConfig: https://github.com/openconfig/public + +Presentations +~~~~~~~~~~~~~ + +- FRR Advanced Northbound API (May 2018) + + - Slides: + https://www.dropbox.com/s/zhybthruwocbqaw/netdef-frr-northbound.pdf?dl=1 + +- Ok, We Got Data Models, Now What? + + - Video: https://www.youtube.com/watch?v=2oqkiZ83vAA + - Slides: + https://www.nanog.org/sites/default/files/20161017_Alvarez_Ok_We_Got_v1.pdf + +- Data Model-Driven Management: Latest Industry and Tool Developments + + - Video: https://www.youtube.com/watch?v=n_oKGJ_jgYQ + - Slides: + https://pc.nanog.org/static/published/meetings/NANOG72/1559/20180219_Claise_Data_Modeling-Driven_Management__v1.pdf + +- Network Automation And Programmability: Reality Versus The Vendor + Hype When Considering Legacy And NFV Networks + + - Video: https://www.youtube.com/watch?v=N5wbYncUS9o + - Slides: + https://www.nanog.org/sites/default/files/1_Moore_Network_Automation_And_Programmability.pdf + +- Lightning Talk: The API is the new CLI? + + - Video: https://www.youtube.com/watch?v=ngi0erGNi58 + - Slides: + https://pc.nanog.org/static/published/meetings/NANOG72/1638/20180221_Grundemann_Lightning_Talk_The_v1.pdf + +- Lightning Talk: OpenConfig - progress toward vendor-neutral network + management + + - Video: https://www.youtube.com/watch?v=10rSUbeMmT4 + - Slides: + https://pc.nanog.org/static/published/meetings/NANOG71/1535/20171004_Shaikh_Lightning_Talk_Openconfig_v1.pdf + +- Getting started with OpenConfig + + - Video: https://www.youtube.com/watch?v=L7trUNK8NJI + - Slides: + https://pc.nanog.org/static/published/meetings/NANOG71/1456/20171003_Alvarez_Getting_Started_With_v1.pdf + +- Why NETCONF and YANG + + - Video: https://www.youtube.com/watch?v=mp4h8aSTba8 + +- NETCONF and YANG Concepts + + - Video: https://www.youtube.com/watch?v=UwYYvT7DBvg + +- NETCONF Tutorial + + - Video: https://www.youtube.com/watch?v=N4vov1mI14U + +Whitepapers +~~~~~~~~~~~ + +- Automating Network and Service Configuration Using NETCONF and YANG: + http://www.tail-f.com/wordpress/wp-content/uploads/2013/02/Tail-f-Presentation-Netconf-Yang.pdf +- Creating the Programmable Network: The Business Case for NETCONF/YANG + in Network Devices: + http://www.tail-f.com/wordpress/wp-content/uploads/2013/10/HR-Tail-f-NETCONF-WP-10-08-13.pdf +- NETCONF/YANG: What’s Holding Back Adoption & How to Accelerate It: + https://www.oneaccess-net.com/images/public/wp_heavy_reading.pdf +- Achieving Automation with YANG Modeling Technologies: + https://www.cisco.com/c/dam/en/us/products/collateral/cloud-systems-management/network-services-orchestrator/idc-achieving-automation-wp.pdf + +Blog posts and podcasts +~~~~~~~~~~~~~~~~~~~~~~~ + +- OpenConfig and IETF YANG Models: Can they converge? - + http://rob.sh/post/215/ +- OpenConfig: Standardized Models For Networking - + https://packetpushers.net/openconfig-standardized-models-networking/ +- (Podcast) OpenConfig: From Basics to Implementations - + https://blog.ipspace.net/2017/02/openconfig-from-basics-to.html +- (Podcast) How Did NETCONF Start on Software Gone Wild - + https://blog.ipspace.net/2017/12/how-did-netconf-start-on-software-gone.html +- YANG Data Models in the Industry: Current State of Affairs (March + 2018) - + https://www.claise.be/2018/03/yang-data-models-in-the-industry-current-state-of-affairs-march-2018/ +- Why Data Model-driven Telemetry is the only useful Telemetry? - + https://www.claise.be/2018/02/why-data-model-driven-telemetry-is-the-only-useful-telemetry/ +- NETCONF versus RESTCONF: Capabilitity Comparisons for Data + Model-driven Management - + https://www.claise.be/2017/10/netconf-versus-restconf-capabilitity-comparisons-for-data-model-driven-management-2/ +- An Introduction to NETCONF/YANG - + https://www.fir3net.com/Networking/Protocols/an-introduction-to-netconf-yang.html +- Network Automation and the Rise of NETCONF - + https://medium.com/@k.okasha/network-automation-and-the-rise-of-netconf-e96cc33fe28 +- YANG and the Road to a Model Driven Network - + https://medium.com/@k.okasha/yang-and-road-to-a-model-driven-network-e9e52d47148d + +Software +~~~~~~~~ + +libyang +^^^^^^^ + + libyang is a YANG data modelling language parser and toolkit written + (and providing API) in C. + +- GitHub page: https://github.com/CESNET/libyang +- Documentaion: https://netopeer.liberouter.org/doc/libyang/master/ + +pyang +^^^^^ + + pyang is a YANG validator, transformator and code generator, written + in python. It can be used to validate YANG modules for correctness, + to transform YANG modules into other formats, and to generate code + from the modules. + +- GitHub page: https://github.com/mbj4668/pyang +- Documentaion: https://github.com/mbj4668/pyang/wiki/Documentation + +ncclient +^^^^^^^^ + + ncclient is a Python library that facilitates client-side scripting + and application development around the NETCONF protocol. + +- GitHub page: https://github.com/ncclient/ncclient +- Documentaion: https://ncclient.readthedocs.io/en/latest/ + +YDK +^^^ + + ydk-gen is a developer tool that can generate API’s that are modeled + in YANG. Currently, it generates language binding for Python, Go and + C++ with planned support for other language bindings in the future. + +- GitHub pages: + + - Generator: https://github.com/CiscoDevNet/ydk-gen + - Python: https://github.com/CiscoDevNet/ydk-py + + - Python samples: https://github.com/CiscoDevNet/ydk-py-samples + + - Go: https://github.com/CiscoDevNet/ydk-go + - C++: https://github.com/CiscoDevNet/ydk-cpp + +- Documentation: + + - Python: http://ydk.cisco.com/py/docs/ + - Go: http://ydk.cisco.com/go/docs/ + - C++: http://ydk.cisco.com/cpp/docs/ + +- (Blog post) Simplifying Network Programmability with Model-Driven + APIs: + https://blogs.cisco.com/sp/simplifying-network-programmability-with-model-driven-apis +- (Video introduction) Infrastructure as a Code Using YANG, OpenConfig + and YDK: https://www.youtube.com/watch?v=G1b6vJW1R5w + +pyangbind +^^^^^^^^^ + + A plugin for pyang that creates Python bindings for a YANG model. + +- GitHub page: https://github.com/robshakir/pyangbind +- Documentation: http://pynms.io/pyangbind/ + +ConfD +^^^^^ + +- Official webpage (for ConfD Basic): + http://www.tail-f.com/confd-basic/ +- Training Videos: http://www.tail-f.com/confd-training-videos/ +- Forum: http://discuss.tail-f.com/ + +Sysrepo +^^^^^^^ + + Sysrepo is an YANG-based configuration and operational state data + store for Unix/Linux applications. + +- GitHub page: https://github.com/sysrepo/sysrepo +- Official webpage: http://www.sysrepo.org/ +- Documentation: http://www.sysrepo.org/static/doc/html/ + +Netopeer2 +^^^^^^^^^ + + Netopeer2 is a set of tools implementing network configuration tools + based on the NETCONF Protocol. This is the second generation of the + toolset, originally available as the Netopeer project. Netopeer2 is + based on the new generation of the NETCONF and YANG libraries - + libyang and libnetconf2. The Netopeer server uses sysrepo as a + NETCONF datastore implementation. + +- GitHub page: https://github.com/CESNET/Netopeer2 + +Clixon +^^^^^^ + + Clixon is an automatic configuration manager where you generate + interactive CLI, NETCONF, RESTCONF and embedded databases with + transaction support from a YANG specification. + +- GitHub page: https://github.com/clicon/clixon +- Project page: http://www.clicon.org/ diff --git a/doc/developer/northbound/northbound.rst b/doc/developer/northbound/northbound.rst new file mode 100644 index 0000000..7dddf06 --- /dev/null +++ b/doc/developer/northbound/northbound.rst @@ -0,0 +1,21 @@ +.. _northbound: + +************** +Northbound API +************** + +.. toctree:: + :maxdepth: 2 + + architecture + transactional-cli + retrofitting-configuration-commands + operational-data-rpcs-and-notifications + plugins-sysrepo + advanced-topics + yang-tools + yang-module-translator + demos + links + ppr-basic-test-topology + ppr-mpls-basic-test-topology diff --git a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst new file mode 100644 index 0000000..554bc17 --- /dev/null +++ b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst @@ -0,0 +1,565 @@ +Operational data +~~~~~~~~~~~~~~~~ + +Writing API-agnostic code for YANG-modeled operational data is +challenging. ConfD and Sysrepo, for instance, have completely different +APIs to fetch operational data. So how can we write API-agnostic +callbacks that can be used by both the ConfD and Sysrepo plugins, and +any other northbound client that might be written in the future? + +As an additional requirement, the callbacks must be designed in a way +that makes in-place XPath filtering possible. As an example, a +management client might want to retrieve only a subset of a large YANG +list (e.g. a BGP table), and for optimal performance it should be +possible to filter out the unwanted elements locally in the managed +devices instead of returning all elements and performing the filtering +on the management application. + +To meet all these requirements, the four callbacks below were introduced +in the northbound architecture: + +.. code:: c + + /* + * Operational data callback. + * + * The callback function should return the value of a specific leaf or + * inform if a typeless value (presence containers or leafs of type + * empty) exists or not. + * + * xpath + * YANG data path of the data we want to get + * + * list_entry + * pointer to list entry + * + * Returns: + * pointer to newly created yang_data structure, or NULL to indicate + * the absence of data + */ + struct yang_data *(*get_elem)(const char *xpath, void *list_entry); + + /* + * Operational data callback for YANG lists. + * + * The callback function should return the next entry in the list. The + * 'list_entry' parameter will be NULL on the first invocation. + * + * list_entry + * pointer to a list entry + * + * Returns: + * pointer to the next entry in the list, or NULL to signal that the + * end of the list was reached + */ + void *(*get_next)(void *list_entry); + + /* + * Operational data callback for YANG lists. + * + * The callback function should fill the 'keys' parameter based on the + * given list_entry. + * + * list_entry + * pointer to a list entry + * + * keys + * structure to be filled based on the attributes of the provided + * list entry + * + * Returns: + * NB_OK on success, NB_ERR otherwise + */ + int (*get_keys)(void *list_entry, struct yang_list_keys *keys); + + /* + * Operational data callback for YANG lists. + * + * The callback function should return a list entry based on the list + * keys given as a parameter. + * + * keys + * structure containing the keys of the list entry + * + * Returns: + * a pointer to the list entry if found, or NULL if not found + */ + void *(*lookup_entry)(struct yang_list_keys *keys); + +These callbacks were designed to provide maximum flexibility, and borrow +a lot of ideas from the ConfD API. Each callback does one and only one +task, they are indivisible primitives that can be combined in several +different ways to iterate over operational data. The extra flexibility +certainly has a performance cost, but it’s the price to pay if we want +to expose FRR operational data using several different management +interfaces (e.g. NETCONF via either ConfD or Sysrepo+Netopeer2). In the +future it might be possible to introduce optional callbacks that do +things like returning multiple objects at once. They would provide +enhanced performance when iterating over large lists, but their use +would be limited by the northbound plugins that can be integrated with +them. + + NOTE: using the northbound callbacks as a base, the ConfD plugin can + provide up to 100 objects between each round trip between FRR and the + *confd* daemon. Preliminary tests showed FRR taking ~7 seconds + (asynchronously, without blocking the main pthread) to return a RIP + table containing 100k routes to a NETCONF client connected to *confd* + (JSON was used as the encoding format). Work needs to be done to find + the bottlenecks and optimize this operation. + +The [[Plugins - Writing Your Own]] page explains how the northbound +plugins can fetch operational data using the aforementioned northbound +callbacks, and how in-place XPath filtering can be implemented. + +Example +^^^^^^^ + +Now let’s move to an example to show how these callbacks are implemented +in practice. The following YANG container is part of the *ietf-rip* +module and contains operational data about RIP neighbors: + +.. code:: yang + + container neighbors { + description + "Neighbor information."; + list neighbor { + key "address"; + description + "A RIP neighbor."; + leaf address { + type inet:ipv4-address; + description + "IP address that a RIP neighbor is using as its + source address."; + } + leaf last-update { + type yang:date-and-time; + description + "The time when the most recent RIP update was + received from this neighbor."; + } + leaf bad-packets-rcvd { + type yang:counter32; + description + "The number of RIP invalid packets received from + this neighbor which were subsequently discarded + for any reason (e.g. a version 0 packet, or an + unknown command type)."; + } + leaf bad-routes-rcvd { + type yang:counter32; + description + "The number of routes received from this neighbor, + in valid RIP packets, which were ignored for any + reason (e.g. unknown address family, or invalid + metric)."; + } + } + } + +We know that this is operational data because the ``neighbors`` +container is within the ``state`` container, which has the +``config false;`` property (which is applied recursively). + +As expected, the ``gen_northbound_callbacks`` tool also generates +skeleton callbacks for nodes that represent operational data: + +.. code:: c + + { + .xpath = "/frr-ripd:ripd/state/neighbors/neighbor", + .cbs.get_next = ripd_state_neighbors_neighbor_get_next, + .cbs.get_keys = ripd_state_neighbors_neighbor_get_keys, + .cbs.lookup_entry = ripd_state_neighbors_neighbor_lookup_entry, + }, + { + .xpath = "/frr-ripd:ripd/state/neighbors/neighbor/address", + .cbs.get_elem = ripd_state_neighbors_neighbor_address_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/neighbors/neighbor/last-update", + .cbs.get_elem = ripd_state_neighbors_neighbor_last_update_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd", + .cbs.get_elem = ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd", + .cbs.get_elem = ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem, + }, + +The ``/frr-ripd:ripd/state/neighbors/neighbor`` list within the +``neighbors`` container has three different callbacks that need to be +implemented. Let’s start with the first one, the ``get_next`` callback: + +.. code:: c + + static void *ripd_state_neighbors_neighbor_get_next(void *list_entry) + { + struct listnode *node; + + if (list_entry == NULL) + node = listhead(peer_list); + else + node = listnextnode((struct listnode *)list_entry); + + return node; + } + +Given a list entry, the job of this callback is to find the next element +from the list. When the ``list_entry`` parameter is NULL, then the first +element of the list should be returned. + +*ripd* uses the ``rip_peer`` structure to represent RIP neighbors, and +the ``peer_list`` global variable (linked list) is used to store all RIP +neighbors. + +In order to be able to iterate over the list of RIP neighbors, the +callback returns a ``listnode`` variable instead of a ``rip_peer`` +variable. The ``listnextnode`` macro can then be used to find the next +element from the linked list. + +Now the second callback, ``get_keys``: + +.. code:: c + + static int ripd_state_neighbors_neighbor_get_keys(void *list_entry, + struct yang_list_keys *keys) + { + struct listnode *node = list_entry; + struct rip_peer *peer = listgetdata(node); + + keys->num = 1; + (void)inet_ntop(AF_INET, &peer->addr, keys->key[0].value, + sizeof(keys->key[0].value)); + + return NB_OK; + } + +This one is easy. First, we obtain the RIP neighbor from the +``listnode`` structure. Then, we fill the ``keys`` parameter according +to the attributes of the RIP neighbor. In this case, the ``neighbor`` +YANG list has only one key: the neighbor IP address. We then use the +``inet_ntop()`` function to transform this binary IP address into a +string (the lingua franca of the FRR northbound). + +The last callback for the ``neighbor`` YANG list is the ``lookup_entry`` +callback: + +.. code:: c + + static void * + ripd_state_neighbors_neighbor_lookup_entry(struct yang_list_keys *keys) + { + struct in_addr address; + + yang_str2ipv4(keys->key[0].value, &address); + + return rip_peer_lookup(&address); + } + +This callback is the counterpart of the ``get_keys`` callback: given an +array of list keys, the associated list entry should be returned. The +``yang_str2ipv4()`` function is used to convert the list key (an IP +address) from a string to an ``in_addr`` structure. Then the +``rip_peer_lookup()`` function is used to find the list entry. + +Finally, each YANG leaf inside the ``neighbor`` list has its associated +``get_elem`` callback: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/state/neighbors/neighbor/address + */ + static struct yang_data * + ripd_state_neighbors_neighbor_address_get_elem(const char *xpath, + void *list_entry) + { + struct rip_peer *peer = list_entry; + + return yang_data_new_ipv4(xpath, &peer->addr); + } + + /* + * XPath: /frr-ripd:ripd/state/neighbors/neighbor/last-update + */ + static struct yang_data * + ripd_state_neighbors_neighbor_last_update_get_elem(const char *xpath, + void *list_entry) + { + /* TODO: yang:date-and-time is tricky */ + return NULL; + } + + /* + * XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd + */ + static struct yang_data * + ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem(const char *xpath, + void *list_entry) + { + struct rip_peer *peer = list_entry; + + return yang_data_new_uint32(xpath, peer->recv_badpackets); + } + + /* + * XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd + */ + static struct yang_data * + ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem(const char *xpath, + void *list_entry) + { + struct rip_peer *peer = list_entry; + + return yang_data_new_uint32(xpath, peer->recv_badroutes); + } + +These callbacks receive the list entry as parameter and return the +corresponding data using the ``yang_data_new_*()`` wrapper functions. +Not much to explain here. + +Iterating over operational data without blocking the main pthread +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +One of the problems we have in FRR is that some “show” commands in the +CLI can take too long, potentially long enough to the point of +triggering some protocol timeouts and bringing sessions down. + +To avoid this kind of problem, northbound clients are encouraged to do +one of the following: \* Create a separate pthread for handling requests +to fetch operational data. \* Iterate over YANG lists and leaf-lists +asynchronously, returning a maximum number of elements per time instead +of returning all elements in one shot. + +In order to handle both cases correctly, the ``get_next`` callbacks need +to use locks to prevent the YANG lists from being modified while they +are being iterated over. If that is not done, the list entry returned by +this callback can become a dangling pointer when used in another +callback. + +Currently the ConfD and Sysrepo plugins run only in the main pthread. +The plan in the short-term is to introduce a separate pthread only for +handling operational data, and use the main pthread only for handling +configuration changes, RPCs and notifications. + +RPCs and Actions +~~~~~~~~~~~~~~~~ + +The FRR northbound supports YANG RPCs and Actions through the ``rpc()`` +callback, which is documented as follows in the *lib/northbound.h* file: + +.. code:: c + + /* + * RPC and action callback. + * + * Both 'input' and 'output' are lists of 'yang_data' structures. The + * callback should fetch all the input parameters from the 'input' list, + * and add output parameters to the 'output' list if necessary. + * + * xpath + * xpath of the YANG RPC or action + * + * input + * read-only list of input parameters + * + * output + * list of output parameters to be populated by the callback + * + * Returns: + * NB_OK on success, NB_ERR otherwise + */ + int (*rpc)(const char *xpath, const struct list *input, + struct list *output); + +Note that the same callback is used for both RPCs and actions, which are +essentially the same thing. In the case of YANG actions, the ``xpath`` +parameter can be consulted to find the data node associated to the +operation. + +As part of the northbound retrofitting process, it’s suggested to model +some EXEC-level commands using YANG so that their functionality is +exposed to other management interfaces other than the CLI. As an +example, if the ``clear bgp`` command is modeled using a YANG RPC, and a +corresponding ``rpc`` callback is written, then it should be possible to +clear BGP neighbors using NETCONF and RESTCONF with that RPC (the ConfD +and Sysrepo plugins have full support for YANG RPCs and actions). + +Here’s an example of a very simple RPC modeled using YANG: + +.. code:: yang + + rpc clear-rip-route { + description + "Clears RIP routes from the IP routing table and routes + redistributed into the RIP protocol."; + } + +This RPC doesn’t have any input or output parameters. Below we can see +the implementation of the corresponding ``rpc`` callback, whose skeleton +was automatically generated by the ``gen_northbound_callbacks`` tool: + +.. code:: c + + /* + * XPath: /frr-ripd:clear-rip-route + */ + static int clear_rip_route_rpc(const char *xpath, const struct list *input, + struct list *output) + { + struct route_node *rp; + struct rip_info *rinfo; + struct list *list; + struct listnode *listnode; + + /* Clear received RIP routes */ + for (rp = route_top(rip->table); rp; rp = route_next(rp)) { + list = rp->info; + if (list == NULL) + continue; + + for (ALL_LIST_ELEMENTS_RO(list, listnode, rinfo)) { + if (!rip_route_rte(rinfo)) + continue; + + if (CHECK_FLAG(rinfo->flags, RIP_RTF_FIB)) + rip_zebra_ipv4_delete(rp); + break; + } + + if (rinfo) { + RIP_TIMER_OFF(rinfo->t_timeout); + RIP_TIMER_OFF(rinfo->t_garbage_collect); + listnode_delete(list, rinfo); + rip_info_free(rinfo); + } + + if (list_isempty(list)) { + list_delete_and_null(&list); + rp->info = NULL; + route_unlock_node(rp); + } + } + + return NB_OK; + } + +If the ``clear-rip-route`` RPC had any input parameters, they would be +available in the ``input`` list given as a parameter to the callback. +Similarly, the ``output`` list can be used to append output parameters +generated by the RPC, if any are defined in the YANG model. + +The northbound clients (CLI and northbound plugins) have the +responsibility to create and delete the ``input`` and ``output`` lists. +However, in the cases where the RPC or action doesn’t have any input or +output parameters, the northbound client can pass NULL pointers to the +``rpc`` callback to avoid creating linked lists unnecessarily. We can +see this happening in the example below: + +.. code:: c + + /* + * XPath: /frr-ripd:clear-rip-route + */ + DEFPY (clear_ip_rip, + clear_ip_rip_cmd, + "clear ip rip", + CLEAR_STR + IP_STR + "Clear IP RIP database\n") + { + return nb_cli_rpc("/frr-ripd:clear-rip-route", NULL, NULL); + } + +``nb_cli_rpc()`` is a helper function that merely finds the appropriate +``rpc`` callback based on the XPath provided in the first argument, and +map the northbound error code from the ``rpc`` callback to a vty error +code (e.g. ``CMD_SUCCESS``, ``CMD_WARNING``). The second and third +arguments provided to the function refer to the ``input`` and ``output`` +lists. In this case, both arguments are set to NULL since the YANG RPC +in question doesn’t have any input/output parameters. + +Notifications +~~~~~~~~~~~~~ + +YANG notifations are sent using the ``nb_notification_send()`` function, +documented in the *lib/northbound.h* file as follows: + +.. code:: c + + /* + * Send a YANG notification. This is a no-op unless the 'nb_notification_send' + * hook was registered by a northbound plugin. + * + * xpath + * xpath of the YANG notification + * + * arguments + * linked list containing the arguments that should be sent. This list is + * deleted after being used. + * + * Returns: + * NB_OK on success, NB_ERR otherwise + */ + extern int nb_notification_send(const char *xpath, struct list *arguments); + +The northbound doesn’t use callbacks for notifications because +notifications are generated locally and sent to the northbound clients. +This way, whenever a notification needs to be sent, it’s possible to +call the appropriate function directly instead of finding a callback +based on the XPath of the YANG notification. + +As an example, the *ietf-rip* module contains the following +notification: + +.. code:: yang + + notification authentication-failure { + description + "This notification is sent when the system + receives a PDU with the wrong authentication + information."; + leaf interface-name { + type string; + description + "Describes the name of the RIP interface."; + } + } + +The following convenience function was implemented in *ripd* to send +*authentication-failure* YANG notifications: + +.. code:: c + + /* + * XPath: /frr-ripd:authentication-failure + */ + void ripd_notif_send_auth_failure(const char *ifname) + { + const char *xpath = "/frr-ripd:authentication-failure"; + struct list *arguments; + char xpath_arg[XPATH_MAXLEN]; + struct yang_data *data; + + arguments = yang_data_list_new(); + + snprintf(xpath_arg, sizeof(xpath_arg), "%s/interface-name", xpath); + data = yang_data_new_string(xpath_arg, ifname); + listnode_add(arguments, data); + + nb_notification_send(xpath, arguments); + } + +Now sending the *authentication-failure* YANG notification should be as +simple as calling the above function and provide the appropriate +interface name. The notification will be processed by all northbound +plugins that subscribed a callback to the ``nb_notification_send`` hook. +The ConfD and Sysrepo plugins, for instance, use this hook to relay the +notifications to the *confd*/*sysrepod* daemons, which can generate +NETCONF notifications to subscribed clients. When no northbound plugin +is loaded, ``nb_notification_send()`` doesn’t do anything and the +notifications are ignored. diff --git a/doc/developer/northbound/plugins-sysrepo.rst b/doc/developer/northbound/plugins-sysrepo.rst new file mode 100644 index 0000000..186c3a0 --- /dev/null +++ b/doc/developer/northbound/plugins-sysrepo.rst @@ -0,0 +1,137 @@ +Installation +------------ + +Required dependencies +^^^^^^^^^^^^^^^^^^^^^ + +:: + + # apt-get install git cmake build-essential bison flex libpcre3-dev libev-dev \ + libavl-dev libprotobuf-c-dev protobuf-c-compiler libcmocka0 \ + libcmocka-dev doxygen libssl-dev libssl-dev libssh-dev + +libyang +^^^^^^^ + +:: + + # apt-get install libyang0.16 libyang-dev + +Sysrepo +^^^^^^^ + +:: + + $ git clone https://github.com/sysrepo/sysrepo.git + $ cd sysrepo/ + $ mkdir build; cd build + $ cmake -DCMAKE_BUILD_TYPE=Release -DGEN_LANGUAGE_BINDINGS=OFF .. && make + # make install + +libnetconf2 +^^^^^^^^^^^ + +:: + + $ git clone https://github.com/CESNET/libnetconf2.git + $ cd libnetconf2/ + $ mkdir build; cd build + $ cmake .. && make + # make install + +netopeer2 +^^^^^^^^^ + +:: + + $ git clone https://github.com/CESNET/Netopeer2.git + $ cd Netopeer2 + $ cd server + $ mkdir build; cd build + $ cmake .. && make + # make install + +**Note:** If ``make install`` fails as it can’t find +``libsysrepo.so.0.7``, then run ``ldconfig`` and try again as it might +not have updated the lib search path + +FRR +^^^ + +Build and install FRR using the ``--enable-sysrepo`` configure-time +option. + +Initialization +-------------- + +Install the FRR YANG modules in the Sysrepo datastore: + +:: + + # sysrepoctl --install /usr/local/share/yang/ietf-interfaces@2018-01-09.yang + # sysrepoctl --install /usr/local/share/yang/frr-vrf.yang + # sysrepoctl --install /usr/local/share/yang/frr-interface.yang + # sysrepoctl --install /usr/local/share/yang/frr-route-types.yang + # sysrepoctl --install /usr/local/share/yang/frr-filter.yang + # sysrepoctl --install /usr/local/share/yang/frr-route-map.yang + # sysrepoctl --install /usr/local/share/yang/frr-isisd.yang + # sysrepoctl --install /usr/local/share/yang/frr-ripd.yang + # sysrepoctl --install /usr/local/share/yang/frr-ripngd.yang + # sysrepoctl -c frr-vrf --owner frr --group frr + # sysrepoctl -c frr-interface --owner frr --group frr + # sysrepoctl -c frr-route-types --owner frr --group frr + # sysrepoctl -c frr-filter --owner frr --group frr + # sysrepoctl -c frr-route-map --owner frr --group frr + # sysrepoctl -c frr-isisd --owner frr --group frr + # sysrepoctl -c frr-ripd --owner frr --group frr + # sysrepoctl -c frr-ripngd --owner frr --group frr + +Start netopeer2-server: + +:: + + # netopeer2-server -d & + +Start the FRR daemons with the sysrepo module: + +:: + + # isisd -M sysrepo --log=stdout + +Managing the configuration +-------------------------- + +The following NETCONF scripts can be used to show and edit the FRR +configuration: +https://github.com/rzalamena/ietf-hackathon-brazil-201907/tree/master/netconf-scripts + +Example: + +:: + + # ./netconf-edit.py 127.0.0.1 + # ./netconf-get-config.py 127.0.0.1 + <?xml version="1.0" encoding="UTF-8"?><data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><isis xmlns="http://frrouting.org/yang/isisd"><instance><area-tag>testnet</area-tag><is-type>level-1</is-type></instance></isis></data> + +.. + + NOTE: the ncclient library needs to be installed first: + ``apt install -y python3-ncclient`` + +The *sysrepocfg* tool can also be used to show/edit the FRR +configuration. Example: + +:: + + # sysrepocfg --format=json --import=frr-isisd.json --datastore=running frr-isisd + # sysrepocfg --format=json --export --datastore=running frr-isisd + { + "frr-isisd:isis": { + "instance": [ + { + "area-tag": "testnet", + "is-type": "level-1" + } + ] + } + } diff --git a/doc/developer/northbound/ppr-basic-test-topology.rst b/doc/developer/northbound/ppr-basic-test-topology.rst new file mode 100644 index 0000000..a680ed7 --- /dev/null +++ b/doc/developer/northbound/ppr-basic-test-topology.rst @@ -0,0 +1,1632 @@ +Table of Contents +~~~~~~~~~~~~~~~~~ + +- `Software <#software>`__ +- `Topology <#topology>`__ +- `Configuration <#configuration>`__ + + - `CLI <#configuration-cli>`__ + - `YANG <#configuration-yang>`__ + +- `Verification - Control Plane <#verification-cplane>`__ +- `Verification - Forwarding Plane <#verification-fplane>`__ + +Software +~~~~~~~~ + +The FRR PPR implementation for IS-IS is available here: +https://github.com/opensourcerouting/frr/tree/isisd-ppr + +Topology +~~~~~~~~ + +In this topology we have an IS-IS network consisting of 12 routers. CE1 +and CE2 are the consumer edges, connected to R11 and R14, respectively. +Three hosts are connected to the CEs using only static routes. + +Router R11 advertises 6 PPR TLVs, which corresponds to three +bi-directional GRE tunnels: \* **6000:1::1 <-> 6000:2::1:** {R11 - R21 - +R22 - R23 - R14} (IPv6 Node Addresses only) \* **6000:1::2 <-> +6000:2::2:** {R11 - R21 - R32 - R41 - R33 - R23 - R14} (IPv6 Node and +Interface Addresses) \* **6000:1::3 <-> 6000:2::3:** {R11 - R21 - R99 - +R23 - R14} (misconfigured path) + +PBR rules are configured on R11 and R14 to route the traffic between +Host 1 and Host 3 using the first PPR tunnel. Traffic between Host 2 and +Host 3 uses the regular IS-IS shortest path. + +Additional information: \* Addresses in the 4000::/16 range refer to +interface addresses, where the last hextet corresponds to the node ID. +\* Addresses in the 5000::/16 range refer to loopback addresses, where +the last hextet corresponds to the node ID. \* Addresses in the +6000::/16 range refer to PPR-ID addresses. + +:: + + +-------+ +-------+ +-------+ + | | | | | | + | HOST1 | | HOST2 | | HOST3 | + | | | | | | + +---+---+ +---+---+ +---+---+ + | | | + |fd00:10:1::/64 | | + +-----+ +------+ fd00:20:1::/64| + | |fd00:10:2::/64 | + | | | + +-+--+--+ +---+---+ + | | | | + | CE1 | | CE2 | + | | | | + +---+---+ +---+---+ + | | + | | + |fd00:10:0::/64 fd00:20:0::/64| + | | + | | + +---+---+ +-------+ +-------+ +---+---+ + | |4000:101::/64| |4000:102::/64| |4000:103::/64| | + | R11 +-------------+ R12 +-------------+ R13 +-------------+ R14 | + | | | | | | | | + +---+---+ +--+-+--+ +--+-+--+ +---+---+ + | | | | | | + |4000:104::/64 | |4000:106::/64 | |4000:108::/64 | + +---------+ +--------+ +--------+ +--------+ +--------+ +---------+ + | |4000:105::/64 | |4000:107::/64 | |4000:109::/64 + | | | | | | + +--+-+--+ +--+-+--+ +--+-+--+ + | |4000:110::/64| |4000:111::/64| | + | R21 +-------------+ R22 +-------------+ R23 | + | | | | | | + +--+-+--+ +--+-+--+ +--+-+--+ + | | | | | | + | |4000:113::/64 | |4000:115::/64 | |4000:117::/64 + +---------+ +--------+ +--------+ +--------+ +--------+ +---------+ + |4000:112::/64 | |4000:114::/64 | |4000:116::/64 | + | | | | | | + +---+---+ +--+-+--+ +--+-+--+ +---+---+ + | |4000:118::/64| |4000:119::/64| |4000:120::/64| | + | R31 +-------------+ R32 +-------------+ R33 +-------------+ R34 | + | | | | | | | | + +-------+ +---+---+ +---+---+ +-------+ + | | + |4000:121::/64 | + +----------+----------+ + | + | + +---+---+ + | | + | R41 | + | | + +-------+ + +Configuration +~~~~~~~~~~~~~ + +PPR TLV processing needs to be enabled on all IS-IS routers using the +``ppr on`` command. The advertisements of all PPR TLVs is done by router +R11. + +CLI configuration +^^^^^^^^^^^^^^^^^ + +.. code:: yaml + + --- + + routers: + + host1: + links: + eth-ce1: + peer: [ce1, eth-host1] + frr: + zebra: + staticd: + config: | + interface eth-ce1 + ipv6 address fd00:10:1::1/64 + ! + ipv6 route ::/0 fd00:10:1::100 + + host2: + links: + eth-ce1: + peer: [ce1, eth-host2] + frr: + zebra: + staticd: + config: | + interface eth-ce1 + ipv6 address fd00:10:2::1/64 + ! + ipv6 route ::/0 fd00:10:2::100 + + host3: + links: + eth-ce2: + peer: [ce2, eth-host3] + frr: + zebra: + staticd: + config: | + interface eth-ce2 + ipv6 address fd00:20:1::1/64 + ! + ipv6 route ::/0 fd00:20:1::100 + + ce1: + links: + eth-host1: + peer: [host1, eth-ce1] + eth-host2: + peer: [host2, eth-ce1] + eth-rt11: + peer: [rt11, eth-ce1] + frr: + zebra: + staticd: + config: | + interface eth-host1 + ipv6 address fd00:10:1::100/64 + ! + interface eth-host2 + ipv6 address fd00:10:2::100/64 + ! + interface eth-rt11 + ipv6 address fd00:10:0::100/64 + ! + ipv6 route ::/0 fd00:10:0::11 + + ce2: + links: + eth-host3: + peer: [host3, eth-ce2] + eth-rt14: + peer: [rt14, eth-ce2] + frr: + zebra: + staticd: + config: | + interface eth-host3 + ipv6 address fd00:20:1::100/64 + ! + interface eth-rt14 + ipv6 address fd00:20:0::100/64 + ! + ipv6 route ::/0 fd00:20:0::14 + + rt11: + links: + lo-ppr: + eth-ce1: + peer: [ce1, eth-rt11] + eth-rt12: + peer: [rt12, eth-rt11] + eth-rt21: + peer: [rt21, eth-rt11] + shell: | + # GRE tunnel for preferred packets (PPR) + ip -6 tunnel add tun-ppr mode ip6gre remote 6000:2::1 local 6000:1::1 ttl 64 + ip link set dev tun-ppr up + # PBR rules + ip -6 rule add from fd00:10:1::/64 to fd00:20:1::/64 iif eth-ce1 lookup 10000 + ip -6 route add default dev tun-ppr table 10000 + frr: + zebra: + staticd: + isisd: + config: | + interface lo-ppr + ipv6 address 6000:1::1/128 + ipv6 address 6000:1::2/128 + ipv6 address 6000:1::3/128 + ! + interface lo + ipv6 address 5000::11/128 + ipv6 router isis 1 + ! + interface eth-ce1 + ipv6 address fd00:10:0::11/64 + ! + interface eth-rt12 + ipv6 address 4000:101::11/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:104::11/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + ipv6 route fd00:10::/32 fd00:10:0::100 + ! + ppr group VOIP + ppr ipv6 6000:1::1/128 prefix 5000::11/128 metric 50 + pde ipv6-node 5000::14/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::22/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::11/128 + ! + ppr ipv6 6000:2::1/128 prefix 5000::14/128 metric 50 + pde ipv6-node 5000::11/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::22/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::14/128 + ! + ! + ppr group INTERFACE_PDES + ppr ipv6 6000:1::2/128 prefix 5000::11/128 + pde ipv6-node 5000::14/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::33/128 + pde ipv6-interface 4000:121::41/64 + pde ipv6-node 5000::32/128 + pde ipv6-interface 4000:113::21/64 + pde ipv6-node 5000::11/128 + ! + ppr ipv6 6000:2::2/128 prefix 5000::14/128 + pde ipv6-node 5000::11/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::32/128 + pde ipv6-interface 4000:121::41/64 + pde ipv6-node 5000::33/128 + pde ipv6-interface 4000:116::23/64 + pde ipv6-node 5000::14/128 + ! + ! + ppr group BROKEN + ppr ipv6 6000:1::3/128 prefix 5000::11/128 metric 1500 + pde ipv6-node 5000::14/128 + pde ipv6-node 5000::23/128 + ! non-existing node!!! + pde ipv6-node 5000::99/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::11/128 + ! + ppr ipv6 6000:2::3/128 prefix 5000::14/128 metric 1500 + pde ipv6-node 5000::11/128 + pde ipv6-node 5000::21/128 + ! non-existing node!!! + pde ipv6-node 5000::99/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::14/128 + ! + ! + router isis 1 + net 49.0000.0000.0000.0011.00 + is-type level-1 + topology ipv6-unicast + ppr on + ppr advertise VOIP + ppr advertise INTERFACE_PDES + ppr advertise BROKEN + ! + + rt12: + links: + eth-rt11: + peer: [rt11, eth-rt12] + eth-rt13: + peer: [rt13, eth-rt12] + eth-rt21: + peer: [rt21, eth-rt12] + eth-rt22: + peer: [rt22, eth-rt12] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::12/128 + ipv6 router isis 1 + ! + interface eth-rt11 + ipv6 address 4000:101::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt13 + ipv6 address 4000:102::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:105::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:106::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0012.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt13: + links: + eth-rt12: + peer: [rt12, eth-rt13] + eth-rt14: + peer: [rt14, eth-rt13] + eth-rt22: + peer: [rt22, eth-rt13] + eth-rt23: + peer: [rt23, eth-rt13] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::13/128 + ipv6 router isis 1 + ! + interface eth-rt12 + ipv6 address 4000:102::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt14 + ipv6 address 4000:103::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:107::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:108::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0013.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt14: + links: + lo-ppr: + eth-ce2: + peer: [ce2, eth-rt14] + eth-rt13: + peer: [rt13, eth-rt14] + eth-rt23: + peer: [rt23, eth-rt14] + shell: | + # GRE tunnel for preferred packets (PPR) + ip -6 tunnel add tun-ppr mode ip6gre remote 6000:1::1 local 6000:2::1 ttl 64 + ip link set dev tun-ppr up + # PBR rules + ip -6 rule add from fd00:20:1::/64 to fd00:10:1::/64 iif eth-ce2 lookup 10000 + ip -6 route add default dev tun-ppr table 10000 + frr: + zebra: + staticd: + isisd: + config: | + interface lo-ppr + ipv6 address 6000:2::1/128 + ipv6 address 6000:2::2/128 + ipv6 address 6000:2::3/128 + ! + interface lo + ipv6 address 5000::14/128 + ipv6 router isis 1 + ! + interface eth-ce2 + ipv6 address fd00:20:0::14/64 + ! + interface eth-rt13 + ipv6 address 4000:103::14/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:109::14/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + ipv6 route fd00:20::/32 fd00:20:0::100 + ! + router isis 1 + net 49.0000.0000.0000.0014.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt21: + links: + eth-rt11: + peer: [rt11, eth-rt21] + eth-rt12: + peer: [rt12, eth-rt21] + eth-rt22: + peer: [rt22, eth-rt21] + eth-rt31: + peer: [rt31, eth-rt21] + eth-rt32: + peer: [rt32, eth-rt21] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::21/128 + ipv6 router isis 1 + ! + interface eth-rt11 + ipv6 address 4000:104::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt12 + ipv6 address 4000:105::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:110::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt31 + ipv6 address 4000:112::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:113::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0021.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt22: + links: + eth-rt12: + peer: [rt12, eth-rt22] + eth-rt13: + peer: [rt13, eth-rt22] + eth-rt21: + peer: [rt21, eth-rt22] + eth-rt23: + peer: [rt23, eth-rt22] + eth-rt32: + peer: [rt32, eth-rt22] + eth-rt33: + peer: [rt33, eth-rt22] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::22/128 + ipv6 router isis 1 + ! + interface eth-rt12 + ipv6 address 4000:106::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt13 + ipv6 address 4000:107::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:110::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:111::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:114::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:115::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0022.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt23: + links: + eth-rt13: + peer: [rt13, eth-rt23] + eth-rt14: + peer: [rt14, eth-rt23] + eth-rt22: + peer: [rt22, eth-rt23] + eth-rt33: + peer: [rt33, eth-rt23] + eth-rt34: + peer: [rt34, eth-rt23] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::23/128 + ipv6 router isis 1 + ! + interface eth-rt13 + ipv6 address 4000:108::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt14 + ipv6 address 4000:109::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:111::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:116::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt34 + ipv6 address 4000:117::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0023.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt31: + links: + eth-rt21: + peer: [rt21, eth-rt31] + eth-rt32: + peer: [rt32, eth-rt31] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::31/128 + ipv6 router isis 1 + ! + interface eth-rt21 + ipv6 address 4000:112::31/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:118::31/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0031.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt32: + links: + eth-rt21: + peer: [rt21, eth-rt32] + eth-rt22: + peer: [rt22, eth-rt32] + eth-rt31: + peer: [rt31, eth-rt32] + eth-rt33: + peer: [rt33, eth-rt32] + eth-sw1: + peer: [sw1, eth-rt32] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::32/128 + ipv6 router isis 1 + ! + interface eth-rt21 + ipv6 address 4000:113::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:114::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt31 + ipv6 address 4000:118::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:119::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-sw1 + ipv6 address 4000:121::32/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0032.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt33: + links: + eth-rt22: + peer: [rt22, eth-rt33] + eth-rt23: + peer: [rt23, eth-rt33] + eth-rt32: + peer: [rt32, eth-rt33] + eth-rt34: + peer: [rt34, eth-rt33] + eth-sw1: + peer: [sw1, eth-rt33] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::33/128 + ipv6 router isis 1 + ! + interface eth-rt22 + ipv6 address 4000:115::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:116::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:119::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt34 + ipv6 address 4000:120::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-sw1 + ipv6 address 4000:121::33/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0033.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt34: + links: + eth-rt23: + peer: [rt23, eth-rt34] + eth-rt33: + peer: [rt33, eth-rt34] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::34/128 + ipv6 router isis 1 + ! + interface eth-rt23 + ipv6 address 4000:117::34/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:120::34/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0034.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + rt41: + links: + eth-sw1: + peer: [sw1, eth-rt41] + frr: + zebra: + isisd: + config: | + interface lo + ipv6 address 5000::41/128 + ipv6 router isis 1 + ! + interface eth-sw1 + ipv6 address 4000:121::41/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0041.00 + is-type level-1 + topology ipv6-unicast + ppr on + ! + + switches: + sw1: + links: + eth-rt32: + peer: [rt32, eth-sw1] + eth-rt33: + peer: [rt33, eth-sw1] + eth-rt41: + peer: [rt41, eth-sw1] + + frr: + base-config: | + hostname %(node) + password 1 + log file %(logdir)/%(node).log + log commands + ! + debug zebra rib + debug isis ppr + debug isis events + debug isis route-events + debug isis spf-events + debug isis lsp-gen + ! + +YANG +^^^^ + +PPR can also be configured using NETCONF, RESTCONF and gRPC based on the +following YANG models: \* +`frr-ppr.yang <https://github.com/opensourcerouting/frr/blob/isisd-ppr/yang/frr-ppr.yang>`__ +\* +`frr-isisd.yang <https://github.com/opensourcerouting/frr/blob/isisd-ppr/yang/frr-isisd.yang>`__ + +As an example, here’s R11 configuration in the XML format: + +.. code:: xml + + <lib xmlns="http://frrouting.org/yang/interface"> + <interface> + <name>lo-ppr</name> + <vrf>default</vrf> + </interface> + <interface> + <name>lo</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + </isis> + </interface> + <interface> + <name>eth-ce1</name> + <vrf>default</vrf> + </interface> + <interface> + <name>eth-rt12</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + <hello> + <multiplier> + <level-1>3</level-1> + <level-2>3</level-2> + </multiplier> + </hello> + <network-type>point-to-point</network-type> + </isis> + </interface> + <interface> + <name>eth-rt21</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + <hello> + <multiplier> + <level-1>3</level-1> + <level-2>3</level-2> + </multiplier> + </hello> + <network-type>point-to-point</network-type> + </isis> + </interface> + </lib> + <ppr xmlns="http://frrouting.org/yang/ppr"> + <group> + <name>VOIP</name> + <ipv6> + <ppr-id>6000:1::1/128</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::22/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>50</ppr-metric> + </attributes> + </ipv6> + <ipv6> + <ppr-id>6000:2::1/128</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::22/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>50</ppr-metric> + </attributes> + </ipv6> + </group> + <group> + <name>INTERFACE_PDES</name> + <ipv6> + <ppr-id>6000:1::2/128</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::33/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>4000:121::41/64</pde-id> + <pde-id-type>ipv6-interface</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::32/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>4000:113::21/64</pde-id> + <pde-id-type>ipv6-interface</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </ipv6> + <ipv6> + <ppr-id>6000:2::2/128</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::32/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>4000:121::41/64</pde-id> + <pde-id-type>ipv6-interface</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::33/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>4000:116::23/64</pde-id> + <pde-id-type>ipv6-interface</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </ipv6> + </group> + <group> + <name>BROKEN</name> + <ipv6> + <ppr-id>6000:1::3/128</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::99/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>1500</ppr-metric> + </attributes> + </ipv6> + <ipv6> + <ppr-id>6000:2::3/128</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::99/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>1500</ppr-metric> + </attributes> + </ipv6> + </group> + </ppr> + <isis xmlns="http://frrouting.org/yang/isisd"> + <instance> + <area-tag>1</area-tag> + <area-address>49.0000.0000.0000.0011.00</area-address> + <multi-topology> + <ipv6-unicast> + </ipv6-unicast> + </multi-topology> + <ppr> + <enable>true</enable> + <ppr-advertise> + <name>VOIP</name> + </ppr-advertise> + <ppr-advertise> + <name>INTERFACE_PDES</name> + </ppr-advertise> + <ppr-advertise> + <name>BROKEN</name> + </ppr-advertise> + </ppr> + </instance> + </isis> + +Verification - Control Plane +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Verify that R11 has flooded the PPR TLVs correctly to all IS-IS routers: + +:: + + # show isis database detail 0000.0000.0011 + Area 1: + IS-IS Level-1 link-state database: + LSP ID PduLen SeqNumber Chksum Holdtime ATT/P/OL + debian.00-00 1233 0x00000009 0x7bd4 683 0/0/0 + Protocols Supported: IPv4, IPv6 + Area Address: 49.0000 + MT Router Info: ipv4-unicast + MT Router Info: ipv6-unicast + Hostname: debian + MT Reachability: 0000.0000.0012.00 (Metric: 10) ipv6-unicast + MT Reachability: 0000.0000.0021.00 (Metric: 10) ipv6-unicast + MT IPv6 Reachability: 5000::11/128 (Metric: 10) ipv6-unicast + MT IPv6 Reachability: 4000:101::/64 (Metric: 10) ipv6-unicast + MT IPv6 Reachability: 4000:104::/64 (Metric: 10) ipv6-unicast + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 6000:1::3/128 (Native IPv6) + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::99/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 1500 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 6000:2::3/128 (Native IPv6) + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::99/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 1500 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 6000:1::2/128 (Native IPv6) + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::33/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 4000:121::41 (IPv6 Interface Address), L:0 N:0 E:0 + PDE: 5000::32/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 4000:113::21 (IPv6 Interface Address), L:0 N:0 E:0 + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 0 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 6000:2::2/128 (Native IPv6) + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::32/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 4000:121::41 (IPv6 Interface Address), L:0 N:0 E:0 + PDE: 5000::33/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 4000:116::23 (IPv6 Interface Address), L:0 N:0 E:0 + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 0 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 6000:1::1/128 (Native IPv6) + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::22/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 50 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 6000:2::1/128 (Native IPv6) + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::22/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 50 + +The PPR TLVs can also be seen using a modified version of Wireshark as +seen below: + +.. figure:: https://user-images.githubusercontent.com/931662/61582441-9551e500-ab01-11e9-8f6f-400ee3fba927.png + :alt: s2 + + s2 + +Using the ``show isis ppr`` command, verify that all routers installed +the PPR-IDs for the paths they are part of. Example: + +Router RT11 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + -------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Tail-End - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Tail-End - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Tail-End - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Head-End Up 00:45:41 + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Head-End Up 00:45:41 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Head-End Up 00:45:41 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:2::1/128 [115/50] via fe80::c2a:54ff:fe39:bff7, eth-rt21, 00:01:33 + I>* 6000:2::2/128 [115/0] via fe80::c2a:54ff:fe39:bff7, eth-rt21, 00:01:33 + I>* 6000:2::3/128 [115/1500] via fe80::c2a:54ff:fe39:bff7, eth-rt21, 00:01:33 + +Router RT12 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Off-Path - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Off-Path - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT13 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Off-Path - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Off-Path - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT14 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + -------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Head-End Up 00:45:45 + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Head-End Up 00:45:45 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Head-End Up 00:45:45 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Tail-End - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Tail-End - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Tail-End - - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::58ea:78ff:fe00:92c1, eth-rt23, 00:01:36 + I>* 6000:1::2/128 [115/0] via fe80::58ea:78ff:fe00:92c1, eth-rt23, 00:01:36 + I>* 6000:1::3/128 [115/1500] via fe80::58ea:78ff:fe00:92c1, eth-rt23, 00:01:36 + +Router RT21 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:45:46 + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Mid-Point Up 00:45:46 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Mid-Point Up 00:45:46 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:45:46 + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Mid-Point Up 00:45:46 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Mid-Point Down - + + # show isis ppr id ipv6 6000:2::3/128 detail + Area 1: + PPR-ID: 6000:2::3/128 (Native IPv6) + PPR-Prefix: 5000::14/128 + PDEs: + 5000::11/128 (IPv6 Node Address) + 5000::21/128 (IPv6 Node Address) [LOCAL] + 5000::99/128 (IPv6 Node Address) [NEXT] + 5000::23/128 (IPv6 Node Address) + 5000::14/128 (IPv6 Node Address) + Attributes: + Metric: 1500 + Position: Mid-Point + Originator: 0000.0000.0011 + Level: L1 + Algorithm: 1 + MT-ID: ipv4-unicast + Status: Down: PDE is unreachable + Last change: 00:00:37 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::142e:79ff:feeb:cffc, eth-rt11, 00:01:38 + I>* 6000:1::2/128 [115/0] via fe80::142e:79ff:feeb:cffc, eth-rt11, 00:01:38 + I>* 6000:1::3/128 [115/1500] via fe80::142e:79ff:feeb:cffc, eth-rt11, 00:01:38 + I>* 6000:2::1/128 [115/50] via fe80::c88e:7fff:fe5f:a08d, eth-rt22, 00:01:38 + I>* 6000:2::2/128 [115/0] via fe80::8b2:9eff:fe98:f66a, eth-rt32, 00:01:38 + +Router RT22 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:45:47 + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Off-Path - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:45:47 + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Off-Path - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::2cb5:edff:fe60:29b1, eth-rt21, 00:01:38 + I>* 6000:2::1/128 [115/50] via fe80::e8d9:63ff:fea3:177b, eth-rt23, 00:01:38 + +Router RT23 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:45:49 + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Mid-Point Up 00:45:49 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Mid-Point Down - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:45:49 + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Mid-Point Up 00:45:49 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Mid-Point Up 00:45:49 + + # show isis ppr id ipv6 6000:1::3/128 detail + Area 1: + PPR-ID: 6000:1::3/128 (Native IPv6) + PPR-Prefix: 5000::11/128 + PDEs: + 5000::14/128 (IPv6 Node Address) + 5000::23/128 (IPv6 Node Address) [LOCAL] + 5000::99/128 (IPv6 Node Address) [NEXT] + 5000::21/128 (IPv6 Node Address) + 5000::11/128 (IPv6 Node Address) + Attributes: + Metric: 1500 + Position: Mid-Point + Originator: 0000.0000.0011 + Level: L1 + Algorithm: 1 + MT-ID: ipv4-unicast + Status: Down: PDE is unreachable + Last change: 00:02:50 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::d09f:1bff:fe31:e9c9, eth-rt22, 00:01:40 + I>* 6000:1::2/128 [115/0] via fe80::c0c3:b3ff:fe9f:b5d3, eth-rt33, 00:01:40 + I>* 6000:2::1/128 [115/50] via fe80::f40a:66ff:fefc:5c32, eth-rt14, 00:01:40 + I>* 6000:2::2/128 [115/0] via fe80::f40a:66ff:fefc:5c32, eth-rt14, 00:01:40 + I>* 6000:2::3/128 [115/1500] via fe80::f40a:66ff:fefc:5c32, eth-rt14, 00:01:40 + +Router RT31 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Off-Path - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Off-Path - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT32 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Mid-Point Up 00:45:51 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Mid-Point Up 00:45:51 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::2/128 [115/0] via 4000:113::21, eth-rt21, 00:01:42 + I>* 6000:2::2/128 [115/0] via 4000:121::41, eth-sw1, 00:01:42 + +Router RT33 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Mid-Point Up 00:45:52 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Mid-Point Up 00:45:52 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::2/128 [115/0] via 4000:121::41, eth-sw1, 00:01:43 + I>* 6000:2::2/128 [115/0] via 4000:116::23, eth-rt23, 00:01:43 + +Router RT34 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Off-Path - - + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Off-Path - - + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT41 +''''''''''' + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:1::2/128 (Native IPv6) 5000::11/128 0 Mid-Point Up 00:45:55 + 1 L1 6000:1::3/128 (Native IPv6) 5000::11/128 1500 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + 1 L1 6000:2::2/128 (Native IPv6) 5000::14/128 0 Mid-Point Up 00:45:55 + 1 L1 6000:2::3/128 (Native IPv6) 5000::14/128 1500 Off-Path - - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::2/128 [115/0] via fe80::b4b9:60ff:feee:3c73, eth-sw1, 00:01:46 + I>* 6000:2::2/128 [115/0] via fe80::bc2a:d9ff:fe65:97f2, eth-sw1, 00:01:46 + +As it can be seen by the output of ``show isis ppr id ipv6 ... detail``, +routers R21 and R23 couldn’t install the third PPR path because of an +unreachable PDE (configuration error). + +Verification - Forwarding Plane +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On Router R11, use the ``traceroute`` tool to ensure that the PPR paths +were installed correctly in the network: + +:: + + root@rt11:~# traceroute 6000:2::1 + traceroute to 6000:2::1 (6000:2::1), 30 hops max, 80 byte packets + 1 4000:104::21 (4000:104::21) 0.612 ms 0.221 ms 0.241 ms + 2 4000:110::22 (4000:110::22) 0.257 ms 0.113 ms 0.105 ms + 3 4000:111::23 (4000:111::23) 0.257 ms 0.151 ms 0.098 ms + 4 6000:2::1 (6000:2::1) 0.346 ms 0.139 ms 0.100 ms + root@rt11:~# + root@rt11:~# traceroute 6000:2::2 + traceroute to 6000:2::2 (6000:2::2), 30 hops max, 80 byte packets + 1 4000:104::21 (4000:104::21) 4.383 ms 4.148 ms 0.044 ms + 2 4000:113::32 (4000:113::32) 0.272 ms 0.065 ms 0.064 ms + 3 4000:121::41 (4000:121::41) 0.263 ms 0.101 ms 0.086 ms + 4 4000:115::33 (4000:115::33) 0.351 ms 4000:119::33 (4000:119::33) 0.249 ms 4000:115::33 (4000:115::33) 0.153 ms + 5 4000:111::23 (4000:111::23) 0.232 ms 0.293 ms 0.131 ms + 6 6000:2::2 (6000:2::2) 0.184 ms 0.212 ms 0.140 ms + root@rt11:~# + root@rt11:~# traceroute 6000:2::3 + traceroute to 6000:2::3 (6000:2::3), 30 hops max, 80 byte packets + 1 4000:104::21 (4000:104::21) 1.537 ms !N 1.347 ms !N 1.075 ms !N + +The failure on the third traceroute is expected since the 6000:2::3 +PPR-ID is misconfigured. + +Now ping Host 3 from Host 1 and use tcpdump or wireshark to verify that +the ICMP packets are being tunneled using GRE and following the {R11 - +R21 - R22 - R23 - R14} path. Here’s a wireshark capture between R11 and +R21: + +.. figure:: https://user-images.githubusercontent.com/931662/61582398-d4cc0180-ab00-11e9-83a8-d219f98010b9.png + :alt: s1 + + s1 + +Using ``traceroute`` it’s also possible to see that the ICMP packets are +being tunneled through the IS-IS network: + +:: + + root@host1:~# traceroute fd00:20:1::1 -s fd00:10:1::1 + traceroute to fd00:20:1::1 (fd00:20:1::1), 30 hops max, 80 byte packets + 1 fd00:10:1::100 (fd00:10:1::100) 0.354 ms 0.092 ms 0.031 ms + 2 fd00:10::11 (fd00:10::11) 0.125 ms 0.022 ms 0.026 ms + 3 * * * + 4 * * * + 5 fd00:20:1::1 (fd00:20:1::1) 0.235 ms 0.106 ms 0.091 ms diff --git a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst new file mode 100644 index 0000000..cedb795 --- /dev/null +++ b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst @@ -0,0 +1,1991 @@ +Table of Contents +~~~~~~~~~~~~~~~~~ + +- `Software <#software>`__ +- `Topology <#topology>`__ +- `Configuration <#configuration>`__ + + - `CLI <#configuration-cli>`__ + - `YANG <#configuration-yang>`__ + +- `Verification - Control Plane <#verification-cplane>`__ +- `Verification - Forwarding Plane <#verification-fplane>`__ + +Software +~~~~~~~~ + +The FRR PPR implementation for IS-IS is available here: +https://github.com/opensourcerouting/frr/tree/isisd-ppr-sr + +Topology +~~~~~~~~ + +In this topology we have an IS-IS network consisting of 12 routers. CE1 +and CE2 are the consumer edges, connected to R11 and R14, respectively. +Three hosts are connected to the CEs using only static routes. + +Router R11 advertises 6 PPR TLVs: \* **IPv6 prefixes 6000:1::1/128 and +6000:2::1/128:** {R11 - R21 - R22 - R23 - R14} (IPv6 Node Addresses). \* +**MPLS SR Prefix-SIDs 500 and 501:** {R11 - R21 - R22 - R23 - R14} (SR +Prefix-SIDs). \* **MPLS SR Prefix-SIDs 502 and 503:** {R11 - R21 - R31 - +R32 - R41 - R33 - R34 - R23 - R14} (SR Prefix-SIDs) + +PBR rules are configured on R11 and R14 to route the traffic between +Host 1 and Host 3 using the first PPR tunnel, whereas all other traffic +between CE1 and CE2 uses the second PPR tunnel. + +Additional information: \* Addresses in the 4000::/16 range refer to +interface addresses, where the last hextet corresponds to the node ID. +\* Addresses in the 5000::/16 range refer to loopback addresses, where +the last hextet corresponds to the node ID. \* Addresses in the +6000::/16 range refer to PPR-ID addresses. + +:: + + +-------+ +-------+ +-------+ + | | | | | | + | HOST1 | | HOST2 | | HOST3 | + | | | | | | + +---+---+ +---+---+ +---+---+ + | | | + |fd00:10:1::/64 | | + +-----+ +------+ fd00:20:1::/64| + | |fd00:10:2::/64 | + | | | + +-+--+--+ +---+---+ + | | | | + | CE1 | | CE2 | + | | | | + +---+---+ +---+---+ + | | + | | + |fd00:10:0::/64 fd00:20:0::/64| + | | + | | + +---+---+ +-------+ +-------+ +---+---+ + | |4000:101::/64| |4000:102::/64| |4000:103::/64| | + | R11 +-------------+ R12 +-------------+ R13 +-------------+ R14 | + | | | | | | | | + +---+---+ +--+-+--+ +--+-+--+ +---+---+ + | | | | | | + |4000:104::/64 | |4000:106::/64 | |4000:108::/64 | + +---------+ +--------+ +--------+ +--------+ +--------+ +---------+ + | |4000:105::/64 | |4000:107::/64 | |4000:109::/64 + | | | | | | + +--+-+--+ +--+-+--+ +--+-+--+ + | |4000:110::/64| |4000:111::/64| | + | R21 +-------------+ R22 +-------------+ R23 | + | | | | | | + +--+-+--+ +--+-+--+ +--+-+--+ + | | | | | | + | |4000:113::/64 | |4000:115::/64 | |4000:117::/64 + +---------+ +--------+ +--------+ +--------+ +--------+ +---------+ + |4000:112::/64 | |4000:114::/64 | |4000:116::/64 | + | | | | | | + +---+---+ +--+-+--+ +--+-+--+ +---+---+ + | |4000:118::/64| |4000:119::/64| |4000:120::/64| | + | R31 +-------------+ R32 +-------------+ R33 +-------------+ R34 | + | | | | | | | | + +-------+ +---+---+ +---+---+ +-------+ + | | + |4000:121::/64 | + +----------+----------+ + | + | + +---+---+ + | | + | R41 | + | | + +-------+ + +Configuration +~~~~~~~~~~~~~ + +PPR TLV processing needs to be enabled on all IS-IS routers using the +``ppr on`` command. The advertisements of all PPR TLVs is done by router +R11. + +CLI configuration +^^^^^^^^^^^^^^^^^ + +.. code:: yaml + + --- + + routers: + + host1: + links: + eth-ce1: + peer: [ce1, eth-host1] + frr: + zebra: + staticd: + config: | + interface eth-ce1 + ipv6 address fd00:10:1::1/64 + ! + ipv6 route ::/0 fd00:10:1::100 + + host2: + links: + eth-ce1: + peer: [ce1, eth-host2] + frr: + zebra: + staticd: + config: | + interface eth-ce1 + ipv6 address fd00:10:2::1/64 + ! + ipv6 route ::/0 fd00:10:2::100 + + host3: + links: + eth-ce2: + peer: [ce2, eth-host3] + frr: + zebra: + staticd: + config: | + interface eth-ce2 + ipv6 address fd00:20:1::1/64 + ! + ipv6 route ::/0 fd00:20:1::100 + + ce1: + links: + eth-host1: + peer: [host1, eth-ce1] + eth-host2: + peer: [host2, eth-ce1] + eth-rt11: + peer: [rt11, eth-ce1] + frr: + zebra: + staticd: + config: | + interface eth-host1 + ipv6 address fd00:10:1::100/64 + ! + interface eth-host2 + ipv6 address fd00:10:2::100/64 + ! + interface eth-rt11 + ipv6 address fd00:10:0::100/64 + ! + ipv6 route ::/0 fd00:10:0::11 label 16501 + + ce2: + links: + eth-host3: + peer: [host3, eth-ce2] + eth-rt14: + peer: [rt14, eth-ce2] + frr: + zebra: + staticd: + config: | + interface eth-host3 + ipv6 address fd00:20:1::100/64 + ! + interface eth-rt14 + ipv6 address fd00:20:0::100/64 + ! + ipv6 route ::/0 fd00:20:0::14 label 16500 + + rt11: + links: + lo: + mpls: yes + lo-ppr: + eth-ce1: + peer: [ce1, eth-rt11] + mpls: yes + eth-rt12: + peer: [rt12, eth-rt11] + mpls: yes + eth-rt21: + peer: [rt21, eth-rt11] + mpls: yes + shell: | + # GRE tunnel for preferred packets (PPR) + ip -6 tunnel add tun-ppr mode ip6gre remote 6000:2::1 local 6000:1::1 ttl 64 + ip link set dev tun-ppr up + # PBR rules + ip -6 rule add from fd00:10:1::/64 to fd00:20:1::/64 iif eth-ce1 lookup 10000 + ip -6 route add default dev tun-ppr table 10000 + frr: + zebra: + staticd: + isisd: + config: | + interface lo-ppr + ipv6 address 6000:1::1/128 + ! + interface lo + ip address 10.0.0.11/32 + ipv6 address 5000::11/128 + ipv6 router isis 1 + ! + interface eth-ce1 + ipv6 address fd00:10:0::11/64 + ! + interface eth-rt12 + ipv6 address 4000:101::11/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:104::11/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + ipv6 route fd00:10::/32 fd00:10:0::100 + ! + ppr group PPR_IPV6 + ppr ipv6 6000:1::1/128 prefix 5000::11/128 metric 50 + pde ipv6-node 5000::14/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::22/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::11/128 + ! + ppr ipv6 6000:2::1/128 prefix 5000::14/128 metric 50 + pde ipv6-node 5000::11/128 + pde ipv6-node 5000::21/128 + pde ipv6-node 5000::22/128 + pde ipv6-node 5000::23/128 + pde ipv6-node 5000::14/128 + ! + ! + ppr group PPR_MPLS_1 + ppr mpls 500 prefix 5000::11/128 + pde prefix-sid 14 + pde prefix-sid 23 + pde prefix-sid 22 + pde prefix-sid 21 + pde prefix-sid 11 + ! + ppr mpls 501 prefix 5000::14/128 + pde prefix-sid 11 + pde prefix-sid 21 + pde prefix-sid 22 + pde prefix-sid 23 + pde prefix-sid 14 + ! + ! + ppr group PPR_MPLS_2 + ppr mpls 502 prefix 5000::11/128 + pde prefix-sid 14 + pde prefix-sid 23 + pde prefix-sid 34 + pde prefix-sid 33 + pde prefix-sid 41 + pde prefix-sid 32 + pde prefix-sid 31 + pde prefix-sid 21 + pde prefix-sid 11 + ! + ppr mpls 503 prefix 5000::14/128 + pde prefix-sid 11 + pde prefix-sid 21 + pde prefix-sid 31 + pde prefix-sid 32 + pde prefix-sid 41 + pde prefix-sid 33 + pde prefix-sid 34 + pde prefix-sid 23 + pde prefix-sid 14 + ! + ! + router isis 1 + net 49.0000.0000.0000.0011.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::11/128 index 11 no-php-flag + ppr on + ppr advertise PPR_IPV6 + ppr advertise PPR_MPLS_1 + ppr advertise PPR_MPLS_2 + ! + + rt12: + links: + lo: + mpls: yes + eth-rt11: + peer: [rt11, eth-rt12] + mpls: yes + eth-rt13: + peer: [rt13, eth-rt12] + mpls: yes + eth-rt21: + peer: [rt21, eth-rt12] + mpls: yes + eth-rt22: + peer: [rt22, eth-rt12] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.12/32 + ipv6 address 5000::12/128 + ipv6 router isis 1 + ! + interface eth-rt11 + ipv6 address 4000:101::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt13 + ipv6 address 4000:102::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:105::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:106::12/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0012.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::12/128 index 12 no-php-flag + ppr on + ! + + rt13: + links: + lo: + mpls: yes + eth-rt12: + peer: [rt12, eth-rt13] + mpls: yes + eth-rt14: + peer: [rt14, eth-rt13] + mpls: yes + eth-rt22: + peer: [rt22, eth-rt13] + mpls: yes + eth-rt23: + peer: [rt23, eth-rt13] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.13/32 + ipv6 address 5000::13/128 + ipv6 router isis 1 + ! + interface eth-rt12 + ipv6 address 4000:102::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt14 + ipv6 address 4000:103::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:107::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:108::13/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0013.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::13/128 index 13 no-php-flag + ppr on + ! + + rt14: + links: + lo: + mpls: yes + lo-ppr: + eth-ce2: + peer: [ce2, eth-rt14] + mpls: yes + eth-rt13: + peer: [rt13, eth-rt14] + mpls: yes + eth-rt23: + peer: [rt23, eth-rt14] + mpls: yes + shell: | + # GRE tunnel for preferred packets (PPR) + ip -6 tunnel add tun-ppr mode ip6gre remote 6000:1::1 local 6000:2::1 ttl 64 + ip link set dev tun-ppr up + # PBR rules + ip -6 rule add from fd00:20:1::/64 to fd00:10:1::/64 iif eth-ce2 lookup 10000 + ip -6 route add default dev tun-ppr table 10000 + frr: + zebra: + staticd: + isisd: + config: | + interface lo-ppr + ipv6 address 6000:2::1/128 + ! + interface lo + ip address 10.0.0.14/32 + ipv6 address 5000::14/128 + ipv6 router isis 1 + ! + interface eth-ce2 + ipv6 address fd00:20:0::14/64 + ! + interface eth-rt13 + ipv6 address 4000:103::14/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:109::14/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + ipv6 route fd00:20::/32 fd00:20:0::100 + ! + router isis 1 + net 49.0000.0000.0000.0014.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::14/128 index 14 no-php-flag + ppr on + ! + + rt21: + links: + lo: + mpls: yes + eth-rt11: + peer: [rt11, eth-rt21] + mpls: yes + eth-rt12: + peer: [rt12, eth-rt21] + mpls: yes + eth-rt22: + peer: [rt22, eth-rt21] + mpls: yes + eth-rt31: + peer: [rt31, eth-rt21] + mpls: yes + eth-rt32: + peer: [rt32, eth-rt21] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.21/32 + ipv6 address 5000::21/128 + ipv6 router isis 1 + ! + interface eth-rt11 + ipv6 address 4000:104::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt12 + ipv6 address 4000:105::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:110::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt31 + ipv6 address 4000:112::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:113::21/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0021.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::21/128 index 21 no-php-flag + ppr on + ! + + rt22: + links: + lo: + mpls: yes + eth-rt12: + peer: [rt12, eth-rt22] + mpls: yes + eth-rt13: + peer: [rt13, eth-rt22] + mpls: yes + eth-rt21: + peer: [rt21, eth-rt22] + mpls: yes + eth-rt23: + peer: [rt23, eth-rt22] + mpls: yes + eth-rt32: + peer: [rt32, eth-rt22] + mpls: yes + eth-rt33: + mpls: yes + peer: [rt33, eth-rt22] + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.22/32 + ipv6 address 5000::22/128 + ipv6 router isis 1 + ! + interface eth-rt12 + ipv6 address 4000:106::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt13 + ipv6 address 4000:107::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt21 + ipv6 address 4000:110::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:111::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:114::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:115::22/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0022.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::22/128 index 22 no-php-flag + ppr on + ! + + rt23: + links: + lo: + mpls: yes + eth-rt13: + peer: [rt13, eth-rt23] + mpls: yes + eth-rt14: + peer: [rt14, eth-rt23] + mpls: yes + eth-rt22: + peer: [rt22, eth-rt23] + mpls: yes + eth-rt33: + peer: [rt33, eth-rt23] + mpls: yes + eth-rt34: + peer: [rt34, eth-rt23] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.23/32 + ipv6 address 5000::23/128 + ipv6 router isis 1 + ! + interface eth-rt13 + ipv6 address 4000:108::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt14 + ipv6 address 4000:109::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:111::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:116::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt34 + ipv6 address 4000:117::23/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0023.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing global-block 20000 27999 + segment-routing prefix 5000::23/128 index 23 no-php-flag + ppr on + ! + + rt31: + links: + lo: + mpls: yes + eth-rt21: + peer: [rt21, eth-rt31] + mpls: yes + eth-rt32: + peer: [rt32, eth-rt31] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.31/32 + ipv6 address 5000::31/128 + ipv6 router isis 1 + ! + interface eth-rt21 + ipv6 address 4000:112::31/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:118::31/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0031.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::31/128 index 31 no-php-flag + ppr on + ! + + rt32: + links: + lo: + mpls: yes + eth-rt21: + peer: [rt21, eth-rt32] + mpls: yes + eth-rt22: + peer: [rt22, eth-rt32] + mpls: yes + eth-rt31: + peer: [rt31, eth-rt32] + mpls: yes + eth-rt33: + peer: [rt33, eth-rt32] + mpls: yes + eth-sw1: + peer: [sw1, eth-rt32] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.32/32 + ipv6 address 5000::32/128 + ipv6 router isis 1 + ! + interface eth-rt21 + ipv6 address 4000:113::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt22 + ipv6 address 4000:114::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt31 + ipv6 address 4000:118::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:119::32/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-sw1 + ipv6 address 4000:121::32/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0032.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::32/128 index 32 no-php-flag + ppr on + ! + + rt33: + links: + lo: + mpls: yes + eth-rt22: + peer: [rt22, eth-rt33] + mpls: yes + eth-rt23: + peer: [rt23, eth-rt33] + mpls: yes + eth-rt32: + peer: [rt32, eth-rt33] + mpls: yes + eth-rt34: + peer: [rt34, eth-rt33] + mpls: yes + eth-sw1: + peer: [sw1, eth-rt33] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.33/32 + ipv6 address 5000::33/128 + ipv6 router isis 1 + ! + interface eth-rt22 + ipv6 address 4000:115::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt23 + ipv6 address 4000:116::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt32 + ipv6 address 4000:119::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt34 + ipv6 address 4000:120::33/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-sw1 + ipv6 address 4000:121::33/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0033.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::33/128 index 33 no-php-flag + ppr on + ! + + rt34: + links: + lo: + mpls: yes + eth-rt23: + peer: [rt23, eth-rt34] + mpls: yes + eth-rt33: + peer: [rt33, eth-rt34] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.34/32 + ipv6 address 5000::34/128 + ipv6 router isis 1 + ! + interface eth-rt23 + ipv6 address 4000:117::34/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + interface eth-rt33 + ipv6 address 4000:120::34/64 + ipv6 router isis 1 + isis network point-to-point + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0034.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::34/128 index 34 no-php-flag + ppr on + ! + + rt41: + links: + lo: + mpls: yes + eth-sw1: + peer: [sw1, eth-rt41] + mpls: yes + frr: + zebra: + isisd: + config: | + interface lo + ip address 10.0.0.41/32 + ipv6 address 5000::41/128 + ipv6 router isis 1 + ! + interface eth-sw1 + ipv6 address 4000:121::41/64 + ipv6 router isis 1 + isis hello-multiplier 3 + ! + router isis 1 + net 49.0000.0000.0000.0041.00 + is-type level-1 + topology ipv6-unicast + segment-routing on + segment-routing prefix 5000::41/128 index 41 no-php-flag + ppr on + ! + + switches: + sw1: + links: + eth-rt32: + peer: [rt32, eth-sw1] + eth-rt33: + peer: [rt33, eth-sw1] + eth-rt41: + peer: [rt41, eth-sw1] + + frr: + #valgrind: yes + base-config: | + hostname %(node) + password 1 + log file %(logdir)/%(node).log + log commands + ! + debug zebra rib + debug isis sr-events + debug isis ppr + debug isis events + debug isis route-events + debug isis spf-events + debug isis lsp-gen + ! + +.. + + NOTE: it’s of fundamental importance to enable MPLS processing on the + loopback interfaces, otherwise the tail-end routers of the PPR-MPLS + tunnels will drop the labeled packets they receive. + +YANG +^^^^ + +PPR can also be configured using NETCONF, RESTCONF and gRPC based on the +following YANG models: \* +`frr-ppr.yang <https://github.com/opensourcerouting/frr/blob/isisd-ppr/yang/frr-ppr.yang>`__ +\* +`frr-isisd.yang <https://github.com/opensourcerouting/frr/blob/isisd-ppr/yang/frr-isisd.yang>`__ + +As an example, here’s R11 configuration in the XML format: + +.. code:: xml + + <lib xmlns="http://frrouting.org/yang/interface"> + <interface> + <name>lo-ppr</name> + <vrf>default</vrf> + </interface> + <interface> + <name>lo</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + </isis> + </interface> + <interface> + <name>eth-ce1</name> + <vrf>default</vrf> + </interface> + <interface> + <name>eth-rt12</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + <hello> + <multiplier> + <level-1>3</level-1> + <level-2>3</level-2> + </multiplier> + </hello> + <network-type>point-to-point</network-type> + </isis> + </interface> + <interface> + <name>eth-rt21</name> + <vrf>default</vrf> + <isis xmlns="http://frrouting.org/yang/isisd"> + <area-tag>1</area-tag> + <ipv6-routing>true</ipv6-routing> + <hello> + <multiplier> + <level-1>3</level-1> + <level-2>3</level-2> + </multiplier> + </hello> + <network-type>point-to-point</network-type> + </isis> + </interface> + </lib> + <ppr xmlns="http://frrouting.org/yang/ppr"> + <group> + <name>PPR_IPV6</name> + <ipv6> + <ppr-id>6000:1::1/128</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::22/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>50</ppr-metric> + </attributes> + </ipv6> + <ipv6> + <ppr-id>6000:2::1/128</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>5000::11/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::21/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::22/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::23/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>5000::14/128</pde-id> + <pde-id-type>ipv6-node</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <attributes> + <ppr-metric>50</ppr-metric> + </attributes> + </ipv6> + </group> + <group> + <name>PPR_MPLS_1</name> + <mpls> + <ppr-id>500</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>14</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>23</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>22</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>21</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>11</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </mpls> + <mpls> + <ppr-id>501</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>11</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>21</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>22</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>23</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>14</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </mpls> + </group> + <group> + <name>PPR_MPLS_2</name> + <mpls> + <ppr-id>502</ppr-id> + <ppr-prefix>5000::11/128</ppr-prefix> + <ppr-pde> + <pde-id>14</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>23</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>34</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>33</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>41</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>32</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>31</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>21</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>11</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </mpls> + <mpls> + <ppr-id>503</ppr-id> + <ppr-prefix>5000::14/128</ppr-prefix> + <ppr-pde> + <pde-id>11</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>21</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>31</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>32</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>41</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>33</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>34</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>23</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + <ppr-pde> + <pde-id>14</pde-id> + <pde-id-type>prefix-sid</pde-id-type> + <pde-type>topological</pde-type> + </ppr-pde> + </mpls> + </group> + </ppr> + <isis xmlns="http://frrouting.org/yang/isisd"> + <instance> + <area-tag>1</area-tag> + <area-address>49.0000.0000.0000.0011.00</area-address> + <multi-topology> + <ipv6-unicast> + </ipv6-unicast> + </multi-topology> + <segment-routing> + <enabled>true</enabled> + <prefix-sid-map> + <prefix-sid> + <prefix>5000::11/128</prefix> + <sid-value>11</sid-value> + <last-hop-behavior>no-php</last-hop-behavior> + </prefix-sid> + </prefix-sid-map> + </segment-routing> + <ppr> + <enable>true</enable> + <ppr-advertise> + <name>PPR_IPV6</name> + </ppr-advertise> + <ppr-advertise> + <name>PPR_MPLS_1</name> + </ppr-advertise> + <ppr-advertise> + <name>PPR_MPLS_2</name> + </ppr-advertise> + </ppr> + </instance> + </isis> + +Verification - Control Plane +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Verify that R11 has flooded the PPR TLVs correctly to all IS-IS routers: + +:: + + # show isis database detail 0000.0000.0011 + Area 1: + IS-IS Level-1 link-state database: + LSP ID PduLen SeqNumber Chksum Holdtime ATT/P/OL + debian.00-00 * 980 0x00000003 0x3b69 894 0/0/0 + Protocols Supported: IPv4, IPv6 + Area Address: 49.0000 + MT Router Info: ipv4-unicast + MT Router Info: ipv6-unicast + Hostname: debian + TE Router ID: 10.0.0.11 + Router Capability: 10.0.0.11 , D:0, S:0 + Segment Routing: I:1 V:1, SRGB Base: 16000 Range: 8000 + Algorithm: 0: SPF 0: Strict SPF + MT Reachability: 0000.0000.0012.00 (Metric: 10) ipv6-unicast + Adjacency-SID: 16, Weight: 0, Flags: F:1 B:0, V:1, L:1, S:0, P:0 + MT Reachability: 0000.0000.0021.00 (Metric: 10) ipv6-unicast + Adjacency-SID: 17, Weight: 0, Flags: F:1 B:0, V:1, L:1, S:0, P:0 + IPv4 Interface Address: 10.0.0.11 + Extended IP Reachability: 10.0.0.11/32 (Metric: 10) + MT IPv6 Reachability: 5000::11/128 (Metric: 10) ipv6-unicast + Subtlvs: + SR Prefix-SID Index: 11, Algorithm: 0, Flags: NO-PHP + MT IPv6 Reachability: 4000:101::/64 (Metric: 10) ipv6-unicast + MT IPv6 Reachability: 4000:104::/64 (Metric: 10) ipv6-unicast + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 6000:1::1/128 (Native IPv6) + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::22/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 50 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 6000:2::1/128 (Native IPv6) + PDE: 5000::11/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::21/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::22/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::23/128 (IPv6 Node Address), L:0 N:0 E:0 + PDE: 5000::14/128 (IPv6 Node Address), L:0 N:1 E:0 + Metric: 50 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 500 (MPLS) + PDE: 14 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 23 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 22 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 21 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 11 (SR-MPLS Prefix SID), L:0 N:1 E:0 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 501 (MPLS) + PDE: 11 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 21 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 22 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 23 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 14 (SR-MPLS Prefix SID), L:0 N:1 E:0 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::11/128 + ID: 502 (MPLS) + PDE: 14 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 23 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 34 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 33 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 41 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 32 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 31 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 21 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 11 (SR-MPLS Prefix SID), L:0 N:1 E:0 + PPR: Fragment ID: 0, MT-ID: ipv4-unicast, Algorithm: SPF, F:0 D:0 A:0 U:1 + PPR Prefix: 5000::14/128 + ID: 503 (MPLS) + PDE: 11 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 21 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 31 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 32 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 41 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 33 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 34 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 23 (SR-MPLS Prefix SID), L:0 N:0 E:0 + PDE: 14 (SR-MPLS Prefix SID), L:0 N:1 E:0 + +Using the ``show isis ppr`` command, verify that all routers installed +the PPR-IDs for the paths they are part of. Example: + +Router RT11 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + -------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Tail-End Up 00:00:42 + 1 L1 501 (MPLS) 5000::14/128 0 Head-End Up 00:00:41 + 1 L1 502 (MPLS) 5000::11/128 0 Tail-End Up 00:00:42 + 1 L1 503 (MPLS) 5000::14/128 0 Head-End Up 00:00:41 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Tail-End - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Head-End Up 00:00:41 + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 implicit-null + 17 SR (IS-IS) fe80::345f:dfff:fea4:913d implicit-null + 16011 SR (IS-IS) lo - + 16012 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16012 + 16013 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16013 + 16014 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16014 + 16021 SR (IS-IS) fe80::345f:dfff:fea4:913d 16021 + 16022 SR (IS-IS) fe80::345f:dfff:fea4:913d 16022 + 16022 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16022 + 16023 SR (IS-IS) fe80::345f:dfff:fea4:913d 16023 + 16023 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16023 + 16031 SR (IS-IS) fe80::345f:dfff:fea4:913d 16031 + 16032 SR (IS-IS) fe80::345f:dfff:fea4:913d 16032 + 16033 SR (IS-IS) fe80::345f:dfff:fea4:913d 16033 + 16033 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16033 + 16034 SR (IS-IS) fe80::345f:dfff:fea4:913d 16034 + 16034 SR (IS-IS) fe80::2065:5ff:fe72:d6c5 16034 + 16041 SR (IS-IS) fe80::345f:dfff:fea4:913d 16041 + 16500 PPR (IS-IS) lo - + 16501 PPR (IS-IS) fe80::345f:dfff:fea4:913d 16501 + 16502 PPR (IS-IS) lo - + 16503 PPR (IS-IS) fe80::345f:dfff:fea4:913d 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:2::1/128 [115/50] via fe80::345f:dfff:fea4:913d, eth-rt21, 00:00:41 + +Router RT12 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 503 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ---------------------------------------------------------------------- + 16 SR (IS-IS) fe80::60ad:96ff:fe3f:9989 implicit-null + 17 SR (IS-IS) fe80::9cd2:25ff:febc:84c4 implicit-null + 18 SR (IS-IS) fe80::941c:12ff:fe55:8a12 implicit-null + 19 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 implicit-null + 16011 SR (IS-IS) fe80::60ad:96ff:fe3f:9989 16011 + 16012 SR (IS-IS) lo - + 16013 SR (IS-IS) fe80::9cd2:25ff:febc:84c4 16013 + 16014 SR (IS-IS) fe80::9cd2:25ff:febc:84c4 16014 + 16021 SR (IS-IS) fe80::941c:12ff:fe55:8a12 16021 + 16022 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16022 + 16023 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16023 + 16023 SR (IS-IS) fe80::9cd2:25ff:febc:84c4 16023 + 16031 SR (IS-IS) fe80::941c:12ff:fe55:8a12 16031 + 16032 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16032 + 16032 SR (IS-IS) fe80::941c:12ff:fe55:8a12 16032 + 16033 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16033 + 16034 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16034 + 16034 SR (IS-IS) fe80::9cd2:25ff:febc:84c4 16034 + 16041 SR (IS-IS) fe80::78a7:59ff:fedc:48b8 16041 + 16041 SR (IS-IS) fe80::941c:12ff:fe55:8a12 16041 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT13 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + ------------------------------------------------------------------------------------------ + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 503 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ---------------------------------------------------------------------- + 16 SR (IS-IS) fe80::1c70:63ff:fe40:3a35 implicit-null + 17 SR (IS-IS) fe80::20:56ff:feff:b218 implicit-null + 18 SR (IS-IS) fe80::44c5:3fff:fe1e:f34a implicit-null + 19 SR (IS-IS) fe80::387d:34ff:fe02:87c3 implicit-null + 16011 SR (IS-IS) fe80::20:56ff:feff:b218 16011 + 16012 SR (IS-IS) fe80::20:56ff:feff:b218 16012 + 16013 SR (IS-IS) lo - + 16014 SR (IS-IS) fe80::1c70:63ff:fe40:3a35 16014 + 16021 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16021 + 16021 SR (IS-IS) fe80::20:56ff:feff:b218 16021 + 16022 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16022 + 16023 SR (IS-IS) fe80::44c5:3fff:fe1e:f34a 20023 + 16031 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16031 + 16031 SR (IS-IS) fe80::20:56ff:feff:b218 16031 + 16032 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16032 + 16033 SR (IS-IS) fe80::44c5:3fff:fe1e:f34a 20033 + 16033 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16033 + 16034 SR (IS-IS) fe80::44c5:3fff:fe1e:f34a 20034 + 16041 SR (IS-IS) fe80::44c5:3fff:fe1e:f34a 20041 + 16041 SR (IS-IS) fe80::387d:34ff:fe02:87c3 16041 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT14 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + -------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Head-End Up 00:00:46 + 1 L1 501 (MPLS) 5000::14/128 0 Tail-End Up 00:00:47 + 1 L1 502 (MPLS) 5000::11/128 0 Head-End Up 00:00:46 + 1 L1 503 (MPLS) 5000::14/128 0 Tail-End Up 00:00:47 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Head-End Up 00:00:46 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Tail-End - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad implicit-null + 17 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 implicit-null + 16011 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16011 + 16012 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16012 + 16013 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16013 + 16014 SR (IS-IS) lo - + 16021 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20021 + 16021 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16021 + 16022 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20022 + 16022 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16022 + 16023 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20023 + 16031 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20031 + 16031 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16031 + 16032 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20032 + 16032 SR (IS-IS) fe80::bcb5:99ff:fed7:22ad 16032 + 16033 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20033 + 16034 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20034 + 16041 SR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20041 + 16500 PPR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20500 + 16501 PPR (IS-IS) lo - + 16502 PPR (IS-IS) fe80::4c7b:a1ff:fe66:6ca7 20502 + 16503 PPR (IS-IS) lo - + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::4c7b:a1ff:fe66:6ca7, eth-rt23, 00:00:02 + +Router RT21 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:49 + 1 L1 501 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:48 + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:49 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:48 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:00:49 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:00:48 + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::b886:2cff:fe84:a76f implicit-null + 17 SR (IS-IS) fe80::bc7e:bbff:fe7f:ecb0 implicit-null + 18 SR (IS-IS) fe80::e877:a2ff:feb7:4438 implicit-null + 19 SR (IS-IS) fe80::a0c2:82ff:fe39:204c implicit-null + 20 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 implicit-null + 16011 SR (IS-IS) fe80::e877:a2ff:feb7:4438 16011 + 16012 SR (IS-IS) fe80::a0c2:82ff:fe39:204c 16012 + 16013 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16013 + 16013 SR (IS-IS) fe80::a0c2:82ff:fe39:204c 16013 + 16014 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16014 + 16014 SR (IS-IS) fe80::a0c2:82ff:fe39:204c 16014 + 16021 SR (IS-IS) lo - + 16022 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16022 + 16023 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16023 + 16031 SR (IS-IS) fe80::bc7e:bbff:fe7f:ecb0 16031 + 16032 SR (IS-IS) fe80::b886:2cff:fe84:a76f 16032 + 16033 SR (IS-IS) fe80::b886:2cff:fe84:a76f 16033 + 16033 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16033 + 16034 SR (IS-IS) fe80::b886:2cff:fe84:a76f 16034 + 16034 SR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16034 + 16041 SR (IS-IS) fe80::b886:2cff:fe84:a76f 16041 + 16500 PPR (IS-IS) fe80::e877:a2ff:feb7:4438 16500 + 16501 PPR (IS-IS) fe80::ac6a:8aff:fe14:4f36 16501 + 16502 PPR (IS-IS) fe80::e877:a2ff:feb7:4438 16502 + 16503 PPR (IS-IS) fe80::bc7e:bbff:fe7f:ecb0 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::e877:a2ff:feb7:4438, eth-rt11, 00:00:04 + I>* 6000:2::1/128 [115/50] via fe80::ac6a:8aff:fe14:4f36, eth-rt22, 00:00:04 + +Router RT22 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:50 + 1 L1 501 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:50 + 1 L1 502 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 503 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:00:50 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:00:50 + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::3432:84ff:fe9d:2e41 implicit-null + 17 SR (IS-IS) fe80::c436:63ff:feb3:4f5d implicit-null + 18 SR (IS-IS) fe80::56:41ff:fe53:a6b2 implicit-null + 19 SR (IS-IS) fe80::b423:eaff:fea1:8247 implicit-null + 20 SR (IS-IS) fe80::9c2f:11ff:fe0a:ab34 implicit-null + 21 SR (IS-IS) fe80::7402:b8ff:fee9:682e implicit-null + 16011 SR (IS-IS) fe80::b423:eaff:fea1:8247 16011 + 16011 SR (IS-IS) fe80::3432:84ff:fe9d:2e41 16011 + 16012 SR (IS-IS) fe80::3432:84ff:fe9d:2e41 16012 + 16013 SR (IS-IS) fe80::c436:63ff:feb3:4f5d 16013 + 16014 SR (IS-IS) fe80::56:41ff:fe53:a6b2 20014 + 16014 SR (IS-IS) fe80::c436:63ff:feb3:4f5d 16014 + 16021 SR (IS-IS) fe80::b423:eaff:fea1:8247 16021 + 16022 SR (IS-IS) lo - + 16023 SR (IS-IS) fe80::56:41ff:fe53:a6b2 20023 + 16031 SR (IS-IS) fe80::9c2f:11ff:fe0a:ab34 16031 + 16031 SR (IS-IS) fe80::b423:eaff:fea1:8247 16031 + 16032 SR (IS-IS) fe80::9c2f:11ff:fe0a:ab34 16032 + 16033 SR (IS-IS) fe80::7402:b8ff:fee9:682e 16033 + 16034 SR (IS-IS) fe80::7402:b8ff:fee9:682e 16034 + 16034 SR (IS-IS) fe80::56:41ff:fe53:a6b2 20034 + 16041 SR (IS-IS) fe80::7402:b8ff:fee9:682e 16041 + 16041 SR (IS-IS) fe80::9c2f:11ff:fe0a:ab34 16041 + 16500 PPR (IS-IS) fe80::b423:eaff:fea1:8247 16500 + 16501 PPR (IS-IS) fe80::56:41ff:fe53:a6b2 20501 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::b423:eaff:fea1:8247, eth-rt21, 00:00:06 + I>* 6000:2::1/128 [115/50] via fe80::56:41ff:fe53:a6b2, eth-rt23, 00:00:06 + +Router RT23 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:52 + 1 L1 501 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:52 + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:52 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:52 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Mid-Point Up 00:00:52 + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Mid-Point Up 00:00:52 + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::c4ca:41ff:fe2d:de8c implicit-null + 17 SR (IS-IS) fe80::a02b:1eff:fed6:97e4 implicit-null + 18 SR (IS-IS) fe80::5c15:8aff:feea:1d07 implicit-null + 19 SR (IS-IS) fe80::a42f:50ff:fe9c:af9f implicit-null + 20 SR (IS-IS) fe80::d0dc:6eff:fe71:9f19 implicit-null + 20011 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16011 + 20011 SR (IS-IS) fe80::a02b:1eff:fed6:97e4 16011 + 20012 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16012 + 20012 SR (IS-IS) fe80::a02b:1eff:fed6:97e4 16012 + 20013 SR (IS-IS) fe80::a02b:1eff:fed6:97e4 16013 + 20014 SR (IS-IS) fe80::c4ca:41ff:fe2d:de8c 16014 + 20021 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16021 + 20022 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16022 + 20023 SR (IS-IS) lo - + 20031 SR (IS-IS) fe80::a42f:50ff:fe9c:af9f 16031 + 20031 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16031 + 20032 SR (IS-IS) fe80::a42f:50ff:fe9c:af9f 16032 + 20032 SR (IS-IS) fe80::5c15:8aff:feea:1d07 16032 + 20033 SR (IS-IS) fe80::a42f:50ff:fe9c:af9f 16033 + 20034 SR (IS-IS) fe80::d0dc:6eff:fe71:9f19 16034 + 20041 SR (IS-IS) fe80::a42f:50ff:fe9c:af9f 16041 + 20500 PPR (IS-IS) fe80::5c15:8aff:feea:1d07 16500 + 20501 PPR (IS-IS) fe80::c4ca:41ff:fe2d:de8c 16501 + 20502 PPR (IS-IS) fe80::d0dc:6eff:fe71:9f19 16502 + 20503 PPR (IS-IS) fe80::c4ca:41ff:fe2d:de8c 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + Codes: K - kernel route, C - connected, S - static, R - RIPng, + O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, + v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, + f - OpenFabric, + > - selected route, * - FIB route, q - queued route, r - rejected route + + I>* 6000:1::1/128 [115/50] via fe80::5c15:8aff:feea:1d07, eth-rt22, 00:00:07 + I>* 6000:2::1/128 [115/50] via fe80::c4ca:41ff:fe2d:de8c, eth-rt14, 00:00:07 + +Router RT31 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:54 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:54 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 implicit-null + 17 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 implicit-null + 16011 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16011 + 16012 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16012 + 16013 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16013 + 16013 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16013 + 16014 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16014 + 16014 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16014 + 16021 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16021 + 16022 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16022 + 16022 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16022 + 16023 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16023 + 16023 SR (IS-IS) fe80::a067:c6ff:fe2c:3385 16023 + 16031 SR (IS-IS) lo - + 16032 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16032 + 16033 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16033 + 16034 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16034 + 16041 SR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16041 + 16502 PPR (IS-IS) fe80::a067:c6ff:fe2c:3385 16502 + 16503 PPR (IS-IS) fe80::f46d:c8ff:fe8a:a341 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT32 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:55 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:55 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::881f:d3ff:febd:9e8c implicit-null + 17 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 implicit-null + 18 SR (IS-IS) fe80::9863:abff:fed0:d7e implicit-null + 19 SR (IS-IS) fe80::ec65:d1ff:fe32:b508 implicit-null + 20 SR (IS-IS) fe80::a4e9:77ff:feaa:f690 implicit-null + 21 SR (IS-IS) fe80::40c4:e6ff:fe26:767f implicit-null + 16011 SR (IS-IS) fe80::881f:d3ff:febd:9e8c 16011 + 16012 SR (IS-IS) fe80::40c4:e6ff:fe26:767f 16012 + 16012 SR (IS-IS) fe80::881f:d3ff:febd:9e8c 16012 + 16013 SR (IS-IS) fe80::40c4:e6ff:fe26:767f 16013 + 16014 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16014 + 16014 SR (IS-IS) fe80::ec65:d1ff:fe32:b508 16014 + 16014 SR (IS-IS) fe80::40c4:e6ff:fe26:767f 16014 + 16021 SR (IS-IS) fe80::881f:d3ff:febd:9e8c 16021 + 16022 SR (IS-IS) fe80::40c4:e6ff:fe26:767f 16022 + 16023 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16023 + 16023 SR (IS-IS) fe80::ec65:d1ff:fe32:b508 16023 + 16023 SR (IS-IS) fe80::40c4:e6ff:fe26:767f 16023 + 16031 SR (IS-IS) fe80::9863:abff:fed0:d7e 16031 + 16032 SR (IS-IS) lo - + 16033 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16033 + 16033 SR (IS-IS) fe80::ec65:d1ff:fe32:b508 16033 + 16034 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16034 + 16034 SR (IS-IS) fe80::ec65:d1ff:fe32:b508 16034 + 16041 SR (IS-IS) fe80::a4e9:77ff:feaa:f690 16041 + 16502 PPR (IS-IS) fe80::9863:abff:fed0:d7e 16502 + 16503 PPR (IS-IS) fe80::a4e9:77ff:feaa:f690 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT33 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:57 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:57 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::2832:a9ff:fec3:7078 implicit-null + 17 SR (IS-IS) fe80::7806:e1ff:fe72:9b1f implicit-null + 18 SR (IS-IS) fe80::5476:31ff:fe94:c39 implicit-null + 19 SR (IS-IS) fe80::a4e9:77ff:feaa:f690 implicit-null + 20 SR (IS-IS) fe80::68c9:2ff:fe04:5eba implicit-null + 21 SR (IS-IS) fe80::d053:97ff:fee2:1711 implicit-null + 16011 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16011 + 16011 SR (IS-IS) fe80::5476:31ff:fe94:c39 16011 + 16011 SR (IS-IS) fe80::d053:97ff:fee2:1711 16011 + 16012 SR (IS-IS) fe80::d053:97ff:fee2:1711 16012 + 16013 SR (IS-IS) fe80::68c9:2ff:fe04:5eba 20013 + 16013 SR (IS-IS) fe80::d053:97ff:fee2:1711 16013 + 16014 SR (IS-IS) fe80::68c9:2ff:fe04:5eba 20014 + 16021 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16021 + 16021 SR (IS-IS) fe80::5476:31ff:fe94:c39 16021 + 16021 SR (IS-IS) fe80::d053:97ff:fee2:1711 16021 + 16022 SR (IS-IS) fe80::d053:97ff:fee2:1711 16022 + 16023 SR (IS-IS) fe80::68c9:2ff:fe04:5eba 20023 + 16031 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16031 + 16031 SR (IS-IS) fe80::5476:31ff:fe94:c39 16031 + 16032 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16032 + 16032 SR (IS-IS) fe80::5476:31ff:fe94:c39 16032 + 16033 SR (IS-IS) lo - + 16034 SR (IS-IS) fe80::7806:e1ff:fe72:9b1f 16034 + 16041 SR (IS-IS) fe80::a4e9:77ff:feaa:f690 16041 + 16502 PPR (IS-IS) fe80::a4e9:77ff:feaa:f690 16502 + 16503 PPR (IS-IS) fe80::7806:e1ff:fe72:9b1f 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT34 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:00:59 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:00:59 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::ac33:5dff:fe99:81ec implicit-null + 17 SR (IS-IS) fe80::f009:b9ff:fe05:e540 implicit-null + 16011 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16011 + 16011 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20011 + 16012 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16012 + 16012 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20012 + 16013 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20013 + 16014 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20014 + 16021 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16021 + 16021 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20021 + 16022 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16022 + 16022 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20022 + 16023 SR (IS-IS) fe80::f009:b9ff:fe05:e540 20023 + 16031 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16031 + 16032 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16032 + 16033 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16033 + 16034 SR (IS-IS) lo - + 16041 SR (IS-IS) fe80::ac33:5dff:fe99:81ec 16041 + 16502 PPR (IS-IS) fe80::ac33:5dff:fe99:81ec 16502 + 16503 PPR (IS-IS) fe80::f009:b9ff:fe05:e540 20503 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Router RT41 +^^^^^^^^^^^ + +:: + + # show isis ppr + Area Level ID Prefix Metric Position Status Uptime + --------------------------------------------------------------------------------------------- + 1 L1 500 (MPLS) 5000::11/128 0 Off-Path - - + 1 L1 501 (MPLS) 5000::14/128 0 Off-Path - - + 1 L1 502 (MPLS) 5000::11/128 0 Mid-Point Up 00:01:01 + 1 L1 503 (MPLS) 5000::14/128 0 Mid-Point Up 00:01:01 + 1 L1 6000:1::1/128 (Native IPv6) 5000::11/128 50 Off-Path - - + 1 L1 6000:2::1/128 (Native IPv6) 5000::14/128 50 Off-Path - - + + # show mpls table + Inbound Label Type Nexthop Outbound Label + ----------------------------------------------------------------------- + 16 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 implicit-null + 17 SR (IS-IS) fe80::2832:a9ff:fec3:7078 implicit-null + 16011 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16011 + 16012 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16012 + 16012 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16012 + 16013 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16013 + 16013 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16013 + 16014 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16014 + 16021 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16021 + 16022 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16022 + 16022 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16022 + 16023 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16023 + 16031 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16031 + 16032 SR (IS-IS) fe80::2832:a9ff:fec3:7078 16032 + 16033 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16033 + 16034 SR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16034 + 16041 SR (IS-IS) lo - + 16502 PPR (IS-IS) fe80::2832:a9ff:fec3:7078 16502 + 16503 PPR (IS-IS) fe80::1c7e:c3ff:fe5e:7a54 16503 + + # show ipv6 route 6000::/16 longer-prefixes isis + +Notice how R23 uses a different SRGB compared to the other routers in +the network. As such, this router install different labels for PPR-IDs +500 and 501 (e.g. 20500 instead of 16500 using the default SRGB). + +Verification - Forwarding Plane +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ping Host 3 from Host2 and use tcpdump or wireshark to verify that the +ICMP packets are being tunneled using MPLS LSPs and following the {R11 - +R21 - R22 - R23 - R14} path. Here’s a wireshark capture between R11 and +R21: + +.. figure:: https://user-images.githubusercontent.com/931662/64057179-2e980080-cb70-11e9-89c3-ff43e6d66cae.png + :alt: wireshark + + wireshark + +Using ``traceroute`` it’s also possible to see that the ICMP packets are +being tunneled through the IS-IS network: + +:: + + root@host2:~# traceroute -n fd00:20:1::1 -s fd00:10:2::1 + traceroute to fd00:20:1::1 (fd00:20:1::1), 30 hops max, 80 byte packets + 1 fd00:10:2::100 1.996 ms 1.832 ms 1.725 ms + 2 * * * + 3 * * * + 4 * * * + 5 * * * + 6 * * * + 7 * * * + 8 fd00:20::100 0.154 ms 0.191 ms 0.116 ms + 9 fd00:20:1::1 0.125 ms 0.105 ms 0.104 ms diff --git a/doc/developer/northbound/retrofitting-configuration-commands.rst b/doc/developer/northbound/retrofitting-configuration-commands.rst new file mode 100644 index 0000000..b407246 --- /dev/null +++ b/doc/developer/northbound/retrofitting-configuration-commands.rst @@ -0,0 +1,1897 @@ +Retrofitting Configuration Commands +----------------------------------- + +This page explains how to convert existing CLI configuration commands to +the new northbound model. This documentation is meant to be the primary +reference for developers working on the northbound retrofitting process. +We’ll show several examples taken from the ripd northbound conversion to +illustrate some concepts described herein. + +Retrofitting process +-------------------- + +Step 1: writing a YANG module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first step is to write a YANG module that models faithfully the +commands that are going to be converted. As explained in the +[[Architecture]] page, the goal is to introduce the new YANG-based +Northbound API without introducing backward incompatible changes in the +CLI. The northbound retrofitting process should be completely +transparent to FRR users. + +The developer is free to choose whether to write a full YANG module or a +partial YANG module and increment it gradually. For developers who lack +experience with YANG it’s probably a better idea to model one command at +time. + +It’s recommended to reuse definitions from standard YANG models whenever +possible to facilitate the process of writing module translators using +the [[YANG module translator]]. As an example, the frr-ripd YANG module +incorporated several parts of the IETF RIP YANG module. The repositories +below contain big collections of YANG models that might be used as a +reference: \* https://github.com/YangModels/yang \* +https://github.com/openconfig/public + +When writing a YANG module, it’s highly recommended to follow the +guidelines from `RFC 6087 <https://tools.ietf.org/html/rfc6087>`__. In +general, most commands should be modeled fairly easy. Here are a few +guidelines specific to authors of FRR YANG models: \* Use +presence-containers or lists to model commands that change the CLI node +(e.g. ``router rip``, ``interface eth0``). This way, if the +presence-container or list entry is removed, all configuration options +below them are removed automatically (exactly like the CLI behaves when +a configuration object is removed using a *no* command). This +recommendation is orthogonal to the `YANG authoring guidelines for +OpenConfig +models <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ +where the use of presence containers is discouraged. OpenConfig YANG +models however were not designed to replicate the behavior of legacy CLI +commands. \* When using YANG lists, be careful to identify what should +be the key leaves. In the ``offset-list WORD <in|out> (0-16) IFNAME`` +command, for example, both the direction (``<in|out>``) and the +interface name should be the keys of the list. This can be only known by +analyzing the data structures used to store the commands. \* For +clarity, use non-presence containers to group leaves that are associated +to the same configuration command (as we’ll see later, this also +facilitate the process of writing ``cli_show`` callbacks). \* YANG +leaves of type *enumeration* should define explicitly the value of each +*enum* option based on the value used in the FRR source code. \* Default +values should be taken from the source code whenever they exist. + +Some commands are more difficult to model and demand the use of more +advanced YANG constructs like *choice*, *when* and *must* statements. +**One key requirement is that it should be impossible to load an invalid +JSON/XML configuration to FRR**. The YANG modules should model exactly +what the CLI accepts in the form of commands, and all restrictions +imposed by the CLI should be defined in the YANG models whenever +possible. As we’ll see later, not all constraints can be expressed using +the YANG language and sometimes we’ll need to resort to code-level +validation in the northbound callbacks. + + Tip: the :doc:`yang-tools` page details several tools and commands that + might be useful when writing a YANG module, like validating YANG + files, indenting YANG files, validating instance data, etc. + +In the example YANG snippet below, we can see the use of the *must* +statement that prevents ripd from redistributing RIP routes into itself. +Although ripd CLI doesn’t allow the operator to enter *redistribute rip* +under *router rip*, we don’t have the same protection when configuring +ripd using other northbound interfaces (e.g. NETCONF). So without this +constraint it would be possible to feed an invalid configuration to ripd +(i.e. a bug). + +.. code:: yang + + list redistribute { + key "protocol"; + description + "Redistributes routes learned from other routing protocols."; + leaf protocol { + type frr-route-types:frr-route-types-v4; + description + "Routing protocol."; + must '. != "rip"'; + } + [snip] + } + +In the example below, we use the YANG *choice* statement to ensure that +either the ``password`` leaf or the ``key-chain`` leaf is configured, +but not both. This is in accordance to the sanity checks performed by +the *ip rip authentication* commands. + +.. code:: yang + + choice authentication-data { + description + "Choose whether to use a simple password or a key-chain."; + leaf authentication-password { + type string { + length "1..16"; + } + description + "Authentication string."; + } + leaf authentication-key-chain { + type string; + description + "Key-chain name."; + } + } + +Once finished, the new YANG model should be put into the FRR *yang/* top +level directory. This will ensure it will be installed automatically by +``make install``. It’s also encouraged (but not required) to put sample +configurations under *yang/examples/* using either JSON or XML files. + +Step 2: generate skeleton northbound callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the *gen_northbound_callbacks* tool to generate skeleton callbacks +for the YANG module. Example: + +.. code:: sh + + $ tools/gen_northbound_callbacks frr-ripd > ripd/rip_northbound.c + +The tool will look for the given module in the ``YANG_MODELS_PATH`` +directory defined during the installation. For each schema node of the +YANG module, the tool will generate skeleton callbacks based on the +properties of the node. Example: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance + */ + static int ripd_instance_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + /* TODO: implement me. */ + return NB_OK; + } + + static int ripd_instance_delete(enum nb_event event, + const struct lyd_node *dnode) + { + /* TODO: implement me. */ + return NB_OK; + } + + /* + * XPath: /frr-ripd:ripd/instance/allow-ecmp + */ + static int ripd_instance_allow_ecmp_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + /* TODO: implement me. */ + return NB_OK; + } + + [snip] + + const struct frr_yang_module_info frr_ripd_info = { + .name = "frr-ripd", + .nodes = { + { + .xpath = "/frr-ripd:ripd/instance", + .cbs.create = ripd_instance_create, + .cbs.delete = ripd_instance_delete, + }, + { + .xpath = "/frr-ripd:ripd/instance/allow-ecmp", + .cbs.modify = ripd_instance_allow_ecmp_modify, + }, + [snip] + { + .xpath = "/frr-ripd:ripd/state/routes/route", + .cbs.get_next = ripd_state_routes_route_get_next, + .cbs.get_keys = ripd_state_routes_route_get_keys, + .cbs.lookup_entry = ripd_state_routes_route_lookup_entry, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/prefix", + .cbs.get_elem = ripd_state_routes_route_prefix_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/next-hop", + .cbs.get_elem = ripd_state_routes_route_next_hop_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/interface", + .cbs.get_elem = ripd_state_routes_route_interface_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/metric", + .cbs.get_elem = ripd_state_routes_route_metric_get_elem, + }, + { + .xpath = "/frr-ripd:clear-rip-route", + .cbs.rpc = clear_rip_route_rpc, + }, + [snip] + +After the C source file is generated, it’s necessary to add a copyright +header on it and indent the code using ``clang-format``. + +Step 3: update the *frr_yang_module_info* array of all relevant daemons +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We must inform the northbound about which daemons will implement the new +YANG module. This is done by updating the ``frr_daemon_info`` structure +of these daemons, with help of the ``FRR_DAEMON_INFO`` macro. + +When a YANG module is specific to a single daemon, like the frr-ripd +module, then only the corresponding daemon should be updated. When the +YANG module is related to a subset of libfrr (e.g. route-maps), then all +FRR daemons that make use of that subset must be updated. + +Example: + +.. code:: c + + static const struct frr_yang_module_info *ripd_yang_modules[] = { + &frr_interface_info, + &frr_ripd_info, + }; + + FRR_DAEMON_INFO(ripd, RIP, .vty_port = RIP_VTY_PORT, + [snip] + .yang_modules = ripd_yang_modules, + .n_yang_modules = array_size(ripd_yang_modules), ) + +Step 4: implement the northbound configuration callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Implementing the northbound configuration callbacks consists mostly of +copying code from the corresponding CLI commands and make the required +adaptations. + +It’s recommended to convert one command or a small group of related +commands per commit. Small commits are preferred to facilitate the +review process. Both “old” and “new” command can coexist without +problems, so the retrofitting process can happen gradually over time. + +The configuration callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These are the four main northbound configuration callbacks, as defined +in the ``lib/northbound.h`` file: + +.. code:: c + + /* + * Configuration callback. + * + * A presence container, list entry, leaf-list entry or leaf of type + * empty has been created. + * + * For presence-containers and list entries, the callback is supposed to + * initialize the default values of its children (if any) from the YANG + * models. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being created. + * + * resource + * Pointer to store resource(s) allocated during the NB_EV_PREPARE + * phase. The same pointer can be used during the NB_EV_ABORT and + * NB_EV_APPLY phases to either release or make use of the allocated + * resource(s). It's set to NULL when the event is NB_EV_VALIDATE. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_RESOURCE when the callback failed to allocate a resource. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*create)(enum nb_event event, const struct lyd_node *dnode, + union nb_resource *resource); + + /* + * Configuration callback. + * + * The value of a leaf has been modified. + * + * List keys don't need to implement this callback. When a list key is + * modified, the northbound treats this as if the list was deleted and a + * new one created with the updated key value. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being modified + * + * resource + * Pointer to store resource(s) allocated during the NB_EV_PREPARE + * phase. The same pointer can be used during the NB_EV_ABORT and + * NB_EV_APPLY phases to either release or make use of the allocated + * resource(s). It's set to NULL when the event is NB_EV_VALIDATE. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_RESOURCE when the callback failed to allocate a resource. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*modify)(enum nb_event event, const struct lyd_node *dnode, + union nb_resource *resource); + + /* + * Configuration callback. + * + * A presence container, list entry, leaf-list entry or optional leaf + * has been deleted. + * + * The callback is supposed to delete the entire configuration object, + * including its children when they exist. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being deleted. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*delete)(enum nb_event event, const struct lyd_node *dnode); + + /* + * Configuration callback. + * + * A list entry or leaf-list entry has been moved. Only applicable when + * the "ordered-by user" statement is present. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being moved. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*move)(enum nb_event event, const struct lyd_node *dnode); + +Since skeleton northbound callbacks are generated automatically by the +*gen_northbound_callbacks* tool, the developer doesn’t need to worry +about which callbacks need to be implemented. + + NOTE: once a daemon starts, it reads its YANG modules and validates + that all required northbound callbacks were implemented. If any + northbound callback is missing, an error is logged and the program + exists. + +Transaction phases +^^^^^^^^^^^^^^^^^^ + +Configuration transactions and their phases were described in detail in +the [[Architecture]] page. Here’s the definition of the ``nb_event`` +enumeration as defined in the *lib/northbound.h* file: + +.. code:: c + + /* Northbound events. */ + enum nb_event { + /* + * The configuration callback is supposed to verify that the changes are + * valid and can be applied. + */ + NB_EV_VALIDATE, + + /* + * The configuration callback is supposed to prepare all resources + * required to apply the changes. + */ + NB_EV_PREPARE, + + /* + * Transaction has failed, the configuration callback needs to release + * all resources previously allocated. + */ + NB_EV_ABORT, + + /* + * The configuration changes need to be applied. The changes can't be + * rejected at this point (errors are logged and ignored). + */ + NB_EV_APPLY, + }; + +When converting a CLI command, we must identify all error-prone +operations and perform them in the ``NB_EV_PREPARE`` phase of the +northbound callbacks. When the operation in question involves the +allocation of a specific resource (e.g. file descriptors), we can store +the allocated resource in the ``resource`` variable given to the +callback. This way the allocated resource can be obtained in the other +phases of the transaction using the same parameter. + +Here’s the ``create`` northbound callback associated to the +``router rip`` command: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance + */ + static int ripd_instance_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + int socket; + + switch (event) { + case NB_EV_VALIDATE: + break; + case NB_EV_PREPARE: + socket = rip_create_socket(); + if (socket < 0) + return NB_ERR_RESOURCE; + resource->fd = socket; + break; + case NB_EV_ABORT: + socket = resource->fd; + close(socket); + break; + case NB_EV_APPLY: + socket = resource->fd; + rip_create(socket); + break; + } + + return NB_OK; + } + +Note that the socket creation is an error-prone operation since it +depends on the underlying operating system, so the socket must be +created during the ``NB_EV_PREPARE`` phase and stored in +``resource->fd``. This socket is then either closed or used depending on +the outcome of the preparation phase of the whole transaction. + +During the ``NB_EV_VALIDATE`` phase, the northbound callbacks must +validate if the intended changes are valid. As an example, FRR doesn’t +allow the operator to deconfigure active interfaces: + +.. code:: c + + static int lib_interface_delete(enum nb_event event, + const struct lyd_node *dnode) + { + struct interface *ifp; + + ifp = yang_dnode_get_entry(dnode); + + switch (event) { + case NB_EV_VALIDATE: + if (CHECK_FLAG(ifp->status, ZEBRA_INTERFACE_ACTIVE)) { + zlog_warn("%s: only inactive interfaces can be deleted", + __func__); + return NB_ERR_VALIDATION; + } + break; + case NB_EV_PREPARE: + case NB_EV_ABORT: + break; + case NB_EV_APPLY: + if_delete(ifp); + break; + } + + return NB_OK; + } + +Note however that it’s preferred to use YANG to model the validation +constraints whenever possible. Code-level validations should be used +only to validate constraints that can’t be modeled using the YANG +language. + +Most callbacks don’t need to perform any validations nor perform any +error-prone operations, so in these cases we can use the following +pattern to return early if ``event`` is different than ``NB_EV_APPLY``: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/distance/default + */ + static int ripd_instance_distance_default_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + if (event != NB_EV_APPLY) + return NB_OK; + + rip->distance = yang_dnode_get_uint8(dnode, NULL); + + return NB_OK; + } + +During development it’s recommend to use the *debug northbound* command +to debug configuration transactions and see what callbacks are being +called. Example: + +:: + + ripd# conf t + ripd(config)# debug northbound + ripd(config)# router rip + ripd(config-router)# allow-ecmp + ripd(config-router)# network eth0 + ripd(config-router)# redistribute ospf metric 2 + ripd(config-router)# commit + % Configuration committed successfully. + + ripd(config-router)# + +Now the ripd log: + +:: + + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [apply_finish] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(null)] + +Getting the data +^^^^^^^^^^^^^^^^ + +One parameter that is common to all northbound configuration callbacks +is the ``dnode`` parameter. This is a libyang data node structure that +contains information relative to the configuration change that is being +performed. For ``create`` callbacks, it contains the configuration node +that is being added. For ``delete`` callbacks, it contains the +configuration node that is being deleted. For ``modify`` callbacks, it +contains the configuration node that is being modified. + +In order to get the actual data value out of the ``dnode`` variable, we +need to use the ``yang_dnode_get_*()`` wrappers documented in +*lib/yang_wrappers.h*. + +The advantage of passing a ``dnode`` structure to the northbound +callbacks is that the whole candidate being committed is made available, +so the callbacks can obtain values from other portions of the +configuration if necessary. This can be done by providing an xpath +expression to the second parameter of the ``yang_dnode_get_*()`` +wrappers to specify the element we want to get. The example below shows +a callback that gets the values of two leaves that are part of the same +list entry: + +.. code:: c + + static int + ripd_instance_redistribute_metric_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + int type; + uint8_t metric; + + if (event != NB_EV_APPLY) + return NB_OK; + + type = yang_dnode_get_enum(dnode, "../protocol"); + metric = yang_dnode_get_uint8(dnode, NULL); + + rip->route_map[type].metric_config = true; + rip->route_map[type].metric = metric; + rip_redistribute_conf_update(type); + + return NB_OK; + } + +.. + + NOTE: if the wrong ``yang_dnode_get_*()`` wrapper is used, the code + will log an error and abort. An example would be using + ``yang_dnode_get_enum()`` to get the value of a boolean data node. + +No need to check if the configuration value has changed +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A common pattern in CLI commands is this: + +.. code:: c + + DEFUN (...) + { + [snip] + if (new_value == old_value) + return CMD_SUCCESS; + [snip] + } + +Several commands need to check if the new value entered by the user is +the same as the one currently configured. Then, if yes, ignore the +command since nothing was changed. + +The northbound callbacks on the other hand don’t need to perform this +check since they act on effective configuration changes. Using the CLI +as an example, if the operator enters the same command multiple times, +the northbound layer will detect that nothing has changed in the +configuration and will avoid calling the northbound callbacks +unnecessarily. + +In some cases, however, it might be desirable to check for +inconsistencies and notify the northbound when that happens: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/interface + */ + static int ripd_instance_interface_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + const char *ifname; + + if (event != NB_EV_APPLY) + return NB_OK; + + ifname = yang_dnode_get_string(dnode, NULL); + + return rip_enable_if_add(ifname); + } + +.. code:: c + + /* Add interface to rip_enable_if. */ + int rip_enable_if_add(const char *ifname) + { + int ret; + + ret = rip_enable_if_lookup(ifname); + if (ret >= 0) + return NB_ERR_INCONSISTENCY; + + vector_set(rip_enable_interface, + XSTRDUP(MTYPE_RIP_INTERFACE_STRING, ifname)); + + rip_enable_apply_all(); /* TODOVJ */ + + return NB_OK; + } + +In the example above, the ``rip_enable_if_add()`` function should never +return ``NB_ERR_INCONSISTENCY`` in normal conditions. This is because +the northbound layer guarantees that the same interface will never be +added more than once (except when it’s removed and re-added again). But +to be on the safe side it’s probably wise to check for internal +inconsistencies to ensure everything is working as expected. + +Default values +^^^^^^^^^^^^^^ + +Whenever creating a new presence-container or list entry, it’s usually +necessary to initialize certain variables to their default values. FRR +most of the time uses special constants for that purpose +(e.g. ``RIP_DEFAULT_METRIC_DEFAULT``, ``DFLT_BGP_HOLDTIME``, etc). Now +that we have YANG models, we want to fetch the default values from these +models instead. This will allow us to changes default values smoothly +without needing to touch the code. Better yet, it will allow users to +create YANG deviations to define custom default values easily. + +To fetch default values from the loaded YANG models, use the +``yang_get_default_*()`` wrapper functions +(e.g. ``yang_get_default_bool()``) documented in *lib/yang_wrappers.h*. + +Example: + +.. code:: c + + int rip_create(int socket) + { + rip = XCALLOC(MTYPE_RIP, sizeof(struct rip)); + + /* Set initial values. */ + rip->ecmp = yang_get_default_bool("%s/allow-ecmp", RIP_INSTANCE); + rip->default_metric = + yang_get_default_uint8("%s/default-metric", RIP_INSTANCE); + [snip] + } + +Configuration options are edited individually +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Several CLI commands edit multiple configuration options at the same +time. Some examples taken from ripd: \* +``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` - +*/frr-ripd:ripd/instance/timers/flush-interval* - +*/frr-ripd:ripd/instance/timers/holddown-interval* - +*/frr-ripd:ripd/instance/timers/update-interval* \* +``distance (1-255) A.B.C.D/M [WORD]`` - +*/frr-ripd:ripd/instance/distance/source/prefix* - +*/frr-ripd:ripd/instance/distance/source/distance* - +*/frr-ripd:ripd/instance/distance/source/access-list* + +In the new northbound model, there’s one or more separate callbacks for +each configuration option. This usually has implications when converting +code from CLI commands to the northbound commands. An example of this is +the following commit from ripd: +`7cf2f2eaf <https://github.com/opensourcerouting/frr/commit/7cf2f2eaf43ef5df294625d1ab4c708db8293510>`__. +The ``rip_distance_set()`` and ``rip_distance_unset()`` functions were +torn apart and their code split into a few different callbacks. + +For lists and presence-containers, it’s possible to use the +``yang_dnode_set_entry()`` function to attach user data to a libyang +data node, and then retrieve this value in the other callbacks (for the +same node or any of its children) using the ``yang_dnode_get_entry()`` +function. Example: + +.. code:: c + + static int ripd_instance_distance_source_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + struct prefix_ipv4 prefix; + struct route_node *rn; + + if (event != NB_EV_APPLY) + return NB_OK; + + yang_dnode_get_ipv4p(&prefix, dnode, "./prefix"); + + /* Get RIP distance node. */ + rn = route_node_get(rip_distance_table, (struct prefix *)&prefix); + rn->info = rip_distance_new(); + yang_dnode_set_entry(dnode, rn); + + return NB_OK; + } + +.. code:: c + + static int + ripd_instance_distance_source_distance_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + struct route_node *rn; + uint8_t distance; + struct rip_distance *rdistance; + + if (event != NB_EV_APPLY) + return NB_OK; + + /* Set distance value. */ + rn = yang_dnode_get_entry(dnode); + distance = yang_dnode_get_uint8(dnode, NULL); + rdistance = rn->info; + rdistance->distance = distance; + + return NB_OK; + } + +Commands that edit multiple configuration options at the same time can +also use the ``apply_finish`` optional callback, documented as follows +in the *lib/northbound.h* file: + +.. code:: c + + /* + * Optional configuration callback for YANG lists and containers. + * + * The 'apply_finish' callbacks are called after all other callbacks + * during the apply phase (NB_EV_APPLY). These callbacks are called only + * under one of the following two cases: + * * The container or a list entry has been created; + * * Any change is made within the descendants of the list entry or + * container (e.g. a child leaf was modified, created or deleted). + * + * This callback is useful in the cases where a single event should be + * triggered regardless if the container or list entry was changed once + * or multiple times. + * + * dnode + * libyang data node from the YANG list or container. + */ + void (*apply_finish)(const struct lyd_node *dnode); + +Here’s an example of how this callback can be used: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/timers/ + */ + static void ripd_instance_timers_apply_finish(const struct lyd_node *dnode) + { + /* Reset update timer thread. */ + rip_event(RIP_UPDATE_EVENT, 0); + } + +.. code:: c + + { + .xpath = "/frr-ripd:ripd/instance/timers", + .cbs.apply_finish = ripd_instance_timers_apply_finish, + .cbs.cli_show = cli_show_rip_timers, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/flush-interval", + .cbs.modify = ripd_instance_timers_flush_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/holddown-interval", + .cbs.modify = ripd_instance_timers_holddown_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/update-interval", + .cbs.modify = ripd_instance_timers_update_interval_modify, + }, + +In this example, we want to call the ``rip_event()`` function only once +regardless if all RIP timers were modified or only one of them. Without +the ``apply_finish`` callback we’d need to call ``rip_event()`` in the +``modify`` callback of each timer (a YANG leaf), resulting in redundant +call to the ``rip_event()`` function if multiple timers are changed at +once. + +Bonus: libyang user types +^^^^^^^^^^^^^^^^^^^^^^^^^ + +When writing YANG modules, it’s advisable to create derived types for +data types that are used on multiple places (e.g. MAC addresses, IS-IS +networks, etc). Here’s how `RFC +7950 <https://tools.ietf.org/html/rfc7950#page-25>`__ defines derived +types: > YANG can define derived types from base types using the +“typedef” > statement. A base type can be either a built-in type or a +derived > type, allowing a hierarchy of derived types. > > A derived +type can be used as the argument for the “type” statement. > > YANG +Example: > > typedef percent { > type uint8 { > range “0 .. 100”; > } > +} > > leaf completed { > type percent; > } + +Derived types are essentially built-in types with imposed restrictions. +As an example, the ``ipv4-address`` derived type from IETF is defined +using the ``string`` built-in type with a ``pattern`` constraint (a +regular expression): + +:: + + typedef ipv4-address { + type string { + pattern + '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}' + + '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])' + + '(%[\p{N}\p{L}]+)?'; + } + description + "The ipv4-address type represents an IPv4 address in + dotted-quad notation. The IPv4 address may include a zone + index, separated by a % sign. + + The zone index is used to disambiguate identical address + values. For link-local addresses, the zone index will + typically be the interface index number or the name of an + interface. If the zone index is not present, the default + zone of the device will be used. + + The canonical format for the zone index is the numerical + format"; + } + +Sometimes, however, it’s desirable to have a binary representation of +the derived type that is different from the associated built-in type. +Taking the ``ipv4-address`` example above, it would be more convenient +to manipulate this YANG type using ``in_addr`` structures instead of +strings. libyang allow us to do that using the user types plugin: +https://netopeer.liberouter.org/doc/libyang/master/howtoschemaplugins.html#usertypes + +Here’s how the the ``ipv4-address`` derived type is implemented in FRR +(*yang/libyang_plugins/frr_user_types.c*): + +.. code:: c + + static int ipv4_address_store_clb(const char *type_name, const char *value_str, + lyd_val *value, char **err_msg) + { + value->ptr = malloc(sizeof(struct in_addr)); + if (!value->ptr) + return 1; + + if (inet_pton(AF_INET, value_str, value->ptr) != 1) { + free(value->ptr); + return 1; + } + + return 0; + } + +.. code:: c + + struct lytype_plugin_list frr_user_types[] = { + {"ietf-inet-types", "2013-07-15", "ipv4-address", + ipv4_address_store_clb, free}, + {"ietf-inet-types", "2013-07-15", "ipv4-address-no-zone", + ipv4_address_store_clb, free}, + [snip] + {NULL, NULL, NULL, NULL, NULL} /* terminating item */ + }; + +Now, in addition to the string representation of the data value, libyang +will also store the data in the binary format we specified (an +``in_addr`` structure). + +Whenever a new derived type is implemented in FRR, it’s also recommended +to write new wrappers in the *lib/yang_wrappers.c* file +(e.g. ``yang_dnode_get_ipv4()``, ``yang_get_default_ipv4()``, etc). + +Step 5: rewrite the CLI commands as dumb wrappers around the northbound callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the northbound callbacks are implemented, we need to rewrite the +associated CLI commands on top of the northbound layer. This is the +easiest part of the retrofitting process. + +For protocol daemons, it’s recommended to put all CLI commands on a +separate C file (e.g. *ripd/rip_cli.c*). This helps to keep the code +more clean by separating the main protocol code from the user interface. +It should also help when moving the CLI to a separate program in the +future. + +For libfrr commands, it’s not possible to centralize all commands in a +single file because the *extract.pl* script from *vtysh* treats commands +differently depending on the file in which they are defined (e.g. DEFUNs +from *lib/routemap.c* are installed using the ``VTYSH_RMAP`` constant, +which identifies the daemons that support route-maps). In this case, the +CLI commands should be rewritten but maintained in the same file. + +Since all CLI configuration commands from FRR will need to be rewritten, +this is an excellent opportunity to rework this part of the code to make +the commands easier to maintain and extend. These are the three main +recommendations: 1. Always use DEFPY instead of DEFUN to improve code +readability. 2. Always try to join multiple DEFUNs into a single DEFPY +whenever possible. As an example, there’s no need to have both +``distance (1-255) A.B.C.D/M`` and ``distance (1-255) A.B.C.D/M WORD`` +when a single ``distance (1-255) A.B.C.D/M [WORD]`` would suffice. 3. +When necessary, create a separate DEFPY for ``no`` commands so that part +of the configuration command can be made optional for convenience. +Example: +``no timers basic [(5-2147483647) (5-2147483647) (5-2147483647)]``. In +this example, everything after ``no timers basic`` is ignored by FRR, so +it makes sense to accept ``no timers basic`` as a valid command. But it +also makes sense to accept all parameters +(``no timers basic (5-2147483647) (5-2147483647) (5-2147483647)``) to +make it easier to remove the command just by prefixing a “no” to it. + +To rewrite a CLI command as a dumb wrapper around the northbound +callbacks, use the ``nb_cli_cfg_change()`` function. This function +accepts as a parameter an array of ``cli_config_change`` structures that +specify the changes that need to performed on the candidate +configuration. Here’s the declaration of this structure (taken from the +*lib/northbound_cli.h* file): + +.. code:: c + + struct cli_config_change { + /* + * XPath (absolute or relative) of the configuration option being + * edited. + */ + char xpath[XPATH_MAXLEN]; + + /* + * Operation to apply (either NB_OP_CREATE, NB_OP_MODIFY or + * NB_OP_DELETE). + */ + enum nb_operation operation; + + /* + * New value of the configuration option. Should be NULL for typeless + * YANG data (e.g. presence-containers). For convenience, NULL can also + * be used to restore a leaf to its default value. + */ + const char *value; + }; + +The ``nb_cli_cfg_change()`` function positions the CLI command on top on +top of the northbound layer. Instead of changing the running +configuration directly, this function changes the candidate +configuration instead, as described in the [[Transactional CLI]] page. +When the transactional CLI is not in use (i.e. the default mode), then +``nb_cli_cfg_change()`` performs an implicit ``commit`` operation after +changing the candidate configuration. + + NOTE: the ``nb_cli_cfg_change()`` function clones the candidate + configuration before actually editing it. This way, if any error + happens during the editing, the original candidate is restored to + avoid inconsistencies. Either all changes from the configuration + command are performed successfully or none are. It’s like a + mini-transaction but happening on the candidate configuration (thus + the northbound callbacks are not involved). + +Other important details to keep in mind while rewriting the CLI +commands: \* ``nb_cli_cfg_change()`` returns CLI errors codes +(e.g. ``CMD_SUCCESS``, ``CMD_WARNING``), so the return value of this +function can be used as the return value of CLI commands. \* Calls to +``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted to +calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t +necessary anymore and can be removed: ``VTY_DECLVAR_CONTEXT``, +``VTY_DECLVAR_CONTEXT_SUB``, ``VTY_GET_CONTEXT`` and +``VTY_CHECK_CONTEXT``. The ``nb_cli_cfg_change()`` functions uses the +``VTY_CHECK_XPATH`` macro to check if the data node being edited still +exists before doing anything else. + +The examples below provide additional details about how the conversion +should be done. + +Example 1 +^^^^^^^^^ + +In this first example, the *router rip* command becomes a dumb wrapper +around the ``ripd_instance_create()`` callback. Note that we don’t need +to check if the ``/frr-ripd:ripd/instance`` data path already exists +before trying to create it. The northbound will detect when this +presence-container already exists and do nothing. The +``VTY_PUSH_XPATH()`` macro is used to change the vty node and set the +context for other commands under *router rip*. + +.. code:: c + + DEFPY_NOSH (router_rip, + router_rip_cmd, + "router rip", + "Enable a routing process\n" + "Routing Information Protocol (RIP)\n") + { + int ret; + + struct cli_config_change changes[] = { + { + .xpath = "/frr-ripd:ripd/instance", + .operation = NB_OP_CREATE, + .value = NULL, + }, + }; + + ret = nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + if (ret == CMD_SUCCESS) + VTY_PUSH_XPATH(RIP_NODE, changes[0].xpath); + + return ret; + } + +Example 2 +^^^^^^^^^ + +Here we can see the use of relative xpaths (starting with ``./``), which +are more convenient that absolute xpaths (which would be +``/frr-ripd:ripd/instance/default-metric`` in this example). This is +possible because the use of ``VTY_PUSH_XPATH()`` in the *router rip* +command set the vty base xpath to ``/frr-ripd:ripd/instance``. + +.. code:: c + + DEFPY (rip_default_metric, + rip_default_metric_cmd, + "default-metric (1-16)", + "Set a metric of redistribute routes\n" + "Default metric\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./default-metric", + .operation = NB_OP_MODIFY, + .value = default_metric_str, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +In the command below we the ``value`` to NULL to indicate that we want +to set this leaf to its default value. This is better than hardcoding +the default value because the default might change in the future. Also, +users might define custom defaults by using YANG deviations, so it’s +better to write code that works correctly regardless of the default +values defined in the YANG models. + +.. code:: c + + DEFPY (no_rip_default_metric, + no_rip_default_metric_cmd, + "no default-metric [(1-16)]", + NO_STR + "Set a metric of redistribute routes\n" + "Default metric\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./default-metric", + .operation = NB_OP_MODIFY, + .value = NULL, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +Example 3 +^^^^^^^^^ + +This example shows how one command can change multiple leaves at the +same time. + +.. code:: c + + DEFPY (rip_timers, + rip_timers_cmd, + "timers basic (5-2147483647)$update (5-2147483647)$timeout (5-2147483647)$garbage", + "Adjust routing timers\n" + "Basic routing protocol update timers\n" + "Routing table update timer value in second. Default is 30.\n" + "Routing information timeout timer. Default is 180.\n" + "Garbage collection timer. Default is 120.\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./timers/update-interval", + .operation = NB_OP_MODIFY, + .value = update_str, + }, + { + .xpath = "./timers/holddown-interval", + .operation = NB_OP_MODIFY, + .value = timeout_str, + }, + { + .xpath = "./timers/flush-interval", + .operation = NB_OP_MODIFY, + .value = garbage_str, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +Example 4 +^^^^^^^^^ + +This example shows how to create a list entry: + +.. code:: c + + DEFPY (rip_distance_source, + rip_distance_source_cmd, + "distance (1-255) A.B.C.D/M$prefix [WORD$acl]", + "Administrative distance\n" + "Distance value\n" + "IP source prefix\n" + "Access list name\n") + { + char xpath_list[XPATH_MAXLEN]; + struct cli_config_change changes[] = { + { + .xpath = ".", + .operation = NB_OP_CREATE, + }, + { + .xpath = "./distance", + .operation = NB_OP_MODIFY, + .value = distance_str, + }, + { + .xpath = "./access-list", + .operation = acl ? NB_OP_MODIFY : NB_OP_DELETE, + .value = acl, + }, + }; + + snprintf(xpath_list, sizeof(xpath_list), "./distance/source[prefix='%s']", + prefix_str); + + return nb_cli_cfg_change(vty, xpath_list, changes, array_size(changes)); + } + +The ``xpath_list`` variable is used to hold the xpath that identifies +the list entry. The keys of the list entry should be embedded in this +xpath and don’t need to be part of the array of configuration changes. +All entries from the ``changes`` array use relative xpaths which are +based on the xpath of the list entry. + +The ``access-list`` optional leaf can be either modified or deleted +depending whether the optional *WORD* parameter is present or not. + +When deleting a list entry, all non-key leaves can be ignored: + +.. code:: c + + DEFPY (no_rip_distance_source, + no_rip_distance_source_cmd, + "no distance (1-255) A.B.C.D/M$prefix [WORD$acl]", + NO_STR + "Administrative distance\n" + "Distance value\n" + "IP source prefix\n" + "Access list name\n") + { + char xpath_list[XPATH_MAXLEN]; + struct cli_config_change changes[] = { + { + .xpath = ".", + .operation = NB_OP_DELETE, + }, + }; + + snprintf(xpath_list, sizeof(xpath_list), "./distance/source[prefix='%s']", + prefix_str); + + return nb_cli_cfg_change(vty, xpath_list, changes, 1); + } + +Example 5 +^^^^^^^^^ + +This example shows a DEFPY statement that performs two validations +before calling ``nb_cli_cfg_change()``: + +.. code:: c + + DEFPY (ip_rip_authentication_string, + ip_rip_authentication_string_cmd, + "ip rip authentication string LINE$password", + IP_STR + "Routing Information Protocol\n" + "Authentication control\n" + "Authentication string\n" + "Authentication string\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./frr-ripd:rip/authentication/password", + .operation = NB_OP_MODIFY, + .value = password, + }, + }; + + if (strlen(password) > 16) { + vty_out(vty, + "%% RIPv2 authentication string must be shorter than 16\n"); + return CMD_WARNING_CONFIG_FAILED; + } + + if (yang_dnode_exists(vty->candidate_config->dnode, "%s%s", + VTY_GET_XPATH, + "/frr-ripd:rip/authentication/key-chain")) { + vty_out(vty, "%% key-chain configuration exists\n"); + return CMD_WARNING_CONFIG_FAILED; + } + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +These two validations are not strictly necessary since the configuration +change is validated using libyang afterwards. The issue with the libyang +validation is that the error messages from libyang are too verbose: + +:: + + ripd# conf t + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication string XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + % Failed to edit candidate configuration. + + Value "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" does not satisfy the constraint "1..16" (range, length, or pattern). + Failed to create node "authentication-password" as a child of "rip". + YANG path: /frr-interface:lib/interface[name='eth0'][vrf='Default-IP-Routing-Table']/frr-ripd:rip/authentication-password + +On the other hand, the original error message from ripd is much cleaner: + +:: + + ripd# conf t + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication string XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + % RIPv2 authentication string must be shorter than 16 + +The second validation is a bit more complex. If we try to create the +``authentication/password`` leaf when the ``authentication/key-chain`` +leaf already exists (both are under a YANG *choice* statement), libyang +will automatically delete the ``authentication/key-chain`` and create +``authentication/password`` on its place. This is different from the +original ripd behavior where the *ip rip authentication key-chain* +command must be removed before configuring the *ip rip authentication +string* command. + +In the spirit of not introducing any backward-incompatible changes in +the CLI, converted commands should retain some of their validation +checks to preserve their original behavior. + +Step 6: implement the ``cli_show`` callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The traditional method used by FRR to display the running configuration +consists of looping through all CLI nodes all call their ``func`` +callbacks one by one, which in turn read the configuration from internal +variables and dump them to the terminal in the form of CLI commands. + +The problem with this approach is twofold. First, since the callbacks +read the configuration from internal variables, they can’t display +anything other than the running configuration. Second, they don’t have +the ability to display default values when requested by the user +(e.g. *show configuration candidate with-defaults*). + +The new northbound architecture solves these problems by introducing a +new callback: ``cli_show``. Here’s the signature of this function (taken +from the *lib/northbound.h* file): + +.. code:: c + + /* + * Optional callback to show the CLI command associated to the given + * YANG data node. + * + * vty + * the vty terminal to dump the configuration to + * + * dnode + * libyang data node that should be shown in the form of a CLI + * command + * + * show_defaults + * specify whether to display default configuration values or not. + * This parameter can be ignored most of the time since the + * northbound doesn't call this callback for default leaves or + * non-presence containers that contain only default child nodes. + * The exception are commands associated to multiple configuration + * options, in which case it might be desirable to hide one or more + * parts of the command when this parameter is set to false. + */ + void (*cli_show)(struct vty *vty, struct lyd_node *dnode, + bool show_defaults); + +One of the main differences to the old CLI ``func`` callbacks is that +the ``cli_show`` callbacks are associated to YANG data paths and not to +CLI nodes. This means we can define one separate callback for each CLI +command, making the code more modular and easier to maintain (among +other advantages that will be more clear later). For enhanced code +readability, it’s recommended to position the ``cli_show`` callbacks +immediately after their associated command definitions (DEFPYs). + +The ``cli_show`` callbacks are used by the ``nb_cli_show_config_cmds()`` +function to display configurations stored inside ``nb_config`` +structures. The configuration being displayed can be anything from the +running configuration (*show configuration running*), a candidate +configuration (*show configuration candidate*) or a rollback +configuration (*show configuration transaction (1-4294967296)*). The +``nb_cli_show_config_cmds()`` function works by iterating over all data +nodes from the given configuration and calling the ``cli_show`` callback +for the nodes where it’s defined. If a list has dozens of entries, the +``cli_show`` callback associated to this list will be called multiple +times with the ``dnode`` parameter pointing to different list entries on +each iteration. + +For backward compatibility with the *show running-config* command, we +can’t get rid of the CLI ``func`` callbacks at this point in time. +However, we can make the CLI ``func`` callbacks call the corresponding +``cli_show`` callbacks to avoid code duplication. The +``nb_cli_show_dnode_cmds()`` function can be used for that purpose. Once +the CLI retrofitting process finishes for all FRR daemons, we can remove +the legacy CLI ``func`` callbacks and turn *show running-config* into a +shorthand for *show configuration running*. + +Regarding displaying configuration with default values, this is +something that is taken care of by the ``nb_cli_show_config_cmds()`` +function itself. When the *show configuration* command is used without +the *with-defaults* option, ``nb_cli_show_config_cmds()`` will skip +calling ``cli_show`` callbacks for data nodes that contain only default +values (e.g. default leaves or non-presence containers that contain only +default child nodes). There are however some exceptional cases where the +implementer of the ``cli_show`` callback should take into consideration +if default values should be displayed or not. This and other concepts +will be explained in more detail in the examples below. + +.. _example-1-1: + +Example 1 +^^^^^^^^^ + +Command: ``default-metric (1-16)`` + +YANG representation: + +.. code:: yang + + leaf default-metric { + type uint8 { + range "1..16"; + } + default "1"; + description + "Default metric of redistributed routes."; + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance/default-metric", + .cbs.modify = ripd_instance_default_metric_modify, + + .cbs.cli_show = cli_show_rip_default_metric, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_default_metric(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " default-metric %s\n", + yang_dnode_get_string(dnode, NULL)); + } + +In this first example, the *default-metric* command was modeled using a +YANG leaf, and we added a new ``cli_show`` callback attached to the YANG +path of this leaf. + +The callback makes use of the ``yang_dnode_get_string()`` function to +obtain the string value of the configuration option. The following would +also be possible: + +.. code:: c + + vty_out(vty, " default-metric %u\n", + yang_dnode_get_uint8(dnode, NULL)); + +Both options are possible because libyang stores both a binary +representation and a textual representation of all values stored in a +data node (``lyd_node``). For simplicity, it’s recommended to always use +``yang_dnode_get_string()`` in the ``cli_show`` callbacks. + +.. _example-2-1: + +Example 2 +^^^^^^^^^ + +Command: ``router rip`` + +YANG representation: + +.. code:: yang + + container instance { + presence "Present if the RIP protocol is enabled."; + description + "RIP routing instance."; + [snip] + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance", + .cbs.create = ripd_instance_create, + .cbs.delete = ripd_instance_delete, + + .cbs.cli_show = cli_show_router_rip, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_router_rip(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, "!\n"); + vty_out(vty, "router rip\n"); + } + +In this example, the ``cli_show`` callback doesn’t need to obtain any +value from the ``dnode`` parameter since presence-containers don’t hold +any data (apart from their child nodes, but they have their own +``cli_show`` callbacks). + +.. _example-3-1: + +Example 3 +^^^^^^^^^ + +Command: ``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` + +YANG representation: + +.. code:: yang + + container timers { + description + "Settings of basic timers"; + leaf flush-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "120"; + description + "Interval before a route is flushed from the routing + table."; + } + leaf holddown-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "180"; + description + "Interval before better routes are released."; + } + leaf update-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "30"; + description + "Interval at which RIP updates are sent."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + + .xpath = "/frr-ripd:ripd/instance/timers", + + .cbs.cli_show = cli_show_rip_timers, + + }, + + { + .xpath = "/frr-ripd:ripd/instance/timers/flush-interval", + .cbs.modify = ripd_instance_timers_flush_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/holddown-interval", + .cbs.modify = ripd_instance_timers_holddown_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/update-interval", + .cbs.modify = ripd_instance_timers_update_interval_modify, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_timers(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " timers basic %s %s %s\n", + yang_dnode_get_string(dnode, "./update-interval"), + yang_dnode_get_string(dnode, "./holddown-interval"), + yang_dnode_get_string(dnode, "./flush-interval")); + } + +This command is a bit different since it changes three leaves at the +same time. This means we need to have a single ``cli_show`` callback in +order to display the three leaves together in the same line. + +The new ``cli_show_rip_timers()`` callback was added attached to the +*timers* non-presence container that groups the three leaves. Without +the *timers* non-presence container we’d need to display the *timers +basic* command inside the ``cli_show_router_rip()`` callback, which +would break our requirement of having a separate ``cli_show`` callback +for each configuration command. + +.. _example-4-1: + +Example 4 +^^^^^^^^^ + +Command: +``redistribute <kernel|connected|static|ospf|isis|bgp|eigrp|nhrp|table|vnc|babel|sharp> [{metric (0-16)|route-map WORD}]`` + +YANG representation: + +.. code:: yang + + list redistribute { + key "protocol"; + description + "Redistributes routes learned from other routing protocols."; + leaf protocol { + type frr-route-types:frr-route-types-v4; + description + "Routing protocol."; + must '. != "rip"'; + } + leaf route-map { + type string { + length "1..max"; + } + description + "Applies the conditions of the specified route-map to + routes that are redistributed into the RIP routing + instance."; + } + leaf metric { + type uint8 { + range "0..16"; + } + description + "Metric used for the redistributed route. If a metric is + not specified, the metric configured with the + default-metric attribute in RIP router configuration is + used. If the default-metric attribute has not been + configured, the default metric for redistributed routes + is 0."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance/redistribute", + .cbs.create = ripd_instance_redistribute_create, + .cbs.delete = ripd_instance_redistribute_delete, + + .cbs.cli_show = cli_show_rip_redistribute, + }, + { + .xpath = "/frr-ripd:ripd/instance/redistribute/route-map", + .cbs.modify = ripd_instance_redistribute_route_map_modify, + .cbs.delete = ripd_instance_redistribute_route_map_delete, + }, + { + .xpath = "/frr-ripd:ripd/instance/redistribute/metric", + .cbs.modify = ripd_instance_redistribute_metric_modify, + .cbs.delete = ripd_instance_redistribute_metric_delete, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_redistribute(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " redistribute %s", + yang_dnode_get_string(dnode, "./protocol")); + if (yang_dnode_exists(dnode, "./metric")) + vty_out(vty, " metric %s", + yang_dnode_get_string(dnode, "./metric")); + if (yang_dnode_exists(dnode, "./route-map")) + vty_out(vty, " route-map %s", + yang_dnode_get_string(dnode, "./route-map")); + vty_out(vty, "\n"); + } + +Similar to the previous example, the *redistribute* command changes +several leaves at the same time, and we need a single callback to +display all leaves in a single line in accordance to the CLI command. In +this case, the leaves are already grouped by a YANG list so there’s no +need to add a non-presence container. The new ``cli_show`` callback was +attached to the YANG path of the list. + +It’s also worth noting the use of the ``yang_dnode_exists()`` function +to check if optional leaves exist in the configuration before displaying +them. + +.. _example-5-1: + +Example 5 +^^^^^^^^^ + +Command: +``ip rip authentication mode <md5 [auth-length <rfc|old-ripd>]|text>`` + +YANG representation: + +.. code:: yang + + container authentication-scheme { + description + "Specify the authentication scheme for the RIP interface"; + leaf mode { + type enumeration { + [snip] + } + default "none"; + description + "Specify the authentication mode."; + } + leaf md5-auth-length { + when "../mode = 'md5'"; + type enumeration { + [snip] + } + default "20"; + description + "MD5 authentication data length."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + + { + + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme", + + .cbs.cli_show = cli_show_ip_rip_authentication_scheme, + }, + { + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme/mode", + .cbs.modify = lib_interface_rip_authentication_scheme_mode_modify, + }, + { + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme/md5-auth-length", + .cbs.modify = lib_interface_rip_authentication_scheme_md5_auth_length_modify, + .cbs.delete = lib_interface_rip_authentication_scheme_md5_auth_length_delete, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_ip_rip_authentication_scheme(struct vty *vty, + struct lyd_node *dnode, + bool show_defaults) + { + switch (yang_dnode_get_enum(dnode, "./mode")) { + case RIP_NO_AUTH: + vty_out(vty, " no ip rip authentication mode\n"); + break; + case RIP_AUTH_SIMPLE_PASSWORD: + vty_out(vty, " ip rip authentication mode text\n"); + break; + case RIP_AUTH_MD5: + vty_out(vty, " ip rip authentication mode md5"); + if (show_defaults + || !yang_dnode_is_default(dnode, "./md5-auth-length")) { + if (yang_dnode_get_enum(dnode, "./md5-auth-length") + == RIP_AUTH_MD5_SIZE) + vty_out(vty, " auth-length rfc"); + else + vty_out(vty, " auth-length old-ripd"); + } + vty_out(vty, "\n"); + break; + } + } + +This is the most complex ``cli_show`` callback we have in ripd. Its +complexity comes from the following: \* The +``ip rip authentication mode ...`` command changes two YANG leaves at +the same time. \* Part of the command should be hidden when the +``show_defaults`` parameter is set to false. + +This is the behavior we want to implement: + +:: + + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication mode md5 + ripd(config-if)# + ripd(config-if)# show configuration candidate + Configuration: + ! + [snip] + ! + interface eth0 + ip rip authentication mode md5 + ! + end + ripd(config-if)# + ripd(config-if)# show configuration candidate with-defaults + Configuration: + ! + [snip] + ! + interface eth0 + [snip] + ip rip authentication mode md5 auth-length old-ripd + ! + end + +Note that ``auth-length old-ripd`` should be hidden unless the +configuration is shown using the *with-defaults* option. This is why the +``cli_show_ip_rip_authentication_scheme()`` callback needs to consult +the value of the *show_defaults* parameter. It’s expected that only a +very small minority of all ``cli_show`` callbacks will need to consult +the *show_defaults* parameter (there’s a chance this might be the only +case!) + +In the case of the *timers basic* command seen before, we need to +display the value of all leaves even if only one of them has a value +different from the default. Hence the ``cli_show_rip_timers()`` callback +was able to completely ignore the *show_defaults* parameter. + +Step 7: consolidation +~~~~~~~~~~~~~~~~~~~~~ + +As mentioned in the fourth step, the northbound retrofitting process can +happen gradually over time, since both “old” and “new” commands can +coexist without problems. Once all commands from a given daemon were +converted, we can proceed to the consolidation step, which consists of +the following: \* Remove the vty configuration lock, which is enabled by +default in all daemons. Now multiple users should be able to edit the +configuration concurrently, using either shared or private candidate +configurations. \* Reference commit: +`57dccdb1 <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. +\* Stop using the qobj infrastructure to keep track of configuration +objects. This is not necessary anymore, the northbound uses a similar +mechanism to keep track of YANG data nodes in the candidate +configuration. \* Reference commit: +`4e6d63ce <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. +\* Make the daemon SIGHUP handler re-read the configuration file (and +ensure it’s not doing anything other than that). \* Reference commit: +`5e57edb4 <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. + +Final Considerations +-------------------- + +Testing +~~~~~~~ + +Converting CLI commands to the new northbound model can be a complicated +task for beginners, but the more commands one converts, the easier it +gets. It’s highly recommended to perform as much testing as possible on +the converted commands to reduce the likelihood of introducing +regressions. Tools like topotests, ANVL and the `CLI +fuzzer <https://github.com/rwestphal/frr-cli-fuzzer>`__ can be used to +catch hidden bugs that might be present. As usual, it’s also recommended +to use valgrind and static code analyzers to catch other types of +problems like memory leaks. + +Amount of work +~~~~~~~~~~~~~~ + +The output below gives a rough estimate of the total number of +configuration commands that need to be converted per daemon: + +.. code:: sh + + $ for dir in lib zebra bgpd ospfd ospf6d isisd ripd ripngd eigrpd pimd pbrd ldpd nhrpd babeld ; do echo -n "$dir: " && cd $dir && grep -ERn "DEFUN|DEFPY" * | grep -Ev "clippy|show|clear" | wc -l && cd ..; done + lib: 302 + zebra: 181 + bgpd: 569 + ospfd: 198 + ospf6d: 99 + isisd: 126 + ripd: 64 + ripngd: 44 + eigrpd: 58 + pimd: 113 + pbrd: 9 + ldpd: 46 + nhrpd: 24 + babeld: 28 + +As it can be seen, the northbound retrofitting process will demand a lot +of work from FRR developers and should take months to complete. Everyone +is welcome to collaborate! diff --git a/doc/developer/northbound/transactional-cli.rst b/doc/developer/northbound/transactional-cli.rst new file mode 100644 index 0000000..439bb6a --- /dev/null +++ b/doc/developer/northbound/transactional-cli.rst @@ -0,0 +1,244 @@ +Table of Contents +----------------- + +- `Introduction <#introduction>`__ +- `Configuration modes <#config-modes>`__ +- `New commands <#retrofitting-process>`__ + + - `commit check <#cmd1>`__ + - `commit <#cmd2>`__ + - `discard <#cmd3>`__ + - `configuration database max-transactions <#cmd4>`__ + - `configuration load <#cmd5>`__ + - `rollback configuration <#cmd6>`__ + - `show configuration candidate <#cmd7>`__ + - `show configuration compare <#cmd8>`__ + - `show configuration running <#cmd9>`__ + - `show configuration transaction <#cmd10>`__ + - `show yang module <#cmd11>`__ + - `show yang module-translator <#cmd12>`__ + - `update <#cmd13>`__ + - `yang module-translator load <#cmd14>`__ + - `yang module-translator unload <#cmd15>`__ + +Introduction +~~~~~~~~~~~~ + +All FRR daemons have built-in support for the CLI, which can be accessed +either through local telnet or via the vty socket (e.g. by using +*vtysh*). This will not change with the introduction of the Northbound +API. However, a new command-line option will be available for all FRR +daemons: ``--tcli``. When given, this option makes the daemon start with +a transactional CLI and configuration commands behave a bit different. +Instead of editing the running configuration, they will edit the +candidate configuration. In other words, the configuration commands +won’t be applied immediately, that has to be done on a separate step +using the new ``commit`` command. + +The transactional CLI simply leverages the new capabilities provided by +the Northbound API and exposes the concept of candidate configurations +to CLI users too. When the transactional mode is not used, the +configuration commands also edit the candidate configuration, but +there’s an implicit ``commit`` after each command. + +In order for the transactional CLI to work, all configuration commands +need to be converted to the new northbound model. Commands not converted +to the new northbound model will change the running configuration +directly since they bypass the FRR northbound layer. For this reason, +starting a daemon with the transactional CLI is not advisable unless all +of its commands have already been converted. When that’s not the case, +we can run into a situation like this: + +:: + + ospfd(config)# router ospf + ospfd(config-router)# ospf router-id 1.1.1.1 + [segfault in ospfd] + +The segfault above can happen if ``router ospf`` edits the candidate +configuration but ``ospf router-id 1.1.1.1`` edits the running +configuration. The second command tries to set +``ospf->router_id_static`` but, since the previous ``router ospf`` +command hasn’t been commited yet, the ``ospf`` global variable is set to +NULL, which leads to the crash. Besides this problem, having a set of +commands that edit the candidate configuration and others that edit the +running configuration is confusing at best. The ``--tcli`` option should +be used only by developers until the northbound retrofitting process is +complete. + +Configuration modes +~~~~~~~~~~~~~~~~~~~ + +When using the transactional CLI (``--tcli``), FRR supports three +different forms of the ``configure`` command: \* ``configure terminal``: +in this mode, a single candidate configuration is shared by all users. +This means that one user might delete a configuration object that’s +being edited by another user, in which case the CLI will detect and +report the problem. If one user issues the ``commit`` command, all +changes done by all users are committed. \* ``configure private``: users +have a private candidate configuration that is edited separately from +the other users. The ``commit`` command commits only the changes done by +the user. \* ``configure exclusive``: similar to ``configure private``, +but also locks the running configuration to prevent other users from +changing it. The configuration lock is released when the user exits the +configuration mode. + +When using ``configure terminal`` or ``configure private``, the +candidate configuration being edited might become outdated if another +user commits a different candidate configuration on another session. +TODO: show image to illustrate the problem. + +New commands +~~~~~~~~~~~~ + +The list below contains the new CLI commands introduced by Northbound +API. The commands are available when a daemon is started using the +transactional CLI (``--tcli``). Currently ``vtysh`` doesn’t support any +of these new commands. + +Please refer to the [[Demos]] page to see a demo of the transactional +CLI in action. + +-------------- + +``commit check`` +'''''''''''''''' + +Check if the candidate configuration is valid or not. + +``commit [force] [comment LINE...]`` +'''''''''''''''''''''''''''''''''''' + +Commit the changes done in the candidate configuration into the running +configuration. + +Options: \* ``force``: commit even if the candidate configuration is +outdated. It’s usually a better option to use the ``update`` command +instead. \* ``comment LINE...``: assign a comment to the configuration +transaction. This comment is displayed when viewing the recorded +transactions in the output of the ``show configuration transaction`` +command. + +``discard`` +''''''''''' + +Discard the changes done in the candidate configuration. + +``configuration database max-transactions (1-100)`` +''''''''''''''''''''''''''''''''''''''''''''''''''' + +Set the maximum number of transactions to store in the rollback log. + +``configuration load <file [<json|xml> [translate WORD]] FILENAME|transaction (1-4294967296)> [replace]`` +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Load a new configuration into the candidate configuration. When loading +the configuration from a file, it’s assumed that the configuration will +be in the form of CLI commands by default. The ``json`` and ``xml`` +options can be used to load configurations in the JSON and XML formats, +respectively. It’s also possible to load a configuration from a previous +transaction by specifying the desired transaction ID +(``(1-4294967296)``). + +Options: \* ``translate WORD``: translate the JSON/XML configuration +file using the YANG module translator. \* ``replace``: replace the +candidate by the loaded configuration. The default is to merge the +loaded configuration into the candidate configuration. + +``rollback configuration (1-4294967296)`` +''''''''''''''''''''''''''''''''''''''''' + +Roll back the running configuration to a previous configuration +identified by its transaction ID (``(1-4294967296)``). + +``show configuration candidate [<json|xml> [translate WORD]] [<with-defaults|changes>]`` +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Show the candidate configuration. + +Options: \* ``json``: show the configuration in the JSON format. \* +``xml``: show the configuration in the XML format. \* +``translate WORD``: translate the JSON/XML output using the YANG module +translator. \* ``with-defaults``: show default values that are hidden by +default. \* ``changes``: show only the changes done in the candidate +configuration. + +``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]`` +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Show the difference between two different configurations. + +Options: \* ``json``: show the configuration differences in the JSON +format. \* ``xml``: show the configuration differences in the XML +format. \* ``translate WORD``: translate the JSON/XML output using the +YANG module translator. + +``show configuration running [<json|xml> [translate WORD]] [with-defaults]`` +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Show the running configuration. + +Options: \* ``json``: show the configuration in the JSON format. \* +``xml``: show the configuration in the XML format. \* +``translate WORD``: translate the JSON/XML output using the YANG module +translator. \* ``with-defaults``: show default values that are hidden by +default. + + NOTE: ``show configuration running`` shows only the running + configuration as known by the northbound layer. Configuration + commands not converted to the new northbound model will not be + displayed. To show the full running configuration, the legacy + ``show running-config`` command must be used. + +``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]`` +''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +When a transaction ID (``(1-4294967296)``) is given, show the +configuration associated to the previously committed transaction. + +When a transaction ID is not given, show all recorded transactions in +the rollback log. + +Options: \* ``json``: show the configuration in the JSON format. \* +``xml``: show the configuration in the XML format. \* +``translate WORD``: translate the JSON/XML output using the YANG module +translator. \* ``with-defaults``: show default values that are hidden by +default. \* ``changes``: show changes compared to the previous +transaction. + +``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]`` +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +When a YANG module is not given, show all loaded YANG modules. +Otherwise, show detailed information about the given module. + +Options: \* ``module-translator WORD``: change the context to modules +loaded by the specified YANG module translator. \* ``summary``: display +summary information about the module. \* ``tree``: display module in the +tree (RFC 8340) format. \* ``yang``: display module in the YANG format. +\* ``yin``: display module in the YIN format. + +``show yang module-translator`` +''''''''''''''''''''''''''''''' + +Show all loaded YANG module translators. + +``update`` +'''''''''' + +Rebase the candidate configuration on top of the latest running +configuration. Conflicts are resolved automatically by giving preference +to the changes done in the candidate configuration. + +The candidate configuration might be outdated if the running +configuration was updated after the candidate was created. + +``yang module-translator load FILENAME`` +'''''''''''''''''''''''''''''''''''''''' + +Load a YANG module translator from the filesystem. + +``yang module-translator unload WORD`` +'''''''''''''''''''''''''''''''''''''' + +Unload a YANG module translator identified by its name. diff --git a/doc/developer/northbound/yang-module-translator.rst b/doc/developer/northbound/yang-module-translator.rst new file mode 100644 index 0000000..aa527ce --- /dev/null +++ b/doc/developer/northbound/yang-module-translator.rst @@ -0,0 +1,629 @@ +Table of Contents +----------------- + +- `Introduction <#introduction>`__ +- `Deviation Modules <#deviation-modules>`__ +- `Translation Tables <#translation-tables>`__ +- `CLI Demonstration <#cli-demonstration>`__ +- `Implementation Details <#implementation-details>`__ + +Introduction +------------ + +One key requirement for the FRR northbound architecture is that it +should be possible to configure/monitor FRR using different sets of YANG +models. This is especially important considering that the industry +hasn’t reached a consensus to provide a single source of standard models +for network management. At this moment both the IETF and OpenConfig +models are widely implemented and are unlikely to converge, at least not +in the short term. In the ideal scenario, management applications should +be able to use either IETF or OpenConfig models to configure and monitor +FRR programatically (or even both at the same time!). + +But how can FRR support multiple sets of YANG models at the same time? +There must be only a single source of truth that models the existing +implementation accurately (the native models). Writing different code +paths or callbacks for different models would be inviable, it would lead +to a lot of duplicated code and extra maintenance overhead. + +In order to support different sets of YANG modules without introducing +the overhead of writing additional code, the solution is to create a +mechanism that dynamically translates YANG instance data between +non-native models to native models and vice-versa. Based on this idea, +an experimental YANG module translator was implemented within the FRR +northbound layer. The translator works by translating XPaths at runtime +using translation tables provided by the user. The translator itself is +modeled using YANG and users can create translators using simple JSON +files. + +A YANG module translator consists of two components: deviation modules +and translation tables. + +Deviation Modules +----------------- + +The first step when writing a YANG module translator is to create a +`deviations <https://tools.ietf.org/html/rfc7950#page-131>`__ module for +each module that is going be translated. This is necessary because in +most cases it won’t be possible to create a perfect translator that +covers the non-native models on their entirety. Some non-native modules +might contain nodes that can’t be mapped to a corresponding node in the +FRR native models. This is either because the corresponding +functionality is not implemented in FRR or because it’s modeled in a +different way that is incompatible. + +An an example, *ripd* doesn’t have BFD support yet, so we need to create +a YANG deviation to modify the *ietf-rip* module and remove the ``bfd`` +container from it: + +.. code:: yang + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:bfd" { + deviate not-supported; + } + +In the example below, while both the *frr-ripd* and *ietf-rip* modules +support RIP authentication, they model the authentication data in +different ways, making translation not possible given the constraints of +the current module translator. A new deviation is necessary to remove +the ``authentication`` container from the *ietf-rip* module: + +.. code:: yang + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:authentication" { + deviate not-supported; + } + +.. + + NOTE: it should be possible to translate the + ``ietf-rip:authentication`` container if the *frr-ripd* module is + modified to model the corresponding data in a compatible way. Another + option is to improve the module translator to make more complex + translations possible, instead of requiring one-to-one XPath + mappings. + +Sometimes creating a mapping between nodes from the native and +non-native models is possible, but the nodes have different properties +that need to be normalized to allow the translation. In the example +below, a YANG deviation is used to change the type and the default value +from a node from the ``ietf-rip`` module. + +.. code:: yang + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:timers/ietf-rip:flush-interval" { + deviate replace { + default "120"; + } + deviate replace { + type uint32; + } + } + +The deviation modules allow the management applications to know which +parts of the custom modules (e.g. IETF/OC) can be used to configure and +monitor FRR. + +In order to facilitate the process of creating YANG deviation modules, +the *gen_yang_deviations* tool was created to automate part of the +process. This tool creates a “not-supported” deviation for all nodes +from the given non-native module. Example: + +:: + + $ tools/gen_yang_deviations ietf-rip > yang/ietf/frr-deviations-ietf-rip.yang + $ head -n 40 yang/ietf/frr-deviations-ietf-rip.yang + deviation "/ietf-rip:clear-rip-route" { + deviate not-supported; + } + + deviation "/ietf-rip:clear-rip-route/ietf-rip:input" { + deviate not-supported; + } + + deviation "/ietf-rip:clear-rip-route/ietf-rip:input/ietf-rip:rip-instance" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:enabled" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:route-policy" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:default-metric" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:distance" { + deviate not-supported; + } + + deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:triggered-update-threshold" { + deviate not-supported; + } + +Once all existing nodes are listed in the deviation module, it’s easy to +check the deviations that need to be removed or modified. This is more +convenient than starting with a blank deviations module and listing +manually all nodes that need to be deviated. + +After removing and/or modifying the auto-generated deviations, the next +step is to write the module XPath translation table as we’ll see in the +next section. Before that, it’s possible to use the *yanglint* tool to +check how the non-native module looks like after applying the +deviations. Example: + +:: + + $ yanglint -f tree yang/ietf/ietf-rip@2018-02-03.yang yang/ietf/frr-deviations-ietf-rip.yang + module: ietf-rip + + augment /ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol: + +--rw rip + +--rw originate-default-route + | +--rw enabled? boolean <false> + +--rw default-metric? uint8 <1> + +--rw distance? uint8 <0> + +--rw timers + | +--rw update-interval? uint32 <30> + | +--rw holddown-interval? uint32 <180> + | +--rw flush-interval? uint32 <120> + +--rw interfaces + | +--rw interface* [interface] + | +--rw interface ietf-interfaces:interface-ref + | +--rw split-horizon? enumeration <simple> + +--ro ipv4 + +--ro neighbors + | +--ro neighbor* [ipv4-address] + | +--ro ipv4-address ietf-inet-types:ipv4-address + | +--ro last-update? ietf-yang-types:date-and-time + | +--ro bad-packets-rcvd? ietf-yang-types:counter32 + | +--ro bad-routes-rcvd? ietf-yang-types:counter32 + +--ro routes + +--ro route* [ipv4-prefix] + +--ro ipv4-prefix ietf-inet-types:ipv4-prefix + +--ro next-hop? ietf-inet-types:ipv4-address + +--ro interface? ietf-interfaces:interface-ref + +--ro metric? uint8 + + rpcs: + +---x clear-rip-route + +.. + + NOTE: the same output can be obtained using the + ``show yang module module-translator ietf ietf-rip tree`` command in + FRR once the *ietf* module translator is loaded. + +In the example above, it can be seen that the vast majority of the +*ietf-rip* nodes were removed because of the “not-supported” deviations. +When a module translator is loaded, FRR calculates the coverage of the +translator by dividing the number of YANG nodes before applying the +deviations by the number of YANG nodes after applying the deviations. +The calculated coverage is displayed in the output of the +``show yang module-translator`` command: + +:: + + ripd# show yang module-translator + Family Module Deviations Coverage (%) + ----------------------------------------------------------------------- + ietf ietf-interfaces frr-deviations-ietf-interfaces 3.92 + ietf ietf-routing frr-deviations-ietf-routing 1.56 + ietf ietf-rip frr-deviations-ietf-rip 13.60 + +As it can be seen in the output above, the *ietf* module translator +covers only ~13% of the original *ietf-rip* module. This is in part +because the *ietf-rip* module models both RIPv2 and RIPng. Also, +*ietf-rip.yang* contains several knobs that aren’t implemented in *ripd* +yet (e.g. BFD support, per-interface timers, statistics, etc). Work can +be done over time to increase the coverage to a more reasonable number. + +Translation Tables +------------------ + +Below is an example of a translator for the IETF family of models: + +.. code:: json + + { + "frr-module-translator:frr-module-translator": { + "family": "ietf", + "module": [ + { + "name": "ietf-interfaces@2018-01-09", + "deviations": "frr-deviations-ietf-interfaces", + "mappings": [ + { + "custom": "/ietf-interfaces:interfaces/interface[name='KEY1']", + "native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']" + }, + { + "custom": "/ietf-interfaces:interfaces/interface[name='KEY1']/description", + "native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/description" + } + ] + }, + { + "name": "ietf-routing@2018-01-25", + "deviations": "frr-deviations-ietf-routing", + "mappings": [ + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']", + "native": "/frr-ripd:ripd/instance" + } + ] + }, + { + "name": "ietf-rip@2018-02-03", + "deviations": "frr-deviations-ietf-rip", + "mappings": [ + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric", + "native": "/frr-ripd:ripd/instance/default-metric" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/distance", + "native": "/frr-ripd:ripd/instance/distance/default" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/originate-default-route/enabled", + "native": "/frr-ripd:ripd/instance/default-information-originate" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/update-interval", + "native": "/frr-ripd:ripd/instance/timers/update-interval" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/holddown-interval", + "native": "/frr-ripd:ripd/instance/timers/holddown-interval" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/flush-interval", + "native": "/frr-ripd:ripd/instance/timers/flush-interval" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']", + "native": "/frr-ripd:ripd/instance/interface[.='KEY1']" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']/split-horizon", + "native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/frr-ripd:rip/split-horizon" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']", + "native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/last-update", + "native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/last-update" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-packets-rcvd", + "native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-packets-rcvd" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-routes-rcvd", + "native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-routes-rcvd" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']", + "native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/next-hop", + "native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/next-hop" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/interface", + "native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/interface" + }, + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/metric", + "native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/metric" + }, + { + "custom": "/ietf-rip:clear-rip-route", + "native": "/frr-ripd:clear-rip-route" + } + ] + } + ] + } + } + +The main motivation to use YANG itself to model YANG module translators +was a practical one: leverage *libyang* to validate the structure of the +user input (JSON files) instead of doing that manually in the +*lib/yang_translator.c* file (tedious and error-prone work). + +Module translators can be loaded using the following CLI command: + +:: + + ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json + % Module translator "ietf" loaded successfully. + +Module translators can also be loaded/unloaded programatically using the +``yang_translator_load()/yang_translator_unload()`` functions within the +northbound plugins. These functions are documented in the +*lib/yang_translator.h* file. + +Each module translator must be assigned a “family” identifier +(e.g. IETF, OpenConfig), and can contain mappings for multiple +interrelated YANG modules. The mappings consist of pairs of +custom/native XPath expressions that should be equivalent, despite +belonging to different YANG modules. + +Example: + +.. code:: json + + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric", + "native": "/frr-ripd:ripd/instance/default-metric" + }, + +The nodes pointed by the custom and native XPaths must have compatible +types. In the case of the example above, both nodes point to a YANG leaf +of type ``uint8``, so the mapping is valid. + +In the example below, the “custom” XPath points to a YANG list +(typeless), and the “native” XPath points to a YANG leaf-list of +strings. In this exceptional case, the types are also considered to be +compatible. + +.. code:: json + + { + "custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']", + "native": "/frr-ripd:ripd/instance/interface[.='KEY1']" + }, + +The ``KEY1..KEY4`` values have a special meaning and are used to +preserve the list keys while performing the XPath translation. + +Once a YANG module translator is loaded and validated at a syntactic +level using *libyang*, further validations are performed to check for +missing mappings (after loading the deviation modules) and incompatible +YANG types. Example: + +:: + + ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json + % Failed to load "/usr/local/share/yang/ietf/frr-ietf-translator.json" + + Please check the logs for more details. + +:: + + 2018/09/03 15:18:45 RIP: yang_translator_validate_cb: YANG types are incompatible (xpath: "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/default-metric") + 2018/09/03 15:18:45 RIP: yang_translator_validate_cb: missing mapping for "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/distance" + 2018/09/03 15:18:45 RIP: yang_translator_validate: failed to validate "ietf" module translator: 2 error(s) + +Overall, this translation mechanism based on XPath mappings is simple +and functional, but only to a certain extent. The native models need to +be reasonably similar to the models that are going be translated, +otherwise the translation is compromised and a good coverage can’t be +achieved. Other translation techniques must be investigated to address +this shortcoming and make it possible to create more powerful YANG +module translators. + +YANG module translators can be evaluated based on the following metrics: +\* Translation potential: is it possible to make complex translations, +taking several variables into account? \* Complexity: measure of how +easy or hard it is to write a module translator. \* Speed: measure of +how fast the translation can be achieved. Translation speed is of +fundamental importance, especially for operational data. \* Robustness: +can the translator be checked for inconsistencies at load time? A module +translator based on scripts wouldn’t fare well on this metric. \* +Round-trip conversions: can the translated data be translated back to +the original format without information loss? + +CLI Demonstration +----------------- + +As of now the only northbound client that supports the YANG module +translator is the FRR embedded CLI. The confd and sysrepo plugins need +to be extended to support the module translator, which might be used not +only for configuration data, but also for operational data, RPCs and +notifications. + +In this demonstration, we’ll use the CLI ``configuration load`` command +to load the following JSON configuration file specified using the IETF +data hierarchy: + +.. code:: json + + { + "ietf-interfaces:interfaces": { + "interface": [ + { + "description": "Engineering", + "name": "eth0" + } + ] + }, + "ietf-routing:routing": { + "control-plane-protocols": { + "control-plane-protocol": [ + { + "name": "main", + "type": "ietf-rip:ripv2", + "ietf-rip:rip": { + "default-metric": "2", + "distance": "80", + "interfaces": { + "interface": [ + { + "interface": "eth0", + "split-horizon": "poison-reverse" + } + ] + }, + "originate-default-route": { + "enabled": "true" + }, + "timers": { + "flush-interval": "241", + "holddown-interval": "181", + "update-interval": "31" + } + } + } + ] + } + } + } + +In order to load this configuration file, it’s necessary to load the +IETF module translator first. Then, when entering the +``configuration load`` command, the ``translate ietf`` parameters must +be given to specify that the input needs to be translated using the +previously loaded ``ietf`` module translator. Example: + +:: + + ripd(config)# configuration load file json /mnt/renato/git/frr/yang/example/ietf-rip.json + % Failed to load configuration: + + Unknown element "interfaces". + ripd(config)# + ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json + % Module translator "ietf" loaded successfully. + + ripd(config)# + ripd(config)# configuration load file json translate ietf /mnt/renato/git/frr/yang/example/ietf-rip.json + +Now let’s check the candidate configuration to see if the configuration +file was loaded successfully: + +:: + + ripd(config)# show configuration candidate + Configuration: + ! + frr version 5.1-dev + frr defaults traditional + ! + interface eth0 + description Engineering + ip rip split-horizon poisoned-reverse + ! + router rip + default-metric 2 + distance 80 + network eth0 + default-information originate + timers basic 31 181 241 + ! + end + ripd(config)# show configuration candidate json + { + "frr-interface:lib": { + "interface": [ + { + "name": "eth0", + "vrf": "default", + "description": "Engineering", + "frr-ripd:rip": { + "split-horizon": "poison-reverse" + } + } + ] + }, + "frr-ripd:ripd": { + "instance": { + "default-metric": 2, + "distance": { + "default": 80 + }, + "interface": [ + "eth0" + ], + "default-information-originate": true, + "timers": { + "flush-interval": 241, + "holddown-interval": 181, + "update-interval": 31 + } + } + } + } + +As it can be seen, the candidate configuration is identical to the one +defined in the *ietf-rip.json* file, only the structure is different. +This means that the *ietf-rip.json* file was translated successfully. + +The ``ietf`` module translator can also be used to do the translation in +other direction: transform data from the native format to the IETF +format. This is shown below by altering the output of the +``show configuration candidate json`` command using the +``translate ietf`` parameter: + +:: + + ripd(config)# show configuration candidate json translate ietf + { + "ietf-interfaces:interfaces": { + "interface": [ + { + "name": "eth0", + "description": "Engineering" + } + ] + }, + "ietf-routing:routing": { + "control-plane-protocols": { + "control-plane-protocol": [ + { + "type": "ietf-rip:ripv2", + "name": "main", + "ietf-rip:rip": { + "interfaces": { + "interface": [ + { + "interface": "eth0", + "split-horizon": "poison-reverse" + } + ] + }, + "default-metric": 2, + "distance": 80, + "originate-default-route": { + "enabled": true + }, + "timers": { + "flush-interval": 241, + "holddown-interval": 181, + "update-interval": 31 + } + } + } + ] + } + } + } + +As expected, this output is exactly identical to the configuration +defined in the *ietf-rip.json* file. The module translator was able to +do a round-trip conversion without information loss. + +Implementation Details +---------------------- + +A different libyang context is allocated for each YANG module +translator. This is important to avoid collisions and ensure that +non-native data can’t be instantiated in the running and candidate +configurations. diff --git a/doc/developer/northbound/yang-tools.rst b/doc/developer/northbound/yang-tools.rst new file mode 100644 index 0000000..346efca --- /dev/null +++ b/doc/developer/northbound/yang-tools.rst @@ -0,0 +1,112 @@ +Yang Tools +~~~~~~~~~~ + +Here's some information about various tools for working with yang +models. + +yanglint cheat sheet +~~~~~~~~~~~~~~~~~~~~ + + libyang project includes a feature-rich tool called yanglint(1) for + validation and conversion of the schemas and YANG modeled data. The + source codes are located at /tools/lint and can be used to explore + how an application is supposed to use the libyang library. + yanglint(1) binary as well as its man page are installed together + with the library itself. + +Validate a YANG module: + +.. code:: sh + + $ yanglint -p <yang-search-path> module.yang + +Generate tree representation of a YANG module: + +.. code:: sh + + $ yanglint -p <yang-search-path> -f tree module.yang + +Validate JSON/XML instance data: + +.. code:: sh + + $ yanglint -p <yang-search-path> module.yang data.{json,xml} + +Convert JSON/XML instance data to another format: + +.. code:: sh + + $ yanglint -p <yang-search-path> -f xml module.yang data.json + $ yanglint -p <yang-search-path> -f json module.yang data.xml + +*yanglint* also features an interactive mode which is very useful when +needing to validate data from multiple modules at the same time. The +*yanglint* README provides several examples: +https://github.com/CESNET/libyang/blob/master/tools/lint/examples/README.md + +Man page (groff): +https://github.com/CESNET/libyang/blob/master/tools/lint/yanglint.1 + +pyang cheat sheet +~~~~~~~~~~~~~~~~~ + + pyang is a YANG validator, transformator and code generator, written + in python. It can be used to validate YANG modules for correctness, + to transform YANG modules into other formats, and to generate code + from the modules. + +Obtaining and installing pyang: + +.. code:: sh + + $ git clone https://github.com/mbj4668/pyang.git + $ cd pyang/ + $ sudo python setup.py install + +Validate a YANG module: + +.. code:: sh + + $ pyang --ietf -p <yang-search-path> module.yang + +Generate tree representation of a YANG module: + +.. code:: sh + + $ pyang -f tree -p <yang-search-path> module.yang + +Indent a YANG file: + +.. code:: sh + + $ pyang -p <yang-search-path> \ + --keep-comments -f yang --yang-canonical \ + module.yang -o module.yang + +Generate skeleton instance data: \* XML: + +.. code:: sh + + $ pyang -p <yang-search-path> \ + -f sample-xml-skeleton --sample-xml-skeleton-defaults \ + module.yang [augmented-module1.yang ...] -o module.xml + +- JSON: + +.. code:: sh + + $ pyang -p <yang-search-path> \ + -f jsonxsl module.yang -o module.xsl + $ xsltproc -o module.json module.xsl module.xml + +Validate XML instance data (works only with YANG 1.0): + +.. code:: sh + + $ yang2dsdl -v module.xml module.yang + +vim +~~~ + +YANG syntax highlighting for vim: +https://github.com/nathanalderson/yang.vim diff --git a/doc/developer/ospf-api.rst b/doc/developer/ospf-api.rst new file mode 100644 index 0000000..41c31b2 --- /dev/null +++ b/doc/developer/ospf-api.rst @@ -0,0 +1,383 @@ +OSPF API Documentation +====================== + +Disclaimer +---------- + +The OSPF daemon contains an API for application access to the LSA database. +This API and documentation was created by Ralph Keller, originally as patch for +Zebra. Unfortunately, the page containing documentation for the API is no +longer online. This page is an attempt to recreate documentation for the API +(with lots of help from the WayBackMachine). + +Ralph has kindly licensed this documentation under GPLv2+. Please preserve the +acknowledgements at the bottom of this document. + +Introduction +------------ + +This page describes an API that allows external applications to access the +link-state database (LSDB) of the OSPF daemon. The implementation is based on +the OSPF code from FRRouting (forked from Quagga and formerly Zebra) routing +protocol suite and is subject to the GNU General Public License. The OSPF API +provides you with the following functionality: + +- Retrieval of the full or partial link-state database of the OSPF daemon. + This allows applications to obtain an exact copy of the LSDB including router + LSAs, network LSAs and so on. Whenever a new LSA arrives at the OSPF daemon, + the API module immediately informs the application by sending a message. This + way, the application is always synchronized with the LSDB of the OSPF daemon. +- Origination of own opaque LSAs (of type 9, 10, or 11) which are then + distributed transparently to other routers within the flooding scope and + received by other applications through the OSPF API. + +Opaque LSAs, which are described in :rfc:`2370`, allow you to distribute +application-specific information within a network using the OSPF protocol. The +information contained in opaque LSAs is transparent for the routing process but +it can be processed by other modules such as traffic engineering (e.g., +MPLS-TE). + +Architecture +------------ + +The following picture depicts the architecture of the Quagga/Zebra protocol +suite. The OSPF daemon is extended with opaque LSA capabilities and an API for +external applications. The OSPF core module executes the OSPF protocol by +discovering neighbors and exchanging neighbor state. The opaque module, +implemented by Masahiko Endo, provides functions to exchange opaque LSAs +between routers. Opaque LSAs can be generated by several modules such as the +MPLS-TE module or the API server module. These modules then invoke the opaque +module to flood their data to neighbors within the flooding scope. + +The client, which is an application potentially running on a different node +than the OSPF daemon, links against the OSPF API client library. This client +library establishes a socket connection with the API server module of the OSPF +daemon and uses this connection to retrieve LSAs and originate opaque LSAs. + +.. figure:: ../figures/ospf_api_architecture.png + :alt: image + + image + +The OSPF API server module works like any other internal opaque module (such as +the MPLS-TE module), but listens to connections from external applications that +want to communicate with the OSPF daemon. The API server module can handle +multiple clients concurrently. + +One of the main objectives of the implementation is to make as little changes +to the existing Zebra code as possible. + +Installation & Configuration +---------------------------- + +Download FRRouting and unpack it. + +Configure and build FRR (note that ``--enable-opaque-lsa`` also enables the +ospfapi server and ospfclient). + +:: + + % sh ./configure --enable-opaque-lsa + % make + +This should also compile the client library and sample application in +ospfclient. + +Make sure that you have enabled opaque LSAs in your configuration. Add the +``ospf opaque-lsa`` statement to your :file:`ospfd.conf`: + +:: + + ! -*- ospf -*- + ! + ! OSPFd sample configuration file + ! + ! + hostname xxxxx + password xxxxx + + router ospf + router-id 10.0.0.1 + network 10.0.0.1/24 area 1 + neighbor 10.0.0.2 + network 10.0.1.2/24 area 1 + neighbor 10.0.1.1 + ospf opaque-lsa <============ add this statement! + +Usage +----- + +In the following we describe how you can use the sample application to +originate opaque LSAs. The sample application first registers with the OSPF +daemon the opaque type it wants to inject and then waits until the OSPF daemon +is ready to accept opaque LSAs of that type. Then the client application +originates an opaque LSA, waits 10 seconds and then updates the opaque LSA with +new opaque data. After another 20 seconds, the client application deletes the +opaque LSA from the LSDB. If the clients terminates unexpectedly, the OSPF API +module will remove all the opaque LSAs that the application registered. Since +the opaque LSAs are flooded to other routers, we will see the opaque LSAs in +all routers according to the flooding scope of the opaque LSA. + +We have a very simple demo setup, just two routers connected with an ATM +point-to-point link. Start the modified OSPF daemons on two adjacent routers. +First run on msr2: + +.. code-block:: console + + # ./ospfd --apiserver -f /usr/local/etc/ospfd.conf + +And on the neighboring router msr3: + +.. code-block:: console + + # ./ospfd --apiserver -f /usr/local/etc/ospfd.conf + +Now the two routers form adjacency and start exchanging their databases. +Looking at the OSPF daemon of msr2 (or msr3), you see this: + +.. code-block:: console + + ospfd> show ip ospf database + + OSPF Router with ID (10.0.0.1) + + Router Link States (Area 0.0.0.1) + + Link ID ADV Router Age Seq# CkSum Link count + 10.0.0.1 10.0.0.1 55 0x80000003 0xc62f 2 + 10.0.0.2 10.0.0.2 55 0x80000003 0xe3e4 3 + + Net Link States (Area 0.0.0.1) + + Link ID ADV Router Age Seq# CkSum + 10.0.0.2 10.0.0.2 60 0x80000001 0x5fcb + +Now we start the sample main application that originates an opaque LSA. + +.. code-block:: console + + # cd ospfapi/apiclient + # ./main msr2 10 250 20 0.0.0.0 0.0.0.1 + +This originates an opaque LSA of type 10 (area local), with opaque type 250 +(experimental), opaque id of 20 (chosen arbitrarily), interface address 0.0.0.0 +(which is used only for opaque LSAs type 9), and area 0.0.0.1 + +Again looking at the OSPF database you see: + +.. code-block:: console + + ospfd> show ip ospf database + + OSPF Router with ID (10.0.0.1) + + Router Link States (Area 0.0.0.1) + + Link ID ADV Router Age Seq# CkSum Link count + 10.0.0.1 10.0.0.1 437 0x80000003 0xc62f 2 + 10.0.0.2 10.0.0.2 437 0x80000003 0xe3e4 3 + + Net Link States (Area 0.0.0.1) + + Link ID ADV Router Age Seq# CkSum + 10.0.0.2 10.0.0.2 442 0x80000001 0x5fcb + + Area-Local Opaque-LSA (Area 0.0.0.1) + + Opaque-Type/Id ADV Router Age Seq# CkSum + 250.0.0.20 10.0.0.1 0 0x80000001 0x58a6 <=== opaque LSA + +You can take a closer look at this opaque LSA: + +.. code-block:: console + + ospfd> show ip ospf database opaque-area + + OSPF Router with ID (10.0.0.1) + + + Area-Local Opaque-LSA (Area 0.0.0.1) + + LS age: 4 + Options: 66 + LS Type: Area-Local Opaque-LSA + Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID) + Advertising Router: 10.0.0.1 + LS Seq Number: 80000001 + Checksum: 0x58a6 + Length: 24 + Opaque-Type 250 (Private/Experimental) + Opaque-ID 0x14 + Opaque-Info: 4 octets of data + Added using OSPF API: 4 octets of opaque data + Opaque data: 1 0 0 0 <==== counter is 1 + +Note that the main application updates the opaque LSA after 10 seconds, then it +looks as follows: + +.. code-block:: console + + ospfd> show ip ospf database opaque-area + + OSPF Router with ID (10.0.0.1) + + + Area-Local Opaque-LSA (Area 0.0.0.1) + + LS age: 1 + Options: 66 + LS Type: Area-Local Opaque-LSA + Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID) + Advertising Router: 10.0.0.1 + LS Seq Number: 80000002 + Checksum: 0x59a3 + Length: 24 + Opaque-Type 250 (Private/Experimental) + Opaque-ID 0x14 + Opaque-Info: 4 octets of data + Added using OSPF API: 4 octets of opaque data + Opaque data: 2 0 0 0 <==== counter is now 2 + +Note that the payload of the opaque LSA has changed as you can see above. + +Then, again after another 20 seconds, the opaque LSA is flushed from the LSDB. + +Important note: +^^^^^^^^^^^^^^^ + +In order to originate an opaque LSA, there must be at least one active +opaque-capable neighbor. Thus, you cannot originate opaque LSAs if no neighbors +are present. If you try to originate when no neighbors are ready, you will +receive a not ready error message. The reason for this restriction is that it +might be possible that some routers have an identical opaque LSA from a +previous origination in their LSDB that unfortunately could not be flushed due +to a crash, and now if the router comes up again and starts originating a new +opaque LSA, the new opaque LSA is considered older since it has a lower +sequence number and is ignored by other routers (that consider the stalled +opaque LSA as more recent). However, if the originating router first +synchronizes the database before originating opaque LSAs, it will detect the +older opaque LSA and can flush it first. + +Protocol and Message Formats +---------------------------- + +If you are developing your own client application and you don't want to make +use of the client library (due to the GNU license restriction or whatever +reason), you can implement your own client-side message handling. The OSPF API +uses two connections between the client and the OSPF API server: One connection +is used for a synchronous request /reply protocol and another connection is +used for asynchronous notifications (e.g., LSA update, neighbor status change). + +Each message begins with the following header: + +.. figure:: ../figures/ospf_api_msghdr.png + :alt: image + + image + +The message type field can take one of the following values: + ++-------------------------------+---------+ +| Messages to OSPF daemon | Value | ++===============================+=========+ +| MSG\_REGISTER\_OPAQUETYPE | 1 | ++-------------------------------+---------+ +| MSG\_UNREGISTER\_OPAQUETYPE | 2 | ++-------------------------------+---------+ +| MSG\_REGISTER\_EVENT | 3 | ++-------------------------------+---------+ +| MSG\_SYNC\_LSDB | 4 | ++-------------------------------+---------+ +| MSG\_ORIGINATE\_REQUEST | 5 | ++-------------------------------+---------+ +| MSG\_DELETE\_REQUEST | 6 | ++-------------------------------+---------+ + ++-----------------------------+---------+ +| Messages from OSPF daemon | Value | ++=============================+=========+ +| MSG\_REPLY | 10 | ++-----------------------------+---------+ +| MSG\_READY\_NOTIFY | 11 | ++-----------------------------+---------+ +| MSG\_LSA\_UPDATE\_NOTIFY | 12 | ++-----------------------------+---------+ +| MSG\_LSA\_DELETE\_NOTIFY | 13 | ++-----------------------------+---------+ +| MSG\_NEW\_IF | 14 | ++-----------------------------+---------+ +| MSG\_DEL\_IF | 15 | ++-----------------------------+---------+ +| MSG\_ISM\_CHANGE | 16 | ++-----------------------------+---------+ +| MSG\_NSM\_CHANGE | 17 | ++-----------------------------+---------+ + +The synchronous requests and replies have the following message formats: + +.. figure:: ../figures/ospf_api_msgs1.png + :alt: image + + image + +The origin field allows origin-based filtering using the following origin +types: + ++-------------------------+---------+ +| Origin | Value | ++=========================+=========+ +| NON\_SELF\_ORIGINATED | 0 | ++-------------------------+---------+ +| SELF\_ORIGINATED | 1 | ++-------------------------+---------+ +| ANY\_ORIGIN | 2 | ++-------------------------+---------+ + +The reply message has one of the following error codes: + ++--------------------------+---------+ +| Error code | Value | ++==========================+=========+ +| API\_OK | 0 | ++--------------------------+---------+ +| API\_NOSUCHINTERFACE | -1 | ++--------------------------+---------+ +| API\_NOSUCHAREA | -2 | ++--------------------------+---------+ +| API\_NOSUCHLSA | -3 | ++--------------------------+---------+ +| API\_ILLEGALSATYPE | -4 | ++--------------------------+---------+ +| API\_ILLEGALOPAQUETYPE | -5 | ++--------------------------+---------+ +| API\_OPAQUETYPEINUSE | -6 | ++--------------------------+---------+ +| API\_NOMEMORY | -7 | ++--------------------------+---------+ +| API\_ERROR | -99 | ++--------------------------+---------+ +| API\_UNDEF | -100 | ++--------------------------+---------+ + +The asynchronous notifications have the following message formats: + +.. figure:: ../figures/ospf_api_msgs2.png + :alt: image + + image + + +.. Do not delete these acknowledgements! + +Original Acknowledgments from Ralph Keller +------------------------------------------ + +I would like to thank Masahiko Endo, the author of the opaque LSA extension +module, for his great support. His wonderful ASCII graphs explaining the +internal workings of this code, and his invaluable input proved to be crucial +in designing a useful API for accessing the link state database of the OSPF +daemon. Once, he even decided to take the plane from Tokyo to Zurich so that we +could actually meet and have face-to-face discussions, which was a lot of fun. +Clearly, without Masahiko no API would ever be completed. I also would like to +thank Daniel Bauer who wrote an opaque LSA implementation too and was willing +to test the OSPF API code in one of his projects. diff --git a/doc/developer/ospf-sr.rst b/doc/developer/ospf-sr.rst new file mode 100644 index 0000000..1c16443 --- /dev/null +++ b/doc/developer/ospf-sr.rst @@ -0,0 +1,347 @@ +OSPF Segment Routing +==================== + +This is an EXPERIMENTAL support of `RFC 8665`. +DON'T use it for production network. + +Supported Features +------------------ + +* Automatic computation of Primary and Backup Adjacency SID with + Cisco experimental remote IP address +* SRGB & SRLB configuration +* Prefix configuration for Node SID with optional NO-PHP flag (Linux + kernel support both mode) +* Node MSD configuration (with Linux Kernel >= 4.10 a maximum of 32 labels + could be stack) +* Automatic provisioning of MPLS table +* Equal Cost Multi-Path (ECMP) +* Static route configuration with label stack up to 32 labels +* TI-LFA (for P2P interfaces only) + +Interoperability +---------------- + +* Tested on various topology including point-to-point and LAN interfaces + in a mix of FRRouting instance and Cisco IOS-XR 6.0.x +* Check OSPF LSA conformity with latest wireshark release 2.5.0-rc + +Implementation details +---------------------- + +Concepts +^^^^^^^^ + +Segment Routing used 3 different OPAQUE LSA in OSPF to carry the various +information: + +* **Router Information:** flood the Segment Routing capabilities of the node. + This include the supported algorithms, the Segment Routing Global Block + (SRGB) and the Maximum Stack Depth (MSD). +* **Extended Link:** flood the Adjaceny and Lan Adjacency Segment Identifier +* **Extended Prefix:** flood the Prefix Segment Identifier + +The implementation follows previous TE and Router Information codes. It used the +OPAQUE LSA functions defined in ospf_opaque.[c,h] as well as the OSPF API. This +latter is mandatory for the implementation as it provides the Callback to +Segment Routing functions (see below) when an Extended Link / Prefix or Router +Information LSA s are received. + +Overview +^^^^^^^^ + +Following files where modified or added: + +* ospd_ri.[c,h] have been modified to add the new TLVs for Segment Routing. +* ospf_ext.[c,h] implement RFC7684 as base support of Extended Link and Prefix + Opaque LSA. +* ospf_sr.[c,h] implement the earth of Segment Routing. It adds a new Segment + Routing database to manage Segment Identifiers per Link and Prefix and + Segment Routing enable node, Callback functions to process incoming LSA and + install MPLS FIB entry through Zebra. + +The figure below shows the relation between the various files: + +* ospf_sr.c centralized all the Segment Routing processing. It receives Opaque + LSA Router Information (4.0.0.0) from ospf_ri.c and Extended Prefix + (7.0.0.X) Link (8.0.0.X) from ospf_ext.c. Once received, it parse TLVs and + SubTLVs and store information in SRDB (which is defined in ospf_sr.h). For + each received LSA, NHLFE is computed and send to Zebra to add/remove new + MPLS labels entries and FEC. New CLI configurations are also centralized in + ospf_sr.c. This CLI will trigger the flooding of new LSA Router Information + (4.0.0.0), Extended Prefix (7.0.0.X) and Link (8.0.0.X) by ospf_ri.c, + respectively ospf_ext.c. +* ospf_ri.c send back to ospf_sr.c received Router Information LSA and update + Self Router Information LSA with parameters provided by ospf_sr.c i.e. SRGB + and MSD. It use ospf_opaque.c functions to send/received these Opaque LSAs. +* ospf_ext.c send back to ospf_sr.c received Extended Prefix and Link Opaque + LSA and send self Extended Prefix and Link Opaque LSA through ospf_opaque.c + functions. + +:: + + +-----------+ +-------+ + | | | | + | ospf_sr.c +-----+ SRDB | + +-----------+ +--+ | | + | +-^-------^-+ | +-------+ + | | | | | + | | | | | + | | | | +--------+ + | | | | | + +---v----------+ | | | +-----v-------+ + | | | | | | | + | ospf_ri.c +--+ | +-------+ ospf_ext.c | + | LSA 4.0.0.0 | | | LSA 7.0.0.X | + | | | | LSA 8.0.0.X | + +---^----------+ | | | + | | +-----^-------+ + | | | + | | | + | +--------v------------+ | + | | | | + | | ZEBRA: Labels + FEC | | + | | | | + | +---------------------+ | + | | + | | + | +---------------+ | + | | | | + +---------> ospf_opaque.c <---------+ + | | + +---------------+ + + Figure 1: Overview of Segment Routing interaction + +Module interactions +^^^^^^^^^^^^^^^^^^^ + +To process incoming LSA, the code is based on the capability to call `hook()` +functions when LSA are inserted or delete to / from the LSDB and the +possibility to register particular treatment for Opaque LSA. The first point +is provided by the OSPF API feature and the second by the Opaque implementation +itself. Indeed, it is possible to register callback function for a given Opaque +LSA ID (see `ospf_register_opaque_functab()` function defined in +`ospf_opaque.c`). Each time a new LSA is added to the LSDB, the +`new_lsa_hook()` function previously register for this LSA type is called. For +Opaque LSA it is the `ospf_opaque_lsa_install_hook()`. For deletion, it is +`ospf_opaque_lsa_delete_hook()`. + +Note that incoming LSA which is already present in the LSDB will be inserted +after the old instance of this LSA remove from the LSDB. Thus, after the first +time, each incoming LSA will trigger a `delete` following by an `install`. This +is not very helpful to handle real LSA deletion. In fact, LSA deletion is done +by Flushing LSA i.e. flood LSA after setting its age to MAX_AGE. Then, a garbage +function has the role to remove all LSA with `age == MAX_AGE` in the LSDB. So, +to handle LSA Flush, the best is to look to the LSA age to determine if it is +an installation or a future deletion i.e. the flushed LSA is first store in the +LSDB with MAX_AGE waiting for the garbage collector function. + +Router Information LSAs +^^^^^^^^^^^^^^^^^^^^^^^ + +To activate Segment Routing, new CLI command `segment-routing on` has been +introduced. When this command is activated, function +`ospf_router_info_update_sr()` is called to indicate to Router Information +process that Segment Routing TLVs must be flood. Same function is called to +modify the Segment Routing Global Block (SRGB) and Maximum Stack Depth (MSD) +TLV. Only Shortest Path First (SPF) Algorithm is supported, so no possibility +to modify this TLV is offer by the code. + +When Opaque LSA Type 4 i.e. Router Information are stored in LSDB, function +`ospf_opaque_lsa_install_hook()` will call the previously registered function +`ospf_router_info_lsa_update()`. In turn, the function will simply trigger +`ospf_sr_ri_lsa_update()` or `ospf_sr_ri_lsa_delete` in function of the LSA +age. Before, it verifies that the LSA Opaque Type is 4 (Router Information). +Self Opaque LSA are not send back to the Segment Routing functions as +information are already stored. + +Extended Link Prefix LSAs +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Like for Router Information, Segment Routing is activate at the Extended +Link/Prefix level with new `segment-routing on` command. This triggers +automatically the flooding of Extended Link LSA for all ospf interfaces where +adjacency is full. For Extended Prefix LSA, the new CLI command +`segment-routing prefix ...` will trigger the flooding of Prefix SID +TLV/SubTLVs. + +When Opaque LSA Type 7 i.e. Extended Prefix and Type 8 i.e. Extended Link are +store in the LSDB, `ospf_ext_pref_update_lsa()` respectively +`ospf_ext_link_update_lsa()` are called like for Router Information LSA. In +turn, they respectively trigger `ospf_sr_ext_prefix_lsa_update()` / +`ospf_sr_ext_link_lsa_update()` or `ospf_sr_ext_prefix_lsa_delete()` / +`ospf_sr_ext_link_lsa_delete()` if the LSA age is equal to MAX_AGE. + +Zebra +^^^^^ + +When a new MPLS entry or new Forwarding Equivalent Class (FEC) must be added or +deleted in the data plane, `add_sid_nhlfe()` respectively `del_sid_nhlfe()` are +called. Once check the validity of labels, they are send to ZEBRA layer through +`ZEBRA_MPLS_LABELS_ADD` command, respectively `ZEBRA_MPLS_LABELS_DELETE` +command for deletion. This is completed by a new labelled route through +`ZEBRA_ROUTE_ADD` command, respectively `ZEBRA_ROUTE_DELETE` command. + +TI-LFA +^^^^^^ + +Experimental support for Topology Independent LFA (Loop-Free Alternate), see +for example 'draft-bashandy-rtgwg-segment-routing-ti-lfa-05'. The related +files are `ospf_ti_lfa.c/h`. + +The current implementation is rather naive and does not support the advanced +optimizations suggested in e.g. RFC7490 or RFC8102. It focuses on providing +the essential infrastructure which can also later be used to enhance the +algorithmic aspects. + +Supported features: + +* Link and node protection +* Intra-area support +* Proper use of Prefix- and Adjacency-SIDs in label stacks +* Asymmetric weights (using reverse SPF) +* Non-adjacent P/Q spaces +* Protection of Prefix-SIDs + +If configured for every SPF run the routing table is enriched with additional +backup paths for every prefix. The corresponding Prefix-SIDs are updated with +backup paths too within the OSPF SR update task. + +Informal High-Level Algorithm Description: + +:: + + p_spaces = empty_list() + + for every protected_resource (link or node): + p_space = generate_p_space(protected_resource) + p_space.q_spaces = empty_list() + + for every destination that is affected by the protected_resource: + q_space = generate_q_space(destination) + + # The label stack is stored in q_space + generate_label_stack(p_space, q_space) + + # The p_space collects all its q_spaces + p_spaces.q_spaces.add(q_space) + + p_spaces.add(p_space) + + adjust_routing_table(p_spaces) + +Possible Performance Improvements: + +* Improve overall datastructures, get away from linked lists for vertices +* Don't calculate a Q space for every destination, but for a minimum set of + backup paths that cover all destinations in the post-convergence SPF. The + thinking here is that once a backup path is known that it is also a backup + path for all nodes on the path themselves. This can be done by using the + leafs of a trimmed minimum spanning tree generated out of the post- + convergence SPF tree for that particular P space. +* For an alternative (maybe better) optimization look at + https://tools.ietf.org/html/rfc7490#section-5.2.1.3 which describes using + the Q space of the node which is affected by e.g. a link failure. Note that + this optimization is topology dependent. + +It is highly recommended to read e.g. `Segment Routing I/II` by Filsfils to +understand the basics of Ti-LFA. + +Configuration +------------- + +Linux Kernel +^^^^^^^^^^^^ + +In order to use OSPF Segment Routing, you must setup MPLS data plane. Up to +know, only Linux Kernel version >= 4.5 is supported. + +First, the MPLS modules aren't loaded by default, so you'll need to load them +yourself: + +:: + + modprobe mpls_router + modprobe mpls_gso + modprobe mpls_iptunnel + +Then, you must activate MPLS on the interface you would used: + +:: + + sysctl -w net.mpls.conf.enp0s9.input=1 + sysctl -w net.mpls.conf.lo.input=1 + sysctl -w net.mpls.platform_labels=1048575 + +The last line fix the maximum MPLS label value. + +Once OSPFd start with Segment Routing, you could check that MPLS routes are +enable with: + +:: + + ip -M route + ip route + +The first command show the MPLS LFIB table while the second show the FIB +table which contains route with MPLS label encapsulation. + +If you disable Penultimate Hop Popping with the `no-php-flag` (see below), you +MUST check that RP filter is not enable for the interface you intend to use, +especially the `lo` one. For that purpose, disable RP filtering with: + +:: + + systcl -w net.ipv4.conf.all.rp_filter=0 + sysctl -w net.ipv4.conf.lo.rp_filter=0 + +OSPFd +^^^^^ + +Here it is a simple example of configuration to enable Segment Routing. Note +that `opaque capability` and `router information` must be set to activate +Opaque LSA prior to Segment +Routing. + +:: + + router ospf + ospf router-id 192.168.1.11 + capability opaque + segment-routing on + segment-routing global-block 10000 19999 local-block 5000 5999 + segment-routing node-msd 8 + segment-routing prefix 192.168.1.11/32 index 1100 + +The first segment-routing statement enables it. The second and third one set +the SRGB and SRLB respectively, fourth line the MSD and finally, set the +Prefix SID index for a given prefix. + +Note that only prefix of Loopback interface could be configured with a Prefix +SID. It is possible to add `no-php-flag` at the end of the prefix command to +disable Penultimate Hop Popping. This advertises to peers that they MUST NOT pop +the MPLS label prior to sending the packet. + +Known limitations +----------------- + +* Runs only within default VRF +* Only single Area is supported. ABR is not yet supported +* Only SPF algorithm is supported +* Extended Prefix Range is not supported +* With NO Penultimate Hop Popping, it is not possible to express a Segment + Path with an Adjacency SID due to the impossibility for the Linux Kernel to + perform double POP instruction. + +Credits +------- + +* Author: Anselme Sawadogo <anselmesawadogo@gmail.com> +* Author: Olivier Dugeon <olivier.dugeon@orange.com> +* Copyright (C) 2016 - 2018 Orange Labs http://www.orange.com + +This work has been performed in the framework of the H2020-ICT-2014 +project 5GEx (Grant Agreement no. 671636), which is partially funded +by the European Commission. + diff --git a/doc/developer/ospf.rst b/doc/developer/ospf.rst new file mode 100644 index 0000000..837a0bd --- /dev/null +++ b/doc/developer/ospf.rst @@ -0,0 +1,13 @@ +.. _ospfd: + +***** +OSPFD +***** + +.. toctree:: + :maxdepth: 2 + + ospf-api + ospf-sr + cspf + diff --git a/doc/developer/packaging-debian.rst b/doc/developer/packaging-debian.rst new file mode 100644 index 0000000..c2c3b7e --- /dev/null +++ b/doc/developer/packaging-debian.rst @@ -0,0 +1,167 @@ +.. _packaging-debian: + +Packaging Debian +================ + +(Tested on Ubuntu 14.04, 16.04, 17.10, 18.04, Debian jessie, stretch and +buster.) + +1. Install the Debian packaging tools: + + .. code-block:: shell + + sudo apt install fakeroot debhelper devscripts + +2. Checkout FRR under an **unprivileged** user account: + + .. code-block:: shell + + git clone https://github.com/frrouting/frr.git frr + cd frr + + If you wish to build a package for a branch other than master: + + .. code-block:: shell + + git checkout <branch> + +3. Install build dependencies using the `mk-build-deps` tool from the + `devscripts` package: + + .. code-block:: shell + + sudo mk-build-deps --install --remove debian/control + + Alternatively, you can manually install build dependencies for your + platform as outlined in :ref:`building`. + +4. Install `git-buildpackage` package: + + .. code-block:: shell + + sudo apt-get install git-buildpackage + +5. (optional) Append a distribution identifier if needed (see below under + :ref:`multi-dist`.) + +6. Build Debian Binary and/or Source Packages: + + .. code-block:: shell + + gbp buildpackage --git-builder=dpkg-buildpackage --git-debian-branch="$(git rev-parse --abbrev-ref HEAD)" $options + + Where `$options` may contain any or all of the following items: + + * build profiles specified with ``-P``, e.g. + ``-Ppkg.frr.nortrlib,pkg.frr.rtrlib``. + Multiple values are separated by commas and there must not be a space + after the ``-P``. + + The following build profiles are currently available: + + +----------------+-------------------+-----------------------------------------+ + | Profile | Negation | Effect | + +================+===================+=========================================+ + | pkg.frr.rtrlib | pkg.frr.nortrlib | builds frr-rpki-rtrlib package (or not) | + +----------------+-------------------+-----------------------------------------+ + | pkg.frr.lua | pkg.frr.nolua | builds lua scripting extension | + +----------------+-------------------+-----------------------------------------+ + | pkg.frr.pim6d | pkg.frr.nopim6d | builds pim6d (default enabled) | + +----------------+-------------------+-----------------------------------------+ + + * the ``-uc -us`` options to disable signing the packages with your GPG key + + (git builds of the `master` or `stable/X.X` branches won't be signed by + default since their target release is set to ``UNRELEASED``.) + + * the ``--build=type`` accepts following options (see ``dpkg-buildpackage`` manual page): + + * ``source`` builds the source package + * ``any`` builds the architecture specific binary packages + * ``all`` build the architecture independent binary packages + * ``binary`` build the architecture specific and independent binary packages (alias for ``any,all``) + * ``full`` builds everything (alias for ``source,any,all``) + + Alternatively, you might want to replace ``dpkg-buildpackage`` with + ``debuild`` wrapper that also runs ``lintian`` and ``debsign`` on the final + packages. + +7. Done! + + If all worked correctly, then you should end up with the Debian packages in + the parent directory of where `debuild` ran. If distributed, please make sure + you distribute it together with the sources (``frr_*.orig.tar.xz``, + ``frr_*.debian.tar.xz`` and ``frr_*.dsc``) + +.. note:: + + A package created from `master` or `stable/X.X` is slightly different from + a package created from the `debian` branch. The changelog for the former + is autogenerated and sets the Debian revision to ``-0``, which causes an + intentional lintian warning. The `debian` branch on the other hand has + a manually maintained changelog that contains proper Debian release + versioning. + + +.. _multi-dist: + +Multi-Distribution builds +========================= + +You can optionally append a distribution identifier in case you want to +make multiple versions of the package available in the same repository. + +.. code-block:: shell + + dch -l '~deb8u' 'build for Debian 8 (jessie)' + dch -l '~deb9u' 'build for Debian 9 (stretch)' + dch -l '~ubuntu14.04.' 'build for Ubuntu 14.04 (trusty)' + dch -l '~ubuntu16.04.' 'build for Ubuntu 16.04 (xenial)' + dch -l '~ubuntu18.04.' 'build for Ubuntu 18.04 (bionic)' + +Between building packages for specific distributions, the only difference +in the package itself lies in the automatically generated shared library +dependencies, e.g. libjson-c2 or libjson-c3. This means that the +architecture independent packages should **not** have a suffix appended. +Also, the current Debian testing/unstable releases should not have any suffix +appended. + +For example, at the end of 2018 (i.e. ``buster``/Debian 10 is the current +"testing" release), the following is a complete list of `.deb` files for +Debian 8, 9 and 10 packages for FRR 6.0.1-1 with RPKI support:: + + frr_6.0.1-1_amd64.deb + frr_6.0.1-1~deb8u1_amd64.deb + frr_6.0.1-1~deb9u1_amd64.deb + frr-dbg_6.0.1-1_amd64.deb + frr-dbg_6.0.1-1~deb8u1_amd64.deb + frr-dbg_6.0.1-1~deb9u1_amd64.deb + frr-rpki-rtrlib_6.0.1-1_amd64.deb + frr-rpki-rtrlib_6.0.1-1~deb8u1_amd64.deb + frr-rpki-rtrlib_6.0.1-1~deb9u1_amd64.deb + frr-doc_6.0.1-1_all.deb + frr-pythontools_6.0.1-1_all.deb + +Note that there are no extra versions of the `frr-doc` and `frr-pythontools` +packages (because they are for architecture ``all``, not ``amd64``), and the +version for Debian 10 does **not** have a ``~deb10u1`` suffix. + +.. warning:: + + Do not use the ``-`` character in the version suffix. The last ``-`` in + the version number is the separator between upstream version and Debian + version. ``6.0.1-1~foobar-2`` means upstream version ``6.0.1-1~foobar``, + Debian version ``2``. This is not what you want. + + The only allowed characters in the Debian version are ``0-9 A-Z a-z + . ~`` + +.. note:: + + The separating character for the suffix **must** be the tilde (``~``) + because the tilde is ordered in version-comparison before the empty + string. That means the order of the above packages is the following: + + ``6.0.1-1`` newer than ``6.0.1-1~deb9u1`` newer than ``6.0.1-1~deb8u1`` + + If you use another character (e.g. ``+``), the untagged version will be + regarded as the "oldest"! diff --git a/doc/developer/packaging-redhat.rst b/doc/developer/packaging-redhat.rst new file mode 100644 index 0000000..d88f449 --- /dev/null +++ b/doc/developer/packaging-redhat.rst @@ -0,0 +1,98 @@ +.. _packaging-redhat: + +Packaging Red Hat +================= + +Tested on CentOS 6, CentOS 7, CentOS 8 and Fedora 24. + +1. On CentOS 6, refer to :ref:`building-centos6` for details on installing + sufficiently up-to-date package versions to enable building FRR. + + Newer automake/autoconf/bison is only needed to build the RPM and is **not** + needed to install the binary RPM package. + +2. Install the build dependencies for your platform. Refer to the + platform-specific build documentation on how to do this. + +3. Install the following additional packages:: + + yum install rpm-build net-snmp-devel pam-devel libcap-devel + + For CentOS 7 and CentOS 8, the package will be built using python3 + and requires additional python3 packages:: + + yum install python3-devel python3-sphinx + + .. note:: + + For CentOS 8 you need to install ``platform-python-devel`` package + to provide ``/usr/bin/pathfix.py``:: + + yum install platform-python-devel + + + If ``yum`` is not present on your system, use ``dnf`` instead. + + You should enable ``PowerTools`` repo if using CentOS 8 which + is disabled by default. + +4. Checkout FRR:: + + git clone https://github.com/frrouting/frr.git frr + +5. Run Bootstrap and make distribution tar.gz:: + + cd frr + ./bootstrap.sh + ./configure --with-pkg-extra-version=-MyRPMVersion + make dist + + .. note:: + + The only ``configure`` option respected when building RPMs is + ``--with-pkg-extra-version``. + +6. Create RPM directory structure and populate with sources:: + + mkdir rpmbuild + mkdir rpmbuild/SOURCES + mkdir rpmbuild/SPECS + cp redhat/*.spec rpmbuild/SPECS/ + cp frr*.tar.gz rpmbuild/SOURCES/ + +7. Edit :file:`rpm/SPECS/frr.spec` with configuration as needed. + + Look at the beginning of the file and adjust the following parameters to + enable or disable features as required:: + + ############### FRRouting (FRR) configure options ################# + # with-feature options + %{!?with_pam: %global with_pam 0 } + %{!?with_ospfclient: %global with_ospfclient 1 } + %{!?with_ospfapi: %global with_ospfapi 1 } + %{!?with_irdp: %global with_irdp 1 } + %{!?with_rtadv: %global with_rtadv 1 } + %{!?with_ldpd: %global with_ldpd 1 } + %{!?with_nhrpd: %global with_nhrpd 1 } + %{!?with_eigrp: %global with_eigrpd 1 } + %{!?with_shared: %global with_shared 1 } + %{!?with_multipath: %global with_multipath 256 } + %{!?frr_user: %global frr_user frr } + %{!?vty_group: %global vty_group frrvty } + %{!?with_fpm: %global with_fpm 0 } + %{!?with_watchfrr: %global with_watchfrr 1 } + %{!?with_bgp_vnc: %global with_bgp_vnc 0 } + %{!?with_pimd: %global with_pimd 1 } + %{!?with_pim6d: %global with_pim6d 1 } + %{!?with_rpki: %global with_rpki 0 } + +8. Build the RPM:: + + rpmbuild --define "_topdir `pwd`/rpmbuild" -ba rpmbuild/SPECS/frr.spec + + If building with RPKI, then download and install the additional RPKI + packages from + https://ci1.netdef.org/browse/RPKI-RTRLIB/latestSuccessful/artifact + +If all works correctly, then you should end up with the RPMs under +:file:`rpmbuild/RPMS` and the source RPM under :file:`rpmbuild/SRPMS`. diff --git a/doc/developer/packaging.rst b/doc/developer/packaging.rst new file mode 100644 index 0000000..0c072e4 --- /dev/null +++ b/doc/developer/packaging.rst @@ -0,0 +1,10 @@ +******************** +Releases & Packaging +******************** + +.. toctree:: + :maxdepth: 2 + + frr-release-procedure + packaging-debian + packaging-redhat diff --git a/doc/developer/path-internals-daemon.rst b/doc/developer/path-internals-daemon.rst new file mode 100644 index 0000000..29f0172 --- /dev/null +++ b/doc/developer/path-internals-daemon.rst @@ -0,0 +1,115 @@ +PATHD Internals +=============== + +Architecture +------------ + +Overview +........ + +The pathd deamon manages the segment routing policies, it owns the data +structures representing them and can load modules that manipulate them like the +PCEP module. Its responsibility is to select a candidate path for each +configured policy and to install it into Zebra. + +Zebra +..... + +Zebra manages policies that are active or pending to be activated due to the +next hop not being available yet. In zebra, policy data structures and APIs are +defined in `zebra_srte.[hc]`. + +The responsibilities of Zebra are: + + - Store the policies' segment list. + - Install the policies when their next-hop is available. + - Notify other daemons of the status of the policies. + +Adding and removing policies is done using the commands `ZEBRA_SR_POLICY_SET` +and `ZEBRA_SR_POLICY_DELETE` as parameter of the function `zebra_send_sr_policy` +all defined in `zclient.[hc]`. + +If the first segment of the policy is an unknown label, it is kept until +notified by the mpls hooks `zebra_mpls_label_created`, and then it is installed. + +To get notified when a policy status changes, a client can implement the +`sr_policy_notify_status` callback defined in `zclient.[hc]`. + +For encoding/decoding the various data structures used to comunicate with zebra, +the following functions are available from `zclient.[hc]`: +`zapi_sr_policy_encode`, `zapi_sr_policy_decode` and +`zapi_sr_policy_notify_status_decode`. + + +Pathd +..... + + +The pathd daemon manages all the possible candidate paths for the segment +routing policies and selects the best one following the +`segment routing policy draft <https://tools.ietf.org/html/draft-ietf-spring-segment-routing-policy-06#section-2.9>`_. +It also supports loadable modules for handling dynamic candidate paths and the +creation of new policies and candidate paths at runtime. + +The responsibilities of the pathd base daemon, not including any optional +modules, are: + + - Store the policies and all the possible candidate paths for them. + - Select the best candidate path for each policy and send it to Zebra. + - Provide VTYSH configuration to set up policies and candidate paths. + - Provide a Northbound API to manipulate **configured** policies and candidate paths. + - Handle loadable modules for extending the functionality. + - Provide an API to the loadable module to manipulate policies and candidate paths. + + +Threading Model +--------------- + +The daemon runs completely inside the main thread using FRR event model, there +is no threading involved. + + +Source Code +----------- + +Internal Data Structures +........................ + +The main data structures for policies and candidate paths are defined in +`pathd.h` and implemented in `pathd.c`. + +When modifying these structures, either directly or through the functions +exported by `pathd.h`, nothing should be deleted/freed right away. The deletion +or modification flags must be set and when all the changes are done, the +function `srte_apply_changes` must be called. When called, a new candidate path +may be elected and sent to Zebra, and all the structures flagged as deleted +will be freed. In addition, a hook will be called so dynamic modules can perform +any required action when the elected candidate path changes. + + +Northbound API +.............. + +The northbound API is defined in `path_nb.[ch]` and implemented in +`path_nb_config.c` for configuration data and `path_nb_state.c` for operational +data. + + +Command Line Client +................... + +The command-line client (VTYSH) is implemented in `path_cli.c`. + + +Interface with Zebra +.................... + +All the functions interfacing with Zebra are defined and implemented in +`path_zebra.[hc]`. + + +Loadable Module API +................... + +For the time being, the API the loadable module uses is defined by `pathd.h`, +but in the future, it should be moved to a dedicated include file. diff --git a/doc/developer/path-internals-pcep.rst b/doc/developer/path-internals-pcep.rst new file mode 100644 index 0000000..a6b2220 --- /dev/null +++ b/doc/developer/path-internals-pcep.rst @@ -0,0 +1,193 @@ +PCEP Module Internals +===================== + +Introduction +------------ + +The PCEP module for the pathd daemon implements the PCEP protocol described in +:rfc:`5440` to update the policies and candidate paths. + +The protocol encoding/decoding and the basic session management is handled by +the `pceplib external library 1.2 <https://github.com/volta-networks/pceplib/tree/devel-1.2>`_. + +Together with pceplib, this module supports at least partially: + + - :rfc:`5440` + + Most of the protocol defined in the RFC is implemented. + All the messages can be parsed, but this was only tested in the context + of segment routing. Only a very small subset of metric types can be + configured, and there is a known issue with some Cisco routers not + following the IANA numbers for metrics. + + - :rfc:`8231` + + Support delegation of candidate path after performing the initial + computation request. If the PCE does not respond or cannot compute + a path, an empty candidate path is delegated to the PCE. + Only tested in the context of segment routing. + + - :rfc:`8408` + + Only used to comunicate the support for segment routing to the PCE. + + - :rfc:`8664` + + All the NAI types are implemented, but only the MPLS NAI are supported. + If the PCE provide segments that are not MPLS labels, the PCC will + return an error. + +Note that pceplib supports more RFCs and drafts, see pceplib +`README <https://github.com/volta-networks/pceplib/blob/master/README.md>`_ +for more details. + + +Architecture +------------ + +Overview +........ + +The module is separated into multiple layers: + + - pathd interface + - command-line console + - controller + - PCC + - pceplib interface + +The pathd interface handles all the interactions with the daemon API. + +The command-line console handles all the VTYSH configuration commands. + +The controller manages the multiple PCC connections and the interaction between +them and the daemon interface. + +The PCC handles a single connection to a PCE through a pceplib session. + +The pceplib interface abstracts the API of the pceplib. + +.. figure:: ../figures/pcep_module_threading_overview.svg + + +Threading Model +--------------- + +The module requires multiple threads to cooperate: + + - The main thread used by the pathd daemon. + - The controller pthread used to isolate the PCC from the main thread. + - The possible threads started in the pceplib library. + +To ensure thread safety, all the controller and PCC state data structures can +only be read and modified in the controller thread, and all the global data +structures can only be read and modified in the main thread. Most of the +interactions between these threads are done through FRR timers and events. + +The controller is the bridge between the two threads, all the functions that +**MUST** be called from the main thread start with the prefix `pcep_ctrl_` and +all the functions that **MUST** be called from the controller thread start +with the prefix `pcep_thread_`. When an asynchronous action must be taken in +a different thread, an FRR event is sent to the thread. If some synchronous +operation is needed, the calling thread will block and run a callback in the +other thread, there the result is **COPIED** and returned to the calling thread. + +No function other than the controller functions defined for it should be called +from the main thread. The only exception being some utility functions from +`path_pcep_lib.[hc]`. + +All the calls to pathd API functions **MUST** be performed in the main thread, +for that, the controller sends FRR events handled in function +`path_pcep.c:pcep_main_event_handler`. + +For the same reason, the console client only runs in the main thread. It can +freely use the global variable, but **MUST** use controller's `pcep_ctrl_` +functions to interact with the PCCs. + + +Source Code +----------- + +Generic Data Structures +....................... + +The data structures are defined in multiple places, and where they are defined +dictates where they can be used. + +The data structures defined in `path_pcep.h` can be used anywhere in the module. + +Internally, throughout the module, the `struct path` data structure is used +to describe PCEP messages. It is a simplified flattened structure that can +represent multiple complex PCEP message types. The conversion from this +structure to the PCEP data structures used by pceplib is done in the pceplib +interface layer. + +The data structures defined in `path_pcep_controller.h` should only be used +in `path_pcep_controller.c`. Even if a structure pointer is passed as a parameter +to functions defined in `path_pcep_pcc.h`, these should consider it as an opaque +data structure only used to call back controller functions. + +The same applies to the structures defined in `path_pcep_pcc.h`, even if the +controller owns a reference to this data structure, it should never read or +modify it directly, it should be considered an opaque structure. + +The global data structure can be accessed from the pathd interface layer +`path_pcep.c` and the command line client code `path_pcep_cli.c`. + + +Interface With Pathd +.................... + +All the functions calling or called by the pathd daemon are implemented in +`path_pcep.c`. These functions **MUST** run in the main FRR thread, and +all the interactions with the controller and the PCCs **MUST** pass through +the controller's `pcep_ctrl_` prefixed functions. + +To handle asynchronous events from the PCCs, a callback is passed to +`pcep_ctrl_initialize` that is called in the FRR main thread context. + + +Command Line Client +................... + +All the command line configuration commands (VTYSH) are implemented in +`path_pcep_cli.c`. All the functions there run in the main FRR thread and +can freely access the global variables. All the interaction with the +controller's and the PCCs **MUST** pass through the controller `pcep_ctrl_` +prefixed functions. + + +Debugging Helpers +................. + +All the functions formating data structures for debugging and logging purposes +are implemented in `path_pcep_debug.[hc]`. + + +Interface with pceplib +...................... + +All the functions calling the pceplib external library are defined in +`path_pcep_lib.[hc]`. Some functions are called from the main FRR thread, like +`pcep_lib_initialize`, `pcep_lib_finalize`; some can be called from either +thread, like `pcep_lib_free_counters`; some function must be called from the +controller thread, like `pcep_lib_connect`. This will probably be formalized +later on with function prefix like done in the controller. + + +Controller +.......... + +The controller is defined and implemented in `path_pcep_controller.[hc]`. +Part of the controller code runs in FRR main thread and part runs in its own +FRR pthread started to isolate the main thread from the PCCs' event loop. +To communicate between the threads it uses FRR events, timers and +`event_execute` calls. + + +PCC +... + +Each PCC instance owns its state and runs in the controller thread. They are +defined and implemented in `path_pcep_pcc.[hc]`. All the interactions with +the daemon must pass through some controller's `pcep_thread_` prefixed function. diff --git a/doc/developer/path-internals.rst b/doc/developer/path-internals.rst new file mode 100644 index 0000000..2c2df0f --- /dev/null +++ b/doc/developer/path-internals.rst @@ -0,0 +1,11 @@ +.. _path_internals: + +********* +Internals +********* + +.. toctree:: + :maxdepth: 2 + + path-internals-daemon + path-internals-pcep diff --git a/doc/developer/path.rst b/doc/developer/path.rst new file mode 100644 index 0000000..b6d2438 --- /dev/null +++ b/doc/developer/path.rst @@ -0,0 +1,11 @@ +.. _path: + +***** +PATHD +***** + +.. toctree:: + :maxdepth: 2 + + path-internals + diff --git a/doc/developer/pceplib.rst b/doc/developer/pceplib.rst new file mode 100644 index 0000000..774617d --- /dev/null +++ b/doc/developer/pceplib.rst @@ -0,0 +1,781 @@ +.. _pceplib: + +******* +PCEPlib +******* + +Overview +======== + +The PCEPlib is a PCEP implementation library that can be used by either a PCE +or PCC. + +Currently, only the FRR pathd has been implemented as a PCC with the PCEPlib. +The PCEPlib is able to simultaneously connect to multiple PCEP peers and can +maintain persistent PCEP connections. + + +PCEPlib compliance +================== + +The PCEPlib implements version 1 of the PCEP protocol, according to `RFC 5440 <https://tools.ietf.org/html/rfc5440>`_. + +Additionally, the PCEPlib implements the following PCEP extensions: + +- `RFC 8281 <https://tools.ietf.org/html/rfc8281>`_ PCE initiated for PCE-Initiated LSP Setup +- `RFC 8231 <https://tools.ietf.org/html/rfc8231>`_ Extensions for Stateful PCE +- `RFC 8232 <https://tools.ietf.org/html/rfc8232>`_ Optimizations of Label Switched Path State Synchronization Procedures for a Stateful PCE +- `RFC 8282 <https://tools.ietf.org/html/rfc8282>`_ Extensions to PCEP for Inter-Layer MPLS and GMPLS Traffic Engineering +- `RFC 8408 <https://tools.ietf.org/html/rfc8408>`_ Conveying Path Setup Type in PCE Communication Protocol (PCEP) Messages +- `draft-ietf-pce-segment-routing-07 <https://tools.ietf.org/html/draft-ietf-pce-segment-routing-07>`_, + `draft-ietf-pce-segment-routing-16 <https://tools.ietf.org/html/draft-ietf-pce-segment-routing-16>`_, + `RFC 8664 <https://tools.ietf.org/html/rfc8664>`_ Segment routing protocol extensions +- `RFC 7470 <https://tools.ietf.org/html/rfc7470>`_ Conveying Vendor-Specific Constraints +- `Draft-ietf-pce-association-group-10 <https://tools.ietf.org/html/draft-ietf-pce-association-group-10>`_ + Establishing Relationships Between Sets of Label Switched Paths +- `Draft-barth-pce-segment-routing-policy-cp-04 <https://tools.ietf.org/html/draft-barth-pce-segment-routing-policy-cp-04>`_ + Segment Routing Policy Candidate Paths + + +PCEPlib Architecture +==================== + +The PCEPlib is comprised of the following modules, each of which will be +detailed in the following sections. + +- **pcep_messages** + - PCEP messages, objects, and TLVs implementations + +- **pcep_pcc** + - PCEPlib public PCC API with a sample PCC binary + +- **pcep_session_logic** + - PCEP Session handling + +- **pcep_socket_comm** + - Socket communications + +- **pcep_timers** + - PCEP timers + +- **pcep_utils** + - Internal utilities used by the PCEPlib modules. + +The interaction of these modules can be seen in the following diagram. + +PCEPlib Architecture: + +.. image:: images/PCEPlib_design.jpg + + +PCEP Session Logic library +-------------------------- + +The PCEP Session Logic library orchestrates calls to the rest of the PCC libraries. + +PCEP Session Logic library responsibilities: + +- Handle messages received from "PCEP Socket Comm" +- Create and manage "PCEP Session" objects +- Set timers and react to timer expirations +- Manage counters + +The PCEP Session Logic library will have 2 main triggers controlled by a +pthread condition variable: + +- Timer expirations - ``on_timer_expire()`` callback +- Messages received from PCEP SocketComm - ``message_received()`` callback + +The counters are created and managed using the ``pcep_utils/pcep_utils_counters.h`` +counters library. The following are the different counter groups managed: + +- **COUNTER_SUBGROUP_ID_RX_MSG** +- **COUNTER_SUBGROUP_ID_TX_MSG** +- **COUNTER_SUBGROUP_ID_RX_OBJ** +- **COUNTER_SUBGROUP_ID_TX_OBJ** +- **COUNTER_SUBGROUP_ID_RX_SUBOBJ** +- **COUNTER_SUBGROUP_ID_TX_SUBOBJ** +- **COUNTER_SUBGROUP_ID_RX_RO_SR_SUBOBJ** +- **COUNTER_SUBGROUP_ID_TX_RO_SR_SUBOBJ** +- **COUNTER_SUBGROUP_ID_RX_TLV** +- **COUNTER_SUBGROUP_ID_TX_TLV** +- **COUNTER_SUBGROUP_ID_EVENT** + +The counters can be obtained and reset as explained later in the PCEPlib PCC API. + +PCEP Socket Comm library +------------------------ + +PCEP communication can be configured to be handled internally in this simple +library. When this library is instantiated by the PCEP Session Logic, callbacks +are provided to handle received messages and error conditions. + +The following diagram illustrates how the library works. + +PCEPlib Socket Comm: + +.. image:: images/PCEPlib_socket_comm.jpg + + +PCEP Timers library +------------------- + +Timers can be configured to be handled internally by this library. When this +library is instantiated by the PCEP Session Logic, callbacks are provided to +ha:0 +ndle timer expirations. The following timers are implemented and handled, +according to `RFC 5440 <https://tools.ietf.org/html/rfc5440>`_. + +- Open KeepWait (fixed at 60 seconds) + - Set once the PCC sends an Open, and if it expires before receiving a KeepAlive or PCErr, then the PCC should send a PCErr and close the TCP connection + +- Keepalive timer + - How often the PCC should send Keepalive messages to the PCE (and vice-versa) + - The timer will be reset after any message is sent: any message serves as a Keepalive + +- DeadTimer + - If no messages are received before expiration, the session is declared as down + - Reset everytime any message is received + +- PCReq request timer + - How long the PCC waits for the PCE to reply to PCReq messages. + +PCEPlib Timers: + +.. image:: images/PCEPlib_timers.jpg + + +PCEP Messages library +--------------------- + +The PCEP Messages library has all of the implemented PCEP messages, objects, +TLVs, and related functionality. + +The following header files can be used for creating and handling received PCEP +entities. + +- pcep-messages.h +- pcep-objects.h +- pcep-tlvs.h + + +PCEP Messages ++++++++++++++ + +The following PCEP messages can be created and received: + +- ``struct pcep_message* pcep_msg_create_open(...);`` +- ``struct pcep_message* pcep_msg_create_open_with_tlvs(...);`` +- ``struct pcep_message* pcep_msg_create_request(...);`` +- ``struct pcep_message* pcep_msg_create_request_ipv6(...);`` +- ``struct pcep_message* pcep_msg_create_reply(...);`` +- ``struct pcep_message* pcep_msg_create_close(...);`` +- ``struct pcep_message* pcep_msg_create_error(...);`` +- ``struct pcep_message* pcep_msg_create_error_with_objects(...);`` +- ``struct pcep_message* pcep_msg_create_keepalive(...);`` +- ``struct pcep_message* pcep_msg_create_report(...);`` +- ``struct pcep_message* pcep_msg_create_update(...);`` +- ``struct pcep_message* pcep_msg_create_initiate(...);`` + +Refer to ``pcep_messages/include/pcep-messages.h`` and the API section +below for more details. + + +PCEP Objects +++++++++++++ + +The following PCEP objects can be created and received: + +- ``struct pcep_object_open* pcep_obj_create_open(...);`` +- ``struct pcep_object_rp* pcep_obj_create_rp(...);`` +- ``struct pcep_object_notify* pcep_obj_create_notify(...);`` +- ``struct pcep_object_nopath* pcep_obj_create_nopath(...);`` +- ``struct pcep_object_association_ipv4* pcep_obj_create_association_ipv4(...);`` +- ``struct pcep_object_association_ipv6* pcep_obj_create_association_ipv6(...);`` +- ``struct pcep_object_endpoints_ipv4* pcep_obj_create_endpoint_ipv4(...);`` +- ``struct pcep_object_endpoints_ipv6* pcep_obj_create_endpoint_ipv6(...);`` +- ``struct pcep_object_bandwidth* pcep_obj_create_bandwidth(...);`` +- ``struct pcep_object_metric* pcep_obj_create_metric(...);`` +- ``struct pcep_object_lspa* pcep_obj_create_lspa(...);`` +- ``struct pcep_object_svec* pcep_obj_create_svec(...);`` +- ``struct pcep_object_error* pcep_obj_create_error(...);`` +- ``struct pcep_object_close* pcep_obj_create_close(...);`` +- ``struct pcep_object_srp* pcep_obj_create_srp(...);`` +- ``struct pcep_object_lsp* pcep_obj_create_lsp(...);`` +- ``struct pcep_object_vendor_info* pcep_obj_create_vendor_info(...);`` +- ``struct pcep_object_ro* pcep_obj_create_ero(...);`` +- ``struct pcep_object_ro* pcep_obj_create_rro(...);`` +- ``struct pcep_object_ro* pcep_obj_create_iro(...);`` +- ``struct pcep_ro_subobj_ipv4* pcep_obj_create_ro_subobj_ipv4(...);`` +- ``struct pcep_ro_subobj_ipv6* pcep_obj_create_ro_subobj_ipv6(...);`` +- ``struct pcep_ro_subobj_unnum* pcep_obj_create_ro_subobj_unnum(...);`` +- ``struct pcep_ro_subobj_32label* pcep_obj_create_ro_subobj_32label(...);`` +- ``struct pcep_ro_subobj_asn* pcep_obj_create_ro_subobj_asn(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_nonai(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv4_node(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv6_node(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv4_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv6_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_unnumbered_ipv4_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_linklocal_ipv6_adj(...);`` + +Refer to ``pcep_messages/include/pcep-objects.h`` and the API section +below for more details. + + +PCEP TLVs ++++++++++ + +The following PCEP TLVs (Tag, Length, Value) can be created and received: + +- Open Object TLVs + - ``struct pcep_object_tlv_stateful_pce_capability* pcep_tlv_create_stateful_pce_capability(...);`` + - ``struct pcep_object_tlv_lsp_db_version* pcep_tlv_create_lsp_db_version(...);`` + - ``struct pcep_object_tlv_speaker_entity_identifier* pcep_tlv_create_speaker_entity_id(...);`` + - ``struct pcep_object_tlv_path_setup_type* pcep_tlv_create_path_setup_type(...);`` + - ``struct pcep_object_tlv_path_setup_type_capability* pcep_tlv_create_path_setup_type_capability(...);`` + - ``struct pcep_object_tlv_sr_pce_capability* pcep_tlv_create_sr_pce_capability(...);`` + +- LSP Object TLVs + - ``struct pcep_object_tlv_ipv4_lsp_identifier* pcep_tlv_create_ipv4_lsp_identifiers(...);`` + - ``struct pcep_object_tlv_ipv6_lsp_identifier* pcep_tlv_create_ipv6_lsp_identifiers(...);`` + - ``struct pcep_object_tlv_symbolic_path_name* pcep_tlv_create_symbolic_path_name(...);`` + - ``struct pcep_object_tlv_lsp_error_code* pcep_tlv_create_lsp_error_code(...);`` + - ``struct pcep_object_tlv_rsvp_error_spec* pcep_tlv_create_rsvp_ipv4_error_spec(...);`` + - ``struct pcep_object_tlv_rsvp_error_spec* pcep_tlv_create_rsvp_ipv6_error_spec(...);`` + - ``struct pcep_object_tlv_nopath_vector* pcep_tlv_create_nopath_vector(...);`` + - ``struct pcep_object_tlv_vendor_info* pcep_tlv_create_vendor_info(...);`` + - ``struct pcep_object_tlv_arbitrary* pcep_tlv_create_tlv_arbitrary(...);`` + +- SRPAG (SR Association Group) TLVs + - ``struct pcep_object_tlv_srpag_pol_id *pcep_tlv_create_srpag_pol_id_ipv4(...);`` + - ``struct pcep_object_tlv_srpag_pol_id *pcep_tlv_create_srpag_pol_id_ipv6(...);`` + - ``struct pcep_object_tlv_srpag_pol_name *pcep_tlv_create_srpag_pol_name(...);`` + - ``struct pcep_object_tlv_srpag_cp_id *pcep_tlv_create_srpag_cp_id(...);`` + - ``struct pcep_object_tlv_srpag_cp_pref *pcep_tlv_create_srpag_cp_pref(...);`` + +Refer to ``pcep_messages/include/pcep-tlvs.h`` and the API section +below for more details. + + +PCEP PCC +-------- + +This module has a Public PCC API library (explained in detail later) and a +sample PCC binary. The APIs in this library encapsulate other PCEPlib libraries +for simplicity. With this API, the PCEPlib PCC can be started and stopped, and +the PCEPlib event queue can be accessed. The PCEP Messages library is not +encapsulated, and should be used directly. + + +Internal Dependencies +--------------------- + +The following diagram illustrates the internal PCEPlib library dependencies. + +PCEPlib internal dependencies: + +.. image:: images/PCEPlib_internal_deps.jpg + + +External Dependencies +--------------------- + +Originally the PCEPlib was based on the open source `libpcep project <https://www.acreo.se/open-software-libpcep>`_, +but that dependency has been reduced to just one source file (pcep-tools.[ch]). + + +PCEPlib Threading model +----------------------- + +The PCEPlib can be run in stand-alone mode whereby a thread is launched for +timers and socket comm, as is illustrated in the following diagram. + +PCEPlib Threading model: + +.. image:: images/PCEPlib_threading_model.jpg + +The PCEPlib can also be configured to use an external timers and socket +infrastructure like the FRR threads and tasks. In this case, no internal +threads are launched for timers and socket comm, as is illustrated in the +following diagram. + +PCEPlib Threading model with external infra: + +.. image:: images/PCEPlib_threading_model_frr_infra.jpg + + +Building +-------- + +The autotools build system is used and integrated with the frr build system. + +Testing +------- + +The Unit Tests for an individual library are executed with the ``make check`` +command. The Unit Test binary will be written to the project ``build`` directory. +All Unit Tests are executed with Valgrind, and any memory issues reported by +Valgrind will cause the Unit Test to fail. + + +PCEPlib PCC API +=============== + +The following sections describe the PCEPlib PCC API. + + +PCEPlib PCC Initialization and Destruction +------------------------------------------ + +The PCEPlib can be initialized to handle memory, timers, and socket comm +internally in what is called stand-alone mode, or with an external +infrastructure, like FRR. + +PCEPlib PCC Initialization and Destruction in stand-alone mode +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +PCEPlib PCC initialization and destruction functions: + +- ``bool initialize_pcc();`` +- ``bool initialize_pcc_wait_for_completion();`` +- ``bool destroy_pcc();`` + +The PCC can be initialized with either ``initialize_pcc()`` or +``initialize_pcc_wait_for_completion()``. + +- ``initialize_pcc_wait_for_completion()`` blocks until ``destroy_pcc()`` + is called from a separate pthread. +- ``initialize_pcc()`` is non-blocking and will be stopped when + ``destroy_pcc()`` is called. + +Both initialize functions will launch 3 pthreads: + +- 1 Timer pthread +- 1 SocketComm pthread +- 1 SessionLogic pthread + +When ``destroy_pcc()`` is called, all pthreads will be stopped and all +resources will be released. + +All 3 functions return true upon success, and false otherwise. + +PCEPlib PCC Initialization and Destruction with FRR infrastructure +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +PCEPlib PCC initialization and destruction functions: + +- ``bool initialize_pcc_infra(struct pceplib_infra_config *infra_config);`` +- ``bool destroy_pcc();`` + +The ``pceplib_infra_config`` struct has the following fields: + +- **void *pceplib_infra_mt** + - FRR Memory type pointer for infra related memory management + +- **void *pceplib_messages_mt** + - FRR Memory type pointer for PCEP messages related memory management + +- **pceplib_malloc_func mfunc** + - FRR malloc function pointer + +- **pceplib_calloc_func cfunc** + - FRR calloc function pointer + +- **pceplib_realloc_func rfunc** + - FRR realloc function pointer + +- **pceplib_strdup_func sfunc** + - FRR strdup function pointer + +- **pceplib_free_func ffunc** + - FRR free function pointer + +- **void *external_infra_data** + - FRR data used by FRR timers and sockets infrastructure + +- **ext_timer_create timer_create_func** + - FRR timer create function pointer + +- **ext_timer_cancel timer_cancel_func** + - FRR timer cancel function pointer + +- **ext_socket_write socket_write_func** + - FRR socket write function pointer, indicating fd is ready to be written to + +- **ext_socket_read socket_read_func** + - FRR socket write function pointer, indicating fd is ready to be read from + + +PCEPlib PCC configuration +------------------------- + +PCEPlib PCC configuratoin functions: + +- ``pcep_configuration *create_default_pcep_configuration();`` +- ``void destroy_pcep_configuration(pcep_configuration *config);`` + +A ``pcep_configuration`` object with default values is created with +``create_default_pcep_configuration()``. These values can be tailored to +specific use cases. + +Created ``pcep_configuration`` objects are destroyed with +``destroy_pcep_configuration()``. + + +PCEPlib PCC configuration paramaters +++++++++++++++++++++++++++++++++++++ + +The ``pcep_configuration`` object is defined in ``pcep_session_logic/include/pcep_session_logic.h`` +The attributes in the ``pcep_configuration`` object are detailed as follows. + +PCEP Connection parameters: + +- **dst_pcep_port** + - Defaults to 0, in which case the default PCEP TCP destination port + 4189 will be used. + - Set to use a specific PCEP TCP destination port. + +- **src_pcep_port** + - Defaults to 0, in which case the default PCEP TCP source port + 4189 will be used. + - Set to use a specific PCEP TCP source port. + +- **Source IP** + - Defaults to IPv4 INADDR_ANY + - Set **src_ip.src_ipv4** and **is_src_ipv6=false** to set the source IPv4. + - Set **src_ip.src_ipv6** and **is_src_ipv6=true** to set the source IPv6. + +- **socket_connect_timeout_millis** + - Maximum amount of time to wait to connect to the PCE TCP socket + before failing, in milliseconds. + +PCEP Versioning: + +- **pcep_msg_versioning->draft_ietf_pce_segment_routing_07** + - Defaults to false, in which case draft 16 versioning will be used. + - Set to true to use draft 07 versioning. + +PCEP Open Message Parameters: + +- **keep_alive_seconds** + - Sent to PCE in PCEP Open Msg + - Recommended value = 30, Minimum value = 1 + - Disabled by setting value = 0 + +- **dead_timer_seconds** + - Sent to PCE in PCEP Open Msg + - Recommended value = 4 * keepalive timer value + +- Supported value ranges for PCEP Open Message received from the PCE + - **min_keep_alive_seconds**, **max_keep_alive_seconds** + - **min_dead_timer_seconds**, **max_dead_timer_seconds** + +- **request_time_seconds** + - When a PCC sends a PcReq to a PCE, the amount of time a PCC will + wait for a PcRep reply from the PCE. + +- **max_unknown_requests** + - If a PCC/PCE receives PCRep/PCReq messages with unknown requests + at a rate equal or greater than MAX-UNKNOWN-REQUESTS per minute, + the PCC/PCE MUST send a PCEP CLOSE message. + - Recommended value = 5 + +- **max_unknown_messages** + - If a PCC/PCE receives unrecognized messages at a rate equal or + greater than MAX-UNKNOWN-MESSAGES per minute, the PCC/PCE MUST + send a PCEP CLOSE message + - Recommended value = 5 + +Stateful PCE Capability TLV configuration parameters (RFC 8231, 8232, 8281, and +draft-ietf-pce-segment-routing-16): + +- **support_stateful_pce_lsp_update** + - If this flag is true, then a Stateful PCE Capability TLV will + be added to the PCEP Open object, with the LSP Update Capability + U-flag set true. + - The rest of these parameters are used to configure the Stateful + PCE Capability TLV + +- **support_pce_lsp_instantiation** + - Sets the I-flag true, indicating the PCC allows instantiation + of an LSP by a PCE. + +- **support_include_db_version** + - Sets the S-bit true, indicating the PCC will include the + LSP-DB-VERSION TLV in each LSP object. See lsp_db_version below. + +- **support_lsp_triggered_resync** + - Sets the T-bit true, indicating the PCE can trigger resynchronization + of LSPs at any point in the life of the session. + +- **support_lsp_delta_sync** + - Sets the D-bit true, indicating the PCEP speaker allows incremental + (delta) State Synchronization. + +- **support_pce_triggered_initial_sync** + - Sets the F-bit true, indicating the PCE SHOULD trigger initial (first) + State Synchronization + +LSP DB Version TLV configuration parameters: + +- **lsp_db_version** + - If this parameter has a value other than 0, and the above + support_include_db_version flag is true, then an LSP DB + Version TLV will be added to the PCEP Open object. + - This parameter should only be set if LSP-DB survived a restart + and is available. + - This value will be copied over to the pcep_session upon initialization. + +SR PCE Capability sub-TLV configuration parameters (draft-ietf-pce-segment-routing-16): + +- **support_sr_te_pst** + - If this flag is true, then an SR PCE Capability sub-TLV will be + added to a Path Setup type Capability TLV, which will be added + to the PCEP Open object. + - The PST used in the Path Setup type Capability will be 1, + indicating the Path is setup using Segment Routing Traffic Engineering. + +Only set the following fields if the **support_sr_te_pst** flag is true. + +- **pcc_can_resolve_nai_to_sid** + - Sets the N-flag true, indicating that the PCC is capable of resolving + a Node or Adjacency Identifier to a SID + +- **max_sid_depth** + - If set other than 0, then the PCC imposes a limit on the Maximum + SID depth. + - If this parameter is other than 0, then the X bit will be true, + and the parameter value will be set in the MSD field. + + +PCEPlib PCC connections +----------------------- + +PCEPlib PCC connect and disconnect functions: + +- ``pcep_session *connect_pce(pcep_configuration *config, struct in_addr *pce_ip);`` +- ``pcep_session *connect_pce_ipv6(pcep_configuration *config, struct in6_addr *pce_ip);`` +- ``void disconnect_pce(pcep_session *session);`` + +When connecting to a PCE, a ``pcep_session`` will be returned on success, NULL +otherwise. + +Refer to the above PCC configuration parameters section for setting the source +and destination PCEP TCP ports, and the source IP address and version. + + +PCEP Messages, Objects, and TLVs +-------------------------------- + +The PCEP messages, objects, and TLVs created in the PCEPlib are high-level API +structures, meaning they need to be encoded before being sent on-the-wire, and +the raw data received needs to be decoded into these structures. This makes +using these objects much easier for the library consumer, since they do not +need to know the detailed raw format of the PCEP entities. + + +PCEP Messages ++++++++++++++ + +Received messages (in the ``pcep_event`` explained below) are of type +``pcep_message``, which have the following fields: + +- ``struct pcep_message_header *msg_header;`` + - Defines the PCEP version and message type + +- ``double_linked_list *obj_list;`` + - A double linked list of the message objects + - Each entry is a pointer to a ``struct pcep_object_header``, and + using the ``object_class`` and ``object_type`` fields, the pointer + can be cast to the appropriate object structure to access the + rest of the object fields + +- ``uint8_t *encoded_message;`` + - This field is only populated for received messages or once the + ``pcep_encode_message()`` function has been called on the message. + - This field is a pointer to the raw PCEP data for the entire + message, including all objects and TLVs. + +- ``uint16_t encoded_message_length;`` + - This field is only populated for received messages or once the + ``pcep_encode_message()`` function has been called on the message. + - This field is the length of the entire raw message, including + all objects and TLVs. + - This field is in host byte order. + + +PCEP Objects +++++++++++++ + +A PCEP message has a double linked list of pointers to ``struct pcep_object_header`` +structures, which have the following fields: + +- ``enum pcep_object_classes object_class;`` +- ``enum pcep_object_types object_type;`` +- ``bool flag_p;`` + - PCC Processing rule bit: When set, the object MUST be taken into + account, when cleared the object is optional + +- ``bool flag_i;`` + - PCE Ignore bit: indicates to a PCC whether or not an optional + object was processed + +- ``double_linked_list *tlv_list;`` + - A double linked list of the object TLVs + - Each entry is a pointer to a ``struct pcep_object_tlv_header``, and + using the TLV type field, the pointer can be cast to the + appropriate TLV structure to access the rest of the TLV fields + +- ``uint8_t *encoded_object;`` + - This field is only populated for received objects or once the + ``pcep_encode_object()`` (called by ``pcep_encode_message()``) + function has been called on the object. + - Pointer into the encoded_message field (from the pcep_message) + where the raw object PCEP data starts. + +- ``uint16_t encoded_object_length;`` + - This field is only populated for received objects or once the + ``pcep_encode_object()`` (called by ``pcep_encode_message()``) + function has been called on the object. + - This field is the length of the entire raw TLV + - This field is in host byte order. + +The object class and type can be used to cast the ``struct pcep_object_header`` +pointer to the appropriate object structure so the specific object fields can +be accessed. + + +PCEP TLVs ++++++++++ + +A PCEP object has a double linked list of pointers to ``struct pcep_object_tlv_header`` +structures, which have the following fields: + +- ``enum pcep_object_tlv_types type;`` +- ``uint8_t *encoded_tlv;`` + - This field is only populated for received TLVs or once the + ``pcep_encode_tlv()`` (called by ``pcep_encode_message()``) + function has been called on the TLV. + - Pointer into the encoded_message field (from the pcep_message) + where the raw TLV PCEP data starts. + +- ``uint16_t encoded_tlv_length;`` + - This field is only populated for received TLVs or once the + ``pcep_encode_tlv()`` (called by ``pcep_encode_message()``) + function has been called on the TLV. + - This field is the length of the entire raw TLV + - This field is in host byte order. + + +Memory management ++++++++++++++++++ + +Any of the PCEPlib Message Library functions that receive a pointer to a +``double_linked_list``, ``pcep_object_header``, or ``pcep_object_tlv_header``, +transfer the ownership of the entity to the PCEPlib. The memory will be freed +internally when the encapsulating structure is freed. If the memory for any of +these is freed by the caller, then there will be a double memory free error +when the memory is freed internally in the PCEPlib. + +Any of the PCEPlib Message Library functions that receive either a pointer to a +``struct in_addr`` or ``struct in6_addr`` will allocate memory for the IP +address internally and copy the IP address. It is the responsibility of the +caller to manage the memory for the IP address passed into the PCEPlib Message +Library functions. + +For messages received via the event queue (explained below), the message will +be freed when the event is freed by calling ``destroy_pcep_event()``. + +When sending messages, the message will be freed internally in the PCEPlib when +the ``send_message()`` ``pcep_pcc`` API function when the ``free_after_send`` flag +is set true. + +To manually delete a message, call the ``pcep_msg_free_message()`` function. +Internally, this will call ``pcep_obj_free_object()`` and ``pcep_obj_free_tlv()`` +appropriately. + + +Sending a PCEP Report message +----------------------------- + +This section shows how to send a PCEP Report messages from the PCC to the PCE, +and serves as an example of how to send other messages. Refer to the sample +PCC binary located in ``pcep_pcc/src/pcep_pcc.c`` for code examples os sending +a PCEP Report message. + +The Report message must have at least an SRP, LSP, and ERO object. + +The PCEP Report message objects are created with the following APIs: + +- ``struct pcep_object_srp *pcep_obj_create_srp(...);`` +- ``struct pcep_object_lsp *pcep_obj_create_lsp(...);`` +- ``struct pcep_object_ro *pcep_obj_create_ero(...);`` + - Create ero subobjects with the ``pcep_obj_create_ro_subobj_*(...);`` functions + +PCEP Report message is created with the following API: + +- ``struct pcep_header *pcep_msg_create_report(double_linked_list *report_object_list);`` + +A PCEP report messages is sent with the following API: + +- ``void send_message(pcep_session *session, pcep_message *message, bool free_after_send);`` + + +PCEPlib Received event queue +---------------------------- + +PCEP events and messages of interest to the PCEPlib consumer will be stored +internally in a message queue for retrieval. + +The following are the event types: + +- **MESSAGE_RECEIVED** +- **PCE_CLOSED_SOCKET** +- **PCE_SENT_PCEP_CLOSE** +- **PCE_DEAD_TIMER_EXPIRED** +- **PCE_OPEN_KEEP_WAIT_TIMER_EXPIRED** +- **PCC_CONNECTED_TO_PCE** +- **PCC_CONNECTION_FAILURE** +- **PCC_PCEP_SESSION_CLOSED** +- **PCC_RCVD_INVALID_OPEN** +- **PCC_SENT_INVALID_OPEN** +- **PCC_RCVD_MAX_INVALID_MSGS** +- **PCC_RCVD_MAX_UNKOWN_MSGS** + +The following PCEP messages will not be posted on the message queue, as they +are handled internally in the library: + +- **Open** +- **Keep Alive** +- **Close** + +Received event queue API: + +- ``bool event_queue_is_empty();`` + - Returns true if the queue is empty, false otherwise + +- ``uint32_t event_queue_num_events_available();`` + - Return the number of events on the queue, 0 if empty + +- ``struct pcep_event *event_queue_get_event();`` + - Return the next event on the queue, NULL if empty + - The ``message`` pointer will only be non-NULL if ``event_type`` + is ``MESSAGE_RECEIVED`` + +- ``void destroy_pcep_event(struct pcep_event *event);`` + - Free the PCEP Event resources, including the PCEP message if present + + +PCEPlib Counters +---------------- + +The PCEPlib counters are managed in the ``pcep_session_logic`` library, and can +be accessed in the ``pcep_session_counters`` field of the ``pcep_session`` structure. +There are 2 API functions to manage the counters: + +- ``void dump_pcep_session_counters(pcep_session *session);`` + - Dump all of the counters to the logs + +- ``void reset_pcep_session_counters(pcep_session *session);`` + diff --git a/doc/developer/process-architecture.rst b/doc/developer/process-architecture.rst new file mode 100644 index 0000000..06ee6a3 --- /dev/null +++ b/doc/developer/process-architecture.rst @@ -0,0 +1,328 @@ +.. _process-architecture: + +Process Architecture +==================== + +FRR is a suite of daemons that serve different functions. This document +describes internal architecture of daemons, focusing their general design +patterns, and especially how threads are used in the daemons that use them. + +Overview +-------- +The fundamental pattern used in FRR daemons is an `event loop +<https://en.wikipedia.org/wiki/Event_loop>`_. Some daemons use `kernel threads +<https://en.wikipedia.org/wiki/Thread_(computing)#Kernel_threads>`_. In these +daemons, each kernel thread runs its own event loop. The event loop +implementation is constructed to be thread safe and to allow threads other than +its owning thread to schedule events on it. The rest of this document describes +these two designs in detail. + +Terminology +----------- +Because this document describes the architecture for kernel threads as well as +the event system, a digression on terminology is in order here. + +Historically Quagga's loop system was viewed as an implementation of userspace +threading. Because of this design choice, the names for various datastructures +within the event system are variations on the term "thread". The primary +datastructure that holds the state of an event loop in this system is called a +"threadmaster". Events scheduled on the event loop - what would today be called +an 'event' or 'task' in systems such as libevent - are called "threads" and the +datastructure for them is ``struct event``. To add to the confusion, these +"threads" have various types, one of which is "event". To hopefully avoid some +of this confusion, this document refers to these "threads" as a 'task' except +where the datastructures are explicitly named. When they are explicitly named, +they will be formatted ``like this`` to differentiate from the conceptual +names. When speaking of kernel threads, the term used will be "pthread" since +FRR's kernel threading implementation uses the POSIX threads API. + +.. This should be broken into its document under :ref:`libfrr` +.. _event-architecture: + +Event Architecture +------------------ +This section presents a brief overview of the event model as currently +implemented in FRR. This doc should be expanded and broken off into its own +section. For now it provides basic information necessary to understand the +interplay between the event system and kernel threads. + +The core event system is implemented in :file:`lib/event.c` and +:file:`lib/frrevent.h`. The primary +structure is ``struct event_loop``, hereafter referred to as a +``threadmaster``. A ``threadmaster`` is a global state object, or context, that +holds all the tasks currently pending execution as well as statistics on tasks +that have already executed. The event system is driven by adding tasks to this +data structure and then calling a function to retrieve the next task to +execute. At initialization, a daemon will typically create one +``threadmaster``, add a small set of initial tasks, and then run a loop to +fetch each task and execute it. + +These tasks have various types corresponding to their general action. The types +are given by integer macros in :file:`frrevent.h` and are: + +``EVENT_READ`` + Task which waits for a file descriptor to become ready for reading and then + executes. + +``EVENT_WRITE`` + Task which waits for a file descriptor to become ready for writing and then + executes. + +``EVENT_TIMER`` + Task which executes after a certain amount of time has passed since it was + scheduled. + +``EVENT_EVENT`` + Generic task that executes with high priority and carries an arbitrary + integer indicating the event type to its handler. These are commonly used to + implement the finite state machines typically found in routing protocols. + +``EVENT_READY`` + Type used internally for tasks on the ready queue. + +``EVENT_UNUSED`` + Type used internally for ``struct event`` objects that aren't being used. + The event system pools ``struct event`` to avoid heap allocations; this is + the type they have when they're in the pool. + +``EVENT_EXECUTE`` + Just before a task is run its type is changed to this. This is used to show + ``X`` as the type in the output of :clicmd:`show thread cpu`. + +The programmer never has to work with these types explicitly. Each type of task +is created and queued via special-purpose functions (actually macros, but +irrelevant for the time being) for the specific type. For example, to add a +``EVENT_READ`` task, you would call + +:: + + event_add_read(struct event_loop *master, int (*handler)(struct event *), void *arg, int fd, struct event **ref); + +The ``struct event`` is then created and added to the appropriate internal +datastructure within the ``threadmaster``. Note that the ``READ`` and +``WRITE`` tasks are independent - a ``READ`` task only tests for +readability, for example. + +The Event Loop +^^^^^^^^^^^^^^ +To use the event system, after creating a ``threadmaster`` the program adds an +initial set of tasks. As these tasks execute, they add more tasks that execute +at some point in the future. This sequence of tasks drives the lifecycle of the +program. When no more tasks are available, the program dies. Typically at +startup the first task added is an I/O task for VTYSH as well as any network +sockets needed for peerings or IPC. + +To retrieve the next task to run the program calls ``event_fetch()``. +``event_fetch()`` internally computes which task to execute next based on +rudimentary priority logic. Events (type ``EVENT_EVENT``) execute with the +highest priority, followed by expired timers and finally I/O tasks (type +``EVENT_READ`` and ``EVENT_WRITE``). When scheduling a task a function and an +arbitrary argument are provided. The task returned from ``event_fetch()`` is +then executed with ``event_call()``. + +The following diagram illustrates a simplified version of this infrastructure. + +.. todo: replace these with SVG +.. figure:: ../figures/threadmaster-single.png + :align: center + + Lifecycle of a program using a single threadmaster. + +The series of "task" boxes represents the current ready task queue. The various +other queues for other types are not shown. The fetch-execute loop is +illustrated at the bottom. + +Mapping the general names used in the figure to specific FRR functions: + +- ``task`` is ``struct event *`` +- ``fetch`` is ``event_fetch()`` +- ``exec()`` is ``event_call()`` +- ``cancel()`` is ``event_cancel()`` +- ``schedule()`` is any of the various task-specific ``event_add_*`` functions + +Adding tasks is done with various task-specific function-like macros. These +macros wrap underlying functions in :file:`event.c` to provide additional +information added at compile time, such as the line number the task was +scheduled from, that can be accessed at runtime for debugging, logging and +informational purposes. Each task type has its own specific scheduling function +that follow the naming convention ``event_add_<type>``; see :file:`frrevent.h` +for details. + +There are some gotchas to keep in mind: + +- I/O tasks are keyed off the file descriptor associated with the I/O + operation. This means that for any given file descriptor, only one of each + type of I/O task (``EVENT_READ`` and ``EVENT_WRITE``) can be scheduled. For + example, scheduling two write tasks one after the other will overwrite the + first task with the second, resulting in total loss of the first task and + difficult bugs. + +- Timer tasks are only as accurate as the monotonic clock provided by the + underlying operating system. + +- Memory management of the arbitrary handler argument passed in the schedule + call is the responsibility of the caller. + + +Kernel Thread Architecture +-------------------------- +Efforts have begun to introduce kernel threads into FRR to improve performance +and stability. Naturally a kernel thread architecture has long been seen as +orthogonal to an event-driven architecture, and the two do have significant +overlap in terms of design choices. Since the event model is tightly integrated +into FRR, careful thought has been put into how pthreads are introduced, what +role they fill, and how they will interoperate with the event model. + +Design Overview +^^^^^^^^^^^^^^^ +Each kernel thread behaves as a lightweight process within FRR, sharing the +same process memory space. On the other hand, the event system is designed to +run in a single process and drive serial execution of a set of tasks. With this +consideration, a natural choice is to implement the event system within each +kernel thread. This allows us to leverage the event-driven execution model with +the currently existing task and context primitives. In this way the familiar +execution model of FRR gains the ability to execute tasks simultaneously while +preserving the existing model for concurrency. + +The following figure illustrates the architecture with multiple pthreads, each +running their own ``threadmaster``-based event loop. + +.. todo: replace these with SVG +.. figure:: ../figures/threadmaster-multiple.png + :align: center + + Lifecycle of a program using multiple pthreads, each running their own + ``threadmaster`` + +Each roundrect represents a single pthread running the same event loop +described under :ref:`event-architecture`. Note the arrow from the ``exec()`` +box on the right to the ``schedule()`` box in the middle pthread. This +illustrates code running in one pthread scheduling a task onto another +pthread's threadmaster. A global lock for each ``threadmaster`` is used to +synchronize these operations. The pthread names are examples. + + +.. This should be broken into its document under :ref:`libfrr` +.. _kernel-thread-wrapper: + +Kernel Thread Wrapper +^^^^^^^^^^^^^^^^^^^^^ +The basis for the integration of pthreads and the event system is a lightweight +wrapper for both systems implemented in :file:`lib/frr_pthread.[ch]`. The +header provides a core datastructure, ``struct frr_pthread``, that encapsulates +structures from both POSIX threads and :file:`event.c`, :file:`frrevent.h`. +In particular, this +datastructure has a pointer to a ``threadmaster`` that runs within the pthread. +It also has fields for a name as well as start and stop functions that have +signatures similar to the POSIX arguments for ``pthread_create()``. + +Calling ``frr_pthread_new()`` creates and registers a new ``frr_pthread``. The +returned structure has a pre-initialized ``threadmaster``, and its ``start`` +and ``stop`` functions are initialized to defaults that will run a basic event +loop with the given threadmaster. Calling ``frr_pthread_run()`` starts the thread +with the ``start`` function. From there, the model is the same as the regular +event model. To schedule tasks on a particular pthread, simply use the regular +:file:`event.c` functions as usual and provide the ``threadmaster`` pointed to +from the ``frr_pthread``. As part of implementing the wrapper, the +:file:`event.c` functions were made thread-safe. Consequently, it is safe to +schedule events on a ``threadmaster`` belonging both to the calling thread as +well as *any other pthread*. This serves as the basis for inter-thread +communication and boils down to a slightly more complicated method of message +passing, where the messages are the regular task events as used in the +event-driven model. The only difference is thread cancellation, which requires +calling ``event_cancel_async()`` instead of ``event_cancel()`` to cancel a task +currently scheduled on a ``threadmaster`` belonging to a different pthread. +This is necessary to avoid race conditions in the specific case where one +pthread wants to guarantee that a task on another pthread is cancelled before +proceeding. + +In addition, the existing commands to show statistics and other information for +tasks within the event driven model have been expanded to handle multiple +pthreads; running :clicmd:`show thread cpu` will display the usual event +breakdown, but it will do so for each pthread running in the program. For +example, :ref:`bgpd` runs a dedicated I/O pthread and shows the following +output for :clicmd:`show thread cpu`: + +:: + + frr# show thread cpu + + Thread statistics for bgpd: + + Showing statistics for pthread main + ------------------------------------ + CPU (user+system): Real (wall-clock): + Active Runtime(ms) Invoked Avg uSec Max uSecs Avg uSec Max uSecs Type Thread + 0 1389.000 10 138900 248000 135549 255349 T subgroup_coalesce_timer + 0 0.000 1 0 0 18 18 T bgp_startup_timer_expire + 0 850.000 18 47222 222000 47795 233814 T work_queue_run + 0 0.000 10 0 0 6 14 T update_subgroup_merge_check_thread_cb + 0 0.000 8 0 0 117 160 W zclient_flush_data + 2 2.000 1 2000 2000 831 831 R bgp_accept + 0 1.000 1 1000 1000 2832 2832 E zclient_connect + 1 42082.000 240574 174 37000 178 72810 R vtysh_read + 1 152.000 1885 80 2000 96 6292 R zclient_read + 0 549346.000 2997298 183 7000 153 20242 E bgp_event + 0 2120.000 300 7066 14000 6813 22046 T (bgp_holdtime_timer) + 0 0.000 2 0 0 57 59 T update_group_refresh_default_originate_route_map + 0 90.000 1 90000 90000 73729 73729 T bgp_route_map_update_timer + 0 1417.000 9147 154 48000 132 61998 T bgp_process_packet + 300 71807.000 2995200 23 3000 24 11066 T (bgp_connect_timer) + 0 1894.000 12713 148 45000 112 33606 T (bgp_generate_updgrp_packets) + 0 0.000 1 0 0 105 105 W vtysh_write + 0 52.000 599 86 2000 138 6992 T (bgp_start_timer) + 1 1.000 8 125 1000 164 593 R vtysh_accept + 0 15.000 600 25 2000 15 153 T (bgp_routeadv_timer) + 0 11.000 299 36 3000 53 3128 RW bgp_connect_check + + + Showing statistics for pthread BGP I/O thread + ---------------------------------------------- + CPU (user+system): Real (wall-clock): + Active Runtime(ms) Invoked Avg uSec Max uSecs Avg uSec Max uSecs Type Thread + 0 1611.000 9296 173 13000 188 13685 R bgp_process_reads + 0 2995.000 11753 254 26000 182 29355 W bgp_process_writes + + + Showing statistics for pthread BGP Keepalives thread + ----------------------------------------------------- + CPU (user+system): Real (wall-clock): + Active Runtime(ms) Invoked Avg uSec Max uSecs Avg uSec Max uSecs Type Thread + No data to display yet. + +Attentive readers will notice that there is a third thread, the Keepalives +thread. This thread is responsible for -- surprise -- generating keepalives for +peers. However, there are no statistics showing for that thread. Although the +pthread uses the ``frr_pthread`` wrapper, it opts not to use the embedded +``threadmaster`` facilities. Instead it replaces the ``start`` and ``stop`` +functions with custom functions. This was done because the ``threadmaster`` +facilities introduce a small but significant amount of overhead relative to the +pthread's task. In this case since the pthread does not need the event-driven +model and does not need to receive tasks from other pthreads, it is simpler and +more efficient to implement it outside of the provided event facilities. The +point to take away from this example is that while the facilities to make using +pthreads within FRR easy are already implemented, the wrapper is flexible and +allows usage of other models while still integrating with the rest of the FRR +core infrastructure. Starting and stopping this pthread works the same as it +does for any other ``frr_pthread``; the only difference is that event +statistics are not collected for it, because there are no events. + +Notes on Design and Documentation +--------------------------------- +Because of the choice to embed the existing event system into each pthread +within FRR, at this time there is not integrated support for other models of +pthread use such as divide and conquer. Similarly, there is no explicit support +for thread pooling or similar higher level constructs. The currently existing +infrastructure is designed around the concept of long-running worker threads +responsible for specific jobs within each daemon. This is not to say that +divide and conquer, thread pooling, etc. could not be implemented in the +future. However, designs in this direction must be very careful to take into +account the existing codebase. Introducing kernel threads into programs that +have been written under the assumption of a single thread of execution must be +done very carefully to avoid insidious errors and to ensure the program remains +understandable and maintainable. + +In keeping with these goals, future work on kernel threading should be +extensively documented here and FRR developers should be very careful with +their design choices, as poor choices tightly integrated can prove to be +catastrophic for development efforts in the future. diff --git a/doc/developer/rcu.rst b/doc/developer/rcu.rst new file mode 100644 index 0000000..4fd5658 --- /dev/null +++ b/doc/developer/rcu.rst @@ -0,0 +1,269 @@ +.. highlight:: c + +RCU +=== + +Introduction +------------ + +RCU (Read-Copy-Update) is, fundamentally, a paradigm of multithreaded +operation (and not a set of APIs.) The core ideas are: + +* longer, complicated updates to structures are made only on private, + "invisible" copies. Other threads, when they access the structure, see an + older (but consistent) copy. + +* once done, the updated copy is swapped in a single operation so that + other threads see either the old or the new data but no inconsistent state + between. + +* the old instance is only released after making sure that it is impossible + any other thread might still be reading it. + +For more information, please search for general or Linux kernel RCU +documentation; there is no way this doc can be comprehensive in explaining the +interactions: + +* https://en.wikipedia.org/wiki/Read-copy-update +* https://www.kernel.org/doc/html/latest/kernel-hacking/locking.html#avoiding-locks-read-copy-update +* https://lwn.net/Articles/262464/ +* http://www.rdrop.com/users/paulmck/RCU/rclock_OLS.2001.05.01c.pdf +* http://lse.sourceforge.net/locking/rcupdate.html + +RCU, the TL;DR +^^^^^^^^^^^^^^ + +#. data structures are always consistent for reading. That's the "R" part. +#. reading never blocks / takes a lock. +#. rcu_read_lock is not a lock in the traditional sense. Think of it as a + "reservation"; it notes what the *oldest* possible thing the thread might + be seeing is, and which thus can't be deleted yet. +#. you create some object, finish it up, and then publish it. +#. publishing is an ``atomic_*`` call with ``memory_order_release``, which + tells the compiler to make sure prior memory writes have completed before + doing the atomic op. +#. ``ATOMLIST_*`` ``add`` operations do the ``memory_order_release`` for you. +#. you can't touch the object after it is published, except with atomic ops. +#. because you can't touch it, if you want to change it you make a new copy, + work on that, and then publish the new copy. That's the "CU" part. +#. deleting the object is also an atomic op. +#. other threads that started working before you published / deleted an object + might not see the new object / still see the deleted object. +#. because other threads may still see deleted objects, the ``free()`` needs + to be delayed. That's what :c:func:`rcu_free()` is for. + + +When (not) to use RCU +^^^^^^^^^^^^^^^^^^^^^ + +RCU is designed for read-heavy workloads where objects are updated relatively +rarely, but frequently accessed. Do *not* indiscriminately replace locking by +RCU patterns. + +The "copy" part of RCU implies that, while updating, several copies of a given +object exist in parallel. Even after the updated copy is swapped in, the old +object remains queued for freeing until all other threads are guaranteed to +not be accessing it anymore, due to passing a sequence point. In addition to +the increased memory usage, there may be some bursted (due to batching) malloc +contention when the RCU cleanup thread does its thing and frees memory. + +Other useful patterns +^^^^^^^^^^^^^^^^^^^^^ + +In addition to the full "copy object, apply changes, atomically update" +approach, there are 2 "reduced" usage cases that can be done: + +* atomically updating single pieces of a particular object, e.g. some flags + or configuration piece + +* straight up read-only / immutable objects + +Both of these cases can be considered RCU "subsets". For example, when +maintaining an atomic list of items, but these items only have a single +integer value that needs to be updated, that value can be atomically updated +without copying the entire object. However, the object still needs to be +free'd through :c:func:`rcu_free()` since reading/updating and deleting might +be happening concurrently. The same applies for immutable objects; deletion +might still race with reading so they need to be free'd through RCU. + +FRR API +------- + +Before diving into detail on the provided functions, it is important to note +that the FRR RCU API covers the **cleanup part of RCU, not the read-copy-update +paradigm itself**. These parts are handled by standard C11 atomic operations, +and by extension through the atomic data structures (ATOMLIST, ATOMSORT & co.) + +The ``rcu_*`` functions only make sense in conjunction with these RCU access +patterns. If you're calling the RCU API but not using these, something is +wrong. The other way around is not necessarily true; it is possible to use +atomic ops & datastructures with other types of locking, e.g. rwlocks. + +.. c:function:: void rcu_read_lock() +.. c:function:: void rcu_read_unlock() + + These functions acquire / release the RCU read-side lock. All access to + RCU-guarded data must be inside a block guarded by these. Any number of + threads may hold the RCU read-side lock at a given point in time, including + both no threads at all and all threads. + + The functions implement a depth counter, i.e. can be nested. The nested + calls are cheap, since they only increment/decrement the counter. + Therefore, any place that uses RCU data and doesn't have a guarantee that + the caller holds RCU (e.g. ``lib/`` code) should just have its own + rcu_read_lock/rcu_read_unlock pair. + + At the "root" level (e.g. un-nested), these calls can incur the cost of one + syscall (to ``futex()``). That puts them on about the same cost as a + mutex lock/unlock. + + The ``thread_master`` code currently always holds RCU everywhere, except + while doing the actual ``poll()`` syscall. This is both an optimization as + well as an "easement" into getting RCU going. The current implementation + contract is that any ``struct event *`` callback is called with a RCU + holding depth of 1, and that this is owned by the thread so it may (should) + drop and reacquire it when doing some longer-running work. + + .. warning:: + + The RCU read-side lock must be held **continuously** for the entire time + any piece of RCU data is used. This includes any access to RCU data + after the initial ``atomic_load``. If the RCU read-side lock is + released, any RCU-protected pointers as well as the data they refer to + become invalid, as another thread may have called :c:func:`rcu_free` on + them. + +.. c:struct:: rcu_head +.. c:struct:: rcu_head_close +.. c:struct:: rcu_action + + The ``rcu_head`` structures are small (16-byte) bits that contain the + queueing machinery for the RCU sweeper/cleanup mechanisms. + + Any piece of data that is cleaned up by RCU needs to have a matching + ``rcu_head`` embedded in it. If there is more than one cleanup operation + to be done (e.g. closing a file descriptor), more than one ``rcu_head`` may + be embedded. + + .. warning:: + + It is not possible to reuse a ``rcu_head``. It is owned by the RCU code + as soon as ``rcu_*`` is called on it. + + The ``_close`` variant carries an extra ``int fd`` field to store the fd to + be closed. + + To minimize the amount of memory used for ``rcu_head``, details about the + RCU operation to be performed are moved into the ``rcu_action`` structure. + It contains e.g. the MTYPE for :c:func:`rcu_free` calls. The pointer to be + freed is stored as an offset relative to the ``rcu_head``, which means it + must be embedded as a struct field so the offset is constant. + + The ``rcu_action`` structure is an implementation detail. Using + ``rcu_free`` or ``rcu_close`` will set it up correctly without further + code needed. + + The ``rcu_head`` may be put in an union with other data if the other data + is only used during "life" of the data, since the ``rcu_head`` is used only + for the "death" of data. But note that other threads may still be reading + a piece of data while a thread is working to free it. + +.. c:function:: void rcu_free(struct memtype *mtype, struct X *ptr, field) + + Free a block of memory after RCU has ensured no other thread can be + accessing it anymore. The pointer remains valid for any other thread that + has called :c:func:`rcu_read_lock` before the ``rcu_free`` call. + + .. warning:: + + In some other RCU implementations, the pointer remains valid to the + *calling* thread if it is holding the RCU read-side lock. This is not + the case in FRR, particularly when running single-threaded. Enforcing + this rule also allows static analysis to find use-after-free issues. + + ``mtype`` is the libfrr ``MTYPE_FOO`` allocation type to pass to + :c:func:`XFREE`. + + ``field`` must be the name of a ``struct rcu_head`` member field in ``ptr``. + The offset of this field (which must be constant) is used to reduce the + memory size of ``struct rcu_head``. + + .. note:: + + ``rcu_free`` (and ``rcu_close``) calls are more efficient if they are + put close to each other. When freeing several RCU'd resources, try to + move the calls next to each other (even if the data structures do not + directly point to each other.) + + Having the calls bundled reduces the cost of adding the ``rcu_head`` to + the RCU queue; the RCU queue is an atomic data structure whose usage + will require the CPU to acquire an exclusive hold on relevant cache + lines. + +.. c:function:: void rcu_close(struct rcu_head_close *head, int fd) + + Close a file descriptor after ensuring no other thread might be using it + anymore. Same as :c:func:`rcu_free`, except it calls ``close`` instead of + ``free``. + +Internals +^^^^^^^^^ + +.. c:struct:: rcu_thread + + Per-thread state maintained by the RCU code, set up by the following + functions. A pointer to a thread's own ``rcu_thread`` is saved in + thread-local storage. + +.. c:function:: struct rcu_thread *rcu_thread_prepare(void) +.. c:function:: void rcu_thread_unprepare(struct rcu_thread *rcu_thread) +.. c:function:: void rcu_thread_start(struct rcu_thread *rcu_thread) + + Since the RCU code needs to have a list of all active threads, these + functions are used by the ``frr_pthread`` code to set up threads. Teardown + is automatic. It should not be necessary to call these functions. + + Any thread that accesses RCU-protected data needs to be registered with + these functions. Threads that do not access RCU-protected data may call + these functions but do not need to. + + Note that passing a pointer to RCU-protected data to some library which + accesses that pointer makes the library "access RCU-protected data". In + that case, either all of the library's threads must be registered for RCU, + or the code must instead pass a (non-RCU) copy of the data to the library. + +.. c:function:: void rcu_shutdown(void) + + Stop the RCU sweeper thread and make sure all cleanup has finished. + + This function is called on daemon exit by the libfrr code to ensure pending + RCU operations are completed. This is mostly to get a clean exit without + memory leaks from queued RCU operations. It should not be necessary to + call this function as libfrr handles this. + +FRR specifics and implementation details +---------------------------------------- + +The FRR RCU infrastructure has the following characteristics: + +* it is Epoch-based with a 32-bit wrapping counter. (This is somewhat + different from other Epoch-based approaches which may be designed to only + use 3 counter values, but works out to a simple implementation.) + +* instead of tracking CPUs as the Linux kernel does, threads are tracked. This + has exactly zero semantic impact, RCU just cares about "threads of + execution", which the kernel can optimize to CPUs but we can't. But it + really boils down to the same thing. + +* there are no ``rcu_dereference`` and ``rcu_assign_pointer`` - use + ``atomic_load`` and ``atomic_store`` instead. (These didn't exist when the + Linux RCU code was created.) + +* there is no ``synchronize_rcu``; this is a design choice but may be revisited + at a later point. ``synchronize_rcu`` blocks a thread until it is guaranteed + that no other threads might still be accessing data structures that they may + have access to at the beginning of the function call. This is a blocking + design and probably not appropriate for FRR. Instead, ``rcu_call`` can be + used to have the RCU sweeper thread make a callback after the same constraint + is fulfilled in an asynchronous way. Most needs should be covered by + ``rcu_free`` and ``rcu_close``. diff --git a/doc/developer/release-announcement-template.md b/doc/developer/release-announcement-template.md new file mode 100644 index 0000000..658b87e --- /dev/null +++ b/doc/developer/release-announcement-template.md @@ -0,0 +1,40 @@ +<!--- +name: release-announcement-template +about: Template to use when drafing a new release announcement. DELETE THIS + BLOCK BEFORE PUBLISHING. +---> + +We are pleased to announce FRR <version>. + +<!-- Add a brief summary of major changes here --> + +Thank you to all contributors! + +Changelog +--------- + +<!-- List **only** user-visible changes in this section. When listing changes to individual daemons, alphabetize the list by daemon name. --> + +**All daemons:** +- <!-- List changes to all daemons --> + +<!-- If a new daemon was added, list it at the top here --> +**New daemon: <new>** +- Adds support for <protocol/feature> + +**daemon 1** +- <!-- List changes --> + +**daemon 2** +- <!-- List changes --> + +**daemon N** +- <!-- List changes --> + +### Internal improvements + +- <!-- List **only** user-invisible changes here --> + +### Packaging changes + +- <!-- List any new or removed packages here --> diff --git a/doc/developer/scripting.rst b/doc/developer/scripting.rst new file mode 100644 index 0000000..202f003 --- /dev/null +++ b/doc/developer/scripting.rst @@ -0,0 +1,628 @@ +.. _scripting: + +Scripting +========= + +.. seealso:: User docs for scripting + +Overview +-------- + +FRR has the ability to call Lua scripts to perform calculations, make +decisions, or otherwise extend builtin behavior with arbitrary user code. This +is implemented using the standard Lua C bindings. The supported version of Lua +is 5.3. + +C objects may be passed into Lua and Lua objects may be retrieved by C code via +a encoding/decoding system. In this way, arbitrary data from FRR may be passed to +scripts. + +The Lua environment is isolated from the C environment; user scripts cannot +access FRR's address space unless explicitly allowed by FRR. + +For general information on how Lua is used to extend C, refer to Part IV of +"Programming in Lua". + +https://www.lua.org/pil/contents.html#24 + + +Design +------ + +Why Lua +^^^^^^^ + +Lua is designed to be embedded in C applications. It is very small; the +standard library is 220K. It is relatively fast. It has a simple, minimal +syntax that is relatively easy to learn and can be understood by someone with +little to no programming experience. Moreover it is widely used to add +scripting capabilities to applications. In short it is designed for this task. + +Reasons against supporting multiple scripting languages: + +- Each language would require different FFI methods, and specifically + different object encoders; a lot of code +- Languages have different capabilities that would have to be brought to + parity with each other; a lot of work +- Languages have vastly different performance characteristics; this would + create alot of basically unfixable issues, and result in a single de facto + standard scripting language (the fastest) +- Each language would need a dedicated maintainer for the above reasons; + this is pragmatically difficult +- Supporting multiple languages fractures the community and limits the audience + with which a given script can be shared + +General +------- + +FRR's scripting functionality is provided in the form of Lua functions in Lua +scripts (``.lua`` files). One Lua script may contain many Lua functions. These +are respectively encapsulated in the following structures: + +.. code-block:: c + + struct frrscript { + /* Lua file name */ + char *name; + + /* hash of lua_function_states */ + struct hash *lua_function_hash; + }; + + struct lua_function_state { + /* Lua function name */ + char *name; + + lua_State *L; + }; + + +`struct frrscript`: Since all Lua functions are contained within scripts, the +following APIs manipulates this structure. ``name`` contains the +Lua script name and a hash of Lua functions to their function names. + +`struct lua_function_state` is an internal structure, but it essentially contains +the name of the Lua function and its state (a stack), which is run using Lua +library functions. + +In general, to run a Lua function, these steps must take place: + +- Initialization +- Load +- Call +- Delete + +Initialization +^^^^^^^^^^^^^^ + +The ``frrscript`` object encapsulates the Lua function state(s) from +one Lua script file. To create, use ``frrscript_new()`` which takes the +name of the Lua script. +The string ".lua" is appended to the script name, and the resultant filename +will be used to look for the script when we want to load a Lua function from it. + +For example, to create ``frrscript`` for ``/etc/frr/scripts/bingus.lua``: + +.. code-block:: c + + struct frrscript *fs = frrscript_new("bingus"); + + +The script is *not* read at this stage. +This function cannot be used to test for a script's presence. + +Load +^^^^ + +The function to be called must first be loaded. Use ``frrscript_load()`` +which takes a ``frrscript`` object, the name of the Lua function +and a callback function. +The script file will be read to load and compile the function. + +For example, to load the Lua function ``on_foo`` +in ``/etc/frr/scripts/bingus.lua``: + +.. code-block:: c + + int ret = frrscript_load(fs, "on_foo", NULL); + + +This function returns 0 if and only if the Lua function was successfully loaded. +A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error when loading the function. + +During loading the script is validated for syntax and its environment +is set up. By default this does not include the Lua standard library; there are +security issues to consider, though for practical purposes untrusted users +should not be able to write the scripts directory anyway. + +Call +^^^^ + +After loading, a Lua function can be called any number of times. + +Input +""""" + +Inputs to the Lua script should be given by providing a list of parenthesized +pairs, +where the first and second field identify the name of the variable and the +value it is bound to, respectively. +The types of the values must have registered encoders (more below); the compiler +will warn you otherwise. + +These variables are first encoded in-order, then provided as arguments +to the Lua function. In the example, note that ``c`` is passed in as a value +while ``a`` and ``b`` are passed in as pointers. + +.. code-block:: c + + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + + +.. code-block:: lua + + function on_foo(a, b, c) + -- a is 100, b is 200, c is 300 + ... + + +Output +"""""" + +.. code-block:: c + + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + // a is 500, b is 200, c is 300 + + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); + // d is 800 + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { ["a"] = 500, ["c"] = 700, ["d"] = 800 } + end + + +**Lua functions being called must return a single table of string names to +values.** +(Lua functions should return an empty table if there is no output.) +The keys of the table are mapped back to names of variables in C. Note that +the values in the table can also be tables. Since tables are Lua's primary +data structure, this design lets us return any Lua value. + +After the Lua function returns, the names of variables to ``frrscript_call()`` +are matched against keys of the returned table, and then decoded. The types +being decoded must have registered decoders (more below); the compiler will +warn you otherwise. + +In the example, since ``a`` was in the returned table and ``b`` was not, +``a`` was decoded and its value modified, while ``b`` was not decoded. +``c`` was decoded as well, but its decoder is a noop. +What modifications happen given a variable depends whether its name was +in the returned table and the decoder's implementation. + +.. warning:: + Always keep in mind that non const-qualified pointers in + ``frrscript_call()`` may be modified - this may be a source of bugs. + On the other hand, const-qualified pointers and other values cannot + be modified. + + +.. tip:: + You can make a copy of a data structure and pass that in instead, + so that modifications only happen to that copy. + +``frrscript_call()`` returns 0 if and only if the Lua function was successfully +called. A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error from the Lua interpreter. + +In the above example, ``d`` was not an input to ``frrscript_call()``, so its +value must be explicitly retrieved with ``frrscript_get_result``. + +``frrscript_get_result()`` takes a +decoder and string name which is used as a key to search the returned table. +Returns the pointer to the decoded value, or NULL if it was not found. +In the example, ``d`` is a "new" value in C space, +so memory allocation might take place. Hence the caller is +responsible for memory deallocation. + +``frrscript_call()`` may be called multiple times without re-loading with +``frrscript_load()``. Results are not preserved between consecutive calls. + +.. code-block:: c + + frrscript_load(fs, "on_foo"); + + frrscript_call(fs, "on_foo"); + frrscript_get_result(fs, "on_foo", ...); + frrscript_call(fs, "on_foo"); + frrscript_get_result(fs, "on_foo", ...); + + +Delete +^^^^^^ + +To delete a script and the all Lua states associated with it: + +.. code-block:: c + + frrscript_delete(fs); + + +A complete example +"""""""""""""""""" + +So, a typical execution call, with error checking, looks something like this: + +.. code-block:: c + + struct frrscript *fs = frrscript_new("my_script"); // name *without* .lua + + int ret = frrscript_load(fs, "on_foo", NULL); + if (ret != 0) + goto DONE; // Lua script or function might have not been found + + int a = 100, b = 200, c = 300; + ret = frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + if (ret != 0) + goto DONE; // Lua function might have not successfully run + + // a and b might be modified + assert(a == 500); + assert(b == 200); + + // c could not have been modified + assert(c == 300); + + // d is new + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); + + if (!d) + goto DONE; // "d" might not have been in returned table + + assert(*d == 800); + XFREE(MTYPE_SCRIPT_RES, d); // caller responsible for free + + DONE: + frrscript_delete(fs); + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { a = 500, c = 700, d = 800 } + end + + +Note that ``{ a = ...`` is same as ``{ ["a"] = ...``; it is Lua shorthand to +use the variable name as the key in a table. + +Encoding and Decoding +^^^^^^^^^^^^^^^^^^^^^ + +Earlier sections glossed over the types of values that can be passed into +``frrscript_call()`` and how data is passed between C and Lua. Lua, as a +dynamically typed, garbage collected language, cannot directly use C values +without some kind of encoding / decoding system to +translate types between the two runtimes. + +Lua communicates with C code using a stack. C code wishing to provide data to +Lua scripts must provide a function that encodes the C data into a Lua +representation and pushes it on the stack. C code wishing to retrieve data from +Lua must provide a corresponding decoder function that retrieves a Lua +value from the stack and converts it to the corresponding C type. + +Encoders and decoders are provided for common data types. +Developers wishing to pass their own data structures between C and Lua need to +create encoders and decoders for that data type. + +We try to keep them named consistently. +There are three kinds of encoders and decoders: + +1. lua_push*: encodes a value onto the Lua stack. + Required for ``frrscript_call``. + +2. lua_decode*: decodes a value from the Lua stack. + Required for ``frrscript_call``. + Only non const-qualified pointers may be actually decoded (more below). + +3. lua_to*: allocates memory and decodes a value from the Lua stack. + Required for ``frrscript_get_result``. + +This design allows us to combine typesafe *modification* of C values as well as +*allocation* of new C values. + +In the following sections, we will use the encoders/decoders for ``struct prefix`` as an example. + +Encoding +"""""""" + +An encoder function takes a ``lua_State *``, a C type and pushes that value onto +the Lua state (a stack). +For C structs, the usual case, +this will typically be encoded to a Lua table, then pushed onto the Lua stack. + +Here is the encoder function for ``struct prefix``: + +.. code-block:: c + + void lua_pushprefix(lua_State *L, struct prefix *prefix) + { + char buffer[PREFIX_STRLEN]; + + lua_newtable(L); + lua_pushstring(L, prefix2str(prefix, buffer, PREFIX_STRLEN)); + lua_setfield(L, -2, "network"); + lua_pushinteger(L, prefix->prefixlen); + lua_setfield(L, -2, "length"); + lua_pushinteger(L, prefix->family); + lua_setfield(L, -2, "family"); + } + +This function pushes a single value, a table, onto the Lua stack, whose +equivalent in Lua is: + +.. code-block:: c + + { ["network"] = "1.2.3.4/24", ["prefixlen"] = 24, ["family"] = 2 } + + +Decoding +"""""""" + +Decoders are a bit more involved. They do the reverse; a decoder function takes +a ``lua_State *``, pops a value off the Lua stack and converts it back into its +C type. + +There are two: ``lua_decode*`` and ``lua_to*``. The former does no mememory +allocation and is needed for ``frrscript_call``. +The latter performs allocation and is optional. + +A ``lua_decode_*`` function takes a ``lua_State*``, an index, and a pointer +to a C data structure, and directly modifies the structure with values from the +Lua stack. Note that only non const-qualified pointers may be modified; +``lua_decode_*`` for other types will be noops. + +Again, for ``struct prefix *``: + +.. code-block:: c + + void lua_decode_prefix(lua_State *L, int idx, struct prefix *prefix) + { + lua_getfield(L, idx, "network"); + (void)str2prefix(lua_tostring(L, -1), prefix); + /* pop the network string */ + lua_pop(L, 1); + /* pop the prefix table */ + lua_pop(L, 1); + } + + +Note: + - Before ``lua_decode*`` is run, the "prefix" table is already on the top of + the stack. ``frrscript_call`` does this for us. + - However, at the end of ``lua_decode*``, the "prefix" table should be popped. + - The other two fields in the "network" table are disregarded, meaning that any + modification to them is discarded in C space. In this case, this is desired + behavior. + +.. warning:: + + ``lua_decode*`` functions should pop all values that ``lua_to*`` pushed onto + the Lua stack. + For encoders that pushed a table, its decoder should pop the table at the end. + The above is an example. + + + +``int`` is not a non const-qualified pointer, so for ``int``: + +.. code-block:: c + + void lua_decode_int_noop(lua_State *L, int idx, int i) + { //noop + } + + +A ``lua_to*`` function provides identical functionality except that it first +allocates memory for the new C type before decoding the value from the Lua stack, +then returns a pointer to the newly allocated C type. You only need to implement +this function to use with ``frrscript_get_result`` to retrieve a result of +this type. + +This function can and should be implemented using ``lua_decode_*``: + +.. code-block:: c + + void *lua_toprefix(lua_State *L, int idx) + { + struct prefix *p = XCALLOC(MTYPE_SCRIPT_RES, sizeof(struct prefix)); + + lua_decode_prefix(L, idx, p); + return p; + } + + +The returned data must always be copied off the stack and the copy must be +allocated with ``MTYPE_SCRIPT_RES``. This way it is possible to unload the script +(destroy the state) without invalidating any references to values stored in it. +Note that it is the caller's responsibility to free the data. + + +Registering encoders and decoders for frrscript_call +"""""""""""""""""""""""""""""""""""""""""""""""""""" + +To register a new type with its ``lua_push*`` and ``lua_decode*`` functions, +add the mapping in the following macros in ``frrscript.h``: + +.. code-block:: diff + + #define ENCODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_pushpeer \ + + struct peer * : lua_pushpeer, \ + + struct prefix * : lua_pushprefix \ + )((L), (value)) + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_decode_peer \ + + struct peer * : lua_decode_peer, \ + + struct prefix * : lua_decode_prefix \ + )((L), -1, (value)) + + +At compile time, the compiler will search for encoders/decoders for the type of +each value passed in via ``frrscript_call``. If a encoder/decoder cannot be +found, it will appear as a compile warning. Note that the types must +match *exactly*. +In the above example, we defined encoders/decoders for a value of +``struct prefix *``, but not ``struct prefix`` or ``const struct prefix *``. + +``const`` values are a special case. We want to use them in our Lua scripts +but not modify them, so creating a decoder for them would be meaningless. +But we still need a decoder for the type of value so that the compiler will be +satisfied. +For that, use ``lua_decode_noop``: + +.. code-block:: diff + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + + const struct prefix * : lua_decode_noop \ + )(L, -1, value) + + +.. note:: + + Encodable/decodable types are not restricted to simple values like integers, + strings and tables. + It is possible to encode a type such that the resultant object in Lua + is an actual object-oriented object, complete with methods that call + back into defined C functions. See the Lua manual for how to do this; + for a code example, look at how zlog is exported into the script environment. + + +Script Environment +------------------ + +Logging +^^^^^^^ + +For convenience, script environments are populated by default with a ``log`` +object which contains methods corresponding to each of the ``zlog`` levels: + +.. code-block:: lua + + log.info("info") + log.warn("warn") + log.error("error") + log.notice("notice") + log.debug("debug") + +The log messages will show up in the daemon's log output. + + +Examples +-------- + +For a complete code example involving passing custom types, retrieving results, +and doing complex calculations in Lua, look at the implementation of the +``match script SCRIPT`` command for BGP routemaps. This example calls into a +script with a function named ``route_match``, +provides route prefix and attributes received from a peer and expects the +function to return a match / no match / match and update result. + +An example script to use with this follows. This function matches, does not match +or updates a route depending on how many BGP UPDATE messages the peer has +received when the script is called, simply as a demonstration of what can be +accomplished with scripting. + +.. code-block:: lua + + + -- Example route map matching + -- author: qlyoung + -- + -- The following variables are available in the global environment: + -- log + -- logging library, with the usual functions + -- + -- route_match arguments: + -- table prefix + -- the route under consideration + -- table attributes + -- the route's attributes + -- table peer + -- the peer which received this route + -- integer RM_FAILURE + -- status code in case of failure + -- integer RM_NOMATCH + -- status code for no match + -- integer RM_MATCH + -- status code for match + -- integer RM_MATCH_AND_CHANGE + -- status code for match-and-set + -- + -- route_match returns table with following keys: + -- integer action, required + -- resultant status code. Should be one of RM_* + -- table attributes, optional + -- updated route attributes + -- + + function route_match(prefix, attributes, peer, + RM_FAILURE, RM_NOMATCH, RM_MATCH, RM_MATCH_AND_CHANGE) + + log.info("Evaluating route " .. prefix.network .. " from peer " .. peer.remote_id.string) + + function on_match (prefix, attributes) + log.info("Match") + return { + attributes = RM_MATCH + } + end + + function on_nomatch (prefix, attributes) + log.info("No match") + return { + action = RM_NOMATCH + } + end + + function on_match_and_change (prefix, attributes) + log.info("Match and change") + attributes["metric"] = attributes["metric"] + 7 + return { + action = RM_MATCH_AND_CHANGE, + attributes = attributes + } + end + + special_routes = { + ["172.16.10.4/24"] = on_match, + ["172.16.13.1/8"] = on_nomatch, + ["192.168.0.24/8"] = on_match_and_change, + } + + + if special_routes[prefix.network] then + return special_routes[prefix.network](prefix, attributes) + elseif peer.stats.update_in % 3 == 0 then + return on_match(prefix, attributes) + elseif peer.stats.update_in % 2 == 0 then + return on_nomatch(prefix, attributes) + else + return on_match_and_change(prefix, attributes) + end + end diff --git a/doc/developer/static-linking.rst b/doc/developer/static-linking.rst new file mode 100644 index 0000000..5342fbf --- /dev/null +++ b/doc/developer/static-linking.rst @@ -0,0 +1,98 @@ +.. _static-linking: + +Static Linking +============== + +This document describes how to build FRR without hard dependencies on shared +libraries. Note that it's not possible to build FRR *completely* statically. +This document just covers how to statically link the dependencies that aren't +likely to be present on a given platform - libfrr and libyang. The resultant +binaries should still be fairly portable. For example, here is the DSO +dependency list for `bgpd` after using these steps: + +.. code-block:: shell + + $ ldd bgpd + linux-vdso.so.1 (0x00007ffe3a989000) + libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9dc10c0000) + libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f9dc0eba000) + libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9dc0b1c000) + libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9dc0918000) + libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9dc06e0000) + libjson-c.so.3 => /lib/x86_64-linux-gnu/libjson-c.so.3 (0x00007f9dc04d5000) + librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9dc02cd000) + libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9dc00ae000) + libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9dbfe96000) + libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9dbfaa5000) + /lib64/ld-linux-x86-64.so.2 (0x00007f9dc1449000) + +Procedure +--------- +Note that these steps have only been tested with LLVM 9 / clang. + +Today, libfrr can already be statically linked by passing these configure +options:: + + --enable-static --enable-static-bin --enable-shared + +libyang is more complicated. You must build and install libyang as a static +library. To do this, follow the usual libyang build procedure as listed in the +FRR developer docs, but set the ``ENABLE_STATIC`` option in your cmake +invocation. You also need to build with PIC enabled, which today is disabled +when building libyang statically. + +The resultant cmake command is:: + + cmake -DENABLE_STATIC=ON -DENABLE_LYD_PRIV=ON \ + -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE \ + -DCMAKE_BUILD_TYPE:String="Release" .. + +This produces a bunch of ``.a`` static archives that need to ultimately be linked +into FRR. However, not only is it 6 archives rather than the usual ``libyang.so``, +you will now also need to link FRR with ``libpcre.a``. Ubuntu's ``libpcre3-dev`` +package provides this, but it hasn't been built with PIC enabled, so it's not +usable for our purposes. So download ``libpcre`` from +`SourceForge <https://sourceforge.net/projects/pcre/>`_, and build it +like this: + +.. code-block:: shell + + ./configure --with-pic + make + +Hopefully you get a nice, usable, PIC ``libpcre.a``. + +So now we have to link all these static libraries into FRR. Rather than modify +FRR to accommodate this, the best option is to create an archive with all of +libyang's dependencies. Then to avoid making any changes to FRR build foo, +rename this ``libyang.a`` and copy it over the usual static library location. +Ugly but it works. To do this, go into your libyang build directory, which +should have a bunch of ``.a`` files. Copy ``libpcre.a`` into this directory. +Write the following into a shell script and run it: + +.. code-block:: shell + + #!/bin/bash + ar -M <<EOM + CREATE libyang_fat.a + ADDLIB libyang.a + ADDLIB libyangdata.a + ADDLIB libmetadata.a + ADDLIB libnacm.a + ADDLIB libuser_inet_types.a + ADDLIB libuser_yang_types.a + ADDLIB libpcre.a + SAVE + END + EOM + ranlib libyang_fat.a + +``libyang_fat.a`` is your archive. Now copy this over your install +``libyang.a``, which on my machine is located at +``/usr/lib/x86_64-linux-gnu/libyang.a`` (try ``locate libyang.a`` if not). + +Now when you build FRR with the static options enabled as above, clang should +pick up the static libyang and link it, leaving you with FRR binaries that have +no hard DSO dependencies beyond common system libraries. To verify, run ``ldd`` +over the resultant binaries. diff --git a/doc/developer/subdir.am b/doc/developer/subdir.am new file mode 100644 index 0000000..0deb0f5 --- /dev/null +++ b/doc/developer/subdir.am @@ -0,0 +1,116 @@ +# +# doc/developer +# + +dev_RSTFILES = \ + doc/developer/bgp-typecodes.rst \ + doc/developer/bgpd.rst \ + doc/developer/building-frr-for-alpine.rst \ + doc/developer/building-frr-for-archlinux.rst \ + doc/developer/building-frr-for-centos6.rst \ + doc/developer/building-frr-for-centos7.rst \ + doc/developer/building-frr-for-debian8.rst \ + doc/developer/building-frr-for-debian9.rst \ + doc/developer/building-frr-for-debian12.rst \ + doc/developer/building-frr-for-fedora.rst \ + doc/developer/building-frr-for-freebsd10.rst \ + doc/developer/building-frr-for-freebsd11.rst \ + doc/developer/building-frr-for-freebsd13.rst \ + doc/developer/building-frr-for-freebsd9.rst \ + doc/developer/building-frr-for-netbsd6.rst \ + doc/developer/building-frr-for-netbsd7.rst \ + doc/developer/building-frr-for-openbsd6.rst \ + doc/developer/building-frr-for-opensuse.rst \ + doc/developer/building-frr-for-openwrt.rst \ + doc/developer/building-frr-for-ubuntu1404.rst \ + doc/developer/building-frr-for-ubuntu1604.rst \ + doc/developer/building-frr-for-ubuntu1804.rst \ + doc/developer/building-frr-for-ubuntu2004.rst \ + doc/developer/building-frr-for-ubuntu2204.rst \ + doc/developer/building-libunwind-note.rst \ + doc/developer/building-libyang.rst \ + doc/developer/building.rst \ + doc/developer/checkpatch.rst \ + doc/developer/cli.rst \ + doc/developer/conf.py \ + doc/developer/cross-compiling.rst \ + doc/developer/frr-release-procedure.rst \ + doc/developer/grpc.rst \ + doc/developer/hooks.rst \ + doc/developer/include-compile.rst \ + doc/developer/index.rst \ + doc/developer/library.rst \ + doc/developer/link-state.rst \ + doc/developer/lists.rst \ + doc/developer/locking.rst \ + doc/developer/logging.rst \ + doc/developer/memtypes.rst \ + doc/developer/modules.rst \ + doc/developer/next-hop-tracking.rst \ + doc/developer/ospf-api.rst \ + doc/developer/ospf-sr.rst \ + doc/developer/ospf.rst \ + doc/developer/packaging-debian.rst \ + doc/developer/packaging-redhat.rst \ + doc/developer/packaging.rst \ + doc/developer/path-internals-daemon.rst \ + doc/developer/path-internals-pcep.rst \ + doc/developer/path-internals.rst \ + doc/developer/path.rst \ + doc/developer/rcu.rst \ + doc/developer/scripting.rst \ + doc/developer/static-linking.rst \ + doc/developer/tracing.rst \ + doc/developer/testing.rst \ + doc/developer/topotests-snippets.rst \ + doc/developer/topotests-markers.rst \ + doc/developer/topotests.rst \ + doc/developer/workflow.rst \ + doc/developer/xrefs.rst \ + doc/developer/zebra.rst \ + doc/developer/northbound/advanced-topics.rst \ + doc/developer/northbound/architecture.rst \ + doc/developer/northbound/demos.rst \ + doc/developer/northbound/links.rst \ + doc/developer/northbound/northbound.rst \ + doc/developer/northbound/operational-data-rpcs-and-notifications.rst \ + doc/developer/northbound/plugins-sysrepo.rst \ + doc/developer/northbound/ppr-basic-test-topology.rst \ + doc/developer/northbound/ppr-mpls-basic-test-topology.rst \ + doc/developer/northbound/retrofitting-configuration-commands.rst \ + doc/developer/northbound/transactional-cli.rst \ + doc/developer/northbound/yang-module-translator.rst \ + doc/developer/northbound/yang-tools.rst \ + # end + +EXTRA_DIST += \ + $(dev_RSTFILES) \ + doc/developer/draft-zebra-00.ms \ + doc/developer/ldpd-basic-test-setup.md \ + doc/developer/release-announcement-template.md \ + doc/developer/_static/overrides.css \ + # end + +DEVBUILD = doc/developer/_build +$(DEVBUILD)/.doctrees/environment.pickle: $(dev_RSTFILES) + +# +# nothing built automatically for "all" target. +# + +# +# standard targets +# + +developer-info: $(DEVBUILD)/texinfo/frr.info +developer-html: $(DEVBUILD)/html/.buildinfo +developer-pdf: $(DEVBUILD)/latexpdf + +# +# hook-in for clean +# + +.PHONY: clean-devdocs +clean-local: clean-devdocs +clean-devdocs: + -rm -rf "$(DEVBUILD)" diff --git a/doc/developer/testing.rst b/doc/developer/testing.rst new file mode 100644 index 0000000..5865a6b --- /dev/null +++ b/doc/developer/testing.rst @@ -0,0 +1,11 @@ +.. _testing: + +******* +Testing +******* + +.. toctree:: + :maxdepth: 2 + + topotests + topotests-jsontopo diff --git a/doc/developer/topotests-jsontopo.rst b/doc/developer/topotests-jsontopo.rst new file mode 100644 index 0000000..e2cc72c --- /dev/null +++ b/doc/developer/topotests-jsontopo.rst @@ -0,0 +1,454 @@ +.. _topotests-json: + +Topotests with JSON +=================== + +Overview +-------- + +On top of current topotests framework following enhancements are done: + + +* Creating the topology and assigning IPs to router' interfaces dynamically. + It is achieved by using json file, in which user specify the number of + routers, links to each router, interfaces for the routers and protocol + configurations for all routers. + +* Creating the configurations dynamically. It is achieved by using + :file:`/usr/lib/frr/frr-reload.py` utility, which takes running configuration + and the newly created configuration for any particular router and creates a + delta file(diff file) and loads it to router. + + +Logging of test case executions +------------------------------- + +* The execution log for each test is saved in the test specific directory create + under `/tmp/topotests` (e.g., + `/tmp/topotests/<testdirname.testfilename>/exec.log`) + +* Additionally all test logs are captured in the `topotest.xml` results file. + This file will be saved in `/tmp/topotests/topotests.xml`. In order to extract + the logs for a particular test one can use the `analyze.py` utility found in + the topotests base directory. + +* Router's current configuration, as it is changed during the test, can be + displayed on console or sent to logs by adding ``show_router_config = True`` in + :file:`pytest.ini`. + +Note: directory "/tmp/topotests/" is created by topotests by default, making +use of same directory to save execution logs. + +Guidelines +---------- + +Writing New Tests +^^^^^^^^^^^^^^^^^ + +This section will guide you in all recommended steps to produce a standard +topology test. + +This is the recommended test writing routine: + +* Create a json file which will have routers and protocol configurations +* Write and debug the tests +* Format the new code using `black <https://github.com/psf/black>`_ +* Create a Pull Request + +.. Note:: + + BGP tests MUST use generous convergence timeouts - you must ensure that any + test involving BGP uses a convergence timeout that is proportional to the + configured BGP timers. If the timers are not reduced from their defaults this + means 130 seconds; however, it is highly recommended that timers be reduced + from the default values unless the test requires they not be. + +File Hierarchy +^^^^^^^^^^^^^^ + +Before starting to write any tests one must know the file hierarchy. The +repository hierarchy looks like this: + +.. code-block:: console + + $ cd frr/tests/topotests + $ find ./* + ... + ./example_test/ + ./example_test/test_template_json.json # input json file, having topology, interfaces, bgp and other configuration + ./example_test/test_template_json.py # test script to write and execute testcases + ... + ./lib # shared test/topology functions + ./lib/topojson.py # library to create topology and configurations dynamically from json file + ./lib/common_config.py # library to create protocol's common configurations ex- static_routes, prefix_lists, route_maps etc. + ./lib/bgp.py # library to create and test bgp configurations + +Defining the Topology and initial configuration in JSON file +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The first step to write a new test is to define the topology and initial +configuration. User has to define topology and initial configuration in JSON +file. Here is an example of JSON file:: + + BGP neighborship with single phy-link, sample JSON file: + { + "ipv4base": "192.168.0.0", + "ipv4mask": 30, + "ipv6base": "fd00::", + "ipv6mask": 64, + "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, + "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, + "routers": { + "r1": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, + "r2": {"ipv4": "auto", "ipv6": "auto"}, + "r3": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "neighbor": { + "r2": { + "dest_link": { + "r1": {} + } + }, + "r3": { + "dest_link": { + "r1": {} + } + } + } + } + } + } + } + }, + "r2": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, + "r1": {"ipv4": "auto", "ipv6": "auto"}, + "r3": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "redistribute": [ + { + "redist_type": "static" + } + ], + "neighbor": { + "r1": { + "dest_link": { + "r2": {} + } + }, + "r3": { + "dest_link": { + "r2": {} + } + } + } + } + } + } + } + } + ... + + +BGP neighboship with loopback interface, sample JSON file:: + + { + "ipv4base": "192.168.0.0", + "ipv4mask": 30, + "ipv6base": "fd00::", + "ipv6mask": 64, + "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, + "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, + "routers": { + "r1": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", + "add_static_route":"yes"}, + "r2": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "neighbor": { + "r2": { + "dest_link": { + "lo": { + "source_link": "lo" + } + } + } + } + } + } + } + }, + "static_routes": [ + { + "network": "1.0.2.17/32", + "next_hop": "192.168.0.1 + } + ] + }, + "r2": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", + "add_static_route":"yes"}, + "r1": {"ipv4": "auto", "ipv6": "auto"}, + "r3": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "redistribute": [ + { + "redist_type": "static" + } + ], + "neighbor": { + "r1": { + "dest_link": { + "lo": { + "source_link": "lo" + } + } + }, + "r3": { + "dest_link": { + "lo": { + "source_link": "lo" + } + } + } + } + } + } + } + }, + "static_routes": [ + { + "network": "192.0.20.1/32", + "no_of_ip": 9, + "admin_distance": 100, + "next_hop": "192.168.0.1", + "tag": 4001 + } + ], + } + ... + +BGP neighborship with Multiple phy-links, sample JSON file:: + + { + "ipv4base": "192.168.0.0", + "ipv4mask": 30, + "ipv6base": "fd00::", + "ipv6mask": 64, + "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, + "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, + "routers": { + "r1": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, + "r2-link1": {"ipv4": "auto", "ipv6": "auto"}, + "r2-link2": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "neighbor": { + "r2": { + "dest_link": { + "r1-link1": {} + } + } + } + } + } + } + } + }, + "r2": { + "links": { + "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, + "r1-link1": {"ipv4": "auto", "ipv6": "auto"}, + "r1-link2": {"ipv4": "auto", "ipv6": "auto"}, + "r3-link1": {"ipv4": "auto", "ipv6": "auto"}, + "r3-link2": {"ipv4": "auto", "ipv6": "auto"} + }, + "bgp": { + "local_as": "64512", + "address_family": { + "ipv4": { + "unicast": { + "redistribute": [ + { + "redist_type": "static" + } + ], + "neighbor": { + "r1": { + "dest_link": { + "r2-link1": {} + } + }, + "r3": { + "dest_link": { + "r2-link1": {} + } + } + } + } + } + } + } + } + ... + + +JSON File Explained +""""""""""""""""""" + +Mandatory keywords/options in JSON: + +* ``ipv4base`` : base ipv4 address to generate ips, ex - 192.168.0.0 +* ``ipv4mask`` : mask for ipv4 address, ex - 30 +* ``ipv6base`` : base ipv6 address to generate ips, ex - fd00: +* ``ipv6mask`` : mask for ipv6 address, ex - 64 +* ``link_ip_start`` : physical interface base ipv4 and ipv6 address +* ``lo_prefix`` : loopback interface base ipv4 and ipv6 address +* ``routers`` : user can add number of routers as per topology, router's name + can be any logical name, ex- r1 or a0. +* ``r1`` : name of the router +* ``lo`` : loopback interface dict, ipv4 and/or ipv6 addresses generated automatically +* ``type`` : type of interface, to identify loopback interface +* ``links`` : physical interfaces dict, ipv4 and/or ipv6 addresses generated + automatically +* ``r2-link1`` : it will be used when routers have multiple links. 'r2' is router + name, 'link' is any logical name, '1' is to identify link number, + router name and link must be seperated by hyphen (``-``), ex- a0-peer1 + +Optional keywords/options in JSON: + +* ``bgp`` : bgp configuration +* ``local_as`` : Local AS number +* ``unicast`` : All SAFI configuration +* ``neighbor``: All neighbor details +* ``dest_link`` : Destination link to which router will connect +* ``router_id`` : bgp router-id +* ``source_link`` : if user wants to establish bgp neighborship with loopback + interface, add ``source_link``: ``lo`` +* ``keepalivetimer`` : Keep alive timer for BGP neighbor +* ``holddowntimer`` : Hold down timer for BGP neighbor +* ``static_routes`` : create static routes for routers +* ``redistribute`` : redistribute static and/or connected routes +* ``prefix_lists`` : create Prefix-lists for routers + +Building topology and configurations +"""""""""""""""""""""""""""""""""""" + +Topology and initial configuration as well as teardown are invoked through the +use of a pytest fixture:: + + + from lib import fixtures + + tgen = pytest.fixture(fixtures.tgen_json, scope="module") + + + # tgen is defined above + # topo is a fixture defined in ../conftest.py and automatically available + def test_bgp_convergence(tgen, topo): + bgp_convergence = bgp.verify_bgp_convergence(tgen, topo) + assert bgp_convergence + +The `fixtures.topo_json` function calls `topojson.setup_module_from_json()` to +create and return a new `topogen.Topogen()` object using the JSON config file +with the same base filename as the test (i.e., `test_file.py` -> +`test_file.json`). Additionally, the fixture calls `tgen.stop_topology()` after +all the tests have run to cleanup. The function is only invoked once per +file/module (scope="module"), but the resulting object is passed to each +function that has `tgen` as an argument. + +For more info on the powerful pytest fixtures feature please see `FIXTURES`_. + +.. _FIXTURES: https://docs.pytest.org/en/6.2.x/fixture.html + +Creating configuration files +"""""""""""""""""""""""""""" + +Router's configuration would be saved in config file frr_json.conf. Common +configurations are like, static routes, prefixlists and route maps etc configs, +these configs can be used by any other protocols as it is. +BGP config will be specific to BGP protocol testing. + +* json file is passed to API Topogen() which saves the JSON object in + `self.json_topo` +* The Topogen object is then passed to API build_config_from_json(), which looks + for configuration tags in new JSON object. +* If tag is found in the JSON object, configuration is created as per input and + written to file frr_json.conf +* Once JSON parsing is over, frr_json.conf is loaded onto respective router. + Config loading is done using 'vtysh -f <file>'. Initial config at this point + is also saved frr_json_initial.conf. This file can be used to reset + configuration on router, during the course of execution. +* Reset of configuration is done using frr "reload.py" utility, which + calculates the difference between router's running config and user's config + and loads delta file to router. API used - reset_config_on_router() + +Writing Tests +""""""""""""" + +Test topologies should always be bootstrapped from the +`example_test/test_template_json.py` when possible in order to take advantage of +the most recent infrastructure support code. + +Example: + + +* Define a module scoped fixture to setup/teardown and supply the tests with the + `Topogen` object. + +.. code-block:: python + + import pytest + from lib import fixtures + + tgen = pytest.fixture(fixtures.tgen_json, scope="module") + + +* Define test functions using pytest fixtures + +.. code-block:: python + + from lib import bgp + + # tgen is defined above + # topo is a global available fixture defined in ../conftest.py + def test_bgp_convergence(tgen, topo): + "Test for BGP convergence." + + # Don't run this test if we have any failure. + if tgen.routers_have_failure(): + pytest.skip(tgen.errors) + + bgp_convergence = bgp.verify_bgp_convergence(tgen, topo) + assert bgp_convergence diff --git a/doc/developer/topotests-markers.rst b/doc/developer/topotests-markers.rst new file mode 100644 index 0000000..9f92412 --- /dev/null +++ b/doc/developer/topotests-markers.rst @@ -0,0 +1,114 @@ +.. _topotests-markers: + +Markers +-------- + +To allow for automated selective testing on large scale continuous integration +systems, all tests must be marked with at least one of the following markers: + +* babeld +* bfdd +* bgpd +* eigrpd +* isisd +* ldpd +* nhrpd +* ospf6d +* ospfd +* pathd +* pbrd +* pimd +* ripd +* ripngd +* sharpd +* staticd +* vrrpd + +The markers corespond to the daemon subdirectories in FRR's source code and have +to be added to tests on a module level depending on which daemons are used +during the test. + +The goal is to have continuous integration systems scan code submissions, detect +changes to files in a daemons subdirectory and select only tests using that +daemon to run to shorten developers waiting times for test results and save test +infrastructure resources. + +Newly written modules and code changes on tests, which do not contain any or +incorrect markers will be rejected by reviewers. + + +Registering markers +^^^^^^^^^^^^^^^^^^^ +The Registration of new markers takes place in the file +``tests/topotests/pytest.ini``: + +.. code:: python3 + + # tests/topotests/pytest.ini + [pytest] + ... + markers = + babeld: Tests that run against BABELD + bfdd: Tests that run against BFDD + ... + vrrpd: Tests that run against VRRPD + + +Adding markers to tests +^^^^^^^^^^^^^^^^^^^^^^^ +Markers are added to a test by placing a global variable in the test module. + +Adding a single marker: + +.. code:: python3 + + import pytest + ... + + # add after imports, before defining classes or functions: + pytestmark = pytest.mark.bfdd + + ... + + def test_using_bfdd(): + + +Adding multiple markers: + +.. code:: python3 + + import pytest + ... + + # add after imports, before defining classes or functions: + pytestmark = [ + pytest.mark.bgpd, + pytest.mark.ospfd, + pytest.mark.ospf6d + ] + + ... + + def test_using_bgpd_ospfd_ospf6d(): + + +Selecting marked modules for testing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Selecting by a single marker: + +.. code:: bash + + pytest -v -m isisd + +Selecting by multiple markers: + +.. code:: bash + + pytest -v -m "isisd or ldpd or nhrpd" + + +Further Information +^^^^^^^^^^^^^^^^^^^ +The `online pytest documentation <https://docs.pytest.org/en/stable/example/markers.html>`_ +provides further information and usage examples for pytest markers. + diff --git a/doc/developer/topotests-snippets.rst b/doc/developer/topotests-snippets.rst new file mode 100644 index 0000000..fb3c928 --- /dev/null +++ b/doc/developer/topotests-snippets.rst @@ -0,0 +1,272 @@ +.. _topotests-snippets: + +Snippets +-------- + +This document will describe common snippets of code that are frequently needed +to perform some test checks. + +Checking for router / test failures +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following check uses the topogen API to check for software failure (e.g. +zebra died) and/or for errors manually set by ``Topogen.set_error()``. + +.. code:: py + + # Get the topology reference + tgen = get_topogen() + + # Check for errors in the topology + if tgen.routers_have_failure(): + # Skip the test with the topology errors as reason + pytest.skip(tgen.errors) + +Checking FRR routers version +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This code snippet is usually run after the topology setup to make sure all +routers instantiated in the topology have the correct software version. + +.. code:: py + + # Get the topology reference + tgen = get_topogen() + + # Get the router list + router_list = tgen.routers() + + # Run the check for all routers + for router in router_list.values(): + if router.has_version('<', '3'): + # Set topology error, so the next tests are skipped + tgen.set_error('unsupported version') + +A sample of this snippet in a test can be found `here +<ldp-vpls-topo1/test_ldp_vpls_topo1.py>`__. + +Interacting with equipment +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You might want to interact with the topology equipment during the tests and +there are different ways to do so. + +Notes: + +1. When using the Topogen API, all the equipment code derives from ``Topogear`` + (`lib/topogen.py <lib/topogen.py>`__). If you feel brave you can look by + yourself how the abstractions that will be mentioned here work. + +2. When not using the ``Topogen`` API there is only one way to interact with + the equipment, which is by calling the ``mininet`` API functions directly + to spawn commands. + +Interacting with the Linux sandbox +"""""""""""""""""""""""""""""""""" + +Without ``Topogen``: + +.. code:: py + + global net + output = net['r1'].cmd('echo "foobar"') + print 'output is: {}'.format(output) + +With ``Topogen``: + +.. code:: py + + tgen = get_topogen() + output = tgen.gears['r1'].run('echo "foobar"') + print 'output is: {}'.format(output) + +Interacting with VTYSH +"""""""""""""""""""""" + +Without ``Topogen``: + +.. code:: py + + global net + output = net['r1'].cmd('vtysh "show ip route" 2>/dev/null') + print 'output is: {}'.format(output) + +With ``Topogen``: + +.. code:: py + + tgen = get_topogen() + output = tgen.gears['r1'].vtysh_cmd("show ip route") + print 'output is: {}'.format(output) + +``Topogen`` also supports sending multiple lines of command: + +.. code:: py + + tgen = get_topogen() + output = tgen.gears['r1'].vtysh_cmd(""" + configure terminal + router bgp 10 + bgp router-id 10.0.255.1 + neighbor 1.2.3.4 remote-as 10 + ! + router bgp 11 + bgp router-id 10.0.255.2 + ! + """) + print 'output is: {}'.format(output) + +You might also want to run multiple commands and get only the commands that +failed: + +.. code:: py + + tgen = get_topogen() + output = tgen.gears['r1'].vtysh_multicmd(""" + configure terminal + router bgp 10 + bgp router-id 10.0.255.1 + neighbor 1.2.3.4 remote-as 10 + ! + router bgp 11 + bgp router-id 10.0.255.2 + ! + """, pretty_output=false) + print 'output is: {}'.format(output) + +Translating vtysh JSON output into Python structures: + +.. code:: py + + tgen = get_topogen() + json_output = tgen.gears['r1'].vtysh_cmd("show ip route json", isjson=True) + output = json.dumps(json_output, indent=4) + print 'output is: {}'.format(output) + + # You can also access the data structure as normal. For example: + # protocol = json_output['1.1.1.1/32']['protocol'] + # assert protocol == "ospf", "wrong protocol" + +.. note:: + + ``vtysh_(multi)cmd`` is only available for router types of equipment. + +Invoking mininet CLI +^^^^^^^^^^^^^^^^^^^^ + +Without ``Topogen``: + +.. code:: py + + CLI(net) + +With ``Topogen``: + +.. code:: py + + tgen = get_topogen() + tgen.mininet_cli() + +Reading files +^^^^^^^^^^^^^ + +Loading a normal text file content in the current directory: + +.. code:: py + + # If you are using Topogen + # CURDIR = CWD + # + # Otherwise find the directory manually: + CURDIR = os.path.dirname(os.path.realpath(__file__)) + + file_name = '{}/r1/show_ip_route.txt'.format(CURDIR) + file_content = open(file_name).read() + +Loading JSON from a file: + +.. code:: py + + import json + + file_name = '{}/r1/show_ip_route.json'.format(CURDIR) + file_content = json.loads(open(file_name).read()) + +Comparing JSON output +^^^^^^^^^^^^^^^^^^^^^ + +After obtaining JSON output formatted with Python data structures, you may use +it to assert a minimalist schema: + +.. code:: py + + tgen = get_topogen() + json_output = tgen.gears['r1'].vtysh_cmd("show ip route json", isjson=True) + + expect = { + '1.1.1.1/32': { + 'protocol': 'ospf' + } + } + + assertmsg = "route 1.1.1.1/32 was not learned through OSPF" + assert json_cmp(json_output, expect) is None, assertmsg + +``json_cmp`` function description (it might be outdated, you can find the +latest description in the source code at +:file:`tests/topotests/lib/topotest.py` + +.. code:: text + + JSON compare function. Receives two parameters: + * `d1`: json value + * `d2`: json subset which we expect + + Returns `None` when all keys that `d1` has matches `d2`, + otherwise a string containing what failed. + + Note: key absence can be tested by adding a key with value `None`. + +Pausing execution +^^^^^^^^^^^^^^^^^ + +Preferably, choose the ``sleep`` function that ``topotest`` provides, as it +prints a notice during the test execution to help debug topology test execution +time. + +.. code:: py + + # Using the topotest sleep + from lib import topotest + + topotest.sleep(10, 'waiting 10 seconds for bla') + # or just tell it the time: + # topotest.sleep(10) + # It will print 'Sleeping for 10 seconds'. + + # Or you can also use the Python sleep, but it won't show anything + from time import sleep + sleep(5) + +iproute2 Linux commands as JSON +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +``topotest`` has two helpers implemented that parses the output of ``ip route`` +commands to JSON. It might simplify your comparison needs by only needing to +provide a Python dictionary. + +.. code:: py + + from lib import topotest + + tgen = get_topogen() + routes = topotest.ip4_route(tgen.gears['r1']) + expected = { + '10.0.1.0/24': {}, + '10.0.2.0/24': { + 'dev': 'r1-eth0' + } + } + + assertmsg = "failed to find 10.0.1.0/24 and/or 10.0.2.0/24" + assert json_cmp(routes, expected) is None, assertmsg diff --git a/doc/developer/topotests.rst b/doc/developer/topotests.rst new file mode 100644 index 0000000..b8f213b --- /dev/null +++ b/doc/developer/topotests.rst @@ -0,0 +1,1429 @@ +.. _topotests: + +Topotests +========= + +Topotests is a suite of topology tests for FRR built on top of micronet. + +Installation and Setup +---------------------- + +Topotests run under python3. Additionally, for ExaBGP (which is used +in some of the BGP tests) an older python2 version (and the python2 +version of ``pip``) must be installed. + +Tested with Ubuntu 20.04,Ubuntu 18.04, and Debian 11. + +Instructions are the same for all setups (i.e. ExaBGP is only used for +BGP tests). + +Tshark is only required if you enable any packet captures on test runs. + +Valgrind is only required if you enable valgrind on test runs. + +Installing Topotest Requirements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code:: shell + + apt-get install \ + gdb \ + iproute2 \ + net-tools \ + python3-pip \ + iputils-ping \ + tshark \ + valgrind + python3 -m pip install wheel + python3 -m pip install 'pytest>=6.2.4' + python3 -m pip install 'pytest-xdist>=2.3.0' + python3 -m pip install 'scapy>=2.4.5' + python3 -m pip install xmltodict + # Use python2 pip to install older ExaBGP + python2 -m pip install 'exabgp<4.0.0' + useradd -d /var/run/exabgp/ -s /bin/false exabgp + + # To enable the gRPC topotest install: + python3 -m pip install grpcio grpcio-tools + + +Enable Coredumps +"""""""""""""""" + +Optional, will give better output. + +.. code:: shell + + disable apport (which move core files) + +Set ``enabled=0`` in ``/etc/default/apport``. + +Next, update security limits by changing :file:`/etc/security/limits.conf` to:: + + #<domain> <type> <item> <value> + * soft core unlimited + root soft core unlimited + * hard core unlimited + root hard core unlimited + +Reboot for options to take effect. + +SNMP Utilities Installation +""""""""""""""""""""""""""" + +To run SNMP test you need to install SNMP utilities and MIBs. Unfortunately +there are some errors in the upstream MIBS which need to be patched up. The +following steps will get you there on Ubuntu 20.04. + +.. code:: shell + + apt install libsnmp-dev + apt install snmpd snmp + apt install snmp-mibs-downloader + download-mibs + wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/iana/IANA-IPPM-METRICS-REGISTRY-MIB -O /usr/share/snmp/mibs/iana/IANA-IPPM-METRICS-REGISTRY-MIB + wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/ietf/SNMPv2-PDU -O /usr/share/snmp/mibs/ietf/SNMPv2-PDU + wget https://raw.githubusercontent.com/FRRouting/frr-mibs/main/ietf/IPATM-IPMC-MIB -O /usr/share/snmp/mibs/ietf/IPATM-IPMC-MIB + edit /etc/snmp/snmp.conf to look like this + # As the snmp packages come without MIB files due to license reasons, loading + # of MIBs is disabled by default. If you added the MIBs you can reenable + # loading them by commenting out the following line. + mibs +ALL + + +FRR Installation +^^^^^^^^^^^^^^^^ + +FRR needs to be installed separately. It is assume to be configured like the +standard Ubuntu Packages: + +- Binaries in :file:`/usr/lib/frr` +- State Directory :file:`/var/run/frr` +- Running under user ``frr``, group ``frr`` +- vtygroup: ``frrvty`` +- config directory: :file:`/etc/frr` +- For FRR Packages, install the dbg package as well for coredump decoding + +No FRR config needs to be done and no FRR daemons should be run ahead of the +test. They are all started as part of the test. + +Manual FRR build +"""""""""""""""" + +If you prefer to manually build FRR, then use the following suggested config: + +.. code:: shell + + ./configure \ + --prefix=/usr \ + --localstatedir=/var/run/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-vtysh \ + --enable-pimd \ + --enable-pim6d \ + --enable-sharpd \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-snmp=agentx \ + --with-pkg-extra-version=-my-manual-build + +And create ``frr`` user and ``frrvty`` group as follows: + +.. code:: shell + + addgroup --system --gid 92 frr + addgroup --system --gid 85 frrvty + adduser --system --ingroup frr --home /var/run/frr/ \ + --gecos "FRRouting suite" --shell /bin/false frr + usermod -G frrvty frr + +Executing Tests +--------------- + +Configure your sudo environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Topotests must be run as root. Normally this will be accomplished through the +use of the ``sudo`` command. In order for topotests to be able to open new +windows (either XTerm or byobu/screen/tmux windows) certain environment +variables must be passed through the sudo command. One way to do this is to +specify the ``-E`` flag to ``sudo``. This will carry over most if not all +your environment variables include ``PATH``. For example: + +.. code:: shell + + sudo -E python3 -m pytest -s -v + +If you do not wish to use ``-E`` (e.g., to avoid ``sudo`` inheriting +``PATH``) you can modify your `/etc/sudoers` config file to specifically pass +the environment variables required by topotests. Add the following commands to +your ``/etc/sudoers`` config file. + +.. code:: shell + + Defaults env_keep="TMUX" + Defaults env_keep+="TMUX_PANE" + Defaults env_keep+="STY" + Defaults env_keep+="DISPLAY" + +If there was already an ``env_keep`` configuration there be sure to use the +``+=`` rather than ``=`` on the first line above as well. + + +Execute all tests in distributed test mode +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code:: shell + + sudo -E pytest -s -v -nauto --dist=loadfile + +The above command must be executed from inside the topotests directory. + +All test\_\* scripts in subdirectories are detected and executed (unless +disabled in ``pytest.ini`` file). Pytest will execute up to N tests in parallel +where N is based on the number of cores on the host. + +Analyze Test Results (``analyze.py``) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +By default router and execution logs are saved in ``/tmp/topotests`` and an XML +results file is saved in ``/tmp/topotests/topotests.xml``. An analysis tool +``analyze.py`` is provided to archive and analyze these results after the run +completes. + +After the test run completes one should pick an archive directory to store the +results in and pass this value to ``analyze.py``. On first execution the results +are moved to that directory from ``/tmp/topotests``. Subsequent runs of +``analyze.py`` with the same args will use that directories contents for instead +of copying any new results from ``/tmp``. Below is an example of this which also +shows the default behavior which is to display all failed and errored tests in +the run. + +.. code:: shell + + ~/frr/tests/topotests# ./analyze.py -Ar run-save + bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_converge + ospf_basic_functionality/test_ospf_lan.py::test_ospf_lan_tc1_p0 + bgp_gr_functionality_topo2/test_bgp_gr_functionality_topo2.py::test_BGP_GR_10_p2 + bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_routingTable + +Here we see that 4 tests have failed. We can dig deeper by displaying the +captured logs and errors. First let's redisplay the results enumerated by adding +the ``-E`` flag + +.. code:: shell + + ~/frr/tests/topotests# ./analyze.py -Ar run-save -E + 0 bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_converge + 1 ospf_basic_functionality/test_ospf_lan.py::test_ospf_lan_tc1_p0 + 2 bgp_gr_functionality_topo2/test_bgp_gr_functionality_topo2.py::test_BGP_GR_10_p2 + 3 bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_routingTable + +Now to look at the error message for a failed test we use ``-T N`` where N is +the number of the test we are interested in along with ``--errmsg`` option. + +.. code:: shell + + ~/frr/tests/topotests# ./analyze.py -Ar run-save -T0 --errmsg + bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_converge: AssertionError: BGP did not converge: + + IPv4 Unicast Summary (VIEW 1): + BGP router identifier 172.30.1.1, local AS number 100 vrf-id -1 + BGP table version 1 + RIB entries 1, using 184 bytes of memory + Peers 3, using 2169 KiB of memory + + Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc + 172.16.1.1 4 65001 0 0 0 0 0 never Connect 0 N/A + 172.16.1.2 4 65002 0 0 0 0 0 never Connect 0 N/A + 172.16.1.5 4 65005 0 0 0 0 0 never Connect 0 N/A + + Total number of neighbors 3 + + assert False + +Now to look at the error text for a failed test we can use ``-T RANGES`` where +``RANGES`` can be a number (e.g., ``5``), a range (e.g., ``0-10``), or a comma +separated list numbers and ranges (e.g., ``5,10-20,30``) of the test cases we +are interested in along with ``--errtext`` option. In the example below we'll +select the first failed test case. + +.. code:: shell + + ~/frr/tests/topotests# ./analyze.py -Ar run-save -T0 --errtext + bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_converge: def test_bgp_converge(): + "Check for BGP converged on all peers and BGP views" + + global fatal_error + global net + [...] + else: + # Bail out with error if a router fails to converge + bgpStatus = net["r%s" % i].cmd('vtysh -c "show ip bgp view %s summary"' % view) + > assert False, "BGP did not converge:\n%s" % bgpStatus + E AssertionError: BGP did not converge: + E + E IPv4 Unicast Summary (VIEW 1): + E BGP router identifier 172.30.1.1, local AS number 100 vrf-id -1 + [...] + E Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc + E 172.16.1.1 4 65001 0 0 0 0 0 never Connect 0 N/A + E 172.16.1.2 4 65002 0 0 0 0 0 never Connect 0 N/A + [...] + +To look at the full capture for a test including the stdout and stderr which +includes full debug logs, use ``--full`` option, or specify a ``-T RANGES`` without +specifying ``--errmsg`` or ``--errtext``. + +.. code:: shell + + ~/frr/tests/topotests# ./analyze.py -Ar run-save -T0 + @classname: bgp_multiview_topo1.test_bgp_multiview_topo1 + @name: test_bgp_converge + @time: 141.401 + @message: AssertionError: BGP did not converge: + [...] + system-out: --------------------------------- Captured Log --------------------------------- + 2021-08-09 02:55:06,581 DEBUG: lib.micronet_compat.topo: Topo(unnamed): Creating + 2021-08-09 02:55:06,581 DEBUG: lib.micronet_compat.topo: Topo(unnamed): addHost r1 + [...] + 2021-08-09 02:57:16,932 DEBUG: topolog.r1: LinuxNamespace(r1): cmd_status("['/bin/bash', '-c', 'vtysh -c "show ip bgp view 1 summary" 2> /dev/null | grep ^[0-9] | grep -vP " 11\\s+(\\d+)"']", kwargs: {'encoding': 'utf-8', 'stdout': -1, 'stderr': -2, 'shell': False}) + 2021-08-09 02:57:22,290 DEBUG: topolog.r1: LinuxNamespace(r1): cmd_status("['/bin/bash', '-c', 'vtysh -c "show ip bgp view 1 summary" 2> /dev/null | grep ^[0-9] | grep -vP " 11\\s+(\\d+)"']", kwargs: {'encoding': 'utf-8', 'stdout': -1, 'stderr': -2, 'shell': False}) + 2021-08-09 02:57:27,636 DEBUG: topolog.r1: LinuxNamespace(r1): cmd_status("['/bin/bash', '-c', 'vtysh -c "show ip bgp view 1 summary"']", kwargs: {'encoding': 'utf-8', 'stdout': -1, 'stderr': -2, 'shell': False}) + --------------------------------- Captured Out --------------------------------- + system-err: --------------------------------- Captured Err --------------------------------- + +Filtered results +"""""""""""""""" + +There are 4 types of test results, [e]rrored, [f]ailed, [p]assed, and +[s]kipped. One can select the set of results to show with the ``-S`` or +``--select`` flags along with the letters for each type (i.e., ``-S efps`` +would select all results). By default ``analyze.py`` will use ``-S ef`` (i.e., +[e]rrors and [f]ailures) unless the ``--search`` filter is given in which case +the default is to search all results (i.e., ``-S efps``). + +One can find all results which contain a ``REGEXP``. To filter results using a +regular expression use the ``--search REGEXP`` option. In this case, by default, +all result types will be searched for a match against the given ``REGEXP``. If a +test result output contains a match it is selected into the set of results to show. + +An example of using ``--search`` would be to search all tests results for some +log message, perhaps a warning or error. + +Using XML Results File from CI +"""""""""""""""""""""""""""""" + +``analyze.py`` actually only needs the ``topotests.xml`` file to run. This is +very useful for analyzing a CI run failure where one only need download the +``topotests.xml`` artifact from the run and then pass that to ``analyze.py`` +with the ``-r`` or ``--results`` option. + +For local runs if you wish to simply copy the ``topotests.xml`` file (leaving +the log files where they are), you can pass the ``-a`` (or ``--save-xml``) +instead of the ``-A`` (or ``-save``) options. + +Analyze Results from a Container Run +"""""""""""""""""""""""""""""""""""" + +``analyze.py`` can also be used with ``docker`` or ``podman`` containers. +Everything works exactly as with a host run except that you specify the name of +the container, or the container-id, using the `-C` or ``--container`` option. +``analyze.py`` will then use the results inside that containers +``/tmp/topotests`` directory. It will extract and save those results when you +pass the ``-A`` or ``-a`` options just as withe host results. + + +Execute single test +^^^^^^^^^^^^^^^^^^^ + +.. code:: shell + + cd test_to_be_run + sudo -E pytest ./test_to_be_run.py + +For example, and assuming you are inside the frr directory: + +.. code:: shell + + cd tests/topotests/bgp_l3vpn_to_bgp_vrf + sudo -E pytest ./test_bgp_l3vpn_to_bgp_vrf.py + +For further options, refer to pytest documentation. + +Test will set exit code which can be used with ``git bisect``. + +For the simulated topology, see the description in the python file. + +Running Topotests with AddressSanitizer +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Topotests can be run with AddressSanitizer. It requires GCC 4.8 or newer. +(Ubuntu 16.04 as suggested here is fine with GCC 5 as default). For more +information on AddressSanitizer, see +https://github.com/google/sanitizers/wiki/AddressSanitizer. + +The checks are done automatically in the library call of ``checkRouterRunning`` +(ie at beginning of tests when there is a check for all daemons running). No +changes or extra configuration for topotests is required beside compiling the +suite with AddressSanitizer enabled. + +If a daemon crashed, then the errorlog is checked for AddressSanitizer output. +If found, then this is added with context (calling test) to +:file:`/tmp/AddressSanitizer.txt` in Markdown compatible format. + +Compiling for GCC AddressSanitizer requires to use ``gcc`` as a linker as well +(instead of ``ld``). Here is a suggest way to compile frr with AddressSanitizer +for ``master`` branch: + +.. code:: shell + + git clone https://github.com/FRRouting/frr.git + cd frr + ./bootstrap.sh + ./configure \ + --enable-address-sanitizer \ + --prefix=/usr/lib/frr --sysconfdir=/etc/frr \ + --localstatedir=/var/run/frr \ + --sbindir=/usr/lib/frr --bindir=/usr/lib/frr \ + --with-moduledir=/usr/lib/frr/modules \ + --enable-multipath=0 --enable-rtadv \ + --enable-tcp-zebra --enable-fpm --enable-pimd \ + --enable-sharpd + make + sudo make install + # Create symlink for vtysh, so topotest finds it in /usr/lib/frr + sudo ln -s /usr/lib/frr/vtysh /usr/bin/ + +and create ``frr`` user and ``frrvty`` group as shown above. + +Debugging Topotest Failures +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Install and run tests inside ``tmux`` or ``byobu`` for best results. + +``XTerm`` is also fully supported. GNU ``screen`` can be used in most +situations; however, it does not work as well with launching ``vtysh`` or shell +on error. + +For the below debugging options which launch programs or CLIs, topotest should +be run within ``tmux`` (or ``screen``)_, as ``gdb``, the shell or ``vtysh`` will +be launched using that windowing program, otherwise ``xterm`` will be attempted +to launch the given programs. + +NOTE: you must run the topotest (pytest) such that your DISPLAY, STY or TMUX +environment variables are carried over. You can do this by passing the +``-E`` flag to ``sudo`` or you can modify your ``/etc/sudoers`` config to +automatically pass that environment variable through to the ``sudo`` +environment. + +.. _screen: https://www.gnu.org/software/screen/ +.. _tmux: https://github.com/tmux/tmux/wiki + +Capturing Packets +""""""""""""""""" + +One can view and capture packets on any of the networks or interfaces defined by +the topotest by specifying the ``--pcap=NET|INTF|all[,NET|INTF,...]`` CLI option +as shown in the examples below. + +.. code:: shell + + # Capture on all networks in isis_topo1 test + sudo -E pytest isis_topo1 --pcap=all + + # Capture on `sw1` network + sudo -E pytest isis_topo1 --pcap=sw1 + + # Capture on `sw1` network and on interface `eth0` on router `r2` + sudo -E pytest isis_topo1 --pcap=sw1,r2:r2-eth0 + +For each capture a window is opened displaying a live summary of the captured +packets. Additionally, the entire packet stream is captured in a pcap file in +the tests log directory e.g.,: + +.. code:: console + + $ sudo -E pytest isis_topo1 --pcap=sw1,r2:r2-eth0 + ... + $ ls -l /tmp/topotests/isis_topo1.test_isis_topo1/ + -rw------- 1 root root 45172 Apr 19 05:30 capture-r2-r2-eth0.pcap + -rw------- 1 root root 48412 Apr 19 05:30 capture-sw1.pcap + ... + +Viewing Live Daemon Logs +"""""""""""""""""""""""" + +One can live view daemon or the frr logs in separate windows using the +``--logd`` CLI option as shown below. + +.. code:: shell + + # View `ripd` logs on all routers in test + sudo -E pytest rip_allow_ecmp --logd=ripd + + # View `ripd` logs on all routers and `mgmtd` log on `r1` + sudo -E pytest rip_allow_ecmp --logd=ripd --logd=mgmtd,r1 + +For each capture a window is opened displaying a live summary of the captured +packets. Additionally, the entire packet stream is captured in a pcap file in +the tests log directory e.g., + +When using a unified log file ``frr.log`` one substitutes ``frr`` for the +daemon name in the ``--logd`` CLI option, e.g., + +.. code:: shell + + # View `frr` log on all routers in test + sudo -E pytest some_test_suite --logd=frr + +Spawning Debugging CLI, ``vtysh`` or Shells on Routers on Test Failure +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +One can have a debugging CLI invoked on test failures by specifying the +``--cli-on-error`` CLI option as shown in the example below. + +.. code:: shell + + sudo -E pytest --cli-on-error all-protocol-startup + +The debugging CLI can run shell or vtysh commands on any combination of routers +It can also open shells or vtysh in their own windows for any combination of +routers. This is usually the most useful option when debugging failures. Here is +the help command from within a CLI launched on error: + +.. code:: shell + + test_bgp_multiview_topo1/test_bgp_routingTable> help + + Basic Commands: + cli :: open a secondary CLI window + help :: this help + hosts :: list hosts + quit :: quit the cli + + HOST can be a host or one of the following: + - '*' for all hosts + - '.' for the parent munet + - a regex specified between '/' (e.g., '/rtr.*/') + + New Window Commands: + logd HOST [HOST ...] DAEMON :: tail -f on the logfile of the given DAEMON for the given HOST[S] + pcap NETWORK :: capture packets from NETWORK into file capture-NETWORK.pcap the command is run within a new window which also shows packet summaries. NETWORK can also be an interface specified as HOST:INTF. To capture inside the host namespace. + stderr HOST [HOST ...] DAEMON :: tail -f on the stderr of the given DAEMON for the given HOST[S] + stdlog HOST [HOST ...] :: tail -f on the `frr.log` for the given HOST[S] + stdout HOST [HOST ...] DAEMON :: tail -f on the stdout of the given DAEMON for the given HOST[S] + term HOST [HOST ...] :: open terminal[s] (TMUX or XTerm) on HOST[S], * for all + vtysh ROUTER [ROUTER ...] :: + xterm HOST [HOST ...] :: open XTerm[s] on HOST[S], * for all + Inline Commands: + [ROUTER ...] COMMAND :: execute vtysh COMMAND on the router[s] + [HOST ...] sh <SHELL-COMMAND> :: execute <SHELL-COMMAND> on hosts + [HOST ...] shi <INTERACTIVE-COMMAND> :: execute <INTERACTIVE-COMMAND> on HOST[s] + + test_bgp_multiview_topo1/test_bgp_routingTable> r1 show int br + ------ Host: r1 ------ + Interface Status VRF Addresses + --------- ------ --- --------- + erspan0 down default + gre0 down default + gretap0 down default + lo up default + r1-eth0 up default 172.16.1.254/24 + r1-stub up default 172.20.0.1/28 + + ---------------------- + test_bgp_multiview_topo1/test_bgp_routingTable> + +Additionally, one can have ``vtysh`` or a shell launched on all routers when a +test fails. To launch the given process on each router after a test failure +specify one of ``--shell-on-error`` or ``--vtysh-on-error``. + +Spawning ``vtysh`` or Shells on Routers +""""""""""""""""""""""""""""""""""""""" + +Topotest can automatically launch a shell or ``vtysh`` for any or all routers in +a test. This is enabled by specifying 1 of 2 CLI arguments ``--shell`` or +``--vtysh``. Both of these options can be set to a single router value, multiple +comma-seperated values, or ``all``. + +When either of these options are specified topotest will pause after setup and +each test to allow for inspection of the router state. + +Here's an example of launching ``vtysh`` on routers ``rt1`` and ``rt2``. + +.. code:: shell + + sudo -E pytest --vtysh=rt1,rt2 all-protocol-startup + +Debugging with GDB +"""""""""""""""""" + +Topotest can automatically launch any daemon with ``gdb``, possibly setting +breakpoints for any test run. This is enabled by specifying 1 or 2 CLI arguments +``--gdb-routers`` and ``--gdb-daemons``. Additionally ``--gdb-breakpoints`` can +be used to automatically set breakpoints in the launched ``gdb`` processes. + +Each of these options can be set to a single value, multiple comma-seperated +values, or ``all``. If ``--gdb-routers`` is empty but ``--gdb_daemons`` is set +then the given daemons will be launched in ``gdb`` on all routers in the test. +Likewise if ``--gdb_routers`` is set, but ``--gdb_daemons`` is empty then all +daemons on the given routers will be launched in ``gdb``. + +Here's an example of launching ``zebra`` and ``bgpd`` inside ``gdb`` on router +``r1`` with a breakpoint set on ``nb_config_diff`` + +.. code:: shell + + sudo -E pytest --gdb-routers=r1 \ + --gdb-daemons=bgpd,zebra \ + --gdb-breakpoints=nb_config_diff \ + all-protocol-startup + +Reporting Memleaks with FRR Memory Statistics +""""""""""""""""""""""""""""""""""""""""""""" + +FRR reports all allocated FRR memory objects on exit to standard error. +Topotest can be run to report such output as errors in order to check for +memleaks in FRR memory allocations. Specifying the CLI argument +``--memleaks`` will enable reporting FRR-based memory allocations at exit as errors. + +.. code:: shell + + sudo -E pytest --memleaks all-protocol-startup + + +StdErr log from daemos after exit +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When running with ``--memleaks``, to enable the reporting of other, +non-memory related, messages seen on StdErr after the daemons exit, +the following env variable can be set:: + + export TOPOTESTS_CHECK_STDERR=Yes + +(The value doesn't matter at this time. The check is whether the env +variable exists or not.) There is no pass/fail on this reporting; the +Output will be reported to the console. + +Collect Memory Leak Information +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When running with ``--memleaks``, FRR processes report unfreed memory +allocations upon exit. To enable also reporting of memory leaks to a specific +location, define an environment variable ``TOPOTESTS_CHECK_MEMLEAK`` with the +file prefix, i.e.: + +:: + + export TOPOTESTS_CHECK_MEMLEAK="/home/mydir/memleak_" + +For tests that support the TOPOTESTS_CHECK_MEMLEAK environment variable, this +will enable output to the information to files with the given prefix (followed +by testname), e.g.,: +file:`/home/mydir/memcheck_test_bgp_multiview_topo1.txt` in case +of a memory leak. + +Detecting Memleaks with Valgrind +"""""""""""""""""""""""""""""""" + +Topotest can automatically launch all daemons with ``valgrind`` to check for +memleaks. This is enabled by specifying 1 or 2 CLI arguments. +``--valgrind-memleaks`` will enable general memleak detection, and +``--valgrind-extra`` enables extra functionality including generating a +suppression file. The suppression file ``tools/valgrind.supp`` is used when +memleak detection is enabled. + +.. code:: shell + + sudo -E pytest --valgrind-memleaks all-protocol-startup + +Collecting Performance Data using perf(1) +""""""""""""""""""""""""""""""""""""""""" + +Topotest can automatically launch any daemon under ``perf(1)`` to collect +performance data. The daemon is run in non-daemon mode with ``perf record -g``. +The ``perf.data`` file will be saved in the router specific directory under the +tests run directoy. + +Here's an example of collecting performance data from ``mgmtd`` on router ``r1`` +during the config_timing test. + +.. code:: console + + $ sudo -E pytest --perf=mgmtd,r1 config_timing + ... + $ find /tmp/topotests/ -name '*perf.data*' + /tmp/topotests/config_timing.test_config_timing/r1/perf.data + +To specify different arguments for ``perf record``, one can use the +``--perf-options`` this will replace the ``-g`` used by default. + +.. _topotests_docker: + +Running Tests with Docker +------------------------- + +There is a Docker image which allows to run topotests. + +Quickstart +^^^^^^^^^^ + +If you have Docker installed, you can run the topotests in Docker. The easiest +way to do this, is to use the make targets from this repository. + +Your current user needs to have access to the Docker daemon. Alternatively you +can run these commands as root. + +.. code:: console + + make topotests + +This command will pull the most recent topotests image from Dockerhub, compile +FRR inside of it, and run the topotests. + +Advanced Usage +^^^^^^^^^^^^^^ + +Internally, the topotests make target uses a shell script to pull the image and +spawn the Docker container. + +There are several environment variables which can be used to modify the +behavior of the script, these can be listed by calling it with ``-h``: + +.. code:: console + + ./tests/topotests/docker/frr-topotests.sh -h + +For example, a volume is used to cache build artifacts between multiple runs of +the image. If you need to force a complete recompile, you can set +``TOPOTEST_CLEAN``: + +.. code:: console + + TOPOTEST_CLEAN=1 ./tests/topotests/docker/frr-topotests.sh + +By default, ``frr-topotests.sh`` will build frr and run pytest. If you append +arguments and the first one starts with ``/`` or ``./``, they will replace the +call to pytest. If the appended arguments do not match this patttern, they will +be provided to pytest as arguments. So, to run a specific test with more +verbose logging: + +.. code:: console + + ./tests/topotests/docker/frr-topotests.sh -vv -s all-protocol-startup/test_all_protocol_startup.py + +And to compile FRR but drop into a shell instead of running pytest: + +.. code:: console + + ./tests/topotests/docker/frr-topotests.sh /bin/bash + +Development +^^^^^^^^^^^ + +The Docker image just includes all the components to run the topotests, but not +the topotests themselves. So if you just want to write tests and don't want to +make changes to the environment provided by the Docker image. You don't need to +build your own Docker image if you do not want to. + +When developing new tests, there is one caveat though: The startup script of +the container will run a ``git-clean`` on its copy of the FRR tree to avoid any +pollution of the container with build artefacts from the host. This will also +result in your newly written tests being unavailable in the container unless at +least added to the index with ``git-add``. + +If you do want to test changes to the Docker image, you can locally build the +image and run the tests without pulling from the registry using the following +commands: + +.. code:: console + + make topotests-build + TOPOTEST_PULL=0 make topotests + + +.. _topotests-guidelines: + +Guidelines +---------- + +Executing Tests +^^^^^^^^^^^^^^^ + +To run the whole suite of tests the following commands must be executed at the +top level directory of topotest: + +.. code:: shell + + $ # Change to the top level directory of topotests. + $ cd path/to/topotests + $ # Tests must be run as root, since micronet requires it. + $ sudo -E pytest + +In order to run a specific test, you can use the following command: + +.. code:: shell + + $ # running a specific topology + $ sudo -E pytest ospf-topo1/ + $ # or inside the test folder + $ cd ospf-topo1 + $ sudo -E pytest # to run all tests inside the directory + $ sudo -E pytest test_ospf_topo1.py # to run a specific test + $ # or outside the test folder + $ cd .. + $ sudo -E pytest ospf-topo1/test_ospf_topo1.py # to run a specific one + +The output of the tested daemons will be available at the temporary folder of +your machine: + +.. code:: shell + + $ ls /tmp/topotest/ospf-topo1.test_ospf-topo1/r1 + ... + zebra.err # zebra stderr output + zebra.log # zebra log file + zebra.out # zebra stdout output + ... + +You can also run memory leak tests to get reports: + +.. code:: shell + + $ # Set the environment variable to apply to a specific test... + $ sudo -E env TOPOTESTS_CHECK_MEMLEAK="/tmp/memleak_report_" pytest ospf-topo1/test_ospf_topo1.py + $ # ...or apply to all tests adding this line to the configuration file + $ echo 'memleak_path = /tmp/memleak_report_' >> pytest.ini + $ # You can also use your editor + $ $EDITOR pytest.ini + $ # After running tests you should see your files: + $ ls /tmp/memleak_report_* + memleak_report_test_ospf_topo1.txt + +Writing a New Test +^^^^^^^^^^^^^^^^^^ + +This section will guide you in all recommended steps to produce a standard +topology test. + +This is the recommended test writing routine: + +- Write a topology (Graphviz recommended) +- Obtain configuration files +- Write the test itself +- Format the new code using `black <https://github.com/psf/black>`_ +- Create a Pull Request + +Some things to keep in mind: + +- BGP tests MUST use generous convergence timeouts - you must ensure + that any test involving BGP uses a convergence timeout of at least + 130 seconds. +- Topotests are run on a range of Linux versions: if your test + requires some OS-specific capability (like mpls support, or vrf + support), there are test functions available in the libraries that + will help you determine whether your test should run or be skipped. +- Avoid including unstable data in your test: don't rely on link-local + addresses or ifindex values, for example, because these can change + from run to run. +- Using sleep is almost never appropriate. As an example: if the test resets the + peers in BGP, the test should look for the peers re-converging instead of just + sleeping an arbitrary amount of time and continuing on. See + ``verify_bgp_convergence`` as a good example of this. In particular look at + it's use of the ``@retry`` decorator. If you are having troubles figuring out + what to look for, please do not be afraid to ask. +- Don't duplicate effort. There exists many protocol utility functions that can + be found in their eponymous module under ``tests/topotests/lib/`` (e.g., + ``ospf.py``) + + + +Topotest File Hierarchy +""""""""""""""""""""""" + +Before starting to write any tests one must know the file hierarchy. The +repository hierarchy looks like this: + +.. code:: shell + + $ cd path/to/topotest + $ find ./* + ... + ./README.md # repository read me + ./GUIDELINES.md # this file + ./conftest.py # test hooks - pytest related functions + ./example-test # example test folder + ./example-test/__init__.py # python package marker - must always exist. + ./example-test/test_template.jpg # generated topology picture - see next section + ./example-test/test_template.dot # Graphviz dot file + ./example-test/test_template.py # the topology plus the test + ... + ./ospf-topo1 # the ospf topology test + ./ospf-topo1/r1 # router 1 configuration files + ./ospf-topo1/r1/zebra.conf # zebra configuration file + ./ospf-topo1/r1/ospfd.conf # ospf configuration file + ./ospf-topo1/r1/ospfroute.txt # 'show ip ospf' output reference file + # removed other for shortness sake + ... + ./lib # shared test/topology functions + ./lib/topogen.py # topogen implementation + ./lib/topotest.py # topotest implementation + +Guidelines for creating/editing topotest: + +- New topologies that don't fit the existing directories should create its own +- Always remember to add the ``__init__.py`` to new folders, this makes auto + complete engines and pylint happy +- Router (Quagga/FRR) specific code should go on topotest.py +- Generic/repeated router actions should have an abstraction in + topogen.TopoRouter. +- Generic/repeated non-router code should go to topotest.py +- pytest related code should go to conftest.py (e.g. specialized asserts) + +Defining the Topology +""""""""""""""""""""" + +The first step to write a new test is to define the topology. This step can be +done in many ways, but the recommended is to use Graphviz to generate a drawing +of the topology. It allows us to see the topology graphically and to see the +names of equipment, links and addresses. + +Here is an example of Graphviz dot file that generates the template topology +:file:`tests/topotests/example-test/test_template.dot` (the inlined code might +get outdated, please see the linked file):: + + graph template { + label="template"; + + # Routers + r1 [ + shape=doubleoctagon, + label="r1", + fillcolor="#f08080", + style=filled, + ]; + r2 [ + shape=doubleoctagon, + label="r2", + fillcolor="#f08080", + style=filled, + ]; + + # Switches + s1 [ + shape=oval, + label="s1\n192.168.0.0/24", + fillcolor="#d0e0d0", + style=filled, + ]; + s2 [ + shape=oval, + label="s2\n192.168.1.0/24", + fillcolor="#d0e0d0", + style=filled, + ]; + + # Connections + r1 -- s1 [label="eth0\n.1"]; + + r1 -- s2 [label="eth1\n.100"]; + r2 -- s2 [label="eth0\n.1"]; + } + +Here is the produced graph: + +.. graphviz:: + + graph template { + label="template"; + + # Routers + r1 [ + shape=doubleoctagon, + label="r1", + fillcolor="#f08080", + style=filled, + ]; + r2 [ + shape=doubleoctagon, + label="r2", + fillcolor="#f08080", + style=filled, + ]; + + # Switches + s1 [ + shape=oval, + label="s1\n192.168.0.0/24", + fillcolor="#d0e0d0", + style=filled, + ]; + s2 [ + shape=oval, + label="s2\n192.168.1.0/24", + fillcolor="#d0e0d0", + style=filled, + ]; + + # Connections + r1 -- s1 [label="eth0\n.1"]; + + r1 -- s2 [label="eth1\n.100"]; + r2 -- s2 [label="eth0\n.1"]; + } + +Generating / Obtaining Configuration Files +"""""""""""""""""""""""""""""""""""""""""" + +In order to get the configuration files or command output for each router, we +need to run the topology and execute commands in ``vtysh``. The quickest way to +achieve that is writing the topology building code and running the topology. + +To bootstrap your test topology, do the following steps: + +- Copy the template test + +.. code:: shell + + $ mkdir new-topo/ + $ touch new-topo/__init__.py + $ cp example-test/test_template.py new-topo/test_new_topo.py + +- Modify the template according to your dot file + +Here is the template topology described in the previous section in python code: + +.. code:: py + + topodef = { + "s1": "r1" + "s2": ("r1", "r2") + } + +If more specialized topology definitions, or router initialization arguments are +required a build function can be used instead of a dictionary: + +.. code:: py + + def build_topo(tgen): + "Build function" + + # Create 2 routers + for routern in range(1, 3): + tgen.add_router("r{}".format(routern)) + + # Create a switch with just one router connected to it to simulate a + # empty network. + switch = tgen.add_switch("s1") + switch.add_link(tgen.gears["r1"]) + + # Create a connection between r1 and r2 + switch = tgen.add_switch("s2") + switch.add_link(tgen.gears["r1"]) + switch.add_link(tgen.gears["r2"]) + +- Run the topology + +Topogen allows us to run the topology without running any tests, you can do +that using the following example commands: + +.. code:: shell + + $ # Running your bootstraped topology + $ sudo -E pytest -s --topology-only new-topo/test_new_topo.py + $ # Running the test_template.py topology + $ sudo -E pytest -s --topology-only example-test/test_template.py + $ # Running the ospf_topo1.py topology + $ sudo -E pytest -s --topology-only ospf-topo1/test_ospf_topo1.py + +Parameters explanation: + +.. program:: pytest + +.. option:: -s + + Actives input/output capture. If this is not specified a new window will be + opened for the interactive CLI, otherwise it will be activated inline. + +.. option:: --topology-only + + Don't run any tests, just build the topology. + +After executing the commands above, you should get the following terminal +output: + +.. code:: shell + + frr/tests/topotests# sudo -E pytest -s --topology-only ospf_topo1/test_ospf_topo1.py + ============================= test session starts ============================== + platform linux -- Python 3.9.2, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 + rootdir: /home/chopps/w/frr/tests/topotests, configfile: pytest.ini + plugins: forked-1.3.0, xdist-2.3.0 + collected 11 items + + [...] + unet> + +The last line shows us that we are now using the CLI (Command Line +Interface), from here you can call your router ``vtysh`` or even bash. + +Here's the help text: + +.. code:: shell + + unet> help + + Commands: + help :: this help + sh [hosts] <shell-command> :: execute <shell-command> on <host> + term [hosts] :: open shell terminals for hosts + vtysh [hosts] :: open vtysh terminals for hosts + [hosts] <vtysh-command> :: execute vtysh-command on hosts + +Here are some commands example: + +.. code:: shell + + unet> sh r1 ping 10.0.3.1 + PING 10.0.3.1 (10.0.3.1) 56(84) bytes of data. + 64 bytes from 10.0.3.1: icmp_seq=1 ttl=64 time=0.576 ms + 64 bytes from 10.0.3.1: icmp_seq=2 ttl=64 time=0.083 ms + 64 bytes from 10.0.3.1: icmp_seq=3 ttl=64 time=0.088 ms + ^C + --- 10.0.3.1 ping statistics --- + 3 packets transmitted, 3 received, 0% packet loss, time 1998ms + rtt min/avg/max/mdev = 0.083/0.249/0.576/0.231 ms + + unet> r1 show run + Building configuration... + + Current configuration: + ! + frr version 8.1-dev-my-manual-build + frr defaults traditional + hostname r1 + log file /tmp/topotests/ospf_topo1.test_ospf_topo1/r1/zebra.log + [...] + end + + unet> show daemons + ------ Host: r1 ------ + zebra ospfd ospf6d staticd + ------- End: r1 ------ + ------ Host: r2 ------ + zebra ospfd ospf6d staticd + ------- End: r2 ------ + ------ Host: r3 ------ + zebra ospfd ospf6d staticd + ------- End: r3 ------ + ------ Host: r4 ------ + zebra ospfd ospf6d staticd + ------- End: r4 ------ + +After you successfully configured your topology, you can obtain the +configuration files (per-daemon) using the following commands: + +.. code:: shell + + unet> sh r3 vtysh -d ospfd + + Hello, this is FRRouting (version 3.1-devrzalamena-build). + Copyright 1996-2005 Kunihiro Ishiguro, et al. + + r1# show running-config + Building configuration... + + Current configuration: + ! + frr version 3.1-devrzalamena-build + frr defaults traditional + no service integrated-vtysh-config + ! + log file ospfd.log + ! + router ospf + ospf router-id 10.0.255.3 + redistribute kernel + redistribute connected + redistribute static + network 10.0.3.0/24 area 0 + network 10.0.10.0/24 area 0 + network 172.16.0.0/24 area 1 + ! + line vty + ! + end + r1# + +You can also login to the node specified by nsenter using bash, etc. +A pid file for each node will be created in the relevant test dir. +You can run scripts inside the node, or use vtysh's <tab> or <?> feature. + +.. code:: shell + + [unet shell] + # cd tests/topotests/srv6_locator + # ./test_srv6_locator.py --topology-only + unet> r1 show segment-routing srv6 locator + Locator: + Name ID Prefix Status + -------------------- ------- ------------------------ ------- + loc1 1 2001:db8:1:1::/64 Up + loc2 2 2001:db8:2:2::/64 Up + + [Another shell] + # nsenter -a -t $(cat /tmp/topotests/srv6_locator.test_srv6_locator/r1.pid) bash --norc + # vtysh + r1# r1 show segment-routing srv6 locator + Locator: + Name ID Prefix Status + -------------------- ------- ------------------------ ------- + loc1 1 2001:db8:1:1::/64 Up + loc2 2 2001:db8:2:2::/64 Up + +Writing Tests +""""""""""""" + +Test topologies should always be bootstrapped from +:file:`tests/topotests/example_test/test_template.py` because it contains +important boilerplate code that can't be avoided, like: + +Example: + +.. code:: py + + # For all routers arrange for: + # - starting zebra using config file from <rtrname>/zebra.conf + # - starting ospfd using an empty config file. + for rname, router in router_list.items(): + router.load_config(TopoRouter.RD_ZEBRA, "zebra.conf") + router.load_config(TopoRouter.RD_OSPF) + + +- The topology definition or build function + +.. code:: py + + topodef = { + "s1": ("r1", "r2"), + "s2": ("r2", "r3") + } + + def build_topo(tgen): + # topology build code + ... + +- pytest setup/teardown fixture to start the topology and supply ``tgen`` + argument to tests. + +.. code:: py + + + @pytest.fixture(scope="module") + def tgen(request): + "Setup/Teardown the environment and provide tgen argument to tests" + + tgen = Topogen(topodef, module.__name__) + # or + tgen = Topogen(build_topo, module.__name__) + + ... + + # Start and configure the router daemons + tgen.start_router() + + # Provide tgen as argument to each test function + yield tgen + + # Teardown after last test runs + tgen.stop_topology() + + +Requirements: + +- Directory name for a new topotest must not contain hyphen (``-``) characters. + To separate words, use underscores (``_``). For example, ``tests/topotests/bgp_new_example``. +- Test code should always be declared inside functions that begin with the + ``test_`` prefix. Functions beginning with different prefixes will not be run + by pytest. +- Configuration files and long output commands should go into separated files + inside folders named after the equipment. +- Tests must be able to run without any interaction. To make sure your test + conforms with this, run it without the :option:`-s` parameter. +- Use `black <https://github.com/psf/black>`_ code formatter before creating + a pull request. This ensures we have a unified code style. +- Mark test modules with pytest markers depending on the daemons used during the + tests (see :ref:`topotests-markers`) +- Always use IPv4 :rfc:`5737` (``192.0.2.0/24``, ``198.51.100.0/24``, + ``203.0.113.0/24``) and IPv6 :rfc:`3849` (``2001:db8::/32``) ranges reserved + for documentation. + +Tips: + +- Keep results in stack variables, so people inspecting code with ``pdb`` can + easily print their values. + +Don't do this: + +.. code:: py + + assert foobar(router1, router2) + +Do this instead: + +.. code:: py + + result = foobar(router1, router2) + assert result + +- Use ``assert`` messages to indicate where the test failed. + +Example: + +.. code:: py + + for router in router_list: + # ... + assert condition, 'Router "{}" condition failed'.format(router.name) + +Debugging Execution +^^^^^^^^^^^^^^^^^^^ + +The most effective ways to inspect topology tests are: + +- Run pytest with ``--pdb`` option. This option will cause a pdb shell to + appear when an assertion fails + +Example: ``pytest -s --pdb ospf-topo1/test_ospf_topo1.py`` + +- Set a breakpoint in the test code with ``pdb`` + +Example: + +.. code:: py + + # Add the pdb import at the beginning of the file + import pdb + # ... + + # Add a breakpoint where you think the problem is + def test_bla(): + # ... + pdb.set_trace() + # ... + +The `Python Debugger <https://docs.python.org/2.7/library/pdb.html>`__ (pdb) +shell allows us to run many useful operations like: + +- Setting breaking point on file/function/conditions (e.g. ``break``, + ``condition``) +- Inspecting variables (e.g. ``p`` (print), ``pp`` (pretty print)) +- Running python code + +.. tip:: + + The TopoGear (equipment abstraction class) implements the ``__str__`` method + that allows the user to inspect equipment information. + +Example of pdb usage: + +.. code:: shell + + > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(121)test_ospf_convergence() + -> for rnum in range(1, 5): + (Pdb) help + Documented commands (type help <topic>): + ======================================== + EOF bt cont enable jump pp run unt + a c continue exit l q s until + alias cl d h list quit step up + args clear debug help n r tbreak w + b commands disable ignore next restart u whatis + break condition down j p return unalias where + + Miscellaneous help topics: + ========================== + exec pdb + + Undocumented commands: + ====================== + retval rv + + (Pdb) list + 116 title2="Expected output") + 117 + 118 def test_ospf_convergence(): + 119 "Test OSPF daemon convergence" + 120 pdb.set_trace() + 121 -> for rnum in range(1, 5): + 122 router = 'r{}'.format(rnum) + 123 + 124 # Load expected results from the command + 125 reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) + 126 expected = open(reffile).read() + (Pdb) step + > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(122)test_ospf_convergence() + -> router = 'r{}'.format(rnum) + (Pdb) step + > /media/sf_src/topotests/ospf-topo1/test_ospf_topo1.py(125)test_ospf_convergence() + -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) + (Pdb) print rnum + 1 + (Pdb) print router + r1 + (Pdb) tgen = get_topogen() + (Pdb) pp tgen.gears[router] + <lib.topogen.TopoRouter object at 0x7f74e06c9850> + (Pdb) pp str(tgen.gears[router]) + 'TopoGear<name="r1",links=["r1-eth0"<->"s1-eth0","r1-eth1"<->"s3-eth0"]> TopoRouter<>' + (Pdb) l 125 + 120 pdb.set_trace() + 121 for rnum in range(1, 5): + 122 router = 'r{}'.format(rnum) + 123 + 124 # Load expected results from the command + 125 -> reffile = os.path.join(CWD, '{}/ospfroute.txt'.format(router)) + 126 expected = open(reffile).read() + 127 + 128 # Run test function until we get an result. Wait at most 60 seconds. + 129 test_func = partial(compare_show_ip_ospf, router, expected) + 130 result, diff = topotest.run_and_expect(test_func, '', + (Pdb) router1 = tgen.gears[router] + (Pdb) router1.vtysh_cmd('show ip ospf route') + '============ OSPF network routing table ============\r\nN 10.0.1.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth0\r\nN 10.0.2.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.3, r1-eth1\r\nN 10.0.3.0/24 [10] area: 0.0.0.0\r\n directly attached to r1-eth1\r\nN 10.0.10.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.0.0/24 [20] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\nN IA 172.16.1.0/24 [30] area: 0.0.0.0\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF router routing table =============\r\nR 10.0.255.2 [10] area: 0.0.0.0, ASBR\r\n via 10.0.3.3, r1-eth1\r\nR 10.0.255.3 [10] area: 0.0.0.0, ABR, ASBR\r\n via 10.0.3.1, r1-eth1\r\nR 10.0.255.4 IA [20] area: 0.0.0.0, ASBR\r\n via 10.0.3.1, r1-eth1\r\n\r\n============ OSPF external routing table ===========\r\n\r\n\r\n' + (Pdb) tgen.cli() + unet> + +To enable more debug messages in other Topogen subsystems, more +logging messages can be displayed by modifying the test configuration file +``pytest.ini``: + +.. code:: ini + + [topogen] + # Change the default verbosity line from 'info'... + #verbosity = info + # ...to 'debug' + verbosity = debug + +Instructions for use, write or debug topologies can be found in :ref:`topotests-guidelines`. +To learn/remember common code snippets see :ref:`topotests-snippets`. + +Before creating a new topology, make sure that there isn't one already that +does what you need. If nothing is similar, then you may create a new topology, +preferably, using the newest template +(:file:`tests/topotests/example-test/test_template.py`). + +.. include:: topotests-markers.rst + +.. include:: topotests-snippets.rst + +License +------- + +All the configs and scripts are licensed under a ISC-style license. See Python +scripts for details. diff --git a/doc/developer/tracing.rst b/doc/developer/tracing.rst new file mode 100644 index 0000000..76f6004 --- /dev/null +++ b/doc/developer/tracing.rst @@ -0,0 +1,411 @@ +.. _tracing: + +Tracing +======= + +FRR has a small but growing number of static tracepoints available for use with +various tracing systems. These tracepoints can assist with debugging, +performance analysis and to help understand program flow. They can also be used +for monitoring. + +Developers are encouraged to write new static tracepoints where sensible. They +are not compiled in by default, and even when they are, they have no overhead +unless enabled by a tracer, so it is okay to be liberal with them. + + +Supported tracers +----------------- + +Presently two types of tracepoints are supported: + +- `LTTng tracepoints <https://lttng.org/>`_ +- `USDT probes <http://dtrace.org/guide/chp-usdt.html>`_ + +LTTng is a tracing framework for Linux only. It offers extremely low overhead +and very rich tracing capabilities. FRR supports LTTng-UST, which is the +userspace implementation. LTTng tracepoints are very rich in detail. No kernel +modules are needed. Besides only being available for Linux, the primary +downside of LTTng is the need to link to ``lttng-ust``. + +USDT probes originate from Solaris, where they were invented for use with +dtrace. They are a kernel feature. At least Linux and FreeBSD support them. No +library is needed; support is compiled in via a system header +(``<sys/sdt.h>``). USDT probes are much slower than LTTng tracepoints and offer +less flexibility in what information can be gleaned from them. + +LTTng is capable of tracing USDT probes but has limited support for them. +SystemTap and dtrace both work only with USDT probes. + + +Usage +----- + +To compile with tracepoints, use one of the following configure flags: + +.. program:: configure.ac + +.. option:: --enable-lttng=yes + + Generate LTTng tracepoints + +.. option:: --enable-usdt=yes + + Generate USDT probes + +To trace with LTTng, compile with either one (prefer :option:`--enable-lttng` +run the target in non-forking mode (no ``-d``) and use LTTng as usual (refer to +LTTng user manual). When using USDT probes with LTTng, follow the example in +`this article +<https://lttng.org/blog/2019/10/15/new-dynamic-user-space-tracing-in-lttng/>`_. +To trace with dtrace or SystemTap, compile with `--enable-usdt=yes` and +use your tracer as usual. + +To see available USDT probes:: + + readelf -n /usr/lib/frr/bgpd + +Example:: + + root@host ~> readelf -n /usr/lib/frr/bgpd + + Displaying notes found in: .note.ABI-tag + Owner Data size Description + GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) + OS: Linux, ABI: 3.2.0 + + Displaying notes found in: .note.gnu.build-id + Owner Data size Description + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) + Build ID: 4f42933a69dcb42a519bc459b2105177c8adf55d + + Displaying notes found in: .note.stapsdt + Owner Data size Description + stapsdt 0x00000045 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: packet_read + Location: 0x000000000045ee48, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-96(%rbp) 8@-104(%rbp) + stapsdt 0x00000047 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: open_process + Location: 0x000000000047c43b, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-224(%rbp) 2@-226(%rbp) + stapsdt 0x00000049 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: update_process + Location: 0x000000000047c4bf, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-208(%rbp) 2@-210(%rbp) + stapsdt 0x0000004f NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: notification_process + Location: 0x000000000047c557, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-192(%rbp) 2@-194(%rbp) + stapsdt 0x0000004c NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: keepalive_process + Location: 0x000000000047c5db, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-176(%rbp) 2@-178(%rbp) + stapsdt 0x0000004a NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: refresh_process + Location: 0x000000000047c673, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-160(%rbp) 2@-162(%rbp) + stapsdt 0x0000004d NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: capability_process + Location: 0x000000000047c6f7, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 2@-146(%rbp) + stapsdt 0x0000006f NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: output_filter + Location: 0x000000000048e33a, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) + stapsdt 0x0000007d NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: process_update + Location: 0x0000000000491f10, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-800(%rbp) 8@-808(%rbp) 4@-812(%rbp) 4@-816(%rbp) 4@-820(%rbp) 8@-832(%rbp) + stapsdt 0x0000006e NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: input_filter + Location: 0x00000000004940ed, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) + + +To see available LTTng probes, run the target, create a session and then:: + + lttng list --userspace | grep frr + +Example:: + + root@host ~> lttng list --userspace | grep frr + PID: 11157 - Name: /usr/lib/frr/bgpd + frr_libfrr:route_node_get (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_sort (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_delete_node (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_remove (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_add (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:memfree (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:memalloc (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:frr_pthread_stop (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:frr_pthread_run (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:thread_call (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:event_cancel_async (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:event_cancel (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_write (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_read (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_event (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_timer (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_release (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_insert (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_get (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:output_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:input_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:process_update (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:packet_read (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:refresh_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:capability_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:notification_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:update_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:keepalive_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:open_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + +When using LTTng, you can also get zlogs as trace events by enabling +the ``lttng_ust_tracelog:*`` event class. + +To see available SystemTap USDT probes, run:: + + stap -L 'process("/usr/lib/frr/bgpd").mark("*")' + +Example:: + + root@host ~> stap -L 'process("/usr/lib/frr/bgpd").mark("*")' + process("/usr/lib/frr/bgpd").mark("capability_process") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("input_filter") $arg1:long $arg2:long $arg3:long $arg4:long $arg5:long + process("/usr/lib/frr/bgpd").mark("keepalive_process") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("notification_process") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("open_process") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("output_filter") $arg1:long $arg2:long $arg3:long $arg4:long $arg5:long + process("/usr/lib/frr/bgpd").mark("packet_read") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("process_update") $arg1:long $arg2:long $arg3:long $arg4:long $arg5:long $arg6:long + process("/usr/lib/frr/bgpd").mark("refresh_process") $arg1:long $arg2:long + process("/usr/lib/frr/bgpd").mark("update_process") $arg1:long $arg2:long + +When using SystemTap, you can also easily attach to an existing function:: + + stap -L 'process("/usr/lib/frr/bgpd").function("bgp_update_receive")' + +Example:: + + root@host ~> stap -L 'process("/usr/lib/frr/bgpd").function("bgp_update_receive")' + process("/usr/lib/frr/bgpd").function("bgp_update_receive@bgpd/bgp_packet.c:1531") $peer:struct peer* $size:bgp_size_t $attr:struct attr $restart:_Bool $nlris:struct bgp_nlri[] $__func__:char const[] const + +Complete ``bgp.stp`` example using SystemTap to show BGP peer, prefix and aspath +using ``process_update`` USDT:: + + global pkt_size; + probe begin + { + ansi_clear_screen(); + println("Starting..."); + } + probe process("/usr/lib/frr/bgpd").function("bgp_update_receive") + { + pkt_size <<< $size; + } + probe process("/usr/lib/frr/bgpd").mark("process_update") + { + aspath = @cast($arg6, "attr")->aspath; + printf("> %s via %s (%s)\n", + user_string($arg2), + user_string(@cast($arg1, "peer")->host), + user_string(@cast(aspath, "aspath")->str)); + } + probe end + { + if (@count(pkt_size)) + print(@hist_linear(pkt_size, 0, 20, 2)); + } + +Output:: + + Starting... + > 192.168.0.0/24 via 192.168.0.1 (65534) + > 192.168.100.1/32 via 192.168.0.1 (65534) + > 172.16.16.1/32 via 192.168.0.1 (65534 65030) + ^Cvalue |-------------------------------------------------- count + 0 | 0 + 2 | 0 + 4 |@ 1 + 6 | 0 + 8 | 0 + ~ + 18 | 0 + 20 | 0 + >20 |@@@@@ 5 + + +Concepts +-------- + +Tracepoints are statically defined points in code where a developer has +determined that outside observers might gain something from knowing what is +going on at that point. It's like logging but with the ability to dump large +amounts of internal data with much higher performance. LTTng has a good summary +`here <https://lttng.org/docs/#doc-what-is-tracing>`_. + +Each tracepoint has a "provider" and name. The provider is basically a +namespace; for example, ``bgpd`` uses the provider name ``frr_bgp``. The name +is arbitrary, but because providers share a global namespace on the user's +system, all providers from FRR should be prefixed by ``frr_``. The tracepoint +name is just the name of the event. Events are globally named by their provider +and name. For example, the event when BGP reads a packet from a peer is +``frr_bgp:packet_read``. + +To do tracing, the tracing tool of choice is told which events to listen to. +For example, to listen to all events from FRR's BGP implementation, you would +enable the events ``frr_bgp:*``. In the same tracing session you could also +choose to record all memory allocations by enabling the ``malloc`` tracepoints +in ``libc`` as well as all kernel skb operations using the various in-kernel +tracepoints. This allows you to build as complete a view as desired of what the +system is doing during the tracing window (subject to what tracepoints are +available). + +Of particular use are the tracepoints for FRR's internal event scheduler; +tracing these allows you to see all events executed by all event loops for the +target(s) in question. Here's a couple events selected from a trace of BGP +during startup:: + + ... + + [18:41:35.750131763] (+0.000048901) host frr_libfrr:thread_call: { cpu_id = + 1 }, { threadmaster_name = "default", function_name = "zclient_connect", + scheduled_from = "lib/zclient.c", scheduled_on_line = 3877, thread_addr = + 0x0, file_descriptor = 0, event_value = 0, argument_ptr = 0xA37F70, timer = + 0 } + + [18:41:35.750175124] (+0.000020001) host frr_libfrr:thread_call: { cpu_id = + 1 }, { threadmaster_name = "default", function_name = "frr_config_read_in", + scheduled_from = "lib/libfrr.c", scheduled_on_line = 934, thread_addr = 0x0, + file_descriptor = 0, event_value = 0, argument_ptr = 0x0, timer = 0 } + + [18:41:35.753341264] (+0.000010532) host frr_libfrr:thread_call: { cpu_id = + 1 }, { threadmaster_name = "default", function_name = "bgp_event", + scheduled_from = "bgpd/bgpd.c", scheduled_on_line = 142, thread_addr = 0x0, + file_descriptor = 2, event_value = 2, argument_ptr = 0xE4D780, timer = 2 } + + [18:41:35.753404186] (+0.000004910) host frr_libfrr:thread_call: { cpu_id = + 1 }, { threadmaster_name = "default", function_name = "zclient_read", + scheduled_from = "lib/zclient.c", scheduled_on_line = 3891, thread_addr = + 0x0, file_descriptor = 40, event_value = 40, argument_ptr = 0xA37F70, timer + = 40 } + + ... + + +Very useful for getting a time-ordered look into what the process is doing. + + +Adding Tracepoints +------------------ + +Adding new tracepoints is a two step process: + +1. Define the tracepoint +2. Use the tracepoint + +Tracepoint definitions state the "provider" and name of the tracepoint, along +with any values it will produce, and how to format them. This is done with +macros provided by LTTng. USDT probes do not use definitions and are inserted +at the trace site with a single macro. However, to maintain support for both +platforms, you must define an LTTng tracepoint when adding a new one. +``frrtrace()`` will expand to the appropriate ``DTRACE_PROBEn`` macro when USDT +is in use. + +If you are adding new tracepoints to a daemon that has no tracepoints, that +daemon's ``subdir.am`` must be updated to conditionally link ``lttng-ust``. +Look at ``bgpd/subdir.am`` for an example of how to do this; grep for +``UST_LIBS``. Create new files named ``<daemon>_trace.[ch]``. Use +``bgpd/bgp_trace.[h]`` as boilerplate. If you are adding tracepoints to a +daemon that already has them, look for the ``<daemon>_trace.h`` file; +tracepoints are written here. + +Refer to the `LTTng developer docs +<https://lttng.org/docs/#doc-c-application>`_ for details on how to define +tracepoints. + +To use them, simply add a call to ``frrtrace()`` at the point you'd like the +event to be emitted, like so: + +.. code-block:: c + + ... + + switch (type) { + case BGP_MSG_OPEN: + frrtrace(2, frr_bgp, open_process, peer, size); /* tracepoint */ + atomic_fetch_add_explicit(&peer->open_in, 1, + memory_order_relaxed); + mprc = bgp_open_receive(peer, size); + + ... + +After recompiling this tracepoint will now be available, either as a USDT probe +or LTTng tracepoint, depending on your compilation choice. + + +trace.h +^^^^^^^ + +Because FRR supports multiple types of tracepoints, the code for creating them +abstracts away the underlying system being used. This abstraction code is in +``lib/trace.h``. There are 2 function-like macros that are used for working +with tracepoints. + +- ``frrtrace()`` defines tracepoints +- ``frrtrace_enabled()`` checks whether a tracepoint is enabled + +There is also ``frrtracelog()``, which is used in zlog core code to make zlog +messages available as trace events to LTTng. This should not be used elsewhere. + +There is additional documentation in the header. The key thing to note is that +you should never include ``trace.h`` in source where you plan to put +tracepoints; include the tracepoint definition header instead (e.g. +:file:`bgp_trace.h`). + + +Limitations +----------- + +Tracers do not like ``fork()`` or ``dlopen()``. LTTng has some workarounds for +this involving interceptor libraries using ``LD_PRELOAD``. + +If you're running FRR in a typical daemonizing way (``-d`` to the daemons) +you'll need to run the daemons like so: + +.. code-block:: shell + + LD_PRELOAD=liblttng-ust-fork.so <daemon> + + +If you're using systemd this you can accomplish this for all daemons by +modifying ``frr.service`` like so: + +.. code-block:: diff + + --- a/frr.service + +++ b/frr.service + @@ -7,6 +7,7 @@ Before=network.target + OnFailure=heartbeat-failed@%n + + [Service] + +Environment="LD_PRELOAD=liblttng-ust-fork.so" + Nice=-5 + Type=forking + NotifyAccess=all + + +USDT tracepoints are relatively high overhead and probably shouldn't be used +for "flight recorder" functionality, i.e. enabling and passively recording all +events for monitoring purposes. It's generally okay to use LTTng like this, +though. diff --git a/doc/developer/vtysh.rst b/doc/developer/vtysh.rst new file mode 100644 index 0000000..323ea57 --- /dev/null +++ b/doc/developer/vtysh.rst @@ -0,0 +1,212 @@ +.. _vtysh: + +***** +VTYSH +***** + +.. seealso:: :ref:`command-line-interface` + +.. _vtysh-architecture: + +Architecture +============ + +VTYSH is a shell for FRR daemons. It amalgamates all the CLI commands defined +in each of the daemons and presents them to the user in a single shell, which +saves the user from having to telnet to each of the daemons and use their +individual shells. The amalgamation is achieved by +:ref:`extracting <vtysh-command-extraction>` commands from daemons and +injecting them into VTYSH at build time. + +At runtime, VTYSH maintains an instance of a CLI mode tree just like each +daemon. However, the mode tree in VTYSH contains (almost) all commands from +every daemon in the same tree, whereas individual daemons have trees that only +contain commands relevant to themselves. VTYSH also uses the library CLI +facilities to maintain the user's current position in the tree (the current +node). Note that this position must be synchronized with all daemons; if a +daemon receives a command that causes it to change its current node, VTYSH must +also change its node. Since the extraction script does not understand the +handler code of commands, but only their definitions, this and other behaviors +must be manually programmed into VTYSH for every case where the internal state +of VTYSH must change in response to a command. Details on how this is done are +discussed in the :ref:`vtysh-special-defuns` section. + +VTYSH also handles writing and applying the integrated configuration file, +:file:`/etc/frr/frr.conf`. Since it has knowledge of the entire command space +of FRR, it can intelligently distribute configuration commands only to the +daemons that understand them. Similarly, when writing the configuration file it +takes care of combining multiple instances of configuration blocks and +simplifying the output. This is discussed in :ref:`vtysh-configuration`. + +.. _vtysh-command-extraction: + +Command Extraction +------------------ + +To build ``vtysh``, the :file:`python/xref2vtysh.py` script scans through the +:file:`frr.xref` file created earlier in the build process. This file contains +a list of all ``DEFUN`` and ``install_element`` sites in the code, generated +directly from the binaries (and therefore matching exactly what is really +available.) + +This list is collated and transformed into ``DEFSH`` (and ``install_element``) +statements, output to ``vtysh_cmd.c``. Each ``DEFSH`` +contains the name of the command plus ``_vtysh``, as well as a flag that +indicates which daemons the command was found in. When the command is executed +in VTYSH, this flag is inspected to determine which daemons to send the command +to. This way, commands are only sent to the daemons that know about them, +avoiding spurious errors from daemons that don't have the command defined. + +The extraction script contains lots of hardcoded knowledge about what sources +to look at and what flags to use for certain commands. + +.. note:: + + The ``vtysh_scan`` Makefile variable and ``#ifndef VTYSH_EXTRACT_PL`` + checks in source files are no longer used. Remove them when rebasing older + changes. + +.. _vtysh-special-defuns: + +Special DEFUNs +-------------- + +In addition to the vanilla ``DEFUN`` macro for defining CLI commands, there are +several VTYSH-specific ``DEFUN`` variants that each serve different purposes. + +``DEFSH`` + Used almost exclusively by generated VTYSH code. This macro defines a + ``cmd_element`` with no handler function; the command, when executed, is + simply forwarded to the daemons indicated in the daemon flag. + +``DEFUN_NOSH`` + Used by daemons. Has the same expansion as a ``DEFUN``, but ``xref2vtysh.py`` + will skip these definitions when extracting commands. This is typically used + when VTYSH must take some special action upon receiving the command, and the + programmer therefore needs to write VTYSH's copy of the command manually + instead of using the generated version. + +``DEFUNSH`` + The same as ``DEFUN``, but with an argument that allows specifying the + ``->daemon`` field of the generated ``cmd_element``. This is used by VTYSH + to determine which daemons to send the command to. + +``DEFUNSH_ATTR`` + A version of ``DEFUNSH`` that allows setting the ``->attr`` field of the + generated ``cmd_element``. Not used in practice. + +.. _vtysh-configuration: + +Configuration Management +------------------------ + +When integrated configuration is used, VTYSH manages writing, reading and +applying the FRR configuration file. VTYSH can be made to read and apply an +integrated configuration to all running daemons by launching it with ``-f +<file>``. It sends the appropriate configuration lines to the relevant daemons +in the same way that commands entered by the user on VTYSH's shell prompt are +processed. + +Configuration writing is more complicated. VTYSH makes a best-effort attempt to +combine and simplify the configuration as much as possible. A working example +is best to explain this behavior. + +Example +^^^^^^^ + +Suppose we have just *staticd* and *zebra* running on the system, and use VTYSH +to apply the following configuration snippet: + +.. code-block:: frr + + ! + vrf blue + ip protocol static route-map ExampleRoutemap + ip route 192.168.0.0/24 192.168.0.1 + exit-vrf + ! + +Note that *staticd* defines static route commands and *zebra* defines ``ip +protocol`` commands. Therefore if we ask only *zebra* for its configuration, we +get the following:: + + (config)# do sh running-config zebra + Building configuration... + + ... + ! + vrf blue + ip protocol static route-map ExampleRoutemap + exit-vrf + ! + ... + +Note that the static route doesn't show up there. Similarly, if we ask +*staticd* for its configuration, we get:: + + (config)# do sh running-config staticd + + ... + ! + vrf blue + ip route 192.168.0.0/24 192.168.0.1 + exit-vrf + ! + ... + +But when we display the configuration with VTYSH, we see:: + + ubuntu-bionic(config)# do sh running-config + + ... + ! + vrf blue + ip protocol static route-map ExampleRoutemap + ip route 192.168.0.0/24 192.168.0.1 + exit-vrf + ! + ... + +This is because VTYSH asks each daemon for its currently running configuration, +and combines equivalent blocks together. In the above example, it combined the +``vrf blue`` blocks from both *zebra* and *staticd* together into one. This is +done in :file:`vtysh_config.c`. + +Protocol +======== + +VTYSH communicates with FRR daemons by way of domain socket. Each daemon +creates its own socket, typically in :file:`/var/run/frr/<daemon>.vty`. The +protocol is very simple. In the VTYSH to daemon direction, messages are simply +NUL-terminated strings, whose content are CLI commands. Here is a typical +message from VTYSH to a daemon: + +:: + + Request + + 00000000: 646f 2077 7269 7465 2074 6572 6d69 6e61 do write termina + 00000010: 6c0a 00 l.. + + +The response format has some more data in it. First is a NUL-terminated string +containing the plaintext response, which is just the output of the command that +was sent in the request. This is displayed to the user. The plaintext response +is followed by 3 null marker bytes, followed by a 1-byte status code that +indicates whether the command was successful or not. + +:: + + Response + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Plaintext Response | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Marker (0x00) | Status Code | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +The first ``0x00`` byte in the marker also serves to terminate the plaintext +response. diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst new file mode 100644 index 0000000..68834ed --- /dev/null +++ b/doc/developer/workflow.rst @@ -0,0 +1,1740 @@ +.. _process-and-workflow: + +******************* +Process & Workflow +******************* + +.. highlight:: none + +FRR is a large project developed by many different groups. This section +documents standards for code style & quality, commit messages, pull requests +and best practices that all contributors are asked to follow. + +This chapter is "descriptive/post-factual" in that it documents pratices that +are in use; it is not "definitive/pre-factual" in prescribing practices. This +means that when a procedure changes, it is agreed upon, then put into practice, +and then documented here. If this document doesn't match reality, it's the +document that needs to be updated, not reality. + +Mailing Lists +============= + +The FRR development group maintains multiple mailing lists for use by the +community. Italicized lists are private. + ++----------------------------------+--------------------------------+ +| Topic | List | ++==================================+================================+ +| Development | dev@lists.frrouting.org | ++----------------------------------+--------------------------------+ +| Users & Operators | frog@lists.frrouting.org | ++----------------------------------+--------------------------------+ +| Announcements | announce@lists.frrouting.org | ++----------------------------------+--------------------------------+ +| *Security* | security@lists.frrouting.org | ++----------------------------------+--------------------------------+ +| *Technical Steering Committee* | tsc@lists.frrouting.org | ++----------------------------------+--------------------------------+ + +The Development list is used to discuss and document general issues related to +project development and governance. The public +`Slack instance <https://frrouting.slack.com>`_ and weekly technical meetings +provide a higher bandwidth channel for discussions. The results of such +discussions must be reflected in updates, as appropriate, to code (i.e., +merges), `GitHub issues`_, and for governance or process changes, updates to +the Development list and either this file or information posted at +https://frrouting.org/. + +Development & Release Cycle +=========================== + +Development +----------- + +.. figure:: ../figures/git_branches.png + :align: center + :scale: 55% + :alt: Merging Git branches into a central trunk + + Rough outline of FRR development workflow + +The master Git for FRR resides on `GitHub`_. + +There is one main branch for development, ``master``. For each major release +(2.0, 3.0 etc) a new release branch is created based on the master. Significant +bugfixes should be backported to upcoming and existing release branches no more +than 1 year old. As a general rule new features are not backported to release +branches. + +Subsequent point releases based on a major branch are handled with git tags. + +Releases +-------- +FRR employs a ``<MAJOR>.<MINOR>.<BUGFIX>`` versioning scheme. + +``MAJOR`` + Significant new features or multiple minor features. This should mostly + cover any kind of disruptive change that is visible or "risky" to operators. + New features or protocols do not necessarily trigger this. (This was changed + for FRR 7.x after feedback from users that the pace of major version number + increments was too high.) + +``MINOR`` + General incremental development releases, excluding "major" changes + mentioned above. Not necessarily fully backwards compatible, as smaller + (but still visible) changes or deprecated feature removals may still happen. + However, there shouldn't be any huge "surprises" between minor releases. + +``BUGFIX`` + Fixes for actual bugs and/or security issues. Fully compatible. + +Releases are scheduled in a 4-month cycle on the first Tuesday each +March/July/November. Walking backwards from this date: + + - 6 weeks earlier, ``master`` is frozen for new features, and feature PRs + are considered lowest priority (regardless of when they were opened.) + + - 4 weeks earlier, the stable branch separates from master (named + ``dev/MAJOR.MINOR`` at this point) and tagged as ``base_X.Y``. + Master is unfrozen and new features may again proceed. + + Part of unfreezing master is editing the ``AC_INIT`` statement in + :file:`configure.ac` to reflect the new development version that master + now refers to. This is accompanied by a ``frr-X.Y-dev`` tag on master, + which should always be on the first commit on master *after* the stable + branch was forked (even if that is not the edit to ``AC_INIT``; it's more + important to have it on the very first commit on master after the fork.) + + (The :file:`configure.ac` edit and tag push are considered git housekeeping + and are pushed directly to ``master``, not through a PR.) + + Below is the snippet of the commands to use in this step. + + .. code-block:: console + + % git remote --verbose + upstream git@github.com:frrouting/frr (fetch) + upstream git@github.com:frrouting/frr (push) + + % git checkout master + % git pull upstream master + % git checkout -b dev/8.2 + % git tag base_8.2 + % git push upstream base_8.2 + % git push upstream dev/8.2 + % git checkout master + % sed -i 's/8.2-dev/8.3-dev/' configure.ac + % git add configure.ac + % git commit -s -m "build: FRR 8.3 development version" + % git tag -a frr-8.3-dev -m "frr-8.3-dev" + % git push upstream master + % git push upstream frr-8.3-dev + + In this step, we also have to update package versions to reflect + the development version. Versions need to be updated using + a standard way of development (Pull Requests) based on master branch. + + Only change the version number with no other changes. This will produce + packages with the a version number that is higher than any previous + version. Once the release is done, whatever updates we make to changelog + files on the release branch need to be cherry-picked to the master branch. + + Update essential dates in advance for reference table (below) when + the next freeze, dev/X.Y, RC, and release phases are scheduled. This should + go in the ``master`` branch. + + - 2 weeks earlier, a ``frr-X.Y-rc`` release candidate is tagged. + + .. code-block:: console + + % git remote --verbose + upstream git@github.com:frrouting/frr (fetch) + upstream git@github.com:frrouting/frr (push) + + % git checkout dev/8.2 + % git tag frr-8.2-rc + % git push upstream frr-8.2-rc + + - on release date, the branch is renamed to ``stable/MAJOR.MINOR``. + +The 2 week window between each of these events should be used to run any and +all testing possible for the release in progress. However, the current +intention is to stick to the schedule even if known issues remain. This would +hopefully occur only after all avenues of fixing issues are exhausted, but to +achieve this, an as exhaustive as possible list of issues needs to be available +as early as possible, i.e. the first 2-week window. + +For reference, the expected release schedule according to the above is: + ++---------+------------+------------+------------+ +| Release | 2023-11-07 | 2024-03-05 | 2024-07-02 | ++---------+------------+------------+------------+ +| RC | 2023-10-24 | 2024-02-20 | 2024-06-18 | ++---------+------------+------------+------------+ +| dev/X.Y | 2023-10-10 | 2024-02-06 | 2024-06-04 | ++---------+------------+------------+------------+ +| freeze | 2023-09-26 | 2024-01-23 | 2024-05-21 | ++---------+------------+------------+------------+ + +Here is the hint on how to get the dates easily: + + .. code-block:: console + + ~$ # Release date is 2023-11-07 (First Tuesday each March/July/November) + ~$ date +%F --date='2023-11-07 -42 days' # Next freeze date + 2023-09-26 + ~$ date +%F --date='2023-11-07 -28 days' # Next dev/X.Y date + 2023-10-10 + ~$ date +%F --date='2023-11-07 -14 days' # Next RC date + 2023-10-24 + +Each release is managed by one or more volunteer release managers from the FRR +community. These release managers are expected to handle the branch for a period +of one year. To spread and distribute this workload, this should be rotated for +subsequent releases. The release managers are currently assumed/expected to +run a release management meeting during the weeks listed above. Barring other +constraints, this would be scheduled before the regular weekly FRR community +call such that important items can be carried over into that call. + +Bugfixes are applied to the two most recent releases. It is expected that +each bugfix backported should include some reasoning for its inclusion +as well as receiving approval by the release managers for that release before +accepted into the release branch. This does not necessarily preclude backporting of +bug fixes to older than the two most recent releases. + +Security fixes are backported to all releases less than or equal to at least one +year old. Security fixes may also be backported to older releases depending on +severity. + +For detailed instructions on how to produce an FRR release, refer to +:ref:`frr-release-procedure`. + + +Long term support branches ( LTS ) +----------------------------------------- + +This kind of branch is not yet officially supported, and need experimentation +before being effective. + +Previous definition of releases prevents long term support of previous releases. +For instance, bug and security fixes are not applied if the stable branch is too +old. + +Because the FRR users have a need to backport bug and security fixes after the +stable branch becomes too old, there is a need to provide support on a long term +basis on that stable branch. If that support is applied on that stable branch, +then that branch is a long term support branch. + +Having a LTS branch requires extra-work and requires one person to be in charge +of that maintenance branch for a certain amount of time. The amount of time will +be by default set to 4 months, and can be increased. 4 months stands for the time +between two releases, this time can be applied to the decision to continue with a +LTS release or not. In all cases, that time period will be well-defined and +published. Also, a self nomination from a person that proposes to handle the LTS +branch is required. The work can be shared by multiple people. In all cases, there +must be at least one person that is in charge of the maintenance branch. The person +on people responsible for a maintenance branch must be a FRR maintainer. Note that +they may choose to abandon support for the maintenance branch at any time. If +no one takes over the responsibility of the LTS branch, then the support will be +discontinued. + +The LTS branch duties are the following ones: + +- organise meetings on a (bi-)weekly or monthly basis, the handling of issues + and pull requested relative to that branch. When time permits, this may be done + during the regularly scheduled FRR meeting. + +- ensure the stability of the branch, by using and eventually adapting the + checking the CI tools of FRR ( indeed, maintaining may lead to create + maintenance branches for topotests or for CI). + +It will not be possible to backport feature requests to LTS branches. Actually, it +is a false good idea to use LTS for that need. Introducing feature requests may +break the paradigm where all more recent releases should also include the feature +request. This would require the LTS maintainer to ensure that all more recent +releases have support for this feature request. Moreover, introducing features +requests may result in breaking the stability of the branch. LTS branches are first +done to bring long term support for stability. + +Development Branches +-------------------- + +Occassionally the community will desire the ability to work together +on a feature that is considered useful to FRR. In this case the +parties may ask the Maintainers for the creation of a development +branch in the main FRR repository. Requirements for this to happen +are: + +- A one paragraph description of the feature being implemented to + allow for the facilitation of discussion about the feature. This + might include pointers to relevant RFC's or presentations that + explain what is planned. This is intended to set a somewhat + low bar for organization. +- A branch maintainer must be named. This person is responsible for + keeping the branch up to date, and general communication about the + project with the other FRR Maintainers. Additionally this person + must already be a FRR Maintainer. +- Commits to this branch must follow the normal PR and commit process + as outlined in other areas of this document. The goal of this is + to prevent the current state where large features are submitted + and are so large they are difficult to review. + +After a development branch has completed the work together, a final +review can be made and the branch merged into master. If a development +branch is becomes un-maintained or not being actively worked on after +three months then the Maintainers can decide to remove the branch. + +Debian Branches +--------------- + +The Debian project contains "official" packages for FRR. While FRR +Maintainers may participate in creating these, it is entirely the Debian +project's decision what to ship and how to work on this. + +As a courtesy and for FRR's benefit, this packaging work is currently visible +in git branches named ``debian/*`` on the main FRR git repository. These +branches are for the exclusive use by people involved in Debian packaging work +for FRR. Direct commit access may be handed out and FRR git rules (review, +testing, etc.) do not apply. Do not push to these branches without talking +to the people noted under ``Maintainer:`` and ``Uploaders:`` in +``debian/control`` on the target branch -- even if you are a FRR Maintainer. + +Changelog +--------- +The changelog will be the base for the release notes. A changelog entry for +your changes is usually not required and will be added based on your commit +messages by the maintainers. However, you are free to include an update to the +changelog with some better description. + +Accords: non-code community consensus +===================================== + +The FRR repository has a place for "accords" - these are items of +consideration for FRR that influence how we work as a community, but either +haven't resulted in code *yet*, or may *never* result in code being written. +They are placed in the ``doc/accords/`` directory. + +The general idea is to simply pass small blurbs of text through our normal PR +procedures, giving them the same visibility, comment and review mechanisms as +code PRs - and changing them later is another PR. Please refer to the README +file in ``doc/accords/`` for further details. The file names of items in that +directory are hopefully helpful in determining whether some of them might be +relevant to your work. + +Submitting Patches and Enhancements +=================================== + +FRR accepts patches using GitHub pull requests. + +The base branch for new contributions and non-critical bug fixes should be +``master``. Please ensure your pull request is based on this branch when you +submit it. + +Code submitted by pull request will be automatically tested by one or more CI +systems. Once the automated tests succeed, other developers will review your +code for quality and correctness. After any concerns are resolved, your code +will be merged into the branch it was submitted against. + +The title of the pull request should provide a high level technical +summary of the included patches. The description should provide +additional details that will help the reviewer to understand the context +of the included patches. + +Squash commits +-------------- + +Before merging make sure a PR has squashed the following kinds of commits: + +- Fixes/review feedback +- Typos +- Merges and rebases +- Work in progress + +This helps to automatically generate human-readable changelog messages. + +Commit Guidelines +----------------- + +There is a built-in commit linter. Basic rules: + +- Commit messages must be prefixed with the name of the changed subsystem, followed + by a colon and a space and start with an imperative verb. + + `Check <https://github.com/FRRouting/frr/tree/master/.github/commitlint.config.js>`_ all + the supported subsystems. + +- Commit messages must not end with a period ``.`` + +Why was my pull request closed? +------------------------------- + +Pull requests older than 180 days will be closed. Exceptions can be made for +pull requests that have active review comments, or that are awaiting other +dependent pull requests. Closed pull requests are easy to recreate, and little +work is lost by closing a pull request that subsequently needs to be reopened. + +We want to limit the total number of pull requests in flight to: + +- Maintain a clean project +- Remove old pull requests that would be difficult to rebase as the underlying code has changed over time +- Encourage code velocity + +.. _license-for-contributions: + +License for Contributions +------------------------- +FRR is under a “GPLv2 or later” license. Any code submitted must be released +under the same license (preferred) or any license which allows redistribution +under this GPLv2 license (eg MIT License). +It is forbidden to push any code that prevents from using GPLv3 license. This +becomes a community rule, as FRR produces binaries that links with Apache 2.0 +libraries. Apache 2.0 and GPLv2 license are incompatible, if put together. +Please see `<http://www.apache.org/licenses/GPL-compatibility.html>`_ for +more information. This rule guarantees the user to distribute FRR binary code +without any licensing issues. + +Pre-submission Checklist +------------------------ +- Format code (see `Code Formatting <#code-formatting>`__) +- Verify and acknowledge license (see :ref:`license-for-contributions`) +- Ensure you have properly signed off (see :ref:`signing-off`) +- Test building with various configurations: + + - ``buildtest.sh`` + +- Verify building source distribution: + + - ``make dist`` (and try rebuilding from the resulting tar file) + +- Run unit tests: + + - ``make test`` + +- In the case of a major new feature or other significant change, document + plans for continued maintenance of the feature. In addition it is a + requirement that automated testing must be written that exercises + the new feature within our existing CI infrastructure. Also the + addition of automated testing to cover any pull request is encouraged. + +- All new code must use the current latest version of acceptable code. + + - If a daemon is converted to YANG, then new code must use YANG. + - DEFPY's must be used for new cli + - Typesafe lists must be used + - printf formatting changes must be used + +.. _signing-off: + +Signing Off +----------- +Code submitted to FRR must be signed off. We have the same requirements for +using the signed-off-by process as the Linux kernel. In short, you must include +a ``Signed-off-by`` tag in every patch. + +An easy way to do this is to use ``git commit -s`` where ``-s`` will automatically +append a signed-off line to the end of your commit message. Also, if you commit +and forgot to add the line you can use ``git commit --amend -s`` to add the +signed-off line to the last commit. + +``Signed-off-by`` is a developer's certification that they have the right to +submit the patch for inclusion into the project. It is an agreement to the +:ref:`Developer's Certificate of Origin <developers-certificate-of-origin>`. +Code without a proper ``Signed-off-by`` line cannot and will not be merged. + +If you are unfamiliar with this process, you should read the +`official policy at kernel.org <https://www.kernel.org/doc/html/latest/process/submitting-patches.html>`_. +You might also find +`this article <http://www.linuxfoundation.org/content/how-participate-linux-community-0>`_ +about participating in the Linux community on the Linux Foundation website to +be a helpful resource. + +.. _developers-certificate-of-origin: + +In short, when you sign off on a commit, you assert your agreement to all of +the following:: + + Developer's Certificate of Origin 1.1 + + By making a contribution to this project, I certify that: + + (a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + + (b) The contribution is based upon previous work that, to the best + of my knowledge, is covered under an appropriate open source + license and I have the right under that license to submit that + work with modifications, whether created in whole or in part by + me, under the same open source license (unless I am permitted to + submit under a different license), as indicated in the file; or + + (c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified it. + + (d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including all + personal information I submit with it, including my sign-off) is + maintained indefinitely and may be redistributed consistent with + this project or the open source license(s) involved. + +After Submitting Your Changes +----------------------------- + +- Watch for Continuous Integration (CI) test results + + - You should automatically receive an email with the test results + within less than 2 hrs of the submission. If you don’t get the + email, then check status on the GitHub pull request. + - Please notify the development mailing list if you think something + doesn't work. + +- If the tests failed: + + - In general, expect the community to ignore the submission until + the tests pass. + - It is up to you to fix and resubmit. + + - This includes fixing existing unit (“make test”) tests if your + changes broke or changed them. + - It also includes fixing distribution packages for the failing + platforms (ie if new libraries are required). + - Feel free to ask for help on the development list. + + - Go back to the submission process and repeat until the tests pass. + +- If the tests pass: + + - Wait for reviewers. Someone will review your code or be assigned + to review your code. + - Respond to any comments or concerns the reviewer has. Use e-mail or + add a comment via github to respond or to let the reviewer know how + their comment or concern is addressed. + - An author must never delete or manually dismiss someone else's comments + or review. (A review may be overridden by agreement in the weekly + technical meeting.) + - When you have addressed someone's review comments, please click the + "re-request review" button (in the top-right corner of the PR page, next + to the reviewer's name, an icon that looks like "reload") + - The responsibility for keeping a PR moving rests with the author at + least as long as there are either negative CI results or negative review + comments. If you forget to mark a review comment as addressed (by + clicking re-request review), the reviewer may very well not notice and + won't come back to your PR. + - Automatically generated comments, e.g., those generated by CI systems, + may be deleted by authors and others when such comments are not the most + recent results from that automated comment source. + - After all comments and concerns are addressed, expect your patch + to be merged. + +- Watch out for questions on the mailing list. At this time there will + be a manual code review and further (longer) tests by various + community members. +- Your submission is done once it is merged to the master branch. + +Programming Languages, Tools and Libraries +========================================== + +The core of FRR is written in C (gcc or clang supported) and makes +use of GNU compiler extensions. Additionally, the CLI generation +tool, `clippy`, requires Python. A few other non-essential scripts are +implemented in Perl and Python. FRR requires the following tools +to build distribution packages: automake, autoconf, texinfo, libtool and +gawk and various libraries (i.e. libpam and libjson-c). + +If your contribution requires a new library or other tool, then please +highlight this in your description of the change. Also make sure it’s +supported by all FRR platform OSes or provide a way to build +without the library (potentially without the new feature) on the other +platforms. + +Documentation should be written in reStructuredText. Sphinx extensions may be +utilized but pure ReST is preferred where possible. See +:ref:`documentation`. + +Use of C++ +---------- + +While C++ is not accepted for core components of FRR, extensions, modules or +other distinct components may want to use C++ and include FRR header files. +There is no requirement on contributors to work to retain C++ compatibility, +but fixes for C++ compatibility are welcome. + +This implies that the burden of work to keep C++ compatibility is placed with +the people who need it, and they may provide it at their leisure to the extent +it is useful to them. So, if only a subset of header files, or even parts of +a header file are made available to C++, this is perfectly fine. + +Code Reviews +============ + +Code quality is paramount for any large program. Consequently we require +reviews of all submitted patches by at least one person other than the +submitter before the patch is merged. + +Because of the nature of the software, FRR's maintainer list (i.e. those with +commit permissions) tends to contain employees / members of various +organizations. In order to prevent conflicts of interest, we use an honor +system in which submissions from an individual representing one company should +be merged by someone unaffiliated with that company. + +Guidelines for code review +-------------------------- + +- As a rule of thumb, the depth of the review should be proportional to the + scope and / or impact of the patch. + +- Anyone may review a patch. + +- When using GitHub reviews, marking "Approve" on a code review indicates + willingness to merge the PR. + +- For individuals with merge rights, marking "Changes requested" is equivalent + to a NAK. + +- For a PR you marked with "Changes requested", please respond to updates in a + timely manner to avoid impeding the flow of development. + +- Rejected or obsolete PRs are generally closed by the submitter based + on requests and/or agreement captured in a PR comment. The comment + may originate with a reviewer or document agreement reached on Slack, + the Development mailing list, or the weekly technical meeting. + +- Reviewers may ask for new automated testing if they feel that the + code change is large enough/significant enough to warrant such + a requirement. + +For project members with merge permissions, the following patterns have +emerged: + +- a PR with any reviews requesting changes may not be merged. + +- a PR with any negative CI result may not be merged. + +- an open "yellow" review mark ("review requested, but not done") should be + given some time (a few days up to weeks, depending on the size of the PR), + but is not a merge blocker. + +- a "textbubble" review mark ("review comments, but not positive/negative") + should be read through but is not a merge blocker. + +- non-trivial PRs are generally given some time (again depending on the size) + for people to mark an interest in reviewing. Trivial PRs may be merged + immediately when CI is green. + + +Coding Practices & Style +======================== + +Commit messages +--------------- + +Commit messages should be formatted in the same way as Linux kernel +commit messages. The format is roughly:: + + dir: short summary + + extended summary + +``dir`` should be the top level source directory under which the change was +made. For example, a change in :file:`bgpd/rfapi` would be formatted as:: + + bgpd: short summary + + ... + +The first line should be no longer than 50 characters. Subsequent lines should +be wrapped to 72 characters. + +The purpose of commit messages is to briefly summarize what the commit is +changing. Therefore, the extended summary portion should be in the form of an +English paragraph. Brief examples of program output are acceptable but if +present should be short (on the order of 10 lines) and clearly demonstrate what +has changed. The goal should be that someone with only passing familiarity with +the code in question can understand what is being changed. + +Commit messages consisting entirely of program output are *unacceptable*. These +do not describe the behavior changed. For example, putting VTYSH output or the +result of test runs as the sole content of commit messages is unacceptable. + +You must also sign off on your commit. + +.. seealso:: :ref:`signing-off` + + +Source File Header +------------------ + +New files must have a copyright header (see :ref:`license-for-contributions` +above) added to the file. The header should be: + +.. code-block:: c + + // SPDX-License-Identifier: GPL-2.0-or-later + /* + * Title/Function of file + * Copyright (C) YEAR Author’s Name + */ + + #include <zebra.h> + +A ``SPDX-License-Identifier`` header is required in all source files, i.e. +``.c``, ``.h``, ``.cpp`` and ``.py`` files. The license boilerplate should be +removed in these files. Some existing files are missing this header, this is +slowly being fixed. + +A ``SPDX-License-Identifier`` header *and* the full license boilerplate is +required in schema definition files, i.e. ``.yang`` and ``.proto``. The +rationale for this is that these files are likely to be individually copied to +places outside FRR, and having only the SPDX header would become a "dangling +pointer". + +.. warning:: + + **DO NOT REMOVE A "Copyright" LINE OR AUTHOR NAME, EVER.** + + **DO NOT APPLY AN SPDX HEADER WHEN THE LICENSE IS UNCLEAR, UNLESS YOU HAVE + CHECKED WITH *ALL* SIGNIFICANT AUTHORS.** + +Please to keep ``#include <zebra.h>``. The absolute first header included in +any C file **must** be either ``zebra.h`` or ``config.h`` (with HAVE_CONFIG_H +guard.) + + +Adding Copyright Claims to Existing Files +----------------------------------------- + +When adding copyright claims for modifications to an existing file, please +add a ``Portions:`` section as shown below. If this section already exists, add +your new claim at the end of the list. + +.. code-block:: c + + /* + * Title/Function of file + * Copyright (C) YEAR Author’s Name + * Portions: + * Copyright (C) 2010 Entity A .... + * Copyright (C) 2016 Your name [optional brief change description] + * ... + */ + +Defensive coding requirements +----------------------------- + +In general, code submitted into FRR will be rejected if it uses unsafe +programming practices. While there is no enforced overall ruleset, the +following requirements have achieved consensus: + +- ``strcpy``, ``strcat`` and ``sprintf`` are unacceptable without exception. + Use ``strlcpy``, ``strlcat`` and ``snprintf`` instead. (Rationale: even if + you know the operation cannot overflow the buffer, a future code change may + inadvertedly introduce an overflow.) + +- buffer size arguments, particularly to ``strlcpy`` and ``snprintf``, must + use ``sizeof()`` whereever possible. Particularly, do not use a size + constant in these cases. (Rationale: changing a buffer to another size + constant may leave the write operations on a now-incorrect size limit.) + +- For stack allocated structs and arrays that should be zero initialized, + prefer initializer expressions over ``memset()`` wherever possible. This + helps prevent ``memset()`` calls being missed in branches, and eliminates the + error class of an incorrect ``size`` argument to ``memset()``. + + For example, instead of: + + .. code-block:: c + + struct foo mystruct; + ... + memset(&mystruct, 0x00, sizeof(struct foo)); + + Prefer: + + .. code-block:: c + + struct foo mystruct = {}; + +- Do not zero initialize stack allocated values that must be initialized with a + nonzero value in order to be used. This way the compiler and memory checking + tools can catch uninitialized value use that would otherwise be suppressed by + the (incorrect) zero initialization. + +- Usage of ``system()`` or other c library routines that cause signals to + possibly be ignored are not allowed. This includes the ``fork()`` and + ``execXX`` call patterns, which is actually what system() does underneath + the covers. This pattern causes the system shutdown to never work properly + as the SIGINT sent is never received. It is better to just prohibit code + that does this instead of having to debug shutdown issues again. + +Other than these specific rules, coding practices from the Linux kernel as +well as CERT or MISRA C guidelines may provide useful input on safe C code. +However, these rules are not applied as-is; some of them expressly collide +with established practice. + + +Container implementations +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In particular to gain defensive coding benefits from better compiler type +checks, there is a set of replacement container data structures to be found +in :file:`lib/typesafe.h`. They're documented under :ref:`lists`. + +Unfortunately, the FRR codebase is quite large, and migrating existing code to +use these new structures is a tedious and far-reaching process (even if it +can be automated with coccinelle, the patches would touch whole swaths of code +and create tons of merge conflicts for ongoing work.) Therefore, little +existing code has been migrated. + +However, both **new code and refactors of existing code should use the new +containers**. If there are any reasons this can't be done, please work to +remove these reasons (e.g. by adding necessary features to the new containers) +rather than falling back to the old code. + +In order of likelyhood of removal, these are the old containers: + +- :file:`nhrpd/list.*`, ``hlist_*`` ⇒ ``DECLARE_LIST`` +- :file:`nhrpd/list.*`, ``list_*`` ⇒ ``DECLARE_DLIST`` +- :file:`lib/skiplist.*`, ``skiplist_*`` ⇒ ``DECLARE_SKIPLIST`` +- :file:`lib/*_queue.h` (BSD), ``SLIST_*`` ⇒ ``DECLARE_LIST`` +- :file:`lib/*_queue.h` (BSD), ``LIST_*`` ⇒ ``DECLARE_DLIST`` +- :file:`lib/*_queue.h` (BSD), ``STAILQ_*`` ⇒ ``DECLARE_LIST`` +- :file:`lib/*_queue.h` (BSD), ``TAILQ_*`` ⇒ ``DECLARE_DLIST`` +- :file:`lib/hash.*`, ``hash_*`` ⇒ ``DECLARE_HASH`` +- :file:`lib/linklist.*`, ``list_*`` ⇒ ``DECLARE_DLIST`` +- open-coded linked lists ⇒ ``DECLARE_LIST``/``DECLARE_DLIST`` + + +Code Formatting +--------------- + +C Code +^^^^^^ + +For C code, FRR uses Linux kernel style except where noted below. Code which +does not comply with these style guidelines will not be accepted. + +The project provides multiple tools to allow you to correctly style your code +as painlessly as possible, primarily built around ``clang-format``. + +clang-format + + In the project root there is a :file:`.clang-format` configuration file + which can be used with the ``clang-format`` source formatter tool from the + LLVM project. Most of the time, this is the easiest and smartest tool to + use. It can be run in a variety of ways. If you point it at a C source file + or directory of source files, it will format all of them. In the LLVM source + tree there are scripts that allow you to integrate it with ``git``, ``vim`` + and ``emacs``, and there are third-party plugins for other editors. The + ``git`` integration is particularly useful; suppose you have some changes in + your git index. Then, with the integration installed, you can do the + following: + + :: + + git clang-format + + This will format *only* the changes present in your index. If you have just + made a few commits and would like to correctly style only the changes made + in those commits, you can use the following syntax: + + :: + + git clang-format HEAD~X + + Where X is one more than the number of commits back from the tip of your + branch you would like ``clang-format`` to look at (similar to specifying the + target for a rebase). + + The ``vim`` plugin is particularly useful. It allows you to select lines in + visual line mode and press a key binding to invoke ``clang-format`` on only + those lines. + + When using ``clang-format``, it is recommended to use the latest version. + Each consecutive version generally has better handling of various edge + cases. You may notice on occasion that two consecutive runs of + ``clang-format`` over the same code may result in changes being made on the + second run. This is an unfortunate artifact of the tool. Please check with + the kernel style guide if in doubt. + + One stylistic problem with the FRR codebase is the use of ``DEFUN`` macros + for defining CLI commands. ``clang-format`` will happily format these macro + invocations, but the result is often unsightly and difficult to read. + Consequently, FRR takes a more relaxed position with how these are + formatted. In general you should lean towards using the style exemplified in + the section on :ref:`command-line-interface`. Because ``clang-format`` + mangles this style, there is a Python script named ``tools/indent.py`` that + wraps ``clang-format`` and handles ``DEFUN`` macros as well as some other + edge cases specific to FRR. If you are submitting a new file, it is + recommended to run that script over the new file, preferably after ensuring + that the latest stable release of ``clang-format`` is in your ``PATH``. + + Documentation on ``clang-format`` and its various integrations is maintained + on the LLVM website. + + https://clang.llvm.org/docs/ClangFormat.html + +checkpatch.sh +checkpatch.pl + + .. seealso:: :ref:`checkpatch` + + In the Linux kernel source tree there is a Perl script used to check + incoming patches for style errors. FRR uses a shell script front end and an + adapted version of the perl script for the same purpose. These scripts can + be found at :file:`tools/checkpatch.sh` and :file:`tools/checkpatch.pl`. + This script takes a git-formatted diff or patch file, applies it to a clean + FRR tree, and inspects the result to catch potential style errors. Running + this script on your patches before submission is highly recommended. The CI + system runs this script as well and will comment on the PR with the results + if style errors are found. + + It is run like this:: + + ./checkpatch.sh <patch> <tree> + + Reports are generated on ``stderr`` and the exit code indicates whether + issues were found (2, 1) or not (0). + + Where ``<patch>`` is the path to the diff or patch file and ``<tree>`` is + the path to your FRR source tree. The tree should be on the branch that you + intend to submit the patch against. The script will make a best-effort + attempt to save the state of your working tree and index before applying the + patch, and to restore it when it is done, but it is still recommended that + you have a clean working tree as the script does perform a hard reset on + your tree during its run. + + The script reports two classes of issues, namely WARNINGs and ERRORs. Please + pay attention to both of them. The script will generally report WARNINGs + where it cannot be 100% sure that a particular issue is real. In most cases + WARNINGs indicate an issue that needs to be fixed. Sometimes the script will + report false positives; these will be handled in code review on a + case-by-case basis. Since the script only looks at changed lines, + occasionally changing one part of a line can cause the script to report a + style issue already present on that line that is unrelated to the change. + When convenient it is preferred that these be cleaned up inline, but this is + not required. + + In general, a developer should heed the information reported by checkpatch. + However, some flexibility is needed for cases where human judgement yields + better clarity than the script. Accordingly, it may be appropriate to + ignore some checkpatch.sh warnings per discussion among the submitter(s) + and reviewer(s) of a change. Misreporting of errors by the script is + possible. When this occurs, the exception should be handled either by + patching checkpatch to correct the false error report, or by documenting the + exception in this document under :ref:`style-exceptions`. If the incorrect + report is likely to appear again, a checkpatch update is preferred. + + If the script finds one or more WARNINGs it will exit with 1. If it finds + one or more ERRORs it will exit with 2. + + For convenience the Linux documentation for the :file:`tools/checkpatch.pl` + script has been included unmodified (i.e., it has not been updated to + reflect local changes) :doc:`here <checkpatch>` + + +Please remember that while FRR provides these tools for your convenience, +responsibility for properly formatting your code ultimately lies on the +shoulders of the submitter. As such, it is recommended to double-check the +results of these tools to avoid delays in merging your submission. + +In some cases, these tools modify or flag the format in ways that go beyond or +even conflict [#tool_style_conflicts]_ with the canonical documented Linux +kernel style. In these cases, the Linux kernel style takes priority; +non-canonical issues flagged by the tools are not compulsory but rather are +opportunities for discussion among the submitter(s) and reviewer(s) of a change. + +**Whitespace changes in untouched parts of the code are not acceptable +in patches that change actual code.** To change/fix formatting issues, +please create a separate patch that only does formatting changes and +nothing else. + +Kernel and BSD styles are documented externally: + +- https://www.kernel.org/doc/html/latest/process/coding-style.html +- http://man.openbsd.org/style + +For GNU coding style, use ``indent`` with the following invocation: + +:: + + indent -nut -nfc1 file_for_submission.c + + +Historically, FRR used fixed-width integral types that do not exist in any +standard but were defined by most platforms at some point. Officially these +types are not guaranteed to exist. Therefore, please use the fixed-width +integral types introduced in the C99 standard when contributing new code to +FRR. If you need to convert a large amount of code to use the correct types, +there is a shell script in :file:`tools/convert-fixedwidth.sh` that will do the +necessary replacements. + ++-----------+--------------------------+ +| Incorrect | Correct | ++===========+==========================+ +| u_int8_t | uint8_t | ++-----------+--------------------------+ +| u_int16_t | uint16_t | ++-----------+--------------------------+ +| u_int32_t | uint32_t | ++-----------+--------------------------+ +| u_int64_t | uint64_t | ++-----------+--------------------------+ +| u_char | uint8_t or unsigned char | ++-----------+--------------------------+ +| u_short | unsigned short | ++-----------+--------------------------+ +| u_int | unsigned int | ++-----------+--------------------------+ +| u_long | unsigned long | ++-----------+--------------------------+ + +FRR also uses unnamed struct fields, enabled with ``-fms-extensions`` (cf. +https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html). The following two +patterns can/should be used where contextually appropriate: + +.. code-block:: c + + struct outer { + struct inner; + }; + +.. code-block:: c + + struct outer { + union { + struct inner; + struct inner inner_name; + }; + }; + + +.. _style-exceptions: + +Exceptions +"""""""""" + +FRR project code comes from a variety of sources, so there are some +stylistic exceptions in place. They are organized here by branch. + +For ``master``: + +BSD coding style applies to: + +- ``ldpd/`` + +``babeld`` uses, approximately, the following style: + +- K&R style braces +- Indents are 4 spaces +- Function return types are on their own line + +For ``stable/3.0`` and ``stable/2.0``: + +GNU coding style apply to the following parts: + +- ``lib/`` +- ``zebra/`` +- ``bgpd/`` +- ``ospfd/`` +- ``ospf6d/`` +- ``isisd/`` +- ``ripd/`` +- ``ripngd/`` +- ``vtysh/`` + +BSD coding style applies to: + +- ``ldpd/`` + + +Python Code +^^^^^^^^^^^ + +Format all Python code with `black <https://github.com/psf/black>`_. + +In a line:: + + python3 -m black <file.py> + +Run this on any Python files you modify before committing. + +FRR's Python code has been formatted with black version 19.10b. + + +YANG +^^^^ + +FRR uses YANG to define data models for its northbound interface. YANG models +should follow conventions used by the IETF standard models. From a practical +standpoint, this corresponds to the output produced by the ``yanglint`` tool +included in the ``libyang`` project, which is used by FRR to parse and validate +YANG models. You should run the following command on all YANG documents you +write: + +.. code-block:: console + + yanglint -f yang <model> + +The output of this command should be identical to the input file. The sole +exception to this is comments. ``yanglint`` does not support comments and will +strip them from its output. You may include comments in your YANG documents, +but they should be indented appropriately (use spaces). Where possible, +comments should be eschewed in favor of a suitable ``description`` statement. + +In short, a diff between your input file and the output of ``yanglint`` should +either be empty or contain only comments. + +Specific Exceptions +^^^^^^^^^^^^^^^^^^^ + +Most of the time checkpatch errors should be corrected. Occasionally as a group +maintainers will decide to ignore certain stylistic issues. Usually this is +because correcting the issue is not possible without large unrelated code +changes. When an exception is made, if it is unlikely to show up again and +doesn't warrant an update to checkpatch, it is documented here. + ++------------------------------------------+---------------------------------------------------------------+ +| Issue | Ignore Reason | ++==========================================+===============================================================+ +| DEFPY_HIDDEN, DEFPY_ATTR: complex macros | DEF* macros cannot be wrapped in parentheses without updating | +| should be wrapped in parentheses | all usages of the macro, which would be highly disruptive. | ++------------------------------------------+---------------------------------------------------------------+ + +Types of configurables +---------------------- + +.. note:: + + This entire section essentially just argues to not make configuration + unnecessarily involved for the user. Rather than rules, this is more of + a list of conclusions intended to help make FRR usable for operators. + + +Almost every feature FRR has comes with its own set of switches and options. +There are several stages at which configuration can be applied. In order of +preference, these are: + +- at configuration/runtime, through YANG. + + This is the preferred way for all FRR knobs. Not all daemons and features + are fully YANGified yet, so in some cases new features cannot rely on a + YANG interface. If a daemon already implements a YANG interface (even + partial), new CLI options must be implemented through a YANG model. + + .. warning:: + + Unlike everything else in this section being guidelines with some slack, + implementing and using a YANG interface for new CLI options in (even + partially!) YANGified daemons is a hard requirement. + + +- at configuration/runtime, through the CLI. + + The "good old" way for all regular configuration. More involved for users + to automate *correctly* than YANG. + +- at startup, by loading additional modules. + + If a feature introduces a dependency on additional libraries (e.g. libsnmp, + rtrlib, etc.), this is the best way to encapsulate the dependency. Having + a separate module allows the distribution to create a separate package + with the extra dependency, so FRR can still be installed without pulling + everything in. + + A module may also be appropriate if a feature is large and reasonably well + isolated. Reducing the amount of running the code is a security benefit, + so even if there are no new external dependencies, modules can be useful. + + While modules cannot currently be loaded at runtime, this is a tradeoff + decision that was made to allow modules to change/extend code that is very + hard to (re)adjust at runtime. If there is a case for runtime (un)loading + of modules, this tradeoff can absolutely be reevaluated. + +- at startup, with command line options. + + This interface is only appropriate for options that have an effect very + early in FRR startup, i.e. before configuration is loaded. Anything that + affects configuration load itself should be here, as well as options + changing the environment FRR runs in. + + If a tunable can be changed at runtime, a command line option is only + acceptable if the configured value has an effect before configuration is + loaded (e.g. zebra reads routes from the kernel before loading config, so + the netlink buffer size is an appropriate command line option.) + +- at compile time, with ``./configure`` options. + + This is the absolute last preference for tunables, since the distribution + needs to make the decision for the user and/or the user needs to rebuild + FRR in order to change the option. + + "Good" configure options do one of three things: + + - set distribution-specific parameters, most prominently all the path + options. File system layout is a distribution/packaging choice, so the + user would hopefully never need to adjust these. + + - changing toolchain behavior, e.g. instrumentation, warnings, + optimizations and sanitizers. + + - enabling/disabling parts of the build, especially if they need + additional dependencies. Being able to build only parts of FRR, or + without some library, is useful. **The only effect these options should + have is adding or removing files from the build result.** If a knob + in this category causes the same binary to exist in different variants, + it is likely implemented incorrectly! + + .. note:: + + This last guideline is currently ignored by several configure options. + ``vtysh`` in general depends on the entire list of enabled daemons, + and options like ``--enable-bgp-vnc`` and ``--enable-ospfapi`` change + daemons internally. Consider this more of an "ideal" than a "rule". + + +Whenever adding new knobs, please try reasonably hard to go up as far as +possible on the above list. Especially ``./configure`` flags are often enough +the "easy way out" but should be avoided when at all possible. To a lesser +degree, the same applies to command line options. + + +Compile-time conditional code +----------------------------- + +Many users access FRR via binary packages from 3rd party sources; +compile-time code puts inclusion/exclusion in the hands of the package +maintainer. Please think very carefully before making code conditional +at compile time, as it increases regression testing, maintenance +burdens, and user confusion. In particular, please avoid gratuitous +``--enable-…`` switches to the configure script - in general, code +should be of high quality and in working condition, or it shouldn’t be +in FRR at all. + +When code must be compile-time conditional, try have the compiler make +it conditional rather than the C pre-processor so that it will still be +checked by the compiler, even if disabled. For example, + +:: + + if (SOME_SYMBOL) + frobnicate(); + +is preferred to + +:: + + #ifdef SOME_SYMBOL + frobnicate (); + #endif /* SOME_SYMBOL */ + +Note that the former approach requires ensuring that ``SOME_SYMBOL`` will be +defined (watch your ``AC_DEFINE``\ s). + +Debug-guards in code +-------------------- + +Debugging statements are an important methodology to allow developers to fix +issues found in the code after it has been released. The caveat here is that +the developer must remember that people will be using the code at scale and in +ways that can be unexpected for the original implementor. As such debugs +**MUST** be guarded in such a way that they can be turned off. FRR has the +ability to turn on/off debugs from the CLI and it is expected that the +developer will use this convention to allow control of their debugs. + +Custom syntax-like block macros +------------------------------- + +FRR uses some macros that behave like the ``for`` or ``if`` C keywords. These +macros follow these patterns: + +- loop-style macros are named ``frr_each_*`` (and ``frr_each``) +- single run macros are named ``frr_with_*`` +- to avoid confusion, ``frr_with_*`` macros must always use a ``{ ... }`` + block even if the block only contains one statement. The ``frr_each`` + constructs are assumed to be well-known enough to use normal ``for`` rules. +- ``break``, ``return`` and ``goto`` all work correctly. For loop-style + macros, ``continue`` works correctly too. + +Both the ``each`` and ``with`` keywords are inspired by other (more +higher-level) programming languages that provide these constructs. + +There are also some older iteration macros, e.g. ``ALL_LIST_ELEMENTS`` and +``FOREACH_AFI_SAFI``. These macros in some cases do **not** fulfill the above +pattern (e.g. ``break`` does not work in ``FOREACH_AFI_SAFI`` because it +expands to 2 nested loops.) + +Static Analysis and Sanitizers +------------------------------ +Clang/LLVM and GCC come with a variety of tools that can be used to help find +bugs in FRR. + +clang-analyze + This is a static analyzer that scans the source code looking for patterns + that are likely to be bugs. The tool is run automatically on pull requests + as part of CI and new static analysis warnings will be placed in the CI + results. FRR aims for absolutely zero static analysis errors. While the + project is not quite there, code that introduces new static analysis errors + is very unlikely to be merged. + +AddressSanitizer + This is an excellent tool that provides runtime instrumentation for + detecting memory errors. As part of CI FRR is built with this + instrumentation and run through a series of tests to look for any results. + Testing your own code with this tool before submission is encouraged. You + can enable it by passing:: + + --enable-address-sanitizer + + to ``configure``. + +ThreadSanitizer + Similar to AddressSanitizer, this tool provides runtime instrumentation for + detecting data races. If you are working on or around multithreaded code, + extensive testing with this instrumtation enabled is *highly* recommended. + You can enable it by passing:: + + --enable-thread-sanitizer + + to ``configure``. + +MemorySanitizer + Similar to AddressSanitizer, this tool provides runtime instrumentation for + detecting use of uninitialized heap memory. Testing your own code with this + tool before submission is encouraged. You can enable it by passing:: + + --enable-memory-sanitizer + + to ``configure``. + +All of the above tools are available in the Clang/LLVM toolchain since 3.4. +AddressSanitizer and ThreadSanitizer are available in recent versions of GCC, +but are no longer actively maintained. MemorySanitizer is not available in GCC. + +.. note:: + + The different Sanitizers are mostly incompatible with each other. Please + refer to GCC/LLVM documentation for details. + +frr-format plugin + This is a GCC plugin provided with FRR that does extended type checks for + ``%pFX``-style printfrr extensions. To use this plugin, + + 1. install GCC plugin development files, e.g.:: + + apt-get install gcc-10-plugin-dev + + 2. **before** running ``configure``, compile the plugin with:: + + make -C tools/gcc-plugins CXX=g++-10 + + (Edit the GCC version to what you're using, it should work for GCC 9 or + newer.) + + After this, the plugin should be automatically picked up by ``configure``. + The plugin does not change very frequently, so you can keep it around across + work on different FRR branches. After a ``git clean -x``, the ``make`` line + will need to be run again. You can also add ``--with-frr-format`` to the + ``configure`` line to make sure the plugin is used, otherwise if something + is not set up correctly it might be silently ignored. + + .. warning:: + + Do **not** enable this plugin for package/release builds. It is intended + for developer/debug builds only. Since it modifies the compiler, it may + cause silent corruption of the executable files. + + Using the plugin also changes the string for ``PRI[udx]64`` from the + system value to ``%L[udx]`` (normally ``%ll[udx]`` or ``%l[udx]``.) + +Additionally, the FRR codebase is regularly scanned for static analysis +errors with Coverity and pull request changes are scanned as part of the +Continuous Integration (CI) process. Developers can scan their commits for +Coverity static analysis errors prior to submission using the +``scan-build`` command. To use this command, the ``clang-tools`` package must +be installed. For example, this can be accomplished on Ubuntu with the +``sudo apt-get install clang-tools`` command. Then, touch the files you want scanned and +invoke the ``scan-build`` command. For example:: + + cd ~/GitHub/frr + touch ospfd/ospf_flood.c ospfd/ospf_vty.c ospfd/ospf_opaque.c + cd build + scan-build make -j32 + +The results of the scan including any static analysis errors will appear inline. +Additionally, there will a directory in the /tmp containing the Coverity +reports (e.g., scan-build-2023-06-09-120100-473730-1). + +Executing non-installed dynamic binaries +---------------------------------------- + +Since FRR uses the GNU autotools build system, it inherits its shortcomings. +To execute a binary directly from the build tree under a wrapper like +`valgrind`, `gdb` or `strace`, use:: + + ./libtool --mode=execute valgrind [--valgrind-opts] zebra/zebra [--zebra-opts] + +While replacing valgrind/zebra as needed. The `libtool` script is found in +the root of the build directory after `./configure` has completed. Its purpose +is to correctly set up `LD_LIBRARY_PATH` so that libraries from the build tree +are used. (On some systems, `libtool` is also available from PATH, but this is +not always the case.) + +.. _cli-workflow: + +CLI changes +----------- + +CLI's are a complicated ugly beast. Additions or changes to the CLI should use +a DEFPY to encapsulate one setting as much as is possible. Additionally as new +DEFPY's are added to the system, documentation should be provided for the new +commands. + +Backwards Compatibility +----------------------- + +As a general principle, changes to CLI and code in the lib/ directory should be +made in a backwards compatible fashion. This means that changes that are purely +stylistic in nature should be avoided, e.g., renaming an existing macro or +library function name without any functional change. When adding new parameters +to common functions, it is also good to consider if this too should be done in +a backward compatible fashion, e.g., by preserving the old form in addition to +adding the new form. + +This is not to say that minor or even major functional changes to CLI and +common code should be avoided, but rather that the benefit gained from a change +should be weighed against the added cost/complexity to existing code. Also, +that when making such changes, it is good to preserve compatibility when +possible to do so without introducing maintenance overhead/cost. It is also +important to keep in mind, existing code includes code that may reside in +private repositories (and is yet to be submitted) or code that has yet to be +migrated from Quagga to FRR. + +That said, compatibility measures can (and should) be removed when either: + +- they become a significant burden, e.g. when data structures change and the + compatibility measure would need a complex adaptation layer or becomes + flat-out impossible +- some measure of time (dependent on the specific case) has passed, so that + the compatibility grace period is considered expired. + +For CLI commands, the deprecation period is 1 year. + +In all cases, compatibility pieces should be marked with compiler/preprocessor +annotations to print warnings at compile time, pointing to the appropriate +update path. A ``-Werror`` build should fail if compatibility bits are used. To +avoid compilation issues in released code, such compiler/preprocessor +annotations must be ignored non-development branches. For example: + +.. code-block:: c + + #if CONFDATE > 20180403 + CPP_NOTICE("Use of <XYZ> is deprecated, please use <ABC>") + #endif + +Preferably, the shell script :file:`tools/fixup-deprecated.py` will be +updated along with making non-backwards compatible code changes, or an +alternate script should be introduced, to update the code to match the +change. When the script is updated, there is no need to preserve the +deprecated code. Note that this does not apply to user interface +changes, just internal code, macros and libraries. + +Miscellaneous +------------- + +When in doubt, follow the guidelines in the Linux kernel style guide, or ask on +the development mailing list / public Slack instance. + +JSON Output +^^^^^^^^^^^ + +New JSON output in FRR needs to be backed by schema, in particular a YANG model. +When adding new JSON, first search for an existing YANG model, either in FRR or +a standard model (e.g., IETF) and use that model as the basis for any JSON +structure and *especially* for key names and canonical values formats. + +If no YANG model exists to support the JSON then an FRR YANG model needs to be +added to or created to support the JSON format. + +* All JSON keys are to be ``camelCased``, with no spaces. YANG modules almost + always use ``kebab-case`` (i.e., all lower case with hyphens to separate + words), so these identifiers need to be mapped to ``camelCase`` by removing + the hyphen (or symbol) and capitalizing the following letter, for + example "router-id" becomes "routerId" +* Commands which output JSON should produce ``{}`` if they have nothing to + display +* In general JSON commands include a ``json`` keyword typically at the end of + the CLI command (e.g., ``show ip ospf json``) + +Use of const +^^^^^^^^^^^^ + +Please consider using ``const`` when possible: it's a useful hint to +callers about the limits to side-effects from your apis, and it makes +it possible to use your apis in paths that involve ``const`` +objects. If you encounter existing apis that *could* be ``const``, +consider including changes in your own pull-request. + +Help with specific warnings +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +FRR's configure script enables a whole batch of extra warnings, some of which +may not be obvious in how to fix. Here are some notes on specific warnings: + +* ``-Wstrict-prototypes``: you probably just forgot the ``void`` in a function + declaration with no parameters, i.e. ``static void foo() {...}`` rather than + ``static void foo(void) {...}``. + + Without the ``void``, in C, it's a function with *unspecified* parameters + (and varargs calling convention.) This is a notable difference to C++, where + the ``void`` is optional and an empty parameter list means no parameters. + +* ``"strict match required"`` from the frr-format plugin: check if you are + using a cast in a printf parameter list. The frr-format plugin cannot + access correct full type information for casts like + ``printfrr(..., (uint64_t)something, ...)`` and will print incorrect + warnings particularly if ``uint64_t``, ``size_t`` or ``ptrdiff_t`` are + involved. The problem is *not* triggered with a variable or function return + value of the exact same type (without a cast). + + Since these cases are very rare, community consensus is to just work around + the warning even though the code might be correct. If you are running into + this, your options are: + + 1. try to avoid the cast altogether, maybe using a different printf format + specifier (e.g. ``%lu`` instead of ``%zu`` or ``PRIu64``). + 2. fix the type(s) of the function/variable/struct member being printed + 3. create a temporary variable with the value and print that without a cast + (this is the last resort and was not necessary anywhere so far.) + + +.. _documentation: + +Documentation +============= + +FRR uses Sphinx+RST as its documentation system. The document you are currently +reading was generated by Sphinx from RST source in +:file:`doc/developer/workflow.rst`. The documentation is structured as follows: + ++-----------------------+-------------------------------------------+ +| Directory | Contents | ++=======================+===========================================+ +| :file:`doc/user` | User documentation; configuration guides; | +| | protocol overviews | ++-----------------------+-------------------------------------------+ +| :file:`doc/developer` | Developer's documentation; API specs; | +| | datastructures; architecture overviews; | +| | project management procedure | ++-----------------------+-------------------------------------------+ +| :file:`doc/manpages` | Source for manpages | ++-----------------------+-------------------------------------------+ +| :file:`doc/figures` | Images and diagrams | ++-----------------------+-------------------------------------------+ +| :file:`doc/extra` | Miscellaneous Sphinx extensions, scripts, | +| | customizations, etc. | ++-----------------------+-------------------------------------------+ + +Each of these directories, with the exception of :file:`doc/figures` and +:file:`doc/extra`, contains a Sphinx-generated Makefile and configuration +script :file:`conf.py` used to set various document parameters. The makefile +can be used for a variety of targets; invoke `make help` in any of these +directories for a listing of available output formats. For convenience, there +is a top-level :file:`Makefile.am` that has targets for PDF and HTML +documentation for both developer and user documentation, respectively. That +makefile is also responsible for building manual pages packed with distribution +builds. + +Indent and styling should follow existing conventions: + +- 3 spaces for indents under directives +- Cross references may contain only lowercase alphanumeric characters and + hyphens ('-') +- Lines wrapped to 80 characters where possible + +Characters for header levels should follow Python documentation guide: + +- ``#`` with overline, for parts +- ``*`` with overline, for chapters +- ``=``, for sections +- ``-``, for subsections +- ``^``, for subsubsections +- ``"``, for paragraphs + +After you have made your changes, please make sure that you can invoke +``make latexpdf`` and ``make html`` with no warnings. + +The documentation is currently incomplete and needs love. If you find a broken +cross-reference, figure, dead hyperlink, style issue or any other nastiness we +gladly accept documentation patches. + +To build the docs, please ensure you have installed a recent version of +`Sphinx <http://www.sphinx-doc.org/en/stable/install.html>`_. If you want to +build LaTeX or PDF docs, you will also need a full LaTeX distribution +installed. + +Code +---- + +FRR is a large and complex software project developed by many different people +over a long period of time. Without adequate documentation, it can be +exceedingly difficult to understand code segments, APIs and other interfaces. +In the interest of keeping the project healthy and maintainable, you should +make every effort to document your code so that other people can understand +what it does without needing to closely read the code itself. + +Some specific guidelines that contributors should follow are: + +- Functions exposed in header files should have descriptive comments above + their signatures in the header file. At a minimum, a function comment should + contain information about the return value, parameters, and a general summary + of the function's purpose. Documentation on parameter values can be omitted + if it is (very) obvious what they are used for. + + Function comments must follow the style for multiline comments laid out in + the kernel style guide. + + Example: + + .. code-block:: c + + /* + * Determines whether or not a string is cool. + * + * text + * the string to check for coolness + * + * is_clccfc + * whether capslock is cruise control for cool + * + * Returns: + * 7 if the text is cool, 0 otherwise + */ + int check_coolness(const char *text, bool is_clccfc); + + Function comments should make it clear what parameters and return values are + used for. + +- Static functions should have descriptive comments in the same form as above + if what they do is not immediately obvious. Use good engineering judgement + when deciding whether a comment is necessary. If you are unsure, document + your code. +- Global variables, static or not, should have a comment describing their use. +- **For new code in lib/, these guidelines are hard requirements.** + +If you make significant changes to portions of the codebase covered in the +Developer's Manual, add a major subsystem or feature, or gain arcane mastery of +some undocumented or poorly documented part of the codebase, please document +your work so others can benefit. If you add a major feature or introduce a new +API, please document the architecture and API to the best of your abilities in +the Developer's Manual, using good judgement when choosing where to place it. + +Finally, if you come across some code that is undocumented and feel like +going above and beyond, document it! We absolutely appreciate and accept +patches that document previously undocumented code. + +User +---- + +If you are contributing code that adds significant user-visible functionality +please document how to use it in :file:`doc/user`. Use good judgement when +choosing where to place documentation. For example, instructions on how to use +your implementation of a new BGP draft should go in the BGP chapter instead of +being its own chapter. If you are adding a new protocol daemon, please create a +new chapter. + +FRR Specific Markup +------------------- + +FRR has some customizations applied to the Sphinx markup that go a long way +towards making documentation easier to use, write and maintain. + +CLI Commands +^^^^^^^^^^^^ + +When documenting CLI please use the ``.. clicmd::`` directive. This directive +will format the command and generate index entries automatically. For example, +the command :clicmd:`show pony` would be documented as follows: + +.. code-block:: rest + + .. clicmd:: show pony + + Prints an ASCII pony. Example output::: + + >>\. + /_ )`. + / _)`^)`. _.---. _ + (_,' \ `^-)"" `.\ + | | \ + \ / | + / \ /.___.'\ (\ (_ + < ,"|| \ |`. \`-' + \\ () )| )/ + hjw |_>|> /_] // + /_] /_] + + +When documented this way, CLI commands can be cross referenced with the +``:clicmd:`` inline markup like so: + +.. code-block:: rest + + :clicmd:`show pony` + +This is very helpful for users who want to quickly remind themselves what a +particular command does. + +When documenting a cli that has a ``no`` form, please do not include the ``no`` +form. I.e. ``no show pony`` would not be documented anywhere. Since most +commands have ``no`` forms, users should be able to infer these or get help +from vtysh's completions. + +When documenting commands that have lots of possible variants, just document +the single command in summary rather than enumerating each possible variant. +E.g. for ``show pony [foo|bar]``, do not: + +.. code-block:: rest + + .. clicmd:: show pony + .. clicmd:: show pony foo + .. clicmd:: show pony bar + +Do: + +.. code-block:: rest + + .. clicmd:: show pony [foo|bar] + + +Configuration Snippets +^^^^^^^^^^^^^^^^^^^^^^ + +When putting blocks of example configuration please use the +``.. code-block::`` directive and specify ``frr`` as the highlighting language, +as in the following example. This will tell Sphinx to use a custom Pygments +lexer to highlight FRR configuration syntax. + +.. code-block:: rest + + .. code-block:: frr + + ! + ! Example configuration file. + ! + log file /tmp/log.log + service integrated-vtysh-config + ! + ip route 1.2.3.0/24 reject + ipv6 route de:ea:db:ee:ff::/64 reject + ! + + +.. _GitHub: https://github.com/frrouting/frr +.. _GitHub issues: https://github.com/frrouting/frr/issues + +.. rubric:: Footnotes + +.. [#tool_style_conflicts] For example, lines over 80 characters are allowed + for text strings to make it possible to search the code for them: please + see `Linux kernel style (breaking long lines and strings) <https://www.kernel.org/doc/html/v4.10/process/coding-style.html#breaking-long-lines-and-strings>`_ + and `Issue #1794 <https://github.com/FRRouting/frr/issues/1794>`_. diff --git a/doc/developer/xrefs.rst b/doc/developer/xrefs.rst new file mode 100644 index 0000000..e8e07df --- /dev/null +++ b/doc/developer/xrefs.rst @@ -0,0 +1,215 @@ +.. _xrefs: + +Introspection (xrefs) +===================== + +The FRR library provides an introspection facility called "xrefs." The intent +is to provide structured access to annotated entities in the compiled binary, +such as log messages and thread scheduling calls. + +Enabling and use +---------------- + +Support for emitting an xref is included in the macros for the specific +entities, e.g. :c:func:`zlog_info` contains the relevant statements. The only +requirement for the system to work is a GNU compatible linker that supports +section start/end symbols. (The only known linker on any system FRR supports +that does not do this is the Solaris linker.) + +To verify xrefs have been included in a binary or dynamic library, run +``readelf -n binary``. For individual object files, it's +``readelf -S object.o | grep xref_array`` instead. + +Structure and contents +---------------------- + +As a slight improvement to security and fault detection, xrefs are divided into +a ``const struct xref *`` and an optional ``struct xrefdata *``. The required +const part contains: + +.. c:member:: enum xref_type xref.type + + Identifies what kind of object the xref points to. + +.. c:member:: int line +.. c:member:: const char *xref.file +.. c:member:: const char *xref.func + + Source code location of the xref. ``func`` will be ``<global>`` for + xrefs outside of a function. + +.. c:member:: struct xrefdata *xref.xrefdata + + The optional writable part of the xref. NULL if no non-const part exists. + +The optional non-const part has: + +.. c:member:: const struct xref *xrefdata.xref + + Pointer back to the constant part. Since circular pointers are close to + impossible to emit from inside a function body's static variables, this + is initialized at startup. + +.. c:member:: char xrefdata.uid[16] + + Unique identifier, see below. + +.. c:member:: const char *xrefdata.hashstr +.. c:member:: uint32_t xrefdata.hashu32[2] + + Input to unique identifier calculation. These should encompass all + details needed to make an xref unique. If more than one string should + be considered, use string concatenation for the initializer. + +Both structures can be extended by embedding them in a larger type-specific +struct, e.g. ``struct xref_logmsg *``. + +Unique identifiers +------------------ + +All xrefs that have a writable ``struct xrefdata *`` part are assigned an +unique identifier, which is formed as base32 (crockford) SHA256 on: + +- the source filename +- the ``hashstr`` field +- the ``hashu32`` fields + +.. note:: + + Function names and line numbers are intentionally not included to allow + moving items within a file without affecting the identifier. + +For running executables, this hash is calculated once at startup. When +directly reading from an ELF file with external tooling, the value must be +calculated when necessary. + +The identifiers have the form ``AXXXX-XXXXX`` where ``X`` is +``0-9, A-Z except I,L,O,U`` and ``A`` is ``G-Z except I,L,O,U`` (i.e. the +identifiers always start with a letter.) When reading identifiers from user +input, ``I`` and ``L`` should be replaced with ``1`` and ``O`` should be +replaced with ``0``. There are 49 bits of entropy in this identifier. + +Underlying machinery +-------------------- + +Xrefs are nothing other than global variables with some extra glue to make +them possible to find from the outside by looking at the binary. The first +non-obvious part is that they can occur inside of functions, since they're +defined as ``static``. They don't have a visible name -- they don't need one. + +To make finding these variables possible, another global variable, a pointer +to the first one, is created in the same way. However, it is put in a special +ELF section through ``__attribute__((section("xref_array")))``. This is the +section you can see with readelf. + +Finally, on the level of a whole executable or library, the linker will stuff +the individual pointers consecutive to each other since they're in the same +section — hence the array. Start and end of this array is given by the +linker-autogenerated ``__start_xref_array`` and ``__stop_xref_array`` symbols. +Using these, both a constructor to run at startup as well as an ELF note are +created. + +The ELF note is the entrypoint for externally retrieving xrefs from a binary +without having to run it. It can be found by walking through the ELF data +structures even if the binary has been fully stripped of debug and section +information. SystemTap's SDT probes & LTTng's trace points work in the same +way (though they emit 1 note for each probe, while xrefs only emit one note +in total which refers to the array.) Using xrefs does not impact SystemTap +or LTTng, the notes have identifiers they can be distinguished by. + +The ELF structure of a linked binary (library or executable) will look like +this:: + + $ readelf --wide -l -n lib/.libs/libfrr.so + + Elf file type is DYN (Shared object file) + Entry point 0x67d21 + There are 12 program headers, starting at offset 64 + + Program Headers: + Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align + PHDR 0x000040 0x0000000000000040 0x0000000000000040 0x0002a0 0x0002a0 R 0x8 + INTERP 0x125560 0x0000000000125560 0x0000000000125560 0x00001c 0x00001c R 0x10 + [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] + LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x02aff0 0x02aff0 R 0x1000 + LOAD 0x02b000 0x000000000002b000 0x000000000002b000 0x0b2889 0x0b2889 R E 0x1000 + LOAD 0x0de000 0x00000000000de000 0x00000000000de000 0x070048 0x070048 R 0x1000 + LOAD 0x14e428 0x000000000014f428 0x000000000014f428 0x00fb70 0x01a2b8 RW 0x1000 + DYNAMIC 0x157a40 0x0000000000158a40 0x0000000000158a40 0x000270 0x000270 RW 0x8 + NOTE 0x0002e0 0x00000000000002e0 0x00000000000002e0 0x00004c 0x00004c R 0x4 + TLS 0x14e428 0x000000000014f428 0x000000000014f428 0x000000 0x000008 R 0x8 + GNU_EH_FRAME 0x12557c 0x000000000012557c 0x000000000012557c 0x00819c 0x00819c R 0x4 + GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0x10 + GNU_RELRO 0x14e428 0x000000000014f428 0x000000000014f428 0x009bd8 0x009bd8 R 0x1 + + (...) + + Displaying notes found in: .note.gnu.build-id + Owner Data size Description + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) Build ID: 6a1f66be38b523095ebd6ec13cc15820cede903d + + Displaying notes found in: .note.FRR + Owner Data size Description + FRRouting 0x00000010 Unknown note type: (0x46455258) description data: 6c eb 15 00 00 00 00 00 74 ec 15 00 00 00 00 00 + +Where 0x15eb6c…0x15ec74 are the offsets (relative to the note itself) where +the xref array is in the file. Also note the owner is clearly marked as +"FRRouting" and the type is "XREF" in hex. + +For SystemTap's use of ELF notes, refer to +https://libstapsdt.readthedocs.io/en/latest/how-it-works/internals.html as an +entry point. + +.. note:: + + Due to GCC bug 41091, the "xref_array" section is not correctly generated + for C++ code when compiled by GCC. A workaround is present for runtime + functionality, but to extract the xrefs from a C++ source file, it needs + to be built with clang (or a future fixed version of GCC) instead. + +Extraction tool +--------------- + +The FRR source contains a matching tool to extract xref data from compiled ELF +binaries in ``python/xrelfo.py``. This tool uses CPython extensions +implemented in ``clippy`` and must therefore be executed with that. + +``xrelfo.py`` processes input from one or more ELF file (.o, .so, executable), +libtool object (.lo, .la, executable wrapper script) or JSON (output from +``xrelfo.py``) and generates an output JSON file. During standard FRR build, +it is invoked on all binaries and libraries and the result is combined into +``frr.json``. + +ELF files from any operating system, CPU architecture and endianness can be +processed on any host. Any issues with this are bugs in ``xrelfo.py`` +(or clippy's ELF code.) + +``xrelfo.py`` also performs some sanity checking, particularly on log +messages. The following options are available: + +.. option:: -o OUTPUT + + Filename to write JSON output to. As a convention, a ``.xref`` filename + extension is used. + +.. option:: -Wlog-format + + Performs extra checks on log message format strings, particularly checks + for ``\t`` and ``\n`` characters (which should not be used in log messages). + +.. option:: -Wlog-args + + Generates cleanup hints for format string arguments where + :c:func:`printfrr()` extensions could be used, e.g. replacing ``inet_ntoa`` + with ``%pI4``. + +.. option:: --profile + + Runs the Python profiler to identify hotspots in the ``xrelfo.py`` code. + +``xrelfo.py`` uses information about C structure definitions saved in +``python/xrefstructs.json``. This file is included with the FRR sources and +only needs to be regenerated when some of the ``struct xref_*`` definitions +are changed (which should be almost never). The file is written by +``python/tiabwarfo.py``, which uses ``pahole`` to extract the necessary data +from DWARF information. diff --git a/doc/developer/zebra.rst b/doc/developer/zebra.rst new file mode 100644 index 0000000..be2952e --- /dev/null +++ b/doc/developer/zebra.rst @@ -0,0 +1,232 @@ +.. _zebra: + +***** +Zebra +***** + +.. _zebra-protocol: + +Overview of the Zebra Protocol +============================== + +The Zebra protocol (or ``ZAPI``) is used by protocol daemons to +communicate with the **zebra** daemon. + +Each protocol daemon may request and send information to and from the +**zebra** daemon such as interface states, routing state, +nexthop-validation, and so on. Protocol daemons may also install +routes with **zebra**. The **zebra** daemon manages which routes are +installed into the forwarding table with the kernel. Some daemons use +more than one ZAPI connection. This is supported: each ZAPI session is +identified by a tuple of: ``{protocol, instance, session_id}``. LDPD +is an example: it uses a second, synchronous ZAPI session to manage +label blocks. The default value for ``session_id`` is zero; daemons +who use multiple ZAPI sessions must assign unique values to the +sessions' ids. + +The Zebra protocol is a streaming protocol, with a common header. Version 0 +lacks a version field and is implicitly versioned. Version 1 and all subsequent +versions have a version field. Version 0 can be distinguished from all other +versions by examining the 3rd byte of the header, which contains a marker value +of 255 (in Quagga) or 254 (in FRR) for all versions except version 0. The +marker byte corresponds to the command field in version 0, and the marker value +is a reserved command in version 0. + +Version History +--------------- + +- Version 0 + + Used by all versions of GNU Zebra and all version of Quagga up to and + including Quagga 0.98. This version has no ``version`` field, and so is + implicitly versioned as version 0. + +- Version 1 + + Added ``marker`` and ``version`` fields, increased ``command`` field to 16 + bits. Used by Quagga versions 0.99.3 through 0.99.20. + +- Version 2 + + Used by Quagga versions 0.99.21 through 0.99.23. + +- Version 3 + + Added ``vrf_id`` field. Used by Quagga versions 0.99.23 until FRR fork. + +- Version 4 + + Change marker value to 254 to prevent people mixing and matching Quagga and + FRR daemon binaries. Used by FRR versions 2.0 through 3.0.3. + +- Version 5 + + Increased VRF identifier field from 16 to 32 bits. Used by FRR versions 4.0 + through 5.0.1. + +- Version 6 + + Removed the following commands: + + * ZEBRA_IPV4_ROUTE_ADD + * ZEBRA_IPV4_ROUTE_DELETE + * ZEBRA_IPV6_ROUTE_ADD + * ZEBRA_IPV6_ROUTE_DELETE + + Used since FRR version 6.0. + + +Zebra Protocol Definition +========================= + +Zebra Protocol Header Field Definitions +--------------------------------------- + +Length + Total packet length including this header. + +Marker + Static marker. The marker value, when it exists, is 255 in all versions of + Quagga. It is 254 in all versions of FRR. This is to allow version 0 headers + (which do not include version explicitly) to be distinguished from versioned + headers. + +Version + Zebra protocol version number. Clients should not continue processing + messages past the version field for versions they do not recognise. + +Command + The Zebra protocol command. + + +Current Version +^^^^^^^^^^^^^^^ + +:: + + Version 5, 6 + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | Marker | Version | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | VRF ID | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Command | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +Past Versions +^^^^^^^^^^^^^ + +:: + + Version 0 + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | Command | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +:: + + Version 1, 2 + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | Marker | Version | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Command | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +:: + + Version 3, 4 + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | Marker | Version | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | VRF ID | Command | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +Zebra Protocol Commands +----------------------- + +The definitions of zebra protocol commands can be found at ``lib/zclient.h``. + +Dataplane batching +================== + +Dataplane batching is an optimization feature that reduces the processing +time involved in the user space to kernel space transition for every message we +want to send. + +Design +----------- + +With our dataplane abstraction, we create a queue of dataplane context objects +for the messages we want to send to the kernel. In a separate pthread, we +loop over this queue and send the context objects to the appropriate +dataplane. A batching enhancement tightly integrates with the dataplane +context objects so they are able to be batch sent to dataplanes that support +it. + +There is one main change in the dataplane code. It does not call +kernel-dependent functions one-by-one, but instead it hands a list of work down +to the kernel level for processing. + +Netlink +^^^^^^^ + +At the moment, this is the only dataplane that allows for batch sending +messages to it. + +When messages must be sent to the kernel, they are consecutively added +to the batch represented by the `struct nl_batch`. Context objects are firstly +encoded to their binary representation. All the encoding functions use the same +interface: take a context object, a buffer and a size of the buffer as an +argument. It is important that they should handle a situation in which a message +wouldn't fit in the buffer and return a proper error. To achieve a zero-copy +(in the user space only) messages are encoded to the same buffer which will +be passed to the kernel. Hence, we can theoretically hit the boundary of the +buffer. + +Messages stored in the batch are sent if one of the conditions occurs: + +- When an encoding function returns the buffer overflow error. The context + object that caused this error is re-added to the new, empty batch. + +- When the size of the batch hits certain limit. + +- When the namespace of a currently being processed context object is + different from all the previous ones. They have to be sent through + distinct sockets, so the messages cannot share the same buffer. + +- After the last message from the list is processed. + +As mentioned earlier, there is a special threshold which is smaller than +the size of the underlying buffer. It prevents the overflow error and thus +eliminates the case, in which a message is encoded twice. + +The buffer used in the batching is global, since allocating that big amount of +memory every time wouldn't be most effective. However, its size can be changed +dynamically, using hidden vtysh command: +``zebra kernel netlink batch-tx-buf (1-1048576) (1-1048576)``. This feature is +only used in tests and shouldn't be utilized in any other place. + +For every failed message in the batch, the kernel responds with an error +message. Error messages are kept in the same order as they were sent, so parsing the +response is straightforward. We use the two pointer technique to match +requests with responses and then set appropriate status of dataplane context +objects. There is also a global receive buffer and it is assumed that whatever +the kernel sends it will fit in this buffer. The payload of netlink error messages +consists of a error code and the original netlink message of the request, so +the batch response won't be bigger than the batch request increased by +some space for the headers. diff --git a/doc/extra/frrlexer.py b/doc/extra/frrlexer.py new file mode 100644 index 0000000..39ce5e6 --- /dev/null +++ b/doc/extra/frrlexer.py @@ -0,0 +1,28 @@ +# -*- coding: utf-8 -*- +# SPDX-License-Identifier: ISC +# Copyright (c) 2017 Vincent Bernat <bernat@luffy.cx> + +from pygments.lexer import RegexLexer, bygroups +from pygments.token import Text, Comment, Keyword +from pygments.token import String, Number, Name + + +class FRRLexer(RegexLexer): + name = "frr" + aliases = ["frr"] + tokens = { + "root": [ + (r"^[ \t]*!.*?\n", Comment.Singleline), + (r'"(\\\\|\\"|[^"])*"', String.Double), + ( + r"[a-f0-9]*:[a-f0-9]*:[a-f0-9:]*(:\d+\.\d+\.\d+\.\d+)?(/\d+)?", + Number, + ), # IPv6 + (r"\d+\.\d+\.\d+\.\d+(/\d+)?", Number), # IPv4 + (r"^([ \t]*)(no[ \t]+)?([-\w]+)", bygroups(Text, Keyword, Name.Function)), + (r"[ \t]+", Text), + (r"\n", Text), + (r"\d+", Number), + (r"\S+", Text), + ], + } diff --git a/doc/extra/spelling_wordlist.txt b/doc/extra/spelling_wordlist.txt new file mode 100644 index 0000000..271f5e4 --- /dev/null +++ b/doc/extra/spelling_wordlist.txt @@ -0,0 +1,243 @@ +abr +Admin +AFI +agentx +AgentX +APK +apps +ARP +ASes +ASv +autoconfiguration +Autoconfiguration +autoconfigured +autodetected +babeld +backtrace +backtraces +behaviour +bgp +bgpd +BGPD +blackhole +charon +cisco +Cisco +cli +clicmd +conf +config +Config +cr +crashlog +dataplane +de +deconfigured +deserialization +dev +distincts +dstmask +eBGP +eigrp +eigrpd +EOR +eth +ethernet +ethernets +extcommunity +failover +fallback +favour +filesystem +Flavel +frontend +frr +FRR +frrvty +gcc +ge +glibc +gre +hardcoded +holddown +Holddown +holdtime +honour +honoured +hostname +hsls +iBGP +ibm +ietf +igmp +inet +init +insta +interdomain +internet +ip +IP +iptables +ipv +IPv +IPvX +IPv4 +IPv6 +isis +isisd +lan +ldpd +le +libc +libcap +libexecinfo +Loc +localhost +logfile +loopback +lsa +LSA +lsas +LSAs +Masaki +Mbit +Mbits +macvlan +macvlans +mib +motd +mpls +mrib +mroute +mroutes +mrouting +mtu +multicast +Multicast +multicasting +multihop +multipath +Multipath +Multipoint +multiprotocol +Multiprotocol +namespace +nd +neighbour +neighbouring +neighbours +Netlink +netmask +netmasks +nexthop +nexthops +nflog +nhrp +NHRP +nhrpd +noauth +noninterfering +Noninterfering +nopassword +nve +NVE +opennhrp +optimisation +optimisations +ospf +OSPF +ospfd +pathing +pce +peerings +performant +php +pid +pim +pimd +ppp +pre +prepend +Prepend +prepended +proc +protobuf +Protobuf +PtP +ra +readonly +rebalance +reconverge +reestablish +reestimated +requestlist +reservable +Reservable +rfc +RFP +RIBs +ripd +Ripd +ripng +ripngd +Roughan +rp +rpki +rtt +RTT +Rxmt +µs +safi +setgid +sm +smux +smuxpeer +snmp +SNMP +snmpd +snmptrap +snmptrapd +Solaris +src +stateful +stdout +strftime +strongSwan +subagents +subsubsections +summarisation +summarise +summarising +supercedes +synchronisation +sysctl +syslogd +tc +te +Timestamp +tmp +trapsink +txt +unicast +Unicast +unix +urib +useable +username +usr +vertices +vici +vn +VN +VNC +vrf +vrfs +vrrp +vty +Vty +vtysh +wildcard +wildcarded +Wilfong +xDSL +xFF diff --git a/doc/figures/cli-change-client.drawio b/doc/figures/cli-change-client.drawio new file mode 100644 index 0000000..c7a68d4 --- /dev/null +++ b/doc/figures/cli-change-client.drawio @@ -0,0 +1,522 @@ +<mxfile host="Electron" modified="2023-06-19T07:55:43.434Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/21.2.8 Chrome/112.0.5615.165 Electron/24.2.0 Safari/537.36" etag="hHcr6k13KyEFOw_PaIFY" version="21.2.8" type="device"> + <diagram name="Page-1" id="58cdce13-f638-feb5-8d6f-7d28b1aa9fa0"> + <mxGraphModel dx="2074" dy="1264" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" background="none" math="0" shadow="1"> + <root> + <mxCell id="0" /> + <mxCell id="1" parent="0" /> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-239" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;jumpStyle=gap;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-28" target="nUYlmBzm2YxJIW5L2hvB-238" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="265.02000000000004" y="307.47999999999996" /> + <mxPoint x="265.02000000000004" y="307.47999999999996" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-240" value="copy of vty-&gt;cfg_changes<br>to protobuf msg" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-239" vertex="1" connectable="0"> + <mxGeometry x="-0.1005" relative="1" as="geometry"> + <mxPoint x="56" y="-15" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-80" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;jumpStyle=gap;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-77" target="nUYlmBzm2YxJIW5L2hvB-78" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="120.01999999999998" y="672.48" /> + <mxPoint x="120.01999999999998" y="672.48" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-11" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;fontSize=12;startSize=8;endSize=8;strokeColor=#ff0000;labelBackgroundColor=none;endArrow=open;fontFamily=Verdana;align=left;entryX=0;entryY=0.5;entryDx=0;entryDy=0;jumpStyle=gap;exitX=1;exitY=0.5;exitDx=0;exitDy=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-1" target="nUYlmBzm2YxJIW5L2hvB-7" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="260" y="505" /> + <mxPoint x="260" y="505" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-34" value="N" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-11" vertex="1" connectable="0"> + <mxGeometry x="-0.3317" y="1" relative="1" as="geometry"> + <mxPoint x="60" y="-14" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-15" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-1" target="nUYlmBzm2YxJIW5L2hvB-13" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="170.0200000000002" y="492.47999999999996" as="sourcePoint" /> + <mxPoint x="200.0200000000002" y="567.48" as="targetPoint" /> + <Array as="points"> + <mxPoint x="190.01999999999998" y="522.48" /> + <mxPoint x="190.01999999999998" y="567.48" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-35" value="N+1" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-15" vertex="1" connectable="0"> + <mxGeometry x="-0.5391" relative="1" as="geometry"> + <mxPoint x="20" y="2" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-1" value="user cmd:<br>&nbsp;"ip route 10.0.0.0/24 null0"<br>-------------------------------<br><br>" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="467.47999999999996" width="120" height="75" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-3" style="edgeStyle=orthogonalEdgeStyle;html=1;labelBackgroundColor=none;endArrow=open;endSize=8;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;align=left;entryX=0;entryY=0.5;entryDx=0;entryDy=0;jumpStyle=gap;exitX=1;exitY=0.25;exitDx=0;exitDy=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-1" target="nUYlmBzm2YxJIW5L2hvB-5" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="260.02000000000004" y="476.97999999999996" as="sourcePoint" /> + <mxPoint x="300.02000000000004" y="367.47999999999996" as="targetPoint" /> + <Array as="points"> + <mxPoint x="280.02000000000004" y="486.47999999999996" /> + <mxPoint x="280.02000000000004" y="397.47999999999996" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-33" value="1" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-3" vertex="1" connectable="0"> + <mxGeometry x="-0.3723" y="-1" relative="1" as="geometry"> + <mxPoint x="36" y="-76" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-5" value="nb_cli_enqueue_change" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=10;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="345.02000000000004" y="377.47999999999996" width="130" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-7" value="nb_cli_enqueue_change" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=10;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="345.02000000000004" y="484.97999999999996" width="130" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-29" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;jumpStyle=gap;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-13" target="nUYlmBzm2YxJIW5L2hvB-28" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="210" y="345" /> + <mxPoint x="210" y="345" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-31" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-13" target="nUYlmBzm2YxJIW5L2hvB-27" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-32" value="<font style="font-size: 7px;">file or !mgmtd</font>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=7;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-31" vertex="1" connectable="0"> + <mxGeometry x="-0.3307" y="1" relative="1" as="geometry"> + <mxPoint x="11" y="-9" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-13" value="nb_cli_apply_changes" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="210.01999999999998" y="547.48" width="100" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-14" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=4;rounded=1;labelBackgroundColor=none;strokeColor=#000000;fontFamily=Verdana;fontSize=12;fontColor=default;startSize=8;endSize=8;shape=connector;" parent="1" edge="1"> + <mxGeometry width="50" height="50" relative="1" as="geometry"> + <mxPoint x="409.62000000000006" y="467.47999999999996" as="sourcePoint" /> + <mxPoint x="409.62000000000006" y="427.47999999999996" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-17" value="" style="triangle;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;rotation=-90;" parent="1" vertex="1"> + <mxGeometry x="575" y="355" width="20" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-18" value="" style="triangle;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;rotation=-90;" parent="1" vertex="1"> + <mxGeometry x="575" y="385" width="20" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-19" value="" style="triangle;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;rotation=-90;" parent="1" vertex="1"> + <mxGeometry x="575" y="475" width="20" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-20" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=4;rounded=1;labelBackgroundColor=none;strokeColor=#000000;fontFamily=Verdana;fontSize=12;fontColor=default;startSize=8;endSize=8;shape=connector;" parent="1" edge="1"> + <mxGeometry width="50" height="50" relative="1" as="geometry"> + <mxPoint x="584.63" y="465" as="sourcePoint" /> + <mxPoint x="584.63" y="425" as="targetPoint" /> + <Array as="points"> + <mxPoint x="584.63" y="445" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-21" value="<font style="font-size: 10px;">candidate<br>ds</font>" style="shape=datastore;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="530" y="577.48" width="60" height="60" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-23" value="<font style="font-size: 10px;">candidate<br>ds</font>" style="shape=datastore;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="644.98" y="577.48" width="60" height="60" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-24" value="" style="shape=singleArrow;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="600" y="592.48" width="29.98" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-61" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-27" target="nUYlmBzm2YxJIW5L2hvB-59" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-27" value="nb_cli_apply_changes_internal" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="380.02000000000004" y="547.48" width="130" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-38" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;fillColor=#fa6800;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-238" target="nUYlmBzm2YxJIW5L2hvB-37" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="825.02" y="272.47999999999996" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-39" value="<font style="font-size: 10px;"><i>socket connection</i><br>FE client -&gt; adapter SETCFG_REQ<br><br></font>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-38" vertex="1" connectable="0"> + <mxGeometry x="-0.0889" y="2" relative="1" as="geometry"> + <mxPoint x="-27" y="22" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-255" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-28" target="nUYlmBzm2YxJIW5L2hvB-246" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="230" y="200" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-256" value="implicit_commit<br style="font-size: 10px;">(legacy CLI)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-255" vertex="1" connectable="0"> + <mxGeometry x="-0.5348" y="-1" relative="1" as="geometry"> + <mxPoint x="-11" y="9" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-28" value="vty_mgmt_send_config_data" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="200.01999999999998" y="327.47999999999996" width="130" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-238" value="mgmt_fe_send_setcfg_req" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="420.02000000000004" y="252.48000000000002" width="130" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-30" value="<font style="font-size: 7px;">mgmtd</font>" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=16;fontFamily=Verdana;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="235.01999999999998" y="517.48" width="50" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-36" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=4;rounded=1;labelBackgroundColor=none;strokeColor=#000000;fontFamily=Verdana;fontSize=12;fontColor=default;startSize=8;endSize=8;shape=connector;" parent="1" edge="1"> + <mxGeometry width="50" height="50" relative="1" as="geometry"> + <mxPoint x="294.62" y="472.43" as="sourcePoint" /> + <mxPoint x="294.62" y="432.43" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-41" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=doubleBlock;endFill=1;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-37" target="nUYlmBzm2YxJIW5L2hvB-40" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-42" value="<font style="font-size: 10px;">validates input and creates TXN (CONFIG)<br><i>can happen multiple times</i><br></font>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-41" vertex="1" connectable="0"> + <mxGeometry x="0.197" y="1" relative="1" as="geometry"> + <mxPoint x="114" y="-4" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-37" value="mgmt_fe_session_handle_setcfg_req_msg" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="730.02" y="297.47999999999996" width="190" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-55" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-40" target="nUYlmBzm2YxJIW5L2hvB-44" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-56" value="<font style="font-size: 10px;">copy protobuf -&gt; txn_req.set_cfg.cfg_changes<br style="border-color: var(--border-color); font-size: 10px;"></font><span style="font-size: 10px;"><font style="font-size: 10px;">TIMER: MGMTD_TXN_PROC_SETCFG</font><br style="font-size: 10px;"></span>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-55" vertex="1" connectable="0"> + <mxGeometry x="0.2852" y="-1" relative="1" as="geometry"> + <mxPoint x="126" y="-31" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-40" value="mgmt_txn_send_set_config_req" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="752.52" y="377.47999999999996" width="145" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-60" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-44" target="nUYlmBzm2YxJIW5L2hvB-59" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="675.02" y="547.48" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-128" value="1" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-60" vertex="1" connectable="0"> + <mxGeometry x="-0.3733" y="3" relative="1" as="geometry"> + <mxPoint x="21" y="-13" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-69" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.436;entryY=0.026;entryDx=0;entryDy=0;entryPerimeter=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-44" target="nUYlmBzm2YxJIW5L2hvB-68" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-70" value="implicit_commit" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-69" vertex="1" connectable="0"> + <mxGeometry x="-0.1764" y="-3" relative="1" as="geometry"> + <mxPoint x="48" y="-3" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-129" value="2" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-69" vertex="1" connectable="0"> + <mxGeometry x="-0.2682" y="-1" relative="1" as="geometry"> + <mxPoint x="-4" y="-11" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-72" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=baseDash;startSize=8;endSize=8;endFill=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-44" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="980.02" y="517.48" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-130" value="2" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=12;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-72" vertex="1" connectable="0"> + <mxGeometry x="-0.1117" y="-3" relative="1" as="geometry"> + <mxPoint x="-29" y="-13" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-44" value="<div>mgmt_txn_process_set_cfg</div>" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="752.52" y="497.47999999999996" width="145" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-49" value="" style="shape=singleArrow;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="480.02" y="375" width="70" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-50" value="" style="shape=singleArrow;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;rotation=-180;" parent="1" vertex="1"> + <mxGeometry x="615" y="375" width="70" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-59" value="<div>nb_candidate_edit</div>" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="542.5" y="524.98" width="105" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-64" value="struct<br>nb_cfg_change" style="text;html=1;align=center;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;fontSize=10;fontFamily=Verdana;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="535" y="315" width="100" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-68" value="<div>mgmt_txn_send_commit_config_req</div>" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffe6cc;strokeColor=#d79b00;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;" parent="1" vertex="1"> + <mxGeometry x="752.52" y="592.48" width="167.5" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-74" value="user cmd:<br>&nbsp;"ip route 10.0.1.0/24 null0"" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="550" width="100" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-75" value="user cmd:<br>&nbsp;"ip route 10.0.2.0/24 null0"" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="70.01999999999998" y="560" width="100" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-76" value="user cmd:<br>&nbsp;"ip route 10.0.3.0/24 null0"" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="80.01999999999998" y="570" width="100" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-77" value="user cmd:<br>"XFRR_end_configuration"<br>&nbsp;config or EOF" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;dashed=1;dashPattern=1 4;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="626.98" width="120" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-90" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-78" target="nUYlmBzm2YxJIW5L2hvB-84" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-78" value="vty_mgmt_send_commit_config" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="215.01999999999998" y="631.48" width="140" height="31" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-88" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;fillColor=#fa6800;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-84" target="nUYlmBzm2YxJIW5L2hvB-87" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="540" y="715" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-89" value="<i style="font-size: 10px;">socket connection<br style="font-size: 10px;"></i>FE client -&gt; adapter COMMCFG_REQ" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-88" vertex="1" connectable="0"> + <mxGeometry x="-0.0463" y="1" relative="1" as="geometry"> + <mxPoint x="-34" y="30" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-84" value="mgmt_fe_send_commitcfg_req" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="215.01999999999998" y="730" width="140" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-93" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;entryX=0.166;entryY=0.994;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-87" target="nUYlmBzm2YxJIW5L2hvB-68" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-87" value="mgmt_fe_session_handle_commit_config_req_msg<br>create txn if none yet<br>if running DS not locked, lock" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="490.00000000000006" y="700" width="220" height="90" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-95" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-68" target="nUYlmBzm2YxJIW5L2hvB-159" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="835.02" y="632.48" as="sourcePoint" /> + <mxPoint x="883.7977777777774" y="718.48" as="targetPoint" /> + <Array as="points" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-96" value="<span style="font-size: 10px;">curr_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG<br style="font-size: 10px;"></span>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-95" vertex="1" connectable="0"> + <mxGeometry x="0.2852" y="-1" relative="1" as="geometry"> + <mxPoint x="91" y="-21" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-131" value="<span style="font-size: 10px;">next_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG<br style="font-size: 10px;"></span>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="989.9977193457571" y="669.9979556509891" as="geometry"> + <mxPoint x="-46" y="1" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-159" value="TIMER:<br style="font-size: 7px;">MGMTD_TXN_PROC_COMCFG" style="ellipse;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=7;fillColor=#b1ddf0;strokeColor=#10739e;" parent="1" vertex="1"> + <mxGeometry x="800.02" y="717.26" width="120" height="80" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-241" value="<i style="border-color: var(--border-color);">does nothing more</i>:<span style="font-size: 9px;"><br>when</span><b style="font-size: 9px;"> not implicit_commit:</b><br style="font-size: 9px;">&nbsp;<font face="Courier New"><b>mgmt (set|delete)-config</b></font> CLI<br style="font-size: 9px;">(no_implicit_commit == true)<br style="font-size: 9px;">inside <font face="Courier New"><b>XFRR_{start,end}_config</b></font><br style="font-size: 9px;">(pending_allowed == true)" style="rounded=0;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=9;fillColor=#eeeeee;strokeColor=#36393d;" parent="1" vertex="1"> + <mxGeometry x="940.02" y="472.42999999999995" width="140" height="100.05" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-243" value="user cmd:<br>"XFRR_start_configuration"<br>&nbsp;config file read indicator" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;dashed=1;dashPattern=1 4;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="417.47999999999996" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-257" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-245" target="nUYlmBzm2YxJIW5L2hvB-246" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="150" y="243" /> + <mxPoint x="200" y="243" /> + <mxPoint x="200" y="180" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-258" value="NO implicit commit<br style="font-size: 10px;">(vtysh -f file)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;horizontal=0;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-257" vertex="1" connectable="0"> + <mxGeometry x="-0.8771" y="-1" relative="1" as="geometry"> + <mxPoint x="9" y="-41" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-245" value="user cmd:<br>"configure terminal"" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=default;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;strokeWidth=1;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="367.47999999999996" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-248" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;strokeWidth=2;fillColor=#fa6800;startArrow=open;startFill=0;shadow=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-246" target="nUYlmBzm2YxJIW5L2hvB-247" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="625" y="190" as="targetPoint" /> + <Array as="points"> + <mxPoint x="585" y="193" /> + <mxPoint x="585" y="193" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-250" value="<i>socket connection<br style="font-size: 9px;"></i>FE client -&gt; adapter LOCKDS_REQ" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-248" vertex="1" connectable="0"> + <mxGeometry x="-0.0567" y="1" relative="1" as="geometry"> + <mxPoint x="5" y="-16" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-246" value="vty_mgmt_lock_cand_inline" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="380.02" y="172.48" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-247" value="LOCK CANDIDATE" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffe6cc;strokeColor=#d79b00;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;" parent="1" vertex="1"> + <mxGeometry x="680" y="175.00000000000003" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-252" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-251" target="nUYlmBzm2YxJIW5L2hvB-245" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-253" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.75;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;exitX=0.248;exitY=0.923;exitDx=0;exitDy=0;exitPerimeter=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-251" target="nUYlmBzm2YxJIW5L2hvB-243" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="90.01999999999998" y="322.47999999999996" /> + <mxPoint x="50.019999999999996" y="322.47999999999996" /> + <mxPoint x="50.019999999999996" y="443.47999999999996" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-254" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;exitX=0.088;exitY=0.793;exitDx=0;exitDy=0;exitPerimeter=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-251" target="nUYlmBzm2YxJIW5L2hvB-1" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="60" y="510" as="targetPoint" /> + <Array as="points"> + <mxPoint x="71" y="253" /> + <mxPoint x="40" y="253" /> + <mxPoint x="40" y="513" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-251" value="EVENT: VTYSH_READ" style="ellipse;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=7;fillColor=#b1ddf0;strokeColor=#10739e;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999999" y="142.48000000000002" width="120" height="80" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-260" value="UNLOCK CANDIDATE" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffe6cc;strokeColor=#d79b00;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;" parent="1" vertex="1"> + <mxGeometry x="680" y="120.00000000000001" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-265" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-261" target="nUYlmBzm2YxJIW5L2hvB-262" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="10" y="707" /> + <mxPoint x="10" y="130" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-266" value="NO implicit commit" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;horizontal=0;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-265" vertex="1" connectable="0"> + <mxGeometry x="-0.781" y="-1" relative="1" as="geometry"> + <mxPoint x="9" y="-12" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-261" value="user cmd:<br>"end/exit"" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=default;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;strokeWidth=1;" parent="1" vertex="1"> + <mxGeometry x="60.01999999999998" y="690" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-262" value="vty_mgmt_lock_cand_inline" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="380.02" y="120" width="120" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-275" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-270" target="nUYlmBzm2YxJIW5L2hvB-268" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="300" y="35" /> + <mxPoint x="300" y="35" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-270" value="vty_mgmt_set_config_result_notified" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="360" y="11.25" width="180" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-263" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;strokeWidth=2;fillColor=#fa6800;startArrow=open;startFill=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-262" target="nUYlmBzm2YxJIW5L2hvB-260" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="500.02" y="140" as="sourcePoint" /> + <mxPoint x="680.02" y="140" as="targetPoint" /> + <Array as="points" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-264" value="<i>socket connection<br style="font-size: 9px;"></i>FE client -&gt; adapter LOCKDS_REQ" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-263" vertex="1" connectable="0"> + <mxGeometry x="-0.0567" y="1" relative="1" as="geometry"> + <mxPoint x="5" y="-16" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-272" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-267" target="nUYlmBzm2YxJIW5L2hvB-271" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="630" y="76" /> + <mxPoint x="630" y="76" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-273" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-267" target="nUYlmBzm2YxJIW5L2hvB-270" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="630" y="40" /> + <mxPoint x="630" y="40" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-267" value="EVENT: REPLY NOTIFICATIONS" style="ellipse;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=7;fillColor=#b1ddf0;strokeColor=#10739e;" parent="1" vertex="1"> + <mxGeometry x="660" y="5" width="120" height="80" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-269" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;jumpStyle=gap;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-268" target="nUYlmBzm2YxJIW5L2hvB-251" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="80" y="80" /> + <mxPoint x="80" y="80" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-268" value="<div>VTYSH</div>" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#cdeb8b;strokeColor=#36393d;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;" parent="1" vertex="1"> + <mxGeometry x="30" y="15" width="77.48" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-274" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-271" target="nUYlmBzm2YxJIW5L2hvB-268" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="330" y="76" /> + <mxPoint x="330" y="50" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-271" value="vty_mgmt_commit_config_result_notified" style="rounded=1;whiteSpace=wrap;html=1;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=#000000;align=center;" parent="1" vertex="1"> + <mxGeometry x="360" y="58.75" width="180" height="35" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-292" value="" style="group" parent="1" vertex="1" connectable="0"> + <mxGeometry x="950" y="710" width="140" height="130" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-284" value="" style="rounded=0;whiteSpace=wrap;html=1;fontFamily=Verdana;fontSize=12;fontColor=default;" parent="nUYlmBzm2YxJIW5L2hvB-292" vertex="1"> + <mxGeometry width="140" height="130" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-278" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;fillColor=#fa6800;" parent="nUYlmBzm2YxJIW5L2hvB-292" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="20" y="60" as="sourcePoint" /> + <mxPoint x="110.01999999999998" y="60" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-279" value="<i style="font-size: 10px;">socket&nbsp;</i>async" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-278" vertex="1" connectable="0"> + <mxGeometry x="-0.0463" y="1" relative="1" as="geometry"> + <mxPoint x="-8" y="-9" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-282" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;fillColor=#0050ef;" parent="nUYlmBzm2YxJIW5L2hvB-292" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="20" y="90" as="sourcePoint" /> + <mxPoint x="110" y="90" as="targetPoint" /> + <Array as="points"> + <mxPoint x="50" y="89.77000000000001" /> + <mxPoint x="50" y="89.77000000000001" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-283" value="<span style="font-size: 10px;"><font style="font-size: 10px;">timer/event&nbsp;</font>async<br style="font-size: 10px;"></span>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=10;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-282" vertex="1" connectable="0"> + <mxGeometry x="0.2852" y="-1" relative="1" as="geometry"> + <mxPoint x="-28" y="-11" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-285" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;labelBackgroundColor=none;strokeColor=#C73500;fontFamily=Verdana;fontSize=12;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;strokeWidth=2;fillColor=#fa6800;startArrow=open;startFill=0;" parent="nUYlmBzm2YxJIW5L2hvB-292" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="20" y="35" as="sourcePoint" /> + <mxPoint x="110" y="35" as="targetPoint" /> + <Array as="points"> + <mxPoint x="20" y="35" /> + <mxPoint x="20" y="35" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-286" value="<i>socket&nbsp; sync (short-circuit)<br></i>" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-285" vertex="1" connectable="0"> + <mxGeometry x="-0.0567" y="1" relative="1" as="geometry"> + <mxPoint x="5" y="-16" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-287" value="" style="endArrow=open;html=1;rounded=1;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=12;fontColor=default;startSize=8;endSize=8;shape=connector;" parent="nUYlmBzm2YxJIW5L2hvB-292" edge="1"> + <mxGeometry width="50" height="50" relative="1" as="geometry"> + <mxPoint x="20" y="120" as="sourcePoint" /> + <mxPoint x="105" y="119.19999999999999" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-288" value="function sync" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=9;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-287" vertex="1" connectable="0"> + <mxGeometry x="-0.26" y="2" relative="1" as="geometry"> + <mxPoint x="6" y="-8" as="offset" /> + </mxGeometry> + </mxCell> + </root> + </mxGraphModel> + </diagram> +</mxfile> diff --git a/doc/figures/cli-change-client.svg b/doc/figures/cli-change-client.svg new file mode 100644 index 0000000..9194f24 --- /dev/null +++ b/doc/figures/cli-change-client.svg @@ -0,0 +1,3 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1099" height="842" viewBox="-0.5 -0.5 1099 842" style="background-color: rgb(255, 255, 255);"><defs><filter id="dropShadow"><feGaussianBlur in="SourceAlpha" stdDeviation="1.7" result="blur"/><feOffset in="blur" dx="3" dy="3" result="offsetBlur"/><feFlood flood-color="#3D4574" flood-opacity="0.4" result="offsetColor"/><feComposite in="offsetColor" in2="offsetBlur" operator="in" result="offsetBlur"/><feBlend in="SourceGraphic" in2="offsetBlur"/></filter></defs><g filter="url(#dropShadow)"><path d="M 264 322.48 L 264 277.5 Q 264 267.5 274 267.5 L 416.78 267.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 408.9 272 L 417.9 267.5 L 408.9 263" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 253px; margin-left: 360px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">copy of vty->cfg_changes<br />to protobuf msg</div></div></div></foreignObject><text x="360" y="256" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">copy of vty->cfg_changes...</text></switch></g><path d="M 119 661.98 L 119 664.74 Q 119 667.5 119 657.5 L 119 649.75 Q 119 642 129 642 L 211.78 642" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 203.9 646.5 L 212.9 642 L 203.9 637.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 179.02 499.98 L 341.78 499.98" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 333.9 504.48 L 342.9 499.98 L 333.9 495.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 485px; margin-left: 295px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">N</div></div></div></foreignObject><text x="295" y="489" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">N</text></switch></g><path d="M 179.02 517.5 L 184.01 517.5 Q 189 517.5 189 527.5 L 189 552.5 Q 189 562.5 197.89 562.49 L 206.78 562.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 198.91 566.99 L 207.9 562.48 L 198.9 557.99" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 527px; margin-left: 210px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">N+1</div></div></div></foreignObject><text x="210" y="531" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">N+1</text></switch></g><rect x="59.02" y="462.48" width="120" height="75" rx="18" ry="18" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 500px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br /> "ip route 10.0.0.0/24 null0"<br />-------------------------------<br /><br /></div></div></div></foreignObject><text x="119" y="502" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><path d="M 179.02 481.23 L 269 481.2 Q 279 481.2 279 471.2 L 279 402.5 Q 279 392.5 289 392.5 L 341.78 392.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 333.9 396.98 L 342.9 392.48 L 333.9 387.98" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 407px; margin-left: 296px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">1</div></div></div></foreignObject><text x="296" y="410" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">1</text></switch></g><rect x="344.02" y="372.48" width="130" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 392px; margin-left: 345px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">nb_cli_enqueue_change</div></div></div></foreignObject><text x="409" y="395" fill="#000000" font-family="Verdana" font-size="10px" text-anchor="middle">nb_cli_enqueue_change</text></switch></g><rect x="344.02" y="479.98" width="130" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 500px; margin-left: 345px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">nb_cli_enqueue_change</div></div></div></foreignObject><text x="409" y="503" fill="#000000" font-family="Verdana" font-size="10px" text-anchor="middle">nb_cli_enqueue_change</text></switch></g><path d="M 259.02 542.48 L 259.02 502.98 M 259.02 496.98 M 259.02 496.98 L 259.02 484.21 M 259.02 478.21 M 259.02 478.21 L 259.02 364.72" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 263.52 372.6 L 259.02 363.6 L 254.52 372.6" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 309.02 562.48 L 376.78 562.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 368.9 566.98 L 377.9 562.48 L 368.9 557.98" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 553px; margin-left: 344px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 7px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><font style="font-size: 7px;">file or !mgmtd</font></div></div></div></foreignObject><text x="344" y="555" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="7px" text-anchor="middle">file or !mgmtd</text></switch></g><rect x="209.02" y="542.48" width="100" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 98px; height: 1px; padding-top: 562px; margin-left: 210px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">nb_cli_apply_changes</div></div></div></foreignObject><text x="259" y="565" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">nb_cli_apply_changes</text></switch></g><path d="M 408.62 462.48 L 408.62 422.48" fill="none" stroke="#000000" stroke-width="4" stroke-miterlimit="10" stroke-dasharray="4 12" pointer-events="stroke"/><path d="M 574 350 L 594 365 L 574 380 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(-90,584,365)" pointer-events="all"/><path d="M 574 380 L 594 395 L 574 410 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(-90,584,395)" pointer-events="all"/><path d="M 574 470 L 594 485 L 574 500 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(-90,584,485)" pointer-events="all"/><path d="M 583.63 460 L 583.63 450 Q 583.63 440 583.63 430 L 583.63 420" fill="none" stroke="#000000" stroke-width="4" stroke-miterlimit="10" stroke-dasharray="4 12" pointer-events="stroke"/><path d="M 529 580.48 C 529 569.81 589 569.81 589 580.48 L 589 624.48 C 589 635.15 529 635.15 529 624.48 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><path d="M 529 580.48 C 529 588.48 589 588.48 589 580.48 M 529 584.48 C 529 592.48 589 592.48 589 584.48 M 529 588.48 C 529 596.48 589 596.48 589 588.48" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 58px; height: 1px; padding-top: 612px; margin-left: 530px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><font style="font-size: 10px;">candidate<br />ds</font></div></div></div></foreignObject><text x="559" y="616" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">candidate...</text></switch></g><path d="M 643.98 580.48 C 643.98 569.81 703.98 569.81 703.98 580.48 L 703.98 624.48 C 703.98 635.15 643.98 635.15 643.98 624.48 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><path d="M 643.98 580.48 C 643.98 588.48 703.98 588.48 703.98 580.48 M 643.98 584.48 C 643.98 592.48 703.98 592.48 703.98 584.48 M 643.98 588.48 C 643.98 596.48 703.98 596.48 703.98 588.48" fill="none" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 58px; height: 1px; padding-top: 612px; margin-left: 645px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><font style="font-size: 10px;">candidate<br />ds</font></div></div></div></foreignObject><text x="674" y="616" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">candidate...</text></switch></g><path d="M 599 597.98 L 622.98 597.98 L 622.98 587.48 L 628.98 602.48 L 622.98 617.48 L 622.98 606.98 L 599 606.98 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><path d="M 509.02 562.5 L 519.01 562.5 Q 529 562.5 529 552.5 L 529 546.25 Q 529 540 534.13 539.99 L 539.26 539.98" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 531.39 544.5 L 540.38 539.98 L 531.37 535.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="379.02" y="542.48" width="130" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 562px; margin-left: 380px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">nb_cli_apply_changes_internal</div></div></div></foreignObject><text x="444" y="565" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">nb_cli_apply_changes_internal</text></switch></g><path d="M 549.02 267.5 L 814 267.5 Q 824 267.5 824.01 277.5 L 824.02 288.01" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 819.01 280.25 L 824.02 290.24 L 829.01 280.24" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 288px; margin-left: 660px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><font style="font-size: 10px;"><i>socket connection</i><br />FE client -> adapter SETCFG_REQ<br /><br /></font></div></div></div></foreignObject><text x="660" y="291" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">socket connection...</text></switch></g><path d="M 229 322.48 L 229 205 Q 229 195 239 195 L 376.78 195" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 368.9 199.5 L 377.9 195 L 368.9 190.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)rotate(-90 219.5 265.98)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 266px; margin-left: 220px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">implicit_commit<br style="font-size: 10px;" />(legacy CLI)</div></div></div></foreignObject><text x="220" y="269" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">implicit_commit...</text></switch></g><rect x="199.02" y="322.48" width="130" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 342px; margin-left: 200px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_send_config_data</div></div></div></foreignObject><text x="264" y="345" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_send_config_data</text></switch></g><rect x="419.02" y="247.48" width="130" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 267px; margin-left: 420px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_fe_send_setcfg_req</div></div></div></foreignObject><text x="484" y="270" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_fe_send_setcfg_req</text></switch></g><rect x="234.02" y="512.48" width="50" height="30" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 527px; margin-left: 259px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 16px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><font style="font-size: 7px;">mgmtd</font></div></div></div></foreignObject><text x="259" y="532" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="16px" text-anchor="middle">mgmtd</text></switch></g><path d="M 293.62 467.43 L 293.62 427.43" fill="none" stroke="#000000" stroke-width="4" stroke-miterlimit="10" stroke-dasharray="4 12" pointer-events="stroke"/><path d="M 824.02 332.48 L 824.02 357.36" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 824.02 371.36 L 820.52 364.36 L 827.52 364.36 Z M 824.02 364.36 L 820.52 357.36 L 827.52 357.36 Z" fill="#ff0000" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 353px; margin-left: 940px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><font style="font-size: 10px;">validates input and creates TXN (CONFIG)<br /><i>can happen multiple times</i><br /></font></div></div></div></foreignObject><text x="940" y="356" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">validates input and creates TXN (CONFIG)...</text></switch></g><rect x="729.02" y="292.48" width="190" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 188px; height: 1px; padding-top: 312px; margin-left: 730px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_fe_session_handle_setcfg_req_msg</div></div></div></foreignObject><text x="824" y="315" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_fe_session_handle_setcfg_req_msg</text></switch></g><path d="M 824.02 412.48 L 824.02 488.01" fill="none" stroke="#001dbc" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 819.02 480.24 L 824.02 490.24 L 829.02 480.24" fill="none" stroke="#001dbc" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 433px; margin-left: 950px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><font style="font-size: 10px;">copy protobuf -> txn_req.set_cfg.cfg_changes<br style="border-color: var(--border-color); font-size: 10px;" /></font><span style="font-size: 10px;"><font style="font-size: 10px;">TIMER: MGMTD_TXN_PROC_SETCFG</font><br style="font-size: 10px;" /></span></div></div></div></foreignObject><text x="950" y="436" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">copy protobuf -> txn_req.set_cfg.cfg_changes...</text></switch></g><rect x="751.52" y="372.48" width="145" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 392px; margin-left: 753px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_set_config_req</div></div></div></foreignObject><text x="824" y="395" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_set_config_req</text></switch></g><path d="M 751.52 512.5 L 709 512.5 Q 699 512.5 699 522.5 L 699 531.25 Q 699 540 689 540 L 648.74 540" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 656.62 535.5 L 647.62 540 L 656.62 544.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 503px; margin-left: 731px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">1</div></div></div></foreignObject><text x="731" y="507" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">1</text></switch></g><path d="M 824 532.48 L 824 550 Q 824 560 824.19 570 L 824.51 586.28" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 819.86 578.49 L 824.53 587.4 L 828.85 578.32" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 553px; margin-left: 870px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">implicit_commit</div></div></div></foreignObject><text x="870" y="556" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">implicit_commit</text></switch></g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 543px; margin-left: 820px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">2</div></div></div></foreignObject><text x="820" y="547" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">2</text></switch></g><path d="M 896.52 512.5 L 927.8 512.5 Q 937.8 512.5 947.8 512.5 L 979.02 512.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 979.02 517.48 L 979.02 507.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 503px; margin-left: 905px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 12px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">2</div></div></div></foreignObject><text x="905" y="507" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="12px" text-anchor="middle">2</text></switch></g><rect x="751.52" y="492.48" width="145" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 512px; margin-left: 753px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><div>mgmt_txn_process_set_cfg</div></div></div></div></foreignObject><text x="824" y="515" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_process_set_cfg</text></switch></g><path d="M 479.02 384 L 535.02 384 L 535.02 370 L 549.02 390 L 535.02 410 L 535.02 396 L 479.02 396 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" pointer-events="all"/><path d="M 614 384 L 670 384 L 670 370 L 684 390 L 670 410 L 670 396 L 614 396 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-miterlimit="10" transform="rotate(-180,649,390)" pointer-events="all"/><rect x="541.5" y="519.98" width="105" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 103px; height: 1px; padding-top: 540px; margin-left: 543px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><div>nb_candidate_edit</div></div></div></div></foreignObject><text x="594" y="542" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">nb_candidate_edit</text></switch></g><rect x="534" y="310" width="100" height="40" fill="none" stroke="none" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 330px; margin-left: 584px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">struct<br />nb_cfg_change</div></div></div></foreignObject><text x="584" y="333" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">struct...</text></switch></g><rect x="751.52" y="587.48" width="167.5" height="40" rx="9.6" ry="9.6" fill="#ffe6cc" stroke="#d79b00" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 607px; margin-left: 753px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><div>mgmt_txn_send_commit_config_req</div></div></div></div></foreignObject><text x="835" y="610" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_commit_config_req</text></switch></g><rect x="59.02" y="545" width="100" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 98px; height: 1px; padding-top: 565px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br /> "ip route 10.0.1.0/24 null0"</div></div></div></foreignObject><text x="109" y="567" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><rect x="69.02" y="555" width="100" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 98px; height: 1px; padding-top: 575px; margin-left: 70px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br /> "ip route 10.0.2.0/24 null0"</div></div></div></foreignObject><text x="119" y="577" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><rect x="79.02" y="565" width="100" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 98px; height: 1px; padding-top: 585px; margin-left: 80px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br /> "ip route 10.0.3.0/24 null0"</div></div></div></foreignObject><text x="129" y="587" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><rect x="59.02" y="621.98" width="120" height="40" rx="9.6" ry="9.6" fill="#ffffc0" stroke="#ff0000" stroke-dasharray="1 4" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 642px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br />"XFRR_end_configuration"<br /> config or EOF</div></div></div></foreignObject><text x="119" y="644" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><path d="M 284.02 657.48 L 284.02 722.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 279.52 714.88 L 284.02 723.88 L 288.52 714.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="214.02" y="626.48" width="140" height="31" rx="7.44" ry="7.44" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 138px; height: 1px; padding-top: 642px; margin-left: 215px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_send_commit_config</div></div></div></foreignObject><text x="284" y="644" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_send_commit_config</text></switch></g><path d="M 354.02 740 L 484.53 740" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 476.76 745 L 486.76 740 L 476.76 735" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 770px; margin-left: 385px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><i style="font-size: 10px;">socket connection<br style="font-size: 10px;" /></i>FE client -> adapter COMMCFG_REQ</div></div></div></foreignObject><text x="385" y="773" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">socket connection...</text></switch></g><rect x="214.02" y="725" width="140" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 138px; height: 1px; padding-top: 740px; margin-left: 215px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_fe_send_commitcfg_req</div></div></div></foreignObject><text x="284" y="742" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_fe_send_commitcfg_req</text></switch></g><path d="M 709 740 L 769.3 740 Q 779.3 740 779.3 730 L 779.32 629.48" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 783.82 637.36 L 779.32 628.36 L 774.82 637.36" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="489" y="695" width="220" height="90" rx="21.6" ry="21.6" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 218px; height: 1px; padding-top: 740px; margin-left: 490px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_fe_session_handle_commit_config_req_msg<br />create txn if none yet<br />if running DS not locked, lock</div></div></div></foreignObject><text x="599" y="742" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_fe_session_handle_commit_config_req_msg...</text></switch></g><path d="M 835.3 627.48 L 835.3 659.9 Q 835.3 669.9 845.3 669.9 L 852.15 669.9 Q 859 669.9 859 679.9 L 859.02 710.02" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 854.52 702.14 L 859.02 711.14 L 863.52 702.14" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 653px; margin-left: 950px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><span style="font-size: 10px;">curr_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG<br style="font-size: 10px;" /></span></div></div></div></foreignObject><text x="950" y="656" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">curr_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG
</text></switch></g><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 666px; margin-left: 943px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><span style="font-size: 10px;">next_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG<br style="font-size: 10px;" /></span></div></div></div></foreignObject><text x="943" y="669" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">next_phase == MGMTD_COMMIT_PHASE_PREPARE_CFG
</text></switch></g><ellipse cx="859.02" cy="752.26" rx="60" ry="40" fill="#b1ddf0" stroke="#10739e" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 752px; margin-left: 800px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 7px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">TIMER:<br style="font-size: 7px;" />MGMTD_TXN_PROC_COMCFG</div></div></div></foreignObject><text x="859" y="754" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="7px" text-anchor="middle">TIMER:...</text></switch></g><rect x="939.02" y="467.43" width="140" height="100.05" fill="#eeeeee" stroke="#36393d" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 138px; height: 1px; padding-top: 517px; margin-left: 940px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 9px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><i style="border-color: var(--border-color);">does nothing more</i>:<span style="font-size: 9px;"><br />when</span><b style="font-size: 9px;"> not implicit_commit:</b><br style="font-size: 9px;" /> <font face="Courier New"><b>mgmt (set|delete)-config</b></font> CLI<br style="font-size: 9px;" />(no_implicit_commit == true)<br style="font-size: 9px;" />inside <font face="Courier New"><b>XFRR_{start,end}_config</b></font><br style="font-size: 9px;" />(pending_allowed == true)</div></div></div></foreignObject><text x="1009" y="520" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="9px" text-anchor="middle">does nothing more:...</text></switch></g><rect x="59.02" y="412.48" width="120" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="#ff0000" stroke-dasharray="1 4" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 430px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br />"XFRR_start_configuration"<br /> config file read indicator</div></div></div></foreignObject><text x="119" y="432" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><path d="M 149 362.48 L 149 248 Q 149 238 159 238 L 189 238 Q 199 238 199 228 L 199 185 Q 199 175 209 175 L 376.78 175" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 368.9 179.5 L 377.9 175 L 368.9 170.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)rotate(-90 159.5 294.98)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 295px; margin-left: 160px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">NO implicit commit<br style="font-size: 10px;" />(vtysh -f file)</div></div></div></foreignObject><text x="160" y="298" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">NO implicit commit...</text></switch></g><rect x="59.02" y="362.48" width="120" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="rgb(0, 0, 0)" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 380px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br />"configure terminal"</div></div></div></foreignObject><text x="119" y="382" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><path d="M 503.49 188 L 574 188 Q 584 188 594 188 L 674.53 188" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="6 6" pointer-events="stroke"/><path d="M 511.26 183 L 501.26 188 L 511.26 193" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><path d="M 666.76 193 L 676.76 188 L 666.76 183" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 172px; margin-left: 590px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 9px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><i>socket connection<br style="font-size: 9px;" /></i>FE client -> adapter LOCKDS_REQ</div></div></div></foreignObject><text x="590" y="174" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="9px" text-anchor="middle">socket connection...</text></switch></g><rect x="379.02" y="167.48" width="120" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 185px; margin-left: 380px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_lock_cand_inline</div></div></div></foreignObject><text x="439" y="187" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_lock_cand_inline</text></switch></g><rect x="679" y="170" width="120" height="35" rx="8.4" ry="8.4" fill="#ffe6cc" stroke="#d79b00" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 188px; margin-left: 680px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">LOCK CANDIDATE</div></div></div></foreignObject><text x="739" y="190" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">LOCK CANDIDATE</text></switch></g><path d="M 119.02 217.48 L 119.02 360.24" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 114.52 352.36 L 119.02 361.36 L 123.52 352.36" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 88.78 211.32 L 88.8 307.5 Q 88.8 317.5 78.8 317.5 L 59 317.5 Q 49 317.5 49 327.5 L 49 428.7 Q 49 438.7 52.89 438.71 L 56.78 438.72" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 48.89 443.2 L 57.9 438.73 L 48.92 434.2" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 69.58 200.92 L 69.6 238 Q 69.6 248 59.6 248 L 49 248 Q 39 248 39 258 L 39 498 Q 39 508 47.89 508 L 56.78 508" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 48.9 512.5 L 57.9 508 L 48.9 503.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><ellipse cx="119.02" cy="177.48" rx="60" ry="40" fill="#b1ddf0" stroke="#10739e" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 177px; margin-left: 60px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 7px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">EVENT: VTYSH_READ</div></div></div></foreignObject><text x="119" y="180" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="7px" text-anchor="middle">EVENT: VTYSH_READ</text></switch></g><rect x="679" y="115" width="120" height="35" rx="8.4" ry="8.4" fill="#ffe6cc" stroke="#d79b00" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 133px; margin-left: 680px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">UNLOCK CANDIDATE</div></div></div></foreignObject><text x="739" y="135" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">UNLOCK CANDIDATE</text></switch></g><path d="M 59.02 702.5 L 19 702.5 Q 9 702.5 9 692.5 L 9 135 Q 9 125 19 125 L 376.78 125" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 368.9 129.5 L 377.9 125 L 368.9 120.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)rotate(-90 19.5 631.02)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 631px; margin-left: 20px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;">NO implicit commit</div></div></div></foreignObject><text x="20" y="634" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">NO implicit commit</text></switch></g><rect x="59.02" y="685" width="120" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="rgb(0, 0, 0)" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 703px; margin-left: 60px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">user cmd:<br />"end/exit"</div></div></div></foreignObject><text x="119" y="705" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">user cmd:...</text></switch></g><rect x="379.02" y="115" width="120" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 133px; margin-left: 380px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_lock_cand_inline</div></div></div></foreignObject><text x="439" y="135" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_lock_cand_inline</text></switch></g><path d="M 359 30 L 309 30 Q 299 30 289 30 L 31.24 30" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 39.12 25.5 L 30.12 30 L 39.12 34.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="359" y="6.25" width="180" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 178px; height: 1px; padding-top: 24px; margin-left: 360px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_set_config_result_notified</div></div></div></foreignObject><text x="449" y="26" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_set_config_result_notified</text></switch></g><path d="M 503.49 132.5 L 674.53 132.5" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="6 6" pointer-events="stroke"/><path d="M 511.26 127.5 L 501.26 132.5 L 511.26 137.5" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><path d="M 666.76 137.5 L 676.76 132.5 L 666.76 127.5" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 116px; margin-left: 590px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 9px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: nowrap;"><i>socket connection<br style="font-size: 9px;" /></i>FE client -> adapter LOCKDS_REQ</div></div></div></foreignObject><text x="590" y="119" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="9px" text-anchor="middle">socket connection...</text></switch></g><path d="M 681.08 71 L 639 71 Q 629 71 619 71 L 541.24 71" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 549.12 66.5 L 540.12 71 L 549.12 75.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 659.47 35 L 639 35 Q 629 35 619 35 L 541.24 35" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 549.12 30.5 L 540.12 35 L 549.12 39.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><ellipse cx="719" cy="40" rx="60" ry="40" fill="#b1ddf0" stroke="#10739e" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 40px; margin-left: 660px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 7px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">EVENT: REPLY NOTIFICATIONS</div></div></div></foreignObject><text x="719" y="42" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="7px" text-anchor="middle">EVENT: REPLY NOTIFICATIONS</text></switch></g><path d="M 79 50 L 79 122 M 79 128 M 79 128 L 79 145.44" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 74.5 137.56 L 79 146.56 L 83.5 137.56" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="29" y="10" width="77.48" height="40" rx="9.6" ry="9.6" fill="#cdeb8b" stroke="#36393d" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 75px; height: 1px; padding-top: 30px; margin-left: 30px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;"><div>VTYSH</div></div></div></div></foreignObject><text x="68" y="32" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">VTYSH</text></switch></g><path d="M 359 71.3 L 339 71.3 Q 329 71.3 329 61.3 L 329 53.15 Q 329 45 319 45 L 108.72 45" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 116.6 40.5 L 107.6 45 L 116.6 49.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="359" y="53.75" width="180" height="35" rx="8.4" ry="8.4" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 178px; height: 1px; padding-top: 71px; margin-left: 360px;"><div data-drawio-colors="color: #000000; " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">vty_mgmt_commit_config_result_notified</div></div></div></foreignObject><text x="449" y="74" fill="#000000" font-family="Verdana" font-size="8px" text-anchor="middle">vty_mgmt_commit_config_result_notified</text></switch></g><rect x="949" y="705" width="140" height="130" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" pointer-events="none"/><path d="M 969 765 L 1054.55 765" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="none"/><path d="M 1046.78 770 L 1056.78 765 L 1046.78 760" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 756px; margin-left: 1005px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: nowrap;"><i style="font-size: 10px;">socket </i>async</div></div></div></foreignObject><text x="1005" y="759" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">socket async</text></switch></g><path d="M 969 795 L 989 795 Q 999 795 1009 795 L 1054.53 795" fill="none" stroke="#001dbc" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="none"/><path d="M 1046.76 800 L 1056.76 795 L 1046.76 790" fill="none" stroke="#001dbc" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 786px; margin-left: 1000px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 10px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: nowrap;"><span style="font-size: 10px;"><font style="font-size: 10px;">timer/event </font>async<br style="font-size: 10px;" /></span></div></div></div></foreignObject><text x="1000" y="789" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="10px" text-anchor="middle">timer/event async
</text></switch></g><path d="M 973.47 740 L 1054.53 740" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="6 6" pointer-events="none"/><path d="M 981.24 735 L 971.24 740 L 981.24 745" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><path d="M 1046.76 745 L 1056.76 740 L 1046.76 735" fill="none" stroke="#c73500" stroke-width="2" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 724px; margin-left: 1017px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 9px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: nowrap;"><i>socket sync (short-circuit)<br /></i></div></div></div></foreignObject><text x="1017" y="726" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="9px" text-anchor="middle">socket sync (short-circuit)
</text></switch></g><path d="M 969 825 L 1051.76 824.22" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="none"/><path d="M 1043.92 828.8 L 1052.88 824.21 L 1043.84 819.8" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="none"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 1px; height: 1px; padding-top: 815px; margin-left: 1006px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 9px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: none; white-space: nowrap;">function sync</div></div></div></foreignObject><text x="1006" y="818" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="9px" text-anchor="middle">function sync</text></switch></g></g><switch><g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/><a transform="translate(0,-5)" xlink:href="https://www.drawio.com/doc/faq/svg-export-text-problems" target="_blank"><text text-anchor="middle" font-size="10px" x="50%" y="100%">Text is not SVG - cannot display</text></a></switch></svg>
\ No newline at end of file diff --git a/doc/figures/cli-change-mgmtd.drawio b/doc/figures/cli-change-mgmtd.drawio new file mode 100644 index 0000000..e8beade --- /dev/null +++ b/doc/figures/cli-change-mgmtd.drawio @@ -0,0 +1,421 @@ +<mxfile host="Electron" modified="2023-06-19T08:43:10.542Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/21.4.0 Chrome/112.0.5615.204 Electron/24.5.1 Safari/537.36" etag="nT5OZWDjYXR5quOjpvZj" version="21.4.0" type="device"> + <diagram name="Page-1" id="58cdce13-f638-feb5-8d6f-7d28b1aa9fa0"> + <mxGraphModel dx="974" dy="1264" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" background="none" math="0" shadow="1"> + <root> + <mxCell id="0" /> + <mxCell id="1" parent="0" /> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-158" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;jumpStyle=gap;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-65" target="nUYlmBzm2YxJIW5L2hvB-157" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-150" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;startArrow=none;startFill=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-65" target="nUYlmBzm2YxJIW5L2hvB-148" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1350" y="320" /> + <mxPoint x="1350" y="320" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-65" value="mgmt_txn_prepare_cfg" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1280" y="279.78000000000003" width="145" height="20.44" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-160" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-217" target="nUYlmBzm2YxJIW5L2hvB-161" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1920" y="380" as="targetPoint" /> + <Array as="points"> + <mxPoint x="1607" y="590" /> + <mxPoint x="1840" y="590" /> + <mxPoint x="1840" y="660" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-174" value="MESSAGE_TXN_REQ
create" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-160" vertex="1" connectable="0"> + <mxGeometry x="-0.5683" relative="1" as="geometry"> + <mxPoint x="-17" y="-10" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-218" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-66" target="nUYlmBzm2YxJIW5L2hvB-217" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1490" y="590" /> + <mxPoint x="1490" y="590" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-66" value="mgmt_txn_send_be_txn_create" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1280" y="580" width="145" height="20" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-217" value="mgmt_be_send_txn_req" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1505" y="581" width="145" height="20" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-203" value="does nothing
cfg_data replys will cause next transition
" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#eeeeee;strokeColor=#36393d;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1240" y="640" width="180" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-214" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-67" target="nUYlmBzm2YxJIW5L2hvB-213" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-216" value="next_phase =
 PHASE_TXN_DELETE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-214" vertex="1" connectable="0"> + <mxGeometry x="-0.2492" y="-1" relative="1" as="geometry"> + <mxPoint x="10" y="19" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-67" value="mgmt_txn_send_be_cfg_apply" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1200" y="709.9999999999999" width="145" height="20" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-213" value="mgmt_be_send_cfgapply_req" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1440" y="709.9999999999999" width="145" height="20" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-111" value="mgmt_txn_send_commit_cfg_reply" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;fontColor=default;align=center;" parent="1" vertex="1"> + <mxGeometry x="1161.25" y="769.9999999999999" width="145" height="20" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-140" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-94" target="nUYlmBzm2YxJIW5L2hvB-66" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1270" y="590" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-145" value="PHASE_TXN_CREATE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-140" vertex="1" connectable="0"> + <mxGeometry x="0.2148" y="-2" relative="1" as="geometry"> + <mxPoint x="49" y="112" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-141" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-94" target="nUYlmBzm2YxJIW5L2hvB-65" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1300" y="260" /> + <mxPoint x="1353" y="260" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-144" value="PHASE_PREPARE_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-141" vertex="1" connectable="0"> + <mxGeometry x="-0.1955" y="3" relative="1" as="geometry"> + <mxPoint x="13" y="-7" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-142" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-94" target="nUYlmBzm2YxJIW5L2hvB-67" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1210" y="340" /> + <mxPoint x="1210" y="340" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-146" value="PHASE_CFG_APPLY" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-142" vertex="1" connectable="0"> + <mxGeometry x="0.6696" y="2" relative="1" as="geometry"> + <mxPoint x="48" y="68" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-143" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-94" target="nUYlmBzm2YxJIW5L2hvB-111" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1180" y="380" /> + <mxPoint x="1180" y="380" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-147" value="PHASE_TXN_DELETE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-143" vertex="1" connectable="0"> + <mxGeometry x="0.7799" y="3" relative="1" as="geometry"> + <mxPoint x="51" y="48" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-204" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-94" target="nUYlmBzm2YxJIW5L2hvB-203" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1250" y="340" /> + <mxPoint x="1250" y="340" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-205" value="PHASE_SEND_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-204" vertex="1" connectable="0"> + <mxGeometry x="0.857" y="3" relative="1" as="geometry"> + <mxPoint x="37" y="18" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-94" value="mgmt_txn_process_commit_cfg" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1150" y="170" width="167.5" height="70" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-97" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;exitX=1;exitY=0.5;exitDx=0;exitDy=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-65" target="nUYlmBzm2YxJIW5L2hvB-138" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1360" y="281" as="sourcePoint" /> + <mxPoint x="1318" y="225" as="targetPoint" /> + <Array as="points"> + <mxPoint x="1580" y="290" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-113" value="curr_phase =
PHASE_TXN_CREATE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-97" vertex="1" connectable="0"> + <mxGeometry x="0.2534" y="-1" relative="1" as="geometry"> + <mxPoint x="-101" y="35" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-119" value="mgmt_txn_send_commit_config_req" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffe6cc;strokeColor=#d79b00;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1150" y="50" width="167.5" height="40" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-122" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-119" target="nUYlmBzm2YxJIW5L2hvB-138" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="826" y="435" as="sourcePoint" /> + <mxPoint x="824" y="521" as="targetPoint" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-123" value="curr_phase ==
MGMTD_COMMIT_PHASE_PREPARE_CFG
" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-122" vertex="1" connectable="0"> + <mxGeometry x="0.2852" y="-1" relative="1" as="geometry"> + <mxPoint x="-23" y="-21" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-132" value="next_phase ==
MGMTD_COMMIT_PHASE_PREPARE_CFG
" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="1425" y="89.99851851851847" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-139" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-138" target="nUYlmBzm2YxJIW5L2hvB-94" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1540" y="190" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-138" value="TIMER:
MGMTD_TXN_PROC_COMCFG" style="ellipse;whiteSpace=wrap;fontFamily=Verdana;fontSize=8;fillColor=#b1ddf0;strokeColor=#10739e;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1520" y="30" width="120" height="80" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-156" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;startArrow=none;startFill=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-65" target="nUYlmBzm2YxJIW5L2hvB-154" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1353" y="310" /> + <mxPoint x="1513" y="310" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-148" value="GET nb_config_change's
nb_config_diff(cand, run)
or 
txn->commit_cfg_req->req.commit_cfg.cfg_chgs
" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#e1d5e7;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;strokeColor=#9673a6;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1280" y="330" width="145" height="50" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-154" value="mgmt_txn_create_config_batches" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#e1d5e7;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;strokeColor=#9673a6;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1440" y="330" width="145" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-157" value="TIMER:
MGMTD_TXN_
COMMITCFG_TIMEOUT" style="ellipse;whiteSpace=wrap;fontFamily=Verdana;fontSize=8;fillColor=#b1ddf0;strokeColor=#10739e;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1590" y="190" width="90" height="60" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-176" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=doubleBlock;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;endFill=1;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-161" target="nUYlmBzm2YxJIW5L2hvB-175" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1970" y="90" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-177" value="MESSAGE_CFG_DATA_REPLY" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-176" vertex="1" connectable="0"> + <mxGeometry x="0.177" relative="1" as="geometry"> + <mxPoint x="-10" y="255" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-224" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;endFill=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-161" target="nUYlmBzm2YxJIW5L2hvB-223" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1890" y="460" as="targetPoint" /> + <Array as="points"> + <mxPoint x="1940" y="305" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-225" value="MESSAGE_CFG_APPLY_REPLY;" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-224" vertex="1" connectable="0"> + <mxGeometry x="-0.0859" y="3" relative="1" as="geometry"> + <mxPoint x="-7" y="64" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-161" value="Backend Client" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#cdeb8b;strokeColor=#36393d;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1935" y="610" width="205" height="120" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-169" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-163" target="nUYlmBzm2YxJIW5L2hvB-168" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-163" value="mgmt_txn_notify_be_txn_reply" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="2020" y="465" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-192" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=doubleBlock;startSize=8;endSize=8;endFill=1;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-168" target="nUYlmBzm2YxJIW5L2hvB-171" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-168" value="mgmt_txn_send_be_cfg_data" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="2020" y="415" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-188" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-171" target="nUYlmBzm2YxJIW5L2hvB-187" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-189" value="batch: PHASE_TXN_REQ -> PHASE_SEND_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-188" vertex="1" connectable="0"> + <mxGeometry x="-0.176" relative="1" as="geometry"> + <mxPoint x="-9" y="-55" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-171" value="mgmt_be_send_cfgdata_req" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="2025" y="310" width="157.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-200" value="txn: PHASE_TXN_REQ -> PHASE_SEND_CFG" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-187" target="nUYlmBzm2YxJIW5L2hvB-199" edge="1"> + <mxGeometry x="1" y="80" relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="2095" y="240" /> + <mxPoint x="2095" y="240" /> + </Array> + <mxPoint x="75" y="-80" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-187" value="mgmt_move_txn_cfg_batch_to_next" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="2025" y="255" width="157.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-201" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-199" target="nUYlmBzm2YxJIW5L2hvB-138" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="2095" y="30" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-202" value="curr_phase = PHASE_SEND_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-201" vertex="1" connectable="0"> + <mxGeometry x="0.4129" y="-3" relative="1" as="geometry"> + <mxPoint x="121" y="-7" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-199" value="mgmt_try_move_commit_to_next_phase
if all backend clients
have all been sent their batches
move to next phase and post EVENT/TIMER" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="2002.5" y="160" width="185" height="60" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-172" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=doubleBlock;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;endFill=1;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-171" target="nUYlmBzm2YxJIW5L2hvB-161" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1957.5" y="175" as="sourcePoint" /> + <Array as="points"> + <mxPoint x="2000" y="325" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-173" value="MESSAGE_CFG_DATA_REQ" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-172" vertex="1" connectable="0"> + <mxGeometry x="-0.1783" y="-1" relative="1" as="geometry"> + <mxPoint x="-9" y="-30" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-195" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-175" target="nUYlmBzm2YxJIW5L2hvB-179" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-197" value="batch: PHASE_SEND_CFG -> PHASE_APPLY_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-195" vertex="1" connectable="0"> + <mxGeometry x="0.463" y="-1" relative="1" as="geometry"> + <mxPoint x="1" y="-9" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-175" value="mgmt_txn_notify_be_cfgdata_reply" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1770" y="75" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-210" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-179" target="nUYlmBzm2YxJIW5L2hvB-180" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-179" value="mgmt_move_txn_cfg_batch_to_next" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1770" y="160" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-181" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-180" target="nUYlmBzm2YxJIW5L2hvB-138" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1770" y="30" as="targetPoint" /> + <Array as="points"> + <mxPoint x="1720" y="235" /> + <mxPoint x="1720" y="50" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-182" value="curr_phase = PHASE_APPLY_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-181" vertex="1" connectable="0"> + <mxGeometry x="-0.4275" relative="1" as="geometry"> + <mxPoint x="50" y="72" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-180" value="mgmt_try_move_commit_to_next_phase" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1763.75" y="220" width="180" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-164" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-161" target="nUYlmBzm2YxJIW5L2hvB-163" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="2120" y="510" as="targetPoint" /> + <mxPoint x="2104" y="610" as="sourcePoint" /> + <Array as="points"> + <mxPoint x="2104" y="580" /> + <mxPoint x="2104" y="580" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-167" value="MESSAGE_TXN_REPLY" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-164" vertex="1" connectable="0"> + <mxGeometry x="-0.5021" y="1" relative="1" as="geometry"> + <mxPoint x="-13" y="-25" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-183" value="curr_phase =
PHASE_TXN_CREATE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="1470" y="569.9974074074072" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-184" value="next_phase =
PHASE_SEND_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="1459.9985714285715" y="609.998574414664" as="geometry"> + <mxPoint x="5" y="-3" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-186" value="For each of the
batches of cfgdata
send msg and move" style="verticalLabelPosition=middle;verticalAlign=middle;strokeWidth=2;shape=mxgraph.lean_mapping.physical_pull;pointerEvents=1;fontFamily=Verdana;fontSize=8;fontColor=default;labelPosition=right;align=left;horizontal=1;" parent="1" vertex="1"> + <mxGeometry x="2025" y="370" width="30" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-191" value="For each of the
 batches of cfgdata" style="verticalLabelPosition=middle;verticalAlign=middle;strokeWidth=2;shape=mxgraph.lean_mapping.physical_pull;pointerEvents=1;fontFamily=Verdana;fontSize=8;fontColor=default;labelPosition=right;align=left;horizontal=1;" parent="1" vertex="1"> + <mxGeometry x="1935" y="55" width="30" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-211" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-213" target="nUYlmBzm2YxJIW5L2hvB-161" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="1945" y="670" as="targetPoint" /> + <mxPoint x="1435" y="600" as="sourcePoint" /> + <Array as="points"> + <mxPoint x="1850" y="720" /> + <mxPoint x="1850" y="670" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-212" value="MESSAGE_CFG_APPLY" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="nUYlmBzm2YxJIW5L2hvB-211" vertex="1" connectable="0"> + <mxGeometry x="-0.5683" relative="1" as="geometry"> + <mxPoint x="-31" y="-10" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-219" value="batch: comm_phase =
PHASE_TXN_CREATE" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="1470" y="539.9974074074072" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-220" value="batch: comm_phase =
PHASE_APPLY_CFG" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;" parent="1" vertex="1" connectable="0"> + <mxGeometry x="1390" y="699.9974074074072" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-228" value="batch: PHASE_APPLY_CFG -> PHASE_TXN_DELETE" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-223" target="nUYlmBzm2YxJIW5L2hvB-227" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-223" value="mgmt_txn_notify_be_cfg_apply_reply
for each batch id in reply" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1750" y="290" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-226" value="mgmt_txn_send_be_txn_delete" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1750" y="420" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-229" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-227" target="nUYlmBzm2YxJIW5L2hvB-226" edge="1"> + <mxGeometry relative="1" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-233" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-227" target="nUYlmBzm2YxJIW5L2hvB-232" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1730" y="375" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-227" value="mgmt_move_txn_cfg_batch_to_next" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1750" y="360" width="167.5" height="30" as="geometry" /> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-230" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#ff0000;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;dashed=1;dashPattern=1 4;strokeWidth=2;entryX=-0.012;entryY=0.122;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-226" target="nUYlmBzm2YxJIW5L2hvB-161" edge="1"> + <mxGeometry relative="1" as="geometry"> + <mxPoint x="2055" y="699.96" as="targetPoint" /> + <mxPoint x="1700" y="470" as="sourcePoint" /> + <Array as="points"> + <mxPoint x="1870" y="625" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-231" value="MESSAGE_TXN_REQ
delete" style="edgeLabel;align=center;verticalAlign=middle;resizable=0;points=[];fontSize=8;fontFamily=Verdana;fontColor=default;labelBackgroundColor=none;horizontal=0;" parent="nUYlmBzm2YxJIW5L2hvB-230" vertex="1" connectable="0"> + <mxGeometry x="-0.5683" relative="1" as="geometry"> + <mxPoint x="-10" y="9" as="offset" /> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-234" value="" style="edgeStyle=orthogonalEdgeStyle;shape=connector;rounded=1;orthogonalLoop=1;jettySize=auto;labelBackgroundColor=none;strokeColor=#001DBC;fontFamily=Verdana;fontSize=8;fontColor=default;endArrow=open;startSize=8;endSize=8;strokeWidth=1;fillColor=#0050ef;" parent="1" source="nUYlmBzm2YxJIW5L2hvB-232" target="nUYlmBzm2YxJIW5L2hvB-138" edge="1"> + <mxGeometry relative="1" as="geometry"> + <Array as="points"> + <mxPoint x="1690" y="70" /> + </Array> + </mxGeometry> + </mxCell> + <mxCell id="nUYlmBzm2YxJIW5L2hvB-232" value="mgmt_try_move_commit_to_next_phase" style="rounded=1;whiteSpace=wrap;arcSize=24;fillColor=#ffffc0;strokeColor=#ff0000;shadow=0;comic=0;labelBackgroundColor=none;fontFamily=Verdana;fontSize=8;align=center;fontColor=default;" parent="1" vertex="1"> + <mxGeometry x="1560" y="420" width="180" height="30" as="geometry" /> + </mxCell> + </root> + </mxGraphModel> + </diagram> +</mxfile> diff --git a/doc/figures/cli-change-mgmtd.svg b/doc/figures/cli-change-mgmtd.svg new file mode 100644 index 0000000..279f74a --- /dev/null +++ b/doc/figures/cli-change-mgmtd.svg @@ -0,0 +1,3 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1048" height="781" viewBox="-0.5 -0.5 1048 781" style="background-color: rgb(255, 255, 255);"><defs><filter id="dropShadow"><feGaussianBlur in="SourceAlpha" stdDeviation="1.7" result="blur"/><feOffset in="blur" dx="3" dy="3" result="offsetBlur"/><feFlood flood-color="#3D4574" flood-opacity="0.4" result="offsetColor"/><feComposite in="offsetColor" in2="offsetBlur" operator="in" result="offsetBlur"/><feBlend in="SourceGraphic" in2="offsetBlur"/></filter></defs><g filter="url(#dropShadow)"><path d="M 275 274 L 475 274 Q 485 274 485 264 L 485 236.24" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 489.5 244.12 L 485 235.12 L 480.5 244.12" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><path d="M 200 284.22 L 200 294.11 Q 200 304 200 307.88 L 200 311.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 195.5 303.88 L 200 312.88 L 204.5 303.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="130" y="263.78" width="145" height="20.44" rx="4.91" ry="4.91" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 274px; margin-left: 131px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_prepare_cfg</div></div></div></foreignObject><text x="203" y="276" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_prepare_cfg</text></switch></g><path d="M 500 574 L 680 574 Q 690 574 690 584 L 690 634 Q 690 644 700 644 L 780.53 644" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 772.76 649 L 782.76 644 L 772.76 639" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="560" y="562">MESSAGE_TXN_REQ</text><text x="560" y="572">create</text></g><path d="M 275 574 L 330 574 Q 340 574 346.38 574 L 352.76 574" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 344.88 578.5 L 353.88 574 L 344.88 569.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="130" y="564" width="145" height="20" rx="4.8" ry="4.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 574px; margin-left: 131px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_be_txn_create</div></div></div></foreignObject><text x="203" y="576" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_be_txn_create</text></switch></g><rect x="355" y="565" width="145" height="20" rx="4.8" ry="4.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 575px; margin-left: 356px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_be_send_txn_req</div></div></div></foreignObject><text x="428" y="577" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_be_send_txn_req</text></switch></g><rect x="90" y="624" width="180" height="30" rx="7.2" ry="7.2" fill="#eeeeee" stroke="#36393d" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 178px; height: 1px; padding-top: 639px; margin-left: 91px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">does nothing<br />cfg_data replys will cause next transition<div><br /></div></div></div></div></foreignObject><text x="180" y="641" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">does nothing...</text></switch></g><path d="M 195 704 L 287.76 704" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 279.88 708.5 L 288.88 704 L 279.88 699.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="241" y="722">next_phase =</text><text x="241" y="732"> PHASE_TXN_DELETE</text></g><rect x="50" y="694" width="145" height="20" rx="4.8" ry="4.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 704px; margin-left: 51px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_be_cfg_apply</div></div></div></foreignObject><text x="123" y="706" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_be_cfg_apply</text></switch></g><rect x="290" y="694" width="145" height="20" rx="4.8" ry="4.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 704px; margin-left: 291px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_be_send_cfgapply_req</div></div></div></foreignObject><text x="363" y="706" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_be_send_cfgapply_req</text></switch></g><rect x="11.25" y="754" width="145" height="20" rx="4.8" ry="4.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 764px; margin-left: 12px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_commit_cfg_reply</div></div></div></foreignObject><text x="84" y="766" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_commit_cfg_reply</text></switch></g><path d="M 120 224 L 120 564 Q 120 574 123.88 574 L 127.76 574" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 119.88 578.5 L 128.88 574 L 119.88 569.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="167" y="558">PHASE_TXN_CREATE</text></g><path d="M 150 224 L 150 234 Q 150 244 160 244 L 193 244 Q 203 244 203 252.77 L 203 261.54" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 198.5 253.66 L 203 262.66 L 207.5 253.66" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="180" y="237">PHASE_PREPARE_CFG</text></g><path d="M 60 224 L 60 314 Q 60 324 60 334 L 60 691.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 55.5 683.88 L 60 692.88 L 64.5 683.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="110" y="687">PHASE_CFG_APPLY</text></g><path d="M 30 224 L 30 354 Q 30 364 30 374 L 30 751.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 25.5 743.88 L 30 752.88 L 34.5 743.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="84" y="747">PHASE_TXN_DELETE</text></g><path d="M 100 224 L 100 314 Q 100 324 100 334 L 100 621.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 95.5 613.88 L 100 622.88 L 104.5 613.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="140" y="616">PHASE_SEND_CFG</text></g><rect x="0" y="154" width="167.5" height="70" rx="16.8" ry="16.8" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 189px; margin-left: 1px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_process_commit_cfg</div></div></div></foreignObject><text x="84" y="191" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_process_commit_cfg</text></switch></g><path d="M 275 274 L 420 274 Q 430 274 430 264 L 430 96.24" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 434.5 104.12 L 430 95.12 L 425.5 104.12" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="330" y="252">curr_phase =</text><text x="330" y="262">PHASE_TXN_CREATE</text></g><rect x="0" y="34" width="167.5" height="40" rx="9.6" ry="9.6" fill="#ffe6cc" stroke="#d79b00" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 54px; margin-left: 1px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_commit_config_req</div></div></div></foreignObject><text x="84" y="56" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_commit_config_req</text></switch></g><path d="M 167.5 54 L 367.76 54" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 359.88 58.5 L 368.88 54 L 359.88 49.5" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="274.5" y="27">curr_phase ==</text><text x="274.5" y="37">MGMTD_COMMIT_PHASE_PREPARE_CFG</text></g><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="275" y="67">next_phase ==</text><text x="275" y="77">MGMTD_COMMIT_PHASE_PREPARE_CFG</text></g><path d="M 390 83.81 L 390 164 Q 390 174 380 174 L 169.74 174" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 177.62 169.5 L 168.62 174 L 177.62 178.5" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><ellipse cx="430" cy="54" rx="60" ry="40" fill="#b1ddf0" stroke="#10739e" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 118px; height: 1px; padding-top: 54px; margin-left: 371px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">TIMER:<br />MGMTD_TXN_PROC_COMCFG</div></div></div></foreignObject><text x="430" y="56" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">TIMER:...</text></switch></g><path d="M 202.5 284.22 L 202.5 289.11 Q 202.5 294 212.5 294 L 353 294 Q 363 294 363 302.88 L 363 311.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 358.5 303.88 L 363 312.88 L 367.5 303.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="130" y="314" width="145" height="50" rx="12" ry="12" fill="#e1d5e7" stroke="#9673a6" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 339px; margin-left: 131px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">GET nb_config_change's<br />nb_config_diff(cand, run)<br />or <br />txn->commit_cfg_req->req.commit_cfg.cfg_chgs<div><br /></div></div></div></div></foreignObject><text x="203" y="341" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">GET nb_config_change's...</text></switch></g><rect x="290" y="314" width="145" height="30" rx="7.2" ry="7.2" fill="#e1d5e7" stroke="#9673a6" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 143px; height: 1px; padding-top: 329px; margin-left: 291px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_create_config_batches</div></div></div></foreignObject><text x="363" y="331" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_create_config_batches</text></switch></g><ellipse cx="485" cy="204" rx="45" ry="30" fill="#b1ddf0" stroke="#10739e" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 88px; height: 1px; padding-top: 204px; margin-left: 441px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">TIMER:<br />MGMTD_TXN_<br />COMMITCFG_TIMEOUT</div></div></div></foreignObject><text x="485" y="206" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">TIMER:...</text></switch></g><path d="M 820 594 L 820 84 Q 820 74 814.87 74 L 809.74 74" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 789.74 74 L 799.74 69 L 799.74 79 Z M 799.74 74 L 809.74 69 L 809.74 79 Z" fill="#ff0000" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px" transform="rotate(-90,810.5,523.5)"><text x="810" y="526">MESSAGE_CFG_DATA_REPLY</text></g><path d="M 790 594 L 790 299 Q 790 289 780.99 289 L 771.97 289" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 779.74 284 L 769.74 289 L 779.74 294" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px" transform="rotate(-90,780.5,507.5)"><text x="780" y="510">MESSAGE_CFG_APPLY_REPLY;</text></g><rect x="785" y="594" width="205" height="120" rx="28.8" ry="28.8" fill="#cdeb8b" stroke="#36393d" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 203px; height: 1px; padding-top: 654px; margin-left: 786px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">Backend Client</div></div></div></foreignObject><text x="888" y="656" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">Backend Client</text></switch></g><path d="M 953.8 449 L 953.8 439 Q 953.8 429 953.8 439 L 953.8 444 Q 953.8 449 953.8 440.12 L 953.8 431.24" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 958.3 439.12 L 953.8 430.12 L 949.3 439.12" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="870" y="449" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 464px; margin-left: 871px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_notify_be_txn_reply</div></div></div></foreignObject><text x="954" y="466" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_notify_be_txn_reply</text></switch></g><path d="M 953.75 399 L 953.75 343.12" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 953.75 325.12 L 958.25 334.12 L 949.25 334.12 Z M 953.75 334.12 L 958.25 343.12 L 949.25 343.12 Z" fill="#ff0000" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="870" y="399" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 414px; margin-left: 871px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_be_cfg_data</div></div></div></foreignObject><text x="954" y="416" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_be_cfg_data</text></switch></g><path d="M 953.8 294 L 953.8 284 Q 953.8 274 953.8 281.5 L 953.8 285.25 Q 953.8 289 953.8 280.12 L 953.8 271.24" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 958.3 279.12 L 953.8 270.12 L 949.3 279.12" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="944.8" y="225">batch: PHASE_TXN_REQ -> PHASE_SEND_CFG</text></g><rect x="875" y="294" width="157.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 156px; height: 1px; padding-top: 309px; margin-left: 876px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_be_send_cfgdata_req</div></div></div></foreignObject><text x="954" y="311" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_be_send_cfgdata_req</text></switch></g><path d="M 945 239 L 945 231.5 Q 945 224 945 215.12 L 945 206.24" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 949.5 214.12 L 945 205.12 L 940.5 214.12" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="939.5" y="126.5">txn: PHASE_TXN_REQ -> PHASE_SEND_CFG</text></g><rect x="875" y="239" width="157.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 156px; height: 1px; padding-top: 254px; margin-left: 876px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_move_txn_cfg_batch_to_next</div></div></div></foreignObject><text x="954" y="256" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_move_txn_cfg_batch_to_next</text></switch></g><path d="M 945 144 L 945 24 Q 945 14 935 14 L 432.24 14" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 440.12 9.5 L 431.12 14 L 440.12 18.5" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="740" y="7">curr_phase = PHASE_SEND_CFG</text></g><rect x="852.5" y="144" width="185" height="60" rx="14.4" ry="14.4" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 183px; height: 1px; padding-top: 174px; margin-left: 854px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_try_move_commit_to_next_phase<br />if all backend clients<br />have all been sent their batches<br />move to next phase and post EVENT/TIMER</div></div></div></foreignObject><text x="945" y="176" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_try_move_commit_to_next_phase...</text></switch></g><path d="M 875 309 L 860 309 Q 850 309 850 319 L 850 571.76" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 850 591.76 L 845 581.76 L 855 581.76 Z M 850 581.76 L 845 571.76 L 855 571.76 Z" fill="#ff0000" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px" transform="rotate(-90,840.5,380.5)"><text x="840" y="383">MESSAGE_CFG_DATA_REQ</text></g><path d="M 703.75 89 L 703.75 141.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 699.25 133.88 L 703.75 142.88 L 708.25 133.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="703.75" y="123">batch: PHASE_SEND_CFG -> PHASE_APPLY_CFG</text></g><rect x="620" y="59" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 74px; margin-left: 621px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_notify_be_cfgdata_reply</div></div></div></foreignObject><text x="704" y="76" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_notify_be_cfgdata_reply</text></switch></g><path d="M 703.8 174 L 703.8 184 Q 703.8 194 703.8 197.88 L 703.8 201.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 699.3 193.88 L 703.8 202.88 L 708.3 193.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="620" y="144" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 159px; margin-left: 621px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_move_txn_cfg_batch_to_next</div></div></div></foreignObject><text x="704" y="161" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_move_txn_cfg_batch_to_next</text></switch></g><path d="M 613.75 219 L 580 219 Q 570 219 570 209 L 570 44 Q 570 34 560 34 L 484.2 34" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 492.08 29.5 L 483.08 34 L 492.08 38.5" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="620" y="246.75">curr_phase = PHASE_APPLY_CFG</text></g><rect x="613.75" y="204" width="180" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 178px; height: 1px; padding-top: 219px; margin-left: 615px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_try_move_commit_to_next_phase</div></div></div></foreignObject><text x="704" y="221" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_try_move_commit_to_next_phase</text></switch></g><path d="M 954 594 L 954 574 Q 954 564 954 554 L 954 483.47" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 959 491.24 L 954 481.24 L 949 491.24" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px" transform="rotate(-90,940.5,539.5)"><text x="940" y="542">MESSAGE_TXN_REPLY</text></g><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="320" y="552">curr_phase =</text><text x="320" y="562">PHASE_TXN_CREATE</text></g><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="315" y="589">next_phase =</text><text x="315" y="599">PHASE_SEND_CFG</text></g><path d="M 896.96 356.21 C 891.58 352.95 884.78 353.59 880.07 357.81 C 875.37 362.03 873.83 368.87 876.25 374.79 C 878.67 380.7 884.51 384.35 890.73 383.85 C 896.96 383.35 902.17 378.8 903.66 372.57" fill="none" stroke="rgb(0, 0, 0)" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><path d="M 902.21 372.57 L 904.38 368.85 L 905 373.31 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px"><text x="906.5" y="361.5">For each of the</text><text x="906.5" y="371.5">batches of cfgdata</text><text x="906.5" y="381.5">send msg and move</text></g><path d="M 806.96 41.21 C 801.58 37.95 794.78 38.59 790.07 42.81 C 785.37 47.03 783.83 53.87 786.25 59.79 C 788.67 65.7 794.51 69.35 800.73 68.85 C 806.96 68.35 812.17 63.8 813.66 57.57" fill="none" stroke="rgb(0, 0, 0)" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><path d="M 812.21 57.57 L 814.38 53.85 L 815 58.31 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px"><text x="816.5" y="51.5">For each of the</text><text x="816.5" y="61.5"> batches of cfgdata</text></g><path d="M 435 704 L 690 704 Q 700 704 700 694 L 700 664 Q 700 654 710 654 L 780.53 654" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 772.76 659 L 782.76 654 L 772.76 649" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="490" y="697">MESSAGE_CFG_APPLY</text></g><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="320" y="522">batch: comm_phase =</text><text x="320" y="532">PHASE_TXN_CREATE</text></g><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="240" y="682">batch: comm_phase =</text><text x="240" y="692">PHASE_APPLY_CFG</text></g><path d="M 683.75 304 L 683.75 341.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 679.25 333.88 L 683.75 342.88 L 688.25 333.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px"><text x="683.25" y="326.5">batch: PHASE_APPLY_CFG -> PHASE_TXN_DELETE</text></g><rect x="600" y="274" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 289px; margin-left: 601px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_notify_be_cfg_apply_reply<br />for each batch id in reply</div></div></div></foreignObject><text x="684" y="291" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_notify_be_cfg_apply_reply...</text></switch></g><rect x="600" y="404" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 419px; margin-left: 601px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_txn_send_be_txn_delete</div></div></div></foreignObject><text x="684" y="421" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_txn_send_be_txn_delete</text></switch></g><path d="M 683.8 374 L 683.8 384 Q 683.8 394 683.8 389 L 683.8 386.5 Q 683.8 384 683.8 392.88 L 683.8 401.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 679.3 393.88 L 683.8 402.88 L 688.3 393.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><path d="M 600 359 L 590 359 Q 580 359 580 369 L 580 401.76" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 575.5 393.88 L 580 402.88 L 584.5 393.88" fill="none" stroke="#ff0000" stroke-miterlimit="10" pointer-events="all"/><rect x="600" y="344" width="167.5" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 166px; height: 1px; padding-top: 359px; margin-left: 601px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_move_txn_cfg_batch_to_next</div></div></div></foreignObject><text x="684" y="361" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_move_txn_cfg_batch_to_next</text></switch></g><path d="M 720 434 L 720 598.6 Q 720 608.6 730 608.61 L 778.07 608.64" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" stroke-dasharray="2 8" pointer-events="stroke"/><path d="M 770.3 613.63 L 780.3 608.64 L 770.31 603.63" fill="none" stroke="#ff0000" stroke-width="2" stroke-miterlimit="10" pointer-events="all"/><g fill="rgb(0, 0, 0)" font-family="Verdana" text-anchor="middle" font-size="8px" transform="rotate(-90,710.5,493.5)"><text x="710" y="491">MESSAGE_TXN_REQ</text><text x="710" y="501">delete</text></g><path d="M 540 404 L 540 64 Q 540 54 530 54 L 492.24 54" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="stroke"/><path d="M 500.12 49.5 L 491.12 54 L 500.12 58.5" fill="none" stroke="#001dbc" stroke-miterlimit="10" pointer-events="all"/><rect x="410" y="404" width="180" height="30" rx="7.2" ry="7.2" fill="#ffffc0" stroke="#ff0000" pointer-events="all"/><g transform="translate(-0.5 -0.5)"><switch><foreignObject pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility" style="overflow: visible; text-align: left;"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 178px; height: 1px; padding-top: 419px; margin-left: 411px;"><div data-drawio-colors="color: rgb(0, 0, 0); " style="box-sizing: border-box; font-size: 0px; text-align: center;"><div style="display: inline-block; font-size: 8px; font-family: Verdana; color: rgb(0, 0, 0); line-height: 1.2; pointer-events: all; white-space: normal; overflow-wrap: normal;">mgmt_try_move_commit_to_next_phase</div></div></div></foreignObject><text x="500" y="421" fill="rgb(0, 0, 0)" font-family="Verdana" font-size="8px" text-anchor="middle">mgmt_try_move_commit_to_next_phase</text></switch></g></g><switch><g requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"/><a transform="translate(0,-5)" xlink:href="https://www.drawio.com/doc/faq/svg-export-text-problems" target="_blank"><text text-anchor="middle" font-size="10px" x="50%" y="100%">Text is not SVG - cannot display</text></a></switch></svg>
\ No newline at end of file diff --git a/doc/figures/cligraph.png b/doc/figures/cligraph.png Binary files differnew file mode 100644 index 0000000..96b84e8 --- /dev/null +++ b/doc/figures/cligraph.png diff --git a/doc/figures/cligraph.svg b/doc/figures/cligraph.svg new file mode 100644 index 0000000..a1dd017 --- /dev/null +++ b/doc/figures/cligraph.svg @@ -0,0 +1,211 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by graphviz version 2.38.0 (20140413.2041) + --> +<!-- Title: %3 Pages: 1 --> +<svg width="300pt" height="980pt" + viewBox="0.00 0.00 299.50 980.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> +<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 976)"> +<title>%3</title> +<polygon fill="white" stroke="none" points="-4,4 -4,-976 295.5,-976 295.5,4 -4,4"/> +<!-- n0xd46960 --> +<g id="node1" class="node"><title>n0xd46960</title> +<polygon fill="#ccffcc" stroke="black" points="158,-972 86,-972 86,-936 158,-936 158,-972"/> +<text text-anchor="start" x="94" y="-952.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">START_TKN</text> +</g> +<!-- n0xd46be0 --> +<g id="node2" class="node"><title>n0xd46be0</title> +<polygon fill="#ffffff" stroke="black" points="159,-900 85,-900 85,-864 159,-864 159,-900"/> +<text text-anchor="start" x="93" y="-885.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">WORD_TKN</text> +<text text-anchor="start" x="101.5" y="-875.2" font-family="Fira Mono" font-size="9.00">"</text> +<text text-anchor="start" x="105.5" y="-875.2" font-family="Fira Mono" font-weight="bold" font-size="11.00" fill="#0055ff">show</text> +<text text-anchor="start" x="138.5" y="-875.2" font-family="Fira Mono" font-size="9.00">"</text> +</g> +<!-- n0xd46960->n0xd46be0 --> +<g id="edge1" class="edge"><title>n0xd46960->n0xd46be0</title> +<path fill="none" stroke="black" d="M122,-935.697C122,-927.983 122,-918.712 122,-910.112"/> +<polygon fill="black" stroke="black" points="125.5,-910.104 122,-900.104 118.5,-910.104 125.5,-910.104"/> +</g> +<!-- n0xd47f80 --> +<g id="node3" class="node"><title>n0xd47f80</title> +<polygon fill="#aaddff" stroke="black" points="156.5,-828 87.5,-828 87.5,-792 156.5,-792 156.5,-828"/> +<text text-anchor="start" x="95.5" y="-808.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">FORK_TKN</text> +</g> +<!-- n0xd46be0->n0xd47f80 --> +<g id="edge2" class="edge"><title>n0xd46be0->n0xd47f80</title> +<path fill="none" stroke="black" d="M122,-863.697C122,-855.983 122,-846.712 122,-838.112"/> +<polygon fill="black" stroke="black" points="125.5,-838.104 122,-828.104 118.5,-838.104 125.5,-838.104"/> +</g> +<!-- n0xd47c70 --> +<g id="node4" class="node"><title>n0xd47c70</title> +<polygon fill="#ffffff" stroke="black" points="127,-756 53,-756 53,-720 127,-720 127,-756"/> +<text text-anchor="start" x="61" y="-741.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">WORD_TKN</text> +<text text-anchor="start" x="80.5" y="-731.2" font-family="Fira Mono" font-size="9.00">"</text> +<text text-anchor="start" x="84.5" y="-731.2" font-family="Fira Mono" font-weight="bold" font-size="11.00" fill="#0055ff">ip</text> +<text text-anchor="start" x="95.5" y="-731.2" font-family="Fira Mono" font-size="9.00">"</text> +</g> +<!-- n0xd47f80->n0xd47c70 --> +<g id="edge3" class="edge"><title>n0xd47f80->n0xd47c70</title> +<path fill="none" stroke="black" d="M114.09,-791.697C110.447,-783.728 106.046,-774.1 102.006,-765.264"/> +<polygon fill="black" stroke="black" points="105.16,-763.744 97.8191,-756.104 98.7936,-766.654 105.16,-763.744"/> +</g> +<!-- n0xd484c0 --> +<g id="node5" class="node"><title>n0xd484c0</title> +<polygon fill="#ddaaff" stroke="black" points="153.5,-684 90.5,-684 90.5,-648 153.5,-648 153.5,-684"/> +<text text-anchor="start" x="98.5" y="-664.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">JOIN_TKN</text> +</g> +<!-- n0xd47f80->n0xd484c0 --> +<g id="edge20" class="edge"><title>n0xd47f80->n0xd484c0</title> +<path fill="none" stroke="black" d="M127.824,-791.56C130.931,-781.33 134.431,-768.08 136,-756 138.06,-740.133 138.06,-735.867 136,-720 134.897,-711.506 132.839,-702.434 130.634,-694.24"/> +<polygon fill="black" stroke="black" points="133.945,-693.087 127.824,-684.44 127.216,-695.017 133.945,-693.087"/> +</g> +<!-- n0xd47c70->n0xd484c0 --> +<g id="edge4" class="edge"><title>n0xd47c70->n0xd484c0</title> +<path fill="none" stroke="black" d="M97.9101,-719.697C101.553,-711.728 105.954,-702.1 109.994,-693.264"/> +<polygon fill="black" stroke="black" points="113.206,-694.654 114.181,-684.104 106.84,-691.744 113.206,-694.654"/> +</g> +<!-- n0xd47ca0 --> +<g id="node6" class="node"><title>n0xd47ca0</title> +<polygon fill="#ffffff" stroke="black" points="159,-612 85,-612 85,-576 159,-576 159,-612"/> +<text text-anchor="start" x="93" y="-597.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">WORD_TKN</text> +<text text-anchor="start" x="106.5" y="-587.2" font-family="Fira Mono" font-size="9.00">"</text> +<text text-anchor="start" x="110.5" y="-587.2" font-family="Fira Mono" font-weight="bold" font-size="11.00" fill="#0055ff">bgp</text> +<text text-anchor="start" x="133.5" y="-587.2" font-family="Fira Mono" font-size="9.00">"</text> +</g> +<!-- n0xd484c0->n0xd47ca0 --> +<g id="edge5" class="edge"><title>n0xd484c0->n0xd47ca0</title> +<path fill="none" stroke="black" d="M122,-647.697C122,-639.983 122,-630.712 122,-622.112"/> +<polygon fill="black" stroke="black" points="125.5,-622.104 122,-612.104 118.5,-622.104 125.5,-622.104"/> +</g> +<!-- n0xd48540 --> +<g id="node7" class="node"><title>n0xd48540</title> +<polygon fill="#ffffff" stroke="black" points="164.5,-540 79.5,-540 79.5,-504 164.5,-504 164.5,-540"/> +<text text-anchor="start" x="93" y="-525.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">WORD_TKN</text> +<text text-anchor="start" x="87.5" y="-515.2" font-family="Fira Mono" font-size="9.00">"</text> +<text text-anchor="start" x="91.5" y="-515.2" font-family="Fira Mono" font-weight="bold" font-size="11.00" fill="#0055ff">neighbors</text> +<text text-anchor="start" x="152.5" y="-515.2" font-family="Fira Mono" font-size="9.00">"</text> +</g> +<!-- n0xd47ca0->n0xd48540 --> +<g id="edge6" class="edge"><title>n0xd47ca0->n0xd48540</title> +<path fill="none" stroke="black" d="M122,-575.697C122,-567.983 122,-558.712 122,-550.112"/> +<polygon fill="black" stroke="black" points="125.5,-550.104 122,-540.104 118.5,-550.104 125.5,-550.104"/> +</g> +<!-- n0xd490c0 --> +<g id="node8" class="node"><title>n0xd490c0</title> +<polygon fill="#aaddff" stroke="black" points="156.5,-468 87.5,-468 87.5,-432 156.5,-432 156.5,-468"/> +<text text-anchor="start" x="95.5" y="-448.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">FORK_TKN</text> +</g> +<!-- n0xd48540->n0xd490c0 --> +<g id="edge7" class="edge"><title>n0xd48540->n0xd490c0</title> +<path fill="none" stroke="black" d="M122,-503.697C122,-495.983 122,-486.712 122,-478.112"/> +<polygon fill="black" stroke="black" points="125.5,-478.104 122,-468.104 118.5,-478.104 125.5,-478.104"/> +</g> +<!-- n0xd48fc0 --> +<g id="node9" class="node"><title>n0xd48fc0</title> +<polygon fill="#ffffff" stroke="black" points="64,-396 0,-396 0,-360 64,-360 64,-396"/> +<text text-anchor="start" x="8" y="-380.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">IPV4_TKN</text> +<text text-anchor="start" x="15" y="-371.8" font-family="Fira Mono" font-size="9.00">A.B.C.D</text> +</g> +<!-- n0xd490c0->n0xd48fc0 --> +<g id="edge8" class="edge"><title>n0xd490c0->n0xd48fc0</title> +<path fill="none" stroke="black" d="M99.7528,-431.697C88.4181,-422.881 74.4698,-412.032 62.1811,-402.474"/> +<polygon fill="black" stroke="black" points="64.0336,-399.481 53.9913,-396.104 59.736,-405.007 64.0336,-399.481"/> +</g> +<!-- n0xd491e0 --> +<g id="node10" class="node"><title>n0xd491e0</title> +<polygon fill="#ddaaff" stroke="black" points="153.5,-324 90.5,-324 90.5,-288 153.5,-288 153.5,-324"/> +<text text-anchor="start" x="98.5" y="-304.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">JOIN_TKN</text> +</g> +<!-- n0xd490c0->n0xd491e0 --> +<g id="edge19" class="edge"><title>n0xd490c0->n0xd491e0</title> +<path fill="none" stroke="black" d="M117.536,-431.953C115.065,-421.63 112.248,-408.153 111,-396 109.366,-380.084 109.366,-375.916 111,-360 111.877,-351.455 113.531,-342.255 115.294,-333.958"/> +<polygon fill="black" stroke="black" points="118.743,-334.573 117.536,-324.047 111.915,-333.028 118.743,-334.573"/> +</g> +<!-- n0xd49340 --> +<g id="node15" class="node"><title>n0xd49340</title> +<polygon fill="#ffffff" stroke="black" points="184,-396 120,-396 120,-360 184,-360 184,-396"/> +<text text-anchor="start" x="128" y="-380.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">IPV6_TKN</text> +<text text-anchor="start" x="135" y="-371.8" font-family="Fira Mono" font-size="9.00">X:X::X:X</text> +</g> +<!-- n0xd490c0->n0xd49340 --> +<g id="edge15" class="edge"><title>n0xd490c0->n0xd49340</title> +<path fill="none" stroke="black" d="M129.416,-431.697C132.794,-423.813 136.87,-414.304 140.623,-405.546"/> +<polygon fill="black" stroke="black" points="143.947,-406.675 144.67,-396.104 137.513,-403.917 143.947,-406.675"/> +</g> +<!-- n0xd49480 --> +<g id="node16" class="node"><title>n0xd49480</title> +<polygon fill="#ffffff" stroke="black" points="291.5,-396 202.5,-396 202.5,-360 291.5,-360 291.5,-396"/> +<text text-anchor="start" x="210.5" y="-380.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">VARIABLE_TKN</text> +<text text-anchor="start" x="233" y="-371.8" font-family="Fira Mono" font-size="9.00">WORD</text> +</g> +<!-- n0xd490c0->n0xd49480 --> +<g id="edge17" class="edge"><title>n0xd490c0->n0xd49480</title> +<path fill="none" stroke="black" d="M152.578,-431.876C169.074,-422.639 189.624,-411.131 207.336,-401.212"/> +<polygon fill="black" stroke="black" points="209.289,-404.13 216.304,-396.19 205.869,-398.022 209.289,-404.13"/> +</g> +<!-- n0xd48fc0->n0xd491e0 --> +<g id="edge9" class="edge"><title>n0xd48fc0->n0xd491e0</title> +<path fill="none" stroke="black" d="M54.2472,-359.697C65.5819,-350.881 79.5302,-340.032 91.8189,-330.474"/> +<polygon fill="black" stroke="black" points="94.264,-333.007 100.009,-324.104 89.9664,-327.481 94.264,-333.007"/> +</g> +<!-- n0xd496e0 --> +<g id="node11" class="node"><title>n0xd496e0</title> +<polygon fill="#aaddff" stroke="black" points="156.5,-252 87.5,-252 87.5,-216 156.5,-216 156.5,-252"/> +<text text-anchor="start" x="95.5" y="-232.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">FORK_TKN</text> +</g> +<!-- n0xd491e0->n0xd496e0 --> +<g id="edge10" class="edge"><title>n0xd491e0->n0xd496e0</title> +<path fill="none" stroke="black" d="M122,-287.697C122,-279.983 122,-270.712 122,-262.112"/> +<polygon fill="black" stroke="black" points="125.5,-262.104 122,-252.104 118.5,-262.104 125.5,-262.104"/> +</g> +<!-- n0xd495e0 --> +<g id="node12" class="node"><title>n0xd495e0</title> +<polygon fill="#ffffff" stroke="black" points="127,-180 53,-180 53,-144 127,-144 127,-180"/> +<text text-anchor="start" x="61" y="-165.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">WORD_TKN</text> +<text text-anchor="start" x="73.5" y="-155.2" font-family="Fira Mono" font-size="9.00">"</text> +<text text-anchor="start" x="77.5" y="-155.2" font-family="Fira Mono" font-weight="bold" font-size="11.00" fill="#0055ff">json</text> +<text text-anchor="start" x="102.5" y="-155.2" font-family="Fira Mono" font-size="9.00">"</text> +</g> +<!-- n0xd496e0->n0xd495e0 --> +<g id="edge11" class="edge"><title>n0xd496e0->n0xd495e0</title> +<path fill="none" stroke="black" d="M114.09,-215.697C110.447,-207.728 106.046,-198.1 102.006,-189.264"/> +<polygon fill="black" stroke="black" points="105.16,-187.744 97.8191,-180.104 98.7936,-190.654 105.16,-187.744"/> +</g> +<!-- n0xd497c0 --> +<g id="node13" class="node"><title>n0xd497c0</title> +<polygon fill="#ddaaff" stroke="black" points="153.5,-108 90.5,-108 90.5,-72 153.5,-72 153.5,-108"/> +<text text-anchor="start" x="98.5" y="-88.8" font-family="Fira Mono" font-weight="bold" font-size="9.00">JOIN_TKN</text> +</g> +<!-- n0xd496e0->n0xd497c0 --> +<g id="edge14" class="edge"><title>n0xd496e0->n0xd497c0</title> +<path fill="none" stroke="black" d="M127.824,-215.56C130.931,-205.33 134.431,-192.08 136,-180 138.06,-164.133 138.06,-159.867 136,-144 134.897,-135.506 132.839,-126.434 130.634,-118.24"/> +<polygon fill="black" stroke="black" points="133.945,-117.087 127.824,-108.44 127.216,-119.017 133.945,-117.087"/> +</g> +<!-- n0xd495e0->n0xd497c0 --> +<g id="edge12" class="edge"><title>n0xd495e0->n0xd497c0</title> +<path fill="none" stroke="black" d="M97.9101,-143.697C101.553,-135.728 105.954,-126.1 109.994,-117.264"/> +<polygon fill="black" stroke="black" points="113.206,-118.654 114.181,-108.104 106.84,-115.744 113.206,-118.654"/> +</g> +<!-- end0xd49900 --> +<g id="node14" class="node"><title>end0xd49900</title> +<polygon fill="#ffddaa" stroke="black" points="149,-36 95,-36 95,-0 149,-0 149,-36"/> +<text text-anchor="start" x="112.5" y="-15.8" font-family="Fira Mono" font-size="9.00">end</text> +</g> +<!-- n0xd497c0->end0xd49900 --> +<g id="edge13" class="edge"><title>n0xd497c0->end0xd49900</title> +<path fill="none" stroke="black" d="M122,-71.6966C122,-63.9827 122,-54.7125 122,-46.1124"/> +<polygon fill="black" stroke="black" points="125.5,-46.1043 122,-36.1043 118.5,-46.1044 125.5,-46.1043"/> +</g> +<!-- n0xd49340->n0xd491e0 --> +<g id="edge16" class="edge"><title>n0xd49340->n0xd491e0</title> +<path fill="none" stroke="black" d="M144.584,-359.697C141.206,-351.813 137.13,-342.304 133.377,-333.546"/> +<polygon fill="black" stroke="black" points="136.487,-331.917 129.33,-324.104 130.053,-334.675 136.487,-331.917"/> +</g> +<!-- n0xd49480->n0xd491e0 --> +<g id="edge18" class="edge"><title>n0xd49480->n0xd491e0</title> +<path fill="none" stroke="black" d="M216.422,-359.876C199.926,-350.639 179.376,-339.131 161.664,-329.212"/> +<polygon fill="black" stroke="black" points="163.131,-326.022 152.696,-324.19 159.711,-332.13 163.131,-326.022"/> +</g> +</g> +</svg> diff --git a/doc/figures/fig-normal-processing.dia b/doc/figures/fig-normal-processing.dia new file mode 100644 index 0000000..c9e8e68 --- /dev/null +++ b/doc/figures/fig-normal-processing.dia @@ -0,0 +1,1738 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#A4#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="5"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="5"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true"> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="7.6,4.75"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.55,4.7;16.45,9.4"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="7.6,4.75"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="8.8000000000000007"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4.5999999999999996"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#8f8f8f"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="20,2.95607"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.8989,2.49893;24.1055,3.50107"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20,2.95607"/> + <dia:point val="24,3"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O38" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="0,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.1,2.5;4.1,3.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="0,3"/> + <dia:point val="4,3"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O36" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="15.9914,5.70858"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.8513,2.57356;18.5463,5.84872"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="15.9914,5.70858"/> + <dia:point val="18.0879,2.95607"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O7" connection="2"/> + <dia:connection handle="1" to="O38" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="6,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.86018,2.86018;8.42045,6.03565"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,3"/> + <dia:point val="7.95858,5.65858"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O36" connection="4"/> + <dia:connection handle="1" to="O5" connection="0"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="7.9,5.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.85,5.55;11.95,8.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="7.9,5.6"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="9.9,6.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.725,6;11.075,8.55"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Best +Path +Selection#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="9.9,6.6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="13.45,5.65"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.4,5.6;16.1,8.55"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="13.45,5.65"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2.6000000000000147"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2.8500000000000005"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="14.75,6.8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.075,6.2;15.425,7.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Local +RIB#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.75,6.8"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="6,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.86335,5.86335;8.23706,7.58282"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,6"/> + <dia:point val="7.9,7.1"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O34" connection="4"/> + <dia:connection handle="1" to="O5" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="20,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.9,5.5;24.1,6.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20,6"/> + <dia:point val="24,6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O40" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="16.05,7.075"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.9141,5.51385;18.329,7.21085"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.05,7.075"/> + <dia:point val="18,6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O7" connection="4"/> + <dia:connection handle="1" to="O40" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="0,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.1,5.5;4.1,6.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="0,6"/> + <dia:point val="4,6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O34" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="0,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.1,8.5;4.1,9.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="0,9"/> + <dia:point val="4,9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O32" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="20,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.9,8.5;24.1,9.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20,9"/> + <dia:point val="24,9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O42" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="16.05,7.075"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.9086,6.93358;18.4224,9.42608"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.05,7.075"/> + <dia:point val="18,9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O7" connection="4"/> + <dia:connection handle="1" to="O42" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="6,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.85858,6.67574;8.32426,9.14142"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,9"/> + <dia:point val="7.9,7.1"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O32" connection="4"/> + <dia:connection handle="1" to="O5" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="20,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.9,11.5;24.1,12.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20,12"/> + <dia:point val="24,12"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O44" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="15.9914,8.44142"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.8552,8.30518;18.4846,12.3329"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="15.9914,8.44142"/> + <dia:point val="18,12"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O7" connection="7"/> + <dia:connection handle="1" to="O44" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="6,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.86371,8.20802;8.44294,12.1363"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,12"/> + <dia:point val="7.95858,8.54142"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O30" connection="4"/> + <dia:connection handle="1" to="O5" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="0,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.1,11.5;4.1,12.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="0,12"/> + <dia:point val="4,12"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O30" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="-5,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-5,2.22625;-0.82625,3.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From Peer A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-5,3"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="-5,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-5,5.22625;-0.82625,6.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From Peer B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-5,6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="-5,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-5,8.22625;-0.77625,9.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From Peer C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-5,9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="-5,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-5,11.2263;-0.77625,12.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From Peer D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-5,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="24,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24,2.22625;27.3238,3.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To Peer A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24,3"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="24,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24,5.22625;27.3238,6.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To Peer B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24,6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="24,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24,8.22625;27.3738,9.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To Peer C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24,9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="24,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24,11.2263;27.3738,12.4975"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To Peer D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="11.9,7.1"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.7476,6.07271;13.6161,8.07729"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="11.9,7.1"/> + <dia:point val="13.45,7.075"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="3"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.99999999999999967"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="4"/> + <dia:connection handle="1" to="O7" connection="3"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="4,11"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,10.95;6.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,11"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,12.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,11.3;5.4,12.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,12.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O32"> + <dia:attribute name="obj_pos"> + <dia:point val="4,8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,7.95;6.05,10.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,8"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O33"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,9.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,8.3;5.4,9.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,9.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O34"> + <dia:attribute name="obj_pos"> + <dia:point val="4,5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,4.95;6.05,7.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O35"> + <dia:attribute name="obj_pos"> + <dia:point val="4.7,6.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.7,5.3;5.4,6.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.7,6.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O36"> + <dia:attribute name="obj_pos"> + <dia:point val="4,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,1.95;6.05,4.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O37"> + <dia:attribute name="obj_pos"> + <dia:point val="4.75,3.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.75,2.3;5.45,3.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.75,3.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O38"> + <dia:attribute name="obj_pos"> + <dia:point val="18.0879,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.0379,1.95;20.05,3.96213"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="18.0879,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559645"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O39"> + <dia:attribute name="obj_pos"> + <dia:point val="18.65,3.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.65,2.4;19.35,4.05"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="18.65,3.45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O40"> + <dia:attribute name="obj_pos"> + <dia:point val="18,5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.95,4.95;20.05,7.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="18,5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O41"> + <dia:attribute name="obj_pos"> + <dia:point val="18.7,6.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.7,5.4;19.4,7.05"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="18.7,6.45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O42"> + <dia:attribute name="obj_pos"> + <dia:point val="18,8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.95,7.95;20.05,10.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="18,8"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O43"> + <dia:attribute name="obj_pos"> + <dia:point val="18.6,9.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.6,8.4;19.35,10.05"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="18.6,9.45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O44"> + <dia:attribute name="obj_pos"> + <dia:point val="18.0879,11"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.0379,10.95;20.05,12.9621"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="18.0879,11"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559649"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O45"> + <dia:attribute name="obj_pos"> + <dia:point val="18.7,12.4"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.7,11.35;19.45,13"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="18.7,12.4"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O46"> + <dia:attribute name="obj_pos"> + <dia:point val="13,15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="12.95,14.95;22.05,19.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="13,15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O47"> + <dia:attribute name="obj_pos"> + <dia:point val="14,16"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.95,15.95;15.9621,17.9621"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14,16"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559649"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O48"> + <dia:attribute name="obj_pos"> + <dia:point val="14.6121,17.4"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.6121,16.35;15.3121,18"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.6121,17.4"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Text" version="0" id="O49"> + <dia:attribute name="obj_pos"> + <dia:point val="19,17"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.525,15.95;21.475,19"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#“Out” Filter +for Peer X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19,17"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O50"> + <dia:attribute name="obj_pos"> + <dia:point val="3,15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.95,14.95;12.05,19.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="3,15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O51"> + <dia:attribute name="obj_pos"> + <dia:point val="9,17"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9,16.4;9,17.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="9,17"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O52"> + <dia:attribute name="obj_pos"> + <dia:point val="9,17"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="6.675,15.95;11.325,19"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#“In” Filter +for Peer X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="9,17"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O53"> + <dia:attribute name="obj_pos"> + <dia:point val="4,16"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,15.95;6.05,18.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,16"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O54"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,17.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,16.3;5.35,17.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,17.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-normal-processing.png b/doc/figures/fig-normal-processing.png Binary files differnew file mode 100644 index 0000000..e4b8fdc --- /dev/null +++ b/doc/figures/fig-normal-processing.png diff --git a/doc/figures/fig-normal-processing.txt b/doc/figures/fig-normal-processing.txt new file mode 100644 index 0000000..01f0e17 --- /dev/null +++ b/doc/figures/fig-normal-processing.txt @@ -0,0 +1,11 @@ + + _______________________________ + / _________ _________ \ +From Peer A --->|(A)-|Best | | |-[A]|--->To Peer A +From Peer B --->|(B)-|Path |-->|Local-RIB|-[B]|--->To Peer B +From Peer C --->|(C)-|Selection| | |-[C]|--->To Peer C +From Peer D --->|(D)-|_________| |_________|-[D]|--->To Peer D + \_______________________________/ + +Key: (X) - 'In' Filter applied to Peer X's announcements + [X] - 'Out' Filter applied to announcements to Peer X diff --git a/doc/figures/fig-rs-processing.dia b/doc/figures/fig-rs-processing.dia new file mode 100644 index 0000000..b2bf213 --- /dev/null +++ b/doc/figures/fig-rs-processing.dia @@ -0,0 +1,4239 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#A4#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true"> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="11.95,5.25"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.9,5.2;20.8,9.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="11.95,5.25"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="8.8000000000000007"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4.5999999999999996"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#8f8f8f"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="12.25,6.1"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="12.2,6.05;16.3,9.15"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12.25,6.1"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="14.25,7.1"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.075,6.5;15.425,9.05"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Best +Path +Selection#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.25,7.1"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="16.25,7.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.0976,6.57271;17.9661,8.57729"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.25,7.6"/> + <dia:point val="17.8,7.575"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="3"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.99999999999999967"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="17.8,6.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.75,6.1;20.45,9.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.8,6.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2.6000000000000147"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2.8500000000000005"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="19.15,7.4"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.125,6.8;20.175,8.55"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Main +Loc-RIB#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.15,7.4"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:group> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="-1,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-1.1,2.5;4.1,3.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="-1,3"/> + <dia:point val="4,3"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="6"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O23" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="20.4,7.575"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="20.2995,7.04947;25.1906,8.05053"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20.4,7.575"/> + <dia:point val="25.0879,7.55"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="4"/> + <dia:connection handle="1" to="O129" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="6,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.86581,2.86581;12.6219,6.65044"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,3"/> + <dia:point val="12.3086,6.15858"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="4"/> + <dia:connection handle="1" to="O1" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="6,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.87832,5.87832;12.4709,8.10918"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,6"/> + <dia:point val="12.25,7.6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O21" connection="4"/> + <dia:connection handle="1" to="O1" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="-1,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-1.1,5.5;4.1,6.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="-1,6"/> + <dia:point val="4,6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O21" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="-1,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-1.1,8.5;4.1,9.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="-1,9"/> + <dia:point val="4,9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O19" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="6,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.88056,7.09023;12.4569,9.11944"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,9"/> + <dia:point val="12.25,7.6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O19" connection="4"/> + <dia:connection handle="1" to="O1" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="6,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.867,8.54627;12.6114,12.133"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,12"/> + <dia:point val="12.3086,9.04142"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="4"/> + <dia:connection handle="1" to="O1" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="-1,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-1.1,11.5;4.1,12.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="-1,12"/> + <dia:point val="4,12"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="2"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O17" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="-6,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-6,2.25;-1.85,3.45"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From Peer A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-6,3"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="-7,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-7,5.25;-1.25,6.45"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From RS-Client B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-7,6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="4,11"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,10.95;6.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,11"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,12.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,11.3;5.4,12.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,12.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="4,8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,7.95;6.05,10.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,8"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,9.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,8.3;5.4,9.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,9.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="4,5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,4.95;6.05,7.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="4.7,6.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.7,5.3;5.4,6.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.7,6.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="4,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,1.95;6.05,4.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="4.75,3.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.75,2.3;5.45,3.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.75,3.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="5,43"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.95,42.95;14.05,47.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="5,43"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="6,44"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.95,43.95;7.96213,45.9621"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="6,44"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559649"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="6.61213,45.4"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="6.61213,44.35;7.31213,46"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="6.61213,45.4"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Text" version="0" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="11,45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.525,43.95;13.475,47"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#“Out” Filter +for Peer X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="11,45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="-6,43"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-6.05,42.95;3.05,47.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="-6,43"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="0,45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0,44.4;0,45.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0,45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="0,45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-2.325,43.95;2.325,47"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#“In” Filter +for Peer X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0,45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O32"> + <dia:attribute name="obj_pos"> + <dia:point val="-5,44"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-5.05,43.95;-2.95,46.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="-5,44"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#eb2020"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O33"> + <dia:attribute name="obj_pos"> + <dia:point val="-4.35,45.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-4.35,44.3;-3.65,45.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-4.35,45.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:group> + <dia:object type="Standard - Text" version="0" id="O34"> + <dia:attribute name="obj_pos"> + <dia:point val="-7,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-7,11.25;-1.2,12.45"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From RS-Client D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-7,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O35"> + <dia:attribute name="obj_pos"> + <dia:point val="-7,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-7,8.25;-1.2,9.45"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#From RS-Client C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="-7,9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O36"> + <dia:attribute name="obj_pos"> + <dia:point val="12,26"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.95,25.95;20.85,30.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12,26"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="8.8000000000000007"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4.5999999999999996"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#8f8f8f"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O37"> + <dia:attribute name="obj_pos"> + <dia:point val="12.3,26.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="12.25,26.8;16.35,29.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12.3,26.85"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O38"> + <dia:attribute name="obj_pos"> + <dia:point val="14.3,27.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.125,27.25;15.475,29.8"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Best +Path +Selection#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.3,27.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Box" version="0" id="O39"> + <dia:attribute name="obj_pos"> + <dia:point val="17.85,26.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.8,26.85;20.5,29.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.85,26.9"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2.6000000000000147"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2.8500000000000005"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O40"> + <dia:attribute name="obj_pos"> + <dia:point val="19.1,28.2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.075,27.6;20.125,29.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Loc-RIB +For C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.1,28.2"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O41"> + <dia:attribute name="obj_pos"> + <dia:point val="16.3,28.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.1476,27.3227;18.0161,29.3273"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.3,28.35"/> + <dia:point val="17.85,28.325"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="3"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.99999999999999967"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O39" connection="3"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O42"> + <dia:attribute name="obj_pos"> + <dia:point val="12,36"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.95,35.95;20.85,40.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12,36"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="8.8000000000000007"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4.5999999999999996"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#8f8f8f"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O43"> + <dia:attribute name="obj_pos"> + <dia:point val="12.3,36.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="12.25,36.8;16.35,39.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12.3,36.85"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O44"> + <dia:attribute name="obj_pos"> + <dia:point val="14.3,37.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.125,37.25;15.475,39.8"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Best +Path +Selection#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.3,37.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Box" version="0" id="O45"> + <dia:attribute name="obj_pos"> + <dia:point val="17.85,36.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.8,36.85;20.5,39.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.85,36.9"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2.6000000000000147"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2.8500000000000005"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O46"> + <dia:attribute name="obj_pos"> + <dia:point val="19.1,38.2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.075,37.6;20.125,39.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Loc-RIB +For D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.1,38.2"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O47"> + <dia:attribute name="obj_pos"> + <dia:point val="16.3,38.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.1476,37.3227;18.0161,39.3273"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.3,38.35"/> + <dia:point val="17.85,38.325"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="3"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.99999999999999967"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O45" connection="3"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O48"> + <dia:attribute name="obj_pos"> + <dia:point val="12,15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.95,14.95;20.85,19.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12,15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="8.8000000000000007"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="4.5999999999999996"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#8f8f8f"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O49"> + <dia:attribute name="obj_pos"> + <dia:point val="12.3,15.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="12.25,15.8;16.35,18.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="12.3,15.85"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="4"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O50"> + <dia:attribute name="obj_pos"> + <dia:point val="14.3,16.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="13.125,16.25;15.475,18.8"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Best +Path +Selection#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="14.3,16.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Box" version="0" id="O51"> + <dia:attribute name="obj_pos"> + <dia:point val="17.85,15.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.8,15.85;20.5,18.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="17.85,15.9"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2.6000000000000147"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2.8500000000000005"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3fa9bd"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O52"> + <dia:attribute name="obj_pos"> + <dia:point val="19.1,17.2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.075,16.6;20.125,18.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Loc-RIB +For B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.1,17.2"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O53"> + <dia:attribute name="obj_pos"> + <dia:point val="16.3,17.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.1476,16.3227;18.0161,18.3273"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.3,17.35"/> + <dia:point val="17.85,17.325"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="3"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.59999999999999998"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.99999999999999967"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - ZigZagLine" version="0" id="O54"> + <dia:attribute name="obj_pos"> + <dia:point val="0.666667,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.566667,11.9;4.1,21.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="0.666667,12"/> + <dia:point val="0.666667,21"/> + <dia:point val="4,21"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O14" connection="0"/> + <dia:connection handle="1" to="O55" connection="3"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O55"> + <dia:attribute name="obj_pos"> + <dia:point val="4,20"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,19.95;6.05,22.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,20"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O56"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,21.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,20.3;5.4,21.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,21.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O57"> + <dia:attribute name="obj_pos"> + <dia:point val="6,21"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,20.5;8.1,21.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,21"/> + <dia:point val="8,21"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O55" connection="4"/> + <dia:connection handle="1" to="O65" connection="3"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O58"> + <dia:attribute name="obj_pos"> + <dia:point val="4,17"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,16.95;6.05,19.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,17"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O59"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,18.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,17.3;5.4,18.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,18.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O60"> + <dia:attribute name="obj_pos"> + <dia:point val="10,21"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.85865,18.3581;12.7734,21.1413"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,21"/> + <dia:point val="12.3586,18.7914"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O65" connection="4"/> + <dia:connection handle="1" to="O49" connection="5"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O61"> + <dia:attribute name="obj_pos"> + <dia:point val="8,14"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,13.95;10.05,16.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,14"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O62"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,15.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,14.3;9.35,15.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,15.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O63"> + <dia:attribute name="obj_pos"> + <dia:point val="8,17"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,16.95;10.05,19.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,17"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O64"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,18.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,17.3;9.35,18.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,18.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O65"> + <dia:attribute name="obj_pos"> + <dia:point val="8,20"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,19.95;10.05,22.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,20"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O66"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,21.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,20.3;9.35,21.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,21.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O67"> + <dia:attribute name="obj_pos"> + <dia:point val="8,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,23.95;10.05,26.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,24"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O68"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,25.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,24.3;9.4,25.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,25.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O69"> + <dia:attribute name="obj_pos"> + <dia:point val="8,30"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,29.95;10.05,32.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,30"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O70"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,31.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,30.3;9.4,31.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,31.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O71"> + <dia:attribute name="obj_pos"> + <dia:point val="8,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,26.95;10.05,29.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O72"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,28.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,27.3;9.4,28.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,28.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O73"> + <dia:attribute name="obj_pos"> + <dia:point val="4,30"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,29.95;6.05,32.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,30"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O74"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,31.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,30.3;5.4,31.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,31.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O75"> + <dia:attribute name="obj_pos"> + <dia:point val="4,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,26.95;6.05,29.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O76"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,28.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,27.3;5.35,28.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,28.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O77"> + <dia:attribute name="obj_pos"> + <dia:point val="8,34"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,33.95;10.05,36.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,34"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O78"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,35.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,34.3;9.4,35.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,35.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O79"> + <dia:attribute name="obj_pos"> + <dia:point val="8,40"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,39.95;10.05,42.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,40"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O80"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,41.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,40.3;9.4,41.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,41.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O81"> + <dia:attribute name="obj_pos"> + <dia:point val="8,37"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,36.95;10.05,39.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,37"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O82"> + <dia:attribute name="obj_pos"> + <dia:point val="8.65,38.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.65,37.3;9.4,38.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.65,38.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O83"> + <dia:attribute name="obj_pos"> + <dia:point val="4,37"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,36.95;6.05,39.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,37"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O84"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,38.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,37.3;5.35,38.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,38.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O85"> + <dia:attribute name="obj_pos"> + <dia:point val="4,40"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,39.95;6.05,42.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,40"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O86"> + <dia:attribute name="obj_pos"> + <dia:point val="4.65,41.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.65,40.3;5.4,41.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.65,41.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - ZigZagLine" version="0" id="O87"> + <dia:attribute name="obj_pos"> + <dia:point val="1.5,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="1.4,8.9;4.1,18.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="1.5,9"/> + <dia:point val="1.5,18"/> + <dia:point val="4,18"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="1"/> + <dia:connection handle="1" to="O58" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O88"> + <dia:attribute name="obj_pos"> + <dia:point val="1.5,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="1.4,8.9;4.1,41.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="1.5,9"/> + <dia:point val="1.5,41"/> + <dia:point val="4,41"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="1"/> + <dia:connection handle="1" to="O85" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O89"> + <dia:attribute name="obj_pos"> + <dia:point val="2.75,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.65,5.9;4.14142,28.1"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="2.75,6"/> + <dia:point val="2.75,28"/> + <dia:point val="4,28"/> + <dia:point val="4,28"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O75" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O90"> + <dia:attribute name="obj_pos"> + <dia:point val="2.75,6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.65,5.9;4.14142,38.1"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="2.75,6"/> + <dia:point val="2.75,38"/> + <dia:point val="4,38"/> + <dia:point val="4,38"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + <dia:enum val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O83" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O91"> + <dia:attribute name="obj_pos"> + <dia:point val="0.666667,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.566667,11.9;4.1,31.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="0.666667,12"/> + <dia:point val="0.666667,31"/> + <dia:point val="4,31"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O14" connection="0"/> + <dia:connection handle="1" to="O73" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O92"> + <dia:attribute name="obj_pos"> + <dia:point val="-0.285714,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.385714,2.9;8.1,15.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="-0.285714,3"/> + <dia:point val="-0.285714,15"/> + <dia:point val="8,15"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O6" connection="0"/> + <dia:connection handle="1" to="O61" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O93"> + <dia:attribute name="obj_pos"> + <dia:point val="-0.285714,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.385714,2.9;8.1,25.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="-0.285714,3"/> + <dia:point val="-0.285714,25"/> + <dia:point val="8,25"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O6" connection="0"/> + <dia:connection handle="1" to="O67" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - ZigZagLine" version="0" id="O94"> + <dia:attribute name="obj_pos"> + <dia:point val="-0.285714,3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.385714,2.9;8.1,35.5"/> + </dia:attribute> + <dia:attribute name="orth_points"> + <dia:point val="-0.285714,3"/> + <dia:point val="-0.285714,35"/> + <dia:point val="8,35"/> + </dia:attribute> + <dia:attribute name="orth_orient"> + <dia:enum val="1"/> + <dia:enum val="0"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O6" connection="0"/> + <dia:connection handle="1" to="O77" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O95"> + <dia:attribute name="obj_pos"> + <dia:point val="6,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,30.5;8.1,31.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,31"/> + <dia:point val="8,31"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O73" connection="4"/> + <dia:connection handle="1" to="O69" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O96"> + <dia:attribute name="obj_pos"> + <dia:point val="10,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.8654,29.3008;12.6756,31.1346"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,31"/> + <dia:point val="12.3586,29.7914"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O69" connection="4"/> + <dia:connection handle="1" to="O37" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O97"> + <dia:attribute name="obj_pos"> + <dia:point val="10,41"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.8654,39.3008;12.6756,41.1346"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,41"/> + <dia:point val="12.3586,39.7914"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O79" connection="4"/> + <dia:connection handle="1" to="O43" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O98"> + <dia:attribute name="obj_pos"> + <dia:point val="6,41"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,40.5;8.1,41.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,41"/> + <dia:point val="8,41"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O85" connection="4"/> + <dia:connection handle="1" to="O79" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O99"> + <dia:attribute name="obj_pos"> + <dia:point val="10,18"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.87657,16.8416;12.5322,18.1234"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,18"/> + <dia:point val="12.3,17.35"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O63" connection="4"/> + <dia:connection handle="1" to="O49" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O100"> + <dia:attribute name="obj_pos"> + <dia:point val="6,18"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,17.5;8.1,18.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,18"/> + <dia:point val="8,18"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O58" connection="4"/> + <dia:connection handle="1" to="O63" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O101"> + <dia:attribute name="obj_pos"> + <dia:point val="6,28"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,27.5;8.1,28.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,28"/> + <dia:point val="8,28"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O75" connection="4"/> + <dia:connection handle="1" to="O71" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O102"> + <dia:attribute name="obj_pos"> + <dia:point val="6,38"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="5.9,37.5;8.1,38.5"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="6,38"/> + <dia:point val="8,38"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O83" connection="4"/> + <dia:connection handle="1" to="O81" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O103"> + <dia:attribute name="obj_pos"> + <dia:point val="10,38"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.88609,37.8406;12.4741,38.8594"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,38"/> + <dia:point val="12.3,38.35"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O81" connection="4"/> + <dia:connection handle="1" to="O43" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O104"> + <dia:attribute name="obj_pos"> + <dia:point val="10,28"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.88609,27.8406;12.4741,28.8594"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,28"/> + <dia:point val="12.3,28.35"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="3"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O71" connection="4"/> + <dia:connection handle="1" to="O37" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O105"> + <dia:attribute name="obj_pos"> + <dia:point val="10,15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.87074,14.8707;12.6317,16.4111"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,15"/> + <dia:point val="12.3586,15.9086"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O61" connection="4"/> + <dia:connection handle="1" to="O49" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O106"> + <dia:attribute name="obj_pos"> + <dia:point val="10,25"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.85936,24.8594;12.7509,27.3602"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,25"/> + <dia:point val="12.3586,26.9086"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O67" connection="4"/> + <dia:connection handle="1" to="O37" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O107"> + <dia:attribute name="obj_pos"> + <dia:point val="10,35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.85936,34.8594;12.7509,37.3602"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="10,35"/> + <dia:point val="12.3586,36.9086"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O77" connection="4"/> + <dia:connection handle="1" to="O43" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O108"> + <dia:attribute name="obj_pos"> + <dia:point val="20.45,17.325"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="20.3481,16.7482;24.5095,17.7518"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20.45,17.325"/> + <dia:point val="24.4,17.25"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O51" connection="4"/> + <dia:connection handle="1" to="O123" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O109"> + <dia:attribute name="obj_pos"> + <dia:point val="20.45,28.325"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="20.3494,27.8494;24.4532,28.8506"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20.45,28.325"/> + <dia:point val="24.35,28.35"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O39" connection="4"/> + <dia:connection handle="1" to="O126" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O110"> + <dia:attribute name="obj_pos"> + <dia:point val="20.45,38.325"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="20.3494,37.8494;24.4411,38.8506"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="20.45,38.325"/> + <dia:point val="24.3379,38.35"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O45" connection="4"/> + <dia:connection handle="1" to="O134" connection="3"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O111"> + <dia:attribute name="obj_pos"> + <dia:point val="16,43"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.95,42.95;25.05,46.735"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="16,43"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.6849999999999952"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O112"> + <dia:attribute name="obj_pos"> + <dia:point val="22,45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22,44.4;22,45.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="22,45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O113"> + <dia:attribute name="obj_pos"> + <dia:point val="21.45,44.7"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.15,43.65;24.75,46.7"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Export Policy +of RS-Client X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="21.45,44.7"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O114"> + <dia:attribute name="obj_pos"> + <dia:point val="16.3,43.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.25,43.3;18.35,45.4"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="16.3,43.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#00c0bb"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O115"> + <dia:attribute name="obj_pos"> + <dia:point val="17.015,44.735"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.015,43.685;17.715,45.335"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="17.015,44.735"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O116"> + <dia:attribute name="obj_pos"> + <dia:point val="27,43"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.95,42.95;36.05,46.735"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="27,43"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.6849999999999952"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#b7b2b2"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O117"> + <dia:attribute name="obj_pos"> + <dia:point val="33,45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="33,44.4;33,45.35"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="33,45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O118"> + <dia:attribute name="obj_pos"> + <dia:point val="32.45,44.7"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.15,43.65;35.75,46.7"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Import Policy +of RS-Client X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="32.45,44.7"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O119"> + <dia:attribute name="obj_pos"> + <dia:point val="27.3,43.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="27.25,43.3;29.35,45.4"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="27.3,43.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#a8e400"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O120"> + <dia:attribute name="obj_pos"> + <dia:point val="28.015,44.735"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.015,43.685;28.715,45.335"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#X#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="28.015,44.735"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Line" version="0" id="O121"> + <dia:attribute name="obj_pos"> + <dia:point val="26.4,17.25"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.3,16.75;30.5,17.75"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="26.4,17.25"/> + <dia:point val="30.4,17.25"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#3c8d37"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O122"> + <dia:attribute name="obj_pos"> + <dia:point val="30.4,17.25"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="30.4,16.5;35.3,17.7"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To RS-Client B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="30.4,17.25"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O123"> + <dia:attribute name="obj_pos"> + <dia:point val="24.4,16.25"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.35,16.2;26.45,18.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="24.4,16.25"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O124"> + <dia:attribute name="obj_pos"> + <dia:point val="25.1,17.7"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="25.1,16.65;25.8,18.3"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#B#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="25.1,17.7"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:group> + <dia:group> + <dia:object type="Standard - Line" version="0" id="O125"> + <dia:attribute name="obj_pos"> + <dia:point val="26.35,28.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.2488,27.8988;30.5561,28.9012"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="26.35,28.35"/> + <dia:point val="30.45,28.4"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#972360"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O126"> + <dia:attribute name="obj_pos"> + <dia:point val="24.35,27.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.3,27.3;26.4,29.4"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="24.35,27.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O127"> + <dia:attribute name="obj_pos"> + <dia:point val="24.95,28.8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.95,27.75;25.7,29.4"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.95,28.8"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Text" version="0" id="O128"> + <dia:attribute name="obj_pos"> + <dia:point val="30.35,28.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="30.35,27.6;35.3,28.8"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To RS-Client C#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="30.35,28.35"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O129"> + <dia:attribute name="obj_pos"> + <dia:point val="25.0879,6.59393"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="25.0379,6.54393;27.05,8.55606"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="25.0879,6.59393"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559645"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O130"> + <dia:attribute name="obj_pos"> + <dia:point val="25.65,8.04393"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="25.65,6.99393;26.35,8.64393"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="25.65,8.04393"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Line" version="0" id="O131"> + <dia:attribute name="obj_pos"> + <dia:point val="27,7.55"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.8988,7.09885;31.306,8.10117"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="27,7.55"/> + <dia:point val="31.2001,7.60001"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#e5aa2a"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O132"> + <dia:attribute name="obj_pos"> + <dia:point val="31.3,7.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="31.3,6.85;34.6,8.05"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To Peer A#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="31.3,7.6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:group> + <dia:group> + <dia:object type="Standard - Line" version="0" id="O133"> + <dia:attribute name="obj_pos"> + <dia:point val="26.25,38.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="26.1499,37.8438;30.5386,38.844"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="26.25,38.35"/> + <dia:point val="30.4379,38.3439"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#618ccd"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O134"> + <dia:attribute name="obj_pos"> + <dia:point val="24.3379,37.3939"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.2879,37.3439;26.3,39.356"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="24.3379,37.3939"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="1.9121320343559631"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.9121320343559649"/> + </dia:attribute> + <dia:attribute name="inner_color"> + <dia:color val="#3756d7"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="corner_radius"> + <dia:real val="0.29999999999999999"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O135"> + <dia:attribute name="obj_pos"> + <dia:point val="24.95,38.7939"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.95,37.7439;25.7,39.3939"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1.3999999999999999"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.95,38.7939"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Text" version="0" id="O136"> + <dia:attribute name="obj_pos"> + <dia:point val="30.4379,38.3439"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="30.4379,37.5939;35.3879,38.7939"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#To RS-Client D#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="30.4379,38.3439"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-rs-processing.png b/doc/figures/fig-rs-processing.png Binary files differnew file mode 100644 index 0000000..1f77263 --- /dev/null +++ b/doc/figures/fig-rs-processing.png diff --git a/doc/figures/fig-rs-processing.txt b/doc/figures/fig-rs-processing.txt new file mode 100644 index 0000000..eafe146 --- /dev/null +++ b/doc/figures/fig-rs-processing.txt @@ -0,0 +1,47 @@ +From Peer A + | From RS-Client B + | | From RS-Client C + | | | From RS-Client D + | | | | + | | | | Main / Normal RIB + | | | | ________________________________ + | | | | / _________ _________ \ + | | | +--->|(D)-|Best | | Main | | + | | +--|--->|(C)-|Path |-->|Local-RIB|->[A]|--->To Peer A + | +--|--|--->|(B)-|Selection| | | | + +--|--|--|--->|(A)-|_________| |_________| | + | | | | \________________________________/ + | | | | + | | | | ________________________________ + | | | | / _________ _________ \ + | | | +--->*D*->|{B}-|Best | |RS-Client| | + | | +--|--->*C*->|{B}-|Path |-->|Local-RIB|->[B]|--->To RS-Client B + | | | | | |Selection| | for B | | + +--|--|--|-------->|{B}-|_________| |_________| | + | | | | \________________________________/ + | | | | + | | | | ________________________________ + | | | | / _________ _________ \ + | | | +--->*D*->|{C}-|Best | |RS-Client| | + | | | | | |Path |-->|Local-RIB|->[C]|--->To RS-Client C + | +--|--|--->*B*->|{C}-|Selection| | for C | | + +--|--|--|-------->|{C}-|_________| |_________| | + | | | \________________________________/ + | | | + | | | ________________________________ + | | | / _________ _________ \ + | | | | |Best | |RS-Client| | + | | +------>*C*->|{D}-|Path |-->|Local-RIB|->[D]|--->To RS-Client D + | +--------->*B*->|{D}-|Selection| | for D | | + +----------------->|{D}-|_________| |_________| | + \________________________________/ + + +Key: (X) - 'In' Filter applied to Peer X's announcements before + considering announcement for the normal main Local-RIB + [X] - 'Out' Filter applied to announcements to Peer X + *X* - 'Export' Filter of RS-Client X, to apply X's policies + before its routes may be considered for other RS-Clients + RIBs. + {X} - 'Import' Filter of RS-Client X, to apply X's policies + on routes before allowing them into X's RIB. diff --git a/doc/figures/fig-vnc-commercial-route-reflector.dia b/doc/figures/fig-vnc-commercial-route-reflector.dia new file mode 100644 index 0000000..0da5bd1 --- /dev/null +++ b/doc/figures/fig-vnc-commercial-route-reflector.dia @@ -0,0 +1,794 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,39.095;64.0901,42.445"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,34.1975;64.0901,37.5475"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,29.3;64.0901,32.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,39.245;8.9726,42.595"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,34.3475;8.9726,37.6975"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,29.45;8.9726,32.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="22.5347,32.178"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.4642,23.359;30.091,32.2485"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="22.5347,32.178"/> + <dia:point val="30.0205,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="43.1205,32.4705"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="36.8099,23.3599;43.1901,32.5401"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="43.1205,32.4705"/> + <dia:point val="36.8795,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.29385,30.5113;7.76004,32.1149"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.88385,35.4088;8.17004,37.0124"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 5 +VN 172.16.130.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.8826,40.3063;8.17129,41.9099"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 6 +VN 172.16.132.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,30.3613;62.8788,31.9649"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 7 +VN 172.16.6.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,35.2588;62.8788,36.8624"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 8 +VN 172.16.8.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.0026,40.1563;63.2863,41.7599"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 9 +VN 172.16.134.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86406,31.0665;15.979,32.529"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,31.125"/> + <dia:point val="15.9205,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="4"/> + <dia:connection handle="1" to="O24" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.87151,35.8489;14.5511,36.0736"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,36.0225"/> + <dia:point val="14.5,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="4"/> + <dia:connection handle="1" to="O24" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86276,39.2697;15.9803,40.9798"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,40.92"/> + <dia:point val="15.9205,39.3295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="4"/> + <dia:connection handle="1" to="O24" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.9204,30.9159;57.2892,32.5296"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,30.975"/> + <dia:point val="49.9795,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="3"/> + <dia:connection handle="1" to="O22" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.3498,35.8223;57.2803,35.9502"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,35.8725"/> + <dia:point val="51.4,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="3"/> + <dia:connection handle="1" to="O22" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="50.9399,37.8657;57.2963,40.8362"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,40.77"/> + <dia:point val="51.0061,37.9319"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="3"/> + <dia:connection handle="1" to="O22" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="34.8,20.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="34.8,19.855;34.8,20.6025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="34.8,20.45"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.65,31;51.45,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="44.2012,35.305;48.8987,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 3 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O22" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.45,31;24.25,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.0013,35.305;21.6988,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O24" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.55,15.1;38.35,24.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="33.45,19.7"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="30.9863,19.105;35.9138,21.4525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Commercial Router +Route Reflector +192.168.1.104#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="33.45,19.7"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-commercial-route-reflector.png b/doc/figures/fig-vnc-commercial-route-reflector.png Binary files differnew file mode 100644 index 0000000..ca8a248 --- /dev/null +++ b/doc/figures/fig-vnc-commercial-route-reflector.png diff --git a/doc/figures/fig-vnc-commercial-route-reflector.txt b/doc/figures/fig-vnc-commercial-route-reflector.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-commercial-route-reflector.txt diff --git a/doc/figures/fig-vnc-frr-route-reflector.dia b/doc/figures/fig-vnc-frr-route-reflector.dia new file mode 100644 index 0000000..634f0b1 --- /dev/null +++ b/doc/figures/fig-vnc-frr-route-reflector.dia @@ -0,0 +1,763 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,39.095;64.0901,42.445"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,34.1975;64.0901,37.5475"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,29.3;64.0901,32.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,39.245;8.9726,42.595"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,34.3475;8.9726,37.6975"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,29.45;8.9726,32.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="22.5347,32.178"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.4642,23.359;30.091,32.2485"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="22.5347,32.178"/> + <dia:point val="30.0205,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="43.1205,32.4705"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="36.8099,23.3599;43.1901,32.5401"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="43.1205,32.4705"/> + <dia:point val="36.8795,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.29385,30.5113;7.76004,32.1149"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.88385,35.4088;8.17004,37.0124"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 5 +VN 172.16.130.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.8826,40.3063;8.17129,41.9099"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 6 +VN 172.16.132.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,30.3613;62.8788,31.9649"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 7 +VN 172.16.6.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,35.2588;62.8788,36.8624"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 8 +VN 172.16.8.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.0026,40.1563;63.2863,41.7599"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 9 +VN 172.16.134.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86406,31.0665;15.979,32.529"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,31.125"/> + <dia:point val="15.9205,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="4"/> + <dia:connection handle="1" to="O23" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.87151,35.8489;14.5511,36.0736"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,36.0225"/> + <dia:point val="14.5,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="4"/> + <dia:connection handle="1" to="O23" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86276,39.2697;15.9803,40.9798"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,40.92"/> + <dia:point val="15.9205,39.3295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="4"/> + <dia:connection handle="1" to="O23" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.9204,30.9159;57.2892,32.5296"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,30.975"/> + <dia:point val="49.9795,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="3"/> + <dia:connection handle="1" to="O25" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.3498,35.8223;57.2803,35.9502"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,35.8725"/> + <dia:point val="51.4,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="3"/> + <dia:connection handle="1" to="O25" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="50.9399,37.8657;57.2963,40.8362"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,40.77"/> + <dia:point val="51.0061,37.9319"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="3"/> + <dia:connection handle="1" to="O25" connection="8"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.55,15.1;38.35,24.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="33.45,20"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.9225,19.405;36.9775,20.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#BGP Route Reflector 1 +192.168.1.100#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="33.45,20"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O21" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.45,31;24.25,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.0013,35.305;21.6988,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.65,31;51.45,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="44.2012,35.305;48.8987,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 3 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O25" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-frr-route-reflector.png b/doc/figures/fig-vnc-frr-route-reflector.png Binary files differnew file mode 100644 index 0000000..4770521 --- /dev/null +++ b/doc/figures/fig-vnc-frr-route-reflector.png diff --git a/doc/figures/fig-vnc-frr-route-reflector.txt b/doc/figures/fig-vnc-frr-route-reflector.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-frr-route-reflector.txt diff --git a/doc/figures/fig-vnc-gw-rr.dia b/doc/figures/fig-vnc-gw-rr.dia new file mode 100644 index 0000000..dab27f7 --- /dev/null +++ b/doc/figures/fig-vnc-gw-rr.dia @@ -0,0 +1,1155 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Text" version="1" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="29.5,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.45,14.8;47.05,14.9"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.5,14.85"/> + <dia:point val="47,14.85"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="4"/> + <dia:connection handle="1" to="O37" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="29.5,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.45,28.8;47.05,28.9"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.5,28.85"/> + <dia:point val="47,28.85"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="4"/> + <dia:connection handle="1" to="O35" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.6,19.65;24.7,24.05"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="24.65,24"/> + <dia:point val="24.65,19.7"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="1"/> + <dia:connection handle="1" to="O31" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.8,19.65;51.9,24.05"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="51.85,24"/> + <dia:point val="51.85,19.7"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="1"/> + <dia:connection handle="1" to="O37" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="47.4933,26.6076"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.9394,17.0251;47.5606,26.6749"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="47.4933,26.6076"/> + <dia:point val="29.0067,17.0924"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="29.0067,26.6076"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.9394,17.0251;47.5606,26.6749"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.0067,26.6076"/> + <dia:point val="47.4933,17.0924"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="16.8594,28.6941"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.8084,28.6431;19.8026,28.803"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.8594,28.6941"/> + <dia:point val="19.7516,28.752"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="8"/> + <dia:connection handle="1" to="O33" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="16.81,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.7433,30.909;20.302,32.6917"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.81,32.625"/> + <dia:point val="20.2352,30.9757"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O13" connection="4"/> + <dia:connection handle="1" to="O33" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="59.9506,31.4965"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.4448,30.3087;60.0137,31.5595"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="59.9506,31.4965"/> + <dia:point val="56.5079,30.3717"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="8"/> + <dia:connection handle="1" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="59.9506,28.6923"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.6981,28.6413;60.0016,28.8056"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="59.9506,28.6923"/> + <dia:point val="56.7491,28.7546"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O15" connection="8"/> + <dia:connection handle="1" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="10,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,26.95;16.86,30.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="13.405,28.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.1862,28.03;15.6237,29.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 1 +VN 172.16.1.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13.405,28.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="8"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="10,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,30.95;16.86,34.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,31"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="13.405,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.1862,32.03;15.6237,33.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 2 +VN 172.16.2.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13.405,32.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O13" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="60,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,26.95;66.86,30.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="63.405,28.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.1863,28.03;65.6238,29.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 3 +VN 172.16.3.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.405,28.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O15" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="60,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,30.95;66.86,34.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,31"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="63.405,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.1863,32.03;65.6238,33.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.405,32.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="16.1073,12.7602"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.0469,12.6997;19.9513,13.7462"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.1073,12.7602"/> + <dia:point val="19.8909,13.6858"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + <dia:connection handle="1" to="O31" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="16.1475,18.0292"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.0831,16.5016;20.1253,18.0935"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.1475,18.0292"/> + <dia:point val="20.0609,16.5659"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O25" connection="8"/> + <dia:connection handle="1" to="O31" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="60.0374,12.8937"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.5559,12.8335;60.0976,13.7714"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="60.0374,12.8937"/> + <dia:point val="56.6162,13.7112"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O27" connection="8"/> + <dia:connection handle="1" to="O37" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="60.3187,18.1413"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.3502,16.5594;60.3834,18.206"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="60.3187,18.1413"/> + <dia:point val="56.4149,16.6241"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="8"/> + <dia:connection handle="1" to="O37" connection="8"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="9.85,8.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.8,8.8;16.2,15.2"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="9.85,8.85"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="13,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.27,11.405;14.73,12.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 1 +172.16.1.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="10,16"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,15.95;16.35,22.35"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,16"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="13,19"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.27,18.405;14.73,19.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 2 +172.16.2.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13,19"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="60,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,8.95;66.35,15.35"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,9"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="63,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.27,11.405;64.73,12.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 3 +172.16.3.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="60.15,16.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="60.1,16.1;66.5,22.5"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60.15,16.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="63.3,19.3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.57,18.705;65.03,20.2525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 4 +172.16.4.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.3,19.3"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="19.8,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.75,9.95;29.55,19.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="19.8,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O32"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.15,14.255;27.15,15.8025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#VNC Gateway 1 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.65,14.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O33"> + <dia:attribute name="obj_pos"> + <dia:point val="19.8,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.75,23.95;29.55,33.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="19.8,24"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O34"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="21.4637,28.255;27.8362,29.8025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 1 (NVA) +192.168.1.103#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.65,28.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O35"> + <dia:attribute name="obj_pos"> + <dia:point val="47,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="46.95,23.95;56.75,33.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="47,24"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O36"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="48.6638,28.255;55.0363,29.8025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 (NVA) +192.168.1.104#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="51.85,28.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O37"> + <dia:attribute name="obj_pos"> + <dia:point val="47,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="46.95,9.95;56.75,19.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="47,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O38"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.35,14.255;54.35,15.8025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#VNC Gateway 2 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="51.85,14.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O37" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O39"> + <dia:attribute name="obj_pos"> + <dia:point val="35,36"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="34.95,35.95;41.35,42.35"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="35,36"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O40"> + <dia:attribute name="obj_pos"> + <dia:point val="38.15,39.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="35.8087,38.555;40.4912,40.1025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RR +192.168.1.105#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="38.15,39.15"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O39" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O41"> + <dia:attribute name="obj_pos"> + <dia:point val="40.7075,37.2272"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="40.6374,31.7234;48.005,37.2972"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="40.7075,37.2272"/> + <dia:point val="47.935,31.7934"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O39" connection="8"/> + <dia:connection handle="1" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O42"> + <dia:attribute name="obj_pos"> + <dia:point val="35.6122,37.2137"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.4724,31.7497;35.6822,37.2838"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="35.6122,37.2137"/> + <dia:point val="28.5425,31.8198"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O39" connection="8"/> + <dia:connection handle="1" to="O33" connection="8"/> + </dia:connections> + </dia:object> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-gw-rr.png b/doc/figures/fig-vnc-gw-rr.png Binary files differnew file mode 100644 index 0000000..7ae0630 --- /dev/null +++ b/doc/figures/fig-vnc-gw-rr.png diff --git a/doc/figures/fig-vnc-gw-rr.txt b/doc/figures/fig-vnc-gw-rr.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-gw-rr.txt diff --git a/doc/figures/fig-vnc-gw.dia b/doc/figures/fig-vnc-gw.dia new file mode 100644 index 0000000..8270e20 --- /dev/null +++ b/doc/figures/fig-vnc-gw.dia @@ -0,0 +1,1058 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Text" version="1" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="29.5,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.45,14.8;47.05,14.9"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.5,14.85"/> + <dia:point val="47,14.85"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="4"/> + <dia:connection handle="1" to="O37" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="29.5,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.45,28.8;47.05,28.9"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.5,28.85"/> + <dia:point val="47,28.85"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="4"/> + <dia:connection handle="1" to="O35" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.6,19.65;24.7,24.05"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="24.65,24"/> + <dia:point val="24.65,19.7"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="1"/> + <dia:connection handle="1" to="O31" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.8,19.65;51.9,24.05"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="51.85,24"/> + <dia:point val="51.85,19.7"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="1"/> + <dia:connection handle="1" to="O37" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="47.4933,26.6076"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.9394,17.0251;47.5606,26.6749"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="47.4933,26.6076"/> + <dia:point val="29.0067,17.0924"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="29.0067,26.6076"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.9394,17.0251;47.5606,26.6749"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="29.0067,26.6076"/> + <dia:point val="47.4933,17.0924"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="16.8594,28.6941"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.8084,28.6431;19.8026,28.803"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.8594,28.6941"/> + <dia:point val="19.7516,28.752"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="8"/> + <dia:connection handle="1" to="O33" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="16.81,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.7433,30.909;20.302,32.6917"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.81,32.625"/> + <dia:point val="20.2352,30.9757"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O13" connection="4"/> + <dia:connection handle="1" to="O33" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="59.9506,31.4965"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.4448,30.3087;60.0137,31.5595"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="59.9506,31.4965"/> + <dia:point val="56.5079,30.3717"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="8"/> + <dia:connection handle="1" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="59.9506,28.6923"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.6981,28.6413;60.0016,28.8056"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="59.9506,28.6923"/> + <dia:point val="56.7491,28.7546"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O15" connection="8"/> + <dia:connection handle="1" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="10,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,26.95;16.86,30.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="13.405,28.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.1862,28.03;15.6237,29.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 1 +VN 172.16.1.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13.405,28.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O11" connection="8"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="10,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,30.95;16.86,34.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,31"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="13.405,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.1862,32.03;15.6237,33.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 2 +VN 172.16.2.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13.405,32.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O13" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="60,27"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,26.95;66.86,30.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,27"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="63.405,28.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.1863,28.03;65.6238,29.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 3 +VN 172.16.3.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.405,28.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O15" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="60,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,30.95;66.86,34.3"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,31"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="63.405,32.625"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.1863,32.03;65.6238,33.5775"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.405,32.625"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="16.1073,12.7602"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.0469,12.6997;19.9513,13.7462"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.1073,12.7602"/> + <dia:point val="19.8909,13.6858"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + <dia:connection handle="1" to="O31" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="16.1475,18.0292"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.0831,16.5016;20.1253,18.0935"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="16.1475,18.0292"/> + <dia:point val="20.0609,16.5659"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O25" connection="8"/> + <dia:connection handle="1" to="O31" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="60.0374,12.8937"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.5559,12.8335;60.0976,13.7714"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="60.0374,12.8937"/> + <dia:point val="56.6162,13.7112"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O27" connection="8"/> + <dia:connection handle="1" to="O37" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="60.3187,18.1413"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="56.3502,16.5594;60.3834,18.206"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="60.3187,18.1413"/> + <dia:point val="56.4149,16.6241"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="8"/> + <dia:connection handle="1" to="O37" connection="8"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="9.85,8.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.8,8.8;16.2,15.2"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="9.85,8.85"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="13,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.27,11.405;14.73,12.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 1 +172.16.1.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="10,16"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9.95,15.95;16.35,22.35"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="10,16"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="13,19"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="11.27,18.405;14.73,19.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 2 +172.16.2.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="13,19"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="60,9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.95,8.95;66.35,15.35"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60,9"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="63,12"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.27,11.405;64.73,12.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 3 +172.16.3.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63,12"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="60.15,16.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="60.1,16.1;66.5,22.5"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="60.15,16.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="6.2999992370605469"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="63.3,19.3"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="61.57,18.705;65.03,20.2525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#CE 4 +172.16.4.2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="63.3,19.3"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="19.8,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.75,9.95;29.55,19.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="19.8,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O32"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.15,14.2363;27.1687,15.8399"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#VNC Gateway 1 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.65,14.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O33"> + <dia:attribute name="obj_pos"> + <dia:point val="19.8,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.75,23.95;29.55,33.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="19.8,24"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O34"> + <dia:attribute name="obj_pos"> + <dia:point val="24.65,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="21.4637,28.2363;27.8549,29.8399"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 1 (NVA) +192.168.1.103#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="24.65,28.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O35"> + <dia:attribute name="obj_pos"> + <dia:point val="47,24"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="46.95,23.95;56.75,33.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="47,24"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O36"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,28.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="48.6638,28.2363;55.0549,29.8399"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 (NVA) +192.168.1.104#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="51.85,28.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O37"> + <dia:attribute name="obj_pos"> + <dia:point val="47,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="46.95,9.95;56.75,19.75"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="47,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O38"> + <dia:attribute name="obj_pos"> + <dia:point val="51.85,14.85"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.35,14.2363;54.3687,15.8399"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#VNC Gateway 2 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="51.85,14.85"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O37" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-gw.png b/doc/figures/fig-vnc-gw.png Binary files differnew file mode 100644 index 0000000..df8f23f --- /dev/null +++ b/doc/figures/fig-vnc-gw.png diff --git a/doc/figures/fig-vnc-gw.txt b/doc/figures/fig-vnc-gw.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-gw.txt diff --git a/doc/figures/fig-vnc-mesh.dia b/doc/figures/fig-vnc-mesh.dia new file mode 100644 index 0000000..a8f702f --- /dev/null +++ b/doc/figures/fig-vnc-mesh.dia @@ -0,0 +1,1071 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Line" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="24.2,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.15,35.85;41.75,35.95"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="24.2,35.9"/> + <dia:point val="41.7,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="22.5347,32.178"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.4642,23.359;30.091,32.2485"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="22.5347,32.178"/> + <dia:point val="30.0205,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="43.1205,32.4705"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="36.8099,23.3599;43.1901,32.5401"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="43.1205,32.4705"/> + <dia:point val="36.8795,23.4295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="23.2957,9.72508"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="23.225,9.65437;30.0912,16.6412"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="23.2957,9.72508"/> + <dia:point val="30.0205,16.5705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="8"/> + <dia:connection handle="1" to="O15" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="33.0601,9.675"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="33.0098,9.62471;33.1421,15.1003"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="33.0601,9.675"/> + <dia:point val="33.0918,15.05"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="44.4726,9.675"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="36.8089,9.60437;44.5432,16.6411"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="44.4726,9.675"/> + <dia:point val="36.8795,16.5705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O37" connection="6"/> + <dia:connection handle="1" to="O15" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86406,31.0665;15.979,32.529"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,31.125"/> + <dia:point val="15.9205,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O27" connection="4"/> + <dia:connection handle="1" to="O17" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.87151,35.8489;14.5511,36.0736"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,36.0225"/> + <dia:point val="14.5,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="4"/> + <dia:connection handle="1" to="O17" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86276,39.2697;15.9803,40.9798"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,40.92"/> + <dia:point val="15.9205,39.3295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="4"/> + <dia:connection handle="1" to="O17" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.9204,30.9159;57.2892,32.5296"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,30.975"/> + <dia:point val="49.9795,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O21" connection="3"/> + <dia:connection handle="1" to="O19" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.3498,35.8223;57.2803,35.9502"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,35.8725"/> + <dia:point val="51.4,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="3"/> + <dia:connection handle="1" to="O19" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="50.9399,37.8657;57.2963,40.8362"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,40.77"/> + <dia:point val="51.0061,37.9319"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O25" connection="3"/> + <dia:connection handle="1" to="O19" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="62.55,31.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="62.55,31.005;62.55,31.7525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="62.55,31.6"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="59.65,31.65"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="59.65,31.055;59.65,31.8025"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="59.65,31.65"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="28.55,15.1;38.35,24.9"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="28.6,15.15"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="33.45,20"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="31.1025,19.405;35.7975,20.9525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 1 +192.168.1.100#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="33.45,20"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O15" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.45,31;24.25,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14.5,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="17.0013,35.305;21.6988,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.35,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O17" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.65,31;51.45,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.7,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="44.2012,35.305;48.8987,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 3 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="46.55,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O19" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:object type="Standard - Box" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,29.3;64.0901,32.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,30.3613;62.8788,31.9649"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 7 +VN 172.16.6.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O21" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,34.1975;64.0901,37.5475"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,35.2588;62.8788,36.8624"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 8 +VN 172.16.8.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O23" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,39.095;64.0901,42.445"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.0026,40.1563;63.2863,41.7599"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 9 +VN 172.16.134.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O25" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,29.45;8.9726,32.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.29385,30.5113;7.76004,32.1149"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O27" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,34.3475;8.9726,37.6975"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.88385,35.4088;8.17004,37.0124"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 5 +VN 172.16.130.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O29" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,39.245;8.9726,42.595"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O32"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.8826,40.3063;8.17129,41.9099"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 6 +VN 172.16.132.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O31" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O33"> + <dia:attribute name="obj_pos"> + <dia:point val="18.2451,6.425"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.1951,6.375;25.1051,9.725"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="18.2451,6.425"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O34"> + <dia:attribute name="obj_pos"> + <dia:point val="21.6501,8.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="19.4251,7.43631;23.8938,9.03988"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 1 +VN 172.16.0.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="21.6501,8.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O33" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O35"> + <dia:attribute name="obj_pos"> + <dia:point val="29.6551,6.425"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="29.6051,6.375;36.5151,9.725"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="29.6551,6.425"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O36"> + <dia:attribute name="obj_pos"> + <dia:point val="33.0601,8.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="30.8338,7.43631;35.305,9.03988"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 2 +VN 172.16.2.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="33.0601,8.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O35" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O37"> + <dia:attribute name="obj_pos"> + <dia:point val="41.0676,6.425"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.0176,6.375;47.9276,9.725"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.0676,6.425"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O38"> + <dia:attribute name="obj_pos"> + <dia:point val="44.4726,8.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.8376,7.43631;47.1263,9.03988"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 3 +VN 172.16.128.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="44.4726,8.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O37" connection="8"/> + </dia:connections> + </dia:object> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-mesh.png b/doc/figures/fig-vnc-mesh.png Binary files differnew file mode 100644 index 0000000..fa0762d --- /dev/null +++ b/doc/figures/fig-vnc-mesh.png diff --git a/doc/figures/fig-vnc-mesh.txt b/doc/figures/fig-vnc-mesh.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-mesh.txt diff --git a/doc/figures/fig-vnc-redundant-route-reflectors.dia b/doc/figures/fig-vnc-redundant-route-reflectors.dia new file mode 100644 index 0000000..4065b8b --- /dev/null +++ b/doc/figures/fig-vnc-redundant-route-reflectors.dia @@ -0,0 +1,871 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#Letter#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.5399999618530273"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true" active="true"> + <dia:object type="Standard - Box" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,39.095;64.0901,42.445"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,39.145"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,34.1975;64.0901,37.5475"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,34.2475"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="57.1801,29.3;64.0901,32.65"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="57.2301,29.35"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,39.245;8.9726,42.595"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,39.295"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,34.3475;8.9726,37.6975"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,34.3975"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Box" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.0626,29.45;8.9726,32.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="2.1126,29.5"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="6.8100000000000023"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3.2500000000000018"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="22.5347,32.178"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.4648,17.4572;42.6316,32.2479"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="22.5347,32.178"/> + <dia:point val="42.5617,17.5271"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O26" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="43.1205,32.4705"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="22.6348,17.9948;43.1902,32.5402"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="43.1205,32.4705"/> + <dia:point val="22.7045,18.0645"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="1" to="O24" connection="7"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.5501,4.905;16.5501,5.6525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>##</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="16.5501,5.5"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.29385,30.5113;7.76004,32.1149"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 4 +VN 172.16.4.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,31.125"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.88385,35.4088;8.17004,37.0124"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 5 +VN 172.16.130.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,36.0225"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.8826,40.3063;8.17129,41.9099"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 6 +VN 172.16.132.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="5.5176,40.92"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,30.3613;62.8788,31.9649"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 7 +VN 172.16.6.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,30.975"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.4101,35.2588;62.8788,36.8624"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 8 +VN 172.16.8.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,35.8725"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O14"> + <dia:attribute name="obj_pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="58.0026,40.1563;63.2863,41.7599"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVE 9 +VN 172.16.134.1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="60.6351,40.77"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O15"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,31.125"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86398,31.0664;15.9041,32.5291"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,31.125"/> + <dia:point val="15.8455,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O5" connection="4"/> + <dia:connection handle="1" to="O28" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O16"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,36.0225"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.8715,35.8489;14.4761,36.0736"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,36.0225"/> + <dia:point val="14.425,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="4"/> + <dia:connection handle="1" to="O28" connection="3"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O17"> + <dia:attribute name="obj_pos"> + <dia:point val="8.9226,40.92"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.86267,39.2696;15.9054,40.9799"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.9226,40.92"/> + <dia:point val="15.8455,39.3295"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O3" connection="4"/> + <dia:connection handle="1" to="O28" connection="5"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O18"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,30.975"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="49.8855,30.916;57.2891,32.5295"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,30.975"/> + <dia:point val="49.9445,32.4705"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O2" connection="3"/> + <dia:connection handle="1" to="O30" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O19"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,35.8725"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="51.3148,35.8223;57.2803,35.9502"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,35.8725"/> + <dia:point val="51.365,35.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O1" connection="3"/> + <dia:connection handle="1" to="O30" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O20"> + <dia:attribute name="obj_pos"> + <dia:point val="57.2301,40.77"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="50.9097,37.8613;57.2963,40.8362"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="57.2301,40.77"/> + <dia:point val="50.9759,37.9275"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O0" connection="3"/> + <dia:connection handle="1" to="O30" connection="8"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O21"> + <dia:attribute name="obj_pos"> + <dia:point val="24.25,15.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="24.1999,14.9999;41.6501,15.1501"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="24.25,15.05"/> + <dia:point val="41.6,15.1"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O22"> + <dia:attribute name="obj_pos"> + <dia:point val="19.1,31"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="18.9998,19.5998;19.1502,31.0502"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="19.1,31"/> + <dia:point val="19.05,19.65"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O23"> + <dia:attribute name="obj_pos"> + <dia:point val="46.4,19.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="46.35,19.55;46.45,30.95"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="46.4,19.6"/> + <dia:point val="46.4,30.9"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O24"> + <dia:attribute name="obj_pos"> + <dia:point val="14.425,9.785"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.375,9.735;24.175,19.535"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14.425,9.785"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O25"> + <dia:attribute name="obj_pos"> + <dia:point val="19.275,14.635"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="15.7475,14.04;22.8025,15.5875"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#BGP Route Reflector 1 +192.168.1.100#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.275,14.635"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O24" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O26"> + <dia:attribute name="obj_pos"> + <dia:point val="41.665,9.785"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.615,9.735;51.415,19.535"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.665,9.785"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O27"> + <dia:attribute name="obj_pos"> + <dia:point val="46.565,14.285"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="44.1013,13.69;49.0288,16.0375"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#Commercial Router +Route Reflector +192.168.1.104#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="46.565,14.285"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O28"> + <dia:attribute name="obj_pos"> + <dia:point val="14.425,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="14.375,31;24.175,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="14.425,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000007629394531"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O29"> + <dia:attribute name="obj_pos"> + <dia:point val="19.275,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="16.9262,35.305;21.6238,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 2 +192.168.1.101#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="19.275,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O28" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O30"> + <dia:attribute name="obj_pos"> + <dia:point val="41.665,31.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="41.615,31;51.415,40.8"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="41.665,31.05"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="9.7000026702880859"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="1" id="O31"> + <dia:attribute name="obj_pos"> + <dia:point val="46.515,35.9"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="44.1662,35.305;48.8637,36.8525"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#NVA 3 +192.168.1.102#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="0.80000000000000004"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="46.515,35.9"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="1"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="valign"> + <dia:enum val="3"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O30" connection="8"/> + </dia:connections> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig-vnc-redundant-route-reflectors.png b/doc/figures/fig-vnc-redundant-route-reflectors.png Binary files differnew file mode 100644 index 0000000..06a27b6 --- /dev/null +++ b/doc/figures/fig-vnc-redundant-route-reflectors.png diff --git a/doc/figures/fig-vnc-redundant-route-reflectors.txt b/doc/figures/fig-vnc-redundant-route-reflectors.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/doc/figures/fig-vnc-redundant-route-reflectors.txt diff --git a/doc/figures/fig_dmvpn_topologies.png b/doc/figures/fig_dmvpn_topologies.png Binary files differnew file mode 100644 index 0000000..a0dcc3e --- /dev/null +++ b/doc/figures/fig_dmvpn_topologies.png diff --git a/doc/figures/fig_topologies_full.dia b/doc/figures/fig_topologies_full.dia new file mode 100644 index 0000000..7ec3398 --- /dev/null +++ b/doc/figures/fig_topologies_full.dia @@ -0,0 +1,533 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#A4#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true"> + <dia:object type="Standard - Line" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="8,11.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.9,11;8.1,12"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8,11.5"/> + <dia:point val="3,11.5"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O6" connection="3"/> + <dia:connection handle="1" to="O8" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="1.5,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="1,4.9;2,10.1"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="1.5,10"/> + <dia:point val="1.5,5"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O8" connection="1"/> + <dia:connection handle="1" to="O10" connection="6"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="2.56066,4.56066"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.1364,4.1364;8.8636,10.8636"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="2.56066,4.56066"/> + <dia:point val="8.43934,10.4393"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O10" connection="7"/> + <dia:connection handle="1" to="O6" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="8.43934,4.56066"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.1364,4.1364;8.8636,10.8636"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.43934,4.56066"/> + <dia:point val="2.56066,10.4393"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="5"/> + <dia:connection handle="1" to="O8" connection="2"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="8,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,1.95;11.05,5.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="8.15,4.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.15,2.555;10.945,5.04"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.15,4.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="8,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,9.95;11.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="8.15,12.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.105,10.555;10.9,13.04"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF4#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.15,12.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="0,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.05,9.95;3.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="0,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="0.2,12.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.2,10.555;2.995,13.04"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF3#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0.2,12.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="0,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.05,1.95;3.05,5.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="0,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="0.2,4.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.155,2.555;2.95,5.04"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0.2,4.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:object type="Standard - Line" version="0" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="8,3.5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.9,3;8.1,4"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8,3.5"/> + <dia:point val="3,3.5"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="3"/> + <dia:connection handle="1" to="O10" connection="4"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="9.5,5"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="9,4.9;10,10.1"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="9.5,5"/> + <dia:point val="9.5,10"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="6"/> + <dia:connection handle="1" to="O6" connection="1"/> + </dia:connections> + </dia:object> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig_topologies_full.png b/doc/figures/fig_topologies_full.png Binary files differnew file mode 100644 index 0000000..d39e5e2 --- /dev/null +++ b/doc/figures/fig_topologies_full.png diff --git a/doc/figures/fig_topologies_full.txt b/doc/figures/fig_topologies_full.txt new file mode 100644 index 0000000..cc8025a --- /dev/null +++ b/doc/figures/fig_topologies_full.txt @@ -0,0 +1,6 @@ +(RF1)--(RF2) + | \ / | + | \/ | + | /\ | + | / \ | +(RF3)--(RF4) diff --git a/doc/figures/fig_topologies_rs.dia b/doc/figures/fig_topologies_rs.dia new file mode 100644 index 0000000..f8aa18d --- /dev/null +++ b/doc/figures/fig_topologies_rs.dia @@ -0,0 +1,499 @@ +<?xml version="1.0" encoding="UTF-8"?> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/"> + <dia:diagramdata> + <dia:attribute name="background"> + <dia:color val="#ffffff"/> + </dia:attribute> + <dia:attribute name="pagebreak"> + <dia:color val="#000099"/> + </dia:attribute> + <dia:attribute name="paper"> + <dia:composite type="paper"> + <dia:attribute name="name"> + <dia:string>#A4#</dia:string> + </dia:attribute> + <dia:attribute name="tmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="bmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="lmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="rmargin"> + <dia:real val="2.8222000598907471"/> + </dia:attribute> + <dia:attribute name="is_portrait"> + <dia:boolean val="true"/> + </dia:attribute> + <dia:attribute name="scaling"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="fitto"> + <dia:boolean val="false"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + <dia:attribute name="grid"> + <dia:composite type="grid"> + <dia:attribute name="width_x"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="width_y"> + <dia:real val="1"/> + </dia:attribute> + <dia:attribute name="visible_x"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="visible_y"> + <dia:int val="1"/> + </dia:attribute> + <dia:composite type="color"/> + </dia:composite> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#d8e5e5"/> + </dia:attribute> + <dia:attribute name="guides"> + <dia:composite type="guides"> + <dia:attribute name="hguides"/> + <dia:attribute name="vguides"/> + </dia:composite> + </dia:attribute> + </dia:diagramdata> + <dia:layer name="Background" visible="true"> + <dia:object type="Standard - Line" version="0" id="O0"> + <dia:attribute name="obj_pos"> + <dia:point val="4,8.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.09695,8.07589;4.46371,10.8134"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="4,8.45"/> + <dia:point val="2.56066,10.4393"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="5"/> + <dia:connection handle="1" to="O10" connection="2"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O1"> + <dia:attribute name="obj_pos"> + <dia:point val="4,6.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="2.09449,4.19064;4.46617,6.97002"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="4,6.6"/> + <dia:point val="2.56066,4.56066"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="0"/> + <dia:connection handle="1" to="O12" connection="7"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O2"> + <dia:attribute name="obj_pos"> + <dia:point val="7,8.45"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="6.53629,8.07589;8.90305,10.8134"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="7,8.45"/> + <dia:point val="8.43934,10.4393"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O4" connection="7"/> + <dia:connection handle="1" to="O8" connection="0"/> + </dia:connections> + </dia:object> + <dia:object type="Standard - Line" version="0" id="O3"> + <dia:attribute name="obj_pos"> + <dia:point val="8.43934,4.56066"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="6.53383,4.19064;8.90551,6.97002"/> + </dia:attribute> + <dia:attribute name="conn_endpoints"> + <dia:point val="8.43934,4.56066"/> + <dia:point val="7,6.6"/> + </dia:attribute> + <dia:attribute name="numcp"> + <dia:int val="1"/> + </dia:attribute> + <dia:attribute name="line_color"> + <dia:color val="#ff0000"/> + </dia:attribute> + <dia:attribute name="line_width"> + <dia:real val="0.20000000000000001"/> + </dia:attribute> + <dia:attribute name="start_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="start_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="start_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow"> + <dia:enum val="22"/> + </dia:attribute> + <dia:attribute name="end_arrow_length"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:attribute name="end_arrow_width"> + <dia:real val="0.5"/> + </dia:attribute> + <dia:connections> + <dia:connection handle="0" to="O6" connection="5"/> + <dia:connection handle="1" to="O4" connection="2"/> + </dia:connections> + </dia:object> + <dia:group> + <dia:object type="Standard - Box" version="0" id="O4"> + <dia:attribute name="obj_pos"> + <dia:point val="4,6.6"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="3.95,6.55;7.05,8.5"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="4,6.6"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="1.8500000000000014"/> + </dia:attribute> + <dia:attribute name="show_background"> + <dia:boolean val="true"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O5"> + <dia:attribute name="obj_pos"> + <dia:point val="4.55,8"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="4.55,6.55;6.55,8.9"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RS#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="4.55,8"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O6"> + <dia:attribute name="obj_pos"> + <dia:point val="8,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,1.95;11.05,5.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O7"> + <dia:attribute name="obj_pos"> + <dia:point val="8.15,4.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.15,2.6;10.9,4.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF2#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.15,4.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O8"> + <dia:attribute name="obj_pos"> + <dia:point val="8,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="7.95,9.95;11.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="8,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O9"> + <dia:attribute name="obj_pos"> + <dia:point val="8.15,12.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="8.15,10.6;10.9,12.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF4#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="8.15,12.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O10"> + <dia:attribute name="obj_pos"> + <dia:point val="0,10"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.05,9.95;3.05,13.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="0,10"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O11"> + <dia:attribute name="obj_pos"> + <dia:point val="0.2,12.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.2,10.6;2.95,12.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF3#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0.2,12.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + <dia:group> + <dia:object type="Standard - Ellipse" version="0" id="O12"> + <dia:attribute name="obj_pos"> + <dia:point val="0,2"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="-0.05,1.95;3.05,5.05"/> + </dia:attribute> + <dia:attribute name="elem_corner"> + <dia:point val="0,2"/> + </dia:attribute> + <dia:attribute name="elem_width"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="elem_height"> + <dia:real val="3"/> + </dia:attribute> + <dia:attribute name="aspect"> + <dia:enum val="2"/> + </dia:attribute> + </dia:object> + <dia:object type="Standard - Text" version="0" id="O13"> + <dia:attribute name="obj_pos"> + <dia:point val="0.2,4.05"/> + </dia:attribute> + <dia:attribute name="obj_bb"> + <dia:rectangle val="0.2,2.6;2.95,4.95"/> + </dia:attribute> + <dia:attribute name="text"> + <dia:composite type="text"> + <dia:attribute name="string"> + <dia:string>#RF1#</dia:string> + </dia:attribute> + <dia:attribute name="font"> + <dia:font family="sans" style="0" name="Helvetica"/> + </dia:attribute> + <dia:attribute name="height"> + <dia:real val="2"/> + </dia:attribute> + <dia:attribute name="pos"> + <dia:point val="0.2,4.05"/> + </dia:attribute> + <dia:attribute name="color"> + <dia:color val="#000000"/> + </dia:attribute> + <dia:attribute name="alignment"> + <dia:enum val="0"/> + </dia:attribute> + </dia:composite> + </dia:attribute> + </dia:object> + </dia:group> + </dia:layer> +</dia:diagram> diff --git a/doc/figures/fig_topologies_rs.png b/doc/figures/fig_topologies_rs.png Binary files differnew file mode 100644 index 0000000..014225c --- /dev/null +++ b/doc/figures/fig_topologies_rs.png diff --git a/doc/figures/fig_topologies_rs.txt b/doc/figures/fig_topologies_rs.txt new file mode 100644 index 0000000..0bd1730 --- /dev/null +++ b/doc/figures/fig_topologies_rs.txt @@ -0,0 +1,5 @@ +(RF1) (RF2) + \ / + [RS] + / \ +(RF3) (RF4) diff --git a/doc/figures/frr-icon.svg b/doc/figures/frr-icon.svg new file mode 100644 index 0000000..f7bdebc --- /dev/null +++ b/doc/figures/frr-icon.svg @@ -0,0 +1,39 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + viewBox="0 0 255.16251 264.35001" + height="264.35001" + width="255.16251" + xml:space="preserve" + version="1.1" + id="svg2"><metadata + id="metadata8"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title></dc:title></cc:Work></rdf:RDF></metadata><defs + id="defs6" /><g + transform="matrix(1.25,0,0,-1.25,0,264.35)" + id="g10"><g + transform="scale(0.1,0.1)" + id="g12"><path + id="path14" + style="fill:#f36f16;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 260.984,905.711 c 0,-431.5 345.223,-786.52 788.866,-832.2696 l 0,290.4566 c -274.799,44.071 -483.912,269.543 -484.475,541.813 0.563,272.249 209.676,497.819 484.475,541.849 l 0,290.5 C 606.207,1692.18 261.063,1337.24 260.984,905.711" /><path + id="path16" + style="fill:#f36f16;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 709.863,908.891 c 0,-197.121 146.297,-362.243 343.647,-408.469 l 0,816.938 C 856.16,1271.14 709.863,1106.06 709.863,908.891" /><path + id="path18" + style="fill:#f36f16;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 748.035,2114.83 1.395,-202.42 C 349.336,1773.27 53.4609,1430.82 2.73828,1017.01 L 0,994.961 l 133.66,0 2.309,17.319 c 60.746,449.24 447.246,801.35 928.931,838.53 l 19.35,1.55 -336.215,262.47" /><path + id="path20" + style="fill:#393536;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 1395.52,1123.39 c 0,-51.43 -41.66,-93.14 -93.14,-93.14 -51.42,0 -93.17,41.71 -93.17,93.14 0,51.46 41.75,93.19 93.17,93.19 51.48,0 93.14,-41.73 93.14,-93.19" /><path + id="path22" + style="fill:#f36f16;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 1529.56,476.52 49.41,-72.372 c 118.33,73.59 206.07,171.122 264.08,280.274 l -90.08,18.258 C 1701.77,615.23 1627.9,537.191 1529.56,476.52 Z m 338.86,848.67 -95.9,0 c 44.87,-92.73 68.09,-194.02 68.42,-295.5 0,-11.12 -0.48,-22.22 -1.06,-33.311 l 80.2,-75.578 c 5.3,36.097 8.22,72.5 8.14,108.889 0.34,100.55 -20.33,201.15 -59.8,295.5" /><path + id="path24" + style="fill:#f36f16;fill-opacity:1;fill-rule:nonzero;stroke:none" + d="m 1663.19,790.07 378.12,-76.73 -321,302.64 c 0.23,6.25 0.73,12.42 0.73,18.71 0,284.04 -218.31,520.69 -508.02,574.72 l 0,-127.07 c 215.24,-51.78 373.82,-232.3 374.38,-447.49 -0.56,-215.42 -159.14,-395.901 -374.38,-447.709 l 0,-127.129 C 1214.79,460.289 1537.38,0 1537.38,0 l 164.55,0 -353.76,500.422 c 139.05,58.219 251.25,161.488 315.02,289.648" /></g></g></svg>
\ No newline at end of file diff --git a/doc/figures/frr-logo-icon.png b/doc/figures/frr-logo-icon.png Binary files differnew file mode 100644 index 0000000..37973fc --- /dev/null +++ b/doc/figures/frr-logo-icon.png diff --git a/doc/figures/frr-logo-medium.png b/doc/figures/frr-logo-medium.png Binary files differnew file mode 100644 index 0000000..04e18eb --- /dev/null +++ b/doc/figures/frr-logo-medium.png diff --git a/doc/figures/frr-logo-small.png b/doc/figures/frr-logo-small.png Binary files differnew file mode 100644 index 0000000..2a9a2c7 --- /dev/null +++ b/doc/figures/frr-logo-small.png diff --git a/doc/figures/frr-logo.png b/doc/figures/frr-logo.png Binary files differnew file mode 100644 index 0000000..6e66414 --- /dev/null +++ b/doc/figures/frr-logo.png diff --git a/doc/figures/git_branches.png b/doc/figures/git_branches.png Binary files differnew file mode 100644 index 0000000..21001c3 --- /dev/null +++ b/doc/figures/git_branches.png diff --git a/doc/figures/git_branches.svg b/doc/figures/git_branches.svg new file mode 100644 index 0000000..0c2c96e --- /dev/null +++ b/doc/figures/git_branches.svg @@ -0,0 +1,720 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + version="1.1" + viewBox="52 37 349.25195 651.46875" + width="349.25195" + height="651.46875" + id="svg2" + inkscape:version="0.91 r13725" + sodipodi:docname="git_branches.svg"> + <metadata + id="metadata376"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <defs + id="defs374" /> + <sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1596" + inkscape:window-height="848" + id="namedview372" + showgrid="false" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" + inkscape:zoom="0.89517435" + inkscape:cx="53.149337" + inkscape:cy="353.95197" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" + inkscape:current-layer="svg2" /> + <g + id="g12" + transform="translate(-3.09375,-3.7812531)"> + <path + d="M 216,98.125 216,684" + id="path14" + inkscape:connector-curvature="0" + style="fill:none;stroke:#000100;stroke-width:8" /> + <path + d="M 205,99.5 216,72 227,99.5" + id="path16" + inkscape:connector-curvature="0" + style="fill:#000100" /> + <path + d="M 225.625,692.25 216,684 l -9.625,8.25 0,-30.25 9.625,-8.25 9.625,8.25" + id="path18" + inkscape:connector-curvature="0" + style="fill:#000100" /> + </g> + <path + d="m 212.90625,644.21875 -72,-36" + id="path20" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,428.21875 -72,-36" + id="path22" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <text + font-weight="bold" + font-size="12" + xml:space="preserve" + id="text30" + style="font-weight:bold;font-size:12px;font-family:Helvetica;text-anchor:middle;fill:#ff0000" + x="143.89426" + y="427.82574"><tspan + id="tspan32">1.0</tspan><tspan + x="143.89426" + y="443.82574" + id="tspan34">Release</tspan><tspan + x="143.89426" + y="459.82574" + id="tspan36">Branch</tspan></text> + <path + d="m 213.73825,392.21875 -72.832,-36" + id="path42" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.73825,356.21875 -71.832,-36" + id="path44" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 220.90625,644.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path46" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,464.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path48" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,428.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path50" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,608.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path52" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,140.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path54" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,356.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path58" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,392.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path60" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <text + font-weight="bold" + font-size="12" + xml:space="preserve" + id="text62" + style="font-weight:bold;font-size:12px;font-family:Helvetica;text-anchor:middle;fill:#000100" + x="213.21825" + y="45.748047"><tspan + id="tspan64">Master</tspan><tspan + x="213.21825" + y="61.748047" + id="tspan66">(Stable)</tspan></text> + <g + id="g82" + transform="translate(-3.09375,-3.7812531)"> + <path + d="m 144.832,612 0,-117.875" + id="path84" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 133.832,495.5 11,-27.5 11,27.5" + id="path86" + inkscape:connector-curvature="0" + style="fill:#ff0000" /> + </g> + <g + id="g88" + transform="translate(-3.09375,-3.7812531)"> + <path + d="m 144.832,396 0,-189.875" + id="path90" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 133.832,207.5 11,-27.5 11,27.5" + id="path92" + inkscape:connector-curvature="0" + style="fill:#ff0000" /> + </g> + <text + font-weight="bold" + font-size="12" + xml:space="preserve" + id="text94" + style="font-weight:bold;font-size:12px;font-family:Helvetica;text-anchor:middle;fill:#ff0000" + x="143.89426" + y="138.82574"><tspan + id="tspan96">1.1</tspan><tspan + x="143.89426" + y="154.82574" + id="tspan98">Release</tspan><tspan + x="143.89426" + y="170.82574" + id="tspan100">Branch</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text102" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="51.90625" + y="611.97876"><tspan + id="tspan104">Version 1.</tspan><tspan + id="tspan106">0.a1</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text108" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.679649" + y="395.97876"><tspan + id="tspan110">Version 1.1.a1</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text112" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="51.90625" + y="359.97876"><tspan + id="tspan114">Version 1.1.a2</tspan></text> + <path + d="m 148.90625,608.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path116" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,392.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path118" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,356.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path120" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,320.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path122" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <text + font-size="12" + xml:space="preserve" + id="text124" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.679649" + y="323.97876"><tspan + id="tspan126">Version 1.</tspan><tspan + id="tspan128">1.b1</tspan></text> + <g + id="g156" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,540 -127.375,0" + id="path158" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,547 -17.5,-7 17.5,-7" + id="path160" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g162" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,504 -127.375,0" + id="path164" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,511 -17.5,-7 17.5,-7" + id="path166" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g168" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,468 -127.375,0" + id="path170" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,475 -17.5,-7 17.5,-7" + id="path172" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g174" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,432 -127.375,0" + id="path176" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,439 -17.5,-7 17.5,-7" + id="path178" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g180" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,396 -127.375,0" + id="path182" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,403 -17.5,-7 17.5,-7" + id="path184" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g186" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,360 -127.375,0" + id="path188" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,367 -17.5,-7 17.5,-7" + id="path190" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g192" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,324 -127.375,0" + id="path194" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,331 -17.5,-7 17.5,-7" + id="path196" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g198" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,288 -127.375,0" + id="path200" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,295 -17.5,-7 17.5,-7" + id="path202" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g204" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,252 -127.375,0" + id="path206" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,259 -17.5,-7 17.5,-7" + id="path208" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g210" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,216 -127.375,0" + id="path212" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,223 -17.5,-7 17.5,-7" + id="path214" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g216" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 404,180 -99.375,0" + id="path218" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,187 -17.5,-7 17.5,-7" + id="path220" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g222" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,180 -127.375,0" + id="path224" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,187 -17.5,-7 17.5,-7" + id="path226" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g228" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,144 -127.375,0" + id="path230" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,151 -17.5,-7 17.5,-7" + id="path232" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <text + font-size="12" + xml:space="preserve" + id="text234" + x="256.90625" + y="531.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan236">Patch Email (Patchwork)</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text238" + x="256.90625" + y="495.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan240">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text242" + x="256.90625" + y="459.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan244">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text246" + x="256.90625" + y="423.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan248">Patch Email (Patchwork)</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text250" + x="256.90625" + y="387.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan252">Patch Email (Patchwork)</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text254" + x="256.90625" + y="351.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan256">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text258" + x="256.90625" + y="315.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan260">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text262" + x="256.90625" + y="279.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan264">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text266" + x="256.90625" + y="243.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan268">Patch Email (Patchwork)</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text270" + x="256.90625" + y="207.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan272">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text274" + x="256.90625" + y="171.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan276">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text278" + x="256.90625" + y="135.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan280">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text282" + x="256.90625" + y="603.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan284">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text286" + x="256.90625" + y="567.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan288">Github Pull Request</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text290" + x="256.90625" + y="567.21875" + style="font-size:12px;font-family:Helvetica;fill:#000100"><tspan + id="tspan292">Github Pull Request</tspan></text> + <g + id="g294" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,612 -127.72,0" + id="path296" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.155,619 -17.5,-7 17.5,-7" + id="path298" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <g + id="g300" + transform="translate(-67.09375,-3.7812531)"> + <path + d="m 432,576 -127.375,0" + id="path302" + inkscape:connector-curvature="0" + style="fill:none;stroke:#666666;stroke-width:4" /> + <path + d="m 305.5,583 -17.5,-7 17.5,-7" + id="path304" + inkscape:connector-curvature="0" + style="fill:#666666" /> + </g> + <text + font-size="12" + xml:space="preserve" + id="text306" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.90625" + y="577.21875"><tspan + id="tspan308">Version 1.</tspan><tspan + id="tspan310">0.a2</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text312" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.90625" + y="541.21875"><tspan + id="tspan314">Version 1.</tspan><tspan + id="tspan316">0.b1</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text318" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.90625" + y="505.21875"><tspan + id="tspan320">Version 1.</tspan><tspan + id="tspan322">0.0</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text324" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.679649" + y="288.97876"><tspan + id="tspan326">Version 1.1.0</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text328" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="51.90625" + y="252.97874"><tspan + id="tspan330">Version 1.1.1</tspan></text> + <text + font-size="12" + xml:space="preserve" + id="text332" + style="font-size:12px;font-family:Helvetica;fill:#000100" + x="52.679649" + y="216.97874"><tspan + id="tspan334">Version 1.</tspan><tspan + id="tspan336">1.2</tspan></text> + <path + d="m 212.90625,608.21875 -72,-36" + id="path338" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,572.21875 -72,-36" + id="path340" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,536.21875 -72,-36" + id="path342" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,320.21875 -72,-36" + id="path344" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,284.21875 -72,-36" + id="path346" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 212.90625,248.21875 -72,-36" + id="path348" + inkscape:connector-curvature="0" + style="fill:none;stroke:#ff0000;stroke-width:8" /> + <path + d="m 148.90625,571.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path350" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,535.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path352" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,499.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path354" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,284.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path356" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,248.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path358" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 148.90625,212.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path360" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,572.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path362" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,536.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path364" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,320.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path366" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,248.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path368" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> + <path + d="m 220.90625,284.21875 c 0,4.418 -3.582,8 -8,8 -4.418,0 -8,-3.582 -8,-8 0,-4.418 3.582,-8 8,-8 4.418,0 8,3.582 8,8 z" + id="path370" + inkscape:connector-curvature="0" + style="fill:#fffffe;stroke:#000100" /> +</svg> diff --git a/doc/figures/nodes.dot b/doc/figures/nodes.dot new file mode 100644 index 0000000..4ce147b --- /dev/null +++ b/doc/figures/nodes.dot @@ -0,0 +1,63 @@ +/* + * FRR CLI modes and their relationships. + * + * Each edge is labeled with the command that causes a transition along that + * edge. Exit commands and their back edges are implicit. + */ +digraph climodes { + ratio = "auto" + mincross = 2.0 + graph [fontsize = 9] + rankdir = LR + + AUTH_NODE -> VIEW_NODE -> ENABLE_NODE; + VIEW_NODE -> AUTH_ENABLE_NODE -> ENABLE_NODE; + ENABLE_NODE -> CONFIG_NODE [ label="configure terminal" ]; + CONFIG_NODE -> RIP_NODE [ label="router rip" ]; + CONFIG_NODE -> RIPNG_NODE [ label="router ripng" ]; + CONFIG_NODE -> BABEL_NODE [ label="router babel" ]; + CONFIG_NODE -> EIGRP_NODE [ label="router eigrp (1-65535)" ]; + CONFIG_NODE -> BGP_NODE [ label="router bgp ASN" ]; + subgraph cluster0 { + BGP_NODE -> BGP_VPNV4_NODE [ label="address-family vpnv4 [unicast]" ]; + BGP_NODE -> BGP_VPNV6_NODE [ label="address-family vpnv6 [unicast]" ]; + BGP_NODE -> BGP_IPV4_NODE [ label="address-family ipv4 [unicast]" ]; + BGP_NODE -> BGP_IPV4L_NODE [ label="address-family ipv4 labeled-unicast" ]; + BGP_NODE -> BGP_IPV4M_NODE [ label="address-family ipv4 multicast" ]; + BGP_NODE -> BGP_IPV6_NODE [ label="address-family ipv6 [unicast]" ]; + BGP_NODE -> BGP_IPV6L_NODE [ label="address-family ipv6 labeled-unicast" ]; + BGP_NODE -> BGP_IPV6M_NODE [ label="address-family ipv6 multicast" ]; + BGP_NODE -> BGP_FLOWSPECV4_NODE [ label="address-family ipv4 flowspec" ]; + BGP_NODE -> BGP_FLOWSPECV6_NODE [ label="address-family ipv6 flowspec" ]; + BGP_NODE -> BGP_EVPN_NODE [ label="address-family l2vpn evpn" ]; + BGP_EVPN_NODE -> BGP_EVPN_VNI_NODE [ label="vni (1-16777215)" ]; + BGP_NODE -> BGP_VRF_POLICY_NODE [ label="vrf-policy NAME" ]; + BGP_NODE -> BGP_VNC_DEFAULTS_NODE [ label="vnc defaults" ]; + BGP_NODE -> BGP_VNC_NVE_GROUP_NODE [ label="vnc nve-group NAME" ]; + BGP_NODE -> BGP_VNC_L2_GROUP_NODE [ label="vnc l2-group NAME" ]; + } + subgraph cluster1 { + LDP_NODE -> LDP_IPV4_NODE [ label="address-family ipv4" ]; + LDP_NODE -> LDP_IPV6_NODE [ label="address-family ipv6" ]; + LDP_IPV4_NODE -> LDP_IPV4_IFACE_NODE [ label="interface IFNAME" ]; + LDP_IPV6_NODE -> LDP_IPV6_IFACE_NODE [ label="interface IFNAME" ]; + LDP_NODE -> LDP_L2VPN_NODE [ label="address-family l2vpn WORD type vpls" ]; + LDP_NODE -> LDP_PSEUDOWIRE_NODE [ label="member pseudowire IFNAME" ]; + } + CONFIG_NODE -> OSPF_NODE [ label="router ospf [(1-65535)] [vrf NAME]" ]; + CONFIG_NODE -> OSPF6_NODE [ label="router ospf6" ]; + CONFIG_NODE -> LDP_NODE [ label="mpls ldp" ]; + CONFIG_NODE -> ISIS_NODE [ label="router isis WORD [vrf NAME]" ]; + CONFIG_NODE -> RMAP_NODE [ label="route-map WORD <deny|permit> (1-65535)" ]; + CONFIG_NODE -> PW_NODE [ label="pseudowire IFNAME" ]; + CONFIG_NODE -> VTY_NODE [ label="line vty" ]; + CONFIG_NODE -> KEYCHAIN_NODE [ label="key chain WORD" ]; + CONFIG_NODE -> KEYCHAIN_KEY_NODE [ label="key (0-2147483647)" ]; + KEYCHAIN_NODE -> KEYCHAIN_KEY_NODE [ label="key (0-2147483647)" ]; + KEYCHAIN_KEY_NODE -> KEYCHAIN_NODE [ label="no key (0-2147483647)" ]; + CONFIG_NODE -> VRF_NODE [ label="vrf NAME" ]; + CONFIG_NODE -> INTERFACE_NODE [ label="interface IFNAME vrf NAME" ]; + INTERFACE_NODE -> LINK_PARAMS_NODE [ label="link-params" ]; + CONFIG_NODE -> NH_GROUP_NODE [ label="nexthop-group NAME" ]; + CONFIG_NODE -> RPKI_NODE [ label="rpki" ]; +} diff --git a/doc/figures/ospf_api_architecture.png b/doc/figures/ospf_api_architecture.png Binary files differnew file mode 100644 index 0000000..bd10a38 --- /dev/null +++ b/doc/figures/ospf_api_architecture.png diff --git a/doc/figures/ospf_api_msghdr.png b/doc/figures/ospf_api_msghdr.png Binary files differnew file mode 100644 index 0000000..5bc840f --- /dev/null +++ b/doc/figures/ospf_api_msghdr.png diff --git a/doc/figures/ospf_api_msgs1.png b/doc/figures/ospf_api_msgs1.png Binary files differnew file mode 100644 index 0000000..a23cc61 --- /dev/null +++ b/doc/figures/ospf_api_msgs1.png diff --git a/doc/figures/ospf_api_msgs2.png b/doc/figures/ospf_api_msgs2.png Binary files differnew file mode 100644 index 0000000..8997ac4 --- /dev/null +++ b/doc/figures/ospf_api_msgs2.png diff --git a/doc/figures/pcep_module_threading_overview.svg b/doc/figures/pcep_module_threading_overview.svg new file mode 100644 index 0000000..4d2d2a2 --- /dev/null +++ b/doc/figures/pcep_module_threading_overview.svg @@ -0,0 +1,481 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg version="1.2" width="210mm" height="297mm" viewBox="0 0 21000 29700" preserveAspectRatio="xMidYMid" fill-rule="evenodd" stroke-width="28.222" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" xmlns:ooo="http://xml.openoffice.org/svg/export" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:presentation="http://sun.com/xmlns/staroffice/presentation" xmlns:smil="http://www.w3.org/2001/SMIL20/" xmlns:anim="urn:oasis:names:tc:opendocument:xmlns:animation:1.0" xml:space="preserve"> + <defs class="ClipPathGroup"> + <clipPath id="presentation_clip_path" clipPathUnits="userSpaceOnUse"> + <rect x="0" y="0" width="21000" height="29700"/> + </clipPath> + <clipPath id="presentation_clip_path_shrink" clipPathUnits="userSpaceOnUse"> + <rect x="21" y="29" width="20958" height="29641"/> + </clipPath> + </defs> + <defs> + <font id="EmbeddedFont_1" horiz-adv-x="2048"> + <font-face font-family="Liberation Sans embedded" units-per-em="2048" font-weight="normal" font-style="normal" ascent="1852" descent="423"/> + <missing-glyph horiz-adv-x="2048" d="M 0,0 L 2047,0 2047,2047 0,2047 0,0 Z"/> + <glyph unicode="u" horiz-adv-x="874" d="M 314,1082 L 314,396 C 314,325 321,269 335,230 349,191 371,162 402,145 433,128 478,119 537,119 624,119 692,149 742,208 792,267 817,350 817,455 L 817,1082 997,1082 997,231 C 997,105 999,28 1003,0 L 833,0 C 832,3 832,12 831,27 830,42 830,59 829,78 828,97 826,132 825,185 L 822,185 C 781,110 733,58 679,27 624,-4 557,-20 476,-20 357,-20 271,10 216,69 161,128 133,225 133,361 L 133,1082 314,1082 Z"/> + <glyph unicode="t" horiz-adv-x="531" d="M 554,8 C 495,-8 434,-16 372,-16 228,-16 156,66 156,229 L 156,951 31,951 31,1082 163,1082 216,1324 336,1324 336,1082 536,1082 536,951 336,951 336,268 C 336,216 345,180 362,159 379,138 408,127 450,127 474,127 509,132 554,141 L 554,8 Z"/> + <glyph unicode="r" horiz-adv-x="530" d="M 142,0 L 142,830 C 142,906 140,990 136,1082 L 306,1082 C 311,959 314,886 314,861 L 318,861 C 347,954 380,1017 417,1051 454,1085 507,1102 575,1102 599,1102 623,1099 648,1092 L 648,927 C 624,934 592,937 552,937 477,937 420,905 381,841 342,776 322,684 322,564 L 322,0 142,0 Z"/> + <glyph unicode="p" horiz-adv-x="953" d="M 1053,546 C 1053,169 920,-20 655,-20 488,-20 376,43 319,168 L 314,168 C 317,163 318,106 318,-2 L 318,-425 138,-425 138,861 C 138,972 136,1046 132,1082 L 306,1082 C 307,1079 308,1070 309,1054 310,1037 312,1012 314,978 315,944 316,921 316,908 L 320,908 C 352,975 394,1024 447,1055 500,1086 569,1101 655,1101 788,1101 888,1056 954,967 1020,878 1053,737 1053,546 Z M 864,542 C 864,693 844,800 803,865 762,930 698,962 609,962 538,962 482,947 442,917 401,887 371,840 350,777 329,713 318,630 318,528 318,386 341,281 386,214 431,147 505,113 607,113 696,113 762,146 803,212 844,277 864,387 864,542 Z"/> + <glyph unicode="o" horiz-adv-x="980" d="M 1053,542 C 1053,353 1011,212 928,119 845,26 724,-20 565,-20 407,-20 288,28 207,125 126,221 86,360 86,542 86,915 248,1102 571,1102 736,1102 858,1057 936,966 1014,875 1053,733 1053,542 Z M 864,542 C 864,691 842,800 798,868 753,935 679,969 574,969 469,969 393,935 346,866 299,797 275,689 275,542 275,399 298,292 345,221 391,149 464,113 563,113 671,113 748,148 795,217 841,286 864,395 864,542 Z"/> + <glyph unicode="n" horiz-adv-x="874" d="M 825,0 L 825,686 C 825,757 818,813 804,852 790,891 768,920 737,937 706,954 661,963 602,963 515,963 447,933 397,874 347,815 322,732 322,627 L 322,0 142,0 142,851 C 142,977 140,1054 136,1082 L 306,1082 C 307,1079 307,1070 308,1055 309,1040 310,1024 311,1005 312,986 313,950 314,897 L 317,897 C 358,972 406,1025 461,1056 515,1087 582,1102 663,1102 782,1102 869,1073 924,1014 979,955 1006,857 1006,721 L 1006,0 825,0 Z"/> + <glyph unicode="l" horiz-adv-x="187" d="M 138,0 L 138,1484 318,1484 318,0 138,0 Z"/> + <glyph unicode="i" horiz-adv-x="187" d="M 137,1312 L 137,1484 317,1484 317,1312 137,1312 Z M 137,0 L 137,1082 317,1082 317,0 137,0 Z"/> + <glyph unicode="h" horiz-adv-x="874" d="M 317,897 C 356,968 402,1020 457,1053 511,1086 580,1102 663,1102 780,1102 867,1073 923,1015 978,956 1006,858 1006,721 L 1006,0 825,0 825,686 C 825,762 818,819 804,856 790,893 767,920 735,937 703,954 659,963 602,963 517,963 450,934 399,875 348,816 322,737 322,638 L 322,0 142,0 142,1484 322,1484 322,1098 C 322,1057 321,1015 319,972 316,929 315,904 314,897 L 317,897 Z"/> + <glyph unicode="e" horiz-adv-x="980" d="M 276,503 C 276,379 302,283 353,216 404,149 479,115 578,115 656,115 719,131 766,162 813,193 844,233 861,281 L 1019,236 C 954,65 807,-20 578,-20 418,-20 296,28 213,123 129,218 87,360 87,548 87,727 129,864 213,959 296,1054 416,1102 571,1102 889,1102 1048,910 1048,527 L 1048,503 276,503 Z M 862,641 C 852,755 823,838 775,891 727,943 658,969 568,969 481,969 412,940 361,882 310,823 282,743 278,641 L 862,641 Z"/> + <glyph unicode="d" horiz-adv-x="927" d="M 821,174 C 788,105 744,55 689,25 634,-5 565,-20 484,-20 347,-20 247,26 183,118 118,210 86,349 86,536 86,913 219,1102 484,1102 566,1102 634,1087 689,1057 744,1027 788,979 821,914 L 823,914 821,1035 821,1484 1001,1484 1001,223 C 1001,110 1003,36 1007,0 L 835,0 C 833,11 831,35 829,74 826,113 825,146 825,174 L 821,174 Z M 275,542 C 275,391 295,282 335,217 375,152 440,119 530,119 632,119 706,154 752,225 798,296 821,405 821,554 821,697 798,802 752,869 706,936 633,969 532,969 441,969 376,936 336,869 295,802 275,693 275,542 Z"/> + <glyph unicode="c" horiz-adv-x="901" d="M 275,546 C 275,402 298,295 343,226 388,157 457,122 548,122 612,122 666,139 709,174 752,209 778,262 788,334 L 970,322 C 956,218 912,135 837,73 762,11 668,-20 553,-20 402,-20 286,28 207,124 127,219 87,359 87,542 87,724 127,863 207,959 287,1054 402,1102 551,1102 662,1102 754,1073 827,1016 900,959 945,880 964,779 L 779,765 C 770,825 746,873 708,908 670,943 616,961 546,961 451,961 382,929 339,866 296,803 275,696 275,546 Z"/> + <glyph unicode="b" horiz-adv-x="953" d="M 1053,546 C 1053,169 920,-20 655,-20 573,-20 505,-5 451,25 396,54 352,102 318,168 L 316,168 C 316,147 315,116 312,74 309,31 307,7 306,0 L 132,0 C 136,36 138,110 138,223 L 138,1484 318,1484 318,1061 C 318,1018 317,967 314,908 L 318,908 C 351,977 396,1027 451,1057 506,1087 574,1102 655,1102 792,1102 892,1056 957,964 1021,872 1053,733 1053,546 Z M 864,540 C 864,691 844,800 804,865 764,930 699,963 609,963 508,963 434,928 388,859 341,790 318,680 318,529 318,387 341,282 386,215 431,147 505,113 607,113 698,113 763,147 804,214 844,281 864,389 864,540 Z"/> + <glyph unicode="a" horiz-adv-x="1060" d="M 414,-20 C 305,-20 224,9 169,66 114,123 87,202 87,302 87,414 124,500 198,560 271,620 390,652 554,656 L 797,660 797,719 C 797,807 778,870 741,908 704,946 645,965 565,965 484,965 426,951 389,924 352,897 330,853 323,793 L 135,810 C 166,1005 310,1102 569,1102 705,1102 807,1071 876,1009 945,946 979,856 979,738 L 979,272 C 979,219 986,179 1000,152 1014,125 1041,111 1080,111 1097,111 1117,113 1139,118 L 1139,6 C 1094,-5 1047,-10 1000,-10 933,-10 885,8 855,43 824,78 807,132 803,207 L 797,207 C 751,124 698,66 637,32 576,-3 501,-20 414,-20 Z M 455,115 C 521,115 580,130 631,160 682,190 723,231 753,284 782,336 797,390 797,445 L 797,534 600,530 C 515,529 451,520 408,504 364,488 330,463 307,430 284,397 272,353 272,299 272,240 288,195 320,163 351,131 396,115 455,115 Z"/> + <glyph unicode="_" horiz-adv-x="1218" d="M -31,-407 L -31,-277 1162,-277 1162,-407 -31,-407 Z"/> + <glyph unicode="Y" horiz-adv-x="1298" d="M 777,584 L 777,0 587,0 587,584 45,1409 255,1409 684,738 1111,1409 1321,1409 777,584 Z"/> + <glyph unicode="X" horiz-adv-x="1298" d="M 1112,0 L 689,616 257,0 46,0 582,732 87,1409 298,1409 690,856 1071,1409 1282,1409 800,739 1323,0 1112,0 Z"/> + <glyph unicode="V" horiz-adv-x="1377" d="M 782,0 L 584,0 9,1409 210,1409 600,417 684,168 768,417 1156,1409 1357,1409 782,0 Z"/> + <glyph unicode="T" horiz-adv-x="1192" d="M 720,1253 L 720,0 530,0 530,1253 46,1253 46,1409 1204,1409 1204,1253 720,1253 Z"/> + <glyph unicode="S" horiz-adv-x="1192" d="M 1272,389 C 1272,259 1221,158 1120,87 1018,16 875,-20 690,-20 347,-20 148,99 93,338 L 278,375 C 299,290 345,228 414,189 483,149 578,129 697,129 820,129 916,150 983,193 1050,235 1083,297 1083,379 1083,425 1073,462 1052,491 1031,520 1001,543 963,562 925,581 880,596 827,609 774,622 716,635 652,650 541,675 456,699 399,724 341,749 295,776 262,807 229,837 203,872 186,913 168,954 159,1000 159,1053 159,1174 205,1267 298,1332 390,1397 522,1430 694,1430 854,1430 976,1406 1061,1357 1146,1308 1205,1224 1239,1106 L 1051,1073 C 1030,1148 991,1202 933,1236 875,1269 795,1286 692,1286 579,1286 493,1267 434,1230 375,1193 345,1137 345,1063 345,1020 357,984 380,956 403,927 436,903 479,884 522,864 609,840 738,811 781,801 825,791 868,781 911,770 952,758 991,744 1030,729 1067,712 1102,693 1136,674 1166,650 1191,622 1216,594 1236,561 1251,523 1265,485 1272,440 1272,389 Z"/> + <glyph unicode="R" horiz-adv-x="1244" d="M 1164,0 L 798,585 359,585 359,0 168,0 168,1409 831,1409 C 990,1409 1112,1374 1199,1303 1285,1232 1328,1133 1328,1006 1328,901 1298,813 1237,742 1176,671 1091,626 984,607 L 1384,0 1164,0 Z M 1136,1004 C 1136,1086 1108,1149 1053,1192 997,1235 917,1256 812,1256 L 359,1256 359,736 820,736 C 921,736 999,760 1054,807 1109,854 1136,919 1136,1004 Z"/> + <glyph unicode="P" horiz-adv-x="1112" d="M 1258,985 C 1258,852 1215,746 1128,667 1041,588 922,549 773,549 L 359,549 359,0 168,0 168,1409 761,1409 C 919,1409 1041,1372 1128,1298 1215,1224 1258,1120 1258,985 Z M 1066,983 C 1066,1165 957,1256 738,1256 L 359,1256 359,700 746,700 C 959,700 1066,794 1066,983 Z"/> + <glyph unicode="O" horiz-adv-x="1430" d="M 1495,711 C 1495,564 1467,435 1411,324 1354,213 1273,128 1168,69 1063,10 938,-20 795,-20 650,-20 526,9 421,68 316,127 235,212 180,323 125,434 97,563 97,711 97,936 159,1113 282,1240 405,1367 577,1430 797,1430 940,1430 1065,1402 1170,1345 1275,1288 1356,1205 1412,1096 1467,987 1495,859 1495,711 Z M 1300,711 C 1300,886 1256,1024 1169,1124 1081,1224 957,1274 797,1274 636,1274 511,1225 423,1126 335,1027 291,889 291,711 291,534 336,394 425,291 514,187 637,135 795,135 958,135 1083,185 1170,286 1257,386 1300,528 1300,711 Z"/> + <glyph unicode="N" horiz-adv-x="1165" d="M 1082,0 L 328,1200 333,1103 338,936 338,0 168,0 168,1409 390,1409 1152,201 C 1144,332 1140,426 1140,485 L 1140,1409 1312,1409 1312,0 1082,0 Z"/> + <glyph unicode="M" horiz-adv-x="1377" d="M 1366,0 L 1366,940 C 1366,1044 1369,1144 1375,1240 1342,1121 1313,1027 1287,960 L 923,0 789,0 420,960 364,1130 331,1240 334,1129 338,940 338,0 168,0 168,1409 419,1409 794,432 C 807,393 820,351 833,306 845,261 853,228 857,208 862,235 874,275 891,330 908,384 919,418 925,432 L 1293,1409 1538,1409 1538,0 1366,0 Z"/> + <glyph unicode="L" horiz-adv-x="927" d="M 168,0 L 168,1409 359,1409 359,156 1071,156 1071,0 168,0 Z"/> + <glyph unicode="I" horiz-adv-x="213" d="M 189,0 L 189,1409 380,1409 380,0 189,0 Z"/> + <glyph unicode="H" horiz-adv-x="1165" d="M 1121,0 L 1121,653 359,653 359,0 168,0 168,1409 359,1409 359,813 1121,813 1121,1409 1312,1409 1312,0 1121,0 Z"/> + <glyph unicode="F" horiz-adv-x="1006" d="M 359,1253 L 359,729 1145,729 1145,571 359,571 359,0 168,0 168,1409 1169,1409 1169,1253 359,1253 Z"/> + <glyph unicode="E" horiz-adv-x="1138" d="M 168,0 L 168,1409 1237,1409 1237,1253 359,1253 359,801 1177,801 1177,647 359,647 359,156 1278,156 1278,0 168,0 Z"/> + <glyph unicode="D" horiz-adv-x="1218" d="M 1381,719 C 1381,574 1353,447 1296,338 1239,229 1159,145 1055,87 951,29 831,0 695,0 L 168,0 168,1409 634,1409 C 873,1409 1057,1349 1187,1230 1316,1110 1381,940 1381,719 Z M 1189,719 C 1189,894 1141,1027 1046,1119 950,1210 811,1256 630,1256 L 359,1256 359,153 673,153 C 776,153 867,176 946,221 1024,266 1084,332 1126,417 1168,502 1189,603 1189,719 Z"/> + <glyph unicode="C" horiz-adv-x="1324" d="M 792,1274 C 636,1274 515,1224 428,1124 341,1023 298,886 298,711 298,538 343,400 434,295 524,190 646,137 800,137 997,137 1146,235 1245,430 L 1401,352 C 1343,231 1262,138 1157,75 1052,12 930,-20 791,-20 649,-20 526,10 423,69 319,128 240,212 186,322 131,431 104,561 104,711 104,936 165,1112 286,1239 407,1366 575,1430 790,1430 940,1430 1065,1401 1166,1342 1267,1283 1341,1196 1388,1081 L 1207,1021 C 1174,1103 1122,1166 1050,1209 977,1252 891,1274 792,1274 Z"/> + <glyph unicode="B" horiz-adv-x="1112" d="M 1258,397 C 1258,272 1212,174 1121,105 1030,35 903,0 740,0 L 168,0 168,1409 680,1409 C 1011,1409 1176,1295 1176,1067 1176,984 1153,914 1106,857 1059,800 993,762 908,743 1020,730 1106,692 1167,631 1228,569 1258,491 1258,397 Z M 984,1044 C 984,1120 958,1174 906,1207 854,1240 779,1256 680,1256 L 359,1256 359,810 680,810 C 782,810 858,829 909,868 959,906 984,965 984,1044 Z M 1065,412 C 1065,578 948,661 715,661 L 359,661 359,153 730,153 C 847,153 932,175 985,218 1038,261 1065,326 1065,412 Z"/> + <glyph unicode="A" horiz-adv-x="1377" d="M 1167,0 L 1006,412 364,412 202,0 4,0 579,1409 796,1409 1362,0 1167,0 Z M 685,1265 L 676,1237 C 659,1182 635,1111 602,1024 L 422,561 949,561 768,1026 C 749,1072 731,1124 712,1182 L 685,1265 Z"/> + <glyph unicode=" " horiz-adv-x="556"/> + </font> + </defs> + <defs class="TextShapeIndex"> + <g ooo:slide="id1" ooo:id-list="id3 id4 id5 id6 id7 id8 id9 id10 id11 id12 id13 id14 id15 id16 id17 id18 id19 id20 id21 id22 id23 id24 id25 id26 id27 id28"/> + </defs> + <defs class="EmbeddedBulletChars"> + <g id="bullet-char-template-57356" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 580,1141 L 1163,571 580,0 -4,571 580,1141 Z"/> + </g> + <g id="bullet-char-template-57354" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 8,1128 L 1137,1128 1137,0 8,0 8,1128 Z"/> + </g> + <g id="bullet-char-template-10146" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 174,0 L 602,739 174,1481 1456,739 174,0 Z M 1358,739 L 309,1346 659,739 1358,739 Z"/> + </g> + <g id="bullet-char-template-10132" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 2015,739 L 1276,0 717,0 1260,543 174,543 174,936 1260,936 717,1481 1274,1481 2015,739 Z"/> + </g> + <g id="bullet-char-template-10007" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 0,-2 C -7,14 -16,27 -25,37 L 356,567 C 262,823 215,952 215,954 215,979 228,992 255,992 264,992 276,990 289,987 310,991 331,999 354,1012 L 381,999 492,748 772,1049 836,1024 860,1049 C 881,1039 901,1025 922,1006 886,937 835,863 770,784 769,783 710,716 594,584 L 774,223 C 774,196 753,168 711,139 L 727,119 C 717,90 699,76 672,76 641,76 570,178 457,381 L 164,-76 C 142,-110 111,-127 72,-127 30,-127 9,-110 8,-76 1,-67 -2,-52 -2,-32 -2,-23 -1,-13 0,-2 Z"/> + </g> + <g id="bullet-char-template-10004" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 285,-33 C 182,-33 111,30 74,156 52,228 41,333 41,471 41,549 55,616 82,672 116,743 169,778 240,778 293,778 328,747 346,684 L 369,508 C 377,444 397,411 428,410 L 1163,1116 C 1174,1127 1196,1133 1229,1133 1271,1133 1292,1118 1292,1087 L 1292,965 C 1292,929 1282,901 1262,881 L 442,47 C 390,-6 338,-33 285,-33 Z"/> + </g> + <g id="bullet-char-template-9679" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 813,0 C 632,0 489,54 383,161 276,268 223,411 223,592 223,773 276,916 383,1023 489,1130 632,1184 813,1184 992,1184 1136,1130 1245,1023 1353,916 1407,772 1407,592 1407,412 1353,268 1245,161 1136,54 992,0 813,0 Z"/> + </g> + <g id="bullet-char-template-8226" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 346,457 C 273,457 209,483 155,535 101,586 74,649 74,723 74,796 101,859 155,911 209,963 273,989 346,989 419,989 480,963 531,910 582,859 608,796 608,723 608,648 583,586 532,535 482,483 420,457 346,457 Z"/> + </g> + <g id="bullet-char-template-8211" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M -4,459 L 1135,459 1135,606 -4,606 -4,459 Z"/> + </g> + <g id="bullet-char-template-61548" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 173,740 C 173,903 231,1043 346,1159 462,1274 601,1332 765,1332 928,1332 1067,1274 1183,1159 1299,1043 1357,903 1357,740 1357,577 1299,437 1183,322 1067,206 928,148 765,148 601,148 462,206 346,322 231,437 173,577 173,740 Z"/> + </g> + </defs> + <defs class="TextEmbeddedBitmaps"/> + <g> + <g id="id2" class="Master_Slide"> + <g id="bg-id2" class="Background"/> + <g id="bo-id2" class="BackgroundObjects"/> + </g> + </g> + <g class="SlideGroup"> + <g> + <g id="container-id1"> + <g id="id1" class="Slide" clip-path="url(#presentation_clip_path)"> + <g class="Page"> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id3"> + <rect class="BoundingBox" stroke="none" fill="none" x="9227" y="2229" width="7040" height="8959"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12771,11160 L 12747,11160 12642,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12578,11160 L 12474,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12410,11160 L 12306,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12242,11160 L 12138,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12074,11160 L 11970,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11906,11160 L 11802,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11738,11160 L 11634,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11570,11160 L 11466,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11402,11160 L 11298,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11234,11160 L 11130,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11066,11160 L 10962,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10898,11160 L 10794,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10730,11160 L 10626,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10562,11160 L 10458,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10394,11160 L 10290,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10226,11160 L 10122,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10058,11160 L 9954,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9890,11160 L 9786,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9722,11160 L 9618,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9554,11160 L 9450,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9386,11160 L 9282,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,11124 L 9254,11020"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10956 L 9254,10852"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10788 L 9254,10684"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10620 L 9254,10516"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10452 L 9254,10348"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10284 L 9254,10180"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,10116 L 9254,10012"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9948 L 9254,9844"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9780 L 9254,9676"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9612 L 9254,9508"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9444 L 9254,9340"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9276 L 9254,9172"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,9108 L 9254,9004"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8940 L 9254,8836"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8772 L 9254,8668"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8604 L 9254,8500"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8436 L 9254,8332"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8268 L 9254,8164"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,8100 L 9254,7996"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7932 L 9254,7828"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7764 L 9254,7660"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7596 L 9254,7492"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7428 L 9254,7324"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7260 L 9254,7156"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,7092 L 9254,6988"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6924 L 9254,6820"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6756 L 9254,6652"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6588 L 9254,6484"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6420 L 9254,6316"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6252 L 9254,6148"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,6084 L 9254,5980"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5916 L 9254,5812"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5748 L 9254,5644"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5580 L 9254,5476"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5412 L 9254,5308"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5244 L 9254,5140"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,5076 L 9254,4972"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4908 L 9254,4804"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4740 L 9254,4636"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4572 L 9254,4467"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4404 L 9254,4299"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4236 L 9254,4131"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,4068 L 9254,3963"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3900 L 9254,3795"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3732 L 9254,3627"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3564 L 9254,3459"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3396 L 9254,3291"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3228 L 9254,3123"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,3060 L 9254,2955"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,2892 L 9254,2787"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,2724 L 9254,2619"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,2556 L 9254,2451"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9254,2388 L 9254,2283"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9290,2256 L 9395,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9458,2256 L 9563,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9626,2256 L 9731,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9794,2256 L 9899,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 9962,2256 L 10067,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10130,2256 L 10235,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10298,2256 L 10403,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10466,2256 L 10571,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10634,2256 L 10739,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10802,2256 L 10907,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 10970,2256 L 11075,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11138,2256 L 11243,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11306,2256 L 11411,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11474,2256 L 11579,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11642,2256 L 11747,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11810,2256 L 11915,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 11978,2256 L 12083,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12146,2256 L 12251,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12314,2256 L 12419,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12482,2256 L 12587,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12650,2256 L 12755,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12818,2256 L 12923,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12986,2256 L 13091,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13154,2256 L 13259,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13322,2256 L 13427,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13490,2256 L 13595,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13659,2256 L 13763,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13827,2256 L 13931,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13995,2256 L 14099,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14163,2256 L 14267,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14331,2256 L 14435,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14499,2256 L 14603,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14667,2256 L 14771,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14835,2256 L 14939,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15003,2256 L 15107,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15171,2256 L 15275,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15339,2256 L 15443,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15507,2256 L 15611,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15675,2256 L 15779,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15843,2256 L 15947,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16011,2256 L 16115,2256"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16179,2256 L 16239,2256 16239,2300"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,2364 L 16239,2468"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,2532 L 16239,2636"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,2700 L 16239,2804"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,2868 L 16239,2972"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3036 L 16239,3140"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3204 L 16239,3308"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3372 L 16239,3476"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3540 L 16239,3644"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3708 L 16239,3812"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,3876 L 16239,3980"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4044 L 16239,4148"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4212 L 16239,4316"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4380 L 16239,4484"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4548 L 16239,4652"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4716 L 16239,4820"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,4884 L 16239,4988"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5052 L 16239,5156"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5220 L 16239,5324"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5388 L 16239,5492"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5556 L 16239,5660"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5724 L 16239,5828"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,5892 L 16239,5996"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6060 L 16239,6164"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6228 L 16239,6332"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6396 L 16239,6500"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6564 L 16239,6668"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6732 L 16239,6836"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,6900 L 16239,7004"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7068 L 16239,7172"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7236 L 16239,7340"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7404 L 16239,7508"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7572 L 16239,7676"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7740 L 16239,7844"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,7908 L 16239,8012"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8076 L 16239,8180"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8244 L 16239,8348"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8412 L 16239,8516"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8580 L 16239,8684"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8748 L 16239,8852"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,8916 L 16239,9020"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9084 L 16239,9188"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9252 L 16239,9356"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9420 L 16239,9524"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9588 L 16239,9693"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9756 L 16239,9861"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,9924 L 16239,10029"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10092 L 16239,10197"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10260 L 16239,10365"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10428 L 16239,10533"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10596 L 16239,10701"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10764 L 16239,10869"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,10932 L 16239,11037"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16239,11100 L 16239,11160 16194,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 16131,11160 L 16026,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15963,11160 L 15858,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15795,11160 L 15690,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15627,11160 L 15522,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15459,11160 L 15354,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15291,11160 L 15186,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 15123,11160 L 15018,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14955,11160 L 14850,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14787,11160 L 14682,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14619,11160 L 14514,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14451,11160 L 14346,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14283,11160 L 14178,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 14115,11160 L 14010,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13947,11160 L 13842,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13779,11160 L 13674,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13611,11160 L 13506,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13443,11160 L 13338,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13275,11160 L 13170,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 13107,11160 L 13002,11160"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="53" stroke-linejoin="round" d="M 12939,11160 L 12834,11160"/> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id4"> + <rect class="BoundingBox" stroke="none" fill="none" x="3128" y="2360" width="5723" height="840"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="529px" font-weight="400"><tspan class="TextPosition" x="3378" y="2963"><tspan fill="rgb(0,102,179)" stroke="none">MAIN PTHREAD</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id5"> + <rect class="BoundingBox" stroke="none" fill="none" x="9640" y="2360" width="8013" height="840"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="529px" font-weight="400"><tspan class="TextPosition" x="9890" y="2963"><tspan fill="rgb(0,102,179)" stroke="none">CONTROLLER PTHREAD</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id6"> + <rect class="BoundingBox" stroke="none" fill="none" x="3315" y="5190" width="2797" height="1273"/> + <path fill="rgb(114,159,207)" stroke="none" d="M 4713,6461 L 3316,6461 3316,5191 6110,5191 6110,6461 4713,6461 Z"/> + <path fill="none" stroke="rgb(52,101,164)" d="M 4713,6461 L 3316,6461 3316,5191 6110,5191 6110,6461 4713,6461 Z"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="4358" y="5985"><tspan fill="rgb(0,0,0)" stroke="none">CLI</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id7"> + <rect class="BoundingBox" stroke="none" fill="none" x="3315" y="7349" width="2797" height="1273"/> + <path fill="rgb(114,159,207)" stroke="none" d="M 4713,8620 L 3316,8620 3316,7350 6110,7350 6110,8620 4713,8620 Z"/> + <path fill="none" stroke="rgb(52,101,164)" d="M 4713,8620 L 3316,8620 3316,7350 6110,7350 6110,8620 4713,8620 Z"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="3972" y="7888"><tspan fill="rgb(0,0,0)" stroke="none">PATHD</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="3430" y="8400"><tspan fill="rgb(0,0,0)" stroke="none">INTERFACE</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id8"> + <rect class="BoundingBox" stroke="none" fill="none" x="12459" y="6079" width="2797" height="1273"/> + <path fill="rgb(114,159,207)" stroke="none" d="M 13857,7350 L 12460,7350 12460,6080 15254,6080 15254,7350 13857,7350 Z"/> + <path fill="none" stroke="rgb(52,101,164)" d="M 13857,7350 L 12460,7350 12460,6080 15254,6080 15254,7350 13857,7350 Z"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="13375" y="6874"><tspan fill="rgb(0,0,0)" stroke="none">PCC</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id9"> + <rect class="BoundingBox" stroke="none" fill="none" x="1665" y="7578" width="2033" height="205"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 1956,7680 L 3406,7680"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 1665,7680 L 1970,7782 1970,7579 1665,7680 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 3697,7680 L 3393,7579 3393,7782 3697,7680 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id10"> + <rect class="BoundingBox" stroke="none" fill="none" x="1892" y="5137" width="1451" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="2142" y="5520"><tspan fill="rgb(0,0,0)" stroke="none">VTYSH</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id11"> + <rect class="BoundingBox" stroke="none" fill="none" x="1568" y="5738" width="1976" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1818" y="6121"><tspan fill="rgb(0,0,0)" stroke="none">Northbound</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id12"> + <rect class="BoundingBox" stroke="none" fill="none" x="1738" y="7235" width="1735" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1988" y="7618"><tspan fill="rgb(0,0,0)" stroke="none">pathd API</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.ClosedBezierShape"> + <g id="id13"> + <rect class="BoundingBox" stroke="none" fill="none" x="2299" y="2229" width="6957" height="8959"/> + <path fill="rgb(52,101,164)" stroke="none" d="M 2299,11134 L 2396,11134 2396,11160 2396,11186 2299,11186 2299,11134 Z M 2299,2230 L 2333,2230 2333,2256 2333,2283 2299,2283 2299,2230 Z M 2399,2283 L 2399,2256 2399,2230 2506,2230 2506,2256 2506,2283 2399,2283 Z M 2571,2283 L 2571,2256 2571,2230 2679,2230 2679,2256 2679,2283 2571,2283 Z M 2744,2283 L 2744,2256 2744,2230 2852,2230 2852,2256 2852,2283 2744,2283 Z M 2917,2283 L 2917,2256 2917,2230 3025,2230 3025,2256 3025,2283 2917,2283 Z M 3090,2283 L 3090,2256 3090,2230 3198,2230 3198,2256 3198,2283 3090,2283 Z M 3263,2283 L 3263,2256 3263,2230 3371,2230 3371,2256 3371,2283 3263,2283 Z M 3436,2283 L 3436,2256 3436,2230 3543,2230 3543,2256 3543,2283 3436,2283 Z M 3609,2283 L 3609,2256 3609,2230 3716,2230 3716,2256 3716,2283 3609,2283 Z M 3782,2283 L 3782,2256 3782,2230 3889,2230 3889,2256 3889,2283 3782,2283 Z M 3955,2283 L 3955,2256 3955,2230 4062,2230 4062,2256 4062,2283 3955,2283 Z M 4128,2283 L 4128,2256 4128,2230 4235,2230 4235,2256 4235,2283 4128,2283 Z M 4301,2283 L 4301,2256 4301,2230 4408,2230 4408,2256 4408,2283 4301,2283 Z M 4473,2283 L 4473,2256 4473,2230 4581,2230 4581,2256 4581,2283 4473,2283 Z M 4646,2283 L 4646,2256 4646,2230 4754,2230 4754,2256 4754,2283 4646,2283 Z M 4819,2283 L 4819,2256 4819,2230 4927,2230 4927,2256 4927,2283 4819,2283 Z M 4992,2283 L 4992,2256 4992,2230 5100,2230 5100,2256 5100,2283 4992,2283 Z M 5165,2283 L 5165,2256 5165,2230 5273,2230 5273,2256 5273,2283 5165,2283 Z M 5338,2283 L 5338,2256 5338,2230 5445,2230 5445,2256 5445,2283 5338,2283 Z M 5511,2283 L 5511,2256 5511,2230 5618,2230 5618,2256 5618,2283 5511,2283 Z M 5684,2283 L 5684,2256 5684,2230 5791,2230 5791,2256 5791,2283 5684,2283 Z M 5857,2283 L 5857,2256 5857,2230 5964,2230 5964,2256 5964,2283 5857,2283 Z M 6030,2283 L 6030,2256 6030,2230 6137,2230 6137,2256 6137,2283 6030,2283 Z M 6203,2283 L 6203,2256 6203,2230 6310,2230 6310,2256 6310,2283 6203,2283 Z M 6375,2283 L 6375,2256 6375,2230 6483,2230 6483,2256 6483,2283 6375,2283 Z M 6548,2283 L 6548,2256 6548,2230 6656,2230 6656,2256 6656,2283 6548,2283 Z M 6721,2283 L 6721,2256 6721,2230 6829,2230 6829,2256 6829,2283 6721,2283 Z M 6894,2283 L 6894,2256 6894,2230 7002,2230 7002,2256 7002,2283 6894,2283 Z M 7067,2283 L 7067,2256 7067,2230 7175,2230 7175,2256 7175,2283 7067,2283 Z M 7240,2283 L 7240,2256 7240,2230 7347,2230 7347,2256 7347,2283 7240,2283 Z M 7413,2283 L 7413,2256 7413,2230 7520,2230 7520,2256 7520,2283 7413,2283 Z M 7586,2283 L 7586,2256 7586,2230 7693,2230 7693,2256 7693,2283 7586,2283 Z M 7759,2283 L 7759,2256 7759,2230 7866,2230 7866,2256 7866,2283 7759,2283 Z M 7932,2283 L 7932,2256 7932,2230 8039,2230 8039,2256 8039,2283 7932,2283 Z M 8105,2283 L 8105,2256 8105,2230 8212,2230 8212,2256 8212,2283 8105,2283 Z M 8277,2283 L 8277,2256 8277,2230 8385,2230 8385,2256 8385,2283 8277,2283 Z M 8450,2283 L 8450,2256 8450,2230 8558,2230 8558,2256 8558,2283 8450,2283 Z M 8623,2283 L 8623,2256 8623,2230 8731,2230 8731,2256 8731,2283 8623,2283 Z M 8796,2283 L 8796,2256 8796,2230 8904,2230 8904,2256 8904,2283 8796,2283 Z M 8969,2283 L 8969,2256 8969,2230 9077,2230 9077,2256 9077,2283 8969,2283 Z M 9142,2283 L 9142,2256 9142,2230 9227,2230 C 9232,2230 9237,2231 9241,2233 9245,2235 9248,2239 9251,2243 9253,2247 9254,2251 9254,2256 L 9254,2278 9227,2278 9227,2283 9142,2283 Z M 9200,2341 L 9227,2341 9254,2341 9254,2446 9227,2446 9200,2446 9200,2341 Z M 9200,2509 L 9227,2509 9254,2509 9254,2614 9227,2614 9200,2614 9200,2509 Z M 9200,2677 L 9227,2677 9254,2677 9254,2782 9227,2782 9200,2782 9200,2677 Z M 9200,2845 L 9227,2845 9254,2845 9254,2950 9227,2950 9200,2950 9200,2845 Z M 9200,3013 L 9227,3013 9254,3013 9254,3118 9227,3118 9200,3118 9200,3013 Z M 9200,3181 L 9227,3181 9254,3181 9254,3286 9227,3286 9200,3286 9200,3181 Z M 9200,3349 L 9227,3349 9254,3349 9254,3454 9227,3454 9200,3454 9200,3349 Z M 9200,3517 L 9227,3517 9254,3517 9254,3622 9227,3622 9200,3622 9200,3517 Z M 9200,3685 L 9227,3685 9254,3685 9254,3790 9227,3790 9200,3790 9200,3685 Z M 9200,3853 L 9227,3853 9254,3853 9254,3958 9227,3958 9200,3958 9200,3853 Z M 9200,4021 L 9227,4021 9254,4021 9254,4126 9227,4126 9200,4126 9200,4021 Z M 9200,4189 L 9227,4189 9254,4189 9254,4294 9227,4294 9200,4294 9200,4189 Z M 9200,4357 L 9227,4357 9254,4357 9254,4462 9227,4462 9200,4462 9200,4357 Z M 9200,4525 L 9227,4525 9254,4525 9254,4630 9227,4630 9200,4630 9200,4525 Z M 9200,4693 L 9227,4693 9254,4693 9254,4798 9227,4798 9200,4798 9200,4693 Z M 9200,4861 L 9227,4861 9254,4861 9254,4966 9227,4966 9200,4966 9200,4861 Z M 9200,5029 L 9227,5029 9254,5029 9254,5134 9227,5134 9200,5134 9200,5029 Z M 9200,5197 L 9227,5197 9254,5197 9254,5302 9227,5302 9200,5302 9200,5197 Z M 9200,5365 L 9227,5365 9254,5365 9254,5470 9227,5470 9200,5470 9200,5365 Z M 9200,5533 L 9227,5533 9254,5533 9254,5638 9227,5638 9200,5638 9200,5533 Z M 9200,5701 L 9227,5701 9254,5701 9254,5806 9227,5806 9200,5806 9200,5701 Z M 9200,5869 L 9227,5869 9254,5869 9254,5974 9227,5974 9200,5974 9200,5869 Z M 9200,6037 L 9227,6037 9254,6037 9254,6142 9227,6142 9200,6142 9200,6037 Z M 9200,6205 L 9227,6205 9254,6205 9254,6310 9227,6310 9200,6310 9200,6205 Z M 9200,6373 L 9227,6373 9254,6373 9254,6478 9227,6478 9200,6478 9200,6373 Z M 9200,6541 L 9227,6541 9254,6541 9254,6646 9227,6646 9200,6646 9200,6541 Z M 9200,6709 L 9227,6709 9254,6709 9254,6814 9227,6814 9200,6814 9200,6709 Z M 9200,6877 L 9227,6877 9254,6877 9254,6982 9227,6982 9200,6982 9200,6877 Z M 9200,7045 L 9227,7045 9254,7045 9254,7150 9227,7150 9200,7150 9200,7045 Z M 9200,7213 L 9227,7213 9254,7213 9254,7318 9227,7318 9200,7318 9200,7213 Z M 9200,7381 L 9227,7381 9254,7381 9254,7486 9227,7486 9200,7486 9200,7381 Z M 9200,7549 L 9227,7549 9254,7549 9254,7654 9227,7654 9200,7654 9200,7549 Z M 9200,7717 L 9227,7717 9254,7717 9254,7822 9227,7822 9200,7822 9200,7717 Z M 9200,7886 L 9227,7886 9254,7886 9254,7990 9227,7990 9200,7990 9200,7886 Z M 9200,8054 L 9227,8054 9254,8054 9254,8158 9227,8158 9200,8158 9200,8054 Z M 9200,8222 L 9227,8222 9254,8222 9254,8326 9227,8326 9200,8326 9200,8222 Z M 9200,8390 L 9227,8390 9254,8390 9254,8494 9227,8494 9200,8494 9200,8390 Z M 9200,8558 L 9227,8558 9254,8558 9254,8662 9227,8662 9200,8662 9200,8558 Z M 9200,8726 L 9227,8726 9254,8726 9254,8830 9227,8830 9200,8830 9200,8726 Z M 9200,8894 L 9227,8894 9254,8894 9254,8998 9227,8998 9200,8998 9200,8894 Z M 9200,9062 L 9227,9062 9254,9062 9254,9166 9227,9166 9200,9166 9200,9062 Z M 9200,9230 L 9227,9230 9254,9230 9254,9334 9227,9334 9200,9334 9200,9230 Z M 9200,9398 L 9227,9398 9254,9398 9254,9502 9227,9502 9200,9502 9200,9398 Z M 9200,9566 L 9227,9566 9254,9566 9254,9670 9227,9670 9200,9670 9200,9566 Z M 9200,9734 L 9227,9734 9254,9734 9254,9838 9227,9838 9200,9838 9200,9734 Z M 9200,9902 L 9227,9902 9254,9902 9254,10006 9227,10006 9200,10006 9200,9902 Z M 9200,10070 L 9227,10070 9254,10070 9254,10174 9227,10174 9200,10174 9200,10070 Z M 9200,10238 L 9227,10238 9254,10238 9254,10342 9227,10342 9200,10342 9200,10238 Z M 9200,10406 L 9227,10406 9254,10406 9254,10510 9227,10510 9200,10510 9200,10406 Z M 9200,10574 L 9227,10574 9254,10574 9254,10678 9227,10678 9200,10678 9200,10574 Z M 9200,10742 L 9227,10742 9254,10742 9254,10846 9227,10846 9200,10846 9200,10742 Z M 9200,10910 L 9227,10910 9254,10910 9254,11014 9227,11014 9200,11014 9200,10910 Z M 9200,11078 L 9227,11078 9254,11078 9254,11160 C 9254,11165 9253,11169 9251,11173 9248,11177 9245,11181 9241,11183 9237,11185 9232,11186 9227,11186 L 9204,11186 9204,11160 9200,11160 9200,11078 Z M 9139,11134 L 9139,11160 9139,11186 9032,11186 9032,11160 9032,11134 9139,11134 Z M 8966,11134 L 8966,11160 8966,11186 8859,11186 8859,11160 8859,11134 8966,11134 Z M 8793,11134 L 8793,11160 8793,11186 8686,11186 8686,11160 8686,11134 8793,11134 Z M 8620,11134 L 8620,11160 8620,11186 8513,11186 8513,11160 8513,11134 8620,11134 Z M 8447,11134 L 8447,11160 8447,11186 8340,11186 8340,11160 8340,11134 8447,11134 Z M 8274,11134 L 8274,11160 8274,11186 8167,11186 8167,11160 8167,11134 8274,11134 Z M 8102,11134 L 8102,11160 8102,11186 7994,11186 7994,11160 7994,11134 8102,11134 Z M 7929,11134 L 7929,11160 7929,11186 7821,11186 7821,11160 7821,11134 7929,11134 Z M 7756,11134 L 7756,11160 7756,11186 7648,11186 7648,11160 7648,11134 7756,11134 Z M 7583,11134 L 7583,11160 7583,11186 7475,11186 7475,11160 7475,11134 7583,11134 Z M 7410,11134 L 7410,11160 7410,11186 7302,11186 7302,11160 7302,11134 7410,11134 Z M 7237,11134 L 7237,11160 7237,11186 7130,11186 7130,11160 7130,11134 7237,11134 Z M 7064,11134 L 7064,11160 7064,11186 6957,11186 6957,11160 6957,11134 7064,11134 Z M 6891,11134 L 6891,11160 6891,11186 6784,11186 6784,11160 6784,11134 6891,11134 Z M 6718,11134 L 6718,11160 6718,11186 6611,11186 6611,11160 6611,11134 6718,11134 Z M 6545,11134 L 6545,11160 6545,11186 6438,11186 6438,11160 6438,11134 6545,11134 Z M 6373,11134 L 6373,11160 6373,11186 6265,11186 6265,11160 6265,11134 6373,11134 Z M 6200,11134 L 6200,11160 6200,11186 6092,11186 6092,11160 6092,11134 6200,11134 Z M 6027,11134 L 6027,11160 6027,11186 5919,11186 5919,11160 5919,11134 6027,11134 Z M 5854,11134 L 5854,11160 5854,11186 5746,11186 5746,11160 5746,11134 5854,11134 Z M 5681,11134 L 5681,11160 5681,11186 5573,11186 5573,11160 5573,11134 5681,11134 Z M 5508,11134 L 5508,11160 5508,11186 5401,11186 5401,11160 5401,11134 5508,11134 Z M 5335,11134 L 5335,11160 5335,11186 5228,11186 5228,11160 5228,11134 5335,11134 Z M 5162,11134 L 5162,11160 5162,11186 5055,11186 5055,11160 5055,11134 5162,11134 Z M 4989,11134 L 4989,11160 4989,11186 4882,11186 4882,11160 4882,11134 4989,11134 Z M 4816,11134 L 4816,11160 4816,11186 4709,11186 4709,11160 4709,11134 4816,11134 Z M 4643,11134 L 4643,11160 4643,11186 4536,11186 4536,11160 4536,11134 4643,11134 Z M 4471,11134 L 4471,11160 4471,11186 4363,11186 4363,11160 4363,11134 4471,11134 Z M 4298,11134 L 4298,11160 4298,11186 4190,11186 4190,11160 4190,11134 4298,11134 Z M 4125,11134 L 4125,11160 4125,11186 4017,11186 4017,11160 4017,11134 4125,11134 Z M 3952,11134 L 3952,11160 3952,11186 3844,11186 3844,11160 3844,11134 3952,11134 Z M 3779,11134 L 3779,11160 3779,11186 3671,11186 3671,11160 3671,11134 3779,11134 Z M 3606,11134 L 3606,11160 3606,11186 3499,11186 3499,11160 3499,11134 3606,11134 Z M 3433,11134 L 3433,11160 3433,11186 3326,11186 3326,11160 3326,11134 3433,11134 Z M 3260,11134 L 3260,11160 3260,11186 3153,11186 3153,11160 3153,11134 3260,11134 Z M 3087,11134 L 3087,11160 3087,11186 2980,11186 2980,11160 2980,11134 3087,11134 Z M 2914,11134 L 2914,11160 2914,11186 2807,11186 2807,11160 2807,11134 2914,11134 Z M 2741,11134 L 2741,11160 2741,11186 2634,11186 2634,11160 2634,11134 2741,11134 Z M 2569,11134 L 2569,11160 2569,11186 2461,11186 2461,11160 2461,11134 2569,11134 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id14"> + <rect class="BoundingBox" stroke="none" fill="none" x="7086" y="3539" width="4194" height="1273"/> + <path fill="rgb(114,159,207)" stroke="none" d="M 9183,4810 L 7087,4810 7087,3540 11278,3540 11278,4810 9183,4810 Z"/> + <path fill="none" stroke="rgb(52,101,164)" d="M 9183,4810 L 7087,4810 7087,3540 11278,3540 11278,4810 9183,4810 Z"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="8218" y="4078"><tspan fill="rgb(0,0,0)" stroke="none">PCEPLIB</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="7900" y="4590"><tspan fill="rgb(0,0,0)" stroke="none">INTERFACE</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id15"> + <rect class="BoundingBox" stroke="none" fill="none" x="1665" y="6105" width="2033" height="205"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 1956,6207 L 3406,6207"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 1665,6207 L 1970,6309 1970,6106 1665,6207 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 3697,6207 L 3393,6106 3393,6309 3697,6207 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id16"> + <rect class="BoundingBox" stroke="none" fill="none" x="1665" y="5470" width="2033" height="205"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 1956,5572 L 3406,5572"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 1665,5572 L 1970,5674 1970,5471 1665,5572 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 3697,5572 L 3393,5471 3393,5674 3697,5572 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.CustomShape"> + <g id="id17"> + <rect class="BoundingBox" stroke="none" fill="none" x="4584" y="9553" width="9528" height="1146"/> + <path fill="rgb(114,159,207)" stroke="none" d="M 9348,10697 L 4585,10697 4585,9554 14110,9554 14110,10697 9348,10697 Z"/> + <path fill="none" stroke="rgb(52,101,164)" d="M 9348,10697 L 4585,10697 4585,9554 14110,9554 14110,10697 9348,10697 Z"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="459px" font-weight="400"><tspan class="TextPosition" x="7786" y="10285"><tspan fill="rgb(0,0,0)" stroke="none">CONTROLLER</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id18"> + <rect class="BoundingBox" stroke="none" fill="none" x="6491" y="9448" width="2383" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="6741" y="9831"><tspan fill="rgb(0,0,0)" stroke="none">pcep_ctrl_XXX</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id19"> + <rect class="BoundingBox" stroke="none" fill="none" x="9920" y="9448" width="2802" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="10170" y="9831"><tspan fill="rgb(0,0,0)" stroke="none">pcep_thread_XXX</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id20"> + <rect class="BoundingBox" stroke="none" fill="none" x="5953" y="8366" width="1682" height="1271"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 6185,8541 L 7402,9461"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 5953,8366 L 6135,8631 6257,8469 5953,8366 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 7634,9636 L 7452,9371 7330,9533 7634,9636 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id21"> + <rect class="BoundingBox" stroke="none" fill="none" x="5953" y="6207" width="2287" height="3430"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 6114,6449 L 8078,9394"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 5953,6207 L 6037,6517 6206,6404 5953,6207 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 8239,9636 L 8155,9326 7986,9439 8239,9636 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id22"> + <rect class="BoundingBox" stroke="none" fill="none" x="5953" y="4683" width="1652" height="763"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 6217,5323 L 7340,4805"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 5953,5445 L 6272,5410 6187,5225 5953,5445 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 7604,4683 L 7285,4718 7370,4903 7604,4683 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id23"> + <rect class="BoundingBox" stroke="none" fill="none" x="5953" y="4683" width="2287" height="3049"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 6128,7498 L 8064,4916"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 5953,7731 L 6217,7548 6054,7427 5953,7731 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 8239,4683 L 7975,4866 8138,4988 8239,4683 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id24"> + <rect class="BoundingBox" stroke="none" fill="none" x="8772" y="1635" width="205" height="2033"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 8874,3376 L 8874,1926"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 8874,3667 L 8976,3363 8773,3363 8874,3667 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 8874,1635 L 8773,1940 8976,1940 8874,1635 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id25"> + <rect class="BoundingBox" stroke="none" fill="none" x="8347" y="1127" width="1925" height="570"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="8597" y="1510"><tspan fill="rgb(0,0,0)" stroke="none">pceplib API</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id26"> + <rect class="BoundingBox" stroke="none" fill="none" x="11033" y="4683" width="1779" height="1525"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 11254,4872 L 12590,6018"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 11033,4683 L 11198,4958 11330,4804 11033,4683 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 12811,6207 L 12646,5932 12514,6086 12811,6207 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id27"> + <rect class="BoundingBox" stroke="none" fill="none" x="11063" y="7223" width="1749" height="2414"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 11234,9400 L 12640,7459"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 11063,9636 L 11324,9449 11159,9330 11063,9636 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 12811,7223 L 12550,7410 12715,7529 12811,7223 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.LineShape"> + <g id="id28"> + <rect class="BoundingBox" stroke="none" fill="none" x="9534" y="1635" width="205" height="2033"/> + <path fill="none" stroke="rgb(102,102,102)" stroke-width="35" stroke-linejoin="round" d="M 9636,3376 L 9636,1926"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 9636,3667 L 9738,3363 9535,3363 9636,3667 Z"/> + <path fill="rgb(102,102,102)" stroke="none" d="M 9636,1635 L 9535,1940 9738,1940 9636,1635 Z"/> + </g> + </g> + </g> + </g> + </g> + </g> + </g> +</svg>
\ No newline at end of file diff --git a/doc/figures/threadmaster-multiple.png b/doc/figures/threadmaster-multiple.png Binary files differnew file mode 100644 index 0000000..2ded50c --- /dev/null +++ b/doc/figures/threadmaster-multiple.png diff --git a/doc/figures/threadmaster-single.png b/doc/figures/threadmaster-single.png Binary files differnew file mode 100644 index 0000000..a068389 --- /dev/null +++ b/doc/figures/threadmaster-single.png diff --git a/doc/figures/threadmaster.svg b/doc/figures/threadmaster.svg new file mode 100644 index 0000000..a8d2c6a --- /dev/null +++ b/doc/figures/threadmaster.svg @@ -0,0 +1,42 @@ +<svg width="640" height="480" xmlns="http://www.w3.org/2000/svg" xmlns:svg="http://www.w3.org/2000/svg"> + <!-- Created with SVG-edit - http://svg-edit.googlecode.com/ --> + <g> + <title>Layer 1</title> + <rect stroke="#000000" id="svg_14" height="209.999998" width="78.999998" y="42.000002" x="44" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="#ffaaaa"/> + <rect stroke="#000000" id="svg_7" height="391.000012" width="278.000011" y="20" x="24.999992" fill="#e5e5e5"/> + <rect stroke="#000000" id="svg_8" height="371.999978" width="259.000006" y="27.5" x="34.5" fill="#aad4ff"/> + <text stroke="#000000" transform="matrix(0.6990958452224731,0,0,0.7454545497894287,7.038336245343089,56.89090812206268) " opacity="0.95" xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" id="svg_19" y="259.756097" x="80.760476" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">I/O</text> + <text id="svg_35" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="139.834812" x="60.305235" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task3</text> + <text id="svg_36" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="143.062739" x="62.867657" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task3</text> + <rect stroke="#000000" id="svg_9" height="196.000002" width="53" y="35" x="44" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="#ffaaaa"/> + <text id="svg_40" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="144.681173" x="161.373439" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <rect id="svg_15" stroke="#000000" height="196.000002" width="53" y="35" x="104" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="#ffaaaa"/> + <rect id="svg_16" stroke="#000000" height="196.000002" width="53" y="35" x="164" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="#ffaaaa"/> + <rect id="svg_17" stroke="#000000" height="196.000002" width="53" y="35" x="225" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="#a2f9a2"/> + <text id="svg_20" stroke="#000000" transform="matrix(0.5625633135745076,0,0,0.5774844344081629,24.86726517334988,96.24168390657653) " opacity="0.95" xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="264.951043" x="181.542341" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">Timer</text> + <text id="svg_21" stroke="#000000" transform="matrix(0.5749753656942643,0,0,0.6334746061662456,38.254388934566045,86.97425370192815) " opacity="0.95" xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="256.836039" x="263.041264" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">Event</text> + <text id="svg_22" stroke="#000000" transform="matrix(0.5985540174236015,0,0,0.6031454293478589,42.9067138136931,90.27170546109113) " opacity="0.95" xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="263.072047" x="346.818415" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">Ready</text> + <line id="svg_24" y2="62" x2="278.034186" y1="62" x1="44" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="none"/> + <line id="svg_25" y2="90" x2="278.034186" y1="90" x1="44" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="none"/> + <line id="svg_26" y2="118.5" x2="278.034186" y1="118.5" x1="44" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="none"/> + <rect id="svg_27" height="45" width="182" y="287" x="103" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="#cccccc"/> + <rect id="svg_28" height="45" width="182" y="342" x="103" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="#cccccc"/> + <text stroke="#000000" transform="matrix(0.791700541973114,0,0,0.7857142686843872,21.471117571927607,58.28571891784668) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" id="svg_31" y="325.818183" x="222.3407" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">thread_fetch()</text> + <text stroke="#000000" transform="matrix(0.8357433386864528,0,0,0.7633649135312126,6.654719490831912,79.40761438745744) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" id="svg_32" y="380.181209" x="228.949154" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">thread_call()</text> + <text stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" id="svg_33" y="47" x="56" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task1</text> + <text style="cursor: move;" id="svg_34" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="94.51867" x="58.592092" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task2</text> + <text style="cursor: move;" id="svg_37" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="141.444306" x="58.584825" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task9</text> + <text style="cursor: move;" id="svg_38" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="50.812022" x="163.086583" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task4</text> + <text style="cursor: move;" id="svg_39" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="92.891296" x="164.799726" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">task7</text> + <text id="svg_41" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="138.207438" x="158.803698" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_42" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="54.046518" x="265.881445" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_43" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="96.125793" x="264.168301" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_44" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="143.060369" x="259.02887" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_45" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="50.809651" x="370.383203" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_46" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="99.36266" x="372.096346" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <text id="svg_47" stroke="#000000" transform="matrix(0.5837222373092742,0,0,0.6178813716933291,36.307626092448714,21.493443100305285) " xml:space="preserve" text-anchor="middle" font-family="Monospace" font-size="24" y="141.441935" x="372.096346" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="0" fill="#000000">...</text> + <line stroke="#000000" id="svg_54" y2="286.009435" x2="275" y1="230.500001" x1="275" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="none"/> + <line stroke="#000000" id="svg_57" y2="278.49643" x2="267.246429" y1="286" x1="275.5" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" fill="none"/> + <line id="svg_58" y2="285.754309" x2="274.995691" y1="278.5" x1="282.25" opacity="0.95" stroke-linecap="null" stroke-linejoin="null" stroke-dasharray="null" stroke-width="null" stroke="#000000" fill="none"/> + </g> +</svg> diff --git a/doc/licenses/BSD-2-Clause b/doc/licenses/BSD-2-Clause new file mode 100644 index 0000000..d11142d --- /dev/null +++ b/doc/licenses/BSD-2-Clause @@ -0,0 +1,19 @@ +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR +IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES +OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT +NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF +THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/doc/licenses/BSD-3-Clause b/doc/licenses/BSD-3-Clause new file mode 100644 index 0000000..46b46ec --- /dev/null +++ b/doc/licenses/BSD-3-Clause @@ -0,0 +1,23 @@ +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. +3. Neither the name of the project nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. diff --git a/doc/licenses/GPL-2.0 b/doc/licenses/GPL-2.0 new file mode 100644 index 0000000..d159169 --- /dev/null +++ b/doc/licenses/GPL-2.0 @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + <signature of Ty Coon>, 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/doc/licenses/GPL-3.0 b/doc/licenses/GPL-3.0 new file mode 100644 index 0000000..94a9ed0 --- /dev/null +++ b/doc/licenses/GPL-3.0 @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + <program> Copyright (C) <year> <name of author> + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +<http://www.gnu.org/licenses/>. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +<http://www.gnu.org/philosophy/why-not-lgpl.html>. diff --git a/doc/licenses/ISC b/doc/licenses/ISC new file mode 100644 index 0000000..02add5e --- /dev/null +++ b/doc/licenses/ISC @@ -0,0 +1,11 @@ +Permission to use, copy, modify, and distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. diff --git a/doc/licenses/LGPL-2.1 b/doc/licenses/LGPL-2.1 new file mode 100644 index 0000000..4362b49 --- /dev/null +++ b/doc/licenses/LGPL-2.1 @@ -0,0 +1,502 @@ + GNU LESSER GENERAL PUBLIC LICENSE + Version 2.1, February 1999 + + Copyright (C) 1991, 1999 Free Software Foundation, Inc. + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the Lesser GPL. It also counts + as the successor of the GNU Library Public License, version 2, hence + the version number 2.1.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Lesser General Public License, applies to some +specially designated software packages--typically libraries--of the +Free Software Foundation and other authors who decide to use it. You +can use it too, but we suggest you first think carefully about whether +this license or the ordinary General Public License is the better +strategy to use in any particular case, based on the explanations below. + + When we speak of free software, we are referring to freedom of use, +not price. Our General Public Licenses are designed to make sure that +you have the freedom to distribute copies of free software (and charge +for this service if you wish); that you receive source code or can get +it if you want it; that you can change the software and use pieces of +it in new free programs; and that you are informed that you can do +these things. + + To protect your rights, we need to make restrictions that forbid +distributors to deny you these rights or to ask you to surrender these +rights. These restrictions translate to certain responsibilities for +you if you distribute copies of the library or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link other code with the library, you must provide +complete object files to the recipients, so that they can relink them +with the library after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + We protect your rights with a two-step method: (1) we copyright the +library, and (2) we offer you this license, which gives you legal +permission to copy, distribute and/or modify the library. + + To protect each distributor, we want to make it very clear that +there is no warranty for the free library. Also, if the library is +modified by someone else and passed on, the recipients should know +that what they have is not the original version, so that the original +author's reputation will not be affected by problems that might be +introduced by others. + + Finally, software patents pose a constant threat to the existence of +any free program. We wish to make sure that a company cannot +effectively restrict the users of a free program by obtaining a +restrictive license from a patent holder. Therefore, we insist that +any patent license obtained for a version of the library must be +consistent with the full freedom of use specified in this license. + + Most GNU software, including some libraries, is covered by the +ordinary GNU General Public License. This license, the GNU Lesser +General Public License, applies to certain designated libraries, and +is quite different from the ordinary General Public License. We use +this license for certain libraries in order to permit linking those +libraries into non-free programs. + + When a program is linked with a library, whether statically or using +a shared library, the combination of the two is legally speaking a +combined work, a derivative of the original library. The ordinary +General Public License therefore permits such linking only if the +entire combination fits its criteria of freedom. The Lesser General +Public License permits more lax criteria for linking other code with +the library. + + We call this license the "Lesser" General Public License because it +does Less to protect the user's freedom than the ordinary General +Public License. It also provides other free software developers Less +of an advantage over competing non-free programs. These disadvantages +are the reason we use the ordinary General Public License for many +libraries. However, the Lesser license provides advantages in certain +special circumstances. + + For example, on rare occasions, there may be a special need to +encourage the widest possible use of a certain library, so that it becomes +a de-facto standard. To achieve this, non-free programs must be +allowed to use the library. A more frequent case is that a free +library does the same job as widely used non-free libraries. In this +case, there is little to gain by limiting the free library to free +software only, so we use the Lesser General Public License. + + In other cases, permission to use a particular library in non-free +programs enables a greater number of people to use a large body of +free software. For example, permission to use the GNU C Library in +non-free programs enables many more people to use the whole GNU +operating system, as well as its variant, the GNU/Linux operating +system. + + Although the Lesser General Public License is Less protective of the +users' freedom, it does ensure that the user of a program that is +linked with the Library has the freedom and the wherewithal to run +that program using a modified version of the Library. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, whereas the latter must +be combined with the library in order to run. + + GNU LESSER GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library or other +program which contains a notice placed by the copyright holder or +other authorized party saying it may be distributed under the terms of +this Lesser General Public License (also called "this License"). +Each licensee is addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control compilation +and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. + + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. + + 6. As an exception to the Sections above, you may also combine or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (1) uses at run time a + copy of the library already present on the user's computer system, + rather than copying library functions into the executable, and (2) + will operate properly with a modified version of the library, if + the user installs one, as long as the modified version is + interface-compatible with the version that the work was made with. + + c) Accompany the work with a written offer, valid for at + least three years, to give the same user the materials + specified in Subsection 6a, above, for a charge no more + than the cost of performing this distribution. + + d) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + e) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the materials to be distributed need not include anything that is +normally distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. + + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties with +this License. + + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under any +particular circumstance, the balance of the section is intended to apply, +and the section as a whole is intended to apply in other circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License may add +an explicit geographical distribution limitation excluding those countries, +so that distribution is permitted only in or among countries not thus +excluded. In such case, this License incorporates the limitation as if +written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Lesser General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. + + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms of the +ordinary General Public License). + + To apply these terms, attach the following notices to the library. It is +safest to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least the +"copyright" line and a pointer to where the full notice is found. + + <one line to give the library's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with this library; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +Also add information on how to contact you by electronic and paper mail. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the library, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James Random Hacker. + + <signature of Ty Coon>, 1 April 1990 + Ty Coon, President of Vice + +That's all there is to it! diff --git a/doc/licenses/LicenseRef-Skiplist-BSD-0-Clause b/doc/licenses/LicenseRef-Skiplist-BSD-0-Clause new file mode 100644 index 0000000..764f7cc --- /dev/null +++ b/doc/licenses/LicenseRef-Skiplist-BSD-0-Clause @@ -0,0 +1,15 @@ +Redistribution and use in source and binary forms, with or without +modification, are permitted. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. diff --git a/doc/licenses/MIT b/doc/licenses/MIT new file mode 100644 index 0000000..73e41b1 --- /dev/null +++ b/doc/licenses/MIT @@ -0,0 +1,17 @@ +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/doc/licenses/Unlicense b/doc/licenses/Unlicense new file mode 100644 index 0000000..efb9808 --- /dev/null +++ b/doc/licenses/Unlicense @@ -0,0 +1,24 @@ +This is free and unencumbered software released into the public domain. + +Anyone is free to copy, modify, publish, use, compile, sell, or +distribute this software, either in source code form or as a compiled +binary, for any purpose, commercial or non-commercial, and by any +means. + +In jurisdictions that recognize copyright laws, the author or authors +of this software dedicate any and all copyright interest in the +software to the public domain. We make this dedication for the benefit +of the public at large and to the detriment of our heirs and +successors. We intend this dedication to be an overt act of +relinquishment in perpetuity of all present and future rights to this +software under copyright law. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. + +For more information, please refer to <https://unlicense.org/> diff --git a/doc/manpages/.gitignore b/doc/manpages/.gitignore new file mode 100644 index 0000000..81c60dc --- /dev/null +++ b/doc/manpages/.gitignore @@ -0,0 +1,2 @@ +/_templates +/_build diff --git a/doc/manpages/Makefile b/doc/manpages/Makefile new file mode 100644 index 0000000..7cccfa2 --- /dev/null +++ b/doc/manpages/Makefile @@ -0,0 +1,12 @@ +all: ALWAYS + @$(MAKE) -s -C ../.. doc/manpages/man.stamp +help: ALWAYS + @$(MAKE) -s -C ../.. doc/help +%: ALWAYS + @$(MAKE) -s -C ../.. doc/manpages/_build/$@ + +Makefile: + #nothing +ALWAYS: +.PHONY: ALWAYS makefiles +.SUFFIXES: diff --git a/doc/manpages/bfd-options.rst b/doc/manpages/bfd-options.rst new file mode 100644 index 0000000..e335ed1 --- /dev/null +++ b/doc/manpages/bfd-options.rst @@ -0,0 +1,10 @@ +BFD SOCKET +---------- + +The following option controls the BFD daemon control socket location. + +.. option:: --bfdctl bfd-control-socket + + Opens the BFD daemon control socket located at the pointed location. + + (default: |INSTALL_PREFIX_STATE|/bfdd.sock) diff --git a/doc/manpages/common-options.rst b/doc/manpages/common-options.rst new file mode 100644 index 0000000..a47a233 --- /dev/null +++ b/doc/manpages/common-options.rst @@ -0,0 +1,163 @@ +HELP AND VERSION +---------------- + +.. option:: -h, --help + + Print a short description of the daemon's command line options. + +.. option:: -v, --version + + Print version and build information for the daemon. + +Both of these options inhibit normal operation and will immediately exit. + +PROCESS CONTROL +--------------- +These options control background operation: + +.. option:: -d, --daemon + + Launches the process in background/daemon mode, forking and detaching from the terminal. + + The parent process will delay its exit until the daemon/child has finished its initialization and has entered its main loop. This is important for zebra startup because the other daemons will attempt to connect to zebra. A return from zebra -d guarantees its readiness to accept these connections. + +.. option:: -t, --terminal + + Opens an interactive VTY session on the terminal, allowing for both state and configuration operations. Note that the terminal starts operating after startup has completed and the configuration file has been loaded. + + The process will exit when end of file is detected on the terminal. It is possible to daemonize a process started with -t (but without -d) by sending SIGQUIT to the process (normally mapped to a ^\ keypress.) + + +The combination of :option:`--daemon` and :option:`--terminal` will delay the daemon from going into background until the terminal session ends (by end of file.) + +If the process receives SIGINT (e.g. a ^C keypress) in this mode, it will exit instead of daemonizing. + +It is safe to suspend (SIGTSTP / ^Z) the terminal session opened by the previous two options; this will only stop the terminal but not the protocol daemon itself (which runs in a separate second process.) + +CONFIGURATION AND PATHS +----------------------- +The following options control configuration and file system locations for frr processes: + +.. option:: -f, --config_file config-file + + Specify a configuration file to be used instead of the default /etc/frr/<daemon>.conf file. + + Note that the daemon will attempt to write to this file if the write file command is issued on its VTY interface or through vtysh. + +.. option:: -C, --dryrun + + Load the configuration file and check its validity, then exit. + +.. option:: -i, --pid_file pid-file + + Output a pid file to a location other than the default /var/run/frr/<daemon>.pid. + +.. option:: -z, --socket zclient-path + + Override the path of the ZAPI socket used to communicate between zebra and the various protocol daemons. The default is /var/run/frr/zserv.api. The value of this option must be the same across all daemons. + +.. option:: -N, --pathspace pathspace + + Insert pathspace into all default paths, changing the defaults to: + + /etc/frr/pathspace/<daemon>.conf + /var/run/frr/pathspace/<daemon>.pid + /var/run/frr/pathspace/<daemon>.vty + /var/run/frr/pathspace/zserv.api + + ´.´ and ´/´ characters will not be accepted in pathspace, but the empty string will be accepted. + + Note that this only changes the respective defaults, it has no effect on the respective path if the -f, -i, -z or --vty_socket options are used. + + The purpose of this option is to easily group all file system related bits together for running multiple fully-separate "logical routers" on a system, particularly with Linux network namespaces. Groups of daemons running with distinct pathspace values will be completely unaware of each other and not interact in any way. + + This option does not do any system setup (like network namespaces.) This must be done by the user, for example by running: + + ip netns exec namespace <daemon> -N namespace + + +PROCESS CREDENTIALS +------------------- +.. option:: -u, --user user + + (default: frr) + +.. option:: -g, --group group + + (default: frr) + + Change the user/group which the daemon will switch to. + +.. option:: -S, --skip_runas + + Skip setting the process effective user and group. + + +Note that there is an additional group, frrvty, which controls group ownership of the VTY sockets. The name of this group cannot currently be changed, and user must be a member of this group. + + +VTY SETUP +--------- +These following options control the daemon's VTY (interactive command line) interface. The interface is available over TCP, using the telnet protocol, as well as through the vtysh frontend. + +.. option:: -A, --vty_addr vty-addr + + Specify an IP/IPv6 address to bind the TCP VTY interface to. It is generally recommended to specify ::1 or 127.0.0.1. For reasons of backwards compatibility, the default is to listen on all interfaces. + +.. option:: -P, --vty_port vty-port + + Override the daemon's default TCP VTY port (each daemon has a different default value upwards of 2600, listed below.) Specifying 0 disables the TCP VTY interface. + + Default ports are::: + + zebra 2601 + ripd 2602 + ripngd 2603 + ospfd 2604 + bgpd 2605 + ospf6d 2606 + isisd 2608 + babeld 2609 + nhrpd 2610 + pimd 2611 + ldpd 2612 + eigrpd 2613 + pbrd 2615 + staticd 2616 + bfdd 2617 + fabricd 2618 + vrrpd 2619 + + Port 2607 is used for ospfd's Opaque LSA API. + +.. option:: --vty_socket vty-path + + Overrides the directory used for the <daemon>.vty sockets. vtysh connects to these sockets in order to access each daemon's VTY. + Default: /var/run/frr[/<pathspace>] + + NB: Unlike the other options, this option specifies a directory, not a full path. + + This option is primarily used by the SNAP packaging system, its semantics may change. It should not be necessary in most other scenarios. + +MODULE LOADING +-------------- +frr supports optional dynamically loadable modules, although these can only be loaded at startup. The set of available modules may vary across distributions and packages, and modules may be available for installation as separate packages. + +.. option:: -M, --module module[:options] + + Load a module named module, optionally passing options to it. + + If there is a ´/´ character in module, the value is assumed to be a pathname to a module. + + If there is no ´/´ character, the module directory (see next option) is searched first for a module named "<daemon>_<module>.so", then for "<module>.so". This allows for a module to exist in variations appropriate for particular daemons, e.g. zebra_snmp and bgp_snmp, with the correct one selected by -M snmp. + + The meaning of options is specific to the module being loaded. Most modules currently ignore it. + + Modules are loaded in the order as listed on the command line. This is not generally relevant. + +.. option:: --moduledir module-path + + Look for modules in the module-path directory instead of the default /usr/lib/frr/modules. (This path is not affected by the -N option.) + +The list of loaded modules can be inspected at runtime with the show modules VTY command. + diff --git a/doc/manpages/conf.py b/doc/manpages/conf.py new file mode 100644 index 0000000..73dea09 --- /dev/null +++ b/doc/manpages/conf.py @@ -0,0 +1,365 @@ +# -*- coding: utf-8 -*- +# +# FRR documentation build configuration file, created by +# sphinx-quickstart on Tue Jan 31 16:00:52 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys +import os +import re + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +needs_sphinx = "1.0" + +# prolog for various variable substitutions +rst_prolog = "" + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = ["sphinx.ext.todo"] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ["_templates"] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst'] +source_suffix = ".rst" + +# The encoding of source files. +# source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = "index" + +# General information about the project. +project = u"FRR" +copyright = u"2017, FRR" +author = u"FRR authors" + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. + +# The short X.Y version. +version = u"?.?" +# The full version, including alpha/beta/rc tags. +release = u"?.?-?" + + +# ----------------------------------------------------------------------------- +# Extract values from codebase for substitution into docs. +# ----------------------------------------------------------------------------- + +# Various installation prefixes. Values are extracted from config.status. +# Reasonable defaults are set in case that file does not exist. +replace_vars = { + "AUTHORS": author, + "COPYRIGHT_YEAR": "1999-2005", + "COPYRIGHT_STR": "Copyright (c) 1999-2005", + "PACKAGE_NAME": project.lower(), + "PACKAGE_TARNAME": project.lower(), + "PACKAGE_STRING": project.lower() + " latest", + "PACKAGE_URL": "https://frrouting.org/", + "PACKAGE_VERSION": "latest", + "INSTALL_PREFIX_ETC": "/etc/frr", + "INSTALL_PREFIX_SBIN": "/usr/lib/frr", + "INSTALL_PREFIX_STATE": "/var/run/frr", + "INSTALL_PREFIX_MODULES": "/usr/lib/frr/modules", + "INSTALL_USER": "frr", + "INSTALL_GROUP": "frr", + "INSTALL_VTY_GROUP": "frrvty", + "GROUP": "frr", + "USER": "frr", +} + +# extract version information, installation location, other stuff we need to +# use when building final documents +val = re.compile('^S\["([^"]+)"\]="(.*)"$') +try: + with open("../../config.status", "r") as cfgstatus: + for ln in cfgstatus.readlines(): + m = val.match(ln) + if not m or m.group(1) not in replace_vars.keys(): + continue + replace_vars[m.group(1)] = m.group(2) +except IOError: + # if config.status doesn't exist, just ignore it + pass + +# manually fill out some of these we can't get from config.status +replace_vars["COPYRIGHT_STR"] = "Copyright (c)" +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["COPYRIGHT_YEAR"]) +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["AUTHORS"]) +release = replace_vars["PACKAGE_VERSION"] +version = release.split("-")[0] + +# add substitutions to prolog +for key, value in replace_vars.items(): + rst_prolog += ".. |{0}| replace:: {1}\n".format(key, value) + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +# today = '' +# Else, today_fmt is used as the format for a strftime call. +# today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = [ + "_build", + "common-options.rst", + "epilogue.rst", + "defines.rst", + "bfd-options.rst", +] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +# default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +# add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +# add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +# show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = "sphinx" + +# A list of ignored prefixes for module index sorting. +# modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +# keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = "default" + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +# html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# "<project> v<release> documentation". +# html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +# html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +# html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +# html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = [] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +# html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +# html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +# html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +# html_additional_pages = {} + +# If false, no module index is generated. +# html_domain_indices = True + +# If false, no index is generated. +# html_use_index = True + +# If true, the index is split into individual pages for each letter. +# html_split_index = False + +# If true, links to the reST sources are added to the pages. +# html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +# html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +# html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a <link> tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +# html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +# html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr' +# html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +# html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +# html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = "FRRdoc" + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + #'papersize': 'letterpaper', + # The font size ('10pt', '11pt' or '12pt'). + #'pointsize': '10pt', + # Additional stuff for the LaTeX preamble. + #'preamble': '', + # Latex figure (float) alignment + #'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, "FRR.tex", u"FRR User Manual", u"FRR", "manual"), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +# latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +# latex_use_parts = False + +# If true, show page references after internal links. +# latex_show_pagerefs = False + +# If true, show URL addresses after external links. +# latex_show_urls = False + +# Documents to append as an appendix to all manuals. +# latex_appendices = [] + +# If false, no module index is generated. +# latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). + +# If true, show URL addresses after external links. +# man_show_urls = False + +fwfrr = "{0} routing engine for use with FRRouting." + +man_pages = [ + ("frr-bfdd", "frr-bfdd", fwfrr.format("a bfd"), [], 8), + ("frr-bgpd", "frr-bgpd", fwfrr.format("a BGPv4, BGPv4+, BGPv4-"), [], 8), + ("frr-eigrpd", "frr-eigrpd", fwfrr.format("an EIGRP"), [], 8), + ("frr-fabricd", "frr-fabricd", fwfrr.format("an OpenFabric"), [], 8), + ("frr-isisd", "frr-isisd", fwfrr.format("an IS-IS"), [], 8), + ("frr-ldpd", "frr-ldpd", fwfrr.format("an LDP"), [], 8), + ("frr-nhrpd", "frr-nhrpd", fwfrr.format("a Next Hop Routing Protocol"), [], 8), + ("frr-ospf6d", "frr-ospf6d", fwfrr.format("an OSPFv3"), [], 8), + ("frr-ospfclient", "frr-ospfclient", "an example ospf-api client", [], 8), + ("frr-ospfd", "frr-ospfd", fwfrr.format("an OSPFv2"), [], 8), + ("frr-pbrd", "frr-pbrd", fwfrr.format("a PBR"), [], 8), + ("frr-pimd", "frr-pimd", fwfrr.format("a PIM"), [], 8), + ("frr-ripd", "frr-ripd", fwfrr.format("a RIP"), [], 8), + ("frr-ripngd", "frr-ripngd", fwfrr.format("a RIPNG"), [], 8), + ("frr-sharpd", "frr-sharpd", fwfrr.format("a SHARP"), [], 8), + ("frr-staticd", "frr-staticd", fwfrr.format("a static route manager"), [], 8), + ("frr-vrrpd", "frr-vrrpd", fwfrr.format("a VRRP"), [], 8), + ( + "frr-watchfrr", + "frr-watchfrr", + "a program to monitor the status of FRRouting daemons", + [], + 8, + ), + ( + "frr-zebra", + "frr-zebra", + "a routing manager for use with associated FRRouting components.", + [], + 8, + ), + ("frr", "frr", "a systemd interaction script", [], 1), + ("mtracebis", "mtracebis", "a multicast trace client", [], 8), + ("vtysh", "vtysh", "an integrated shell for FRRouting.", [], 1), +] + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) + +# Documents to append as an appendix to all manuals. +# texinfo_appendices = [] + +# If false, no module index is generated. +# texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +# texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +# texinfo_no_detailmenu = False + +# custom extensions here diff --git a/doc/manpages/defines.rst b/doc/manpages/defines.rst new file mode 100644 index 0000000..ac24cfa --- /dev/null +++ b/doc/manpages/defines.rst @@ -0,0 +1,3 @@ +.. |synopsis-options| replace:: [-d|-t|-dt] [-C] [-f config-file] [-i pid-file] [-z zclient-path] [-u user] [-g group] [-A vty-addr] [-P vty-port] [-M module[:options]] [-N pathspace] [--vty_socket vty-path] [--moduledir module-path] +.. |synopsis-options-hv| replace:: [-h] [-v] +.. |seealso-programs| replace:: frr-zebra(8), vtysh(1), frr-ripd(8), frr-ripngd(8), frr-ospfd(8), frr-ospf6d(8), frr-bgpd(8), frr-isisd(8), frr-babeld(8), frr-nhrpd(8), frr-pimd(8), frr-pbrd(8), frr-ldpd(8), frr-eigrpd(8), frr-staticd(8), frr-fabricd(8), frr-vrrpd(8), mtracebis(8) diff --git a/doc/manpages/epilogue.rst b/doc/manpages/epilogue.rst new file mode 100644 index 0000000..77fed81 --- /dev/null +++ b/doc/manpages/epilogue.rst @@ -0,0 +1,16 @@ +WARNING +======= +This man page is intended to be a quick reference for command line options. The definitive document is the info file |PACKAGE_STRING| or the documentation available on the project website at |PACKAGE_URL|. + +DIAGNOSTICS +=========== +The daemon may log to standard output, to a VTY, to a log file, or through syslog to the system logs. FRR supports many debugging options, see the Info file, web docs or source for details. + +SEE ALSO +======== +|seealso-programs| +|PACKAGE_URL| + +BUGS +==== +FRR eats bugs for breakfast. If you have food for the maintainers, please email <dev@lists.frrouting.org>. diff --git a/doc/manpages/frr-bfdd.rst b/doc/manpages/frr-bfdd.rst new file mode 100644 index 0000000..1f8b147 --- /dev/null +++ b/doc/manpages/frr-bfdd.rst @@ -0,0 +1,40 @@ +**** +BFDD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: bfdd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a communication failure detection component that works with +the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst +.. include:: bfd-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-bgpd.rst b/doc/manpages/frr-bgpd.rst new file mode 100644 index 0000000..f7e2026 --- /dev/null +++ b/doc/manpages/frr-bgpd.rst @@ -0,0 +1,82 @@ +**** +BGPD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: bgpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +.. option:: -p, --bgp_port <port> + + Set the bgp protocol's port number. When port number is 0, that means do not + listen bgp port. + +.. option:: -l, --listenon + + Specify a specific IP address for bgpd to listen on, rather than its default + of ``0.0.0.0`` / ``::``. This can be useful to constrain bgpd to an internal + address, or to run multiple bgpd processes on one host. + +.. option:: -n, --no_kernel + + Do not install learned routes into the linux kernel. This option is useful + for a route-reflector environment or if you are running multiple bgp + processes in the same namespace. This option is different than the --no_zebra + option in that a ZAPI connection is made. + +.. option:: -e, --ecmp + + Run BGP with a limited ecmp capability, that is different than what BGP + was compiled with. The value specified must be greater than 0 and less + than or equal to the MULTIPATH_NUM specified on compilation. + +.. option:: -Z, --no_zebra + + Do not communicate with zebra at all. This is different than the --no_kernel + option in that we do not even open a ZAPI connection to the zebra process. + +.. option:: -s, --socket_size + + When opening tcp connections to our peers, set the socket send buffer + size that the kernel will use for the peers socket. This option + is only really useful at a very large scale. Experimentation should + be done to see if this is helping or not at the scale you are running + at. + +LABEL MANAGER +------------- + +.. option:: -I, --int_num + + Set zclient id. This is required when using Zebra label manager in proxy mode. + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-eigrpd.rst b/doc/manpages/frr-eigrpd.rst new file mode 100644 index 0000000..bc82446 --- /dev/null +++ b/doc/manpages/frr-eigrpd.rst @@ -0,0 +1,38 @@ +****** +EIGRPD +****** + +.. include:: defines.rst +.. |DAEMON| replace:: eigrpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-fabricd.rst b/doc/manpages/frr-fabricd.rst new file mode 100644 index 0000000..c14c076 --- /dev/null +++ b/doc/manpages/frr-fabricd.rst @@ -0,0 +1,38 @@ +******* +FABRICD +******* + +.. include:: defines.rst +.. |DAEMON| replace:: fabricd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-isisd.rst b/doc/manpages/frr-isisd.rst new file mode 100644 index 0000000..68761f6 --- /dev/null +++ b/doc/manpages/frr-isisd.rst @@ -0,0 +1,38 @@ +***** +ISISD +***** + +.. include:: defines.rst +.. |DAEMON| replace:: isisd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-ldpd.rst b/doc/manpages/frr-ldpd.rst new file mode 100644 index 0000000..113f066 --- /dev/null +++ b/doc/manpages/frr-ldpd.rst @@ -0,0 +1,38 @@ +**** +LDPD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: ldpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-nhrpd.rst b/doc/manpages/frr-nhrpd.rst new file mode 100644 index 0000000..cae01c6 --- /dev/null +++ b/doc/manpages/frr-nhrpd.rst @@ -0,0 +1,38 @@ +***** +NHRPD +***** + +.. include:: defines.rst +.. |DAEMON| replace:: nhrpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-ospf6d.rst b/doc/manpages/frr-ospf6d.rst new file mode 100644 index 0000000..cfc6860 --- /dev/null +++ b/doc/manpages/frr-ospf6d.rst @@ -0,0 +1,39 @@ +****** +OSPF6D +****** + +.. include:: defines.rst +.. |DAEMON| replace:: ospf6d + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-ospfclient.rst b/doc/manpages/frr-ospfclient.rst new file mode 100644 index 0000000..c52b108 --- /dev/null +++ b/doc/manpages/frr-ospfclient.rst @@ -0,0 +1,43 @@ +********** +OSPFCLIENT +********** + +.. include:: defines.rst + +SYNOPSIS +======== +ospfclient <ospfd> <lsatype> <opaquetype> <opaqueid> <ifaddr> <areaid> + +DESCRIPTION +=========== +ospfclient is an example ospf-api client to test the ospfd daemon. + +OPTIONS +======= + +.. option:: ospfd + + A router where the API-enabled OSPF daemon is running. + +.. option:: lsatype + + The value has to be either "9", "10", or "11", depending on the flooding scope. + +.. option:: opaquetype + + The value has to be in the range of 0-255 (for example, experimental applications might use opaquetype larger than 128). + +.. option:: opaqueid + + Arbitrary application instance (24 bits). + +.. option:: ifaddr + + Interface IP address for type 9, otherwise it will be ignored. + +.. option:: areaid + + Area in the IP address format for type 10, otherwise it will be ignored. + + +.. include:: epilogue.rst diff --git a/doc/manpages/frr-ospfd.rst b/doc/manpages/frr-ospfd.rst new file mode 100644 index 0000000..951a022 --- /dev/null +++ b/doc/manpages/frr-ospfd.rst @@ -0,0 +1,43 @@ +***** +OSPFD +***** + +.. include:: defines.rst +.. |DAEMON| replace:: ospfd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +.. option:: -a, --apiserver + + Enable the OSPF API server. + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-pbrd.rst b/doc/manpages/frr-pbrd.rst new file mode 100644 index 0000000..d9a80b1 --- /dev/null +++ b/doc/manpages/frr-pbrd.rst @@ -0,0 +1,38 @@ +**** +PBRD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: pbrd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + + diff --git a/doc/manpages/frr-pimd.rst b/doc/manpages/frr-pimd.rst new file mode 100644 index 0000000..d758266 --- /dev/null +++ b/doc/manpages/frr-pimd.rst @@ -0,0 +1,38 @@ +**** +PIMD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: pimd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-ripd.rst b/doc/manpages/frr-ripd.rst new file mode 100644 index 0000000..af4590c --- /dev/null +++ b/doc/manpages/frr-ripd.rst @@ -0,0 +1,38 @@ +**** +RIPD +**** + +.. include:: defines.rst +.. |DAEMON| replace:: ripd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-ripngd.rst b/doc/manpages/frr-ripngd.rst new file mode 100644 index 0000000..aedd689 --- /dev/null +++ b/doc/manpages/frr-ripngd.rst @@ -0,0 +1,38 @@ +****** +RIPNGD +****** + +.. include:: defines.rst +.. |DAEMON| replace:: ripngd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-sharpd.rst b/doc/manpages/frr-sharpd.rst new file mode 100644 index 0000000..016f3f9 --- /dev/null +++ b/doc/manpages/frr-sharpd.rst @@ -0,0 +1,38 @@ +****** +SHARPD +****** + +.. include:: defines.rst +.. |DAEMON| replace:: sharpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + + diff --git a/doc/manpages/frr-staticd.rst b/doc/manpages/frr-staticd.rst new file mode 100644 index 0000000..ccbcf32 --- /dev/null +++ b/doc/manpages/frr-staticd.rst @@ -0,0 +1,38 @@ +******* +STATICD +******* + +.. include:: defines.rst +.. |DAEMON| replace:: staticd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting engine. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + + diff --git a/doc/manpages/frr-vrrpd.rst b/doc/manpages/frr-vrrpd.rst new file mode 100644 index 0000000..0e73b07 --- /dev/null +++ b/doc/manpages/frr-vrrpd.rst @@ -0,0 +1,40 @@ +***** +VRRPD +***** + +.. include:: defines.rst +.. |DAEMON| replace:: vrrpd + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing component that works with the FRRouting routing engine. +It implements the Virtual Router Redundancy Protocol. Support for both VRRPv2 +and VRRPv3 is present. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-watchfrr.rst b/doc/manpages/frr-watchfrr.rst new file mode 100644 index 0000000..d8c82ea --- /dev/null +++ b/doc/manpages/frr-watchfrr.rst @@ -0,0 +1,134 @@ +******** +WATCHFRR +******** + +.. include:: defines.rst +.. |DAEMON| replace:: watchfrr + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| [option...] <daemon>... + + +DESCRIPTION +=========== +|DAEMON| is a watchdog program that monitors the status of supplied frr daemons and tries to restart them in case they become unresponsive or shut down. + +To determine whether a daemon is running, it tries to connect to the daemon's VTY UNIX stream socket, and send echo commands to ensure the daemon responds. When the daemon crashes, EOF is received from the socket, so that |DAEMON| can react immediately. + +In order to avoid restarting the daemons in quick succession, you can supply the -m and -M options to set the minimum and maximum delay between the restart commands. The minimum restart delay is recalculated each time a restart is attempted. If the time since the last restart attempt exceeds twice the value of -M, the restart delay is set to the value of -m, otherwise the interval is doubled (but capped at the value of -M). + +OPTIONS +======= + +.. option:: --dry + + Run |DAEMON| in "dry-run" mode, only monitoring the specified daemons but not performing any start/stop/restart actions. + +.. option:: -d, --daemon + + Run in daemon mode. When supplied, error messages are sent to Syslog instead of standard output (stdout). + +.. option:: -S <directory>, --statedir <directory> + + Set the VTY socket directory (the default value is "/var/run/frr"). + +.. option:: -N <name>, --pathspace <name> + + Insert the given name into paths used by the FRR daemons. This is appended + to the VTY socket directory and passed to the daemons which also add it to + their paths in /etc. + +.. option:: --netns[=<name>] + + (Linux only.) Switch network namespaces when starting watchfrr. The name + defaults to the value passed with -N (which it should be used in conjunction + with.) If the name is not specified, the option has no effect. + + If the network namespace does not exist, it is created in a manner + compatible with iproute2. Network namespaces are not removed by FRR, this + must be done with "ip netns delete". + +.. option:: -l <level>, --loglevel <level> + + Set the logging level (the default value is "6"). The value should range from 0 (LOG_EMERG) to 7 (LOG_DEBUG), but higher number can be supplied if extra debugging messages are required. + +.. option:: --min-restart-interval <number> + + Set the minimum number of seconds to wait between invocations of the daemon restart commands (the default value is "60"). + +.. option:: --max-restart-interval <number> + + Set the maximum number of seconds to wait between invocations of the daemon restart commands (the default value is "600"). + +.. option:: -i <number>, --interval <number> + + Set the status polling interval in seconds (the default value is "5"). + +.. option:: -t <number>, --timeout <number> + + Set the unresponsiveness timeout in seconds (the default value is "10"). + +.. option:: -T <number>, --restart-timeout <number> + + Set the restart (kill) timeout in seconds (the default value is "20"). If any background jobs are still running after this period has elapsed, they will be killed. + +.. option:: -p <filename>, --pid-file <filename> + + Set the process identifier filename (the default value is "/var/run/frr/|DAEMON|.pid"). + +.. option:: -b <string>, --blank-string <string> + + When the supplied string is found in any of the command line option arguments (i.e., -r, -s, or -k), replace it with a space. + + This is an ugly hack to circumvent problems with passing the command line arguments containing embedded spaces. + +.. option:: -v, --version + + Display the version information and exit. + +.. option:: -h, --help + + Display the usage information and exit. + +The following 3 options specify scripts that |DAEMON| uses to perform start/stop/restart actions. Reasonable default values are built into watchfrr, so the use of these options should no longer be necessary: + +.. option:: -s command, --start-command command + + Supply a Bourne shell command to start a single daemon. The command string should contain the '%s' placeholder to be substituted with the daemon name. + +.. option:: -k command, --kill-command command + + Supply a Bourne shell command to stop a single daemon. The command string should contain the '%s' placeholder to be substituted with the daemon name. + +.. option:: -r command, --restart command + + Supply a Bourne shell command to restart a single daemon. The command string should contain the '%s' placeholder to be substituted with the daemon name. + +PREVIOUS OPTIONS +================ +Prior versions of |DAEMON| supported some additional options that no longer exist::: + + -a, -A, -e, -R, -z + +The ``-a``, ``-A`` and ``-R`` options were used to select alternate monitoring modes that offered different patterns of restarting daemons. The "correct" mode (phased restart) is now the default. The -e and -z options used to disable some monitoring aspects, |DAEMON| now always has all monitoring features enabled. + +Removing these options should result in correct operation, if it does not please file a bug report. + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr-zebra.rst b/doc/manpages/frr-zebra.rst new file mode 100644 index 0000000..6cc46b8 --- /dev/null +++ b/doc/manpages/frr-zebra.rst @@ -0,0 +1,68 @@ +***** +ZEBRA +***** + +.. include:: defines.rst +.. |DAEMON| replace:: zebra + +SYNOPSIS +======== +|DAEMON| |synopsis-options-hv| + +|DAEMON| |synopsis-options| + +DESCRIPTION +=========== +|DAEMON| is a routing manager that implements the zebra route engine. zebra supports all protocol daemons in the FRRouting suite. + +OPTIONS +======= +OPTIONS available for the |DAEMON| command: + +.. include:: common-options.rst + +.. option:: -b, --batch + + Runs in batch mode, zebra parses its config and exits. + +.. option:: -s, --nl-bufsize <netlink-buffer-size> + + Set netlink receive buffer size. There are cases where zebra daemon can't handle flood of netlink messages from kernel. If you ever see "recvmsg overrun" messages in zebra log, you are in trouble. + + Solution is to increase receive buffer of netlink socket. Note that kernel < 2.6.14 doesn't allow increasing it over maximum value defined in /proc/sys/net/core/rmem_max. If you want to do it, you have to increase maximum before starting zebra. + + Note that this affects Linux only. + + +.. option:: -n, --vrfwnetns + + Enable namespace VRF backend. By default, the VRF backend relies on VRF-lite support from the Linux kernel. This option permits discovering Linux named network namespaces and mapping it to FRR VRF contexts. + +ROUTES +------ + +.. option:: -r, --retain + + When the program terminates, do not flush routes installed by zebra from the kernel. + +.. option:: -R, --routing-table <tableno> + + Specify which kernel routing table *Zebra* should communicate with. + If this option is not specified the default table (RT_TABLE_MAIN) is used. + + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|DAEMON| + The default location of the |DAEMON| binary. + +|INSTALL_PREFIX_ETC|/|DAEMON|.conf + The default location of the |DAEMON| config file. + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/manpages/frr.rst b/doc/manpages/frr.rst new file mode 100644 index 0000000..3d3479a --- /dev/null +++ b/doc/manpages/frr.rst @@ -0,0 +1,44 @@ +*** +FRR +*** + +.. include:: defines.rst + +SYNOPSIS +======== +frr [ start ] + +frr [ stop ] + +frr [ reload ] + +frr [ restart ] + +frr [ status ] + + +DESCRIPTION +=========== +frr is a systemd interaction script for the FRRouting routing engine. + +OPTIONS +======= +Options available for the frr command: + +start + Start enabled FRR daemons + +stop + Stop enabled FRR daemons + +reload + Reload modified configuration files + +restart + Stop all running daemons and then restart them + +status + Status of all the daemon + +.. include:: epilogue.rst + diff --git a/doc/manpages/index.rst b/doc/manpages/index.rst new file mode 100644 index 0000000..58a1d9e --- /dev/null +++ b/doc/manpages/index.rst @@ -0,0 +1,30 @@ +.. FRR documentation master file, created by + sphinx-quickstart on Wed Jan 31 12:00:55 2018. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +.. toctree:: + :maxdepth: 2 + + frr + frr-bfdd + frr-bgpd + frr-eigrpd + frr-isisd + frr-fabricd + frr-ldpd + frr-nhrpd + frr-ospf6d + frr-ospfclient + frr-ospfd + frr-pimd + frr-pbrd + frr-ripd + frr-ripngd + frr-sharpd + frr-staticd + frr-watchfrr + frr-zebra + frr-vrrpd + mtracebis + vtysh diff --git a/doc/manpages/mtracebis.rst b/doc/manpages/mtracebis.rst new file mode 100644 index 0000000..d3ba803 --- /dev/null +++ b/doc/manpages/mtracebis.rst @@ -0,0 +1,44 @@ +********* +MTRACEBIS +********* + +.. include:: defines.rst +.. |PROGRAM| replace:: mtracebis + +SYNOPSIS +======== +|PROGRAM| |synopsis-options-hv| + +|PROGRAM| <multicast source> [<multicast group>] + +DESCRIPTION +=========== +|PROGRAM| is a program for initiating multicast traceroute, or "mtrace", queries. + +It can initiate two types of mtrace queries: weak and group. + +Weak tests whether the interfaces towards the source are multicast enabled and is +initiated by supplying only the multicast source address. + +Group tests whether there is multicast routing protocol state for particular +multicast group and is initiated by supplying mutlicast source and group. + +The first query sent is a full query, capable of crossing the network all the way +to the source. If this fails, hop-by-hop queries are initiated. + +Hop-by-hop queries start by requesting only a response from the nearest router. +Following that, next query is extended to the next two routers, and so on... +until a set of routers is tested for connectivity. + +FILES +===== + +|INSTALL_PREFIX_SBIN|/|PROGRAM| + The default location of the |PROGRAM| binary. + +.. include:: epilogue.rst + +AUTHORS +======= + +Mladen Sablic diff --git a/doc/manpages/subdir.am b/doc/manpages/subdir.am new file mode 100644 index 0000000..9284212 --- /dev/null +++ b/doc/manpages/subdir.am @@ -0,0 +1,75 @@ +# +# doc/manpages +# + +man_RSTFILES = \ + doc/manpages/bfd-options.rst \ + doc/manpages/common-options.rst \ + doc/manpages/conf.py \ + doc/manpages/defines.rst \ + doc/manpages/epilogue.rst \ + doc/manpages/frr-bfdd.rst \ + doc/manpages/frr-bgpd.rst \ + doc/manpages/frr-eigrpd.rst \ + doc/manpages/frr-fabricd.rst \ + doc/manpages/frr-isisd.rst \ + doc/manpages/frr-ldpd.rst \ + doc/manpages/frr-nhrpd.rst \ + doc/manpages/frr-ospf6d.rst \ + doc/manpages/frr-ospfclient.rst \ + doc/manpages/frr-ospfd.rst \ + doc/manpages/frr-pbrd.rst \ + doc/manpages/frr-pimd.rst \ + doc/manpages/frr-ripd.rst \ + doc/manpages/frr-ripngd.rst \ + doc/manpages/frr-sharpd.rst \ + doc/manpages/frr-staticd.rst \ + doc/manpages/frr-vrrpd.rst \ + doc/manpages/frr-watchfrr.rst \ + doc/manpages/frr-zebra.rst \ + doc/manpages/frr.rst \ + doc/manpages/index.rst \ + doc/manpages/mtracebis.rst \ + doc/manpages/vtysh.rst \ + # end + +EXTRA_DIST += $(man_RSTFILES) + +MANPARENT = doc/manpages/_build +MANBUILD = $(MANPARENT)/man +doc/manpages/_build/.doctrees/environment.pickle: $(man_RSTFILES) + +# +# automake integration +# + +rstman1dir = $(mandir)/man1 +rstman8dir = $(mandir)/man8 + +rstman1_DATA = +rstman8_DATA = + +if DOC +rstman1_DATA += $(man1) +rstman8_DATA += $(man8) +endif # DOC + +man1 = $(MANBUILD)/frr.1 +man8 = + +# dependency +$(man8) $(man1): $(MANBUILD)/man.stamp + +# +# hook-ins for clean / doc +# (install is handled by automake _DATA) +# + +clean-local: clean-manpages +.PHONY: clean-manpages +clean-manpages: + -rm -rf $(MANPARENT) + +doc: doc-man +.PHONY: doc-man +doc-man: $(rstman8_DATA) $(rstman1_DATA) diff --git a/doc/manpages/vtysh.rst b/doc/manpages/vtysh.rst new file mode 100644 index 0000000..396fcfc --- /dev/null +++ b/doc/manpages/vtysh.rst @@ -0,0 +1,107 @@ +***** +VTYSH +***** + +.. include:: defines.rst +.. |DAEMON| replace:: vtysh + +SYNOPSIS +======== +vtysh [ -b ] + +vtysh [ -E ] [ -d *daemon* ] [ -c *command* ] + +DESCRIPTION +=========== +vtysh is an integrated shell for the FRRouting suite of protocol daemons. + +OPTIONS +======= +OPTIONS available for the vtysh command: + +.. option:: -b, --boot + + Execute boot startup configuration. It makes sense only if integrated config file is in use (not default in FRRouting). See Info file frr for more info. + +.. option:: -c, --command command + + Specify command to be executed under batch mode. It behaves like -c option in any other shell - command is executed and vtysh exits. + + It's useful for gathering info from FRRouting daemons or reconfiguring daemons from inside shell scripts, etc. Note that multiple commands may be executed by using more than one -c option and/or embedding linefeed characters inside the command string. + +.. option:: -d, --daemon daemon_name + + Specify which daemon to connect to. By default, vtysh attempts to connect to all FRRouting daemons running on the system. With this flag, one can specify a single daemon to connect to instead. For example, specifying '-d ospfd' will connect only to ospfd. This can be particularly useful inside scripts with -c where the command is targeted for a single daemon. + +.. option:: -e, --execute command + + Alias for -c. It's here only for compatibility with Zebra routing software and older FRR versions. This will be removed in future. + +.. option:: -E, --echo + + When the -c option is being used, this flag will cause the standard vtysh prompt and command to be echoed prior to displaying the results. This is particularly useful to separate the results when executing multiple commands. + +.. option:: -C, --dryrun + + When the -C option is being used, this flag will check the config for syntatic validity. + +.. option:: -m, --markfile + + Mark the input file with context ends, useful for cleanup of a config file that has a lot of extraneous space and end markers + +.. option:: -n, --noerror + + When executing cli that does not invoke a vtysh shell, if an error ocurrs ignore it for purposes of return codes from vtysh. + +.. option:: -H, --histfile + + Override the history file for vtysh commands. You can set ``vtysh -H /dev/null`` to turn logging of at all. + +.. option:: -u, --user + + Restrict access to configuration commands by preventing use of the "enable" command. This option provides the same limited "security" as password-protected telnet access. *This security should not be relied on in production environments.* + + Caveat emptor: VTYSH was never designed to be a privilege broker and is not built using secure coding practices. No guarantees of security are provided for this option and under no circumstances should this option be used to provide any semblance of secure read-only access to FRR. + +.. option:: -h, --help + + Display a usage message on standard output and exit. + +.. option:: -t, --timestamp + + Print a timestamp before going to shell or reading the configuration file. + +.. option:: --no-fork + + When used in conjunction with ``-b``, prevents vtysh from forking children to handle configuring each target daemon. + + +ENVIRONMENT VARIABLES +===================== +VTYSH_PAGER + This should be the name of the pager to use. Default is more. + +VTYSH_HISTFILE + Override the history file for vtysh commands. Logging can be turned off using ``VTYSH_HISTFILE=/dev/null vtysh``. + Environment is preferred way to override the history file path over command line argument (-H/--histfile). + +FILES +===== +|INSTALL_PREFIX_SBIN|/vtysh + The default location of the vtysh binary. + +|INSTALL_PREFIX_ETC|/vtysh.conf + The default location of the vtysh config file. + +|INSTALL_PREFIX_ETC|/frr.conf + The default location of the integrated FRRouting routing engine config file if integrated config file is in use. + +${HOME}/.history_frr + Location of history of commands entered via cli + +$(PWD)/|DAEMON|.log + If the |DAEMON| process is configured to output logs to a file, then you + will find this file in the directory where you started |DAEMON|. + +.. include:: epilogue.rst + diff --git a/doc/mpls/ChangeLog.opaque.txt b/doc/mpls/ChangeLog.opaque.txt new file mode 100644 index 0000000..afcfaa3 --- /dev/null +++ b/doc/mpls/ChangeLog.opaque.txt @@ -0,0 +1,192 @@ +----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- +Changes 2013.07.01 + +1. Feature enhancements + + 1.1 Update ospf_te.[c,h] in conformance to RFC3630 and clean the code. + Add new directive to enable MPLS-TE per interface instead of globally + + 1.2 Add support for RFC4970 "Router Information" and RFC5088 "PCE + Capabilities announcement". + + 1.3 Incorporate the mpls documentation into the main stream doc. + +----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- +Changes 2001.12.03 + +1. Bug fixes + + 1.1 Though a new member "oi" has added to "struct ospf_lsa" to control + flooding scope of type-9 Opaque-LSAs, the value was always NULL + because no one set it. + + 1.2 In the function "show_ip_ospf_database_summary()" and "show_lsa_ + detail_adv_router()", VTY output for type-11 Opaque-LSAs did not + work properly. + + 1.3 URL for the opaque-type assignment reference has changed. + + 1.4 In the file "ospf_mpls_te.c", printf formats have changed to + avoid compiler warning messages; "%lu" -> "%u", "%lx" -> "%x". + Note that this hack depends on OS, compiler and their versions. + + 1.5 One of attached documentation "opaque_lsa.txt" has changed to + reflect the latest coding. + +2. Feature enhancements + + 2.1 Knowing that it is an ugly hack, an "officially unallocated" + opaque-type value 0 has newly introduced as a "wildcard", + which matches to all opaque-type. + This value must not be flooded to the network, of course. + + 2.2 The Opaque-core module makes use of newly introduced hooks to + dispatch every LSDB change (LSA installation and deletion) to + preregistered opaque users. + Therefore, by providing appropriate callback functions as new + parameters of "ospf_register_opaque_functab()", an opaque user + can refer to every LSA instance to be installed into, or to be + deleted from, the LSDB. + +----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- +Changes 2001.10.31 + +1. Bug fixes + + 1.1 Since each LSA has their own lifetime, they will remain in a + routing domain (being stored in LSDB of each router), until their + age naturally reach to MaxAge or explicitly being flushed by the + originated router. Therefore, if a router restarted with a short + downtime, it is possible that previously flooded self-originated + LSAs might received if the NSM status is not less than Exchange. + + There were some problems in the way of handling self-originated + Opaque-LSAs if they are contained in a received LSUpd message, + but not installed to the local LSDB yet. + Regardless of some conditions to start originating Opaque-LSAs + (there should be at least one opaque-capable full-state neighbor), + the function "ospf_flood()" will be called to flood and install + this brand-new looking LSA. + As the result, when the NSM of an opaque-capable neighbor gets + full, internal state inconsistency happens; a user of Opaque-LSA + such as MPLS-TE can refer to self-originated LSAs in the local + LSDB, but cannot modify their contents... + + Above problems have fixed with a policy "flush it from the whole + routing domain and keep silent until the flushing completed". + By using this sweeping technique, we can be free from confusion + caused by self-originated LSAs received via network. + + 1.2 The function "ospf_opaque_type_name()" contained massive ifdefs + corresponding to each "opaque-type". + These unnecessary ifdefs are removed completely. + + 1.3 In the function "ospf_delete_opaque_functab()", there was an + improper loop control that causes illegal memory access. + Original coding was "next = nextnode (node)". + + 1.4 The function "ospf_mpls_te_ism_change()" could not handle the + case when the ISM changes from Waiting to DR/BDR/Other. + So, there was a case that even if one of an ISM become + operational and MPLS-TE module has started, the corresponding + Opaque-LSA cannot be originated. + + 1.5 The function "ospf_opaque_lsa_reoriginate_schedule()" did not + allow to be called multiple times, simply because handling + module for the given "lsa-type & opaque-type" already exists. + But this assumption seems to be wrong. + Change the policy to allow this function to be called multiple + times and let the caller to decide what should do when the + corresponding callback function "(* functab->lsa_originator)()" + is called. + +2. Feature enhancements + + 2.1 The global bitmap "opaque" has introduced instead of former flag + "OpaqueCapable", to store complex conditions to handle Opaque-LSAs. + + 2.2 The MPLS-TE module now referes to "draft-katz-yeung-ospf-traffic + -06.txt", no significant changes with 05 version, though. + +----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- +Changes 2001.08.03 + +1. Bug fixes + + 1.1 Even if the ospfd started with opaque capability enabled, when + the ospfd receives an unknown opaque-type (unregistered by the + function "ospf_register_opaque_functab()" beforehand), the LSA + was discarded. As the result, only the opaque-LSAs that have + commonly registered by opaque-capable ospf routers can be + flooded in a routing domain. + + This behavior has fixed so that arbitrary opaque-type LSAs can + be flooded among opaque-capable ospf routers. + If the ospfd has opaque-LSA capability but disabled at runtime, + received opaque-LSAs can be accepted and registered to LSDB as + is, but not be flooded to the network; those opaque LSAs will + remain in LSDB until explicitly flushed by incoming LSUpd + messages with MaxAge, or their age naturally reaches to MaxAge. + + 1.2 The function "ospf_register_opaque_functab()" did not check + if the entry corresponding to the given "lsa-type, opaque-type" + combination already exists or not. + This problem has fixed not to allow multiple registration. + + 1.3 Since type-11 (AS external) LSAs will be flooded beyond areas, + there is little relationship between "struct lsa" and "struct + area". More specifically, the pointer address "lsa->area" can + be NULL if the lsa-type is 11, thus an illegal memory access + will happen. This problem has fixed. + + 1.4 When self-originated opaque-LSAs are received via network and + if the corresponding opaque-type functions are not available + (they have already deleted) at that time, those LSAs were + dropped due to "unknown opaque-type" error. + After the problem 1.1 has fixed, those "self-originated" LSAs + were registered to LSDB and then flooded to the network, even + if the processing functions did not exist... + + After all, this problem has fixed so that those LSAs should + explicitly be flushed from the routing domain immediately, if + the processing functions cannot find at that time. + + 1.5 Some typo have fixed. + + --- EXAMPLE --- + static int + opaque_lsa_originate_callback (list funclist, void *lsa_type_dependent) + ^^^^^ + --- EXAMPLE --- + +2. Feature enhancements + + 2.1 According to the description of rfc2328 in section 10.8, any + change in the router's optional capabilities should trigger + the option re-negotiation procedures with neighbors. + + --- EXCERPT --- + If for some reason the router's optional + capabilities change, the Database Exchange procedure should be + restarted by reverting to neighbor state ExStart. + --- EXCERPT --- + + For the opaque-capability changes, this feature has implemented. + More specifically, if "ospf opaque-lsa" or "no ospf opaque-lsa" + VTY command is given at runtime, all self-originated LSAs will + be flushed immediately and then all neighbor status will be + forced to ExStart by generating SeqNumberMismatch events. + + 2.1 When we change opaque-capability dynamically (ON -> OFF -> ON), + there was no trigger at "OFF->ON" timing to reactivate opaque + LSA handling modules (such as MPLS-TE) that have once forcibly + stopped at "ON->OFF" timing. + Now this dynamic reactivation feature has added. + + 2.2 The MPLS-TE module now referes to "draft-katz-yeung-ospf-traffic + -05.txt", no significant changes with 04 version, though. + +----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- * ----- +Changes 2001.03.28 + + Initial release of Opaque-LSA/MPLS-TE extensions for the zebra/ospfd. diff --git a/doc/mpls/cli_summary.txt b/doc/mpls/cli_summary.txt new file mode 100644 index 0000000..c60d0ae --- /dev/null +++ b/doc/mpls/cli_summary.txt @@ -0,0 +1,90 @@ +Summary of CLI commands, expanded for Opaque-LSA/MPLS-TE. +--------------------------------------------------------- + +router> + + show ip ospf database (asbr-summary|external|max-age|network|router|self-originate|summary|opaque-link|opaque-area|opaque-external) + + show ip ospf database (asbr-summary|external|network|router|summary|opaque-link|opaque-area|opaque-external) (self-originate|) + + show ip ospf database (asbr-summary|external|network|router|summary|opaque-link|opaque-area|opaque-external) A.B.C.D + + show ip ospf database (asbr-summary|external|network|router|summary|opaque-link|opaque-area|opaque-external) A.B.C.D (self-originate|) + + show ip ospf database (asbr-summary|external|network|router|summary|opaque-link|opaque-area|opaque-external) A.B.C.D adv-router A.B.C.D + + show ip ospf database (asbr-summary|external|network|router|summary|opaque-link|opaque-area|opaque-external) adv-router A.B.C.D + + --> Add database items: opaque-link, opaque-area, opaque-external + + show mpls-te interface [INTERFACE] + + --> Show current MPLS-TE link-TLV parameters. + If [INTERFACE] is omitted, all interfaces will be displayed. + + show mpls-te router + + --> Show current MPLS-TE Router-TLV parameters. + +router> enable +router# +router# configure terminal +router(config)# interface [INTERFACE] +router(config-if)# + + mpls-te link max-bw BANDWIDTH + + --> Set MPLS-TE link-TLV parameter: Maximum Bandwidth (Bytes/sec). + In integer or floating point format (1000, or 1.0e3) + + mpls-te link max-rsv-bw BANDWIDTH + + --> Set MPLS-TE link-TLV parameter: Maximum Reservable Bandwidth (Bytes/sec). + In integer or floating point format (1000, or 1.0e3) + + mpls-te link metric <0-4294967295> + + --> Set MPLS-TE link-TLV parameter: MPLS-TE metric. + + mpls-te link rsc-clsclr BITPATTERN + + --> Set MPLS-TE link-TLV parameter: Resource Class/Color. + In 32-bit hexadecimal format, with leading "0x" (0x0 - 0xffffffff) + + mpls-te link unrsv-bw <0-7> BANDWIDTH + + --> Set MPLS-TE link-TLV parameter: Unreserved Bandwidth (Bytes/sec). + In integer or floating point format (1000, or 1.0e3) + +router(config-if)# exit +router(config)# router ospf +router(config-router)# + + mpls-te + + --> Enable MPLS-TE functionality. + Note that master-switch "ospf opaque-lsa" must also be specified. + + mpls-te on + + --> Alias of "mpls-te" command. + + mpls-te router-address A.B.C.D + + --> Set MPLS-TE Router-TLV parameter: Router Address. + + no mpls-te + + --> Disable MPLS-TE functionality. + + no ospf opaque-lsa + + --> Disable Opaque-LSAs capability. + This node behaves Opaque-incapable node. + + ospf opaque-lsa + + --> Enable Opaque-LSAs capability. + This is the master-switch to make this node Opaque-capable. + +router# exit diff --git a/doc/mpls/opaque_lsa.txt b/doc/mpls/opaque_lsa.txt new file mode 100644 index 0000000..ae89ee3 --- /dev/null +++ b/doc/mpls/opaque_lsa.txt @@ -0,0 +1,365 @@ +1. List of "opaque-type dependent" callback functions per LSA-type. + + <ospf_opaque_lsaN_functab> (N = 9,10,11) + | + | struct + | list struct struct + +-> +-------+ listnode listnode + | head |-----> +------+ +------ + | tail | | next |--------------------> | next + | count | /--| prev |<---------------------| prev + +-------+ | data |----+ | + |///////| +------+ | + +-------+ | + | + struct | + ospf_opaque_tabent | + +----------------------+ <--+ + | opaque_type | + +----------------------+ + | (Callback functions) | + +----------------------+ + + +2. Self-originated Opaque-LSAs per LSA-type. + +2.1 Type-11 (AS-external) Opaque-LSAs + + struct + ospf + +---> +-------------------+ + | |///////////////////| + | +-------------------+ + | | opaque | + | +-------------------+ + | |///////////////////| + | +-------------------+ + | | opaque_lsa_self |---+ + | +-------------------+ | + | |///////////////////| | + | +-------------------+ | + | | + ......|.............................|....................................... + : | | Almost common for type-9,10,11 LSA : + : | +-----------------------+ : + : | | : + : | | struct : + : | | list struct struct : + : | +-> +-------+ listnode listnode : + : | | head |-----> +------+ +------ : + : | | tail | | next |--------------------> | next : + : | | count | /--| prev |<---------------------| prev : + : | +-------+ | data |---+ | : + : | |///////| +------+ | : + : | +-------+ | : + : | | : + : | struct | : + : | opaque_info_per_type | : + : | +-------------------+ <--------+ : + : | | opaque_type | <------------+ : + : | +-------------------+ | : + : | | status | | : + : | +-------------------+ | : + : | | t_opaque_lsa_self | | : + : | +-------------------+ | : + : +-----| owner | | struct : + : +-------------------+ | ospf_opaque_tabent : + : | functab |-------------------> +---------------- : + : +-------------------+ | | opaque_type : + : | id_list |---+ | |(Callback Funcs) : + : +-------------------+ | | | : + : | | : + : +-----------------------+ | : + : | | : + : | struct | : + : | list struct | struct : + : +-> +-------+ listnode | listnode : + : | head |-----> +------+ | +------ : + : | tail | | next |--------------------> | next : + : | count | /--| prev |<---------------------| prev : + : +-------+ | data |---+ | | : + : |///////| +------+ | | : + : +-------+ | | : + : | | : + : struct | | : + : opaque_info_per_id | | : + : +-------------------+ <--------+ | : + : | opaque_id | | : + : +-------------------+ | : + : | t_opaque_lsa_self | | : + : +-------------------+ | : + : | opqctl_type |--------------+ : + : +-------------------+ : + : | lsa |---+ : + : +-------------------+ | : + : | : + : struct | : + : ospf_lsa | : + : +-------------+ <-------+ : + : |/////////////| struct : + : +-------------+ lsa_header : + : | data |--------------> +-------- : + : +-------------+ | : + : |/////////////| : + : +-------------+ : + : +--------| area | : + : | +-------------+ : + : --- |/////////////| : + : +-------------+ : + : +-----| oi | : + : | +-------------+ : + : --- : + :..........................................................................: + +2.2 Type-10 (area-local) Opaque-LSAs + + struct + ospf + +---------+ <-----------+ + |/////////| | + +---------+ | + | + struct | + ospf_area | + +--+---> +-----------------+ | + | | | top |-----+ + | | +-----------------+ + | | |/////////////////| struct + | | +-----------------+ ospf_lsa + | | | router_lsa_self |-----------> +--------- + | | +-----------------+ | + | | | opaque_lsa_self |-----+ | + | | +-----------------+ | + | | |/////////////////| | + | | +-----------------+ | + | | | + ...|..|.............................|....................................... + : | | | Almost common for type-9,10,11 LSA : + : | | +-----------------------+ : + : | | | : + : | | | struct : + : | | | list struct struct : + : | | +-> +-------+ listnode listnode : + : | | | head |-----> +------+ +------ : + : | | | tail | | next |--------------------> | next : + : | | | count | /--| prev |<---------------------| prev : + : | | +-------+ | data |---+ | : + : | | |///////| +------+ | : + : | | +-------+ | : + : | | | : + : | | struct | : + : | | opaque_info_per_type | : + : | | +-------------------+ <--------+ : + : | | | opaque_type | <------------+ : + : | | +-------------------+ | : + : | | | status | | : + : | | +-------------------+ | : + : | | | t_opaque_lsa_self | | : + : | | +-------------------+ | : + : | +-----| owner | | struct : + : | +-------------------+ | ospf_opaque_tabent : + : | | functab |-------------------> +---------------- : + : | +-------------------+ | | opaque_type : + : | | id_list |---+ | |(Callback Funcs) : + : | +-------------------+ | | | : + : | | | : + : | +-----------------------+ | : + : | | | : + : | | struct | : + : | | list struct | struct : + : | +-> +-------+ listnode | listnode : + : | | head |-----> +------+ | +------ : + : | | tail | | next |--------------------> | next : + : | | count | /--| prev |<---------------------| prev : + : | +-------+ | data |---+ | | : + : | |///////| +------+ | | : + : | +-------+ | | : + : | | | : + : | struct | | : + : | opaque_info_per_id | | : + : | +-------------------+ <--------+ | : + : | | opaque_id | | : + : | +-------------------+ | : + : | | t_opaque_lsa_self | | : + : | +-------------------+ | : + : | | opqctl_type |--------------+ : + : | +-------------------+ : + : | | lsa |---+ : + : | +-------------------+ | : + : | | : + : | struct | : + : | ospf_lsa | : + : | +-------------+ <-------+ : + : | |/////////////| struct : + : | +-------------+ lsa_header : + : | | data |--------------> +-------- : + : | +-------------+ | : + : | |/////////////| : + : | +-------------+ : + : +--------| area | : + : +-------------+ : + : |/////////////| : + : +-------------+ : + : +-----| oi | : + : | +-------------+ : + : --- : + :..........................................................................: + +2.3 Type-9 (link-local) Opaque-LSAs + + struct + ospf_area + +------> +---------+ <---------+ + | |/////////| | + | +---------+ | + | | + | struct | + | ospf_interface | + | +-+-> +-----------------+ | + | | | |/////////////////| | + | | | +-----------------+ | + | | | | area |---+ + | | | +-----------------+ + | | | |/////////////////| struct + | | | +-----------------+ ospf_lsa + | | | |network_lsa_self |-----------> +--------- + | | | +-----------------+ | + | | | | opaque_lsa_self |-----+ | + | | | +-----------------+ | + | | | |/////////////////| | + | | | +-----------------+ | + | | | | + ...|..|.|...........................|....................................... + : | | | | Almost common for type-9,10,11 LSA : + : | | | +-----------------------+ : + : | | | | : + : | | | | struct : + : | | | | list struct struct : + : | | | +-> +-------+ listnode listnode : + : | | | | head |-----> +------+ +------ : + : | | | | tail | | next |--------------------> | next : + : | | | | count | /--| prev |<---------------------| prev : + : | | | +-------+ | data |---+ | : + : | | | |///////| +------+ | : + : | | | +-------+ | : + : | | | | : + : | | | struct | : + : | | | opaque_info_per_type | : + : | | | +-------------------+ <--------+ : + : | | | | opaque_type | <------------+ : + : | | | +-------------------+ | : + : | | | | status | | : + : | | | +-------------------+ | : + : | | | | t_opaque_lsa_self | | : + : | | | +-------------------+ | : + : | | +---| owner | | struct : + : | | +-------------------+ | ospf_opaque_tabent : + : | | | functab |-------------------> +---------------- : + : | | +-------------------+ | | opaque_type : + : | | | id_list |---+ | |(Callback Funcs) : + : | | +-------------------+ | | | : + : | | | | : + : | | +-----------------------+ | : + : | | | | : + : | | | struct | : + : | | | list struct | struct : + : | | +-> +-------+ listnode | listnode : + : | | | head |-----> +------+ | +------ : + : | | | tail | | next |--------------------> | next : + : | | | count | /--| prev |<---------------------| prev : + : | | +-------+ | data |---+ | | : + : | | |///////| +------+ | | : + : | | +-------+ | | : + : | | | | : + : | | struct | | : + : | | opaque_info_per_id | | : + : | | +-------------------+ <--------+ | : + : | | | opaque_id | | : + : | | +-------------------+ | : + : | | | t_opaque_lsa_self | | : + : | | +-------------------+ | : + : | | | opqctl_type |--------------+ : + : | | +-------------------+ : + : | | | lsa |---+ : + : | | +-------------------+ | : + : | | | : + : | | struct | : + : | | ospf_lsa | : + : | | +-------------+ <-------+ : + : | | |/////////////| struct : + : | | +-------------+ lsa_header : + : | | | data |--------------> +-------- : + : | | +-------------+ | : + : | | |/////////////| : + : | | +-------------+ : + : +--|-----| area | : + : | +-------------+ : + : | |/////////////| : + : | +-------------+ : + : +-----| oi | : + : +-------------+ : + :..........................................................................: + + +3. Internal structures for MPLS-TE parameter management. + + struct + ospf_mpls_te + +-------------+ + | status | + +-------------+ + | iflist |---+ + +-------------+ | + |(Router-TLV) | | + +-------------+ | + | + +---------------------+ + | + | struct + | list struct struct + +---> +-------+ listnode listnode + | head |-----> +------+ +------ + | tail | | next |--------------------> | next + | count | /--| prev |<---------------------| prev + +-------+ | data |---+ | + |///////| +------+ | + +-------+ | + | + +--------------------------------+ + | + | struct + | ospf_mpls_te_linkparms + +-> +----------------+ + | instance | struct + +----------------+ interface + | ifp |--------------------> +----------+ + +----------------+ +----> |//////////| + | area |----+ | +----------+ + +----------------+ | | | info |-----+ + | flags | | | +----------+ | + +----------------+ | | |//////////| | + | (Link-TLV) | | | +----------+ | + +----------------+ | | | + | (Link-SubTLVs) | | | struct | + +----------------+ | | ospf_if_info | + | | +----------+ <---+ + | | |//////////| + struct | | +----------+ + ospf_area | | | oifs |-----+ + +-> +--------------+ <----+ | +----------+ | + | |//////////////| | | + | +--------------+ | struct | + | | route_table | + | struct | +-----------+ <--+ + | ospf_interface | | route_top | - - - - -. + | +--------------+ <----+ | +-----------+ . + | |//////////////| | | . + | +--------------+ | | struct . + | | ifp |------|----------+ route_node . + | +--------------+ | +-----------+ < - - - - + | |//////////////| | |///////////| + | +--------------+ | +-----------+ + +---| area | +-----------------| info | + +--------------+ +-----------+ + |//////////////| |///////////| + +--------------+ +-----------+ diff --git a/doc/mpls/ospfd.conf b/doc/mpls/ospfd.conf new file mode 100644 index 0000000..2b15fa4 --- /dev/null +++ b/doc/mpls/ospfd.conf @@ -0,0 +1,76 @@ +! +! Zebra configuration saved from vty +! 2001/03/16 22:07:53 +! +hostname HOSTNAME +password PASSWORD +log file /var/log/ospfd.log +! +debug ospf ism +debug ospf nsm +debug ospf lsa +debug ospf zebra +debug ospf event +debug ospf packet all detail +! +! +interface fxp0 + ip ospf hello-interval 60 + ip ospf dead-interval 240 + mpls-te on + mpls-te link metric 999 + mpls-te link max-bw 1.25e+06 + mpls-te link max-rsv-bw 1.25e+06 + mpls-te link unrsv-bw 0 1.25e+06 + mpls-te link unrsv-bw 1 1.25e+06 + mpls-te link unrsv-bw 2 1.25e+06 + mpls-te link unrsv-bw 3 1.25e+06 + mpls-te link unrsv-bw 4 1.25e+06 + mpls-te link unrsv-bw 5 1.25e+06 + mpls-te link unrsv-bw 6 1.25e+06 + mpls-te link unrsv-bw 7 1.25e+06 + mpls-te link rsc-clsclr 0xab +! +interface de1 + ip ospf hello-interval 60 + ip ospf dead-interval 240 + mpls-te link metric 111 + mpls-te link max-bw 1.25e+06 + mpls-te link max-rsv-bw 1.25e+06 + mpls-te link unrsv-bw 0 1.25e+06 + mpls-te link unrsv-bw 1 1.25e+06 + mpls-te link unrsv-bw 2 1.25e+06 + mpls-te link unrsv-bw 3 1.25e+06 + mpls-te link unrsv-bw 4 1.25e+06 + mpls-te link unrsv-bw 5 1.25e+06 + mpls-te link unrsv-bw 6 1.25e+06 + mpls-te link unrsv-bw 7 1.25e+06 + mpls-te link rsc-clsclr 0xcd +! +interface de0 + mpls-te link metric 0 + mpls-te link rsc-clsclr 0x0 +! +interface lp0 + ip ospf network point-to-point +! +interface tun0 + ip ospf network point-to-point +! +interface sl0 + ip ospf network point-to-point +! +interface ppp0 + ip ospf network point-to-point +! +interface lo0 +! +router ospf + compatible rfc1583 + network 192.168.0.0/16 area 1 + ospf opaque-lsa + mpls-te + mpls-te router-address 1.2.3.4 +! +line vty +! diff --git a/doc/subdir.am b/doc/subdir.am new file mode 100644 index 0000000..2795326 --- /dev/null +++ b/doc/subdir.am @@ -0,0 +1,179 @@ +# +# doc +# + +# You can set these variables from the command line. +SPHINXOPTS ?= +PAPER ?= + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) + +### + +AM_V_SPHINX = $(am__v_SPHINX_$(V)) +am__v_SPHINX_ = $(am__v_SPHINX_$(AM_DEFAULT_VERBOSITY)) +am__v_SPHINX_0 = @echo " SPHINX " $@; +am__v_SPHINX_1 = +AM_V_MAKEINFO = $(am__v_MAKEINFO_$(V)) +am__v_MAKEINFO_ = $(am__v_MAKEINFO_$(AM_DEFAULT_VERBOSITY)) +am__v_MAKEINFO_0 = @echo " MAKEINFO" $@; +am__v_MAKEINFO_1 = + +# +# real-file sphinx targets that work for dependencies +# + +doc/%/_build/.doctrees/environment.pickle: + $(AM_V_SPHINX) ( \ + subdoc="$@"; subdoc="$${subdoc#doc/}"; subdoc="doc/$${subdoc%%/*}"; \ + $(PYTHON) $(PYSPHINX) -a -q -b text -d "$${subdoc}/_build/.doctrees" \ + $(ALLSPHINXOPTS) "$(top_srcdir)/$${subdoc}" "$${subdoc}/_build/text" \ + ) +doc/%/_build/html/.buildinfo: doc/%/_build/.doctrees/environment.pickle + $(AM_V_SPHINX) ( \ + subdoc="$@"; subdoc="$${subdoc#doc/}"; subdoc="doc/$${subdoc%%/*}"; \ + $(PYTHON) $(PYSPHINX) -q -b html -d "$${subdoc}/_build/.doctrees" \ + $(ALLSPHINXOPTS) "$(top_srcdir)/$${subdoc}" "$${subdoc}/_build/html" \ + ) +.PRECIOUS: doc/%/_build/texinfo/frr.texi +doc/%/_build/texinfo/frr.texi: doc/%/_build/.doctrees/environment.pickle + $(AM_V_SPHINX) ( \ + subdoc="$@"; subdoc="$${subdoc#doc/}"; subdoc="doc/$${subdoc%%/*}"; \ + $(PYTHON) $(PYSPHINX) -q -b texinfo -d "$${subdoc}/_build/.doctrees" \ + $(ALLSPHINXOPTS) "$(top_srcdir)/$${subdoc}" "$${subdoc}/_build/texinfo" \ + ) +doc/%/_build/texinfo/frr.info: doc/%/_build/texinfo/frr.texi + $(AM_V_MAKEINFO)$(MAKEINFO) --no-split -o '$@' '$<' +doc/%/_build/man/man.stamp: doc/%/_build/.doctrees/environment.pickle + $(AM_V_SPHINX) ( \ + subdoc="$@"; subdoc="$${subdoc#doc/}"; subdoc="doc/$${subdoc%%/*}"; \ + $(MKDIR_P) "$${subdoc}/_build/man"; touch $@.tmp; \ + $(PYTHON) $(PYSPHINX) -a -q -b man -d "$${subdoc}/_build/.doctrees" \ + $(ALLSPHINXOPTS) "$(top_srcdir)/$${subdoc}" "$${subdoc}/_build/man" && \ + mv -f $@.tmp $@ \ + ) + +# +# auxiliary sphinx targets (output name = directory, +# deps will not work very well) +# + +SPHINXTARGETS = \ + html dirhtml singlehtml pickle json \ + htmlhelp qthelp applehelp devhelp \ + epub latex text man texinfo gettext \ + changes linkcheck doctest coverage \ + xml pseudoxml \ + # end + +M_SPHINXTARGETS = $(addprefix doc/%/_build/,$(SPHINXTARGETS)) +.PRECIOUS: $(M_SPHINXTARGETS) +$(M_SPHINXTARGETS): doc/%/_build/.doctrees/environment.pickle + $(AM_V_SPHINX) ( \ + target="$@"; \ + builder="$${target##*/}"; \ + subdoc="$${target#doc/}"; subdoc="doc/$${subdoc%%/*}"; \ + rm -rf "$@"; \ + $(PYTHON) $(PYSPHINX) -q -b $${builder} -d $${subdoc}/_build/.doctrees \ + $(ALLSPHINXOPTS) $(top_srcdir)/$${subdoc} $@ \ + ) + +.PHONY: doc/%/_build/latexpdf +doc/%/_build/latexpdf: doc/%/_build/latex + @make -C $< all-pdf + +# If you want to build the developer's docs in other formats, try the +# following: +# +# $ cd developer +# $ make help + +# dist tarballs want doc sources +EXTRA_DIST += \ + doc/mpls/ChangeLog.opaque.txt \ + doc/mpls/ospfd.conf \ + doc/mpls/cli_summary.txt \ + doc/mpls/opaque_lsa.txt \ + doc/figures/cli-change-client.drawio \ + doc/figures/cli-change-client.svg \ + doc/figures/cli-change-mgmtd.drawio \ + doc/figures/cli-change-mgmtd.svg \ + doc/figures/cligraph.png \ + doc/figures/cligraph.svg \ + doc/figures/fig-normal-processing.dia \ + doc/figures/fig-normal-processing.png \ + doc/figures/fig-normal-processing.txt \ + doc/figures/fig-rs-processing.dia \ + doc/figures/fig-rs-processing.png \ + doc/figures/fig-rs-processing.txt \ + doc/figures/fig_topologies_full.dia \ + doc/figures/fig_topologies_full.png \ + doc/figures/fig_topologies_full.txt \ + doc/figures/fig_topologies_rs.dia \ + doc/figures/fig_topologies_rs.png \ + doc/figures/fig_topologies_rs.txt \ + doc/figures/fig-vnc-commercial-route-reflector.dia \ + doc/figures/fig-vnc-commercial-route-reflector.png \ + doc/figures/fig-vnc-commercial-route-reflector.txt \ + doc/figures/fig-vnc-frr-route-reflector.dia \ + doc/figures/fig-vnc-frr-route-reflector.png \ + doc/figures/fig-vnc-frr-route-reflector.txt \ + doc/figures/fig-vnc-gw.dia \ + doc/figures/fig-vnc-gw.png \ + doc/figures/fig-vnc-gw-rr.dia \ + doc/figures/fig-vnc-gw-rr.png \ + doc/figures/fig-vnc-gw-rr.txt \ + doc/figures/fig-vnc-gw.txt \ + doc/figures/fig-vnc-mesh.dia \ + doc/figures/fig-vnc-mesh.png \ + doc/figures/fig-vnc-mesh.txt \ + doc/figures/fig-vnc-redundant-route-reflectors.dia \ + doc/figures/fig-vnc-redundant-route-reflectors.png \ + doc/figures/fig-vnc-redundant-route-reflectors.txt \ + doc/figures/frr-icon.svg \ + doc/figures/frr-logo-icon.png \ + doc/figures/frr-logo-medium.png \ + doc/figures/frr-logo.png \ + doc/figures/frr-logo-small.png \ + doc/figures/git_branches.png \ + doc/figures/git_branches.svg \ + doc/figures/ospf_api_architecture.png \ + doc/figures/ospf_api_msghdr.png \ + doc/figures/ospf_api_msgs1.png \ + doc/figures/ospf_api_msgs2.png \ + doc/extra/frrlexer.py \ + # end + + +.PHONY: doc/help +doc/help: + @echo "Please use \`make doc/{user,manpages,developer}/<target>' where <target> is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " applehelp to make an Apple Help Book" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " texinfo to make Texinfo files" + @echo " info to make Texinfo files and run them through makeinfo" + @echo " gettext to make PO message catalogs" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " xml to make Docutils-native XML files" + @echo " pseudoxml to make pseudoxml-XML files for display purposes" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + @echo " coverage to run coverage check of the documentation (if enabled)" diff --git a/doc/user/.gitignore b/doc/user/.gitignore new file mode 100644 index 0000000..81c60dc --- /dev/null +++ b/doc/user/.gitignore @@ -0,0 +1,2 @@ +/_templates +/_build diff --git a/doc/user/.readthedocs.yaml b/doc/user/.readthedocs.yaml new file mode 100644 index 0000000..c5a11da --- /dev/null +++ b/doc/user/.readthedocs.yaml @@ -0,0 +1,12 @@ +# Required +version: 2 + +# Set the version of Python and other tools you might need +build: + os: ubuntu-22.04 + tools: + python: "3.11" + +# Build documentation in the docs/ directory with Sphinx +sphinx: + configuration: doc/user/conf.py diff --git a/doc/user/Makefile b/doc/user/Makefile new file mode 100644 index 0000000..840ee5b --- /dev/null +++ b/doc/user/Makefile @@ -0,0 +1,16 @@ +all: ALWAYS + @$(MAKE) -s -C ../.. doc-user +help: ALWAYS + @$(MAKE) -s -C ../.. doc/help +pdf: ALWAYS + @$(MAKE) -s -C ../.. doc/user/_build/latexpdf +info: ALWAYS + @$(MAKE) -s -C ../.. doc/user/_build/texinfo/frr.info +%: ALWAYS + @$(MAKE) -s -C ../.. doc/user/_build/$@ + +Makefile: + #nothing +ALWAYS: +.PHONY: ALWAYS makefiles +.SUFFIXES: diff --git a/doc/user/Useful_Sysctl_Settings.md b/doc/user/Useful_Sysctl_Settings.md new file mode 100644 index 0000000..0ebf911 --- /dev/null +++ b/doc/user/Useful_Sysctl_Settings.md @@ -0,0 +1,59 @@ +# Useful Sysctl Settings +Sysctl on Linux systems can tweak many useful behaviors. When it comes to a routing protocol suite like FRRouting there are numerous values depending on your use case that make sense to optimize. + +The below sysctl values provide a logical set of defaults which can be further optimized. + + +``` +# /etc/sysctl.d/99frr_defaults.conf +# Place this file at the location above and reload the device. +# or run the sysctl -p /etc/sysctl.d/99frr_defaults.conf + +# Enables IPv4/IPv6 Routing +net.ipv4.ip_forward = 1 +net.ipv6.conf.all.forwarding=1 + +# Routing +net.ipv6.route.max_size=131072 +net.ipv4.conf.all.ignore_routes_with_linkdown=1 +net.ipv6.conf.all.ignore_routes_with_linkdown=1 + +# Best Settings for Peering w/ BGP Unnumbered +# and OSPF Neighbors +net.ipv4.conf.all.rp_filter = 0 +net.ipv4.conf.default.rp_filter = 0 +net.ipv4.conf.lo.rp_filter = 0 +net.ipv4.conf.all.forwarding = 1 +net.ipv4.conf.default.forwarding = 1 +net.ipv4.conf.default.arp_announce = 2 +net.ipv4.conf.default.arp_notify = 1 +net.ipv4.conf.default.arp_ignore=1 +net.ipv4.conf.all.arp_announce = 2 +net.ipv4.conf.all.arp_notify = 1 +net.ipv4.conf.all.arp_ignore=1 +net.ipv4.icmp_errors_use_inbound_ifaddr=1 + +# Miscellaneous Settings + +# Keep ipv6 permanent addresses on an admin down +net.ipv6.conf.all.keep_addr_on_down=1 +net.ipv6.route.skip_notify_on_dev_down=1 + +# igmp +net.ipv4.igmp_max_memberships=1000 +net.ipv4.neigh.default.mcast_solicit = 10 + +# MLD +net.ipv6.mld_max_msf=512 + +# Garbage Collection Settings for ARP and Neighbors +net.ipv4.neigh.default.gc_thresh2=7168 +net.ipv4.neigh.default.gc_thresh3=8192 +net.ipv4.neigh.default.base_reachable_time_ms=14400000 +net.ipv6.neigh.default.gc_thresh2=3584 +net.ipv6.neigh.default.gc_thresh3=4096 +net.ipv6.neigh.default.base_reachable_time_ms=14400000 + +# Use neigh information on selection of nexthop for multipath hops +net.ipv4.fib_multipath_use_neigh=1 +``` diff --git a/doc/user/_static/overrides.css b/doc/user/_static/overrides.css new file mode 100644 index 0000000..638f619 --- /dev/null +++ b/doc/user/_static/overrides.css @@ -0,0 +1,295 @@ +/* remove max-width restriction */ +div.body { + max-width: none; +} + +/* styling for the protocols vs. OS table in overview.rst */ +/* first, general bits */ +div.body td.mark { + text-align: center; + border-left: 1px solid #ccc; +} +table.mark th { + text-align: center; +} +table.mark td { + vertical-align: middle; +} +table.mark cite { + font-weight: bold; +} + +/* individual Y/N/... cells */ +td.mark { + width: 4.5em; +} +table.mark strong { + display:block; + text-align: center; + margin:auto; + padding-top: 8pt; + padding-bottom: 2pt; +} +td.mark span { + display: block; + padding: 3px 1px; + border: 1px dotted #666; + width: 36pt; + margin:auto; + text-align:center; +} +table.mark tr td:first-child { + padding-left:1.5em; +} +table.mark tr td:first-child cite { + margin-left:-1.5em; +} +span.mark-y { background-color: #77ffaa; } +span.mark-geq { background-color: #aaff77; } +span.mark-cp { background-color: #ffbb55; } +span.mark-n { background-color: #ff8877; } +span.mark-dag { background-color: #ffee99; font-size: 8pt; padding:0px 1px; border-top:0px; } + +/* for the legend below */ +li span.mark { + display: inline-block; + padding: 3px 1px; + border: 1px dotted #666; + width: 36pt; + text-align: center; +} + +/* Palette URL: http://paletton.com/#uid=70p0p0kt6uvcDRAlhBavokxLJ6w */ + +:root { +--primary-0: #F36F16; /* Main Primary color */ +--primary-1: #FFC39A; +--primary-2: #FF9A55; +--primary-3: #A34403; +--primary-4: #341500; +--primary-9: #FFF3EB; + +--secondary-1-0: #F39C16; /* Main Secondary color (1) */ +--secondary-1-1: #FFD79A; +--secondary-1-2: #FFBC55; +--secondary-1-3: #A36403; +--secondary-1-4: #341F00; +--secondary-1-9: #FFF7EB; + +--secondary-2-0: #1A599F; /* Main Secondary color (2) */ +--secondary-2-1: #92B9E5; +--secondary-2-2: #477CB8; +--secondary-2-3: #0A386B; +--secondary-2-4: #011122; +--secondary-2-9: #E3EBF4; + +--complement-0: #0E9A83; /* Main Complement color */ +--complement-1: #8AE4D4; +--complement-2: #3CB4A0; +--complement-3: #026857; +--complement-4: #00211B; +--complement-9: #E0F4F0; +} + +/* new */ + +body { + font-family: "Fira Sans", Helvetica, Arial, sans-serif; + font-weight:400; +} +h1, h2, h3, h4, h5, h6 { + font-family: "Fira Sans", Helvetica, Arial, sans-serif; + font-weight:500; +} +code, pre, tt { + font-family: "Fira Mono"; +} +h1 { + background-color:var(--secondary-1-1); + border-bottom:1px solid var(--secondary-1-0); + font-weight:300; +} +h2 { + margin-top:36pt; +} + +a, +a:hover, +a:visited, +.code-block-caption a.headerlink:hover, +.rst-content dl:not(.docutils) dt .headerlink { + color: var(--complement-0); +} +.code-block-caption a.headerlink { + visibility:hidden; +} + +/* admonitions */ + +.admonition.warning { + border:1px dashed var(--primary-2); +} +.admonition.warning .admonition-title { + color: var(--primary-3); + background-color: var(--primary-1); +} +.admonition.note, +.admonition.hint { + border:1px dashed var(--complement-2); +} +.admonition.note .admonition-title, +.admonition.hint .admonition-title { + color: var(--complement-3); + background-color: var(--complement-1); +} +.admonition.seealso, +div.seealso { + background-color:var(--complement-9); +} +.admonition.seealso .admonition-title { + color: var(--complement-3); + background-color:var(--complement-1); + border-bottom:1px solid var(--complement-2); +} +.admonition.admonition-todo .admonition-title { + background-image: repeating-linear-gradient( + 135deg, + #ffa, + #ffa 14.14213452px, + #bbb 14.14213452px, + #bbb 28.28427124px + ); + color:#000; +} +.admonition.admonition-todo { + background-image: repeating-linear-gradient( + 135deg, + #ffd, + #ffd 14.14213452px, + #eed 14.14213452px, + #eed 28.28427124px + ); +} + +.rst-content dl .admonition p.last { + margin-bottom:0 !important; +} + +/* file block */ + +.code-block-caption { +/* border-radius: 4px; */ + font-style:italic; + font-weight:300; + border-bottom: 1px solid var(--secondary-2-1); + background-color: var(--secondary-2-9); + padding:2px 8px; +} + +/* navbar */ + +.wy-nav-side { + background-color: var(--secondary-1-4); + border-right:2px solid var(--primary-3); +} +.wy-menu-vertical a, +.wy-menu-vertical a:visited, +.wy-menu-vertical a:hover, +.wy-side-nav-search>a, +.wy-side-nav-search .wy-dropdown>a { + color: var(--primary-0); +} + +nav div.wy-side-nav-search { + background-color: #eee; +} +nav div.wy-side-scroll { + background-color: var(--secondary-1-4); +} +nav .wy-menu-vertical a:hover { + background-color:var(--primary-0); + color:var(--primary-4); +} +nav .wy-menu-vertical li.current ul a:hover { + background-color:var(--secondary-1-2); + color:var(--primary-4); +} +nav .wy-menu-vertical li.current ul a { + background-color:var(--secondary-1-1); + color:var(--primary-3); +} +nav .wy-menu-vertical li.on a:hover, +nav .wy-menu-vertical li.current>a:hover { + background-color:#fcfcfc; +} +.wy-side-nav-search input[type=text] { + border-color:var(--primary-2); +} +.wy-menu-vertical li.toctree-l1.current>a { + border-top:1px solid var(--secondary-1-3); + border-bottom:1px solid var(--secondary-1-3); +} +.wy-menu-vertical li.toctree-l2.current>a { + background-color:var(--secondary-1-2); +} +.wy-menu-vertical li.toctree-l2.current li.toctree-l3>a { + background-color:var(--secondary-1-9); +} + +.wy-nav-content { + padding: 25pt 40pt; +} +div[role=navigation] > hr { + display:none; +} +div[role=navigation] { + margin-bottom:15pt; +} +h1 { + margin-left:-40pt; + margin-right:-40pt; + padding:5pt 40pt 5pt 40pt; +} + +.rst-content pre.literal-block, .rst-content div[class^='highlight'] { + border-color:var(--secondary-1-1); +} + +span.pre { + color: var(--complement-3); +} +pre { + background-color: var(--secondary-1-9); + border-color: var(--secondary-1-1); +} +.highlight .p { color: var(--secondary-2-3); } +.highlight .k { color: var(--secondary-2-0); } +.highlight .kt { color: var(--complement-0); } +.highlight .cm { color: var(--primary-3); } +.highlight .ow { color: var(--primary-3); } +.highlight .na { color: var(--primary-2); } +.highlight .nv { color: var(--complement-0); } + +strong { + font-weight:500; +} +.rst-content dl:not(.docutils) dt { + font-family:Fira Mono; + font-weight:600; + background-color:var(--secondary-2-9); + color:var(--secondary-2-3); + border-top:2px solid var(--secondary-2-2); +} +dt code.descname { + color: var(--secondary-2-4); +} + +@media (min-width: 1200px) { + .container { width: auto; } +} +@media (min-width: 992px) { + .container { width: auto; } +} +@media (min-width: 768px) { + .container { width: auto; } +} diff --git a/doc/user/_static/overrides.js b/doc/user/_static/overrides.js new file mode 100644 index 0000000..73bf612 --- /dev/null +++ b/doc/user/_static/overrides.js @@ -0,0 +1,13 @@ +/* special styling for the protocols vs. OS table in overview.rst + * + * unfortunately this can't be done in straight CSS because we're changing + * the styling on the parent. + */ +$(document).ready(function() { + $("span.mark:contains('Y')" ).addClass("mark-y" ).parent("td").addClass("mark"); + $("span.mark:contains('≥')" ).addClass("mark-geq").parent("td").addClass("mark"); + $("span.mark:contains('N')" ).addClass("mark-n" ).parent("td").addClass("mark"); + $("span.mark:contains('CP')").addClass("mark-cp" ).parent("td").addClass("mark"); + $("span.mark:contains('†')" ).addClass("mark-dag").parent("td").addClass("mark"); + $('td.mark').parents('table').addClass("mark").children('colgroup').remove(); +}); diff --git a/doc/user/affinitymap.rst b/doc/user/affinitymap.rst new file mode 100644 index 0000000..b7dcb14 --- /dev/null +++ b/doc/user/affinitymap.rst @@ -0,0 +1,32 @@ +.. _affinity-map: + +************* +Affinity Maps +************* + +Affinity maps provide a means of configuring Standard Admininistrative-Group +(RFC3630, RFC5305 and RFC5329) and Extended Admininistrative-Group (RFC7308). +An affinity-map maps a specific bit position to a human readable-name. + +An affinity refers to a color or a ressource class in the Traffic Engineering +terminology. The bit position means the position of the bit set starting from +the least significant bit. For example, if the affinity 'blue' has bit position +0 the extended Admin-Group value will be 0x01. If the affinity 'red' bit +position 2 was added to a link in combination with the 'blue' affinity, the +Admin-Group value would be 0x05. + +Command +------- + +.. clicmd:: affinity-map NAME bit-position (0-1023) + + Map the affinity name NAME to the bit-position. The bit-position is the key + so that only one name can be mapped to particular bit-position. + +.. clicmd:: no affinity-map NAME + + Remove the affinity-map mapping. + +Affinity-maps with a bit-position value higher than 31 are not compatible with +Standard Admininistrative-Group. The CLI disallow the usage of such +affinity-maps when Standard Admininistrative-Groups are required.
\ No newline at end of file diff --git a/doc/user/babeld.rst b/doc/user/babeld.rst new file mode 100644 index 0000000..bda0045 --- /dev/null +++ b/doc/user/babeld.rst @@ -0,0 +1,273 @@ +.. _babel: + +***** +Babel +***** + +Babel is an interior gateway protocol that is suitable both for wired networks +and for wireless mesh networks. Babel has been described as 'RIP on speed' -- +it is based on the same principles as RIP, but includes a number of refinements +that make it react much faster to topology changes without ever counting to +infinity, and allow it to perform reliable link quality estimation on wireless +links. Babel is a double-stack routing protocol, meaning that a single Babel +instance is able to perform routing for both IPv4 and IPv6. + +FRR implements Babel as described in :rfc:`6126`. + +.. _configuring-babeld: + +Configuring babeld +================== + +The *babeld* daemon can be invoked with any of the common +options (:ref:`common-invocation-options`). + +The *zebra* daemon must be running before *babeld* is +invoked. Also, if *zebra* is restarted then *babeld* +must be too. + +Configuration of *babeld* is done in its configuration file +:file:`babeld.conf`. + +.. _babel-configuration: + +Babel configuration +=================== + +.. clicmd:: router babel + + Enable or disable Babel routing. + +.. clicmd:: babel diversity + + Enable or disable routing using radio frequency diversity. This is + highly recommended in networks with many wireless nodes. + If you enable this, you will probably want to set `babel + diversity-factor` and `babel channel` below. + + +.. clicmd:: babel diversity-factor (1-256) + + Sets the multiplicative factor used for diversity routing, in units of + 1/256; lower values cause diversity to play a more important role in + route selection. The default it 256, which means that diversity plays + no role in route selection; you will probably want to set that to 128 + or less on nodes with multiple independent radios. + +.. clicmd:: network IFNAME + + Enable or disable Babel on the given interface. + + +.. clicmd:: babel <wired|wireless> + + Specifies whether this interface is wireless, which disables a number + of optimisations that are only correct on wired interfaces. + Specifying `wireless` (the default) is always correct, but may + cause slower convergence and extra routing traffic. + +.. clicmd:: babel split-horizon + + Specifies whether to perform split-horizon on the interface. Specifying + ``no babel split-horizon`` is always correct, while ``babel + split-horizon`` is an optimisation that should only be used on symmetric + and transitive (wired) networks. The default is ``babel split-horizon`` + on wired interfaces, and ``no babel split-horizon`` on wireless + interfaces. This flag is reset when the wired/wireless status of an + interface is changed. + + +.. clicmd:: babel hello-interval (20-655340) + + Specifies the time in milliseconds between two scheduled hellos. On + wired links, Babel notices a link failure within two hello intervals; + on wireless links, the link quality value is reestimated at every + hello interval. The default is 4000 ms. + + +.. clicmd:: babel update-interval (20-655340) + + Specifies the time in milliseconds between two scheduled updates. Since + Babel makes extensive use of triggered updates, this can be set to fairly + high values on links with little packet loss. The default is 20000 ms. + + +.. clicmd:: babel channel (1-254) +.. clicmd:: babel channel interfering +.. clicmd:: babel channel noninterfering + + Set the channel number that diversity routing uses for this interface (see + `babel diversity` above). Noninterfering interfaces are assumed to only + interfere with themselves, interfering interfaces are assumed to interfere + with all other channels except noninterfering channels, and interfaces with + a channel number interfere with interfering interfaces and interfaces with + the same channel number. The default is ``babel channel interfering`` for + wireless interfaces, and ``babel channel noninterfering`` for wired + interfaces. This is reset when the wired/wireless status of an interface is + changed. + + +.. clicmd:: babel rxcost (1-65534) + + Specifies the base receive cost for this interface. For wireless + interfaces, it specifies the multiplier used for computing the ETX + reception cost (default 256); for wired interfaces, it specifies the + cost that will be advertised to neighbours. This value is reset when + the wired/wireless attribute of the interface is changed. + +.. note:: + Do not use this command unless you know what you are doing; in most + networks, acting directly on the cost using route maps is a better + technique. + + +.. clicmd:: babel rtt-decay (1-256) + + This specifies the decay factor for the exponential moving average of + RTT samples, in units of 1/256. Higher values discard old samples + faster. The default is 42. + + +.. clicmd:: babel rtt-min (1-65535) + + This specifies the minimum RTT, in milliseconds, starting from which we + increase the cost to a neighbour. The additional cost is linear in + (rtt - rtt-min). The default is 10 ms. + + +.. clicmd:: babel rtt-max (1-65535) + + This specifies the maximum RTT, in milliseconds, above which we don't + increase the cost to a neighbour. The default is 120 ms. + + +.. clicmd:: babel max-rtt-penalty (0-65535) + + This specifies the maximum cost added to a neighbour because of RTT, i.e. + when the RTT is higher or equal than rtt-max. The default is 150. Setting it + to 0 effectively disables the use of a RTT-based cost. + + +.. clicmd:: babel enable-timestamps + + Enable or disable sending timestamps with each Hello and IHU message in + order to compute RTT values. The default is `no babel enable-timestamps`. + + +.. clicmd:: babel resend-delay (20-655340) + + Specifies the time in milliseconds after which an 'important' request or + update will be resent. The default is 2000 ms. You probably don't want to + tweak this value. + + +.. clicmd:: babel smoothing-half-life (0-65534) + + Specifies the time constant, in seconds, of the smoothing algorithm used for + implementing hysteresis. Larger values reduce route oscillation at the cost + of very slightly increasing convergence time. The value 0 disables + hysteresis, and is suitable for wired networks. The default is 4 s. + +.. _babel-redistribution: + +Babel redistribution +==================== + + +.. clicmd:: redistribute <ipv4|ipv6> KIND + + Specify which kind of routes should be redistributed into Babel. + +.. _show-babel-information: + +Show Babel information +====================== + +These commands dump various parts of *babeld*'s internal state. + + +.. clicmd:: show babel route + + +.. clicmd:: show babel route A.B.C.D + + +.. clicmd:: show babel route X:X::X:X + + +.. clicmd:: show babel route A.B.C.D/M + + +.. clicmd:: show babel route X:X::X:X/M + + +.. clicmd:: show babel interface + + +.. clicmd:: show babel interface IFNAME + + +.. clicmd:: show babel neighbor + + +.. clicmd:: show babel parameters + +Babel debugging commands +======================== + + simple: debug babel KIND + simple: no debug babel KIND + +.. clicmd:: debug babel KIND + + Enable or disable debugging messages of a given kind. ``KIND`` can + be one of: + + - ``common`` + - ``filter`` + - ``timeout`` + - ``interface`` + - ``route`` + - ``all`` + +.. note:: + If you have compiled with the ``NO_DEBUG`` flag, then these commands aren't + available. + + +Babel sample configuration file +=============================== + +.. code-block:: frr + + debug babel common + !debug babel kernel + !debug babel filter + !debug babel timeout + !debug babel interface + !debug babel route + !debug babel all + + router babel + ! network wlan0 + ! network eth0 + ! redistribute ipv4 kernel + ! no redistribute ipv6 static + + ! The defaults are fine for a wireless interface + + !interface wlan0 + + ! A few optimisation tweaks are optional but recommended on a wired interface + ! Disable link quality estimation, enable split horizon processing, and + ! increase the hello and update intervals. + + !interface eth0 + ! babel wired + ! babel split-horizon + ! babel hello-interval 12000 + ! babel update-interval 36000 + + ! log file /var/log/frr/babeld.log + log stdout + diff --git a/doc/user/basic.rst b/doc/user/basic.rst new file mode 100644 index 0000000..bbf24c5 --- /dev/null +++ b/doc/user/basic.rst @@ -0,0 +1,1065 @@ +.. _basic-commands: + +************** +Basic Commands +************** + +The following sections discuss commands common to all the routing daemons. + +.. _config-commands: + +Config Commands +=============== + + + + + +In a config file, you can write the debugging options, a vty's password, +routing daemon configurations, a log file name, and so forth. This information +forms the initial command set for a routing beast as it is starting. + +Config files are generally found in |INSTALL_PREFIX_ETC|. + +Config Methods +-------------- + +There are two ways of configuring FRR. + +Traditionally each of the daemons had its own config file. The daemon name plus +``.conf`` was the default config file name. For example, zebra's default config +file was :file:`zebra.conf`. This method is deprecated. + +Because of the amount of config files this creates, and the tendency of one +daemon to rely on others for certain functionality, most deployments now use +"integrated" configuration. In this setup all configuration goes into a single +file, typically :file:`/etc/frr/frr.conf`. When starting up FRR using an init +script or systemd, ``vtysh`` is invoked to read the config file and send the +appropriate portions to only the daemons interested in them. Running +configuration updates are persisted back to this single file using ``vtysh``. +This is the recommended method. To use this method, add the following line to +:file:`/etc/frr/vtysh.conf`: + +.. code-block:: frr + + service integrated-vtysh-config + +If you installed from source or used a package, this is probably already +present. + +If desired, you can specify a config file using the :option:`-f` or +:option:`--config_file` options when starting a daemon. + + +.. _basic-config-commands: + +Basic Config Commands +--------------------- + +.. clicmd:: hostname HOSTNAME + + Set hostname of the router. It is only for current ``vtysh``, it will not be + saved to any configuration file even with ``write file``. + +.. clicmd:: domainname DOMAINNAME + + Set domainname of the router. It is only for current ``vtysh``, it will not + be saved to any configuration file even with ``write file``. + +.. clicmd:: password PASSWORD + + Set password for vty interface. The ``no`` form of the command deletes the + password. If there is no password, a vty won't accept connections. + +.. clicmd:: enable password PASSWORD + + Set enable password. The ``no`` form of the command deletes the enable + password. + +.. clicmd:: service cputime-stats + + Collect CPU usage statistics for individual FRR event handlers and CLI + commands. This is enabled by default and can be disabled if the extra + overhead causes a noticeable slowdown on your system. + + Disabling these statistics will also make the + :clicmd:`service cputime-warning (1-4294967295)` limit non-functional. + +.. clicmd:: service cputime-warning (1-4294967295) + + Warn if the CPU usage of an event handler or CLI command exceeds the + specified limit (in milliseconds.) Such warnings are generally indicative + of some routine in FRR mistakenly blocking/hogging the processing loop and + should be reported as a FRR bug. + + This command has no effect if :clicmd:`service cputime-stats` is disabled. + +.. clicmd:: service walltime-warning (1-4294967295) + + Warn if the total wallclock time spent handling an event or executing a CLI + command exceeds the specified limit (in milliseconds.) This includes time + spent waiting for I/O or other tasks executing and may produce excessive + warnings if the system is overloaded. (This may still be useful to + provide an immediate sign that FRR is not operating correctly due to + externally caused starvation.) + +.. clicmd:: log trap LEVEL + + These commands are deprecated and are present only for historical + compatibility. The log trap command sets the current logging level for all + enabled logging destinations, and it sets the default for all future logging + commands that do not specify a level. The normal default logging level is + debugging. The ``no`` form of the command resets the default level for + future logging commands to debugging, but it does not change the logging + level of existing logging destinations. + + +.. clicmd:: log stdout LEVEL + + Enable logging output to stdout. If the optional second argument specifying + the logging level is not present, the default logging level (typically + debugging) will be used. The ``no`` form of the command disables logging to + stdout. The ``LEVEL`` argument must have one of these values: emergencies, + alerts, critical, errors, warnings, notifications, informational, or + debugging. Note that the existing code logs its most important messages with + severity ``errors``. + + .. note:: + + If ``systemd`` is in use and stdout is connected to systemd, FRR will + automatically switch to ``journald`` extended logging for this target. + + .. warning:: + + FRRouting uses the ``writev()`` system call to write log messages. This + call is supposed to be atomic, but in reality this does not hold for + pipes or terminals, only regular files. This means that in rare cases, + concurrent log messages from distinct threads may get jumbled in + terminal output. Use a log file and ``tail -f`` if this rare chance is + inacceptable to your setup. + +.. clicmd:: log file [FILENAME [LEVEL]] + + If you want to log into a file, please specify ``filename`` as + in this example: + + :: + + log file /var/log/frr/bgpd.log informational + + If the optional second argument specifying the logging level is not present, + the default logging level (typically debugging, but can be changed using the + deprecated ``log trap`` command) will be used. The ``no`` form of the command + disables logging to a file. + +.. clicmd:: log syslog [LEVEL] + + Enable logging output to syslog. If the optional second argument specifying + the logging level is not present, the default logging level (typically + debugging, but can be changed using the deprecated ``log trap`` command) will + be used. The ``no`` form of the command disables logging to syslog. + + .. note:: + + This uses the system's ``syslog()`` API, which does not support message + batching or structured key/value data pairs. If possible, use + :clicmd:`log extended EXTLOGNAME` with + :clicmd:`destination syslog [supports-rfc5424]` instead of this. + +.. clicmd:: log extended EXTLOGNAME + + Create an extended logging target with the specified name. The name has + no further meaning and is only used to identify the target. Multiple + targets can be created and deleted with the ``no`` form. + + Refer to :ref:`ext-log-target` for further details and suboptions. + +.. clicmd:: log monitor [LEVEL] + + This command is deprecated and does nothing. + +.. clicmd:: log facility [FACILITY] + + This command changes the facility used in syslog messages. The default + facility is ``daemon``. The ``no`` form of the command resets the facility + to the default ``daemon`` facility. + +.. clicmd:: log record-priority + + To include the severity in all messages logged to a file, to stdout, or to + a terminal monitor (i.e. anything except syslog), + use the ``log record-priority`` global configuration command. + To disable this option, use the ``no`` form of the command. By default, + the severity level is not included in logged messages. Note: some + versions of syslogd can be configured to include the facility and + level in the messages emitted. + +.. clicmd:: log timestamp precision [(0-6)] + + This command sets the precision of log message timestamps to the given + number of digits after the decimal point. Currently, the value must be in + the range 0 to 6 (i.e. the maximum precision is microseconds). To restore + the default behavior (1-second accuracy), use the ``no`` form of the + command, or set the precision explicitly to 0. + + :: + + log timestamp precision 3 + + In this example, the precision is set to provide timestamps with + millisecond accuracy. + +.. clicmd:: log commands + + This command enables the logging of all commands typed by a user to all + enabled log destinations. The note that logging includes full command lines, + including passwords. If the daemon startup option `--command-log-always` + is used to start the daemon then this command is turned on by default + and cannot be turned off and the [no] form of the command is dissallowed. + +.. clicmd:: log filtered-file [FILENAME [LEVEL]] + + Configure a destination file for filtered logs with the + :clicmd:`log filter-text WORD` command. + +.. clicmd:: log filter-text WORD + + This command forces logs to be filtered on a specific string. A log message + will only be printed if it matches on one of the filters in the log-filter + table. The filter only applies to file logging targets configured with + :clicmd:`log filtered-file [FILENAME [LEVEL]]`. + + .. note:: + + Log filters help when you need to turn on debugs that cause significant + load on the system (enabling certain debugs can bring FRR to a halt). + Log filters prevent this but you should still expect a small performance + hit due to filtering each of all those logs. + + .. note:: + + This setting is not saved to ``frr.conf`` and not shown in + :clicmd:`show running-config`. It is intended for ephemeral debugging + purposes only. + +.. clicmd:: clear log filter-text + + This command clears all current filters in the log-filter table. + + +.. clicmd:: log immediate-mode + + Use unbuffered output for log and debug messages; normally there is + some internal buffering. + +.. clicmd:: log unique-id + + Include ``[XXXXX-XXXXX]`` log message unique identifier in the textual part + of log messages. This is enabled by default, but can be disabled with + ``no log unique-id``. Please make sure the IDs are enabled when including + logs for FRR bug reports. + + The unique identifiers are automatically generated based on source code + file name, format string (before filling out) and severity. They do not + change "randomly", but some cleanup work may cause large chunks of ID + changes between releases. The IDs always start with a letter, consist of + letters and numbers (and a dash for readability), are case insensitive, and + ``I``, ``L``, ``O`` & ``U`` are excluded. + + This option will not affect future logging targets which allow putting the + unique identifier in auxiliary metadata outside the log message text + content. (No such logging target exists currently, but RFC5424 syslog and + systemd's journald both support it.) + +.. clicmd:: debug unique-id XXXXX-XXXXX backtrace + + Print backtraces (call stack) for specific log messages, identified by + their unique ID (see above.) Includes source code location and current + event handler being executed. On some systems you may need to install a + `debug symbols` package to get proper function names rather than raw code + pointers. + + This command can be issued inside and outside configuration mode, and is + saved to configuration only if it was given in configuration mode. + + .. warning:: + + Printing backtraces can significantly slow down logging calls and cause + log files to quickly balloon in size. Remember to disable backtraces + when they're no longer needed. + +.. clicmd:: debug routemap [detail] + + This command turns on debugging of routemaps. When detail is specified + more data is provided to the operator about the reasoning about what + is going on in the routemap code. + +.. clicmd:: service password-encryption + + Encrypt password. + +.. clicmd:: service advanced-vty + + Enable advanced mode VTY. + +.. clicmd:: service terminal-length (0-512) + + Set system wide line configuration. This configuration command applies to + all VTY interfaces. + +.. clicmd:: line vty + + Enter vty configuration mode. + +.. clicmd:: banner motd default + + Set default motd string. + +.. clicmd:: banner motd file FILE + + Set motd string from file. The file must be in directory specified + under ``--sysconfdir``. + +.. clicmd:: banner motd line LINE + + Set motd string from an input. + +.. clicmd:: exec-timeout MINUTE [SECOND] + + Set VTY connection timeout value. When only one argument is specified + it is used for timeout value in minutes. Optional second argument is + used for timeout value in seconds. Default timeout value is 10 minutes. + When timeout value is zero, it means no timeout. + + Not setting this, or setting the values to 0 0, means a timeout will not be + enabled. + +.. clicmd:: access-class ACCESS-LIST + + Restrict vty connections with an access list. + +.. clicmd:: allow-reserved-ranges + + Allow using IPv4 reserved (Class E) IP ranges for daemons. E.g.: setting + IPv4 addresses for interfaces or allowing reserved ranges in BGP next-hops. + + If you need multiple FRR instances (or FRR + any other daemon) running in a + single router and peering via 127.0.0.0/8, it's also possible to use this + knob if turned on. + + Default: off. + +.. _sample-config-file: + +Sample Config File +------------------ + +Below is a sample configuration file for the zebra daemon. + +.. code-block:: frr + + ! + ! Zebra configuration file + ! + frr version 6.0 + frr defaults traditional + ! + hostname Router + password zebra + enable password zebra + ! + log stdout + ! + ! + + +``!`` and ``#`` are comment characters. If the first character of the word is +one of the comment characters then from the rest of the line forward will be +ignored as a comment. + +.. code-block:: frr + + password zebra!password + +If a comment character is not the first character of the word, it's a normal +character. So in the above example ``!`` will not be regarded as a comment and +the password is set to ``zebra!password``. + + +Configuration versioning, profiles and upgrade behavior +------------------------------------------------------- + +All |PACKAGE_NAME| daemons share a mechanism to specify a configuration profile +and version for loading and saving configuration. Specific configuration +settings take different default values depending on the selected profile and +version. + +While the profile can be selected by user configuration and will remain over +upgrades, |PACKAGE_NAME| will always write configurations using its current +version. This means that, after upgrading, a ``write file`` may write out a +slightly different configuration than what was read in. + +Since the previous configuration is loaded with its version's defaults, but +the new configuration is written with the new defaults, any default that +changed between versions will result in an appropriate configuration entry +being written out. **FRRouting configuration is sticky, staying consistent +over upgrades.** Changed defaults will only affect new configuration. + +Note that the loaded version persists into interactive configuration +sessions. Commands executed in an interactive configuration session are +no different from configuration loaded at startup. This means that when, +say, you configure a new BGP peer, the defaults used for configuration +are the ones selected by the last ``frr version`` command. + +.. warning:: + + Saving the configuration does not bump the daemons forward to use the new + version for their defaults, but restarting them will, since they will then + apply the new ``frr version`` command that was written out. Manually + execute the ``frr version`` command in ``show running-config`` to avoid + this intermediate state. + +This is visible in ``show running-config``: + +.. code-block:: frr + + Current configuration: + ! + ! loaded from 6.0 + frr version 6.1-dev + frr defaults traditional + ! + +If you save and then restart with this configuration, the old defaults will +no longer apply. Similarly, you could execute ``frr version 6.1-dev``, causing +the new defaults to apply and the ``loaded from 6.0`` comment to disappear. + + +Profiles +^^^^^^^^ + +|PACKAGE_NAME| provides configuration profiles to adapt its default settings +to various usage scenarios. Currently, the following profiles are +implemented: + +* ``traditional`` - reflects defaults adhering mostly to IETF standards or + common practices in wide-area internet routing. +* ``datacenter`` - reflects a single administrative domain with intradomain + links using aggressive timers. + +Your distribution/installation may pre-set a profile through the ``-F`` command +line option on all daemons. All daemons must be configured for the same +profile. The value specified on the command line is only a pre-set and any +``frr defaults`` statement in the configuration will take precedence. + +.. note:: + + The profile must be the same across all daemons. Mismatches may result + in undefined behavior. + +You can freely switch between profiles without causing any interruption or +configuration changes. All settings remain at their previous values, and +``show running-configuration`` output will have new output listing the previous +default values as explicit configuration. New configuration, e.g. adding a +BGP peer, will use the new defaults. To apply the new defaults for existing +configuration, the previously-invisible old defaults that are now shown must +be removed from the configuration. + + +Upgrade practices for interactive configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you configure |PACKAGE_NAME| interactively and use the configuration +writing functionality to make changes persistent, the following +recommendations apply in regards to upgrades: + +1. Skipping major versions should generally work but is still inadvisable. + To avoid unneeded issue, upgrade one major version at a time and write + out the configuration after each update. + +2. After installing a new |PACKAGE_NAME| version, check the configuration + for differences against your old configuration. If any defaults changed + that affect your setup, lines may appear or disappear. If a new line + appears, it was previously the default (or not supported) and is now + necessary to retain previous behavior. If a line disappears, it + previously wasn't the default, but now is, so it is no longer necessary. + +3. Check the log files for deprecation warnings by using ``grep -i deprecat``. + +4. After completing each upgrade, save the configuration and either restart + |PACKAGE_NAME| or execute ``frr version <CURRENT>`` to ensure defaults of + the new version are fully applied. + + +Upgrade practices for autogenerated configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When using |PACKAGE_NAME| with generated configurations (e.g. Ansible, +Puppet, etc.), upgrade considerations differ somewhat: + +1. Always write out a ``frr version`` statement in the configurations you + generate. This ensures that defaults are applied consistently. + +2. Try to not run more distinct versions of |PACKAGE_NAME| than necessary. + Each version may need to be checked individually. If running a mix of + older and newer installations, use the oldest version for the + ``frr version`` statement. + +3. When rolling out upgrades, generate a configuration as usual with the old + version identifier and load it. Check for any differences or deprecation + warnings. If there are differences in the configuration, propagate these + back to the configuration generator to minimize relying on actual default + values. + +4. After the last installation of an old version is removed, change the + configuration generation to a newer ``frr version`` as appropriate. Perform + the same checks as when rolling out upgrades. + + +.. _terminal-mode-commands: + +Terminal Mode Commands +====================== + +.. clicmd:: write terminal + + Displays the current configuration to the vty interface. + +.. clicmd:: write file + + Write current configuration to configuration file. + +.. clicmd:: configure [terminal] + + Change to configuration mode. This command is the first step to + configuration. + +.. clicmd:: terminal length (0-512) + + Set terminal display length to ``(0-512)``. If length is 0, no display + control is performed. + +.. clicmd:: who + + Show a list of currently connected vty sessions. + +.. clicmd:: list + + List all available commands. + +.. clicmd:: show version + + Show the current version of |PACKAGE_NAME| and its build host information. + +.. clicmd:: show logging + + Shows the current configuration of the logging system. This includes the + status of all logging destinations. + +.. clicmd:: show log-filter + + Shows the current log filters applied to each daemon. + +.. clicmd:: show memory [DAEMON] + + Show information on how much memory is used for which specific things in + |PACKAGE_NAME|. Output may vary depending on system capabilities but will + generally look something like this: + + :: + + frr# show memory + System allocator statistics: + Total heap allocated: 1584 KiB + Holding block headers: 0 bytes + Used small blocks: 0 bytes + Used ordinary blocks: 1484 KiB + Free small blocks: 2096 bytes + Free ordinary blocks: 100 KiB + Ordinary blocks: 2 + Small blocks: 60 + Holding blocks: 0 + (see system documentation for 'mallinfo' for meaning) + --- qmem libfrr --- + Buffer : 3 24 72 + Buffer data : 1 4120 4120 + Host config : 3 (variably sized) 72 + Command Tokens : 3427 72 247160 + Command Token Text : 2555 (variably sized) 83720 + Command Token Help : 2555 (variably sized) 61720 + Command Argument : 2 (variably sized) 48 + Command Argument Name : 641 (variably sized) 15672 + [...] + --- qmem Label Manager --- + --- qmem zebra --- + ZEBRA VRF : 1 912 920 + Route Entry : 11 80 968 + Static route : 1 192 200 + RIB destination : 8 48 448 + RIB table info : 4 16 96 + Nexthop tracking object : 1 200 200 + Zebra Name Space : 1 312 312 + --- qmem Table Manager --- + + To understand system allocator statistics, refer to your system's + :manpage:`mallinfo(3)` man page. + + Below these statistics, statistics on individual memory allocation types + in |PACKAGE_NAME| (so-called `MTYPEs`) is printed: + + * the first column of numbers is the current count of allocations made for + the type (the number decreases when items are freed.) + * the second column is the size of each item. This is only available if + allocations on a type are always made with the same size. + * the third column is the total amount of memory allocated for the + particular type, including padding applied by malloc. This means that + the number may be larger than the first column multiplied by the second. + Overhead incurred by malloc's bookkeeping is not included in this, and + the column may be missing if system support is not available. + + When executing this command from ``vtysh``, each of the daemons' memory + usage is printed sequentially. You can specify the daemon's name to print + only its memory usage. + +.. clicmd:: show motd + + Show current motd banner. + +.. clicmd:: show history + + Dump the vtysh cli history. + +.. clicmd:: logmsg LEVEL MESSAGE + + Send a message to all logging destinations that are enabled for messages of + the given severity. + +.. clicmd:: find REGEX... + + This command performs a regex search across all defined commands in all + modes. As an example, suppose you're in enable mode and can't remember where + the command to turn OSPF segment routing on is: + + :: + + frr# find segment-routing on + (ospf) segment-routing on + (isis) segment-routing on + + + The CLI mode is displayed next to each command. In this example, + :clicmd:`segment-routing on` is under the `router ospf` mode. + + Similarly, suppose you want a listing of all commands that contain "l2vpn" + and "neighbor": + + :: + + frr# find l2vpn.*neighbor + (view) show [ip] bgp l2vpn evpn neighbors <A.B.C.D|X:X::X:X|WORD> advertised-routes [json] + (view) show [ip] bgp l2vpn evpn neighbors <A.B.C.D|X:X::X:X|WORD> routes [json] + (view) show [ip] bgp l2vpn evpn rd ASN:NN_OR_IP-ADDRESS:NN neighbors <A.B.C.D|X:X::X:X|WORD> advertised-routes [json] + (view) show [ip] bgp l2vpn evpn rd ASN:NN_OR_IP-ADDRESS:NN neighbors <A.B.C.D|X:X::X:X|WORD> routes [json] + ... + + + Note that when entering spaces as part of a regex specification, repeated + spaces will be compressed into a single space for matching purposes. This is + a consequence of spaces being used to delimit CLI tokens. If you need to + match more than one space, use the ``\s`` escape. + + POSIX Extended Regular Expressions are supported. + + +.. _common-show-commands: + +.. clicmd:: show thread cpu [r|w|t|e|x] + + This command displays system run statistics for all the different event + types. If no options is specified all different run types are displayed + together. Additionally you can ask to look at (r)ead, (w)rite, (t)imer, + (e)vent and e(x)ecute thread event types. + +.. clicmd:: show thread poll + + This command displays FRR's poll data. It allows a glimpse into how + we are setting each individual fd for the poll command at that point + in time. + +.. clicmd:: show thread timers + + This command displays FRR's timer data for timers that will pop in + the future. + +.. clicmd:: show yang operational-data XPATH [{format <json|xml>|translate TRANSLATOR|with-config}] DAEMON + + Display the YANG operational data starting from XPATH. The default + format is JSON, but can be displayed in XML as well. + + Normally YANG operational data are located inside containers marked + as `read-only`. + + Optionally it is also possible to display configuration leaves in + addition to operational data with the option `with-config`. This + option enables the display of configuration leaves with their + currently configured value (if the leaf is optional it will only show + if it was created or has a default value). + +.. _common-invocation-options: + +Common Invocation Options +========================= + +These options apply to all |PACKAGE_NAME| daemons. + + +.. option:: -d, --daemon + + Run in daemon mode. + +.. option:: -f, --config_file <file> + + Set configuration file name. + +.. option:: -h, --help + + Display this help and exit. + +.. option:: -i, --pid_file <file> + + Upon startup the process identifier of the daemon is written to a file, + typically in :file:`/var/run`. This file can be used by the init system + to implement commands such as ``.../init.d/zebra status``, + ``.../init.d/zebra restart`` or ``.../init.d/zebra stop``. + + The file name is an run-time option rather than a configure-time option so + that multiple routing daemons can be run simultaneously. This is useful when + using |PACKAGE_NAME| to implement a routing looking glass. One machine can + be used to collect differing routing views from differing points in the + network. + +.. option:: -A, --vty_addr <address> + + Set the VTY local address to bind to. If set, the VTY socket will only be + bound to this address. + +.. option:: -P, --vty_port <port> + + Set the VTY TCP port number. If set to 0 then the TCP VTY sockets will not + be opened. + +.. option:: -u <user> + + Set the user and group to run as. + +.. option:: -N <namespace> + + Set the namespace that the daemon will run in. A "/<namespace>" will + be added to all files that use the statedir. If you have "/var/run/frr" + as the default statedir then it will become "/var/run/frr/<namespace>". + +.. option:: -o, --vrfdefaultname <name> + + Set the name used for the *Default VRF* in CLI commands and YANG models. + This option must be the same for all running daemons. By default, the name + is "default". + + .. seealso:: :ref:`zebra-vrf` + +.. option:: -v, --version + + Print program version. + +.. option:: --command-log-always + + Cause the daemon to always log commands entered to the specified log file. + This also makes the `no log commands` command dissallowed. Enabling this + is suggested if you have need to track what the operator is doing on + this router. + +.. option:: --log <stdout|syslog|file:/path/to/log/file> + + When initializing the daemon, setup the log to go to either stdout, + syslog or to a file. These values will be displayed as part of + a show run. Additionally they can be overridden at runtime if + desired via the normal log commands. + +.. option:: --log-level <emergencies|alerts|critical|errors|warnings|notifications|informational|debugging> + + When initializing the daemon, allow the specification of a default + log level at startup from one of the specified levels. + +.. option:: --tcli + + Enable the transactional CLI mode. + +.. option:: --limit-fds <number> + + Limit the number of file descriptors that will be used internally + by the FRR daemons. By default, the daemons use the system ulimit + value. + +.. _loadable-module-support: + +Loadable Module Support +======================= + +FRR supports loading extension modules at startup. Loading, reloading or +unloading modules at runtime is not supported (yet). To load a module, use +the following command line option at daemon startup: + + +.. option:: -M, --module <module:options> + + Load the specified module, optionally passing options to it. If the module + name contains a slash (/), it is assumed to be a full pathname to a file to + be loaded. If it does not contain a slash, the |INSTALL_PREFIX_MODULES| + directory is searched for a module of the given name; first with the daemon + name prepended (e.g. ``zebra_mod`` for ``mod``), then without the daemon + name prepended. + + This option is available on all daemons, though some daemons may not have + any modules available to be loaded. + + +The SNMP Module +--------------- + +If SNMP is enabled during compile-time and installed as part of the package, +the ``snmp`` module can be loaded for the *Zebra*, *bgpd*, *ospfd*, *ospf6d* +and *ripd* daemons. + +The module ignores any options passed to it. Refer to :ref:`snmp-support` for +information on its usage. + + +The FPM Module +-------------- + +If FPM is enabled during compile-time and installed as part of the package, the +``fpm`` module can be loaded for the *zebra* daemon. This provides the +Forwarding Plane Manager ("FPM") API. + +The module expects its argument to be either ``Netlink`` or ``protobuf``, +specifying the encapsulation to use. ``Netlink`` is the default, and +``protobuf`` may not be available if the module was built without protobuf +support. Refer to :ref:`zebra-fib-push-interface` for more information. + + +.. _virtual-terminal-interfaces: + +Virtual Terminal Interfaces +=========================== + +VTY -- Virtual Terminal [aka TeletYpe] Interface is a command line +interface (CLI) for user interaction with the routing daemon. + + +.. _vty-overview: + +VTY Overview +------------ + +VTY stands for Virtual TeletYpe interface. It means you can connect to +the daemon via the telnet protocol. + +To enable a VTY interface, you have to setup a VTY password. If there +is no VTY password, one cannot connect to the VTY interface at all. + +:: + + % telnet localhost 2601 + Trying 127.0.0.1... + Connected to localhost. + Escape character is '^]'. + + Hello, this is |PACKAGE_NAME| (version |PACKAGE_VERSION|) + |COPYRIGHT_STR| + + User Access Verification + + Password: XXXXX + Router> ? + enable . . . Turn on privileged commands + exit . . . Exit current mode and down to previous mode + help . . . Description of the interactive help system + list . . . Print command list + show . . . Show system inform + + wh. . . Display who is on a vty + Router> enable + Password: XXXXX + Router# configure terminal + Router(config)# interface eth0 + Router(config-if)# ip address 10.0.0.1/8 + Router(config-if)# ^Z + Router# + + +.. _vty-modes: + +VTY Modes +--------- + +There are three basic VTY modes: + +There are commands that may be restricted to specific VTY modes. + +.. _vty-view-mode: + +VTY View Mode +^^^^^^^^^^^^^ + +This mode is for read-only access to the CLI. One may exit the mode by +leaving the system, or by entering `enable` mode. + +.. _vty-enable-mode: + +VTY Enable Mode +^^^^^^^^^^^^^^^ + +This mode is for read-write access to the CLI. One may exit the mode by +leaving the system, or by escaping to view mode. + +.. _vty-other-modes: + +VTY Other Modes +^^^^^^^^^^^^^^^ + +This page is for describing other modes. + +.. _vty-cli-commands: + +VTY CLI Commands +---------------- + +Commands that you may use at the command-line are described in the following +three subsubsections. + +.. _cli-movement-commands: + +CLI Movement Commands +^^^^^^^^^^^^^^^^^^^^^ + +These commands are used for moving the CLI cursor. The :kbd:`C` character +means press the Control Key. + +:kbd:`C-f` / :kbd:`LEFT` + Move forward one character. + +:kbd:`C-b` / :kbd:`RIGHT` + Move backward one character. + +:kbd:`M-f` + Move forward one word. + +:kbd:`M-b` + Move backward one word. + +:kbd:`C-a` + Move to the beginning of the line. + +:kbd:`C-e` + Move to the end of the line. + + +.. _cli-editing-commands: + +CLI Editing Commands +^^^^^^^^^^^^^^^^^^^^ + +These commands are used for editing text on a line. The :kbd:`C` +character means press the Control Key. + + +:kbd:`C-h` / :kbd:`DEL` + Delete the character before point. + + +:kbd:`C-d` + Delete the character after point. + + +:kbd:`M-d` + Forward kill word. + + +:kbd:`C-w` + Backward kill word. + + +:kbd:`C-k` + Kill to the end of the line. + + +:kbd:`C-u` + Kill line from the beginning, erasing input. + + +:kbd:`C-t` + Transpose character. + + +CLI Advanced Commands +^^^^^^^^^^^^^^^^^^^^^ + +There are several additional CLI commands for command line completions, +insta-help, and VTY session management. + + +:kbd:`C-c` + Interrupt current input and moves to the next line. + + +:kbd:`C-z` + End current configuration session and move to top node. + + +:kbd:`C-n` / :kbd:`DOWN` + Move down to next line in the history buffer. + + +:kbd:`C-p` / :kbd:`UP` + Move up to previous line in the history buffer. + + +:kbd:`TAB` + Use command line completion by typing :kbd:`TAB`. + + +:kbd:`?` + You can use command line help by typing ``help`` at the beginning of the + line. Typing :kbd:`?` at any point in the line will show possible + completions. + +Pipe Actions +^^^^^^^^^^^^ + +VTY supports optional modifiers at the end of commands that perform +postprocessing on command output or modify the action of commands. These do not +show up in the :kbd:`?` or :kbd:`TAB` suggestion lists. + +``... | include REGEX`` + Filters the output of the preceding command, including only lines which + match the POSIX Extended Regular Expression ``REGEX``. Do not put the regex + in quotes. + + Examples: + + :: + + frr# show ip bgp sum json | include remoteAs + "remoteAs":0, + "remoteAs":455, + "remoteAs":99, + + :: + + frr# show run | include neigh.*[0-9]{2}\.0\.[2-4]\.[0-9]* + neighbor 10.0.2.106 remote-as 99 + neighbor 10.0.2.107 remote-as 99 + neighbor 10.0.2.108 remote-as 99 + neighbor 10.0.2.109 remote-as 99 + neighbor 10.0.2.110 remote-as 99 + neighbor 10.0.3.111 remote-as 111 + diff --git a/doc/user/bfd.rst b/doc/user/bfd.rst new file mode 100644 index 0000000..6c57822 --- /dev/null +++ b/doc/user/bfd.rst @@ -0,0 +1,766 @@ +.. _bfd: + +********************************** +Bidirectional Forwarding Detection +********************************** + +:abbr:`BFD (Bidirectional Forwarding Detection)` stands for +Bidirectional Forwarding Detection and it is described and extended by +the following RFCs: + +* :rfc:`5880` +* :rfc:`5881` +* :rfc:`5882` +* :rfc:`5883` + +Currently, there are two implementations of the BFD commands in FRR: + +* :abbr:`PTM (Prescriptive Topology Manager)`: an external daemon which + implements BFD; +* ``bfdd``: a BFD implementation that is able to talk with remote peers; + +This document will focus on the later implementation: *bfdd*. + + +.. _bfd-starting: + +Starting BFD +============ + +*bfdd* default configuration file is :file:`bfdd.conf`. *bfdd* searches +the current directory first then |INSTALL_PREFIX_ETC|/bfdd.conf. All of +*bfdd*'s command must be configured in :file:`bfdd.conf`. + +*bfdd* specific invocation options are described below. Common options +may also be specified (:ref:`common-invocation-options`). + +.. program:: bfdd + +.. option:: --bfdctl <unix-socket> + + Set the BFD daemon control socket location. If using a non-default + socket location:: + + /usr/lib/frr/bfdd --bfdctl /tmp/bfdd.sock + + + The default UNIX socket location is: + + #define BFDD_CONTROL_SOCKET "|INSTALL_PREFIX_STATE|/bfdd.sock" + + This option overrides the location addition that the -N option provides + to the bfdd.sock + +.. option:: --dplaneaddr <type>:<address>[<:port>] + + Configure the distributed BFD data plane listening socket bind address. + + One would expect the data plane to run in the same machine as FRR, so + the suggested configuration would be: + + --dplaneaddr unix:/var/run/frr/bfdd_dplane.sock + + Or using IPv4: + + --dplaneaddr ipv4:127.0.0.1 + + Or using IPv6: + + --dplaneaddr ipv6:[::1] + + It is also possible to specify a port (for IPv4/IPv6 only): + + --dplaneaddr ipv6:[::1]:50701 + + (if ommited the default port is ``50700``). + + It is also possible to operate in client mode (instead of listening for + connections). To connect to a data plane server append the letter 'c' to + the protocol, example: + + --dplaneaddr ipv4c:127.0.0.1 + +.. note:: + + When using UNIX sockets don't forget to check the file permissions + before attempting to use it. + + +.. _bfd-commands: + +BFDd Commands +============= + +.. clicmd:: bfd + + Opens the BFD daemon configuration node. + +.. clicmd:: peer <A.B.C.D|X:X::X:X> [{multihop|local-address <A.B.C.D|X:X::X:X>|interface IFNAME|vrf NAME}] + + Creates and configures a new BFD peer to listen and talk to. + + `multihop` tells the BFD daemon that we should expect packets with + TTL less than 254 (because it will take more than one hop) and to + listen on the multihop port (4784). When using multi-hop mode + `echo-mode` will not work (see :rfc:`5883` section 3). + + `local-address` provides a local address that we should bind our + peer listener to and the address we should use to send the packets. + This option is mandatory for IPv6. + + `interface` selects which interface we should use. + + `vrf` selects which domain we want to use. + + +.. clicmd:: profile WORD + + Creates a peer profile that can be configured in multiple peers. + + Deleting the profile will cause all peers using it to reset to the default + values. + + +.. clicmd:: show bfd [vrf NAME] peers [json] + + Show all configured BFD peers information and current status. + +.. clicmd:: show bfd [vrf NAME] peer <WORD|<A.B.C.D|X:X::X:X> [{multihop|local-address <A.B.C.D|X:X::X:X>|interface IFNAME}]> [json] + + Show status for a specific BFD peer. + +.. clicmd:: show bfd [vrf NAME] peers brief [json] + + Show all configured BFD peers information and current status in brief. + +.. clicmd:: show bfd distributed + + Show the BFD data plane (distributed BFD) statistics. + + +.. _bfd-peer-config: + +Peer / Profile Configuration +---------------------------- + +BFD peers and profiles share the same BFD session configuration commands. + +.. clicmd:: detect-multiplier (2-255) + + Configures the detection multiplier to determine packet loss. The + remote transmission interval will be multiplied by this value to + determine the connection loss detection timer. The default value is + 3. + + Example: when the local system has `detect-multiplier 3` and the + remote system has `transmission interval 300`, the local system will + detect failures only after 900 milliseconds without receiving + packets. + +.. clicmd:: receive-interval (10-60000) + + Configures the minimum interval that this system is capable of + receiving control packets. The default value is 300 milliseconds. + +.. clicmd:: transmit-interval (10-60000) + + The minimum transmission interval (less jitter) that this system + wants to use to send BFD control packets. Defaults to 300ms. + +.. clicmd:: echo receive-interval <disabled|(10-60000)> + + Configures the minimum interval that this system is capable of + receiving echo packets. Disabled means that this system doesn't want + to receive echo packets. The default value is 50 milliseconds. + +.. clicmd:: echo transmit-interval (10-60000) + + The minimum transmission interval (less jitter) that this system + wants to use to send BFD echo packets. Defaults to 50ms. + +.. clicmd:: echo-mode + + Enables or disables the echo transmission mode. This mode is disabled + by default. If you are not using distributed BFD then echo mode works + only when the peer is also FRR. + + It is recommended that the transmission interval of control packets + to be increased after enabling echo-mode to reduce bandwidth usage. + For example: `transmit-interval 2000`. + + Echo mode is not supported on multi-hop setups (see :rfc:`5883` + section 3). + +.. clicmd:: shutdown + + Enables or disables the peer. When the peer is disabled an + 'administrative down' message is sent to the remote peer. + + +.. clicmd:: passive-mode + + Mark session as passive: a passive session will not attempt to start + the connection and will wait for control packets from peer before it + begins replying. + + This feature is useful when you have a router that acts as the + central node of a star network and you want to avoid sending BFD + control packets you don't need to. + + The default is active-mode (or ``no passive-mode``). + +.. clicmd:: minimum-ttl (1-254) + + For multi hop sessions only: configure the minimum expected TTL for + an incoming BFD control packet. + + This feature serves the purpose of thightening the packet validation + requirements to avoid receiving BFD control packets from other + sessions. + + The default value is 254 (which means we only expect one hop between + this system and the peer). + + +BFD Peer Specific Commands +-------------------------- + +.. clicmd:: profile BFDPROF + + Configure peer to use the profile configurations. + + Notes: + + - Profile configurations can be overridden on a peer basis by specifying + non-default parameters in peer configuration node. + - Non existing profiles can be configured and they will only be applied + once they start to exist. + - If the profile gets updated the new configuration will be applied to all + peers with the profile without interruptions. + + +.. _bfd-bgp-peer-config: + +BGP BFD Configuration +--------------------- + +The following commands are available inside the BGP configuration node. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> bfd + + Listen for BFD events registered on the same target as this BGP + neighbor. When BFD peer goes down it immediately asks BGP to shutdown + the connection with its neighbor and, when it goes back up, notify + BGP to try to connect to it. + + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> bfd check-control-plane-failure + + Allow to write CBIT independence in BFD outgoing packets. Also allow to + read both C-BIT value of BFD and lookup BGP peer status. This command is + useful when a BFD down event is caught, while the BGP peer requested that + local BGP keeps the remote BGP entries as staled if such issue is detected. + This is the case when graceful restart is enabled, and it is wished to + ignore the BD event while waiting for the remote router to restart. + + Disabling this disables presence of CBIT independence in BFD outgoing + packets and pays attention to BFD down notifications. This is the default. + + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> bfd profile BFDPROF + + Same as command ``neighbor <A.B.C.D|X:X::X:X|WORD> bfd``, but applies the + BFD profile to the sessions it creates or that already exist. + + +.. _bfd-isis-peer-config: + +IS-IS BFD Configuration +----------------------- + +The following commands are available inside the interface configuration node. + +.. clicmd:: isis bfd + + Listen for BFD events on peers created on the interface. Every time + a new neighbor is found a BFD peer is created to monitor the link + status for fast convergence. + + Note that there will be just one BFD session per interface. In case both + IPv4 and IPv6 support are configured then just a IPv6 based session is + created. + +.. clicmd:: isis bfd profile BFDPROF + + Use a BFD profile BFDPROF as provided in the BFD configuration. + + +.. _bfd-ospf-peer-config: + +OSPF BFD Configuration +---------------------- + +The following commands are available inside the interface configuration node. + +.. clicmd:: ip ospf bfd + + Listen for BFD events on peers created on the interface. Every time + a new neighbor is found a BFD peer is created to monitor the link + status for fast convergence. + +.. clicmd:: ip ospf bfd profile BFDPROF + + Same as command ``ip ospf bfd``, but applies the BFD profile to the sessions + it creates or that already exist. + + +.. _bfd-ospf6-peer-config: + +OSPF6 BFD Configuration +----------------------- + +The following commands are available inside the interface configuration node. + +.. clicmd:: ipv6 ospf6 bfd [profile BFDPROF] + + Listen for BFD events on peers created on the interface. Every time + a new neighbor is found a BFD peer is created to monitor the link + status for fast convergence. + + Optionally uses the BFD profile ``BFDPROF`` in the created sessions under + that interface. + + +.. _bfd-pim-peer-config: + +PIM BFD Configuration +--------------------- + +The following commands are available inside the interface configuration node. + +.. clicmd:: ip pim bfd [profile BFDPROF] + + Listen for BFD events on peers created on the interface. Every time + a new neighbor is found a BFD peer is created to monitor the link + status for fast convergence. + + Optionally uses the BFD profile ``BFDPROF`` in the created sessions under + that interface. + + +.. _bfd-rip-peer-config: + +RIP BFD configuration +--------------------- + +The following commands are available inside the interface configuration node: + +.. clicmd:: ip rip bfd + + Automatically create BFD session for each RIP peer discovered in this + interface. When the BFD session monitor signalize that the link is down + the RIP peer is removed and all the learned routes associated with that + peer are removed. + + +.. clicmd:: ip rip bfd profile BFD_PROFILE_NAME + + Selects a BFD profile for the BFD sessions created in this interface. + + +The following command is available in the RIP router configuration node: + +.. clicmd:: bfd default-profile BFD_PROFILE_NAME + + Selects a default BFD profile for all sessions without a profile specified. + + +.. _bfd-static-peer-config: + +BFD Static Route Monitoring Configuration +----------------------------------------- + +A monitored static route conditions the installation to the RIB on the +BFD session running state: when BFD session is up the route is installed +to RIB, but when the BFD session is down it is removed from the RIB. + +The following commands are available inside the configuration node: + +.. clicmd:: ip route A.B.C.D/M A.B.C.D bfd [{multi-hop|source A.B.C.D|profile BFDPROF}] + + Configure a static route for ``A.B.C.D/M`` using gateway ``A.B.C.D`` and use + the gateway address as BFD peer destination address. + +.. clicmd:: ipv6 route X:X::X:X/M [from X:X::X:X/M] X:X::X:X bfd [{multi-hop|source X:X::X:X|profile BFDPROF}] + + Configure a static route for ``X:X::X:X/M`` using gateway + ``X:X::X:X`` and use the gateway address as BFD peer destination + address. + +The static routes when uninstalled will no longer show up in the output of +the command ``show ip route`` or ``show ipv6 route``, instead we must use the +BFD static route show command to see these monitored route status. + +.. clicmd:: show bfd static route [json] + + Show all monitored static routes and their status. + + Example output: + + :: + + Showing BFD monitored static routes: + + Route groups: + rtg1 peer 172.16.0.1 (status: uninstalled): + 2001:db8::100/128 + + Next hops: + VRF default IPv4 Unicast: + 192.168.100.0/24 peer 172.16.0.1 (status: uninstalled) + + VRF default IPv4 Multicast: + + VRF default IPv6 Unicast: + +.. _bfd-configuration: + +Configuration +============= + +Before applying ``bfdd`` rules to integrated daemons (like BGPd), we must +create the corresponding peers inside the ``bfd`` configuration node. + +Here is an example of BFD configuration: + +:: + + bfd + peer 192.168.0.1 + no shutdown + ! + ! + router bgp 65530 + neighbor 192.168.0.1 remote-as 65531 + neighbor 192.168.0.1 bfd + neighbor 192.168.0.2 remote-as 65530 + neighbor 192.168.0.2 bfd + neighbor 192.168.0.3 remote-as 65532 + neighbor 192.168.0.3 bfd + ! + +Peers can be identified by its address (use ``multihop`` when you need +to specify a multi hop peer). + +Here are the available peer configurations: + +:: + + bfd + ! Configure a fast profile + profile fast + receive-interval 150 + transmit-interval 150 + ! + + ! Configure peer with fast profile + peer 192.168.0.6 + profile fast + no shutdown + ! + + ! Configure peer with fast profile and override receive speed. + peer 192.168.0.7 + profile fast + receive-interval 500 + no shutdown + ! + + ! configure a peer on an specific interface + peer 192.168.0.1 interface eth0 + no shutdown + ! + + ! configure a multihop peer + peer 192.168.0.2 multihop local-address 192.168.0.3 + shutdown + ! + + ! configure a peer in a different vrf + peer 192.168.0.3 vrf foo + shutdown + ! + + ! configure a peer with every option possible + peer 192.168.0.4 + detect-multiplier 50 + receive-interval 60000 + transmit-interval 3000 + shutdown + ! + + ! configure a peer on an interface from a separate vrf + peer 192.168.0.5 interface eth1 vrf vrf2 + no shutdown + ! + + ! remove a peer + no peer 192.168.0.3 vrf foo + + +.. _bfd-status: + +Status +====== + +You can inspect the current BFD peer status with the following commands: + +:: + + frr# show bfd peers + BFD Peers: + peer 192.168.0.1 + ID: 1 + Remote ID: 1 + Status: up + Uptime: 1 minute(s), 51 second(s) + Diagnostics: ok + Remote diagnostics: ok + Peer Type: dynamic + Local timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + Echo transmission interval: disabled + Remote timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + + peer 192.168.1.1 + ID: 2 + Remote ID: 2 + Status: up + Uptime: 1 minute(s), 53 second(s) + Diagnostics: ok + Remote diagnostics: ok + Peer Type: configured + Local timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + Echo transmission interval: disabled + Remote timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + + frr# show bfd peer 192.168.1.1 + BFD Peer: + peer 192.168.1.1 + ID: 2 + Remote ID: 2 + Status: up + Uptime: 3 minute(s), 4 second(s) + Diagnostics: ok + Remote diagnostics: ok + Peer Type: dynamic + Local timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + Echo transmission interval: disabled + Remote timers: + Detect-multiplier: 3 + Receive interval: 300ms + Transmission interval: 300ms + Echo receive interval: 50ms + + frr# show bfd peer 192.168.0.1 json + {"multihop":false,"peer":"192.168.0.1","id":1,"remote-id":1,"status":"up","uptime":161,"diagnostic":"ok","remote-diagnostic":"ok","receive-interval":300,"transmit-interval":300,"echo-receive-interval":50,"echo-transmit-interval":0,"detect-multiplier":3,"remote-receive-interval":300,"remote-transmit-interval":300,"remote-echo-receive-interval":50,"remote-detect-multiplier":3,"peer-type":"dynamic"} + +If you are running IPV4 BFD Echo, on a Linux platform, we also +calculate round trip time for the packets. We display minimum, +average and maximum time it took to receive the looped Echo packets +in the RTT fields. + +You can inspect the current BFD peer status in brief with the following commands: + +:: + + frr# show bfd peers brief + Session count: 1 + SessionId LocalAddress PeerAddress Status + ========= ============ =========== ====== + 1 192.168.0.1 192.168.0.2 up + + +You can also inspect peer session counters with the following commands: + +:: + + frr# show bfd peers counters + BFD Peers: + peer 192.168.2.1 interface r2-eth2 + Control packet input: 28 packets + Control packet output: 28 packets + Echo packet input: 0 packets + Echo packet output: 0 packets + Session up events: 1 + Session down events: 0 + Zebra notifications: 2 + + peer 192.168.0.1 + Control packet input: 54 packets + Control packet output: 103 packets + Echo packet input: 965 packets + Echo packet output: 966 packets + Session up events: 1 + Session down events: 0 + Zebra notifications: 4 + + frr# show bfd peer 192.168.0.1 counters + peer 192.168.0.1 + Control packet input: 126 packets + Control packet output: 247 packets + Echo packet input: 2409 packets + Echo packet output: 2410 packets + Session up events: 1 + Session down events: 0 + Zebra notifications: 4 + + frr# show bfd peer 192.168.0.1 counters json + {"multihop":false,"peer":"192.168.0.1","control-packet-input":348,"control-packet-output":685,"echo-packet-input":6815,"echo-packet-output":6816,"session-up":1,"session-down":0,"zebra-notifications":4} + +You can also clear packet counters per session with the following commands, only the packet counters will be reset: + +:: + + frr# clear bfd peers counters + + frr# show bfd peers counters + BFD Peers: + peer 192.168.2.1 interface r2-eth2 + Control packet input: 0 packets + Control packet output: 0 packets + Echo packet input: 0 packets + Echo packet output: 0 packets + Session up events: 1 + Session down events: 0 + Zebra notifications: 2 + + peer 192.168.0.1 + Control packet input: 0 packets + Control packet output: 0 packets + Echo packet input: 0 packets + Echo packet output: 0 packets + Session up events: 1 + Session down events: 0 + Zebra notifications: 4 + + +.. _bfd-distributed: + +Distributed BFD +=============== + +The distributed BFD is the separation of the BFD protocol control plane from +the data plane. FRR implements its own BFD data plane protocol so vendors can +study and include it in their own software/hardware without having to modify +the FRR source code. The protocol definitions can be found at +``bfdd/bfddp_packet.h`` header (or the installed +``/usr/include/frr/bfdd/bfddp_packet.h``). + +To use this feature the BFD daemon needs to be started using the command line +option :option:`--dplaneaddr`. When operating using this option the BFD daemon +will not attempt to establish BFD sessions, but it will offload all its work to +the data plane that is (or will be) connected. Data plane reconnection is also +supported. + +The BFD data plane will be responsible for: + +* Sending/receiving the BFD protocol control/echo packets + +* Notifying BFD sessions state changes + +* Keeping the number of packets/bytes received/transmitted per session + + +The FRR BFD daemon will be responsible for: + +* Adding/updating BFD session settings + +* Asking for BFD session counters + +* Redistributing the state changes to the integrated protocols (``bgpd``, + ``ospfd`` etc...) + + +BFD daemon will also keep record of data plane communication statistics with +the command :clicmd:`show bfd distributed`. + +Sample output: + +:: + + frr# show bfd distributed + Data plane + ========== + File descriptor: 16 + Input bytes: 1296 + Input bytes peak: 72 + Input messages: 42 + Input current usage: 0 + Output bytes: 568 + Output bytes peak: 136 + Output messages: 19 + Output full events: 0 + Output current usage: 0 + + +.. _bfd-debugging: + +Debugging +========= + +By default only informational, warning and errors messages are going to be +displayed. If you want to get debug messages and other diagnostics then make +sure you have `debugging` level enabled: + +:: + + config + log file /var/log/frr/frr.log debugging + log syslog debugging + +You may also fine tune the debug messages by selecting one or more of the +debug levels: + +.. clicmd:: debug bfd distributed + + Toggle BFD data plane (distributed BFD) debugging. + + Activates the following debug messages: + + * Data plane received / send messages + * Connection events + +.. clicmd:: debug bfd network + + Toggle network events: show messages about socket failures and unexpected + BFD messages that may not belong to registered peers. + +.. clicmd:: debug bfd peer + + Toggle peer event log messages: show messages about peer creation/removal + and state changes. + +.. clicmd:: debug bfd zebra + + Toggle zebra message events: show messages about interfaces, local + addresses, VRF and daemon peer registrations. diff --git a/doc/user/bgp.rst b/doc/user/bgp.rst new file mode 100644 index 0000000..5b8ec11 --- /dev/null +++ b/doc/user/bgp.rst @@ -0,0 +1,5277 @@ +.. _bgp: + +*** +BGP +*** + +:abbr:`BGP` stands for Border Gateway Protocol. The latest BGP version is 4. +BGP-4 is one of the Exterior Gateway Protocols and the de facto standard +interdomain routing protocol. BGP-4 is described in :rfc:`1771` and updated by +:rfc:`4271`. :rfc:`2858` adds multiprotocol support to BGP-4. + +.. _starting-bgp: + +Starting BGP +============ + +The default configuration file of *bgpd* is :file:`bgpd.conf`. *bgpd* searches +the current directory first, followed by |INSTALL_PREFIX_ETC|/bgpd.conf. All of +*bgpd*'s commands must be configured in :file:`bgpd.conf` when the integrated +config is not being used. + +*bgpd* specific invocation options are described below. Common options may also +be specified (:ref:`common-invocation-options`). + +.. program:: bgpd + +.. option:: -p, --bgp_port <port> + + Set the bgp protocol's port number. When port number is 0, that means do not + listen bgp port. + +.. option:: -l, --listenon + + Specify specific IP addresses for bgpd to listen on, rather than its default + of ``0.0.0.0`` / ``::``. This can be useful to constrain bgpd to an internal + address, or to run multiple bgpd processes on one host. Multiple addresses + can be specified. + + In the following example, bgpd is started listening for connections on the + addresses 100.0.1.2 and fd00::2:2. The options -d (runs in daemon mode) and + -f (uses specific configuration file) are also used in this example as we + are likely to run multiple bgpd instances, each one with different + configurations, when using -l option. + + Note that this option implies the --no_kernel option, and no learned routes will be installed into the linux kernel. + +.. code-block:: shell + + # /usr/lib/frr/bgpd -d -f /some-folder/bgpd.conf -l 100.0.1.2 -l fd00::2:2 + +.. option:: -n, --no_kernel + + Do not install learned routes into the linux kernel. This option is useful + for a route-reflector environment or if you are running multiple bgp + processes in the same namespace. This option is different than the --no_zebra + option in that a ZAPI connection is made. + + This option can also be toggled during runtime by using the + ``[no] bgp no-rib`` commands in VTY shell. + + Note that this option will persist after saving the configuration during + runtime, unless unset by the ``no bgp no-rib`` command in VTY shell prior to + a configuration write operation. + +.. option:: -S, --skip_runas + + Skip the normal process of checking capabilities and changing user and group + information. + +.. option:: -e, --ecmp + + Run BGP with a limited ecmp capability, that is different than what BGP + was compiled with. The value specified must be greater than 0 and less + than or equal to the MULTIPATH_NUM specified on compilation. + +.. option:: -Z, --no_zebra + + Do not communicate with zebra at all. This is different than the --no_kernel + option in that we do not even open a ZAPI connection to the zebra process. + +.. option:: -s, --socket_size + + When opening tcp connections to our peers, set the socket send buffer + size that the kernel will use for the peers socket. This option + is only really useful at a very large scale. Experimentation should + be done to see if this is helping or not at the scale you are running + at. + +.. option:: --v6-with-v4-nexthops + + Allow BGP to peer in the V6 afi, when the interface only has v4 addresses. + This allows bgp to install the v6 routes with a v6 nexthop that has the + v4 address encoded in the nexthop. Zebra's equivalent option currently + overrides the bgp setting. This setting is only really usable when + the operator has turned off communication to zebra and is running bgpd + as a complete standalone process. + +LABEL MANAGER +------------- + +.. option:: -I, --int_num + + Set zclient id. This is required when using Zebra label manager in proxy mode. + +.. _bgp-basic-concepts: + +Basic Concepts +============== + +.. _bgp-autonomous-systems: + +Autonomous Systems +------------------ + +From :rfc:`1930`: + + An AS is a connected group of one or more IP prefixes run by one or more + network operators which has a SINGLE and CLEARLY DEFINED routing policy. + +Each AS has an identifying number associated with it called an :abbr:`ASN +(Autonomous System Number)`. This is a two octet value ranging in value from 1 +to 65535. The AS numbers 64512 through 65535 are defined as private AS numbers. +Private AS numbers must not be advertised on the global Internet. + +The :abbr:`ASN (Autonomous System Number)` is one of the essential elements of +BGP. BGP is a distance vector routing protocol, and the AS-Path framework +provides distance vector metric and loop detection to BGP. + +.. seealso:: :rfc:`1930` + +.. _bgp-address-families: + +Address Families +---------------- + +Multiprotocol extensions enable BGP to carry routing information for multiple +network layer protocols. BGP supports an Address Family Identifier (AFI) for +IPv4 and IPv6. Support is also provided for multiple sets of per-AFI +information via the BGP Subsequent Address Family Identifier (SAFI). FRR +supports SAFIs for unicast information, labeled information (:rfc:`3107` and +:rfc:`8277`), and Layer 3 VPN information (:rfc:`4364` and :rfc:`4659`). + +.. _bgp-route-selection: + +Route Selection +--------------- + +The route selection process used by FRR's BGP implementation uses the following +decision criterion, starting at the top of the list and going towards the +bottom until one of the factors can be used. + +1. **Weight check** + + Prefer higher local weight routes to lower routes. + +2. **Local preference check** + + Prefer higher local preference routes to lower. + + If ``bgp bestpath aigp`` is enabled, and both paths that are compared have + AIGP attribute, BGP uses AIGP tie-breaking unless both of the paths have the + AIGP metric attribute. This means that the AIGP attribute is not evaluated + during the best path selection process between two paths when one path does + not have the AIGP attribute. + +3. **Local route check** + + Prefer local routes (statics, aggregates, redistributed) to received routes. + +4. **AS path length check** + + Prefer shortest hop-count AS_PATHs. + +5. **Origin check** + + Prefer the lowest origin type route. That is, prefer IGP origin routes to + EGP, to Incomplete routes. + +6. **MED check** + + Where routes with a MED were received from the same AS, prefer the route + with the lowest MED. :ref:`bgp-med`. + +7. **External check** + + Prefer the route received from an external, eBGP peer over routes received + from other types of peers. + +8. **IGP cost check** + + Prefer the route with the lower IGP cost. + +9. **Multi-path check** + + If multi-pathing is enabled, then check whether the routes not yet + distinguished in preference may be considered equal. If + :clicmd:`bgp bestpath as-path multipath-relax` is set, all such routes are + considered equal, otherwise routes received via iBGP with identical AS_PATHs + or routes received from eBGP neighbours in the same AS are considered equal. + +10. **Already-selected external check** + + Where both routes were received from eBGP peers, then prefer the route + which is already selected. Note that this check is not applied if + :clicmd:`bgp bestpath compare-routerid` is configured. This check can + prevent some cases of oscillation. + +11. **Router-ID check** + + Prefer the route with the lowest `router-ID`. If the route has an + `ORIGINATOR_ID` attribute, through iBGP reflection, then that router ID is + used, otherwise the `router-ID` of the peer the route was received from is + used. + +12. **Cluster-List length check** + + The route with the shortest cluster-list length is used. The cluster-list + reflects the iBGP reflection path the route has taken. + +13. **Peer address** + + Prefer the route received from the peer with the higher transport layer + address, as a last-resort tie-breaker. + +.. _bgp-capability-negotiation: + +Capability Negotiation +---------------------- + +When adding IPv6 routing information exchange feature to BGP. There were some +proposals. :abbr:`IETF (Internet Engineering Task Force)` +:abbr:`IDR (Inter Domain Routing)` adopted a proposal called Multiprotocol +Extension for BGP. The specification is described in :rfc:`2283`. The protocol +does not define new protocols. It defines new attributes to existing BGP. When +it is used exchanging IPv6 routing information it is called BGP-4+. When it is +used for exchanging multicast routing information it is called MBGP. + +*bgpd* supports Multiprotocol Extension for BGP. So if a remote peer supports +the protocol, *bgpd* can exchange IPv6 and/or multicast routing information. + +Traditional BGP did not have the feature to detect a remote peer's +capabilities, e.g. whether it can handle prefix types other than IPv4 unicast +routes. This was a big problem using Multiprotocol Extension for BGP in an +operational network. :rfc:`2842` adopted a feature called Capability +Negotiation. *bgpd* use this Capability Negotiation to detect the remote peer's +capabilities. If a peer is only configured as an IPv4 unicast neighbor, *bgpd* +does not send these Capability Negotiation packets (at least not unless other +optional BGP features require capability negotiation). + +By default, FRR will bring up peering with minimal common capability for the +both sides. For example, if the local router has unicast and multicast +capabilities and the remote router only has unicast capability the local router +will establish the connection with unicast only capability. When there are no +common capabilities, FRR sends Unsupported Capability error and then resets the +connection. + +.. _bgp-router-configuration: + +BGP Router Configuration +======================== + +ASN and Router ID +----------------- + +First of all you must configure BGP router with the :clicmd:`router bgp ASN` +command. The AS number is an identifier for the autonomous system. The AS +identifier can either be a number or two numbers separated by a period. The +BGP protocol uses the AS identifier for detecting whether the BGP connection is +internal or external. + +.. clicmd:: router bgp ASN + + Enable a BGP protocol process with the specified ASN. After + this statement you can input any `BGP Commands`. + +.. clicmd:: bgp router-id A.B.C.D + + This command specifies the router-ID. If *bgpd* connects to *zebra* it gets + interface and address information. In that case default router ID value is + selected as the largest IP Address of the interfaces. When `router zebra` is + not enabled *bgpd* can't get interface information so `router-id` is set to + 0.0.0.0. So please set router-id by hand. + + +.. _bgp-multiple-autonomous-systems: + +Multiple Autonomous Systems +--------------------------- + +FRR's BGP implementation is capable of running multiple autonomous systems at +once. Each configured AS corresponds to a :ref:`zebra-vrf`. In the past, to get +the same functionality the network administrator had to run a new *bgpd* +process; using VRFs allows multiple autonomous systems to be handled in a +single process. + +When using multiple autonomous systems, all router config blocks after the +first one must specify a VRF to be the target of BGP's route selection. This +VRF must be unique within respect to all other VRFs being used for the same +purpose, i.e. two different autonomous systems cannot use the same VRF. +However, the same AS can be used with different VRFs. + +.. note:: + + The separated nature of VRFs makes it possible to peer a single *bgpd* + process to itself, on one machine. Note that this can be done fully within + BGP without a corresponding VRF in the kernel or Zebra, which enables some + practical use cases such as :ref:`route reflectors <bgp-route-reflector>` + and route servers. + +Configuration of additional autonomous systems, or of a router that targets a +specific VRF, is accomplished with the following command: + +.. clicmd:: router bgp ASN vrf VRFNAME + + ``VRFNAME`` is matched against VRFs configured in the kernel. When ``vrf + VRFNAME`` is not specified, the BGP protocol process belongs to the default + VRF. + +An example configuration with multiple autonomous systems might look like this: + +.. code-block:: frr + + router bgp 1 + neighbor 10.0.0.1 remote-as 20 + neighbor 10.0.0.2 remote-as 30 + ! + router bgp 2 vrf blue + neighbor 10.0.0.3 remote-as 40 + neighbor 10.0.0.4 remote-as 50 + ! + router bgp 3 vrf red + neighbor 10.0.0.5 remote-as 60 + neighbor 10.0.0.6 remote-as 70 + ... + +.. seealso:: :ref:`bgp-vrf-route-leaking` +.. seealso:: :ref:`zebra-vrf` + + +.. _bgp-views: + +Views +----- + +In addition to supporting multiple autonomous systems, FRR's BGP implementation +also supports *views*. + +BGP views are almost the same as normal BGP processes, except that routes +selected by BGP are not installed into the kernel routing table. Each BGP view +provides an independent set of routing information which is only distributed +via BGP. Multiple views can be supported, and BGP view information is always +independent from other routing protocols and Zebra/kernel routes. BGP views use +the core instance (i.e., default VRF) for communication with peers. + +.. clicmd:: router bgp AS-NUMBER view NAME + + Make a new BGP view. You can use an arbitrary word for the ``NAME``. Routes + selected by the view are not installed into the kernel routing table. + + With this command, you can setup Route Server like below. + + .. code-block:: frr + + ! + router bgp 1 view 1 + neighbor 10.0.0.1 remote-as 2 + neighbor 10.0.0.2 remote-as 3 + ! + router bgp 2 view 2 + neighbor 10.0.0.3 remote-as 4 + neighbor 10.0.0.4 remote-as 5 + +.. clicmd:: show [ip] bgp view NAME + + Display the routing table of BGP view ``NAME``. + + +Route Selection +--------------- + +.. clicmd:: bgp bestpath as-path confed + + This command specifies that the length of confederation path sets and + sequences should should be taken into account during the BGP best path + decision process. + +.. clicmd:: bgp bestpath as-path multipath-relax + + This command specifies that BGP decision process should consider paths + of equal AS_PATH length candidates for multipath computation. Without + the knob, the entire AS_PATH must match for multipath computation. + +.. clicmd:: bgp bestpath compare-routerid + + Ensure that when comparing routes where both are equal on most metrics, + including local-pref, AS_PATH length, IGP cost, MED, that the tie is broken + based on router-ID. + + If this option is enabled, then the already-selected check, where + already selected eBGP routes are preferred, is skipped. + + If a route has an `ORIGINATOR_ID` attribute because it has been reflected, + that `ORIGINATOR_ID` will be used. Otherwise, the router-ID of the peer the + route was received from will be used. + + The advantage of this is that the route-selection (at this point) will be + more deterministic. The disadvantage is that a few or even one lowest-ID + router may attract all traffic to otherwise-equal paths because of this + check. It may increase the possibility of MED or IGP oscillation, unless + other measures were taken to avoid these. The exact behaviour will be + sensitive to the iBGP and reflection topology. + +.. clicmd:: bgp bestpath peer-type multipath-relax + + This command specifies that BGP decision process should consider paths + from all peers for multipath computation. If this option is enabled, + paths learned from any of eBGP, iBGP, or confederation neighbors will + be multipath if they are otherwise considered equal cost. + +.. clicmd:: bgp bestpath aigp + + Use the bgp bestpath aigp command to evaluate the AIGP attribute during + the best path selection process between two paths that have the AIGP + attribute. + + When bgp bestpath aigp is disabled, BGP does not use AIGP tie-breaking + rules unless paths have the AIGP attribute. + + Disabled by default. + +.. clicmd:: maximum-paths (1-128) + + Sets the maximum-paths value used for ecmp calculations for this + bgp instance in EBGP. The maximum value listed, 128, can be limited by + the ecmp cli for bgp or if the daemon was compiled with a lower + ecmp value. This value can also be set in ipv4/ipv6 unicast/labeled + unicast to only affect those particular afi/safi's. + +.. clicmd:: maximum-paths ibgp (1-128) [equal-cluster-length] + + Sets the maximum-paths value used for ecmp calculations for this + bgp instance in IBGP. The maximum value listed, 128, can be limited by + the ecmp cli for bgp or if the daemon was compiled with a lower + ecmp value. This value can also be set in ipv4/ipv6 unicast/labeled + unicast to only affect those particular afi/safi's. + +.. _bgp-distance: + +Administrative Distance Metrics +------------------------------- + +.. clicmd:: distance bgp (1-255) (1-255) (1-255) + + This command changes distance value of BGP. The arguments are the distance + values for external routes, internal routes and local routes + respectively. + +.. clicmd:: distance (1-255) A.B.C.D/M + +.. clicmd:: distance (1-255) A.B.C.D/M WORD + + Sets the administrative distance for a particular route. + + If the system has a static route configured from the kernel, it has a + distance of 0. In some cases, it might be useful to override the route + from the FRR. E.g.: Kernel has a statically configured default route, + and you received another default route from the BGP and want to install + it to be preferred over the static route. In such a case, you MUST set + a higher distance from the kernel. + + .. seealso:: :ref:`administrative-distance` + +.. _bgp-requires-policy: + +Require policy on EBGP +---------------------- + +.. clicmd:: bgp ebgp-requires-policy + + This command requires incoming and outgoing filters to be applied + for eBGP sessions as part of RFC-8212 compliance. Without the incoming + filter, no routes will be accepted. Without the outgoing filter, no + routes will be announced. + + This is enabled by default for the traditional configuration and + turned off by default for datacenter configuration. + + When you enable/disable this option you MUST clear the session. + + When the incoming or outgoing filter is missing you will see + "(Policy)" sign under ``show bgp summary``: + + .. code-block:: frr + + exit1# show bgp summary + + IPv4 Unicast Summary (VRF default): + BGP router identifier 10.10.10.1, local AS number 65001 vrf-id 0 + BGP table version 4 + RIB entries 7, using 1344 bytes of memory + Peers 2, using 43 KiB of memory + + Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc + 192.168.0.2 4 65002 8 10 0 0 0 00:03:09 5 (Policy) N/A + fe80:1::2222 4 65002 9 11 0 0 0 00:03:09 (Policy) (Policy) N/A + + Additionally a `show bgp neighbor` command would indicate in the `For address family:` + block that: + + .. code-block:: frr + + exit1# show bgp neighbor + ... + For address family: IPv4 Unicast + Update group 1, subgroup 1 + Packet Queue length 0 + Inbound soft reconfiguration allowed + Community attribute sent to this neighbor(all) + Inbound updates discarded due to missing policy + Outbound updates discarded due to missing policy + 0 accepted prefixes + +Reject routes with AS_SET or AS_CONFED_SET types +------------------------------------------------ + +.. clicmd:: bgp reject-as-sets + + This command enables rejection of incoming and outgoing routes having AS_SET or AS_CONFED_SET type. + +Suppress duplicate updates +-------------------------- + +.. clicmd:: bgp suppress-duplicates + + For example, BGP routers can generate multiple identical announcements with + empty community attributes if stripped at egress. This is an undesired behavior. + Suppress duplicate updates if the route actually not changed. + Default: enabled. + +Send Hard Reset CEASE Notification for Administrative Reset +----------------------------------------------------------- + +.. clicmd:: bgp hard-administrative-reset + + Send Hard Reset CEASE Notification for 'Administrative Reset' events. + + When disabled, and Graceful Restart Notification capability is exchanged + between the peers, Graceful Restart procedures apply, and routes will be + retained. + + Enabled by default. + +Disable checking if nexthop is connected on EBGP sessions +--------------------------------------------------------- + +.. clicmd:: bgp disable-ebgp-connected-route-check + + This command is used to disable the connection verification process for EBGP peering sessions + that are reachable by a single hop but are configured on a loopback interface or otherwise + configured with a non-directly connected IP address. + +.. _bgp-route-flap-dampening: + +Route Flap Dampening +-------------------- + +.. clicmd:: bgp dampening (1-45) (1-20000) (1-50000) (1-255) + + This command enables BGP route-flap dampening and specifies dampening parameters. + + half-life + Half-life time for the penalty + + reuse-threshold + Value to start reusing a route + + suppress-threshold + Value to start suppressing a route + + max-suppress + Maximum duration to suppress a stable route + + The route-flap damping algorithm is compatible with :rfc:`2439`. The use of + this command is not recommended nowadays. + + At the moment, route-flap dampening is not working per VRF and is working only + for IPv4 unicast and multicast. + +.. seealso:: + https://www.ripe.net/publications/docs/ripe-378 + +.. _bgp-med: + +Multi-Exit Discriminator +------------------------ + +The BGP :abbr:`MED (Multi-Exit Discriminator)` attribute has properties which +can cause subtle convergence problems in BGP. These properties and problems +have proven to be hard to understand, at least historically, and may still not +be widely understood. The following attempts to collect together and present +what is known about MED, to help operators and FRR users in designing and +configuring their networks. + +The BGP :abbr:`MED` attribute is intended to allow one AS to indicate its +preferences for its ingress points to another AS. The MED attribute will not be +propagated on to another AS by the receiving AS - it is 'non-transitive' in the +BGP sense. + +E.g., if AS X and AS Y have 2 different BGP peering points, then AS X might set +a MED of 100 on routes advertised at one and a MED of 200 at the other. When AS +Y selects between otherwise equal routes to or via AS X, AS Y should prefer to +take the path via the lower MED peering of 100 with AS X. Setting the MED +allows an AS to influence the routing taken to it within another, neighbouring +AS. + +In this use of MED it is not really meaningful to compare the MED value on +routes where the next AS on the paths differs. E.g., if AS Y also had a route +for some destination via AS Z in addition to the routes from AS X, and AS Z had +also set a MED, it wouldn't make sense for AS Y to compare AS Z's MED values to +those of AS X. The MED values have been set by different administrators, with +different frames of reference. + +The default behaviour of BGP therefore is to not compare MED values across +routes received from different neighbouring ASes. In FRR this is done by +comparing the neighbouring, left-most AS in the received AS_PATHs of the routes +and only comparing MED if those are the same. + +Unfortunately, this behaviour of MED, of sometimes being compared across routes +and sometimes not, depending on the properties of those other routes, means MED +can cause the order of preference over all the routes to be undefined. That is, +given routes A, B, and C, if A is preferred to B, and B is preferred to C, then +a well-defined order should mean the preference is transitive (in the sense of +orders [#med-transitivity-rant]_) and that A would be preferred to C. + +However, when MED is involved this need not be the case. With MED it is +possible that C is actually preferred over A. So A is preferred to B, B is +preferred to C, but C is preferred to A. This can be true even where BGP +defines a deterministic 'most preferred' route out of the full set of A,B,C. +With MED, for any given set of routes there may be a deterministically +preferred route, but there need not be any way to arrange them into any order +of preference. With unmodified MED, the order of preference of routes literally +becomes undefined. + +That MED can induce non-transitive preferences over routes can cause issues. +Firstly, it may be perceived to cause routing table churn locally at speakers; +secondly, and more seriously, it may cause routing instability in iBGP +topologies, where sets of speakers continually oscillate between different +paths. + +The first issue arises from how speakers often implement routing decisions. +Though BGP defines a selection process that will deterministically select the +same route as best at any given speaker, even with MED, that process requires +evaluating all routes together. For performance and ease of implementation +reasons, many implementations evaluate route preferences in a pair-wise fashion +instead. Given there is no well-defined order when MED is involved, the best +route that will be chosen becomes subject to implementation details, such as +the order the routes are stored in. That may be (locally) non-deterministic, +e.g.: it may be the order the routes were received in. + +This indeterminism may be considered undesirable, though it need not cause +problems. It may mean additional routing churn is perceived, as sometimes more +updates may be produced than at other times in reaction to some event . + +This first issue can be fixed with a more deterministic route selection that +ensures routes are ordered by the neighbouring AS during selection. +:clicmd:`bgp deterministic-med`. This may reduce the number of updates as routes +are received, and may in some cases reduce routing churn. Though, it could +equally deterministically produce the largest possible set of updates in +response to the most common sequence of received updates. + +A deterministic order of evaluation tends to imply an additional overhead of +sorting over any set of n routes to a destination. The implementation of +deterministic MED in FRR scales significantly worse than most sorting +algorithms at present, with the number of paths to a given destination. That +number is often low enough to not cause any issues, but where there are many +paths, the deterministic comparison may quickly become increasingly expensive +in terms of CPU. + +Deterministic local evaluation can *not* fix the second, more major, issue of +MED however. Which is that the non-transitive preference of routes MED can +cause may lead to routing instability or oscillation across multiple speakers +in iBGP topologies. This can occur with full-mesh iBGP, but is particularly +problematic in non-full-mesh iBGP topologies that further reduce the routing +information known to each speaker. This has primarily been documented with iBGP +:ref:`route-reflection <bgp-route-reflector>` topologies. However, any +route-hiding technologies potentially could also exacerbate oscillation with MED. + +This second issue occurs where speakers each have only a subset of routes, and +there are cycles in the preferences between different combinations of routes - +as the undefined order of preference of MED allows - and the routes are +distributed in a way that causes the BGP speakers to 'chase' those cycles. This +can occur even if all speakers use a deterministic order of evaluation in route +selection. + +E.g., speaker 4 in AS A might receive a route from speaker 2 in AS X, and from +speaker 3 in AS Y; while speaker 5 in AS A might receive that route from +speaker 1 in AS Y. AS Y might set a MED of 200 at speaker 1, and 100 at speaker +3. I.e, using ASN:ID:MED to label the speakers: + +:: + + . + /---------------\\ + X:2------|--A:4-------A:5--|-Y:1:200 + Y:3:100--|-/ | + \\---------------/ + + + +Assuming all other metrics are equal (AS_PATH, ORIGIN, 0 IGP costs), then based +on the RFC4271 decision process speaker 4 will choose X:2 over Y:3:100, based +on the lower ID of 2. Speaker 4 advertises X:2 to speaker 5. Speaker 5 will +continue to prefer Y:1:200 based on the ID, and advertise this to speaker 4. +Speaker 4 will now have the full set of routes, and the Y:1:200 it receives +from 5 will beat X:2, but when speaker 4 compares Y:1:200 to Y:3:100 the MED +check now becomes active as the ASes match, and now Y:3:100 is preferred. +Speaker 4 therefore now advertises Y:3:100 to 5, which will also agrees that +Y:3:100 is preferred to Y:1:200, and so withdraws the latter route from 4. +Speaker 4 now has only X:2 and Y:3:100, and X:2 beats Y:3:100, and so speaker 4 +implicitly updates its route to speaker 5 to X:2. Speaker 5 sees that Y:1:200 +beats X:2 based on the ID, and advertises Y:1:200 to speaker 4, and the cycle +continues. + +The root cause is the lack of a clear order of preference caused by how MED +sometimes is and sometimes is not compared, leading to this cycle in the +preferences between the routes: + +:: + + . + /---> X:2 ---beats---> Y:3:100 --\\ + | | + | | + \\---beats--- Y:1:200 <---beats---/ + + + +This particular type of oscillation in full-mesh iBGP topologies can be +avoided by speakers preferring already selected, external routes rather than +choosing to update to new a route based on a post-MED metric (e.g. router-ID), +at the cost of a non-deterministic selection process. FRR implements this, as +do many other implementations, so long as it is not overridden by setting +:clicmd:`bgp bestpath compare-routerid`, and see also +:ref:`bgp-route-selection`. + +However, more complex and insidious cycles of oscillation are possible with +iBGP route-reflection, which are not so easily avoided. These have been +documented in various places. See, e.g.: + +- [bgp-route-osci-cond]_ +- [stable-flexible-ibgp]_ +- [ibgp-correctness]_ + +for concrete examples and further references. + +There is as of this writing *no* known way to use MED for its original purpose; +*and* reduce routing information in iBGP topologies; *and* be sure to avoid the +instability problems of MED due the non-transitive routing preferences it can +induce; in general on arbitrary networks. + +There may be iBGP topology specific ways to reduce the instability risks, even +while using MED, e.g.: by constraining the reflection topology and by tuning +IGP costs between route-reflector clusters, see :rfc:`3345` for details. In the +near future, the Add-Path extension to BGP may also solve MED oscillation while +still allowing MED to be used as intended, by distributing "best-paths per +neighbour AS". This would be at the cost of distributing at least as many +routes to all speakers as a full-mesh iBGP would, if not more, while also +imposing similar CPU overheads as the "Deterministic MED" feature at each +Add-Path reflector. + +More generally, the instability problems that MED can introduce on more +complex, non-full-mesh, iBGP topologies may be avoided either by: + +- Setting :clicmd:`bgp always-compare-med`, however this allows MED to be compared + across values set by different neighbour ASes, which may not produce + coherent desirable results, of itself. +- Effectively ignoring MED by setting MED to the same value (e.g.: 0) using + :clicmd:`set metric METRIC` on all received routes, in combination with + setting :clicmd:`bgp always-compare-med` on all speakers. This is the simplest + and most performant way to avoid MED oscillation issues, where an AS is happy + not to allow neighbours to inject this problematic metric. + +As MED is evaluated after the AS_PATH length check, another possible use for +MED is for intra-AS steering of routes with equal AS_PATH length, as an +extension of the last case above. As MED is evaluated before IGP metric, this +can allow cold-potato routing to be implemented to send traffic to preferred +hand-offs with neighbours, rather than the closest hand-off according to the +IGP metric. + +Note that even if action is taken to address the MED non-transitivity issues, +other oscillations may still be possible. E.g., on IGP cost if iBGP and IGP +topologies are at cross-purposes with each other - see the Flavel and Roughan +paper above for an example. Hence the guideline that the iBGP topology should +follow the IGP topology. + +.. clicmd:: bgp deterministic-med + + Carry out route-selection in way that produces deterministic answers + locally, even in the face of MED and the lack of a well-defined order of + preference it can induce on routes. Without this option the preferred route + with MED may be determined largely by the order that routes were received + in. + + Setting this option will have a performance cost that may be noticeable when + there are many routes for each destination. Currently in FRR it is + implemented in a way that scales poorly as the number of routes per + destination increases. + + The default is that this option is not set. + +Note that there are other sources of indeterminism in the route selection +process, specifically, the preference for older and already selected routes +from eBGP peers, :ref:`bgp-route-selection`. + +.. clicmd:: bgp always-compare-med + + Always compare the MED on routes, even when they were received from + different neighbouring ASes. Setting this option makes the order of + preference of routes more defined, and should eliminate MED induced + oscillations. + + If using this option, it may also be desirable to use + :clicmd:`set metric METRIC` to set MED to 0 on routes received from external + neighbours. + + This option can be used, together with :clicmd:`set metric METRIC` to use + MED as an intra-AS metric to steer equal-length AS_PATH routes to, e.g., + desired exit points. + + +.. _bgp-graceful-restart: + +Graceful Restart +---------------- + +BGP graceful restart functionality as defined in +`RFC-4724 <https://tools.ietf.org/html/rfc4724/>`_ defines the mechanisms that +allows BGP speaker to continue to forward data packets along known routes +while the routing protocol information is being restored. + + +Usually, when BGP on a router restarts, all the BGP peers detect that the +session went down and then came up. This "down/up" transition results in a +"routing flap" and causes BGP route re-computation, generation of BGP routing +updates, and unnecessary churn to the forwarding tables. + +The following functionality is provided by graceful restart: + +1. The feature allows the restarting router to indicate to the helping peer the + routes it can preserve in its forwarding plane during control plane restart + by sending graceful restart capability in the OPEN message sent during + session establishment. Graceful restart notification flag and/or restart + time can also be changed during the dynamic BGP capabilities. If using + dynamic capabilities, no session reset is required, thus it's very useful + to increase restart time before doing a software upgrade or so. +2. The feature allows helping router to advertise to all other peers the routes + received from the restarting router which are preserved in the forwarding + plane of the restarting router during control plane restart. + + +:: + + + + (R1)-----------------------------------------------------------------(R2) + + 1. BGP Graceful Restart Capability exchanged between R1 & R2. + + <---------------------------------------------------------------------> + + 2. Kill BGP Process at R1. + + ----------------------------------------------------------------------> + + 3. R2 Detects the above BGP Restart & verifies BGP Restarting + Capability of R1. + + 4. Start BGP Process at R1. + + 5. Re-establish the BGP session between R1 & R2. + + <---------------------------------------------------------------------> + + 6. R2 Send initial route updates, followed by End-Of-Rib. + + <---------------------------------------------------------------------- + + 7. R1 was waiting for End-Of-Rib from R2 & which has been received + now. + + 8. R1 now runs BGP Best-Path algorithm. Send Initial BGP Update, + followed by End-Of Rib + + <---------------------------------------------------------------------> + + +.. _bgp-GR-preserve-forwarding-state: + +BGP-GR Preserve-Forwarding State +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +BGP OPEN message carrying optional capabilities for Graceful Restart has +8 bit “Flags for Address Family” for given AFI and SAFI. This field contains +bit flags relating to routes that were advertised with the given AFI and SAFI. + +.. code-block:: frr + + 0 1 2 3 4 5 6 7 + +-+-+-+-+-+-+-+-+ + |F| Reserved | + +-+-+-+-+-+-+-+-+ + +The most significant bit is defined as the Forwarding State (F) bit, which +can be used to indicate whether the forwarding state for routes that were +advertised with the given AFI and SAFI has indeed been preserved during the +previous BGP restart. When set (value 1), the bit indicates that the +forwarding state has been preserved. +The remaining bits are reserved and MUST be set to zero by the sender and +ignored by the receiver. + +.. clicmd:: bgp graceful-restart preserve-fw-state + +FRR gives us the option to enable/disable the "F" flag using this specific +vty command. However, it doesn't have the option to enable/disable +this flag only for specific AFI/SAFI i.e. when this command is used, it +applied to all the supported AFI/SAFI combinations for this peer. + +.. _bgp-end-of-rib-message: + +End-of-RIB (EOR) message +^^^^^^^^^^^^^^^^^^^^^^^^ + +An UPDATE message with no reachable Network Layer Reachability Information +(NLRI) and empty withdrawn NLRI is specified as the End-of-RIB marker that can +be used by a BGP speaker to indicate to its peer the completion of the initial +routing update after the session is established. + +For the IPv4 unicast address family, the End-of-RIB marker is an UPDATE message +with the minimum length. For any other address family, it is an UPDATE message +that contains only the MP_UNREACH_NLRI attribute with no withdrawn routes for +that <AFI, SAFI>. + +Although the End-of-RIB marker is specified for the purpose of BGP graceful +restart, it is noted that the generation of such a marker upon completion of +the initial update would be useful for routing convergence in general, and thus +the practice is recommended. + +.. _bgp-route-selection-deferral-timer: + +Route Selection Deferral Timer +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Specifies the time the restarting router defers the route selection process +after restart. + +Restarting Router : The usage of route election deferral timer is specified +in https://tools.ietf.org/html/rfc4724#section-4.1 + +Once the session between the Restarting Speaker and the Receiving Speaker is +re-established, the Restarting Speaker will receive and process BGP messages +from its peers. + +However, it MUST defer route selection for an address family until it either. + +1. Receives the End-of-RIB marker from all its peers (excluding the ones with + the "Restart State" bit set in the received capability and excluding the ones + that do not advertise the graceful restart capability). +2. The Selection_Deferral_Timer timeout. + +.. clicmd:: bgp graceful-restart select-defer-time (0-3600) + + This is command, will set deferral time to value specified. + + +.. clicmd:: bgp graceful-restart rib-stale-time (1-3600) + + This is command, will set the time for which stale routes are kept in RIB. + +.. clicmd:: bgp graceful-restart restart-time (0-4095) + + Set the time to wait to delete stale routes before a BGP open message + is received. + + Using with Long-lived Graceful Restart capability, this is recommended + setting this timer to 0 and control stale routes with + ``bgp long-lived-graceful-restart stale-time``. + + Default value is 120. + +.. clicmd:: bgp graceful-restart stalepath-time (1-4095) + + This is command, will set the max time (in seconds) to hold onto + restarting peer's stale paths. + + It also controls Enhanced Route-Refresh timer. + + If this command is configured and the router does not receive a Route-Refresh EoRR + message, the router removes the stale routes from the BGP table after the timer + expires. The stale path timer is started when the router receives a Route-Refresh + BoRR message. + +.. clicmd:: bgp graceful-restart notification + + Indicate Graceful Restart support for BGP NOTIFICATION messages. + + After changing this parameter, you have to reset the peers in order to advertise + N-bit in Graceful Restart capability. + + Without Graceful-Restart Notification capability (N-bit not set), GR is not + activated when receiving CEASE/HOLDTIME expire notifications. + + When sending ``CEASE/Administrative Reset`` (``clear bgp``), the session is closed + and routes are not retained. When N-bit is set and ``bgp hard-administrative-reset`` + is turned off Graceful-Restart is activated and routes are retained. + + Enabled by default. + +.. _bgp-per-peer-graceful-restart: + +BGP Per Peer Graceful Restart +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Ability to enable and disable graceful restart, helper and no GR at all mode +functionality at peer level. + +So bgp graceful restart can be enabled at modes global BGP level or at per +peer level. There are two FSM, one for BGP GR global mode and other for peer +per GR. + +Default global mode is helper and default peer per mode is inherit from global. +If per peer mode is configured, the GR mode of this particular peer will +override the global mode. + +.. _bgp-GR-global-mode-cmd: + +BGP GR Global Mode Commands +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: bgp graceful-restart + + This command will enable BGP graceful restart functionality at the global + level. + +.. clicmd:: bgp graceful-restart disable + + This command will disable both the functionality graceful restart and helper + mode. + + +.. _bgp-GR-peer-mode-cmd: + +BGP GR Peer Mode Commands +^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: neighbor A.B.C.D graceful-restart + + This command will enable BGP graceful restart functionality at the peer + level. + +.. clicmd:: neighbor A.B.C.D graceful-restart-helper + + This command will enable BGP graceful restart helper only functionality + at the peer level. + +.. clicmd:: neighbor A.B.C.D graceful-restart-disable + + This command will disable the entire BGP graceful restart functionality + at the peer level. + + +Long-lived Graceful Restart +--------------------------- + +Currently, only restarter mode is supported. This capability is advertised only +if graceful restart capability is negotiated. + +.. clicmd:: bgp long-lived-graceful-restart stale-time (1-16777215) + + Specifies the maximum time to wait before purging long-lived stale routes for + helper routers. + + Default is 0, which means the feature is off by default. Only graceful + restart takes into account. + +.. _bgp-shutdown: + +Administrative Shutdown +----------------------- + +.. clicmd:: bgp shutdown [message MSG...] + + Administrative shutdown of all peers of a bgp instance. Drop all BGP peers, + but preserve their configurations. The peers are notified in accordance with + `RFC 8203 <https://tools.ietf.org/html/rfc8203/>`_ by sending a + ``NOTIFICATION`` message with error code ``Cease`` and subcode + ``Administrative Shutdown`` prior to terminating connections. This global + shutdown is independent of the neighbor shutdown, meaning that individually + shut down peers will not be affected by lifting it. + + An optional shutdown message `MSG` can be specified. + + +.. _bgp-network: + +Networks +-------- + +.. clicmd:: network A.B.C.D/M + + This command adds the announcement network. + + .. code-block:: frr + + router bgp 1 + address-family ipv4 unicast + network 10.0.0.0/8 + exit-address-family + + This configuration example says that network 10.0.0.0/8 will be + announced to all neighbors. Some vendors' routers don't advertise + routes if they aren't present in their IGP routing tables; `bgpd` + doesn't care about IGP routes when announcing its routes. + + +.. clicmd:: bgp network import-check + + This configuration modifies the behavior of the network statement. + If you have this configured the underlying network must exist in + the rib. If you have the [no] form configured then BGP will not + check for the networks existence in the rib. For versions 7.3 and + before frr defaults for datacenter were the network must exist, + traditional did not check for existence. For versions 7.4 and beyond + both traditional and datacenter the network must exist. + +.. _bgp-ipv6-support: + +IPv6 Support +------------ + +.. clicmd:: neighbor A.B.C.D activate + + This configuration modifies whether to enable an address family for a + specific neighbor. By default only the IPv4 unicast address family is + enabled. + + .. code-block:: frr + + router bgp 1 + address-family ipv6 unicast + neighbor 2001:0DB8::1 activate + network 2001:0DB8:5009::/64 + exit-address-family + + This configuration example says that network 2001:0DB8:5009::/64 will be + announced and enables the neighbor 2001:0DB8::1 to receive this announcement. + + By default, only the IPv4 unicast address family is announced to all + neighbors. Using the 'no bgp default ipv4-unicast' configuration overrides + this default so that all address families need to be enabled explicitly. + + .. code-block:: frr + + router bgp 1 + no bgp default ipv4-unicast + neighbor 10.10.10.1 remote-as 2 + neighbor 2001:0DB8::1 remote-as 3 + address-family ipv4 unicast + neighbor 10.10.10.1 activate + network 192.168.1.0/24 + exit-address-family + address-family ipv6 unicast + neighbor 2001:0DB8::1 activate + network 2001:0DB8:5009::/64 + exit-address-family + + This configuration demonstrates how the 'no bgp default ipv4-unicast' might + be used in a setup with two upstreams where each of the upstreams should only + receive either IPv4 or IPv6 announcements. + + Using the ``bgp default ipv6-unicast`` configuration, IPv6 unicast + address family is enabled by default for all new neighbors. + + +.. _bgp-route-aggregation: + +Route Aggregation +----------------- + +.. _bgp-route-aggregation-ipv4: + +Route Aggregation-IPv4 Address Family +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: aggregate-address A.B.C.D/M + + This command specifies an aggregate address. + + In order to advertise an aggregated prefix, a more specific (longer) prefix + MUST exist in the BGP table. For example, if you want to create an + ``aggregate-address 10.0.0.0/24``, you should make sure you have something + like ``10.0.0.5/32`` or ``10.0.0.0/26``, or any other smaller prefix in the + BGP table. The routing information table (RIB) is not enough, you have to + redistribute them into the BGP table. + +.. clicmd:: aggregate-address A.B.C.D/M route-map NAME + + Apply a route-map for an aggregated prefix. + +.. clicmd:: aggregate-address A.B.C.D/M origin <egp|igp|incomplete> + + Override ORIGIN for an aggregated prefix. + +.. clicmd:: aggregate-address A.B.C.D/M as-set + + This command specifies an aggregate address. Resulting routes include + AS set. + +.. clicmd:: aggregate-address A.B.C.D/M summary-only + + This command specifies an aggregate address. + + Longer prefixes advertisements of more specific routes to all neighbors are suppressed. + +.. clicmd:: aggregate-address A.B.C.D/M matching-MED-only + + Configure the aggregated address to only be created when the routes MED + match, otherwise no aggregated route will be created. + +.. clicmd:: aggregate-address A.B.C.D/M suppress-map NAME + + Similar to `summary-only`, but will only suppress more specific routes that + are matched by the selected route-map. + + + This configuration example sets up an ``aggregate-address`` under the ipv4 + address-family. + + .. code-block:: frr + + router bgp 1 + address-family ipv4 unicast + aggregate-address 10.0.0.0/8 + aggregate-address 20.0.0.0/8 as-set + aggregate-address 40.0.0.0/8 summary-only + aggregate-address 50.0.0.0/8 route-map aggr-rmap + exit-address-family + + +.. _bgp-route-aggregation-ipv6: + +Route Aggregation-IPv6 Address Family +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: aggregate-address X:X::X:X/M + + This command specifies an aggregate address. + +.. clicmd:: aggregate-address X:X::X:X/M route-map NAME + + Apply a route-map for an aggregated prefix. + +.. clicmd:: aggregate-address X:X::X:X/M origin <egp|igp|incomplete> + + Override ORIGIN for an aggregated prefix. + +.. clicmd:: aggregate-address X:X::X:X/M as-set + + This command specifies an aggregate address. Resulting routes include + AS set. + +.. clicmd:: aggregate-address X:X::X:X/M summary-only + + This command specifies an aggregate address. + + Longer prefixes advertisements of more specific routes to all neighbors are suppressed + +.. clicmd:: aggregate-address X:X::X:X/M matching-MED-only + + Configure the aggregated address to only be created when the routes MED + match, otherwise no aggregated route will be created. + +.. clicmd:: aggregate-address X:X::X:X/M suppress-map NAME + + Similar to `summary-only`, but will only suppress more specific routes that + are matched by the selected route-map. + + + This configuration example sets up an ``aggregate-address`` under the ipv6 + address-family. + + .. code-block:: frr + + router bgp 1 + address-family ipv6 unicast + aggregate-address 10::0/64 + aggregate-address 20::0/64 as-set + aggregate-address 40::0/64 summary-only + aggregate-address 50::0/64 route-map aggr-rmap + exit-address-family + + +.. _bgp-redistribute-to-bgp: + +Redistribution +-------------- + +Redistribution configuration should be placed under the ``address-family`` +section for the specific AF to redistribute into. Protocol availability for +redistribution is determined by BGP AF; for example, you cannot redistribute +OSPFv3 into ``address-family ipv4 unicast`` as OSPFv3 supports IPv6. + +.. clicmd:: redistribute <babel|connected|eigrp|isis|kernel|openfabric|ospf|ospf6|rip|ripng|sharp|static|table> [metric (0-4294967295)] [route-map WORD] + +Redistribute routes from other protocols into BGP. + +.. clicmd:: redistribute vnc-direct + + Redistribute VNC direct (not via zebra) routes to BGP process. + +.. clicmd:: bgp update-delay MAX-DELAY + +.. clicmd:: bgp update-delay MAX-DELAY ESTABLISH-WAIT + + This feature is used to enable read-only mode on BGP process restart or when + a BGP process is cleared using 'clear ip bgp \*'. Note that this command is + configured at the global level and applies to all bgp instances/vrfs. It + cannot be used at the same time as the "update-delay" command described below, + which is entered in each bgp instance/vrf desired to delay update installation + and advertisements. The global and per-vrf approaches to defining update-delay + are mutually exclusive. + + When applicable, read-only mode would begin as soon as the first peer reaches + Established status and a timer for max-delay seconds is started. During this + mode BGP doesn't run any best-path or generate any updates to its peers. This + mode continues until: + + 1. All the configured peers, except the shutdown peers, have sent explicit EOR + (End-Of-RIB) or an implicit-EOR. The first keep-alive after BGP has reached + Established is considered an implicit-EOR. + If the establish-wait optional value is given, then BGP will wait for + peers to reach established from the beginning of the update-delay till the + establish-wait period is over, i.e. the minimum set of established peers for + which EOR is expected would be peers established during the establish-wait + window, not necessarily all the configured neighbors. + 2. max-delay period is over. + + On hitting any of the above two conditions, BGP resumes the decision process + and generates updates to its peers. + + Default max-delay is 0, i.e. the feature is off by default. + + +.. clicmd:: update-delay MAX-DELAY + +.. clicmd:: update-delay MAX-DELAY ESTABLISH-WAIT + + This feature is used to enable read-only mode on BGP process restart or when + a BGP process is cleared using 'clear ip bgp \*'. Note that this command is + configured under the specific bgp instance/vrf that the feature is enabled for. + It cannot be used at the same time as the global "bgp update-delay" described + above, which is entered at the global level and applies to all bgp instances. + The global and per-vrf approaches to defining update-delay are mutually + exclusive. + + When applicable, read-only mode would begin as soon as the first peer reaches + Established status and a timer for max-delay seconds is started. During this + mode BGP doesn't run any best-path or generate any updates to its peers. This + mode continues until: + + 1. All the configured peers, except the shutdown peers, have sent explicit EOR + (End-Of-RIB) or an implicit-EOR. The first keep-alive after BGP has reached + Established is considered an implicit-EOR. + If the establish-wait optional value is given, then BGP will wait for + peers to reach established from the beginning of the update-delay till the + establish-wait period is over, i.e. the minimum set of established peers for + which EOR is expected would be peers established during the establish-wait + window, not necessarily all the configured neighbors. + 2. max-delay period is over. + + On hitting any of the above two conditions, BGP resumes the decision process + and generates updates to its peers. + + Default max-delay is 0, i.e. the feature is off by default. + +.. clicmd:: table-map ROUTE-MAP-NAME + + This feature is used to apply a route-map on route updates from BGP to + Zebra. All the applicable match operations are allowed, such as match on + prefix, next-hop, communities, etc. Set operations for this attach-point are + limited to metric and next-hop only. Any operation of this feature does not + affect BGPs internal RIB. + + Supported for ipv4 and ipv6 address families. It works on multi-paths as + well, however, metric setting is based on the best-path only. + +.. _bgp-peers: + +Peers +----- + +.. _bgp-defining-peers: + +Defining Peers +^^^^^^^^^^^^^^ + +.. clicmd:: neighbor PEER remote-as ASN + + Creates a new neighbor whose remote-as is ASN. PEER can be an IPv4 address + or an IPv6 address or an interface to use for the connection. + + .. code-block:: frr + + router bgp 1 + neighbor 10.0.0.1 remote-as 2 + + In this case my router, in AS-1, is trying to peer with AS-2 at 10.0.0.1. + + This command must be the first command used when configuring a neighbor. If + the remote-as is not specified, *bgpd* will complain like this: :: + + can't find neighbor 10.0.0.1 + +.. clicmd:: neighbor PEER remote-as internal + + Create a peer as you would when you specify an ASN, except that if the + peers ASN is different than mine as specified under the :clicmd:`router bgp ASN` + command the connection will be denied. + +.. clicmd:: neighbor PEER remote-as external + + Create a peer as you would when you specify an ASN, except that if the + peers ASN is the same as mine as specified under the :clicmd:`router bgp ASN` + command the connection will be denied. + +.. clicmd:: bgp listen range <A.B.C.D/M|X:X::X:X/M> peer-group PGNAME + + Accept connections from any peers in the specified prefix. Configuration + from the specified peer-group is used to configure these peers. + +.. note:: + + When using BGP listen ranges, if the associated peer group has TCP MD5 + authentication configured, your kernel must support this on prefixes. On + Linux, this support was added in kernel version 4.14. If your kernel does + not support this feature you will get a warning in the log file, and the + listen range will only accept connections from peers without MD5 configured. + + Additionally, we have observed that when using this option at scale (several + hundred peers) the kernel may hit its option memory limit. In this situation + you will see error messages like: + + ``bgpd: sockopt_tcp_signature: setsockopt(23): Cannot allocate memory`` + + In this case you need to increase the value of the sysctl + ``net.core.optmem_max`` to allow the kernel to allocate the necessary option + memory. + +.. clicmd:: bgp listen limit <1-65535> + + Define the maximum number of peers accepted for one BGP instance. This + limit is set to 100 by default. Increasing this value will really be + possible if more file descriptors are available in the BGP process. This + value is defined by the underlying system (ulimit value), and can be + overridden by `--limit-fds`. More information is available in chapter + (:ref:`common-invocation-options`). + +.. clicmd:: coalesce-time (0-4294967295) + + The time in milliseconds that BGP will delay before deciding what peers + can be put into an update-group together in order to generate a single + update for them. The default time is 1000. + +.. _bgp-configuring-peers: + +Configuring Peers +^^^^^^^^^^^^^^^^^ + +.. clicmd:: neighbor PEER shutdown [message MSG...] [rtt (1-65535) [count (1-255)]] + + Shutdown the peer. We can delete the neighbor's configuration by + ``no neighbor PEER remote-as ASN`` but all configuration of the neighbor + will be deleted. When you want to preserve the configuration, but want to + drop the BGP peer, use this syntax. + + Optionally you can specify a shutdown message `MSG`. + + Also, you can specify optionally ``rtt`` in milliseconds to automatically + shutdown the peer if round-trip-time becomes higher than defined. + + Additional ``count`` parameter is the number of keepalive messages to count + before shutdown the peer if round-trip-time becomes higher than defined. + +.. clicmd:: neighbor PEER disable-connected-check + + Allow peerings between directly connected eBGP peers using loopback + addresses. + +.. clicmd:: neighbor PEER disable-link-bw-encoding-ieee + + By default bandwidth in extended communities is carried encoded as IEEE + floating-point format, which is according to the draft. + + Older versions have the implementation where extended community bandwidth + value is carried encoded as uint32. To enable backward compatibility we + need to disable IEEE floating-point encoding option per-peer. + +.. clicmd:: neighbor PEER enforce-first-as + + Discard updates received from the specified (eBGP) peer if the AS_PATH + attribute does not contain the PEER's ASN as the first AS_PATH segment. + + Default: disabled. + +.. clicmd:: neighbor PEER extended-optional-parameters + + Force Extended Optional Parameters Length format to be used for OPEN messages. + + By default, it's disabled. If the standard optional parameters length is + higher than one-octet (255), then extended format is enabled automatically. + + For testing purposes, extended format can be enabled with this command. + +.. clicmd:: neighbor PEER ebgp-multihop + + Specifying ``ebgp-multihop`` allows sessions with eBGP neighbors to + establish when they are multiple hops away. When the neighbor is not + directly connected and this knob is not enabled, the session will not + establish. + + If the peer's IP address is not in the RIB and is reachable via the + default route, then you have to enable ``ip nht resolve-via-default``. + +.. clicmd:: neighbor PEER description ... + + Set description of the peer. + +.. clicmd:: neighbor PEER interface IFNAME + + When you connect to a BGP peer over an IPv6 link-local address, you have to + specify the IFNAME of the interface used for the connection. To specify + IPv4 session addresses, see the ``neighbor PEER update-source`` command + below. + +.. clicmd:: neighbor PEER interface remote-as <internal|external|ASN> + + Configure an unnumbered BGP peer. ``PEER`` should be an interface name. The + session will be established via IPv6 link locals. Use ``internal`` for iBGP + and ``external`` for eBGP sessions, or specify an ASN if you wish. + +.. clicmd:: neighbor PEER next-hop-self [force] + + This command specifies an announced route's nexthop as being equivalent to + the address of the bgp router if it is learned via eBGP. This will also + bypass third-party next-hops in favor of the local bgp address. If the + optional keyword ``force`` is specified the modification is done also for + routes learned via iBGP. + +.. clicmd:: neighbor PEER attribute-unchanged [{as-path|next-hop|med}] + + This command specifies attributes to be left unchanged for advertisements + sent to a peer. Use this to leave the next-hop unchanged in ipv6 + configurations, as the route-map directive to leave the next-hop unchanged + is only available for ipv4. + +.. clicmd:: neighbor PEER update-source <IFNAME|ADDRESS> + + Specify the IPv4 source address to use for the :abbr:`BGP` session to this + neighbour, may be specified as either an IPv4 address directly or as an + interface name (in which case the *zebra* daemon MUST be running in order + for *bgpd* to be able to retrieve interface state). + + .. code-block:: frr + + router bgp 64555 + neighbor foo update-source 192.168.0.1 + neighbor bar update-source lo0 + + +.. clicmd:: neighbor PEER default-originate [route-map WORD] + + *bgpd*'s default is to not announce the default route (0.0.0.0/0) even if it + is in routing table. When you want to announce default routes to the peer, + use this command. + + If ``route-map`` keyword is specified, then the default route will be + originated only if route-map conditions are met. For example, announce + the default route only if ``10.10.10.10/32`` route exists and set an + arbitrary community for a default route. + + .. code-block:: frr + + router bgp 64555 + address-family ipv4 unicast + neighbor 192.168.255.1 default-originate route-map default + ! + ip prefix-list p1 seq 5 permit 10.10.10.10/32 + ! + route-map default permit 10 + match ip address prefix-list p1 + set community 123:123 + ! + +.. clicmd:: neighbor PEER port PORT + +.. clicmd:: neighbor PEER password PASSWORD + + Set a MD5 password to be used with the tcp socket that is being used + to connect to the remote peer. Please note if you are using this + command with a large number of peers on linux you should consider + modifying the `net.core.optmem_max` sysctl to a larger value to + avoid out of memory errors from the linux kernel. + +.. clicmd:: neighbor PEER send-community + +.. clicmd:: neighbor PEER weight WEIGHT + + This command specifies a default `weight` value for the neighbor's routes. + +.. clicmd:: neighbor PEER maximum-prefix NUMBER [force] + + Sets a maximum number of prefixes we can receive from a given peer. If this + number is exceeded, the BGP session will be destroyed. + + In practice, it is generally preferable to use a prefix-list to limit what + prefixes are received from the peer instead of using this knob. Tearing down + the BGP session when a limit is exceeded is far more destructive than merely + rejecting undesired prefixes. The prefix-list method is also much more + granular and offers much smarter matching criterion than number of received + prefixes, making it more suited to implementing policy. + + If ``force`` is set, then ALL prefixes are counted for maximum instead of + accepted only. This is useful for cases where an inbound filter is applied, + but you want maximum-prefix to act on ALL (including filtered) prefixes. This + option requires `soft-reconfiguration inbound` to be enabled for the peer. + +.. clicmd:: neighbor PEER maximum-prefix-out NUMBER + + Sets a maximum number of prefixes we can send to a given peer. + + Since sent prefix count is managed by update-groups, this option + creates a separate update-group for outgoing updates. + +.. clicmd:: neighbor PEER local-as AS-NUMBER [no-prepend] [replace-as] + + Specify an alternate AS for this BGP process when interacting with the + specified peer. With no modifiers, the specified local-as is prepended to + the received AS_PATH when receiving routing updates from the peer, and + prepended to the outgoing AS_PATH (after the process local AS) when + transmitting local routes to the peer. + + If the no-prepend attribute is specified, then the supplied local-as is not + prepended to the received AS_PATH. + + If the replace-as attribute is specified, then only the supplied local-as is + prepended to the AS_PATH when transmitting local-route updates to this peer. + + Note that replace-as can only be specified if no-prepend is. + + This command is only allowed for eBGP peers. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> as-override + + Override AS number of the originating router with the local AS number. + + Usually this configuration is used in PEs (Provider Edge) to replace + the incoming customer AS number so the connected CE (Customer Edge) + can use the same AS number as the other customer sites. This allows + customers of the provider network to use the same AS number across + their sites. + + This command is only allowed for eBGP peers. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> allowas-in [<(1-10)|origin>] + + Accept incoming routes with AS path containing AS number with the same value + as the current system AS. + + This is used when you want to use the same AS number in your sites, but you + can't connect them directly. This is an alternative to + `neighbor WORD as-override`. + + The parameter `(1-10)` configures the amount of accepted occurrences of the + system AS number in AS path. + + The parameter `origin` configures BGP to only accept routes originated with + the same AS number as the system. + + This command is only allowed for eBGP peers. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> addpath-tx-all-paths + + Configure BGP to send all known paths to neighbor in order to preserve multi + path capabilities inside a network. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> addpath-tx-bestpath-per-AS + + Configure BGP to send best known paths to neighbor in order to preserve multi + path capabilities inside a network. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> addpath-tx-best-selected (1-6) + + Configure BGP to calculate and send N best known paths to the neighbor. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> disable-addpath-rx + + Do not accept additional paths from this neighbor. + +.. clicmd:: neighbor PEER ttl-security hops NUMBER + + This command enforces Generalized TTL Security Mechanism (GTSM), as + specified in RFC 5082. With this command, only neighbors that are the + specified number of hops away will be allowed to become neighbors. This + command is mutually exclusive with *ebgp-multihop*. + +.. clicmd:: neighbor PEER capability extended-nexthop + + Allow bgp to negotiate the extended-nexthop capability with it's peer. + If you are peering over a v6 LL address then this capability is turned + on automatically. If you are peering over a v6 Global Address then + turning on this command will allow BGP to install v4 routes with + v6 nexthops if you do not have v4 configured on interfaces. + +.. clicmd:: neighbor PEER capability dynamic + + Allow BGP to negotiate the Dynamic Capability with its peers. + + Dynamic Capability defines a new BGP message (CAPABILITY) that can be used + to set/unset BGP capabilities without bringing down a BGP session. + + This includes changing graceful-restart (LLGR also) timers, + enabling/disabling add-path, and other supported capabilities. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> accept-own + + Enable handling of self-originated VPN routes containing ``accept-own`` community. + + This feature allows you to handle self-originated VPN routes, which a BGP speaker + receives from a route-reflector. A 'self-originated' route is one that was + originally advertised by the speaker itself. As per :rfc:`4271`, a BGP speaker rejects + advertisements that originated the speaker itself. However, the BGP ACCEPT_OWN + mechanism enables a router to accept the prefixes it has advertised, when reflected + from a route-reflector that modifies certain attributes of the prefix. + + A special community called ``accept-own`` is attached to the prefix by the + route-reflector, which is a signal to the receiving router to bypass the ORIGINATOR_ID + and NEXTHOP/MP_REACH_NLRI check. + + Default: disabled. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> path-attribute discard (1-255)... + + Drops specified path attributes from BGP UPDATE messages from the specified neighbor. + + If you do not want specific attributes, you can drop them using this command, and + let the BGP proceed by ignoring those attributes. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> path-attribute treat-as-withdraw (1-255)... + + Received BGP UPDATES that contain specified path attributes are treat-as-withdraw. If + there is an existing prefix in the BGP routing table, it will be removed. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> graceful-shutdown + + Mark all routes from this neighbor as less preferred by setting ``graceful-shutdown`` + community, and local-preference to 0. + +.. clicmd:: bgp fast-external-failover + + This command causes bgp to take down ebgp peers immediately + when a link flaps. `bgp fast-external-failover` is the default + and will not be displayed as part of a `show run`. The no form + of the command turns off this ability. + +.. clicmd:: bgp default-originate timer (0-3600) + + Set the period to rerun the default-originate route-map scanner process. The + default is 5 seconds. With a full routing table, it might be useful to increase + this setting to avoid scanning the whole BGP table aggressively. + +.. clicmd:: bgp default ipv4-unicast + + This command allows the user to specify that the IPv4 Unicast address + family is turned on by default or not. This command defaults to on + and is not displayed. + The `no bgp default ipv4-unicast` form of the command is displayed. + +.. clicmd:: bgp default ipv4-multicast + + This command allows the user to specify that the IPv4 Multicast address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv4-multicast` form of the command is displayed. + +.. clicmd:: bgp default ipv4-vpn + + This command allows the user to specify that the IPv4 MPLS VPN address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv4-vpn` form of the command is displayed. + +.. clicmd:: bgp default ipv4-flowspec + + This command allows the user to specify that the IPv4 Flowspec address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv4-flowspec` form of the command is displayed. + +.. clicmd:: bgp default ipv6-unicast + + This command allows the user to specify that the IPv6 Unicast address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv6-unicast` form of the command is displayed. + +.. clicmd:: bgp default ipv6-multicast + + This command allows the user to specify that the IPv6 Multicast address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv6-multicast` form of the command is displayed. + +.. clicmd:: bgp default ipv6-vpn + + This command allows the user to specify that the IPv6 MPLS VPN address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv6-vpn` form of the command is displayed. + +.. clicmd:: bgp default ipv6-flowspec + + This command allows the user to specify that the IPv6 Flowspec address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default ipv6-flowspec` form of the command is displayed. + +.. clicmd:: bgp default l2vpn-evpn + + This command allows the user to specify that the L2VPN EVPN address + family is turned on by default or not. This command defaults to off + and is not displayed. + The `bgp default l2vpn-evpn` form of the command is displayed. + +.. clicmd:: bgp default show-hostname + + This command shows the hostname of the peer in certain BGP commands + outputs. It's easier to troubleshoot if you have a number of BGP peers. + +.. clicmd:: bgp default show-nexthop-hostname + + This command shows the hostname of the next-hop in certain BGP commands + outputs. It's easier to troubleshoot if you have a number of BGP peers + and a number of routes to check. + +.. clicmd:: bgp default software-version-capability + + This command enables software version capability advertisement by default + for all the neighbors. + + For ``datacenter`` profile, this is enabled by default. + + .. code-block:: frr + + IPv4 Unicast Summary (VRF default): + BGP router identifier 10.0.0.6, local AS number 65001 vrf-id 0 + BGP table version 12 + RIB entries 23, using 4600 bytes of memory + Peers 3, using 2174 KiB of memory + + Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc + 10.0.0.4 4 65001 20 22 12 0 0 00:00:11 5 12 FRRouting/8.5.1 + 10.0.0.5 4 65001 21 22 12 0 0 00:00:11 5 12 FRRouting/9.0 + 192.168.67.7 4 65001 27 31 12 0 0 00:00:23 2 10 FRRouting/9.1-dev-MyOwnFRRVersion-g3c8c08dcd9 + + Total number of neighbors 3 + +.. clicmd:: neighbor PEER advertisement-interval (0-600) + + Setup the minimum route advertisement interval(mrai) for the + peer in question. This number is between 0 and 600 seconds, + with the default advertisement interval being 0. + +.. clicmd:: neighbor PEER timers (0-65535) (0-65535) + + Set keepalive and hold timers for a neighbor. The first value is keepalive + and the second is hold time. + +.. clicmd:: neighbor PEER timers connect (1-65535) + + Set connect timer for a neighbor. The connect timer controls how long BGP + waits between connection attempts to a neighbor. + +.. clicmd:: neighbor PEER timers delayopen (1-240) + + This command allows the user enable the + `RFC 4271 <https://tools.ietf.org/html/rfc4271/>` DelayOpenTimer with the + specified interval or disable it with the negating command for the peer. By + default, the DelayOpenTimer is disabled. The timer interval may be set to a + duration of 1 to 240 seconds. + +.. clicmd:: bgp minimum-holdtime (1-65535) + + This command allows user to prevent session establishment with BGP peers + with lower holdtime less than configured minimum holdtime. + When this command is not set, minimum holdtime does not work. + +.. clicmd:: bgp tcp-keepalive (1-65535) (1-65535) (1-30) + + This command allows user to configure TCP keepalive with new BGP peers. + Each parameter respectively stands for TCP keepalive idle timer (seconds), + interval (seconds), and maximum probes. By default, TCP keepalive is + disabled. + +Displaying Information about Peers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: show bgp <afi> <safi> neighbors WORD bestpath-routes [detail] [json] [wide] + + For the given neighbor, WORD, that is specified list the routes selected + by BGP as having the best path. + + If ``detail`` option is specified, the detailed version of all routes + will be displayed. The same format as ``show [ip] bgp [afi] [safi] PREFIX`` + will be used, but for the whole table of received, advertised or filtered + prefixes. + + If ``json`` option is specified, output is displayed in JSON format. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + +.. _bgp-peer-filtering: + +Peer Filtering +^^^^^^^^^^^^^^ + +.. clicmd:: neighbor PEER distribute-list NAME [in|out] + + This command specifies a distribute-list for the peer. `direct` is + ``in`` or ``out``. + +.. clicmd:: neighbor PEER prefix-list NAME [in|out] + +.. clicmd:: neighbor PEER filter-list NAME [in|out] + +.. clicmd:: neighbor PEER route-map NAME [in|out] + + Apply a route-map on the neighbor. `direct` must be `in` or `out`. + +.. clicmd:: bgp route-reflector allow-outbound-policy + + By default, attribute modification via route-map policy out is not reflected + on reflected routes. This option allows the modifications to be reflected as + well. Once enabled, it affects all reflected routes. + +.. clicmd:: neighbor PEER sender-as-path-loop-detection + + Enable the detection of sender side AS path loops and filter the + bad routes before they are sent. + + This setting is disabled by default. + +.. _bgp-peer-group: + +Peer Groups +^^^^^^^^^^^ + +Peer groups are used to help improve scaling by generating the same +update information to all members of a peer group. Note that this means +that the routes generated by a member of a peer group will be sent back +to that originating peer with the originator identifier attribute set to +indicated the originating peer. All peers not associated with a +specific peer group are treated as belonging to a default peer group, +and will share updates. + +.. clicmd:: neighbor WORD peer-group + + This command defines a new peer group. + +.. clicmd:: neighbor PEER peer-group PGNAME + + This command bind specific peer to peer group WORD. + +.. clicmd:: neighbor PEER solo + + This command is used to indicate that routes advertised by the peer + should not be reflected back to the peer. This command only is only + meaningful when there is a single peer defined in the peer-group. + +.. clicmd:: show [ip] bgp peer-group [json] + + This command displays configured BGP peer-groups. + + .. code-block:: frr + + exit1-debian-9# show bgp peer-group + + BGP peer-group test1, remote AS 65001 + Peer-group type is external + Configured address-families: IPv4 Unicast; IPv6 Unicast; + 1 IPv4 listen range(s) + 192.168.100.0/24 + 2 IPv6 listen range(s) + 2001:db8:1::/64 + 2001:db8:2::/64 + Peer-group members: + 192.168.200.1 Active + 2001:db8::1 Active + + BGP peer-group test2 + Peer-group type is external + Configured address-families: IPv4 Unicast; + + Optional ``json`` parameter is used to display JSON output. + + .. code-block:: frr + + { + "test1":{ + "remoteAs":65001, + "type":"external", + "addressFamiliesConfigured":[ + "IPv4 Unicast", + "IPv6 Unicast" + ], + "dynamicRanges":{ + "IPv4":{ + "count":1, + "ranges":[ + "192.168.100.0\/24" + ] + }, + "IPv6":{ + "count":2, + "ranges":[ + "2001:db8:1::\/64", + "2001:db8:2::\/64" + ] + } + }, + "members":{ + "192.168.200.1":{ + "status":"Active" + }, + "2001:db8::1":{ + "status":"Active" + } + } + }, + "test2":{ + "type":"external", + "addressFamiliesConfigured":[ + "IPv4 Unicast" + ] + } + } + +Capability Negotiation +^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: neighbor PEER strict-capability-match + + + Strictly compares remote capabilities and local capabilities. If + capabilities are different, send Unsupported Capability error then reset + connection. + + You may want to disable sending Capability Negotiation OPEN message optional + parameter to the peer when remote peer does not implement Capability + Negotiation. Please use *dont-capability-negotiate* command to disable the + feature. + +.. clicmd:: neighbor PEER dont-capability-negotiate + + Suppress sending Capability Negotiation as OPEN message optional parameter + to the peer. This command only affects the peer is configured other than + IPv4 unicast configuration. + + When remote peer does not have capability negotiation feature, remote peer + will not send any capabilities at all. In that case, bgp configures the peer + with configured capabilities. + + You may prefer locally configured capabilities more than the negotiated + capabilities even though remote peer sends capabilities. If the peer is + configured by *override-capability*, *bgpd* ignores received capabilities + then override negotiated capabilities with configured values. + + Additionally the operator should be reminded that this feature fundamentally + disables the ability to use widely deployed BGP features. BGP unnumbered, + hostname support, AS4, Addpath, Route Refresh, ORF, Dynamic Capabilities, + and graceful restart. + +.. clicmd:: neighbor PEER override-capability + + Override the result of Capability Negotiation with local configuration. + Ignore remote peer's capability value. + +.. clicmd:: neighbor PEER capability software-version + + Send the software version in the BGP OPEN message to the neighbor. This is + very useful in environments with a large amount of peers with different + versions of FRR or any other vendor. + + Disabled by default. + +.. clicmd:: neighbor PEER aigp + + Send and receive AIGP attribute for this neighbor. This is valid only for + eBGP neighbors. + + Disabled by default. iBGP neighbors have this option enabled implicitly. + +.. _bgp-as-path-access-lists: + +AS Path Access Lists +-------------------- + +AS path access list is user defined AS path. + +.. clicmd:: bgp as-path access-list WORD [seq (0-4294967295)] permit|deny LINE + + This command defines a new AS path access list. + +.. clicmd:: show bgp as-path-access-list [json] + + Display all BGP AS Path access lists. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show bgp as-path-access-list WORD [json] + + Display the specified BGP AS Path access list. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. _bgp-bogon-filter-example: + +Bogon ASN filter policy configuration example +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: frr + + bgp as-path access-list 99 permit _0_ + bgp as-path access-list 99 permit _23456_ + bgp as-path access-list 99 permit _1310[0-6][0-9]_|_13107[0-1]_ + bgp as-path access-list 99 seq 20 permit ^65 + +.. _bgp-using-as-path-in-route-map: + +Using AS Path in Route Map +-------------------------- + +.. clicmd:: match as-path WORD + + For a given as-path, WORD, match it on the BGP as-path given for the prefix + and if it matches do normal route-map actions. The no form of the command + removes this match from the route-map. + +.. clicmd:: set as-path prepend AS-PATH + + Prepend the given string of AS numbers to the AS_PATH of the BGP path's NLRI. + The no form of this command removes this set operation from the route-map. + +.. clicmd:: set as-path prepend last-as NUM + + Prepend the existing last AS number (the leftmost ASN) to the AS_PATH. + The no form of this command removes this set operation from the route-map. + +.. clicmd:: set as-path replace <any|ASN> [<ASN>] + + Replace a specific AS number to local AS number or a configured AS number. + ``any`` replaces each AS number in the AS-PATH with either the local AS + number or the configured AS number. + +.. clicmd:: set as-path replace as-path-access-list WORD [<ASN>] + + Replace some AS numbers from the AS_PATH of the BGP path's NLRI. Substituted + AS numbers are conformant with the regex defined in as-path access-list + WORD. Changed AS numbers are replaced either by the local AS number or the + configured AS number. + The no form of this command removes this set operation from the route-map. + +.. clicmd:: set as-path exclude all + + Remove all AS numbers from the AS_PATH of the BGP path's NLRI. The no form of + this command removes this set operation from the route-map. + +.. clicmd:: set as-path exclude as-path-access-list WORD + + Remove some AS numbers from the AS_PATH of the BGP path's NLRI. Removed AS + numbers are conformant with the regex defined in as-path access-list WORD. + The no form of this command removes this set operation from the route-map. + + +.. _bgp-communities-attribute: + +Communities Attribute +--------------------- + +The BGP communities attribute is widely used for implementing policy routing. +Network operators can manipulate BGP communities attribute based on their +network policy. BGP communities attribute is defined in :rfc:`1997` and +:rfc:`1998`. It is an optional transitive attribute, therefore local policy can +travel through different autonomous system. + +The communities attribute is a set of communities values. Each community value +is 4 octet long. The following format is used to define the community value. + +``AS:VAL`` + This format represents 4 octet communities value. ``AS`` is high order 2 + octet in digit format. ``VAL`` is low order 2 octet in digit format. This + format is useful to define AS oriented policy value. For example, + ``7675:80`` can be used when AS 7675 wants to pass local policy value 80 to + neighboring peer. + +``graceful-shutdown`` + ``graceful-shutdown`` represents well-known communities value + ``GRACEFUL_SHUTDOWN`` ``0xFFFF0000`` ``65535:0``. :rfc:`8326` implements + the purpose Graceful BGP Session Shutdown to reduce the amount of + lost traffic when taking BGP sessions down for maintenance. The use + of the community needs to be supported from your peers side to + actually have any effect. + +``accept-own`` + ``accept-own`` represents well-known communities value ``ACCEPT_OWN`` + ``0xFFFF0001`` ``65535:1``. :rfc:`7611` implements a way to signal + to a router to accept routes with a local nexthop address. This + can be the case when doing policing and having traffic having a + nexthop located in another VRF but still local interface to the + router. It is recommended to read the RFC for full details. + +``route-filter-translated-v4`` + ``route-filter-translated-v4`` represents well-known communities value + ``ROUTE_FILTER_TRANSLATED_v4`` ``0xFFFF0002`` ``65535:2``. + +``route-filter-v4`` + ``route-filter-v4`` represents well-known communities value + ``ROUTE_FILTER_v4`` ``0xFFFF0003`` ``65535:3``. + +``route-filter-translated-v6`` + ``route-filter-translated-v6`` represents well-known communities value + ``ROUTE_FILTER_TRANSLATED_v6`` ``0xFFFF0004`` ``65535:4``. + +``route-filter-v6`` + ``route-filter-v6`` represents well-known communities value + ``ROUTE_FILTER_v6`` ``0xFFFF0005`` ``65535:5``. + +``llgr-stale`` + ``llgr-stale`` represents well-known communities value ``LLGR_STALE`` + ``0xFFFF0006`` ``65535:6``. + Assigned and intended only for use with routers supporting the + Long-lived Graceful Restart Capability as described in + [Draft-IETF-uttaro-idr-bgp-persistence]_. + Routers receiving routes with this community may (depending on + implementation) choose allow to reject or modify routes on the + presence or absence of this community. + +``no-llgr`` + ``no-llgr`` represents well-known communities value ``NO_LLGR`` + ``0xFFFF0007`` ``65535:7``. + Assigned and intended only for use with routers supporting the + Long-lived Graceful Restart Capability as described in + [Draft-IETF-uttaro-idr-bgp-persistence]_. + Routers receiving routes with this community may (depending on + implementation) choose allow to reject or modify routes on the + presence or absence of this community. + +``accept-own-nexthop`` + ``accept-own-nexthop`` represents well-known communities value + ``accept-own-nexthop`` ``0xFFFF0008`` ``65535:8``. + [Draft-IETF-agrewal-idr-accept-own-nexthop]_ describes + how to tag and label VPN routes to be able to send traffic between VRFs + via an internal layer 2 domain on the same PE device. Refer to + [Draft-IETF-agrewal-idr-accept-own-nexthop]_ for full details. + +``blackhole`` + ``blackhole`` represents well-known communities value ``BLACKHOLE`` + ``0xFFFF029A`` ``65535:666``. :rfc:`7999` documents sending prefixes to + EBGP peers and upstream for the purpose of blackholing traffic. + Prefixes tagged with the this community should normally not be + re-advertised from neighbors of the originating network. Upon receiving + ``BLACKHOLE`` community from a BGP speaker, ``NO_ADVERTISE`` community + is added automatically. + +``no-export`` + ``no-export`` represents well-known communities value ``NO_EXPORT`` + ``0xFFFFFF01``. All routes carry this value must not be advertised to + outside a BGP confederation boundary. If neighboring BGP peer is part of BGP + confederation, the peer is considered as inside a BGP confederation + boundary, so the route will be announced to the peer. + +``no-advertise`` + ``no-advertise`` represents well-known communities value ``NO_ADVERTISE`` + ``0xFFFFFF02``. All routes carry this value must not be advertise to other + BGP peers. + +``local-AS`` + ``local-AS`` represents well-known communities value ``NO_EXPORT_SUBCONFED`` + ``0xFFFFFF03``. All routes carry this value must not be advertised to + external BGP peers. Even if the neighboring router is part of confederation, + it is considered as external BGP peer, so the route will not be announced to + the peer. + +``no-peer`` + ``no-peer`` represents well-known communities value ``NOPEER`` + ``0xFFFFFF04`` ``65535:65284``. :rfc:`3765` is used to communicate to + another network how the originating network want the prefix propagated. + +When the communities attribute is received duplicate community values in the +attribute are ignored and value is sorted in numerical order. + +.. [Draft-IETF-uttaro-idr-bgp-persistence] <https://tools.ietf.org/id/draft-uttaro-idr-bgp-persistence-04.txt> +.. [Draft-IETF-agrewal-idr-accept-own-nexthop] <https://tools.ietf.org/id/draft-agrewal-idr-accept-own-nexthop-00.txt> + +.. _bgp-community-lists: + +Community Lists +^^^^^^^^^^^^^^^ +Community lists are user defined lists of community attribute values. These +lists can be used for matching or manipulating the communities attribute in +UPDATE messages. + +There are two types of community list: + +standard + This type accepts an explicit value for the attribute. + +expanded + This type accepts a regular expression. Because the regex must be + interpreted on each use expanded community lists are slower than standard + lists. + +.. clicmd:: bgp community-list standard NAME permit|deny COMMUNITY + + This command defines a new standard community list. ``COMMUNITY`` is + communities value. The ``COMMUNITY`` is compiled into community structure. + We can define multiple community list under same name. In that case match + will happen user defined order. Once the community list matches to + communities attribute in BGP updates it return permit or deny by the + community list definition. When there is no matched entry, deny will be + returned. When ``COMMUNITY`` is empty it matches to any routes. + +.. clicmd:: bgp community-list expanded NAME permit|deny COMMUNITY + + This command defines a new expanded community list. ``COMMUNITY`` is a + string expression of communities attribute. ``COMMUNITY`` can be a regular + expression (:ref:`bgp-regular-expressions`) to match the communities + attribute in BGP updates. The expanded community is only used to filter, + not `set` actions. + +.. deprecated:: 5.0 + It is recommended to use the more explicit versions of this command. + +.. clicmd:: bgp community-list NAME permit|deny COMMUNITY + + When the community list type is not specified, the community list type is + automatically detected. If ``COMMUNITY`` can be compiled into communities + attribute, the community list is defined as a standard community list. + Otherwise it is defined as an expanded community list. This feature is left + for backward compatibility. Use of this feature is not recommended. + + Note that all community lists share the same namespace, so it's not + necessary to specify ``standard`` or ``expanded``; these modifiers are + purely aesthetic. + +.. clicmd:: show bgp community-list [NAME detail] + + Displays community list information. When ``NAME`` is specified the + specified community list's information is shown. + + :: + + # show bgp community-list + Named Community standard list CLIST + permit 7675:80 7675:100 no-export + deny internet + Named Community expanded list EXPAND + permit : + + # show bgp community-list CLIST detail + Named Community standard list CLIST + permit 7675:80 7675:100 no-export + deny internet + + +.. _bgp-numbered-community-lists: + +Numbered Community Lists +^^^^^^^^^^^^^^^^^^^^^^^^ + +When number is used for BGP community list name, the number has +special meanings. Community list number in the range from 1 to 99 is +standard community list. Community list number in the range from 100 +to 500 is expanded community list. These community lists are called +as numbered community lists. On the other hand normal community lists +is called as named community lists. + +.. clicmd:: bgp community-list (1-99) permit|deny COMMUNITY + + This command defines a new community list. The argument to (1-99) defines + the list identifier. + +.. clicmd:: bgp community-list (100-500) permit|deny COMMUNITY + + This command defines a new expanded community list. The argument to + (100-500) defines the list identifier. + +.. _bgp-community-alias: + +Community alias +^^^^^^^^^^^^^^^ + +BGP community aliases are useful to quickly identify what communities are set +for a specific prefix in a human-readable format. Especially handy for a huge +amount of communities. Accurately defined aliases can help you faster spot +things on the wire. + +.. clicmd:: bgp community alias NAME ALIAS + + This command creates an alias name for a community that will be used + later in various CLI outputs in a human-readable format. + + .. code-block:: frr + + ~# vtysh -c 'show run' | grep 'bgp community alias' + bgp community alias 65001:14 community-1 + bgp community alias 65001:123:1 lcommunity-1 + + ~# vtysh -c 'show ip bgp 172.16.16.1/32' + BGP routing table entry for 172.16.16.1/32, version 21 + Paths: (2 available, best #2, table default) + Advertised to non peer-group peers: + 65030 + 192.168.0.2 from 192.168.0.2 (172.16.16.1) + Origin incomplete, metric 0, valid, external, best (Neighbor IP) + Community: 65001:12 65001:13 community-1 65001:65534 + Large Community: lcommunity-1 65001:123:2 + Last update: Fri Apr 16 12:51:27 2021 + +.. clicmd:: show bgp [afi] [safi] [all] alias WORD [wide|json] + + Display prefixes with matching BGP community alias. + +.. _bgp-using-communities-in-route-map: + +Using Communities in Route Maps +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In :ref:`route-map` we can match on or set the BGP communities attribute. Using +this feature network operator can implement their network policy based on BGP +communities attribute. + +The following commands can be used in route maps: + +.. clicmd:: match alias WORD + + This command performs match to BGP updates using community alias WORD. When + the one of BGP communities value match to the one of community alias value in + community alias, it is match. + +.. clicmd:: match community WORD exact-match [exact-match] + + This command perform match to BGP updates using community list WORD. When + the one of BGP communities value match to the one of communities value in + community list, it is match. When `exact-match` keyword is specified, match + happen only when BGP updates have completely same communities value + specified in the community list. + +.. clicmd:: set community <none|COMMUNITY> additive + + This command sets the community value in BGP updates. If the attribute is + already configured, the newly provided value replaces the old one unless the + ``additive`` keyword is specified, in which case the new value is appended + to the existing value. + + If ``none`` is specified as the community value, the communities attribute + is not sent. + + It is not possible to set an expanded community list. + +.. clicmd:: set comm-list WORD delete + + This command remove communities value from BGP communities attribute. The + ``word`` is community list name. When BGP route's communities value matches + to the community list ``word``, the communities value is removed. When all + of communities value is removed eventually, the BGP update's communities + attribute is completely removed. + +.. _bgp-communities-example: + +Example Configuration +^^^^^^^^^^^^^^^^^^^^^ + +The following configuration is exemplary of the most typical usage of BGP +communities attribute. In the example, AS 7675 provides an upstream Internet +connection to AS 100. When the following configuration exists in AS 7675, the +network operator of AS 100 can set local preference in AS 7675 network by +setting BGP communities attribute to the updates. + +.. code-block:: frr + + router bgp 7675 + neighbor 192.168.0.1 remote-as 100 + address-family ipv4 unicast + neighbor 192.168.0.1 route-map RMAP in + exit-address-family + ! + bgp community-list 70 permit 7675:70 + bgp community-list 80 permit 7675:80 + bgp community-list 90 permit 7675:90 + ! + route-map RMAP permit 10 + match community 70 + set local-preference 70 + ! + route-map RMAP permit 20 + match community 80 + set local-preference 80 + ! + route-map RMAP permit 30 + match community 90 + set local-preference 90 + + +The following configuration announces ``10.0.0.0/8`` from AS 100 to AS 7675. +The route has communities value ``7675:80`` so when above configuration exists +in AS 7675, the announced routes' local preference value will be set to 80. + +.. code-block:: frr + + router bgp 100 + network 10.0.0.0/8 + neighbor 192.168.0.2 remote-as 7675 + address-family ipv4 unicast + neighbor 192.168.0.2 route-map RMAP out + exit-address-family + ! + ip prefix-list PLIST permit 10.0.0.0/8 + ! + route-map RMAP permit 10 + match ip address prefix-list PLIST + set community 7675:80 + + +The following configuration is an example of BGP route filtering using +communities attribute. This configuration only permit BGP routes which has BGP +communities value (``0:80`` and ``0:90``) or ``0:100``. The network operator can +set special internal communities value at BGP border router, then limit the +BGP route announcements into the internal network. + +.. code-block:: frr + + router bgp 7675 + neighbor 192.168.0.1 remote-as 100 + address-family ipv4 unicast + neighbor 192.168.0.1 route-map RMAP in + exit-address-family + ! + bgp community-list 1 permit 0:80 0:90 + bgp community-list 1 permit 0:100 + ! + route-map RMAP permit in + match community 1 + + +The following example filters BGP routes which have a community value of +``1:1``. When there is no match community-list returns ``deny``. To avoid +filtering all routes, a ``permit`` line is set at the end of the +community-list. + +.. code-block:: frr + + router bgp 7675 + neighbor 192.168.0.1 remote-as 100 + address-family ipv4 unicast + neighbor 192.168.0.1 route-map RMAP in + exit-address-family + ! + bgp community-list standard FILTER deny 1:1 + bgp community-list standard FILTER permit + ! + route-map RMAP permit 10 + match community FILTER + + +The following configuration is an example of communities value deletion. With +this configuration the community values ``100:1`` and ``100:2`` are removed +from BGP updates. For communities value deletion, only ``permit`` +community-list is used. ``deny`` community-list is ignored. + +.. code-block:: frr + + router bgp 7675 + neighbor 192.168.0.1 remote-as 100 + address-family ipv4 unicast + neighbor 192.168.0.1 route-map RMAP in + exit-address-family + ! + bgp community-list standard DEL permit 100:1 100:2 + ! + route-map RMAP permit 10 + set comm-list DEL delete + + +.. _bgp-extended-communities-attribute: + +Extended Communities Attribute +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +BGP extended communities attribute is introduced with MPLS VPN/BGP technology. +MPLS VPN/BGP expands capability of network infrastructure to provide VPN +functionality. At the same time it requires a new framework for policy routing. +With BGP Extended Communities Attribute we can use Route Target or Site of +Origin for implementing network policy for MPLS VPN/BGP. + +BGP Extended Communities Attribute is similar to BGP Communities Attribute. It +is an optional transitive attribute. BGP Extended Communities Attribute can +carry multiple Extended Community value. Each Extended Community value is +eight octet length. + +BGP Extended Communities Attribute provides an extended range compared with BGP +Communities Attribute. Adding to that there is a type field in each value to +provides community space structure. + +There are two format to define Extended Community value. One is AS based format +the other is IP address based format. + +``AS:VAL`` + This is a format to define AS based Extended Community value. ``AS`` part + is 2 octets Global Administrator subfield in Extended Community value. + ``VAL`` part is 4 octets Local Administrator subfield. ``7675:100`` + represents AS 7675 policy value 100. + +``IP-Address:VAL`` + This is a format to define IP address based Extended Community value. + ``IP-Address`` part is 4 octets Global Administrator subfield. ``VAL`` part + is 2 octets Local Administrator subfield. + +.. _bgp-extended-community-lists: + +Extended Community Lists +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: bgp extcommunity-list standard NAME permit|deny EXTCOMMUNITY + + This command defines a new standard extcommunity-list. `extcommunity` is + extended communities value. The `extcommunity` is compiled into extended + community structure. We can define multiple extcommunity-list under same + name. In that case match will happen user defined order. Once the + extcommunity-list matches to extended communities attribute in BGP updates + it return permit or deny based upon the extcommunity-list definition. When + there is no matched entry, deny will be returned. When `extcommunity` is + empty it matches to any routes. + +.. clicmd:: bgp extcommunity-list expanded NAME permit|deny LINE + + This command defines a new expanded extcommunity-list. `line` is a string + expression of extended communities attribute. `line` can be a regular + expression (:ref:`bgp-regular-expressions`) to match an extended communities + attribute in BGP updates. + + Note that all extended community lists shares a single name space, so it's + not necessary to specify their type when creating or destroying them. + +.. clicmd:: show bgp extcommunity-list [NAME detail] + + This command displays current extcommunity-list information. When `name` is + specified the community list's information is shown. + + +.. _bgp-extended-communities-in-route-map: + +BGP Extended Communities in Route Map +""""""""""""""""""""""""""""""""""""" + +.. clicmd:: match extcommunity WORD + +.. clicmd:: set extcommunity none + + This command resets the extended community value in BGP updates. If the attribute is + already configured or received from the peer, the attribute is discarded and set to + none. This is useful if you need to strip incoming extended communities. + +.. clicmd:: set extcommunity rt EXTCOMMUNITY + + This command sets Route Target value. + +.. clicmd:: set extcommunity nt EXTCOMMUNITY + + This command sets Node Target value. + + If the receiving BGP router supports Node Target Extended Communities, + it will install the route with the community that contains it's own + local BGP Identifier. Otherwise, it's not installed. + +.. clicmd:: set extcommunity soo EXTCOMMUNITY + + This command sets Site of Origin value. + +.. clicmd:: set extcomumnity color EXTCOMMUNITY + + This command sets colors values. + +.. clicmd:: set extcommunity bandwidth <(1-25600) | cumulative | num-multipaths> [non-transitive] + + This command sets the BGP link-bandwidth extended community for the prefix + (best path) for which it is applied. The link-bandwidth can be specified as + an ``explicit value`` (specified in Mbps), or the router can be told to use + the ``cumulative bandwidth`` of all multipaths for the prefix or to compute + it based on the ``number of multipaths``. The link bandwidth extended + community is encoded as ``transitive`` unless the set command explicitly + configures it as ``non-transitive``. + +.. seealso:: :ref:`wecmp_linkbw` + +Note that the extended expanded community is only used for `match` rule, not for +`set` actions. + +.. _bgp-large-communities-attribute: + +Large Communities Attribute +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The BGP Large Communities attribute was introduced in Feb 2017 with +:rfc:`8092`. + +The BGP Large Communities Attribute is similar to the BGP Communities Attribute +except that it has 3 components instead of two and each of which are 4 octets +in length. Large Communities bring additional functionality and convenience +over traditional communities, specifically the fact that the ``GLOBAL`` part +below is now 4 octets wide allowing seamless use in networks using 4-byte ASNs. + +``GLOBAL:LOCAL1:LOCAL2`` + This is the format to define Large Community values. Referencing :rfc:`8195` + the values are commonly referred to as follows: + + - The ``GLOBAL`` part is a 4 octet Global Administrator field, commonly used + as the operators AS number. + - The ``LOCAL1`` part is a 4 octet Local Data Part 1 subfield referred to as + a function. + - The ``LOCAL2`` part is a 4 octet Local Data Part 2 field and referred to + as the parameter subfield. + + As an example, ``65551:1:10`` represents AS 65551 function 1 and parameter + 10. The referenced RFC above gives some guidelines on recommended usage. + +.. _bgp-large-community-lists: + +Large Community Lists +""""""""""""""""""""" + +Two types of large community lists are supported, namely `standard` and +`expanded`. + +.. clicmd:: bgp large-community-list standard NAME permit|deny LARGE-COMMUNITY + + This command defines a new standard large-community-list. `large-community` + is the Large Community value. We can add multiple large communities under + same name. In that case the match will happen in the user defined order. + Once the large-community-list matches the Large Communities attribute in BGP + updates it will return permit or deny based upon the large-community-list + definition. When there is no matched entry, a deny will be returned. When + `large-community` is empty it matches any routes. + +.. clicmd:: bgp large-community-list expanded NAME permit|deny LINE + + This command defines a new expanded large-community-list. Where `line` is a + string matching expression, it will be compared to the entire Large + Communities attribute as a string, with each large-community in order from + lowest to highest. `line` can also be a regular expression which matches + this Large Community attribute. + + Note that all community lists share the same namespace, so it's not + necessary to specify ``standard`` or ``expanded``; these modifiers are + purely aesthetic. + +.. clicmd:: show bgp large-community-list + +.. clicmd:: show bgp large-community-list NAME detail + + This command display current large-community-list information. When + `name` is specified the community list information is shown. + +.. clicmd:: show ip bgp large-community-info + + This command displays the current large communities in use. + +.. _bgp-large-communities-in-route-map: + +Large Communities in Route Map +"""""""""""""""""""""""""""""" + +.. clicmd:: match large-community LINE [exact-match] + + Where `line` can be a simple string to match, or a regular expression. It + is very important to note that this match occurs on the entire + large-community string as a whole, where each large-community is ordered + from lowest to highest. When `exact-match` keyword is specified, match + happen only when BGP updates have completely same large communities value + specified in the large community list. + +.. clicmd:: set large-community LARGE-COMMUNITY + +.. clicmd:: set large-community LARGE-COMMUNITY LARGE-COMMUNITY + +.. clicmd:: set large-community LARGE-COMMUNITY additive + + These commands are used for setting large-community values. The first + command will overwrite any large-communities currently present. + The second specifies two large-communities, which overwrites the current + large-community list. The third will add a large-community value without + overwriting other values. Multiple large-community values can be specified. + +Note that the large expanded community is only used for `match` rule, not for +`set` actions. + +.. _bgp-roles-and-only-to-customers: + +BGP Roles and Only to Customers +------------------------------- + +BGP roles are defined in :rfc:`9234` and provide an easy way to route leaks +prevention, detection and mitigation. + +To enable its mechanics, you must set your local role to reflect your type of +peering relationship with your neighbor. Possible values of ``LOCAL-ROLE`` are: + +- provider +- rs-server +- rs-client +- customer +- peer + +The local Role value is negotiated with the new BGP Role capability with a +built-in check of the corresponding value. In case of mismatch the new OPEN +Roles Mismatch Notification <2, 11> would be sent. + +The correct Role pairs are: + +* Provider - Customer +* Peer - Peer +* RS-Server - RS-Client + +.. code-block:: shell + + ~# vtysh -c 'show bgp neighbor' | grep 'Role' + Local Role: customer + Neighbor Role: provider + Role: advertised and received + +If strict-mode is set BGP session won't become established until BGP neighbor +set local Role on its side. This configuration parameter is defined in +:rfc:`9234` and used to enforce corresponding configuration at your +counter-part side. Default value - disabled. + +Routes that sent from provider, rs-server, or peer local-role (or if received +by customer, rs-clinet, or peer local-role) will be marked with a new +Only to Customer (OTC) attribute. + +Routes with this attribute can only be sent to your neighbor if your +local-role is provider or rs-server. Routes with this attribute can be +received only if your local-role is customer or rs-client. + +In case of peer-peer relationship routes can be received only if +OTC value is equal to your neighbor AS number. + +All these rules with OTC help to detect and mitigate route leaks and +happened automatically if local-role is set. + +.. clicmd:: neighbor PEER local-role LOCAL-ROLE [strict-mode] + + This command set your local-role to ``LOCAL-ROLE``: + <provider|rs-server|rs-client|customer|peer>. + + This role helps to detect and prevent route leaks. + + If ``strict-mode`` is set, your neighbor must send you Capability with the + value of his role (by setting local-role on his side). Otherwise, a Role + Mismatch Notification will be sent. + +Labeled unicast +--------------- + +*bgpd* supports labeled information, as per :rfc:`3107`. + +.. clicmd:: bgp labeled-unicast <explicit-null|ipv4-explicit-null|ipv6-explicit-null> + +By default, locally advertised prefixes use the `implicit-null` label to +encode in the outgoing NLRI. The following command uses the `explicit-null` +label value for all the BGP instances. + +.. _bgp-l3vpn-vrfs: + +L3VPN VRFs +---------- + +*bgpd* supports :abbr:`L3VPN (Layer 3 Virtual Private Networks)` :abbr:`VRFs +(Virtual Routing and Forwarding)` for IPv4 :rfc:`4364` and IPv6 :rfc:`4659`. +L3VPN routes, and their associated VRF MPLS labels, can be distributed to VPN +SAFI neighbors in the *default*, i.e., non VRF, BGP instance. VRF MPLS labels +are reached using *core* MPLS labels which are distributed using LDP or BGP +labeled unicast. *bgpd* also supports inter-VRF route leaking. + + +L3VPN over GRE interfaces +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In MPLS-VPN or SRv6-VPN, an L3VPN next-hop entry requires that the path +chosen respectively contains a labelled path or a valid SID IPv6 address. +Otherwise the L3VPN entry will not be installed. It is possible to ignore +that check when the path chosen by the next-hop uses a GRE interface, and +there is a route-map configured at inbound side of ipv4-vpn or ipv6-vpn +address family with following syntax: + +.. clicmd:: set l3vpn next-hop encapsulation gre + +The incoming BGP L3VPN entry is accepted, provided that the next hop of the +L3VPN entry uses a path that takes the GRE tunnel as outgoing interface. The +remote endpoint should be configured just behind the GRE tunnel; remote +device configuration may vary depending whether it acts at edge endpoint or +not: in any case, the expectation is that incoming MPLS traffic received at +this endpoint should be considered as a valid path for L3VPN. + +.. _bgp-vrf-route-leaking: + +VRF Route Leaking +----------------- + +BGP routes may be leaked (i.e. copied) between a unicast VRF RIB and the VPN +SAFI RIB of the default VRF for use in MPLS-based L3VPNs. Unicast routes may +also be leaked between any VRFs (including the unicast RIB of the default BGP +instanced). A shortcut syntax is also available for specifying leaking from one +VRF to another VRF using the default instance's VPN RIB as the intermediary. A +common application of the VRF-VRF feature is to connect a customer's private +routing domain to a provider's VPN service. Leaking is configured from the +point of view of an individual VRF: ``import`` refers to routes leaked from VPN +to a unicast VRF, whereas ``export`` refers to routes leaked from a unicast VRF +to VPN. + +Required parameters +^^^^^^^^^^^^^^^^^^^ + +Routes exported from a unicast VRF to the VPN RIB must be augmented by two +parameters: + +- an :abbr:`RD (Route Distinguisher)` +- an :abbr:`RTLIST (Route-target List)` + +Configuration for these exported routes must, at a minimum, specify these two +parameters. + +Routes imported from the VPN RIB to a unicast VRF are selected according to +their RTLISTs. Routes whose RTLIST contains at least one route-target in +common with the configured import RTLIST are leaked. Configuration for these +imported routes must specify an RTLIST to be matched. + +The RD, which carries no semantic value, is intended to make the route unique +in the VPN RIB among all routes of its prefix that originate from all the +customers and sites that are attached to the provider's VPN service. +Accordingly, each site of each customer is typically assigned an RD that is +unique across the entire provider network. + +The RTLIST is a set of route-target extended community values whose purpose is +to specify route-leaking policy. Typically, a customer is assigned a single +route-target value for import and export to be used at all customer sites. This +configuration specifies a simple topology wherein a customer has a single +routing domain which is shared across all its sites. More complex routing +topologies are possible through use of additional route-targets to augment the +leaking of sets of routes in various ways. + +When using the shortcut syntax for vrf-to-vrf leaking, the RD and RT are +auto-derived. + +General configuration +^^^^^^^^^^^^^^^^^^^^^ + +Configuration of route leaking between a unicast VRF RIB and the VPN SAFI RIB +of the default VRF is accomplished via commands in the context of a VRF +address-family: + +.. clicmd:: rd vpn export AS:NN|IP:nn + + Specifies the route distinguisher to be added to a route exported from the + current unicast VRF to VPN. + +.. clicmd:: rt vpn import|export|both RTLIST... + + Specifies the route-target list to be attached to a route (export) or the + route-target list to match against (import) when exporting/importing between + the current unicast VRF and VPN. + + The RTLIST is a space-separated list of route-targets, which are BGP + extended community values as described in + :ref:`bgp-extended-communities-attribute`. + +.. clicmd:: label vpn export allocation-mode per-vrf|per-nexthop + + Select how labels are allocated in the given VRF. By default, the `per-vrf` + mode is selected, and one label is used for all prefixes from the VRF. The + `per-nexthop` will use a unique label for all prefixes that are reachable + via the same nexthop. + +.. clicmd:: label vpn export (0..1048575)|auto + + Enables an MPLS label to be attached to a route exported from the current + unicast VRF to VPN. If the value specified is ``auto``, the label value is + automatically assigned from a pool maintained by the Zebra daemon. If Zebra + is not running, or if this command is not configured, automatic label + assignment will not complete, which will block corresponding route export. + +.. clicmd:: nexthop vpn export A.B.C.D|X:X::X:X + + Specifies an optional nexthop value to be assigned to a route exported from + the current unicast VRF to VPN. If left unspecified, the nexthop will be set + to 0.0.0.0 or 0:0::0:0 (self). + +.. clicmd:: route-map vpn import|export MAP + + Specifies an optional route-map to be applied to routes imported or exported + between the current unicast VRF and VPN. + +.. clicmd:: import|export vpn + + Enables import or export of routes between the current unicast VRF and VPN. + +.. clicmd:: import vrf VRFNAME + + Shortcut syntax for specifying automatic leaking from vrf VRFNAME to + the current VRF using the VPN RIB as intermediary. The RD and RT + are auto derived and should not be specified explicitly for either the + source or destination VRF's. + + This shortcut syntax mode is not compatible with the explicit + `import vpn` and `export vpn` statements for the two VRF's involved. + The CLI will disallow attempts to configure incompatible leaking + modes. + +.. clicmd:: bgp retain route-target all + +It is possible to retain or not VPN prefixes that are not imported by local +VRF configuration. This can be done via the following command in the context +of the global VPNv4/VPNv6 family. This command defaults to on and is not +displayed. +The `no bgp retain route-target all` form of the command is displayed. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> soo EXTCOMMUNITY + +Without this command, SoO extended community attribute is configured using +an inbound route map that sets the SoO value during the update process. +With the introduction of the new BGP per-neighbor Site-of-Origin (SoO) feature, +two new commands configured in sub-modes under router configuration mode +simplify the SoO value configuration. + +If we configure SoO per neighbor at PEs, the SoO community is automatically +added for all routes from the CPEs. Routes are validated and prevented from +being sent back to the same CPE (e.g.: multi-site). This is especially needed +when using ``as-override`` or ``allowas-in`` to prevent routing loops. + +.. clicmd:: mpls bgp forwarding + +It is possible to permit BGP install VPN prefixes without transport labels, +by issuing the following command under the interface configuration context. +This configuration will install VPN prefixes originated from an e-bgp session, +and with the next-hop directly connected. + +.. clicmd:: mpls bgp l3vpn-multi-domain-switching + +Redistribute labeled L3VPN routes from AS to neighboring AS (RFC-4364 option +B, or within the same AS when the iBGP peer uses ``next-hop-self`` to rewrite +the next-hop attribute). The labeled L3VPN routes received on this interface are +re-advertised with local labels and an MPLS table swap entry is set to bind +the local label to the received label. + +.. _bgp-l3vpn-srv6: + +L3VPN SRv6 +---------- + +.. clicmd:: segment-routing srv6 + + Use SRv6 backend with BGP L3VPN, and go to its configuration node. + +.. clicmd:: locator NAME + + Specify the SRv6 locator to be used for SRv6 L3VPN. The Locator name must + be set in zebra, but user can set it in any order. + +General configuration +^^^^^^^^^^^^^^^^^^^^^ + +Configuration of the SRv6 SID used to advertise a L3VPN for both IPv4 and IPv6 +is accomplished via the following command in the context of a VRF: + +.. clicmd:: sid vpn per-vrf export (1..1048575)|auto + + Enables a SRv6 SID to be attached to a route exported from the current + unicast VRF to VPN. A single SID is used for both IPv4 and IPv6 address + families. If you want to set a SID for only IPv4 address family or IPv6 + address family, you need to use the command ``sid vpn export (1..1048575)|auto`` + in the context of an address-family. If the value specified is ``auto``, + the SID value is automatically assigned from a pool maintained by the Zebra + daemon. If Zebra is not running, or if this command is not configured, automatic + SID assignment will not complete, which will block corresponding route export. + +.. _bgp-evpn: + +Ethernet Virtual Network - EVPN +------------------------------- + +Note: When using EVPN features and if you have a large number of hosts, make +sure to adjust the size of the arp neighbor cache to avoid neighbor table +overflow and/or excessive garbage collection. On Linux, the size of the table +and garbage collection frequency can be controlled via the following +sysctl configurations: + +.. code-block:: shell + + net.ipv4.neigh.default.gc_thresh1 + net.ipv4.neigh.default.gc_thresh2 + net.ipv4.neigh.default.gc_thresh3 + + net.ipv6.neigh.default.gc_thresh1 + net.ipv6.neigh.default.gc_thresh2 + net.ipv6.neigh.default.gc_thresh3 + +For more information, see ``man 7 arp``. + +.. _bgp-enabling-evpn: + +Enabling EVPN +^^^^^^^^^^^^^ + +EVPN should be enabled on the BGP instance corresponding to the VRF acting as +the underlay for the VXLAN tunneling. In most circumstances this will be the +default VRF. The command to enable EVPN for a BGP instance is +``advertise-all-vni`` which lives under ``address-family l2vpn evpn``: + +.. code-block:: frr + + router bgp 65001 + ! + address-family l2vpn evpn + advertise-all-vni + +A more comprehensive configuration example can be found in the :ref:`evpn` page. + +.. _bgp-evpn-l3-route-targets: + +EVPN L3 Route-Targets +^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: route-target <import|export|both> <RTLIST|auto> + +Modify the route-target set for EVPN advertised type-2/type-5 routes. +RTLIST is a list of any of matching +``(A.B.C.D:MN|EF:OPQR|GHJK:MN|*:OPQR|*:MN)`` where ``*`` indicates wildcard +matching for the AS number. It will be set to match any AS number. This is +useful in datacenter deployments with Downstream VNI. ``auto`` is used to +retain the autoconfigure that is default behavior for L3 RTs. + +.. _bgp-evpn-advertise-pip: + +EVPN advertise-PIP +^^^^^^^^^^^^^^^^^^ + +In a EVPN symmetric routing MLAG deployment, all EVPN routes advertised +with anycast-IP as next-hop IP and anycast MAC as the Router MAC (RMAC - in +BGP EVPN Extended-Community). +EVPN picks up the next-hop IP from the VxLAN interface's local tunnel IP and +the RMAC is obtained from the MAC of the L3VNI's SVI interface. +Note: Next-hop IP is used for EVPN routes whether symmetric routing is +deployed or not but the RMAC is only relevant for symmetric routing scenario. + +Current behavior is not ideal for Prefix (type-5) and self (type-2) +routes. This is because the traffic from remote VTEPs routed sub optimally +if they land on the system where the route does not belong. + +The advertise-pip feature advertises Prefix (type-5) and self (type-2) +routes with system's individual (primary) IP as the next-hop and individual +(system) MAC as Router-MAC (RMAC), while leaving the behavior unchanged for +other EVPN routes. + +To support this feature there needs to have ability to co-exist a +(system-MAC, system-IP) pair with a (anycast-MAC, anycast-IP) pair with the +ability to terminate VxLAN-encapsulated packets received for either pair on +the same L3VNI (i.e associated VLAN). This capability is needed per tenant +VRF instance. + +To derive the system-MAC and the anycast MAC, there must be a +separate/additional MAC-VLAN interface corresponding to L3VNI’s SVI. +The SVI interface’s MAC address can be interpreted as system-MAC +and MAC-VLAN interface's MAC as anycast MAC. + +To derive system-IP and anycast-IP, the default BGP instance's router-id is used +as system-IP and the VxLAN interface’s local tunnel IP as the anycast-IP. + +User has an option to configure the system-IP and/or system-MAC value if the +auto derived value is not preferred. + +Note: By default, advertise-pip feature is enabled and user has an option to +disable the feature via configuration CLI. Once the feature is disabled under +bgp vrf instance or MAC-VLAN interface is not configured, all the routes follow +the same behavior of using same next-hop and RMAC values. + +.. clicmd:: advertise-pip [ip <addr> [mac <addr>]] + +Enables or disables advertise-pip feature, specify system-IP and/or system-MAC +parameters. + +EVPN advertise-svi-ip +^^^^^^^^^^^^^^^^^^^^^ +Typically, the SVI IP address is reused on VTEPs across multiple racks. However, +if you have unique SVI IP addresses that you want to be reachable you can use the +advertise-svi-ip option. This option advertises the SVI IP/MAC address as a type-2 +route and eliminates the need for any flooding over VXLAN to reach the IP from a +remote VTEP. + +.. clicmd:: advertise-svi-ip + +Note that you should not enable both the advertise-svi-ip and the advertise-default-gw +at the same time. + +.. _bgp-evpn-overlay-index-gateway-ip: + +EVPN Overlay Index Gateway IP +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +RFC https://datatracker.ietf.org/doc/html/rfc9136 explains the use of overlay +indexes for recursive route resolution for EVPN type-5 route. + +We support gateway IP overlay index. +A gateway IP, advertised with EVPN prefix route, is used to find an EVPN MAC/IP +route with its IP field same as the gateway IP. This MAC/IP entry provides the +nexthop VTEP and the tunnel information required for the VxLAN encapsulation. + +Functionality: + +:: + + . +--------+ BGP +--------+ BGP +--------+ +--------+ + SN1 | | IPv4 | | EVPN | | | | + ======+ Host1 +------+ PE1 +------+ PE2 +------+ Host2 + + | | | | | | | | + +--------+ +--------+ +--------+ +--------+ + +Consider above topology where prefix SN1 is connected behind host1. Host1 +advertises SN1 to PE1 over BGP IPv4 session. PE1 advertises SN1 to PE2 using +EVPN type-5 route with host1 IP as the gateway IP. PE1 also advertises +Host1 MAC/IP as type-2 route which is used to resolve host1 gateway IP. + +PE2 receives this type-5 route and imports it into the vrf based on route +targets. BGP prefix imported into the vrf uses gateway IP as its BGP nexthop. +This route is installed into zebra if following conditions are satisfied: + +1. Gateway IP nexthop is L3 reachable. +2. PE2 has received EVPN type-2 route with IP field set to gateway IP. + +Topology requirements: + +1. This feature is supported for asymmetric routing model only. While + sending packets to SN1, ingress PE (PE2) performs routing and + egress PE (PE1) performs only bridging. +2. This feature supports only traditional(non vlan-aware) bridge model. Bridge + interface associated with L2VNI is an L3 interface. i.e., this interface is + configured with an address in the L2VNI subnet. Note that the gateway IP + should also have an address in the same subnet. +3. As this feature works in asymmetric routing model, all L2VNIs and corresponding + VxLAN and bridge interfaces should be present at all the PEs. +4. L3VNI configuration is required to generate and import EVPN type-5 routes. + L3VNI VxLAN and bridge interfaces also should be present. + +A PE can use one of the following two mechanisms to advertise an EVPN type-5 +route with gateway IP. + +1. CLI to add gateway IP while generating EVPN type-5 route from a BGP IPv4/IPv6 +prefix: + +.. clicmd:: advertise <ipv4|ipv6> unicast [gateway-ip] + +When this CLI is configured for a BGP vrf under L2VPN EVPN address family, EVPN +type-5 routes are generated for BGP prefixes in the vrf. Nexthop of the BGP +prefix becomes the gateway IP of the corresponding type-5 route. + +If the above command is configured without the "gateway-ip" keyword, type-5 +routes are generated without overlay index. + +2. Add gateway IP to EVPN type-5 route using a route-map: + +.. clicmd:: set evpn gateway-ip <ipv4|ipv6> <addr> + +When route-map with above set clause is applied as outbound policy in BGP, it +will set the gateway-ip in EVPN type-5 NLRI. + +Example configuration: + +.. code-block:: frr + + router bgp 100 + neighbor 192.168.0.1 remote-as 101 + ! + address-family ipv4 l2vpn evpn + neighbor 192.168.0.1 route-map RMAP out + exit-address-family + ! + route-map RMAP permit 10 + set evpn gateway-ip 10.0.0.1 + set evpn gateway-ip 10::1 + +A PE that receives a type-5 route with gateway IP overlay index should have +"enable-resolve-overlay-index" configuration enabled to recursively resolve the +overlay index nexthop and install the prefix into zebra. + +.. clicmd:: enable-resolve-overlay-index + +Example configuration: + +.. code-block:: frr + + router bgp 65001 + bgp router-id 192.168.100.1 + no bgp ebgp-requires-policy + neighbor 10.0.1.2 remote-as 65002 + ! + address-family l2vpn evpn + neighbor 10.0.1.2 activate + advertise-all-vni + enable-resolve-overlay-index + exit-address-family + ! + +.. _bgp-evpn-mac-vrf-site-of-origin: + +EVPN MAC-VRF Site-of-Origin +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In some EVPN deployments it is useful to associate a logical VTEP's Layer 2 +domain (MAC-VRF) with a Site-of-Origin "site" identifier. This provides a +BGP topology-independent means of marking and import-filtering EVPN routes +originated from a particular L2 domain. One situation where this is valuable +is when deploying EVPN using anycast VTEPs, i.e. Active/Active MLAG, as it +can be used to avoid ownership conflicts between the two control planes +(EVPN vs MLAG). + +Example Use Case (MLAG Anycast VTEPs): + +During normal operation, an MLAG VTEP will advertise EVPN routes for attached +hosts using a shared anycast IP as the BGP next-hop. It is expected for its +MLAG peer to drop routes originated by the MLAG Peer since they have a Martian +(self) next-hop. However, prior to the anycast IP being assigned to the local +system, the anycast BGP next-hop will not be considered a Martian (self) IP. +This results in a timing window where hosts that are locally attached to the +MLAG pair's L2 domain can be learned both as "local" (via MLAG) or "remote" +(via an EVPN route with a non-local next-hop). This can trigger erroneous MAC +Mobility events, as the host "moves" between one MLAG Peer's Unique VTEP-IP +and the shared anycast VTEP-IP, which causes unnecessary control plane and +data plane events to propagate throughout the EVPN domain. +By associating the MAC-VRF of both MLAG VTEPs with the same site identifier, +EVPN routes originated by one MLAG VTEP will ignored by its MLAG peer, ensuring +that only the MLAG control plane attempts to take ownership of local hosts. + +The EVPN MAC-VRF Site-of-Origin feature works by influencing two behaviors: + +1. All EVPN routes originating from the local MAC-VRF will have a + Site-of-Origin extended community added to the route, matching the + configured value. +2. EVPN routes will be subjected to a "self SoO" check during MAC-VRF + or IP-VRF import processing. If the EVPN route is found to carry a + Site-of-Origin extended community whose value matches the locally + configured MAC-VRF Site-of-Origin, the route will be maintained in + the global EVPN RIB ("show bgp l2vpn evpn route") but will not be + imported into the corresponding MAC-VRF ("show bgp vni") or IP-VRF + ("show bgp [vrf <vrfname>] [ipv4 | ipv6 [unicast]]"). + +The import filtering described in item (2) is constrained just to Type-2 +(MAC-IP) and Type-3 (IMET) EVPN routes. + +The EVPN MAC-VRF Site-of-Origin can be configured using a single CLI command +under ``address-family l2vpn evpn`` of the EVPN underlay BGP instance. + +.. clicmd:: [no] mac-vrf soo <site-of-origin-string> + +Example configuration: + +.. code-block:: frr + + router bgp 100 + neighbor 192.168.0.1 remote-as 101 + ! + address-family ipv4 l2vpn evpn + neighbor 192.168.0.1 activate + advertise-all-vni + mac-vrf soo 100.64.0.0:777 + exit-address-family + +This configuration ensures: + +1. EVPN routes originated from a local L2VNI will have a Site-of-Origin + extended community with the value ``100.64.0.0:777`` +2. Received EVPN routes carrying a Site-of-Origin extended community with the + value ``100.64.0.0:777`` will not be imported into a local MAC-VRF (L2VNI) + or IP-VRF (L3VNI). + +.. _bgp-evpn-mh: + +EVPN Multihoming +^^^^^^^^^^^^^^^^ + +All-Active Multihoming is used for redundancy and load sharing. Servers +are attached to two or more PEs and the links are bonded (link-aggregation). +This group of server links is referred to as an Ethernet Segment. + +Ethernet Segments +""""""""""""""""" +An Ethernet Segment can be configured by specifying a system-MAC and a +local discriminator or a complete ESINAME against the bond interface on the +PE (via zebra) - + +.. clicmd:: evpn mh es-id <(1-16777215)|ESINAME> + +.. clicmd:: evpn mh es-sys-mac X:X:X:X:X:X + +The sys-mac and local discriminator are used for generating a 10-byte, +Type-3 Ethernet Segment ID. ESINAME is a 10-byte, Type-0 Ethernet Segment ID - +"00:AA:BB:CC:DD:EE:FF:GG:HH:II". + +Type-1 (EAD-per-ES and EAD-per-EVI) routes are used to advertise the locally +attached ESs and to learn off remote ESs in the network. Local Type-2/MAC-IP +routes are also advertised with a destination ESI allowing for MAC-IP syncing +between Ethernet Segment peers. +Reference: RFC 7432, RFC 8365 + +EVPN-MH is intended as a replacement for MLAG or Anycast VTEPs. In +multihoming each PE has an unique VTEP address which requires the introduction +of a new dataplane construct, MAC-ECMP. Here a MAC/FDB entry can point to a +list of remote PEs/VTEPs. + +BUM handling +"""""""""""" +Type-4 (ESR) routes are used for Designated Forwarder (DF) election. DFs +forward BUM traffic received via the overlay network. This implementation +uses a preference based DF election specified by draft-ietf-bess-evpn-pref-df. +The DF preference is configurable per-ES (via zebra) - + +.. clicmd:: evpn mh es-df-pref (1-16777215) + +BUM traffic is rxed via the overlay by all PEs attached to a server but +only the DF can forward the de-capsulated traffic to the access port. To +accommodate that non-DF filters are installed in the dataplane to drop +the traffic. + +Similarly traffic received from ES peers via the overlay cannot be forwarded +to the server. This is split-horizon-filtering with local bias. + +Knobs for interop +""""""""""""""""" +Some vendors do not send EAD-per-EVI routes. To interop with them we +need to relax the dependency on EAD-per-EVI routes and activate a remote +ES-PE based on just the EAD-per-ES route. + +Note that by default we advertise and expect EAD-per-EVI routes. + +.. clicmd:: disable-ead-evi-rx + +.. clicmd:: disable-ead-evi-tx + +Fast failover +""""""""""""" +As the primary purpose of EVPN-MH is redundancy keeping the failover efficient +is a recurring theme in the implementation. Following sub-features have +been introduced for the express purpose of efficient ES failovers. + +- Layer-2 Nexthop Groups and MAC-ECMP via L2NHG. + +- Host routes (for symmetric IRB) via L3NHG. + On dataplanes that support layer3 nexthop groups the feature can be turned + on via the following BGP config - + +.. clicmd:: use-es-l3nhg + +- Local ES (MAC/Neigh) failover via ES-redirect. + On dataplanes that do not have support for ES-redirect the feature can be + turned off via the following zebra config - + +.. clicmd:: evpn mh redirect-off + +Uplink/Core tracking +"""""""""""""""""""" +When all the underlay links go down the PE no longer has access to the VxLAN ++overlay. To prevent blackholing of traffic the server/ES links are +protodowned on the PE. A link can be setup for uplink tracking via the +following zebra configuration - + +.. clicmd:: evpn mh uplink + +Proxy advertisements +"""""""""""""""""""" +To handle hitless upgrades support for proxy advertisement has been added +as specified by draft-rbickhart-evpn-ip-mac-proxy-adv. This allows a PE +(say PE1) to proxy advertise a MAC-IP rxed from an ES peer (say PE2). When +the ES peer (PE2) goes down PE1 continues to advertise hosts learnt from PE2 +for a holdtime during which it attempts to establish local reachability of +the host. This holdtime is configurable via the following zebra commands - + +.. clicmd:: evpn mh neigh-holdtime (0-86400) + +.. clicmd:: evpn mh mac-holdtime (0-86400) + +Startup delay +""""""""""""" +When a switch is rebooted we wait for a brief period to allow the underlay +and EVPN network to converge before enabling the ESs. For this duration the +ES bonds are held protodown. The startup delay is configurable via the +following zebra command - + +.. clicmd:: evpn mh startup-delay (0-3600) + +EAD-per-ES fragmentation +"""""""""""""""""""""""" +The EAD-per-ES route carries the EVI route targets for all the broadcast +domains associated with the ES. Depending on the EVI scale the EAD-per-ES +route maybe fragmented. + +The number of EVIs per-EAD route can be configured via the following +BGP command - + +.. clicmd:: [no] ead-es-frag evi-limit (1-1000) + +Sample Configuration +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: frr + + ! + router bgp 5556 + ! + address-family l2vpn evpn + ead-es-frag evi-limit 200 + exit-address-family + ! + ! + +EAD-per-ES route-target +""""""""""""""""""""""" +The EAD-per-ES route by default carries all the EVI route targets. Depending +on EVI scale that can result in route fragmentation. In some cases it maybe +necessary to avoid this fragmentation and that can be done via the following +workaround - +1. Configure a single supplementary BD per-tenant VRF. This SBD needs to +be provisioned on all EVPN PEs associated with the tenant-VRF. +2. Config the SBD's RT as the EAD-per-ES route's export RT. + +Sample Configuration +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: frr + + ! + router bgp 5556 + ! + address-family l2vpn evpn + ead-es-route-target export 5556:1001 + ead-es-route-target export 5556:1004 + ead-es-route-target export 5556:1008 + exit-address-family + ! + +Support with VRF network namespace backend +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +It is possible to separate overlay networks contained in VXLAN interfaces from +underlay networks by using VRFs. VRF-lite and VRF-netns backends can be used for +that. In the latter case, it is necessary to set both bridge and vxlan interface +in the same network namespace, as below example illustrates: + +.. code-block:: shell + + # linux shell + ip netns add vrf1 + ip link add name vxlan101 type vxlan id 101 dstport 4789 dev eth0 local 10.1.1.1 + ip link set dev vxlan101 netns vrf1 + ip netns exec vrf1 ip link set dev lo up + ip netns exec vrf1 brctl addbr bridge101 + ip netns exec vrf1 brctl addif bridge101 vxlan101 + +This makes it possible to separate not only layer 3 networks like VRF-lite networks. +Also, VRF netns based make possible to separate layer 2 networks on separate VRF +instances. + +.. _bgp-conditional-advertisement: + +BGP Conditional Advertisement +----------------------------- +The BGP conditional advertisement feature uses the ``non-exist-map`` or the +``exist-map`` and the ``advertise-map`` keywords of the neighbor advertise-map +command in order to track routes by the route prefix. + +``non-exist-map`` + 1. If a route prefix is not present in the output of non-exist-map command, + then advertise the route specified by the advertise-map command. + + 2. If a route prefix is present in the output of non-exist-map command, + then do not advertise the route specified by the addvertise-map command. + +``exist-map`` + 1. If a route prefix is present in the output of exist-map command, + then advertise the route specified by the advertise-map command. + + 2. If a route prefix is not present in the output of exist-map command, + then do not advertise the route specified by the advertise-map command. + +This feature is useful when some prefixes are advertised to one of its peers +only if the information from the other peer is not present (due to failure in +peering session or partial reachability etc). + +The conditional BGP announcements are sent in addition to the normal +announcements that a BGP router sends to its peer. + +The conditional advertisement process is triggered by the BGP scanner process, +which runs every 60 by default. This means that the maximum time for the +conditional advertisement to take effect is the value of the process timer. + +As an optimization, while the process always runs on each timer expiry, it +determines whether or not the conditional advertisement policy or the routing +table has changed; if neither have changed, no processing is necessary and the +scanner exits early. + +.. clicmd:: neighbor A.B.C.D advertise-map NAME [exist-map|non-exist-map] NAME + + This command enables BGP scanner process to monitor routes specified by + exist-map or non-exist-map command in BGP table and conditionally advertises + the routes specified by advertise-map command. + +.. clicmd:: bgp conditional-advertisement timer (5-240) + + Set the period to rerun the conditional advertisement scanner process. The + default is 60 seconds. + +Sample Configuration +^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: frr + + interface enp0s9 + ip address 10.10.10.2/24 + ! + interface enp0s10 + ip address 10.10.20.2/24 + ! + interface lo + ip address 203.0.113.1/32 + ! + router bgp 2 + bgp log-neighbor-changes + no bgp ebgp-requires-policy + neighbor 10.10.10.1 remote-as 1 + neighbor 10.10.20.3 remote-as 3 + ! + address-family ipv4 unicast + neighbor 10.10.10.1 soft-reconfiguration inbound + neighbor 10.10.20.3 soft-reconfiguration inbound + neighbor 10.10.20.3 advertise-map ADV-MAP non-exist-map EXIST-MAP + exit-address-family + ! + ip prefix-list DEFAULT seq 5 permit 192.0.2.5/32 + ip prefix-list DEFAULT seq 10 permit 192.0.2.1/32 + ip prefix-list EXIST seq 5 permit 10.10.10.10/32 + ip prefix-list DEFAULT-ROUTE seq 5 permit 0.0.0.0/0 + ip prefix-list IP1 seq 5 permit 10.139.224.0/20 + ! + bgp community-list standard DC-ROUTES seq 5 permit 64952:3008 + bgp community-list standard DC-ROUTES seq 10 permit 64671:501 + bgp community-list standard DC-ROUTES seq 15 permit 64950:3009 + bgp community-list standard DEFAULT-ROUTE seq 5 permit 65013:200 + ! + route-map ADV-MAP permit 10 + match ip address prefix-list IP1 + ! + route-map ADV-MAP permit 20 + match community DC-ROUTES + ! + route-map EXIST-MAP permit 10 + match community DEFAULT-ROUTE + match ip address prefix-list DEFAULT-ROUTE + ! + +Sample Output +^^^^^^^^^^^^^ + +When default route is present in R2'2 BGP table, 10.139.224.0/20 and 192.0.2.1/32 are not advertised to R3. + +.. code-block:: frr + + Router2# show ip bgp + BGP table version is 20, local router ID is 203.0.113.1, vrf id 0 + Default local pref 100, local AS 2 + Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, + i internal, r RIB-failure, S Stale, R Removed + Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self + Origin codes: i - IGP, e - EGP, ? - incomplete + RPKI validation codes: V valid, I invalid, N Not found + + Network Next Hop Metric LocPrf Weight Path + *> 0.0.0.0/0 10.10.10.1 0 0 1 i + *> 10.139.224.0/20 10.10.10.1 0 0 1 ? + *> 192.0.2.1/32 10.10.10.1 0 0 1 i + *> 192.0.2.5/32 10.10.10.1 0 0 1 i + + Displayed 4 routes and 4 total paths + Router2# show ip bgp neighbors 10.10.20.3 + + !--- Output suppressed. + + For address family: IPv4 Unicast + Update group 7, subgroup 7 + Packet Queue length 0 + Inbound soft reconfiguration allowed + Community attribute sent to this neighbor(all) + Condition NON_EXIST, Condition-map *EXIST-MAP, Advertise-map *ADV-MAP, status: Withdraw + 0 accepted prefixes + + !--- Output suppressed. + + Router2# show ip bgp neighbors 10.10.20.3 advertised-routes + BGP table version is 20, local router ID is 203.0.113.1, vrf id 0 + Default local pref 100, local AS 2 + Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, + i internal, r RIB-failure, S Stale, R Removed + Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self + Origin codes: i - IGP, e - EGP, ? - incomplete + RPKI validation codes: V valid, I invalid, N Not found + + Network Next Hop Metric LocPrf Weight Path + *> 0.0.0.0/0 0.0.0.0 0 1 i + *> 192.0.2.5/32 0.0.0.0 0 1 i + + Total number of prefixes 2 + +When default route is not present in R2'2 BGP table, 10.139.224.0/20 and 192.0.2.1/32 are advertised to R3. + +.. code-block:: frr + + Router2# show ip bgp + BGP table version is 21, local router ID is 203.0.113.1, vrf id 0 + Default local pref 100, local AS 2 + Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, + i internal, r RIB-failure, S Stale, R Removed + Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self + Origin codes: i - IGP, e - EGP, ? - incomplete + RPKI validation codes: V valid, I invalid, N Not found + + Network Next Hop Metric LocPrf Weight Path + *> 10.139.224.0/20 10.10.10.1 0 0 1 ? + *> 192.0.2.1/32 10.10.10.1 0 0 1 i + *> 192.0.2.5/32 10.10.10.1 0 0 1 i + + Displayed 3 routes and 3 total paths + + Router2# show ip bgp neighbors 10.10.20.3 + + !--- Output suppressed. + + For address family: IPv4 Unicast + Update group 7, subgroup 7 + Packet Queue length 0 + Inbound soft reconfiguration allowed + Community attribute sent to this neighbor(all) + Condition NON_EXIST, Condition-map *EXIST-MAP, Advertise-map *ADV-MAP, status: Advertise + 0 accepted prefixes + + !--- Output suppressed. + + Router2# show ip bgp neighbors 10.10.20.3 advertised-routes + BGP table version is 21, local router ID is 203.0.113.1, vrf id 0 + Default local pref 100, local AS 2 + Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, + i internal, r RIB-failure, S Stale, R Removed + Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self + Origin codes: i - IGP, e - EGP, ? - incomplete + RPKI validation codes: V valid, I invalid, N Not found + + Network Next Hop Metric LocPrf Weight Path + *> 10.139.224.0/20 0.0.0.0 0 1 ? + *> 192.0.2.1/32 0.0.0.0 0 1 i + *> 192.0.2.5/32 0.0.0.0 0 1 i + + Total number of prefixes 3 + Router2# + +.. _bgp-debugging: + +Debugging +--------- + +.. clicmd:: show debug + + Show all enabled debugs. + +.. clicmd:: show bgp listeners + + Display Listen sockets and the vrf that created them. Useful for debugging of when + listen is not working and this is considered a developer debug statement. + +.. clicmd:: debug bgp allow-martian + + Enable or disable BGP accepting martian nexthops from a peer. Please note + this is not an actual debug command and this command is also being deprecated + and will be removed soon. The new command is :clicmd:`bgp allow-martian-nexthop` + +.. clicmd:: debug bgp bfd + + Enable or disable debugging for BFD events. This will show BFD integration + library messages and BGP BFD integration messages that are mostly state + transitions and validation problems. + +.. clicmd:: debug bgp conditional-advertisement + + Enable or disable debugging of BGP conditional advertisement. + +.. clicmd:: debug bgp neighbor-events + + Enable or disable debugging for neighbor events. This provides general + information on BGP events such as peer connection / disconnection, session + establishment / teardown, and capability negotiation. + +.. clicmd:: debug bgp updates + + Enable or disable debugging for BGP updates. This provides information on + BGP UPDATE messages transmitted and received between local and remote + instances. + +.. clicmd:: debug bgp keepalives + + Enable or disable debugging for BGP keepalives. This provides information on + BGP KEEPALIVE messages transmitted and received between local and remote + instances. + +.. clicmd:: debug bgp bestpath <A.B.C.D/M|X:X::X:X/M> + + Enable or disable debugging for bestpath selection on the specified prefix. + +.. clicmd:: debug bgp nht + + Enable or disable debugging of BGP nexthop tracking. + +.. clicmd:: debug bgp update-groups + + Enable or disable debugging of dynamic update groups. This provides general + information on group creation, deletion, join and prune events. + +.. clicmd:: debug bgp zebra + + Enable or disable debugging of communications between *bgpd* and *zebra*. + +Dumping Messages and Routing Tables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: dump bgp all PATH [INTERVAL] + +.. clicmd:: dump bgp all-et PATH [INTERVAL] + + + Dump all BGP packet and events to `path` file. + If `interval` is set, a new file will be created for echo `interval` of + seconds. The path `path` can be set with date and time formatting + (strftime). The type ‘all-et’ enables support for Extended Timestamp Header + (:ref:`packet-binary-dump-format`). + +.. clicmd:: dump bgp updates PATH [INTERVAL] + +.. clicmd:: dump bgp updates-et PATH [INTERVAL] + + + Dump only BGP updates messages to `path` file. + If `interval` is set, a new file will be created for echo `interval` of + seconds. The path `path` can be set with date and time formatting + (strftime). The type ‘updates-et’ enables support for Extended Timestamp + Header (:ref:`packet-binary-dump-format`). + +.. clicmd:: dump bgp routes-mrt PATH + +.. clicmd:: dump bgp routes-mrt PATH INTERVAL + + + Dump whole BGP routing table to `path`. This is heavy process. The path + `path` can be set with date and time formatting (strftime). If `interval` is + set, a new file will be created for echo `interval` of seconds. + + Note: the interval variable can also be set using hours and minutes: 04h20m00. + + +.. _bgp-other-commands: + +Other BGP Commands +------------------ + +The following are available in the top level *enable* mode: + +.. clicmd:: clear bgp \* + + Clear all peers. + +.. clicmd:: clear bgp ipv4|ipv6 \* + + Clear all peers with this address-family activated. + +.. clicmd:: clear bgp ipv4|ipv6 unicast \* + + Clear all peers with this address-family and sub-address-family activated. + +.. clicmd:: clear bgp ipv4|ipv6 PEER + + Clear peers with address of X.X.X.X and this address-family activated. + +.. clicmd:: clear bgp ipv4|ipv6 unicast PEER + + Clear peer with address of X.X.X.X and this address-family and sub-address-family activated. + +.. clicmd:: clear bgp ipv4|ipv6 PEER soft|in|out + + Clear peer using soft reconfiguration in this address-family. + +.. clicmd:: clear bgp ipv4|ipv6 unicast PEER soft|in|out + + Clear peer using soft reconfiguration in this address-family and sub-address-family. + +.. clicmd:: clear bgp [ipv4|ipv6] [unicast] PEER|\* message-stats + + Clear BGP message statistics for a specified peer or for all peers, + optionally filtered by activated address-family and sub-address-family. + +The following are available in the ``router bgp`` mode: + +.. clicmd:: write-quanta (1-64) + + BGP message Tx I/O is vectored. This means that multiple packets are written + to the peer socket at the same time each I/O cycle, in order to minimize + system call overhead. This value controls how many are written at a time. + Under certain load conditions, reducing this value could make peer traffic + less 'bursty'. In practice, leave this settings on the default (64) unless + you truly know what you are doing. + +.. clicmd:: read-quanta (1-10) + + Unlike Tx, BGP Rx traffic is not vectored. Packets are read off the wire one + at a time in a loop. This setting controls how many iterations the loop runs + for. As with write-quanta, it is best to leave this setting on the default. + +The following command is available in ``config`` mode as well as in the +``router bgp`` mode: + +.. clicmd:: bgp graceful-shutdown + + The purpose of this command is to initiate BGP Graceful Shutdown which + is described in :rfc:`8326`. The use case for this is to minimize or + eliminate the amount of traffic loss in a network when a planned + maintenance activity such as software upgrade or hardware replacement + is to be performed on a router. The feature works by re-announcing + routes to eBGP peers with the GRACEFUL_SHUTDOWN community included. + Peers are then expected to treat such paths with the lowest preference. + This happens automatically on a receiver running FRR; with other + routing protocol stacks, an inbound policy may have to be configured. + In FRR, triggering graceful shutdown also results in announcing a + LOCAL_PREF of 0 to iBGP peers. + + Graceful shutdown can be configured per BGP instance or globally for + all of BGP. These two options are mutually exclusive. The no form of + the command causes graceful shutdown to be stopped, and routes will + be re-announced without the GRACEFUL_SHUTDOWN community and/or with + the usual LOCAL_PREF value. Note that if this option is saved to + the startup configuration, graceful shutdown will remain in effect + across restarts of *bgpd* and will need to be explicitly disabled. + +.. clicmd:: bgp input-queue-limit (1-4294967295) + + Set the BGP Input Queue limit for all peers when messaging parsing. Increase + this only if you have the memory to handle large queues of messages at once. + +.. clicmd:: bgp output-queue-limit (1-4294967295) + + Set the BGP Output Queue limit for all peers when messaging parsing. Increase + this only if you have the memory to handle large queues of messages at once. + +.. _bgp-displaying-bgp-information: + +Displaying BGP Information +========================== + +The following four commands display the IPv6 and IPv4 routing tables, depending +on whether or not the ``ip`` keyword is used. +Actually, :clicmd:`show ip bgp` command was used on older `Quagga` routing +daemon project, while :clicmd:`show bgp` command is the new format. The choice +has been done to keep old format with IPv4 routing table, while new format +displays IPv6 routing table. + +.. clicmd:: show ip bgp [all] [wide|json [detail]] + +.. clicmd:: show ip bgp A.B.C.D [json] + +.. clicmd:: show bgp [all] [wide|json [detail]] + +.. clicmd:: show bgp X:X::X:X [json] + + These commands display BGP routes. When no route is specified, the default + is to display all BGP routes. + + :: + + BGP table version is 0, local router ID is 10.1.1.1 + Status codes: s suppressed, d damped, h history, * valid, > best, i - internal + Origin codes: i - IGP, e - EGP, ? - incomplete + + Network Next Hop Metric LocPrf Weight Path + \*> 1.1.1.1/32 0.0.0.0 0 32768 i + + Total number of prefixes 1 + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + This is especially handy dealing with IPv6 prefixes and + if :clicmd:`[no] bgp default show-nexthop-hostname` is enabled. + + If ``all`` option is specified, ``ip`` keyword is ignored, show bgp all and + show ip bgp all commands display routes for all AFIs and SAFIs. + + If ``json`` option is specified, output is displayed in JSON format. + + If ``detail`` option is specified after ``json``, more verbose JSON output + will be displayed. + +Some other commands provide additional options for filtering the output. + +.. clicmd:: show [ip] bgp regexp LINE + + This command displays BGP routes using AS path regular expression + (:ref:`bgp-regular-expressions`). + +.. clicmd:: show [ip] bgp [all] summary [wide] [json] + + Show a bgp peer summary for the specified address family. + +The old command structure :clicmd:`show ip bgp` may be removed in the future +and should no longer be used. In order to reach the other BGP routing tables +other than the IPv6 routing table given by :clicmd:`show bgp`, the new command +structure is extended with :clicmd:`show bgp [afi] [safi]`. + +``wide`` option gives more output like ``LocalAS`` and extended ``Desc`` to +64 characters. + + .. code-block:: frr + + exit1# show ip bgp summary wide + + IPv4 Unicast Summary (VRF default): + BGP router identifier 192.168.100.1, local AS number 65534 vrf-id 0 + BGP table version 3 + RIB entries 5, using 920 bytes of memory + Peers 1, using 27 KiB of memory + + Neighbor V AS LocalAS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc + 192.168.0.2 4 65030 123 15 22 0 0 0 00:07:00 0 1 us-east1-rs1.frrouting.org + + Total number of neighbors 1 + exit1# + +If PfxRcd and/or PfxSnt is shown as ``(Policy)``, that means that the EBGP +default policy is turned on, but you don't have any filters applied for +incoming/outgoing directions. + +.. seealso:: :ref:`bgp-requires-policy` + +.. clicmd:: show bgp [afi] [safi] [all] [wide|json] + +.. clicmd:: show bgp vrfs [<VRFNAME$vrf_name>] [json] + + The command displays all bgp vrf instances basic info like router-id, + configured and established neighbors, + evpn related basic info like l3vni, router-mac, vxlan-interface. + User can get that information as JSON format when ``json`` keyword + at the end of cli is presented. + + .. code-block:: frr + + torc-11# show bgp vrfs + Type Id routerId #PeersCfg #PeersEstb Name + L3-VNI RouterMAC Interface + DFLT 0 17.0.0.6 3 3 default + 0 00:00:00:00:00:00 unknown + VRF 21 17.0.0.6 0 0 sym_1 + 8888 34:11:12:22:22:01 vlan4034_l3 + VRF 32 17.0.0.6 0 0 sym_2 + 8889 34:11:12:22:22:01 vlan4035_l3 + + Total number of VRFs (including default): 3 + +.. clicmd:: show bgp [<ipv4|ipv6> <unicast|multicast|vpn|labeled-unicast|flowspec> | l2vpn evpn] + + These commands display BGP routes for the specific routing table indicated by + the selected afi and the selected safi. If no afi and no safi value is given, + the command falls back to the default IPv6 routing table. + +.. clicmd:: show bgp l2vpn evpn route [type <macip|2|multicast|3|es|4|prefix|5>] + + EVPN prefixes can also be filtered by EVPN route type. + +.. clicmd:: show bgp l2vpn evpn route [detail] [type <ead|1|macip|2|multicast|3|es|4|prefix|5>] self-originate [json] + + Display self-originated EVPN prefixes which can also be filtered by EVPN route type. + +.. clicmd:: show bgp vni <all|VNI> [vtep VTEP] [type <ead|1|macip|2|multicast|3>] [<detail|json>] + + Display per-VNI EVPN routing table in bgp. Filter route-type, vtep, or VNI. + +.. clicmd:: show bgp [afi] [safi] [all] summary [json] + + Show a bgp peer summary for the specified address family, and subsequent + address-family. + +.. clicmd:: show bgp [afi] [safi] [all] summary failed [json] + + Show a bgp peer summary for peers that are not successfully exchanging routes + for the specified address family, and subsequent address-family. + +.. clicmd:: show bgp [afi] [safi] [all] summary established [json] + + Show a bgp peer summary for peers that are successfully exchanging routes + for the specified address family, and subsequent address-family. + +.. clicmd:: show bgp [afi] [safi] [all] summary neighbor [PEER] [json] + + Show a bgp summary for the specified peer, address family, and + subsequent address-family. The neighbor filter can be used in combination + with the failed, established filters. + +.. clicmd:: show bgp [afi] [safi] [all] summary remote-as <internal|external|ASN> [json] + + Show a bgp peer summary for the specified remote-as ASN or type (``internal`` + for iBGP and ``external`` for eBGP sessions), address family, and subsequent + address-family. The remote-as filter can be used in combination with the + failed, established filters. + +.. clicmd:: show bgp [afi] [safi] [all] summary terse [json] + + Shorten the output. Do not show the following information about the BGP + instances: the number of RIB entries, the table version and the used memory. + The ``terse`` option can be used in combination with the remote-as, neighbor, + failed and established filters, and with the ``wide`` option as well. + +.. clicmd:: show bgp [afi] [safi] [neighbor [PEER] [routes|advertised-routes|received-routes] [<A.B.C.D/M|X:X::X:X/M> | detail] [json] + + This command shows information on a specific BGP peer of the relevant + afi and safi selected. + + The ``routes`` keyword displays only routes in this address-family's BGP + table that were received by this peer and accepted by inbound policy. + + The ``advertised-routes`` keyword displays only the routes in this + address-family's BGP table that were permitted by outbound policy and + advertised to to this peer. + + The ``received-routes`` keyword displays all routes belonging to this + address-family (prior to inbound policy) that were received by this peer. + + If a specific prefix is specified, the detailed version of that prefix will + be displayed. + + If ``detail`` option is specified, the detailed version of all routes + will be displayed. The same format as ``show [ip] bgp [afi] [safi] PREFIX`` + will be used, but for the whole table of received, advertised or filtered + prefixes. + + If ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show bgp [<view|vrf> VIEWVRFNAME] [afi] [safi] neighbors PEER received prefix-filter [json] + + Display Address Prefix ORFs received from this peer. + +.. clicmd:: show bgp [afi] [safi] [all] dampening dampened-paths [wide|json] + + Display paths suppressed due to dampening of the selected afi and safi + selected. + +.. clicmd:: show bgp [afi] [safi] [all] dampening flap-statistics [wide|json] + + Display flap statistics of routes of the selected afi and safi selected. + +.. clicmd:: show bgp [afi] [safi] [all] dampening parameters [json] + + Display details of configured dampening parameters of the selected afi and + safi. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show bgp [afi] [safi] [all] version (1-4294967295) [wide|json] + + Display prefixes with matching version numbers. The version number and + above having prefixes will be listed here. + + It helps to identify which prefixes were installed at some point. + + Here is an example of how to check what prefixes were installed starting + with an arbitrary version: + +.. code-block:: shell + + # vtysh -c 'show bgp ipv4 unicast json' | jq '.tableVersion' + 9 + # vtysh -c 'show ip bgp version 9 json' | jq -r '.routes | keys[]' + 192.168.3.0/24 + # vtysh -c 'show ip bgp version 8 json' | jq -r '.routes | keys[]' + 192.168.2.0/24 + 192.168.3.0/24 + +.. clicmd:: show bgp [afi] [safi] statistics + + Display statistics of routes of the selected afi and safi. + +.. clicmd:: show bgp statistics-all + + Display statistics of routes of all the afi and safi. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] cidr-only [wide|json] + + Display routes with non-natural netmasks. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] prefix-list WORD [wide|json] + + Display routes that match the specified prefix-list. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] access-list WORD [wide|json] + + Display routes that match the specified access-list. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] filter-list WORD [wide|json] + + Display routes that match the specified AS-Path filter-list. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] route-map WORD [wide|json] + + Display routes that match the specified route-map. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] <A.B.C.D/M|X:X::X:X/M> longer-prefixes [wide|json] + + Displays the specified route and all more specific routes. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] self-originate [wide|json] + + Display self-originated routes. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] neighbors A.B.C.D [advertised-routes|received-routes|filtered-routes] [<A.B.C.D/M|X:X::X:X/M> | detail] [json|wide] + + Display the routes advertised to a BGP neighbor or received routes + from neighbor or filtered routes received from neighbor based on the + option specified. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + This is especially handy dealing with IPv6 prefixes and + if :clicmd:`[no] bgp default show-nexthop-hostname` is enabled. + + If ``all`` option is specified, ``ip`` keyword is ignored and, + routes displayed for all AFIs and SAFIs. + if afi is specified, with ``all`` option, routes will be displayed for + each SAFI in the selcted AFI + + If a specific prefix is specified, the detailed version of that prefix will + be displayed. + + If ``detail`` option is specified, the detailed version of all routes + will be displayed. The same format as ``show [ip] bgp [afi] [safi] PREFIX`` + will be used, but for the whole table of received, advertised or filtered + prefixes. + + If ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show [ip] bgp [afi] [safi] [all] detail-routes + + Display the detailed version of all routes. The same format as using + ``show [ip] bgp [afi] [safi] PREFIX``, but for the whole BGP table. + + If ``all`` option is specified, ``ip`` keyword is ignored and, + routes displayed for all AFIs and SAFIs. + + If ``afi`` is specified, with ``all`` option, routes will be displayed for + each SAFI in the selected AFI. + +.. clicmd:: show [ip] bgp [<view|vrf> VIEWVRFNAME] [afi] [safi] detail [json] + + Display the detailed version of all routes from the specified bgp vrf table + for a given afi + safi. + + If no vrf is specified, then it is assumed as a default vrf and routes + are displayed from default vrf table. + + If ``all`` option is specified as vrf name, then all bgp vrf tables routes + from a given afi+safi are displayed in the detailed output of routes. + + If ``json`` option is specified, detailed output is displayed in JSON format. + + Following are sample output for few examples of how to use this command. + +.. code-block:: frr + + torm-23# sh bgp ipv4 unicast detail (OR) sh bgp vrf default ipv4 unicast detail + + !--- Output suppressed. + + BGP routing table entry for 172.16.16.1/32 + Paths: (1 available, best #1, table default) + Not advertised to any peer + Local, (Received from a RR-client) + 172.16.16.1 (metric 20) from torm-22(172.16.16.1) (192.168.0.10) + Origin IGP, metric 0, localpref 100, valid, internal + Last update: Fri May 8 12:54:05 2023 + BGP routing table entry for 172.16.16.2/32 + Paths: (1 available, best #1, table default) + Not advertised to any peer + Local + 0.0.0.0 from 0.0.0.0 (172.16.16.2) + Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (First path received) + Last update: Wed May 8 12:54:41 2023 + + Displayed 2 routes and 2 total paths + +.. code-block:: frr + + torm-23# sh bgp vrf all detail + + Instance default: + + !--- Output suppressed. + + BGP routing table entry for 172.16.16.1/32 + Paths: (1 available, best #1, table default) + Not advertised to any peer + Local, (Received from a RR-client) + 172.16.16.1 (metric 20) from torm-22(172.16.16.1) (192.168.0.10) + Origin IGP, metric 0, localpref 100, valid, internal + Last update: Fri May 8 12:44:05 2023 + BGP routing table entry for 172.16.16.2/32 + Paths: (1 available, best #1, table default) + Not advertised to any peer + Local + 0.0.0.0 from 0.0.0.0 (172.16.16.2) + Origin incomplete, metric 0, weight 32768, valid, sourced, bestpath-from-AS Local, best (First path received) + Last update: Wed May 8 12:45:01 2023 + + Displayed 2 routes and 2 total paths + + Instance vrf3: + + !--- Output suppressed. + + BGP routing table entry for 192.168.0.2/32 + Paths: (1 available, best #1, vrf vrf3) + Not advertised to any peer + Imported from 172.16.16.1:12:[2]:[0]:[48]:[00:02:00:00:00:58]:[32]:[192.168.0.2], VNI 1008/4003 + Local + 172.16.16.1 from torm-22(172.16.16.1) (172.16.16.1) announce-nh-self + Origin IGP, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received) + Extended Community: RT:65000:1008 ET:8 Rmac:00:02:00:00:00:58 + Last update: Fri May 8 02:41:55 2023 + BGP routing table entry for 192.168.1.2/32 + Paths: (1 available, best #1, vrf vrf3) + Not advertised to any peer + Imported from 172.16.16.1:13:[2]:[0]:[48]:[00:02:00:00:00:58]:[32]:[192.168.1.2], VNI 1009/4003 + Local + 172.16.16.1 from torm-22(172.16.16.1) (172.16.16.1) announce-nh-self + Origin IGP, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received) + Extended Community: RT:65000:1009 ET:8 Rmac:00:02:00:00:00:58 + Last update: Fri May 8 02:41:55 2023 + + Displayed 2 routes and 2 total paths + + +.. code-block:: frr + + torm-23# sh bgp vrf vrf3 ipv4 unicast detail + + !--- Output suppressed. + + BGP routing table entry for 192.168.0.2/32 + Paths: (1 available, best #1, vrf vrf3) + Not advertised to any peer + Imported from 172.16.16.1:12:[2]:[0]:[48]:[00:02:00:00:00:58]:[32]:[192.168.0.2], VNI 1008/4003 + Local + 172.16.16.1 from torm-22(172.16.16.1) (172.16.16.1) announce-nh-self + Origin IGP, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received) + Extended Community: RT:65000:1008 ET:8 Rmac:00:02:00:00:00:58 + Last update: Fri May 8 02:23:35 2023 + BGP routing table entry for 192.168.1.2/32 + Paths: (1 available, best #1, vrf vrf3) + Not advertised to any peer + Imported from 172.16.16.1:13:[2]:[0]:[48]:[00:02:00:00:00:58]:[32]:[192.168.1.2], VNI 1009/4003 + Local + 172.16.16.1 from torm-22(172.16.16.1) (172.16.16.1) announce-nh-self + Origin IGP, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received) + Extended Community: RT:65000:1009 ET:8 Rmac:00:02:00:00:00:58 + Last update: Fri May 8 02:23:55 2023 + + Displayed 2 routes and 2 total paths + +.. _bgp-display-routes-by-community: + +Displaying Routes by Community Attribute +---------------------------------------- + +The following commands allow displaying routes based on their community +attribute. + +.. clicmd:: show [ip] bgp <ipv4|ipv6> [all] community [wide|json] + +.. clicmd:: show [ip] bgp <ipv4|ipv6> [all] community COMMUNITY [wide|json] + +.. clicmd:: show [ip] bgp <ipv4|ipv6> [all] community COMMUNITY exact-match [wide|json] + + These commands display BGP routes which have the community attribute. + attribute. When ``COMMUNITY`` is specified, BGP routes that match that + community are displayed. When `exact-match` is specified, it display only + routes that have an exact match. + +.. clicmd:: show [ip] bgp <ipv4|ipv6> community-list WORD [json] + +.. clicmd:: show [ip] bgp <ipv4|ipv6> community-list WORD exact-match [json] + + These commands display BGP routes for the address family specified that + match the specified community list. When `exact-match` is specified, it + displays only routes that have an exact match. + + If ``wide`` option is specified, then the prefix table's width is increased + to fully display the prefix and the nexthop. + + This is especially handy dealing with IPv6 prefixes and + if :clicmd:`[no] bgp default show-nexthop-hostname` is enabled. + + If ``all`` option is specified, ``ip`` keyword is ignored and, + routes displayed for all AFIs and SAFIs. + if afi is specified, with ``all`` option, routes will be displayed for + each SAFI in the selcted AFI + + If ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show bgp labelpool <chunks|inuse|ledger|requests|summary> [json] + + These commands display information about the BGP labelpool used for + the association of MPLS labels with routes for L3VPN and Labeled Unicast + + If ``chunks`` option is specified, output shows the current list of label + chunks granted to BGP by Zebra, indicating the start and end label in + each chunk + + If ``inuse`` option is specified, output shows the current inuse list of + label to prefix mappings + + If ``ledger`` option is specified, output shows ledger list of all + label requests made per prefix + + If ``requests`` option is specified, output shows current list of label + requests which have not yet been fulfilled by the labelpool + + If ``summary`` option is specified, output is a summary of the counts for + the chunks, inuse, ledger and requests list along with the count of + outstanding chunk requests to Zebra and the number of zebra reconnects + that have happened + + If ``json`` option is specified, output is displayed in JSON format. + +.. _bgp-display-routes-by-lcommunity: + +Displaying Routes by Large Community Attribute +---------------------------------------------- + +The following commands allow displaying routes based on their +large community attribute. + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community LARGE-COMMUNITY + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community LARGE-COMMUNITY exact-match + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community LARGE-COMMUNITY json + + These commands display BGP routes which have the large community attribute. + attribute. When ``LARGE-COMMUNITY`` is specified, BGP routes that match that + large community are displayed. When `exact-match` is specified, it display + only routes that have an exact match. When `json` is specified, it display + routes in json format. + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community-list WORD + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community-list WORD exact-match + +.. clicmd:: show [ip] bgp <ipv4|ipv6> large-community-list WORD json + + These commands display BGP routes for the address family specified that + match the specified large community list. When `exact-match` is specified, + it displays only routes that have an exact match. When `json` is specified, + it display routes in json format. + +.. _bgp-display-routes-by-as-path: + + +Displaying Routes by AS Path +---------------------------- + +.. clicmd:: show bgp ipv4|ipv6 regexp LINE + + This commands displays BGP routes that matches a regular + expression `line` (:ref:`bgp-regular-expressions`). + +.. clicmd:: show [ip] bgp ipv4 vpn + +.. clicmd:: show [ip] bgp ipv6 vpn + + Print active IPV4 or IPV6 routes advertised via the VPN SAFI. + +.. clicmd:: show bgp ipv4 vpn summary + +.. clicmd:: show bgp ipv6 vpn summary + + Print a summary of neighbor connections for the specified AFI/SAFI combination. + +Displaying Routes by Route Distinguisher +---------------------------------------- + +.. clicmd:: show bgp [<ipv4|ipv6> vpn | l2vpn evpn [route]] rd <all|RD> + + For L3VPN and EVPN address-families, routes can be displayed on a per-RD + (Route Distinguisher) basis or for all RD's. + +.. clicmd:: show bgp l2vpn evpn rd <all|RD> [overlay | tags] + + Use the ``overlay`` or ``tags`` keywords to display the overlay/tag + information about the EVPN prefixes in the selected Route Distinguisher. + +.. clicmd:: show bgp l2vpn evpn route rd <all|RD> mac <MAC> [ip <MAC>] [json] + + For EVPN Type 2 (macip) routes, a MAC address (and optionally an IP address) + can be supplied to the command to only display matching prefixes in the + specified RD. + +Displaying Update Group Information +----------------------------------- + +.. clicmd:: show bgp update-groups [advertise-queue|advertised-routes|packet-queue] + + Display Information about each individual update-group being used. + If SUBGROUP-ID is specified only display about that particular group. If + advertise-queue is specified the list of routes that need to be sent + to the peers in the update-group is displayed, advertised-routes means + the list of routes we have sent to the peers in the update-group and + packet-queue specifies the list of packets in the queue to be sent. + +.. clicmd:: show bgp update-groups statistics + + Display Information about update-group events in FRR. + +Displaying Nexthop Information +------------------------------ +.. clicmd:: show [ip] bgp [<view|vrf> VIEWVRFNAME] nexthop ipv4 [A.B.C.D] [detail] [json] + +.. clicmd:: show [ip] bgp [<view|vrf> VIEWVRFNAME] nexthop ipv6 [X:X::X:X] [detail] [json] + +.. clicmd:: show [ip] bgp [<view|vrf> VIEWVRFNAME] nexthop [<A.B.C.D|X:X::X:X>] [detail] [json] + +.. clicmd:: show [ip] bgp <view|vrf> all nexthop [json] + + Display information about nexthops to bgp neighbors. If a certain nexthop is + specified, also provides information about paths associated with the nexthop. + With detail option provides information about gates of each nexthop. + +.. clicmd:: show [ip] bgp [<view|vrf> VIEWVRFNAME] import-check-table [detail] [json] + + Display information about nexthops from table that is used to check network's + existence in the rib for network statements. + +Segment-Routing IPv6 +-------------------- + +.. clicmd:: show bgp segment-routing srv6 + + This command displays information about SRv6 L3VPN in bgpd. Specifically, + what kind of Locator is being used, and its Locator chunk information. + And the SID of the SRv6 Function that is actually managed on bgpd. + In the following example, bgpd is using a Locator named loc1, and two SRv6 + Functions are managed to perform VPNv6 VRF redirect for vrf10 and vrf20. + +:: + + router# show bgp segment-routing srv6 + locator_name: loc1 + locator_chunks: + - 2001:db8:1:1::/64 + functions: + - sid: 2001:db8:1:1::100 + locator: loc1 + - sid: 2001:db8:1:1::200 + locator: loc1 + bgps: + - name: default + vpn_policy[AFI_IP].tovpn_sid: none + vpn_policy[AFI_IP6].tovpn_sid: none + - name: vrf10 + vpn_policy[AFI_IP].tovpn_sid: none + vpn_policy[AFI_IP6].tovpn_sid: 2001:db8:1:1::100 + - name: vrf20 + vpn_policy[AFI_IP].tovpn_sid: none + vpn_policy[AFI_IP6].tovpn_sid: 2001:db8:1:1::200 + +AS-notation support +------------------- + +By default, the ASN value output follows how the BGP ASN instance is +expressed in the configuration. Three as-notation outputs are available: + +- plain output: both AS4B and AS2B use a single number. + ` router bgp 65536`. + +- dot output: AS4B values are using two numbers separated by a period. + `router bgp 1.1` means that the AS number is 65536. + +- dot+ output: AS2B and AS4B values are using two numbers separated by a + period. `router bgp 0.5` means that the AS number is 5. + +The below option permits forcing the as-notation output: + +.. clicmd:: router bgp ASN as-notation dot|dot+|plain + + The chosen as-notation format will override the BGP ASN output. + +.. _bgp-route-reflector: + +Route Reflector +=============== + +BGP routers connected inside the same AS through BGP belong to an internal +BGP session, or IBGP. In order to prevent routing table loops, IBGP does not +advertise IBGP-learned routes to other routers in the same session. As such, +IBGP requires a full mesh of all peers. For large networks, this quickly becomes +unscalable. Introducing route reflectors removes the need for the full-mesh. + +When route reflectors are configured, these will reflect the routes announced +by the peers configured as clients. A route reflector client is configured +with: + +.. clicmd:: neighbor PEER route-reflector-client + + +To avoid single points of failure, multiple route reflectors can be configured. + +A cluster is a collection of route reflectors and their clients, and is used +by route reflectors to avoid looping. + +.. clicmd:: bgp cluster-id A.B.C.D + +.. clicmd:: bgp no-rib + +To set and unset the BGP daemon ``-n`` / ``--no_kernel`` options during runtime +to disable BGP route installation to the RIB (Zebra), the ``[no] bgp no-rib`` +commands can be used; + +Please note that setting the option during runtime will withdraw all routes in +the daemons RIB from Zebra and unsetting it will announce all routes in the +daemons RIB to Zebra. If the option is passed as a command line argument when +starting the daemon and the configuration gets saved, the option will persist +unless removed from the configuration with the negating command prior to the +configuration write operation. At this point in time non SAFI_UNICAST BGP +data is not properly withdrawn from zebra when this command is issued. + +.. clicmd:: bgp allow-martian-nexthop + +When a peer receives a martian nexthop as part of the NLRI for a route +permit the nexthop to be used as such, instead of rejecting and resetting +the connection. + +.. clicmd:: bgp send-extra-data zebra + +This command turns on the ability of BGP to send extra data to zebra. Currently, +it's the AS-Path, communities, and the path selection reason. The default +behavior in BGP is not to send this data. If the routes were sent to zebra and +the option is changed, bgpd doesn't reinstall the routes to comply with the new +setting. + +.. clicmd:: bgp session-dscp (0-63) + +This command allows bgp to control, at a global level, the TCP dscp values +in the TCP header. + +.. _bgp-suppress-fib: + +Suppressing routes not installed in FIB +======================================= + +The FRR implementation of BGP advertises prefixes learnt from a peer to other +peers even if the routes do not get installed in the FIB. There can be +scenarios where the hardware tables in some of the routers (along the path from +the source to destination) is full which will result in all routes not getting +installed in the FIB. If these routes are advertised to the downstream routers +then traffic will start flowing and will be dropped at the intermediate router. + +The solution is to provide a configurable option to check for the FIB install +status of the prefixes and advertise to peers if the prefixes are successfully +installed in the FIB. The advertisement of the prefixes are suppressed if it is +not installed in FIB. + +The following conditions apply will apply when checking for route installation +status in FIB: + +1. The advertisement or suppression of routes based on FIB install status + applies only for newly learnt routes from peer (routes which are not in + BGP local RIB). +2. If the route received from peer already exists in BGP local RIB and route + attributes have changed (best path changed), the old path is deleted and + new path is installed in FIB. The FIB install status will not have any + effect. Therefore only when the route is received first time the checks + apply. +3. The feature will not apply for routes learnt through other means like + redistribution to bgp from other protocols. This is applicable only to + peer learnt routes. +4. If a route is installed in FIB and then gets deleted from the dataplane, + then routes will not be withdrawn from peers. This will be considered as + dataplane issue. +5. The feature will slightly increase the time required to advertise the routes + to peers since the route install status needs to be received from the FIB +6. If routes are received by the peer before the configuration is applied, then + the bgp sessions need to be reset for the configuration to take effect. +7. If the route which is already installed in dataplane is removed for some + reason, sending withdraw message to peers is not currently supported. + +.. clicmd:: bgp suppress-fib-pending + + This command is applicable at the global level and at an individual + bgp level. If applied at the global level all bgp instances will + wait for fib installation before announcing routes and there is no + way to turn it off for a particular bgp vrf. + +.. _routing-policy: + +Routing Policy +============== + +You can set different routing policy for a peer. For example, you can set +different filter for a peer. + +.. code-block:: frr + + ! + router bgp 1 view 1 + neighbor 10.0.0.1 remote-as 2 + address-family ipv4 unicast + neighbor 10.0.0.1 distribute-list 1 in + exit-address-family + ! + router bgp 1 view 2 + neighbor 10.0.0.1 remote-as 2 + address-family ipv4 unicast + neighbor 10.0.0.1 distribute-list 2 in + exit-address-family + +This means BGP update from a peer 10.0.0.1 goes to both BGP view 1 and view 2. +When the update is inserted into view 1, distribute-list 1 is applied. On the +other hand, when the update is inserted into view 2, distribute-list 2 is +applied. + + +.. _bgp-regular-expressions: + +BGP Regular Expressions +======================= + +BGP regular expressions are based on :t:`POSIX 1003.2` regular expressions. The +following description is just a quick subset of the POSIX regular expressions. + + +.\* + Matches any single character. + +\* + Matches 0 or more occurrences of pattern. + +\+ + Matches 1 or more occurrences of pattern. + +? + Match 0 or 1 occurrences of pattern. + +^ + Matches the beginning of the line. + +$ + Matches the end of the line. + +_ + The ``_`` character has special meanings in BGP regular expressions. It + matches to space and comma , and AS set delimiter ``{`` and ``}`` and AS + confederation delimiter ``(`` and ``)``. And it also matches to the + beginning of the line and the end of the line. So ``_`` can be used for AS + value boundaries match. This character technically evaluates to + ``(^|[,{}()]|$)``. + + +.. _bgp-configuration-examples: + +Miscellaneous Configuration Examples +==================================== + +Example of a session to an upstream, advertising only one prefix to it. + +.. code-block:: frr + + router bgp 64512 + bgp router-id 10.236.87.1 + neighbor upstream peer-group + neighbor upstream remote-as 64515 + neighbor upstream capability dynamic + neighbor 10.1.1.1 peer-group upstream + neighbor 10.1.1.1 description ACME ISP + + address-family ipv4 unicast + network 10.236.87.0/24 + neighbor upstream prefix-list pl-allowed-adv out + exit-address-family + ! + ip prefix-list pl-allowed-adv seq 5 permit 82.195.133.0/25 + ip prefix-list pl-allowed-adv seq 10 deny any + +A more complex example including upstream, peer and customer sessions +advertising global prefixes and NO_EXPORT prefixes and providing actions for +customer routes based on community values. Extensive use is made of route-maps +and the 'call' feature to support selective advertising of prefixes. This +example is intended as guidance only, it has NOT been tested and almost +certainly contains silly mistakes, if not serious flaws. + +.. code-block:: frr + + router bgp 64512 + bgp router-id 10.236.87.1 + neighbor upstream capability dynamic + neighbor cust capability dynamic + neighbor peer capability dynamic + neighbor 10.1.1.1 remote-as 64515 + neighbor 10.1.1.1 peer-group upstream + neighbor 10.2.1.1 remote-as 64516 + neighbor 10.2.1.1 peer-group upstream + neighbor 10.3.1.1 remote-as 64517 + neighbor 10.3.1.1 peer-group cust-default + neighbor 10.3.1.1 description customer1 + neighbor 10.4.1.1 remote-as 64518 + neighbor 10.4.1.1 peer-group cust + neighbor 10.4.1.1 description customer2 + neighbor 10.5.1.1 remote-as 64519 + neighbor 10.5.1.1 peer-group peer + neighbor 10.5.1.1 description peer AS 1 + neighbor 10.6.1.1 remote-as 64520 + neighbor 10.6.1.1 peer-group peer + neighbor 10.6.1.1 description peer AS 2 + + address-family ipv4 unicast + network 10.123.456.0/24 + network 10.123.456.128/25 route-map rm-no-export + neighbor upstream route-map rm-upstream-out out + neighbor cust route-map rm-cust-in in + neighbor cust route-map rm-cust-out out + neighbor cust send-community both + neighbor peer route-map rm-peer-in in + neighbor peer route-map rm-peer-out out + neighbor peer send-community both + neighbor 10.3.1.1 prefix-list pl-cust1-network in + neighbor 10.4.1.1 prefix-list pl-cust2-network in + neighbor 10.5.1.1 prefix-list pl-peer1-network in + neighbor 10.6.1.1 prefix-list pl-peer2-network in + exit-address-family + ! + ip prefix-list pl-default permit 0.0.0.0/0 + ! + ip prefix-list pl-upstream-peers permit 10.1.1.1/32 + ip prefix-list pl-upstream-peers permit 10.2.1.1/32 + ! + ip prefix-list pl-cust1-network permit 10.3.1.0/24 + ip prefix-list pl-cust1-network permit 10.3.2.0/24 + ! + ip prefix-list pl-cust2-network permit 10.4.1.0/24 + ! + ip prefix-list pl-peer1-network permit 10.5.1.0/24 + ip prefix-list pl-peer1-network permit 10.5.2.0/24 + ip prefix-list pl-peer1-network permit 192.168.0.0/24 + ! + ip prefix-list pl-peer2-network permit 10.6.1.0/24 + ip prefix-list pl-peer2-network permit 10.6.2.0/24 + ip prefix-list pl-peer2-network permit 192.168.1.0/24 + ip prefix-list pl-peer2-network permit 192.168.2.0/24 + ip prefix-list pl-peer2-network permit 172.16.1/24 + ! + bgp as-path access-list seq 5 asp-own-as permit ^$ + bgp as-path access-list seq 10 asp-own-as permit _64512_ + ! + ! ################################################################# + ! Match communities we provide actions for, on routes receives from + ! customers. Communities values of <our-ASN>:X, with X, have actions: + ! + ! 100 - blackhole the prefix + ! 200 - set no_export + ! 300 - advertise only to other customers + ! 400 - advertise only to upstreams + ! 500 - set no_export when advertising to upstreams + ! 2X00 - set local_preference to X00 + ! + ! blackhole the prefix of the route + bgp community-list standard cm-blackhole permit 64512:100 + ! + ! set no-export community before advertising + bgp community-list standard cm-set-no-export permit 64512:200 + ! + ! advertise only to other customers + bgp community-list standard cm-cust-only permit 64512:300 + ! + ! advertise only to upstreams + bgp community-list standard cm-upstream-only permit 64512:400 + ! + ! advertise to upstreams with no-export + bgp community-list standard cm-upstream-noexport permit 64512:500 + ! + ! set local-pref to least significant 3 digits of the community + bgp community-list standard cm-prefmod-100 permit 64512:2100 + bgp community-list standard cm-prefmod-200 permit 64512:2200 + bgp community-list standard cm-prefmod-300 permit 64512:2300 + bgp community-list standard cm-prefmod-400 permit 64512:2400 + bgp community-list expanded cme-prefmod-range permit 64512:2... + ! + ! Informational communities + ! + ! 3000 - learned from upstream + ! 3100 - learned from customer + ! 3200 - learned from peer + ! + bgp community-list standard cm-learnt-upstream permit 64512:3000 + bgp community-list standard cm-learnt-cust permit 64512:3100 + bgp community-list standard cm-learnt-peer permit 64512:3200 + ! + ! ################################################################### + ! Utility route-maps + ! + ! These utility route-maps generally should not used to permit/deny + ! routes, i.e. they do not have meaning as filters, and hence probably + ! should be used with 'on-match next'. These all finish with an empty + ! permit entry so as not interfere with processing in the caller. + ! + route-map rm-no-export permit 10 + set community additive no-export + route-map rm-no-export permit 20 + ! + route-map rm-blackhole permit 10 + description blackhole, up-pref and ensure it cannot escape this AS + set ip next-hop 127.0.0.1 + set local-preference 10 + set community additive no-export + route-map rm-blackhole permit 20 + ! + ! Set local-pref as requested + route-map rm-prefmod permit 10 + match community cm-prefmod-100 + set local-preference 100 + route-map rm-prefmod permit 20 + match community cm-prefmod-200 + set local-preference 200 + route-map rm-prefmod permit 30 + match community cm-prefmod-300 + set local-preference 300 + route-map rm-prefmod permit 40 + match community cm-prefmod-400 + set local-preference 400 + route-map rm-prefmod permit 50 + ! + ! Community actions to take on receipt of route. + route-map rm-community-in permit 10 + description check for blackholing, no point continuing if it matches. + match community cm-blackhole + call rm-blackhole + route-map rm-community-in permit 20 + match community cm-set-no-export + call rm-no-export + on-match next + route-map rm-community-in permit 30 + match community cme-prefmod-range + call rm-prefmod + route-map rm-community-in permit 40 + ! + ! ##################################################################### + ! Community actions to take when advertising a route. + ! These are filtering route-maps, + ! + ! Deny customer routes to upstream with cust-only set. + route-map rm-community-filt-to-upstream deny 10 + match community cm-learnt-cust + match community cm-cust-only + route-map rm-community-filt-to-upstream permit 20 + ! + ! Deny customer routes to other customers with upstream-only set. + route-map rm-community-filt-to-cust deny 10 + match community cm-learnt-cust + match community cm-upstream-only + route-map rm-community-filt-to-cust permit 20 + ! + ! ################################################################### + ! The top-level route-maps applied to sessions. Further entries could + ! be added obviously.. + ! + ! Customers + route-map rm-cust-in permit 10 + call rm-community-in + on-match next + route-map rm-cust-in permit 20 + set community additive 64512:3100 + route-map rm-cust-in permit 30 + ! + route-map rm-cust-out permit 10 + call rm-community-filt-to-cust + on-match next + route-map rm-cust-out permit 20 + ! + ! Upstream transit ASes + route-map rm-upstream-out permit 10 + description filter customer prefixes which are marked cust-only + call rm-community-filt-to-upstream + on-match next + route-map rm-upstream-out permit 20 + description only customer routes are provided to upstreams/peers + match community cm-learnt-cust + ! + ! Peer ASes + ! outbound policy is same as for upstream + route-map rm-peer-out permit 10 + call rm-upstream-out + ! + route-map rm-peer-in permit 10 + set community additive 64512:3200 + + +Example of how to set up a 6-Bone connection. + +.. code-block:: frr + + ! bgpd configuration + ! ================== + ! + ! MP-BGP configuration + ! + router bgp 7675 + bgp router-id 10.0.0.1 + neighbor 3ffe:1cfa:0:2:2a0:c9ff:fe9e:f56 remote-as `as-number` + ! + address-family ipv6 + network 3ffe:506::/32 + neighbor 3ffe:1cfa:0:2:2a0:c9ff:fe9e:f56 activate + neighbor 3ffe:1cfa:0:2:2a0:c9ff:fe9e:f56 route-map set-nexthop out + neighbor 3ffe:1cfa:0:2:2c0:4fff:fe68:a231 remote-as `as-number` + neighbor 3ffe:1cfa:0:2:2c0:4fff:fe68:a231 route-map set-nexthop out + exit-address-family + ! + ipv6 access-list all permit any + ! + ! Set output nexthop address. + ! + route-map set-nexthop permit 10 + match ipv6 address all + set ipv6 nexthop global 3ffe:1cfa:0:2:2c0:4fff:fe68:a225 + set ipv6 nexthop local fe80::2c0:4fff:fe68:a225 + ! + log file bgpd.log + ! + +.. _bgp-tcp-mss: + +BGP tcp-mss support +=================== +TCP provides a mechanism for the user to specify the max segment size. +setsockopt API is used to set the max segment size for TCP session. We +can configure this as part of BGP neighbor configuration. + +This document explains how to avoid ICMP vulnerability issues by limiting +TCP max segment size when you are using MTU discovery. Using MTU discovery +on TCP paths is one method of avoiding BGP packet fragmentation. + +TCP negotiates a maximum segment size (MSS) value during session connection +establishment between two peers. The MSS value negotiated is primarily based +on the maximum transmission unit (MTU) of the interfaces to which the +communicating peers are directly connected. However, due to variations in +link MTU on the path taken by the TCP packets, some packets in the network +that are well within the MSS value might be fragmented when the packet size +exceeds the link's MTU. + +This feature is supported with TCP over IPv4 and TCP over IPv6. + +CLI Configuration: +------------------ +Below configuration can be done in router bgp mode and allows the user to +configure the tcp-mss value per neighbor. The configuration gets applied +only after hard reset is performed on that neighbor. If we configure tcp-mss +on both the neighbors then both neighbors need to be reset. + +The configuration takes effect based on below rules, so there is a configured +tcp-mss and a synced tcp-mss value per TCP session. + +By default if the configuration is not done then the TCP max segment size is +set to the Maximum Transmission unit (MTU) – (IP/IP6 header size + TCP header +size + ethernet header). For IPv4 its MTU – (20 bytes IP header + 20 bytes TCP +header + 12 bytes ethernet header) and for IPv6 its MTU – (40 bytes IPv6 header ++ 20 bytes TCP header + 12 bytes ethernet header). + +If the config is done then it reduces 12-14 bytes for the ether header and +uses it after synchronizing in TCP handshake. + +.. clicmd:: neighbor <A.B.C.D|X:X::X:X|WORD> tcp-mss (1-65535) + +When tcp-mss is configured kernel reduces 12-14 bytes for ethernet header. +E.g. if tcp-mss is configured as 150 the synced value will be 138. + +Note: configured and synced value is different since TCP module will reduce +12 bytes for ethernet header. + +Running config: +--------------- + +.. code-block:: frr + + frr# show running-config + Building configuration... + + Current configuration: + ! + router bgp 100 + bgp router-id 192.0.2.1 + neighbor 198.51.100.2 remote-as 100 + neighbor 198.51.100.2 tcp-mss 150 => new entry + neighbor 2001:DB8::2 remote-as 100 + neighbor 2001:DB8::2 tcp-mss 400 => new entry + +Show command: +------------- + +.. code-block:: frr + + frr# show bgp neighbors 198.51.100.2 + BGP neighbor is 198.51.100.2, remote AS 100, local AS 100, internal link + Hostname: frr + BGP version 4, remote router ID 192.0.2.2, local router ID 192.0.2.1 + BGP state = Established, up for 02:15:28 + Last read 00:00:28, Last write 00:00:28 + Hold time is 180, keepalive interval is 60 seconds + Configured tcp-mss is 150, synced tcp-mss is 138 => new display + +.. code-block:: frr + + frr# show bgp neighbors 2001:DB8::2 + BGP neighbor is 2001:DB8::2, remote AS 100, local AS 100, internal link + Hostname: frr + BGP version 4, remote router ID 192.0.2.2, local router ID 192.0.2.1 + BGP state = Established, up for 02:16:34 + Last read 00:00:34, Last write 00:00:34 + Hold time is 180, keepalive interval is 60 seconds + Configured tcp-mss is 400, synced tcp-mss is 388 => new display + +Show command json output: +------------------------- + +.. code-block:: frr + + frr# show bgp neighbors 2001:DB8::2 json + { + "2001:DB8::2":{ + "remoteAs":100, + "localAs":100, + "nbrInternalLink":true, + "hostname":"frr", + "bgpVersion":4, + "remoteRouterId":"192.0.2.2", + "localRouterId":"192.0.2.1", + "bgpState":"Established", + "bgpTimerUpMsec":8349000, + "bgpTimerUpString":"02:19:09", + "bgpTimerUpEstablishedEpoch":1613054251, + "bgpTimerLastRead":9000, + "bgpTimerLastWrite":9000, + "bgpInUpdateElapsedTimeMsecs":8347000, + "bgpTimerHoldTimeMsecs":180000, + "bgpTimerKeepAliveIntervalMsecs":60000, + "bgpTcpMssConfigured":400, => new entry + "bgpTcpMssSynced":388, => new entry + +.. code-block:: frr + + frr# show bgp neighbors 198.51.100.2 json + { + "198.51.100.2":{ + "remoteAs":100, + "localAs":100, + "nbrInternalLink":true, + "hostname":"frr", + "bgpVersion":4, + "remoteRouterId":"192.0.2.2", + "localRouterId":"192.0.2.1", + "bgpState":"Established", + "bgpTimerUpMsec":8370000, + "bgpTimerUpString":"02:19:30", + "bgpTimerUpEstablishedEpoch":1613054251, + "bgpTimerLastRead":30000, + "bgpTimerLastWrite":30000, + "bgpInUpdateElapsedTimeMsecs":8368000, + "bgpTimerHoldTimeMsecs":180000, + "bgpTimerKeepAliveIntervalMsecs":60000, + "bgpTcpMssConfigured":150, => new entry + "bgpTcpMssSynced":138, => new entry + +.. include:: routeserver.rst + +.. include:: rpki.rst + +.. include:: wecmp_linkbw.rst + +.. include:: flowspec.rst + +.. [#med-transitivity-rant] For some set of objects to have an order, there *must* be some binary ordering relation that is defined for *every* combination of those objects, and that relation *must* be transitive. I.e.:, if the relation operator is <, and if a < b and b < c then that relation must carry over and it *must* be that a < c for the objects to have an order. The ordering relation may allow for equality, i.e. a < b and b < a may both be true and imply that a and b are equal in the order and not distinguished by it, in which case the set has a partial order. Otherwise, if there is an order, all the objects have a distinct place in the order and the set has a total order) +.. [bgp-route-osci-cond] McPherson, D. and Gill, V. and Walton, D., "Border Gateway Protocol (BGP) Persistent Route Oscillation Condition", IETF RFC3345 +.. [stable-flexible-ibgp] Flavel, A. and M. Roughan, "Stable and flexible iBGP", ACM SIGCOMM 2009 +.. [ibgp-correctness] Griffin, T. and G. Wilfong, "On the correctness of IBGP configuration", ACM SIGCOMM 2002 + +.. _bgp-fast-convergence: + +BGP fast-convergence support +============================ +Whenever BGP peer address becomes unreachable we must bring down the BGP +session immediately. Currently only single-hop EBGP sessions are brought +down immediately.IBGP and multi-hop EBGP sessions wait for hold-timer +expiry to bring down the sessions. + +This new configuration option helps user to teardown BGP sessions immediately +whenever peer becomes unreachable. + +.. clicmd:: bgp fast-convergence + +This configuration is available at the bgp level. When enabled, configuration +is applied to all the neighbors configured in that bgp instance. + +.. code-block:: frr + + router bgp 64496 + neighbor 10.0.0.2 remote-as 64496 + neighbor fd00::2 remote-as 64496 + bgp fast-convergence + ! + address-family ipv4 unicast + redistribute static + exit-address-family + ! + address-family ipv6 unicast + neighbor fd00::2 activate + exit-address-family diff --git a/doc/user/bmp.rst b/doc/user/bmp.rst new file mode 100644 index 0000000..1983995 --- /dev/null +++ b/doc/user/bmp.rst @@ -0,0 +1,166 @@ +.. _bmp: + +*** +BMP +*** + +:abbr:`BMP` (BGP Monitoring Protocol, :rfc:`7854`) is used to send monitoring +data from BGP routers to network management entities. + +Implementation characteristics +============================== + +The `BMP` implementation in FRR has the following properties: + +- only the :rfc:`7854` features are currently implemented. This means protocol + version 3 without any extensions. It is not possible to use an older draft + protocol version of BMP. + +- the following statistics codes are implemented: + + - 0: count of prefixes rejected + - 2: count of duplicate prefix withdrawals + - 3: count of **prefixes** with loop in cluster id + - 4: count of **prefixes** with loop in AS-path + - 5: count of **prefixes** with loop in originator + - 11: count of updates subjected to :rfc:`7607` "treat as withdrawal" + handling due to errors + - 65531: *experimental* count of prefixes rejected due to invalid next-hop + + Note that stat items 3, 4 and 5 are specified to count updates, but FRR + implements them as prefix-based counters. + +- **route mirroring** is fully implemented, however BGP OPEN messages are not + currently included in route mirroring messages. Their contents can be + extracted from the "peer up" notification for sessions that established + successfully. OPEN messages for failed sessions cannot currently be + mirrored. + +- **route monitoring** is available for IPv4 and IPv6 AFIs, unicast and + multicast SAFIs. Other SAFIs (VPN, Labeled-Unicast, Flowspec, etc.) are not + currently supported. + +- monitoring peers that have BGP **add-path** enabled on the session will + result in somewhat unpredictable behaviour. Currently, the outcome is: + + - route mirroring functions as intended, messages are copied verbatim + - the add-path ID is never included in route monitoring messages + - if multiple paths were received from a peer, an unpredictable path is + picked and sent on the BMP session. The selection will differ for + pre-policy and post-policy monitoring sessions. + - as long as any path is present, something will be advertised on BMP + sessions. Only after the last path is gone a withdrawal will be sent on + BMP sessions. + - updates to additional paths will trigger BMP route monitoring messages. + There is no guarantee on consistency regarding which path is sent in these + messages. + +- monitoring peers with :rfc:`5549` extended next-hops has not been tested. + +Starting BMP +============ + +BMP is implemented as a loadable module. This means that to use BMP, ``bgpd`` +must be started with the ``-M bmp`` option. It is not possible to enable BMP +if ``bgpd`` was started without this option. + +Configuring BMP +=============== + +All of FRR's BMP configuration options are located inside the +:clicmd:`router bgp ASN` block. Configure BGP first before proceeding to BMP +setup. + +There is one option that applies to the BGP instance as a whole: + +.. clicmd:: bmp mirror buffer-limit(0-4294967294) + + This sets the maximum amount of memory used for buffering BGP messages + (updates, keepalives, ...) for sending in BMP Route Mirroring. + + The buffer is for the entire BGP instance; if multiple BMP targets are + configured they reference the same buffer and do not consume additional + memory. Queue overhead is included in accounting this memory, so the + actual space available for BGP messages is slightly less than the value + configured here. + + If the buffer fills up, the oldest messages are removed from the buffer and + any BMP sessions where the now-removed messages were still pending have + their **entire** queue flushed and a "Mirroring Messages Lost" BMP message + is sent. + + BMP Route Monitoring is not affected by this option. + +All other configuration is managed per targets: + +.. clicmd:: bmp targets NAME + + Create/delete a targets group. As implied by the plural name, targets may + cover multiple outbound active BMP sessions as well as inbound passive + listeners. + + If BMP sessions have the same configuration, putting them in the same + ``bmp targets`` will reduce overhead. + +BMP session configuration +------------------------- + +Inside a ``bmp targets`` block, the following commands control session +establishment: + + +.. clicmd:: bmp connect HOSTNAME port (1-65535) {min-retry MSEC|max-retry MSEC} [source-interface WORD] + + Add/remove an active outbound BMP session. HOSTNAME is resolved via DNS, + if multiple addresses are returned they are tried in nondeterministic + order. Only one connection will be established even if multiple addresses + are returned. ``min-retry`` and ``max-retry`` specify (in milliseconds) + bounds for exponential backoff. ``source-interface`` is the local interface on + which the connection has to bind. + +.. warning:: + + ``ip access-list`` and ``ipv6 access-list`` are checked for outbound + connections resulting from ``bmp connect`` statements. + +.. clicmd:: bmp listener <X:X::X:X|A.B.C.D> port (1-65535) + + Accept incoming BMP sessions on the specified address and port. You can + use ``0.0.0.0`` and ``::`` to listen on all IPv4/IPv6 addresses. + +.. clicmd:: ip access-list NAME +.. clicmd:: ipv6 access-list NAME + + Restrict BMP sessions to the addresses allowed by the respective access + lists. The access lists are checked for both passive and active BMP + sessions. Changes do not affect currently established sessions. + +BMP data feed configuration +--------------------------- + +The following commands configure what BMP messages are sent on sessions +associated with a particular ``bmp targets``: + +.. clicmd:: bmp stats [interval (100-86400000)] + + Send BMP Statistics (counter) messages at the specified interval (in + milliseconds.) + +.. clicmd:: bmp monitor AFI SAFI <pre-policy|post-policy> + + Perform Route Monitoring for the specified AFI and SAFI. Only IPv4 and + IPv6 are currently valid for AFI. SAFI valid values are currently + unicast, multicast, evpn and vpn. + Other AFI/SAFI combinations may be added in the future. + + All BGP neighbors are included in Route Monitoring. Options to select + a subset of BGP sessions may be added in the future. + +.. clicmd:: bmp mirror + + Perform Route Mirroring for all BGP neighbors. Since this provides a + direct feed of BGP messages, there are no AFI/SAFI options to be + configured. + + All BGP neighbors are included in Route Mirroring. Options to select + a subset of BGP sessions may be added in the future. diff --git a/doc/user/bugs.rst b/doc/user/bugs.rst new file mode 100644 index 0000000..2af9e31 --- /dev/null +++ b/doc/user/bugs.rst @@ -0,0 +1,70 @@ +.. index:: + pair: bug reports; contact + +.. _bug-reports: + +************** +Reporting Bugs +************** + +This file describes the procedure for reporting FRRouting bugs. You are asked +to follow this format when submitting bug reports. + +Bugs submitted with woefully incomplete information will receive little +attention and are likely to be closed. If you hit a suspected bug in an older +version, you may be asked to test with a later version in your environment. + +Often you may be asked for additional information to help solve the bug. Bugs +may be closed after 30 days of non-response to requests to reconfirm or supply +additional information. + +Please report bugs on the project GitHub issue tracker at +https://github.com/frrouting/frr/issues + +Report Format & Requested Information +===================================== + +When reporting a bug, please provide the following information. + +#. Your FRR version if it is a release build, or the commit hash if you built + from source. + +#. If you compiled from source, please provide your ``./configure`` line, + including all option flags. + +#. A full list of the FRR daemons you run. + +#. Your platform name and version, e.g. ``Ubuntu 18.04``. + +#. Problem description. + + - Provide as much information as possible. + - Copy and paste relevant commands and their output to describe your network + setup. + - Topology diagrams are helpful when reporting bugs involving more than one + box. + - Platform routing tables and interface configurations are useful if you are + reporting a routing issue. + + *Please be sure to review the provided information and censor any sensitive + material.* + +#. All FRR configuration files you use. Again, please be sure to censor any + sensitive information. For sensitive v4 / v6 addresses, we ask that you + censor the inner octets; e.g., ``192.XXX.XXX.32/24``. + +#. If you are reporting a crash and have a core file, please supply a stack + trace using GDB: + + :: + + $ gdb exec_file core_file + (gdb) bt . + +#. Run all FRR daemons with full debugging on and send *only* the portion of + logs which are relevant to your problem. + +#. Patches, workarounds, and fixes are always welcome. + +.. seealso:: :ref:`basic-config-commands` + diff --git a/doc/user/conf.py b/doc/user/conf.py new file mode 100644 index 0000000..728f9c9 --- /dev/null +++ b/doc/user/conf.py @@ -0,0 +1,402 @@ +# -*- coding: utf-8 -*- +# +# FRR documentation build configuration file, created by +# sphinx-quickstart on Tue Jan 31 16:00:52 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys +import os +import re +import pygments +import sphinx +from sphinx.highlighting import lexers + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +needs_sphinx = "1.0" + +# prolog for various variable substitutions +rst_prolog = "" + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = ["sphinx.ext.todo"] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ["_templates"] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst'] +source_suffix = ".rst" + +# The encoding of source files. +# source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = "index" + +# General information about the project. +project = u"FRR" +copyright = u"2017, FRR" +author = u"FRR authors" + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. + +# The short X.Y version. +version = u"?.?" +# The full version, including alpha/beta/rc tags. +release = u"?.?-?" + + +# ----------------------------------------------------------------------------- +# Extract values from codebase for substitution into docs. +# ----------------------------------------------------------------------------- + +# Various installation prefixes. Values are extracted from config.status. +# Reasonable defaults are set in case that file does not exist. +replace_vars = { + "AUTHORS": author, + "COPYRIGHT_YEAR": "1999-2005", + "COPYRIGHT_STR": "Copyright (c) 1999-2005", + "PACKAGE_NAME": project.lower(), + "PACKAGE_TARNAME": project.lower(), + "PACKAGE_STRING": project.lower() + " latest", + "PACKAGE_URL": "https://frrouting.org/", + "PACKAGE_VERSION": "latest", + "INSTALL_PREFIX_ETC": "/etc/frr", + "INSTALL_PREFIX_SBIN": "/usr/lib/frr", + "INSTALL_PREFIX_STATE": "/var/run/frr", + "INSTALL_PREFIX_MODULES": "/usr/lib/frr/modules", + "INSTALL_USER": "frr", + "INSTALL_GROUP": "frr", + "INSTALL_VTY_GROUP": "frrvty", + "GROUP": "frr", + "USER": "frr", +} + +# extract version information, installation location, other stuff we need to +# use when building final documents +val = re.compile('^S\["([^"]+)"\]="(.*)"$') +try: + with open("../../config.status", "r") as cfgstatus: + for ln in cfgstatus.readlines(): + m = val.match(ln) + if not m or m.group(1) not in replace_vars.keys(): + continue + replace_vars[m.group(1)] = m.group(2) +except IOError: + # if config.status doesn't exist, just ignore it + pass + +# manually fill out some of these we can't get from config.status +replace_vars["COPYRIGHT_STR"] = "Copyright (c)" +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["COPYRIGHT_YEAR"]) +replace_vars["COPYRIGHT_STR"] += " {0}".format(replace_vars["AUTHORS"]) +release = replace_vars["PACKAGE_VERSION"] +version = release.split("-")[0] + +# add substitutions to prolog +for key, value in replace_vars.items(): + rst_prolog += ".. |{0}| replace:: {1}\n".format(key, value) + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +# today = '' +# Else, today_fmt is used as the format for a strftime call. +# today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = [ + "_build", + "rpki.rst", + "routeserver.rst", + "ospf_fundamentals.rst", + "flowspec.rst", + "snmptrap.rst", + "wecmp_linkbw.rst", +] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +# default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +# add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +# add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +# show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = "sphinx" + +# A list of ignored prefixes for module index sorting. +# modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +# keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = "default" + +try: + import sphinx_rtd_theme + + html_theme = "sphinx_rtd_theme" +except ImportError: + pass + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# html_theme_options = { +# 'sidebarbgcolor': '#374249' +# } + +# Add any paths that contain custom themes here, relative to this directory. +# html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# "<project> v<release> documentation". +# html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +# html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +html_logo = "../figures/frr-icon.svg" + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +html_favicon = "../figures/frr-logo-icon.png" + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ["_static"] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +# html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +# html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +# html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +# html_additional_pages = {} + +# If false, no module index is generated. +# html_domain_indices = True + +# If false, no index is generated. +# html_use_index = True + +# If true, the index is split into individual pages for each letter. +# html_split_index = False + +# If true, links to the reST sources are added to the pages. +# html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +# html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +# html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a <link> tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +# html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +# html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr' +# html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +# html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +# html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = "FRRdoc" + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + #'papersize': 'letterpaper', + # The font size ('10pt', '11pt' or '12pt'). + #'pointsize': '10pt', + # Additional stuff for the LaTeX preamble. + #'preamble': '', + # Latex figure (float) alignment + #'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, "FRR.tex", u"FRR User Manual", u"FRR", "manual"), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +latex_logo = "../figures/frr-logo-medium.png" + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +# latex_use_parts = False + +# If true, show page references after internal links. +# latex_show_pagerefs = False + +# If true, show URL addresses after external links. +# latex_show_urls = False + +# Documents to append as an appendix to all manuals. +# latex_appendices = [] + +# If false, no module index is generated. +# latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [(master_doc, "frr", u"FRR User Manual", [author], 1)] + +# If true, show URL addresses after external links. +# man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + ( + master_doc, + "frr", + u"FRR User Manual", + author, + "FRR", + "One line description of project.", + "Miscellaneous", + ), +] + +# Documents to append as an appendix to all manuals. +# texinfo_appendices = [] + +# If false, no module index is generated. +# texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +# texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +# texinfo_no_detailmenu = False + +# contents of ../extra/frrlexer.py. +# This is read here to support VPATH build. Since this section is execfile()'d +# with the file location, we can safely use a relative path here to save the +# contents of the lexer file for later use even if our relative path changes +# due to VPATH. +with open("../extra/frrlexer.py", "rb") as lex: + frrlexerpy = lex.read() + +# Parse version string into int array +def vparse(s): + a = [] + + for c in s: + if c != ".": + a.append(int(c)) + + while len(a) < 3: + a.append(0) + + return a[:3] + + +# custom extensions here +def setup(app): + # object type for FRR CLI commands, can be extended to document parent CLI + # node later on + app.add_object_type("clicmd", "clicmd", indextemplate="pair: %s; configuration command") + + # I dont care how stupid this is + if "add_js_file" in dir(app): + app.add_js_file("overrides.js") + else: + app.add_javascript("overrides.js") + + if "add_css_file" in dir(app): + app.add_css_file("overrides.css") + else: + app.add_stylesheet("overrides.css") + + + # load Pygments lexer for FRR config syntax + # + # NB: in Pygments 2.2+ this can be done with `load_lexer_from_file`, but we + # do it manually since not all of our supported build platforms have 2.2 + # yet. + # + # frrlexer = pygments.lexers.load_lexer_from_file('../extra/frrlexer.py', lexername="FRRLexer") + custom_namespace = {} + exec(frrlexerpy, custom_namespace) + lexers["frr"] = custom_namespace["FRRLexer"]() diff --git a/doc/user/eigrpd.rst b/doc/user/eigrpd.rst new file mode 100644 index 0000000..fa157c4 --- /dev/null +++ b/doc/user/eigrpd.rst @@ -0,0 +1,200 @@ +.. _eigrp: + +***** +EIGRP +***** + +.. glossary:: + + DUAL + The *Diffusing Update ALgorithm*, a :term:`Bellman-Ford` based routing + algorithm used by EIGRP. + +EIGRP -- Routing Information Protocol is widely deployed interior gateway +routing protocol. EIGRP was developed in the 1990's. EIGRP is a +:term:`distance-vector` protocol and is based on the :term:`DUAL` algorithms. +As a distance-vector protocol, the EIGRP router send updates to its +neighbors as networks change, thus allowing the convergence to a +known topology. + +*eigrpd* supports EIGRP as described in RFC7868 + +.. _starting-and-stopping-eigrpd: + +Starting and Stopping eigrpd +============================ + +The default configuration file name of *eigrpd*'s is :file:`eigrpd.conf`. When +invocation *eigrpd* searches directory |INSTALL_PREFIX_ETC|. If +:file:`eigrpd.conf` is not there next search current directory. If an +integrated config is specified configuration is written into :file:`frr.conf`. + +The EIGRP protocol requires interface information maintained by *zebra* daemon. +So running *zebra* is mandatory to run *eigrpd*. Thus minimum sequence for +running EIGRP is: + +:: + + # zebra -d + # eigrpd -d + + +Please note that *zebra* must be invoked before *eigrpd*. + +To stop *eigrpd*, please use:: + + kill `cat /var/run/frr/eigrpd.pid` + +Certain signals have special meanings to *eigrpd*. + ++------------------+-----------------------------------------------------------+ +| Signal | Meaning | ++==================+===========================================================+ +| SIGHUP & SIGUSR1 | Rotate the log file | ++------------------+-----------------------------------------------------------+ +| SIGINT & SIGTERM | Sweep all installed EIGRP routes and gracefully terminate | ++------------------+-----------------------------------------------------------+ + + +*eigrpd* invocation options. Common options that can be specified +(:ref:`common-invocation-options`). + +.. program:: eigrpd + +.. _eigrp-configuration: + +EIGRP Configuration +=================== + +.. clicmd:: router eigrp (1-65535) [vrf NAME] + + The `router eigrp` command is necessary to enable EIGRP. To disable EIGRP, + use the `no router eigrp (1-65535)` command. EIGRP must be enabled before + carrying out any of the EIGRP commands. Specify vrf NAME if you want + eigrp to work within the specified vrf. + +.. clicmd:: network NETWORK + + Set the EIGRP enable interface by `network`. The interfaces which + have addresses matching with `network` are enabled. + + This group of commands either enables or disables EIGRP interfaces between + certain numbers of a specified network address. For example, if the + network for 10.0.0.0/24 is EIGRP enabled, this would result in all the + addresses from 10.0.0.0 to 10.0.0.255 being enabled for EIGRP. The `no + network` command will disable EIGRP for the specified network. + + Below is very simple EIGRP configuration. Interface `eth0` and + interface which address match to `10.0.0.0/8` are EIGRP enabled. + + .. code-block:: frr + + ! + router eigrp 1 + network 10.0.0.0/8 + ! + + +.. clicmd:: passive-interface (IFNAME|default) + + + This command sets the specified interface to passive mode. On passive mode + interface, all receiving packets are ignored and eigrpd does not send either + multicast or unicast EIGRP packets except to EIGRP neighbors specified with + `neighbor` command. The interface may be specified as `default` to make + eigrpd default to passive on all interfaces. + + The default is to be passive on all interfaces. + +.. _how-to-announce-eigrp-route: + +How to Announce EIGRP route +=========================== + +Redistribute routes into EIGRP: + +.. clicmd:: redistribute <babel|bgp|connected|isis|kernel|openfabric|ospf|rip|sharp|static|table> [metric (1-4294967295) (0-4294967295) (0-255) (1-255) (1-65535)] + + The ``redistribute`` family of commands imports routing information from + other sources into EIGRP's tables. Redistribution may be disabled with the + ``no`` form of the commands. + + Note that connected routes on interfaces EIGRP is enabled on are announced + by default. + + Optionally, various EIGRP metrics may be specified. These metrics will be + applied to the imported routes. + + +.. _show-eigrp-information: + +Show EIGRP Information +====================== + +.. clicmd:: show ip eigrp [vrf NAME] topology + + Display current EIGRP status. + + :: + + eigrpd> **show ip eigrp topology** + # show ip eigrp topo + + EIGRP Topology Table for AS(4)/ID(0.0.0.0) + + Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply + r - reply Status, s - sia Status + + P 10.0.2.0/24, 1 successors, FD is 256256, serno: 0 + via Connected, enp0s3 + +.. clicmd:: show ip eigrp [vrf NAME] interface + + Display the list of interfaces associated with a particular eigrp + instance. + +.. clicmd:: show ip eigrp [vrf NAME] neighbor + + Display the list of neighbors that have been established within + a particular eigrp instance. + +EIGRP Debug Commands +==================== + +Debug for EIGRP protocol. + +.. clicmd:: debug eigrp packets + + Debug eigrp packets + + ``debug eigrp`` will show EIGRP packets that are sent and received. + +.. clicmd:: debug eigrp transmit + + Debug eigrp transmit events + + ``debug eigrp transmit`` will display detailed information about the EIGRP + transmit events. + +.. clicmd:: show debugging eigrp + + Display *eigrpd*'s debugging option. + + ``show debugging eigrp`` will show all information currently set for eigrpd + debug. + + +Sample configuration +==================== + +.. code-block:: frr + + hostname eigrpd + password zebra + enable password please-set-at-here + ! + router eigrp 4453 + network 192.168.1.0/24 + ! + log stdout + diff --git a/doc/user/evpn.rst b/doc/user/evpn.rst new file mode 100644 index 0000000..7c4d9fe --- /dev/null +++ b/doc/user/evpn.rst @@ -0,0 +1,530 @@ +.. _evpn: + +**** +EVPN +**** + +:abbr:`EVPN` stands for Ethernet Virtual Private Network. This is an extension +of BGP that enables the signaling of bridged (L2) and routed (L3) VPNs over a +common network. EVPN is described in :rfc:`7432` and is updated by several +additional RFCs and IETF drafts including :rfc:`9135` (Integrated Routing +and Bridging in Ethernet VPN), :rfc:`9136` (IP Prefix Advertisement in Ethernet +VPN), :rfc:`8584` (Framework for Ethernet VPN Designated Forwarder Election +Extensibility), and :rfc:`8365` (A Network Virtualization Overlay Solution Using +Ethernet VPN). FRR supports All-Active Layer-2 Multihoming for devices (MHD) via +LACP Ethernet Segments as well as both Symmetric and Asymmetric IRB. +FRR implements MAC-VRFs using a "VLAN-Based Service Interface" (:rfc:`7432`) +and performs processing of Symmetric IRB routes following the +"Interface-less IP-VRF-to-IP-VRF Model" (:rfc:`9136`). + +.. _evpn-concepts: + +EVPN Concepts +============= +BGP-EVPN is the control plane for the transport of Ethernet frames, regardless +of whether those frames are bridged or routed. In the case of a VLAN-Based +Service Interface with VXLAN encap, a single VNI is used to represent an EVPN +Instance (EVI) and will have its own Route Distinguisher and set of +Import/Export Route-Targets. + +A VNI is considered to be either Layer-2 (tied to a MAC-VRF) or Layer-3 +(tied to an IP-VRF), which indicates what kind of information is represented by +the VRF. An IP-VRF represents a routing table (operating in much the same way as +a VRF traditionally operates in L3VPN), while a MAC-VRF represents a bridging +table i.e. MAC (fdb) and ARP/NDP entries. + +A MAC-VRF can be thought of as a VLAN with or without an SVI associated with it. +An SVI is a Layer-3 interface bound to a bridging domain. In Linux an SVI can +either be a traditional bridge or a VLAN subinterface of a VLAN-aware bridge. +If there is an SVI for the VLAN, ARP/NDP entries can be bound to the MACs within +the broadcast domain. Without an SVI, the VLAN operates in traditional L2 +fashion and MACs are the only type of host addresses known within the VLAN. + +In the same way that there can be a many-to-one relationship of SVIs to a VRF, +there can also be a many-to-one relationship of MAC-VRFs (L2VNIs) to an IP-VRF +(L3VNI). In FRR the L3VNI association for an L2VNI is determined by the +presence of an SVI for the VLAN and the VRF membership of the SVI. +If an L2VNI does not have an SVI or its SVI is not enslaved to a VRF, the L2VNI +will be associated with the "default" VRF. If an L2VNI has an SVI whose master +device is a VRF, then that L2VNI will be associated with its master VRF. + +.. _evpn-frr-configuration: + +FRR Configuration +================= +FRR learns about the system's Linux network interface configuration from the +kernel via Netlink, however it does not manage network interfaces directly. +The following sections will include examples of Linux interface configurations +that are compatible with FRR's EVPN implementation. While there are multiple +interface managers that can setup a proper kernel config (e.g. ifupdown2), +these examples will use iproute2 to add/configure the interfaces. + +All of the examples will follow the same basic setup but use different, yet +compatible, interface configurations. + +In this example we will setup the following: + +* An IP-VRF named vrf1, associated with L3VNI 100 +* An IP-VRF named vrf2, associated with L3VNI 200 +* An IP-VRF named vrf3, with no L3VNI associations +* A MAC-VRF using VLAN 10, associated with L2VNI 110 and IP-VRF vrf1 +* A MAC-VRF using VLAN 20, associated with L2VNI 220 and IP-VRF vrf2 +* A MAC-VRF using VLAN 30, associated with L2VNI 330 and IP-VRF vrf3 +* A MAC-VRF using VLAN 40, associated with L2VNI 440 and IP-VRF default +* A MAC-VRF using VLAN 50, associated with L2VNI 550 and operating L2-Only + +.. _evpn-sample-configuration: + +Sample Configuration +-------------------- +This is a sample FRR configuration that implements the above EVPN environment. +The first snippet will be the config in its entiretly, then each config element +will be explained individually later in the document. + +The following snippet will result in a functional EVPN control plane if the +corresponding Linux interface configuration is correct, compatible, and active: + +.. code-block:: frr + + vrf vrf1 + vni 100 + exit-vrf + ! + vrf vrf2 + vni 200 + exit-vrf + ! + router bgp 4200000000 + neighbor 192.168.122.12 remote-as internal + ! + address-family ipv4 unicast + network 100.64.0.1/32 + exit-address-family + ! + address-family l2vpn evpn + neighbor 192.168.122.12 activate + advertise-all-vni + advertise-svi-ip + exit-address-family + exit + ! + router bgp 4200000000 vrf vrf1 + ! + address-family ipv4 unicast + redistribute static + exit-address-family + ! + address-family ipv6 unicast + redistribute static + exit-address-family + ! + address-family l2vpn evpn + advertise ipv4 unicast + advertise ipv6 unicast + exit-address-family + exit + ! + router bgp 4200000000 vrf vrf2 + ! + address-family ipv4 unicast + redistribute static + exit-address-family + ! + address-family ipv6 unicast + redistribute static + exit-address-family + ! + address-family l2vpn evpn + advertise ipv4 unicast + advertise ipv6 unicast + exit-address-family + exit + +A VRF will get its L3VNI association as a result of the ``vni`` command under +the ``vrf`` stanza. Until this L3VNI association is made, zebra will discover +the VNI from netlink but will consider it to be an L2VNI. The current L2 vs L3 +context of a VNI can be seen in the output of ``show evpn vni``. + +In this configuration we are telling zebra to consider VXLAN-ID 100 to be the +L3VNI for vrf1 and VXLAN-ID 200 to be the L3VNI for vrf2. + +.. code-block:: frr + + vrf vrf1 + vni 100 + exit-vrf + ! + vrf vrf2 + vni 200 + exit-vrf + +The VTEP-IP (100.64.0.1) needs to be reachable by other VTEPs in the EVPN +environment in order for VXLAN decapsulation to function. In this example we +will advertise our local VTEP-IP using BGP (via the ``network`` statement), but +static routes or other routing protocols like IS-IS or OSPF can also be used. + +In order to enable EVPN for a BGP instance, we must use the command +``advertise-all-vni``. In this example we will be using the default VRF to +carry the l2vpn evpn address-family, so we will enable EVPN for the default VRF. + +In this example, we plan to exchange EVPN routes with 192.168.122.12, so we +will activate the l2vpn evpn address-family for this peer in order to allow +EVPN NLRI to be advertised and received. + +The ``advertise-svi-ip`` command also belongs in the BGP instance where EVPN is +enabled. This command tells FRR to originate "self" Type-2 routes for all the +MAC/IP pairs associated with the local SVI interfaces. + +.. code-block:: frr + + router bgp 4200000000 + neighbor 192.168.122.12 remote-as internal + ! + address-family ipv4 unicast + network 100.64.0.1/32 + exit-address-family + ! + address-family l2vpn evpn + neighbor 192.168.122.12 activate + advertise-all-vni + advertise-svi-ip + exit-address-family + exit + +IPv4 and IPv6 BGP Prefixes from an IP-VRF are not exported to EVPN as Type-5 +routes until the respective ``advertise <afi> unicast`` command has been +configured in the BGP instance of the VRF in question. All routes in the BGP +RIB (locally originated, learned from a peer, or leaked from another VRF) will +be eligible to be exported to EVPN so long as they are valid and selected in +the VRF's unicast table. + +In this example, the BGP instances for vrf1 and vrf2 will have their static +routes redistributed into the BGP loc-rib for the ipv4 unicast and ipv6 unicast +address-families via the ``redistribute static`` statements. These unicast +prefixes will then be exported into EVPN as Type-5 routes as a result of the +``advertise ipv4 unicast`` and ``advertise ipv6 unicast`` commands. + +.. code-block:: frr + + router bgp 4200000000 vrf vrf1 + ! + address-family ipv4 unicast + redistribute static + exit-address-family + ! + address-family ipv6 unicast + redistribute static + exit-address-family + ! + address-family l2vpn evpn + advertise ipv4 unicast + advertise ipv6 unicast + exit-address-family + exit + ! + router bgp 4200000000 vrf vrf2 + ! + address-family ipv4 unicast + redistribute static + exit-address-family + ! + address-family ipv6 unicast + redistribute static + exit-address-family + ! + address-family l2vpn evpn + advertise ipv4 unicast + advertise ipv6 unicast + exit-address-family + exit + +.. _evpn-linux-interface-configuration: + +Linux Interface Configuration +============================= +The Linux kernel offers several options for configuring netdevices for an +EVPN-VXLAN environment. The following section will include samples of a few +netdev configurations that are compatible with FRR which implement the +environment described above. + +Some high-level config considerations: + +* The local VTEP-IP should always be set to a reachable IP on the lo device. +* An L3VNI should always have an SVI (aka the L3-SVI). +* An L3-SVI should not be assigned an IP address, link-local or otherwise. + + * IPv6 address autoconfiguration can be disabled via ``addrgenmode none``. + +* An SVI for an L2VNI is only needed for routing (IRB) or ARP/ND suppression. + + * ARP/ND suppression is a kernel function, it is not managed by FRR. + * ARP/ND suppression is enabled per bridge_slave via ``neigh_suppress``. + * ARP/ND suppression should only be enabled on vxlan interfaces. + * IPv4/IPv6 forwarding should be disabled on SVIs not used for routing (IRB). + +* Dynamic MAC/VTEP learning should be disabled on VXLAN interfaces used in EVPN. + + * Dynamic MAC learning is a function of the kernel bridge driver, not FRR. + * Dynamic MAC learning is toggled per bridge_slave via ``learning {on|off}``. + * Dynamic VTEP learning is a function of the kernel vxlan driver, not FRR. + * Dynamic VTEP learning is toggled per vxlan interface via ``[no]learning``. + +* The VXLAN interfaces should not have a ``remote`` VTEP defined. + + * Remote VTEPs are learned via EVPN, so static VTEPs are unnecessary. + +.. _evpn-traditional-bridge-traditional-vxlan-devices: + +Traditional Bridges and Traditional VXLAN Devices +------------------------------------------------- +In the traditional bridge model, we use a separate ``bridge`` interface per +MAC-VRF which acts as the SVI for that broadcast domain. A bridge is considered +"traditional" if ``vlan_filtering`` is set to ``0`` (disabled) which indicates +the bridge only has one broadcast domain which does not consider VLAN tags. +Similarly, only one VNI is carried by each "traditional" ``vxlan`` interface. +So in this deployment model, each VXLAN-enabled broadcast domain will have one +traditional vxlan interface enslaved to one traditional bridge. + +Bridges created for an L3VNI broadcast domain should only have one member: the +L3VNI vxlan device. Bridges created for an L2VNI broadcast domain generally +have multiple members: the L2VNI vxlan device, plus any host/network ports +where the L2 domain will be carried. + +To carry the broadcast domains of multiple traditional bridges over the same +host/network port, a tagged ``vlan`` sub-interface of the port must be created +per broadcast domain. The vlan sub-interfaces would then be enslaved to the +traditional bridge, ensuring that only packets tagged with the expected VID are +associated with the expected broadcast domain. + +.. code-block:: shell + + ################### + ## vxlan vtep-ip ## + ################### + ip addr add 100.64.0.1/32 dev lo + + ############################# + ## ip-vrf vrf1 / l3vni 100 ## + ############################# + ip link add vrf1 type vrf table 1100 + ip link set vrf1 up + ip link add br100 type bridge + ip link set br100 master vrf1 addrgenmode none + ip link set br100 addr aa:bb:cc:00:00:64 + ip link add vni100 type vxlan local 100.64.0.1 dstport 4789 id 100 nolearning + ip link set vni100 master br100 addrgenmode none + ip link set vni100 type bridge_slave neigh_suppress on learning off + ip link set vni100 up + ip link set br100 up + + ############################# + ## ip-vrf vrf2 / l3vni 200 ## + ############################# + ip link add vrf2 type vrf table 1200 + ip link set vrf2 up + ip link add br200 type bridge + ip link set br200 master vrf2 addrgenmode none + ip link set br200 addr aa:bb:cc:00:00:c8 + ip link add vni200 type vxlan local 100.64.0.1 dstport 4789 id 200 nolearning + ip link set vni200 master br200 addrgenmode none + ip link set vni200 type bridge_slave neigh_suppress on learning off + ip link set vni200 up + ip link set br200 up + + ################# + ## ip-vrf vrf3 ## + ################# + ip link add vrf3 type vrf table 1300 + ip link set vrf3 up + + ############### + ## l2vni 110 ## + ############### + ip link add br10 type bridge + ip link set br10 master vrf1 + ip link set br10 addr aa:bb:cc:00:00:6e + ip addr add 10.0.10.1/24 dev br10 + ip addr add 2001:db8:0:10::1/64 dev br10 + ip link add vni110 type vxlan local 100.64.0.1 dstport 4789 id 110 nolearning + ip link set vni110 master br10 addrgenmode none + ip link set vni110 type bridge_slave neigh_suppress on learning off + ip link set vni110 up + ip link set br10 up + + ############### + ## l2vni 220 ## + ############### + ip link add br20 type bridge + ip link set br20 master vrf2 + ip link set br20 addr aa:bb:cc:00:00:dc + ip addr add 10.0.20.1/24 dev br20 + ip addr add 2001:db8:0:20::1/64 dev br20 + ip link add vni220 type vxlan local 100.64.0.1 dstport 4789 id 220 nolearning + ip link set vni220 master br20 addrgenmode none + ip link set vni220 type bridge_slave neigh_suppress on learning off + ip link set vni220 up + ip link set br20 up + + ############### + ## l2vni 330 ## + ############### + ip link add br30 type bridge + ip link set br30 master vrf3 + ip link set br30 addr aa:bb:cc:00:01:4a + ip addr add 10.0.30.1/24 dev br30 + ip addr add 2001:db8:0:30::1/64 dev br30 + ip link add vni330 type vxlan local 100.64.0.1 dstport 4789 id 330 nolearning + ip link set vni330 master br30 addrgenmode none + ip link set vni330 type bridge_slave neigh_suppress on learning off + ip link set vni330 up + ip link set br30 up + + ############### + ## l2vni 440 ## + ############### + ip link add br40 type bridge + ip link set br40 addr aa:bb:cc:00:01:b8 + ip addr add 10.0.40.1/24 dev br40 + ip addr add 2001:db8:0:40::1/64 dev br40 + ip link add vni440 type vxlan local 100.64.0.1 dstport 4789 id 440 nolearning + ip link set vni440 master br40 addrgenmode none + ip link set vni440 type bridge_slave neigh_suppress on learning off + ip link set vni440 up + ip link set br40 up + + ############### + ## l2vni 550 ## + ############### + ip link add br50 type bridge + ip link set br50 addrgenmode none + ip link set br50 addr aa:bb:cc:00:02:26 + ip link add vni550 type vxlan local 100.64.0.1 dstport 4789 id 550 nolearning + ip link set vni550 master br50 addrgenmode none + ip link set vni550 type bridge_slave neigh_suppress on learning off + sysctl -w net.ipv4.conf.br50.forwarding=0 + sysctl -w net.ipv6.conf.br50.forwarding=0 + ip link set vni550 up + ip link set br50 up + + ################## + ## create vlan subinterface of eth0 for each l2vni vlan and enslave each + ## subinterface to the corresponding bridge + ################## + ip link set eth0 up + for i in 10 20 30 40 50; do + ip link add link eth0 name eth0.$i type vlan id $i; + ip link set eth0.$i master br$i; + ip link set eth0.$i up; + done + + +To begin with, it creates a ``vrf`` interface named "vrf1" that is bound to the +kernel routing table with ID 1100. This will represent the IP-VRF "vrf1" which +we will later allocate an L3VNI for. + +.. code-block:: shell + + ip link add vrf1 type vrf table 1100 + +This block creates a traditional ``bridge`` interface named "br100", binds it to +the VRF named "vrf1", disables IPv6 address autoconfiguration, and statically +defines the MAC address of "br100". This traditional bridge is used for the +L3VNI broadcast domain mapping to VRF "vrf1", i.e. "br100" is vrf1's L3-SVI. + +.. code-block:: shell + + ip link add br100 type bridge + ip link set br100 master vrf1 addrgenmode none + ip link set br100 addr aa:bb:cc:00:00:64 + +Here a traditional ``vxlan`` interface is created with the name "vni100" which +uses a VTEP-IP of 100.64.0.1, carries VNI 100, and has Dynamic VTEP learning +disabled. IPv6 address autoconfiguration is disabled for "vni100", then the +interface is enslaved to "br100", ARP/ND suppression is enabled, and Dynamic +MAC Learning is disabled. + +.. code-block:: shell + + ip link add vni100 type vxlan local 100.64.0.1 dstport 4789 id 100 nolearning + ip link set vni100 master br100 addrgenmode none + ip link set vni100 type bridge_slave neigh_suppress on learning off + +This completes the necessary configuration for a VRF and L3VNI. + +Here a traditional bridge named "br10" is created. We add "br10" to "vrf1" by +setting "vrf1" as the ``master`` of "br10". It is not necessary to set the SVI +MAC statically, but it is done here for consistency's sake. Since "br10" will +be used for routing, IPv4 and IPv6 addresses are also added to the SVI. + +.. code-block:: shell + + ip link add br10 type bridge + ip link set br10 master vrf1 + ip link set br10 addr aa:bb:cc:00:00:6e + ip addr add 10.0.10.1/24 dev br10 + ip addr add 2001:db8:0:10::1/64 dev br10 + +If the SVI will not be used for routing, IP addresses should not be assigned to +the SVI interface and IPv4/IPv6 "forwarding" should be disabled for the SVI via +the appropriate sysctl nodes. + +.. code-block:: shell + + sysctl -w net.ipv4.conf.<ifname>.forwarding=0 + sysctl -w net.ipv6.conf.<ifname>.forwarding=0 + +The following commands create a ``vxlan`` interface for VNI 100. Other than the +VNI, The interface settings are the same for an L2VNI as they are for an L3VNI. + +.. code-block:: shell + + ip link add vni110 type vxlan local 100.64.0.1 dstport 4789 id 110 nolearning + ip link set vni110 master br10 addrgenmode none + ip link set vni110 type bridge_slave neigh_suppress on learning off + +Finally, to limit a traditional bridge's broadcast domain to traffic matching +specific VLAN-IDs, ``vlan`` subinterfaces of a host/network port need to be +setup. This example shows the creation of a VLAN subinterface of "eth0" +matching VID 10 with the name "eth0.10". By enslaving "eth0.10" to "br10" +(instead of "eth0") we ensure that only Ethernet frames ingressing "eth0" +tagged with VID 10 will be associated with the "br10" broadcast domain. + +.. code-block:: shell + + ip link add link eth0 name eth0.10 type vlan id 10 + ip link set eth0.10 master br10 + +If you do not want to restrict the broadcast domain by VLAN-ID, you can skip +the creation of the VLAN subinterfaces and directly enslave "eth0" to "br10". + +.. code-block:: shell + + ip link set eth0 master br10 + +This completes the necessary configuration for an L2VNI. + +Displaying EVPN information +--------------------------- + +.. clicmd:: show evpn mac vni (1-16777215) detail [json] + + Display detailed information about MAC addresses for + a specified VNI. + +.. clicmd:: show vrf [<NAME$vrf_name|all$vrf_all>] vni [json] + + Displays VRF to L3VNI mapping. It also displays L3VNI associated + router-mac, svi interface and vxlan interface. + User can get that information as JSON format when ``json`` keyword + at the end of cli is presented. + + .. code-block:: frr + + tor2# show vrf vni + VRF VNI VxLAN IF L3-SVI State Rmac + sym_1 9288 vxlan21 vlan210_l3 Up 21:31:36:ff:ff:20 + sym_2 9289 vxlan21 vlan210_l3 Up 21:31:36:ff:ff:20 + sym_3 9290 vxlan21 vlan210_l3 Up 21:31:36:ff:ff:20 + tor2# show vrf sym_1 vni + VRF VNI VxLAN IF L3-SVI State Rmac + sym_1 9288 vxlan21 vlan210_l3 Up 44:38:36:ff:ff:20 diff --git a/doc/user/extlog.rst b/doc/user/extlog.rst new file mode 100644 index 0000000..bb872c6 --- /dev/null +++ b/doc/user/extlog.rst @@ -0,0 +1,189 @@ +.. _ext-log-target: + +*********************** +Extended Logging Target +*********************** + +After creating one or more extended logging targets with the +:clicmd:`log extended EXTLOGNAME` command, the target(s) must be configured +for the desired logging output. + +Each extended log target supports emitting log messages in one of the following +formats: + +- ``rfc5424`` - :rfc:`5424` - modern syslog with ISO 8601 timestamps, time zone and + structured data (key/value pairs) support +- ``rfc3164`` - :rfc:`3164` - legacy BSD syslog, timestamps with 1 second granularity +- ``local-syslog`` - same as :rfc:`3164`, but without the hostname field +- ``journald`` - systemd's `native journald protocol <https://systemd.io/JOURNAL_NATIVE_PROTOCOL/>`_. + This protocol also supports structured data (key/value pairs). + +Destinations +------------ + +The output location is configured with the following subcommands: + +.. clicmd:: destination none + + Disable the target while retaining its remaining configuration. + +.. clicmd:: destination syslog [supports-rfc5424] + + Send log messages to the system's standard log destination + (``/dev/log``). This does not use the C library's ``syslog()`` function, + instead writing directly to ``/dev/log``. + + On NetBSD and FreeBSD, the RFC5424 format is automatically used when + the OS version is recent enough (5.0 for NetBSD, 12.0 for FreeBSD). + Unfortunately, support for this format cannot be autodetected otherwise, + and particularly on Linux systems must be enabled manually. + +.. clicmd:: destination journald + + Send log messages to systemd's journald. + +.. clicmd:: destination <stdout|stderr|fd <(0-63)|envvar WORD>> \ + [format FORMAT] + + Send log messages to one of the daemon's file descriptors. The + ``fd (0-63)`` and ``fd envvar WORD`` variants are intended to work with + the shell's ``command 3>something`` and bash's + ``command {ENVVAR}>something`` I/O redirection specifiers. + + Only file descriptors open at a daemon's startup time can be used for + this; accidental misuse of a file descriptor that has been opened by + FRR itself is prevented. + + Using FIFOs with this option will work but is unsupported and can cause + daemons to hang or crash depending on reader behavior. + + Format defaults to RFC5424 if none is specified. + + .. note:: + + When starting FRR daemons from custom shell scripts, make sure not + to leak / leave extraneous file descriptors open. FRR daemons do not + close these. + +.. clicmd:: destination file PATH \ + [create [{user WORD|group WORD|mode PERMS}]|no-create] \ + [format FORMAT] + + Log to a regular file. File permissions can be specified when FRR creates + the file itself. + + Format defaults to RFC5424 if none is specified. + + .. note:: + + FRR will never change permissions or ownership on an existing log file. + In many cases, FRR will also not have permissions to set user and group + arbitrarily. + +.. clicmd:: destination unix PATH [format FORMAT] + + Connect to a UNIX domain socket and send log messages there. This will + autodetect ``SOCK_STREAM``, ``SOCK_SEQPACKET`` and ``SOCK_DGRAM`` and + adjust behavior appropriately. + +Options +------- + +.. clicmd:: priority PRIORITY + + Select minimum priority of messages to send to this target. Defaults to + `debugging`. + +.. clicmd:: facility FACILITY + + Select syslog facility for messages on this target. Defaults to `daemon`. + The :clicmd:`log facility [FACILITY]` command does not affect extended + targets. + +.. clicmd:: timestamp precision (0-9) + + Set desired number of sub-second timestamp digits. This only has an effect + for RFC5424 and journald format targets; the RFC3164 and local-syslogd + formats do not support any sub-second digits. + +.. clicmd:: timestamp local-time + + Use the local system timezone for timestamps rather than UTC (the default.) + + RFC5424 and journald formats include zone information (``Z`` or ``+-NN:NN`` + suffix in ISO8601). RFC3164 and local-syslogd offer no way of identifying + the time zone used, care must be taken that this option and the receiver + are configured identically, or the timestamp is replaced at the receiver. + + .. note:: + + FRR includes a timestamp in journald messages, but journald always + provides its own timestamp. + +.. clicmd:: structured-data <code-location|version|unique-id|error-category|format-args> + + Select additional key/value data to be included for the RFC5424 and journald + formats. Refer to the next section for details. + + ``unique-id`` and ``error-category`` are enabled by default. + + .. warning:: + + Log messages can grow in size significantly when enabling additional + data. + + +Structured data +--------------- + +When using the RFC5424 or journald formats, FRR can provide additional metadata +for log messages as key/value pairs. The following information can be added +in this way: + ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| Switch | 5424 group | 5424 item(s) | journald field | Contents | ++====================+====================+==============+==================+=============================================+ +| always active | ``location@50145`` | ``tid`` | ``TID`` | Thread ID | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| always active | ``location@50145`` | ``instance`` | ``FRR_INSTANCE`` | Multi-instance number | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``unique-id`` | ``location@50145`` | ``id`` | ``FRR_ID`` | ``XXXXX-XXXXX`` unique message identifier | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``error-category`` | ``location@50145`` | ``ec`` | ``FRR_EC`` | Integer error category number | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``code-location`` | ``location@50145`` | ``file`` | ``CODE_FILE`` | Source code file name | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``code-location`` | ``location@50145`` | ``line`` | ``CODE_LINE`` | Source code line number | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``code-location`` | ``location@50145`` | ``func`` | ``CODE_FUNC`` | Source code function name | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``format-args`` | ``args@50145`` | ``argN`` | ``FRR_ARGn`` | Message printf format arguments (n = 1..16) | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ +| ``version`` | ``origin`` | multiple | n/a | FRR version information (IETF format) | ++--------------------+--------------------+--------------+------------------+---------------------------------------------+ + +The information added by ``version`` is +``[origin enterpriseId="50145" software="FRRouting" swVersion="..."]`` +and is the same for all log messages. (Hence makes little sense to include in +most scenarios.) 50145 is the FRRouting IANA Enterprise Number. + +Crashlogs / backtraces do not include any additional information since it +cannot safely be retrieved from a crash handler. However, all of the above +destinations will deliver crashlogs. + + +Restart and Reconfiguration caveats +----------------------------------- + +FRR uses "add-delete" semantics when reconfiguring log targets of any type +(including both extended targets mentioned here as well as the global +:clicmd:`log stdout LEVEL` and :clicmd:`log syslog [LEVEL]` variants.) This +means that when changing logging configuration, log messages from threads +executing in parallel may be duplicated for a brief window of time. + +For the ``unix``, ``syslog`` and ``journald`` extended destinations, messages +can be lost when the receiver is restarted without the use of socket +activation (i.e. keeping the receiver socket open.) FRR does not buffer +log messages for later delivery, meaning anything logged while the receiver +is unavailable is lost. Since systemd provides socket activation for +journald, no messages will be lost on the ``journald`` target. diff --git a/doc/user/fabricd.rst b/doc/user/fabricd.rst new file mode 100644 index 0000000..48d264f --- /dev/null +++ b/doc/user/fabricd.rst @@ -0,0 +1,304 @@ +.. _fabricd: + +********** +OpenFabric +********** + +OpenFabric, specified in :t:`draft-white-openfabric-06.txt`, is a routing +protocol derived from IS-IS, providing link-state routing with efficient +flooding for topologies like spine-leaf networks. + +FRR implements OpenFabric in a daemon called *fabricd* + +.. _configuring-fabricd: + +Configuring fabricd +=================== + +There are no *fabricd* specific options. Common options can be specified +(:ref:`common-invocation-options`) to *fabricd*. *fabricd* needs to acquire +interface information from *zebra* in order to function. Therefore *zebra* must +be running before invoking *fabricd*. Also, if *zebra* is restarted then *fabricd* +must be too. + +Like other daemons, *fabricd* configuration is done in an OpenFabric specific +configuration file :file:`fabricd.conf`. + +.. _openfabric-router: + +OpenFabric router +================= + +To enable the OpenFabric routing protocol, an OpenFabric router needs to be created +in the configuration: + +.. clicmd:: router openfabric WORD + + Enable or disable the OpenFabric process by specifying the OpenFabric domain with + 'WORD'. + +.. clicmd:: net XX.XXXX. ... .XXX.XX + + Set/Unset network entity title (NET) provided in ISO format. + +.. clicmd:: domain-password [clear | md5] <password> + + Configure the authentication password for a domain, as clear text or md5 one. + +.. clicmd:: attached-bit [receive ignore | send] + + Set attached bit for inter-area traffic: + + - receive + If LSP received with attached bit set, create default route to neighbor + - send + If L1|L2 router, set attached bit in LSP sent to L1 router + +.. clicmd:: log-adjacency-changes + + Log changes in adjacency state. + +.. clicmd:: set-overload-bit + + Set overload bit to avoid any transit traffic. + +.. clicmd:: purge-originator + + + Enable or disable :rfc:`6232` purge originator identification. + +.. clicmd:: fabric-tier (0-14) + + + Configure a static tier number to advertise as location in the fabric + +.. _openfabric-timer: + +OpenFabric Timer +================ + +.. clicmd:: lsp-gen-interval (1-120) + + + Set minimum interval in seconds between regenerating same LSP. + +.. clicmd:: lsp-refresh-interval (1-65235) + + + Set LSP refresh interval in seconds. + +.. clicmd:: max-lsp-lifetime (360-65535) + + + Set LSP maximum LSP lifetime in seconds. + +.. clicmd:: spf-interval (1-120) + + + Set minimum interval between consecutive SPF calculations in seconds. + +.. _openfabric-interface: + +OpenFabric interface +==================== + +.. clicmd:: ip router openfabric WORD + + +.. _ip-router-openfabric-word: + + Activate OpenFabric on this interface. Note that the name + of OpenFabric instance must be the same as the one used to configure the + routing process (see command :clicmd:`router openfabric WORD`). + +.. clicmd:: openfabric csnp-interval (1-600) + + + Set CSNP interval in seconds. + +.. clicmd:: openfabric hello-interval (1-600) + + + Set Hello interval in seconds. + +.. clicmd:: openfabric hello-multiplier (2-100) + + + Set multiplier for Hello holding time. + +.. clicmd:: openfabric metric (0-16777215) + + + Set interface metric value. + +.. clicmd:: openfabric passive + + + Configure the passive mode for this interface. + +.. clicmd:: openfabric password [clear | md5] <password> + + + Configure the authentication password (clear or encoded text) for the + interface. + +.. clicmd:: openfabric psnp-interval (1-120) + + + Set PSNP interval in seconds. + +.. _showing-openfabric-information: + +Showing OpenFabric information +============================== + +.. clicmd:: show openfabric summary + + Show summary information about OpenFabric. + +.. clicmd:: show openfabric hostname + + Show which hostnames are associated with which OpenFabric system ids. + +.. clicmd:: show openfabric interface + +.. clicmd:: show openfabric interface detail + +.. clicmd:: show openfabric interface <interface name> + + Show state and configuration of specified OpenFabric interface, or all interfaces + if no interface is given with or without details. + +.. clicmd:: show openfabric neighbor + +.. clicmd:: show openfabric neighbor <System Id> + +.. clicmd:: show openfabric neighbor detail + + Show state and information of specified OpenFabric neighbor, or all neighbors if + no system id is given with or without details. + +.. clicmd:: show openfabric database + +.. clicmd:: show openfabric database [detail] + +.. clicmd:: show openfabric database <LSP id> [detail] + +.. clicmd:: show openfabric database detail <LSP id> + + Show the OpenFabric database globally, for a specific LSP id without or with + details. + +.. clicmd:: show openfabric topology + + Show calculated OpenFabric paths and associated topology information. + +.. _debugging-openfabric: + +Debugging OpenFabric +==================== + +.. clicmd:: debug openfabric adj-packets + + OpenFabric Adjacency related packets. + +.. clicmd:: debug openfabric checksum-errors + + OpenFabric LSP checksum errors. + +.. clicmd:: debug openfabric events + + OpenFabric Events. + +.. clicmd:: debug openfabric local-updates + + OpenFabric local update packets. + +.. clicmd:: debug openfabric lsp-gen + + Generation of own LSPs. + +.. clicmd:: debug openfabric lsp-sched + + Debug scheduling of generation of own LSPs. + +.. clicmd:: debug openfabric packet-dump + + OpenFabric packet dump. + +.. clicmd:: debug openfabric protocol-errors + + OpenFabric LSP protocol errors. + +.. clicmd:: debug openfabric route-events + + OpenFabric Route related events. + +.. clicmd:: debug openfabric snp-packets + + OpenFabric CSNP/PSNP packets. + +.. clicmd:: debug openfabric spf-events + +.. clicmd:: debug openfabric spf-statistics + +.. clicmd:: debug openfabric spf-triggers + + OpenFabric Shortest Path First Events, Timing and Statistic Data and + triggering events. + +.. clicmd:: debug openfabric update-packets + + Update-related packets. + +.. clicmd:: show debugging openfabric + + Print which OpenFabric debug levels are active. + +Sample configuration +==================== + +A simple example: + +.. code-block:: frr + + ! + interface lo + ip address 192.0.2.1/32 + ip router openfabric 1 + ipv6 address 2001:db8::1/128 + ipv6 router openfabric 1 + ! + interface eth0 + ip router openfabric 1 + ipv6 router openfabric 1 + ! + interface eth1 + ip router openfabric 1 + ipv6 router openfabric 1 + ! + router openfabric 1 + net 49.0000.0000.0001.00 + + +Alternative example: + +.. code-block:: frr + + hostname fabricd + + router openfabric DEAD + net 47.0023.0000.0003.0300.0100.0102.0304.0506.00 + lsp-lifetime 65535 + + hostname isisd-router + domain-password foobar + + interface eth0 + ip router openfabric DEAD + openfabric hello-interval 5 + openfabric lsp-interval 1000 + + ! -- optional + openfabric retransmit-interval 10 + openfabric retransmit-throttle-interval diff --git a/doc/user/filter.rst b/doc/user/filter.rst new file mode 100644 index 0000000..c1146e5 --- /dev/null +++ b/doc/user/filter.rst @@ -0,0 +1,173 @@ +********* +Filtering +********* + +FRR provides many very flexible filtering features. Filtering is used +for both input and output of the routing information. Once filtering is +defined, it can be applied in any direction. + +IP Access List +============== + +.. clicmd:: access-list NAME [seq (1-4294967295)] permit IPV4-NETWORK + +.. clicmd:: access-list NAME [seq (1-4294967295)] deny IPV4-NETWORK + + seq + seq `number` can be set either automatically or manually. In the + case that sequential numbers are set manually, the user may pick any + number less than 4294967295. In the case that sequential number are set + automatically, the sequential number will increase by a unit of five (5) + per list. If a list with no specified sequential number is created + after a list with a specified sequential number, the list will + automatically pick the next multiple of five (5) as the list number. + For example, if a list with number 2 already exists and a new list with + no specified number is created, the next list will be numbered 5. If + lists 2 and 7 already exist and a new list with no specified number is + created, the new list will be numbered 10. + + Basic filtering is done by `access-list` as shown in the + following example. + + .. code-block:: frr + + access-list filter deny 10.0.0.0/9 + access-list filter permit 10.0.0.0/8 + access-list filter seq 13 permit 10.0.0.0/7 + +.. clicmd:: show <ip|ipv6> access-list [json] + + Display all IPv4 or IPv6 access lists. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show <ip|ipv6> access-list WORD [json] + + Display the specified IPv4 or IPv6 access list. + + If the ``json`` option is specified, output is displayed in JSON format. + + +IP Prefix List +============== + +*ip prefix-list* provides the most powerful prefix based +filtering mechanism. In addition to *access-list* functionality, +*ip prefix-list* has prefix length range specification and +sequential number specification. You can add or delete prefix based +filters to arbitrary points of prefix-list using sequential number specification. + +If no ip prefix-list is specified, it acts as permit. If *ip prefix-list* +is defined, and no match is found, default deny is applied. + +.. clicmd:: ip prefix-list NAME (permit|deny) PREFIX [le LEN] [ge LEN] + +.. clicmd:: ip prefix-list NAME seq NUMBER (permit|deny) PREFIX [le LEN] [ge LEN] + + You can create *ip prefix-list* using above commands. + + seq + seq `number` can be set either automatically or manually. In the + case that sequential numbers are set manually, the user may pick any + number less than 4294967295. In the case that sequential number are set + automatically, the sequential number will increase by a unit of five (5) + per list. If a list with no specified sequential number is created + after a list with a specified sequential number, the list will + automatically pick the next multiple of five (5) as the list number. + For example, if a list with number 2 already exists and a new list with + no specified number is created, the next list will be numbered 5. If + lists 2 and 7 already exist and a new list with no specified number is + created, the new list will be numbered 10. + + le + Specifies prefix length. The prefix list will be applied if the prefix + length is less than or equal to the le prefix length. + + ge + Specifies prefix length. The prefix list will be applied if the prefix + length is greater than or equal to the ge prefix length. + + + Less than or equal to prefix numbers and greater than or equal to + prefix numbers can be used together. The order of the le and ge + commands does not matter. + + If a prefix list with a different sequential number but with the exact + same rules as a previous list is created, an error will result. + However, in the case that the sequential number and the rules are + exactly similar, no error will result. + + If a list with the same sequential number as a previous list is created, + the new list will overwrite the old list. + + Matching of IP Prefix is performed from the smaller sequential number to the + larger. The matching will stop once any rule has been applied. + + In the case of no le or ge command, the prefix length must match exactly the + length specified in the prefix list. + + +.. _ip-prefix-list-description: + +ip prefix-list description +-------------------------- + +.. clicmd:: ip prefix-list NAME description DESC + + Descriptions may be added to prefix lists. This command adds a + description to the prefix list. + + +.. _showing-ip-prefix-list: + +Showing ip prefix-list +---------------------- + +.. clicmd:: show ip prefix-list [json] + + Display all IP prefix lists. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show ip prefix-list NAME [json] + + Show IP prefix list can be used with a prefix list name. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show ip prefix-list NAME seq NUM [json] + + Show IP prefix list can be used with a prefix list name and sequential + number. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show ip prefix-list NAME A.B.C.D/M + + If the command longer is used, all prefix lists with prefix lengths equal to + or longer than the specified length will be displayed. If the command first + match is used, the first prefix length match will be displayed. + +.. clicmd:: show ip prefix-list NAME A.B.C.D/M longer +.. clicmd:: show ip prefix-list NAME A.B.C.D/M first-match +.. clicmd:: show ip prefix-list summary [json] +.. clicmd:: show ip prefix-list summary NAME [json] +.. clicmd:: show ip prefix-list detail [json] +.. clicmd:: show ip prefix-list detail NAME [json] + +.. clicmd:: debug prefix-list NAME match <A.B.C.D/M|X:X::X:X/M> [address-mode] + + Execute the prefix list matching code for the specified list and prefix. + Shows which entry matched, if any. (``address-mode`` is used for + PIM RP lookups and skips prefix length checks.) + + The return value from this command is success only if the prefix-list + result is to permit the prefix, so the command can be used in scripting. + +Clear counter of ip prefix-list +------------------------------- + +.. clicmd:: clear ip prefix-list [NAME [A.B.C.D/M]] + + Clears the counters of all IP prefix lists. Clear IP Prefix List can be used + with a specified NAME or NAME and prefix. diff --git a/doc/user/flowspec.rst b/doc/user/flowspec.rst new file mode 100644 index 0000000..faf5973 --- /dev/null +++ b/doc/user/flowspec.rst @@ -0,0 +1,376 @@ +.. _flowspec: + +Flowspec +======== + +.. _features-of-the-current-implementation-flowspec: + +Overview +--------- + +Flowspec introduces a new :abbr:`NLRI (Network Layer Reachability Information)` +encoding format that is used to distribute traffic rule flow specifications. +Basically, instead of simply relying on destination IP address for IP prefixes, +the IP prefix is replaced by a n-tuple consisting of a rule. That rule can be a +more or less complex combination of the following: + + +- Network source/destination (can be one or the other, or both). +- Layer 4 information for UDP/TCP: source port, destination port, or any port. +- Layer 4 information for ICMP type and ICMP code. +- Layer 4 information for TCP Flags. +- Layer 3 information: DSCP value, Protocol type, packet length, fragmentation. +- Misc layer 4 TCP flags. + +Note that if originally Flowspec defined IPv4 rules, this is also possible to use +IPv6 address-family. The same set of combinations as defined for IPv4 can be used. + +A combination of the above rules is applied for traffic filtering. This is +encoded as part of specific BGP extended communities and the action can range +from the obvious rerouting (to nexthop or to separate VRF) to shaping, or +discard. + +The following IETF drafts and RFCs have been used to implement FRR Flowspec: + +- :rfc:`5575` +- [Draft-IETF-IDR-Flowspec-redirect-IP]_ +- [Draft-IETF-IDR-Flow-Spec-V6]_ + +.. _design-principles-flowspec: + +Design Principles +----------------- + +FRR implements the Flowspec client side, that is to say that BGP is able to +receive Flowspec entries, but is not able to act as manager and send Flowspec +entries. + +Linux provides the following mechanisms to implement policy based routing: + +- Filtering the traffic with ``Netfilter``. + ``Netfilter`` provides a set of tools like ``ipset`` and ``iptables`` that are + powerful enough to be able to filter such Flowspec filter rule. + +- using non standard routing tables via ``iproute2`` (via the ``ip rule`` + command provided by ``iproute2``). + ``iproute2`` is already used by FRR's :ref:`pbr` daemon which provides basic + policy based routing based on IP source and destination criterion. + +Below example is an illustration of what Flowspec will inject in the underlying +system: + +.. code-block:: shell + + # linux shell + ipset create match0x102 hash:net,net counters + ipset add match0x102 32.0.0.0/16,40.0.0.0/16 + iptables -N match0x102 -t mangle + iptables -A match0x102 -t mangle -j MARK --set-mark 102 + iptables -A match0x102 -t mangle -j ACCEPT + iptables -i ntfp3 -t mangle -I PREROUTING -m set --match-set match0x102 + src,dst -g match0x102 + ip rule add fwmark 102 lookup 102 + ip route add 40.0.0.0/16 via 44.0.0.2 table 102 + +For handling an incoming Flowspec entry, the following workflow is applied: + +- Incoming Flowspec entries are handled by *bgpd*, stored in the BGP RIB. +- Flowspec entry is installed according to its complexity. + +It will be installed if one of the following filtering action is seen on the +BGP extended community: either redirect IP, or redirect VRF, in conjunction +with rate option, for redirecting traffic. Or rate option set to 0, for +discarding traffic. + +According to the degree of complexity of the Flowspec entry, it will be +installed in *zebra* RIB. For more information about what is supported in the +FRR implementation as rule, see :ref:`flowspec-known-issues` chapter. Flowspec +entry is split in several parts before being sent to *zebra*. + +- *zebra* daemon receives the policy routing configuration + +Policy Based Routing entities necessary to policy route the traffic in the +underlying system, are received by *zebra*. Two filtering contexts will be +created or appended in ``Netfilter``: ``ipset`` and ``iptable`` context. The +former is used to define an IP filter based on multiple criterium. For +instance, an ipset ``net:net`` is based on two ip addresses, while +``net,port,net`` is based on two ip addresses and one port (for ICMP, UDP, or +TCP). The way the filtering is used (for example, is src port or dst port +used?) is defined by the latter filtering context. ``iptable`` command will +reference the ``ipset`` context and will tell how to filter and what to do. In +our case, a marker will be set to indicate ``iproute2`` where to forward the +traffic to. Sometimes, for dropping action, there is no need to add a marker; +the ``iptable`` will tell to drop all packets matching the ``ipset`` entry. + +Configuration Guide +------------------- + +In order to configure an IPv4 Flowspec engine, use the following configuration. +As of today, it is only possible to configure Flowspec on the default VRF. + +.. code-block:: frr + + router bgp <AS> + neighbor <A.B.C.D> remote-as <remoteAS> + neighbor <A:B::C:D> remote-as <remoteAS2> + address-family ipv4 flowspec + neighbor <A.B.C.D> activate + exit + address-family ipv6 flowspec + neighbor <A:B::C:D> activate + exit + exit + +You can see Flowspec entries, by using one of the following show commands: + +.. clicmd:: show bgp ipv4 flowspec [detail | A.B.C.D] + +.. clicmd:: show bgp ipv6 flowspec [detail | A:B::C:D] + +Per-interface configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +One nice feature to use is the ability to apply Flowspec to a specific +interface, instead of applying it to the whole machine. Despite the following +IETF draft [Draft-IETF-IDR-Flowspec-Interface-Set]_ is not implemented, it is +possible to manually limit Flowspec application to some incoming interfaces. +Actually, not using it can result to some unexpected behaviour like accounting +twice the traffic, or slow down the traffic (filtering costs). To limit +Flowspec to one specific interface, use the following command, under +`flowspec address-family` node. + +.. clicmd:: local-install <IFNAME | any> + +By default, Flowspec is activated on all interfaces. Installing it to a named +interface will result in allowing only this interface. Conversely, enabling any +interface will flush all previously configured interfaces. + +VRF redirection +^^^^^^^^^^^^^^^ + +Another nice feature to configure is the ability to redirect traffic to a +separate VRF. This feature does not go against the ability to configure +Flowspec only on default VRF. Actually, when you receive incoming BGP flowspec +entries on that default VRF, you can redirect traffic to an other VRF. + +As a reminder, BGP flowspec entries have a BGP extended community that contains +a Route Target. Finding out a local VRF based on Route Target consists in the +following: + +- A configuration of each VRF must be done, with its Route Target set + Each VRF is being configured within a BGP VRF instance with its own Route + Target list. Route Target accepted format matches the following: + ``A.B.C.D:U16``, or ``U16:U32``, ``U32:U16``. + +- The first VRF with the matching Route Target will be selected to route traffic + to. Use the following command under ipv4 unicast address-family node + +.. clicmd:: rt redirect import RTLIST... + +In order to illustrate, if the Route Target configured in the Flowspec entry is +``E.F.G.H:II``, then a BGP VRF instance with the same Route Target will be set +set. That VRF will then be selected. The below full configuration example +depicts how Route Targets are configured and how VRFs and cross VRF +configuration is done. Note that the VRF are mapped on Linux Network +Namespaces. For data traffic to cross VRF boundaries, virtual ethernet +interfaces are created with private IP addressing scheme. + +.. code-block:: frr + + router bgp <ASx> + neighbor <A.B.C.D> remote-as <ASz> + address-family ipv4 flowspec + neighbor A.B.C.D activate + exit + exit + router bgp <ASy> vrf vrf2 + address-family ipv4 unicast + rt redirect import <E.F.G.H:II> + exit + exit + +Similarly, it is possible to do the same for IPv6 flowspec rules, by using +an IPv6 extended community. The format is defined on :rfc:`5701`, and that +community contains an IPv6 address encoded in the attribute, and matches the +locally configured imported route target IPv6 defined under the appropriate +BGP VRF instance. Below example defines an IPv6 extended community containing +`E:F::G:H` address followed by 2 bytes chosen by admin ( here `JJ`). + +.. code-block:: frr + + router bgp <ASx> + neighbor <A:B::C:D> remote-as <ASz> + address-family ipv6 flowspec + neighbor A:B::C:D activate + exit + exit + router bgp <ASy> vrf vrf2 + address-family ipv6 unicast + rt6 redirect import <E:F::G:H:JJ> + exit + exit + + +Flowspec monitoring & troubleshooting +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You can monitor policy-routing objects by using one of the following commands. +Those command rely on the filtering contexts configured from BGP, and get the +statistics information retrieved from the underlying system. In other words, +those statistics are retrieved from ``Netfilter``. + +.. clicmd:: show pbr ipset IPSETNAME | iptable + +``IPSETNAME`` is the policy routing object name created by ``ipset``. About +rule contexts, it is possible to know which rule has been configured to +policy-route some specific traffic. The :clicmd:`show pbr iptable` command +displays for forwarded traffic, which table is used. Then it is easy to use +that table identifier to dump the routing table that the forwarded traffic will +match. + +.. code-block:: frr + +.. clicmd:: show ip route table TABLEID + + ``TABLEID`` is the table number identifier referencing the non standard + routing table used in this example. + +.. clicmd:: debug bgp flowspec + + You can troubleshoot Flowspec, or BGP policy based routing. For instance, if + you encounter some issues when decoding a Flowspec entry, you should enable + :clicmd:`debug bgp flowspec`. + +.. clicmd:: debug bgp pbr [error] + + If you fail to apply the flowspec entry into *zebra*, there should be some + relationship with policy routing mechanism. Here, + :clicmd:`debug bgp pbr error` could help. + + To get information about policy routing contexts created/removed, only use + :clicmd:`debug bgp pbr` command. + +Ensuring that a Flowspec entry has been correctly installed and that incoming +traffic is policy-routed correctly can be checked as demonstrated below. First +of all, you must check whether the Flowspec entry has been installed or not. + +.. code-block:: frr + + CLI# show bgp ipv4 flowspec 5.5.5.2/32 + BGP flowspec entry: (flags 0x418) + Destination Address 5.5.5.2/32 + IP Protocol = 17 + Destination Port >= 50 , <= 90 + FS:redirect VRF RT:255.255.255.255:255 + received for 18:41:37 + installed in PBR (match0x271ce00) + +This means that the Flowspec entry has been installed in an ``iptable`` named +``match0x271ce00``. Once you have confirmation it is installed, you can check +whether you find the associate entry by executing following command. You can +also check whether incoming traffic has been matched by looking at counter +line. + +.. code-block:: frr + + CLI# show pbr ipset match0x271ce00 + IPset match0x271ce00 type net,port + to 5.5.5.0/24:proto 6:80-120 (8) + pkts 1000, bytes 1000000 + to 5.5.5.2:proto 17:50-90 (5) + pkts 1692918, bytes 157441374 + +As you can see, the entry is present. note that an ``iptable`` entry can be +used to host several Flowspec entries. In order to know where the matching +traffic is redirected to, you have to look at the policy routing rules. The +policy-routing is done by forwarding traffic to a routing table number. That +routing table number is reached by using a ``iptable``. The relationship +between the routing table number and the incoming traffic is a ``MARKER`` that +is set by the IPtable referencing the IPSet. In Flowspec case, ``iptable`` +referencing the ``ipset`` context have the same name. So it is easy to know +which routing table is used by issuing following command: + +.. code-block:: frr + + CLI# show pbr iptable + IPtable match0x271ce00 action redirect (5) + pkts 1700000, bytes 158000000 + table 257, fwmark 257 + ... + +As you can see, by using following Linux commands, the MARKER ``0x101`` is +present in both ``iptable`` and ``ip rule`` contexts. + +.. code-block:: shell + + # iptables -t mangle --list match0x271ce00 -v + Chain match0x271ce00 (1 references) + pkts bytes target prot opt in out source destination + 1700K 158M MARK all -- any any anywhere anywhere + MARK set 0x101 + 1700K 158M ACCEPT all -- any any anywhere anywhere + + # ip rule list + 0:from all lookup local + 0:from all fwmark 0x101 lookup 257 + 32766:from all lookup main + 32767:from all lookup default + +This allows us to see where the traffic is forwarded to. + +.. _flowspec-known-issues: + +Limitations / Known Issues +-------------------------- + +As you can see, Flowspec is rich and can be very complex. As of today, not all +Flowspec rules will be able to be converted into Policy Based Routing actions. + +- The ``Netfilter`` driver is not integrated into FRR yet. Not having this + piece of code prevents from injecting flowspec entries into the underlying + system. + +- There are some limitations around filtering contexts + + If I take example of UDP ports, or TCP ports in Flowspec, the information + can be a range of ports, or a unique value. This case is handled. + However, complexity can be increased, if the flow is a combination of a list + of range of ports and an enumerate of unique values. Here this case is not + handled. Similarly, it is not possible to create a filter for both src port + and dst port. For instance, filter on src port from [1-1000] and dst port = + 80. The same kind of complexity is not possible for packet length, ICMP type, + ICMP code. + +There are some other known issues: + +- The validation procedure depicted in :rfc:`5575` is not available. + + This validation procedure has not been implemented, as this feature was not + used in the existing setups you shared with us. + +- The filtering action shaper value, if positive, is not used to apply shaping. + + If value is positive, the traffic is redirected to the wished destination, + without any other action configured by Flowspec. + It is recommended to configure Quality of Service if needed, more globally on + a per interface basis. + +- Upon an unexpected crash or other event, *zebra* may not have time to flush + PBR contexts. + + That is to say ``ipset``, ``iptable`` and ``ip rule`` contexts. This is also a + consequence due to the fact that ip rule / ipset / iptables are not discovered + at startup (not able to read appropriate contexts coming from Flowspec). + +Appendix +-------- + +More information with a public presentation that explains the design of Flowspec +inside FRRouting. + +[Presentation]_ + +.. [Draft-IETF-IDR-Flowspec-redirect-IP] <https://tools.ietf.org/id/draft-ietf-idr-flowspec-redirect-ip-02.txt> +.. [Draft-IETF-IDR-Flowspec-Interface-Set] <https://tools.ietf.org/id/draft-ietf-idr-flowspec-interfaceset-03.txt> +.. [Draft-IETF-IDR-Flow-Spec-V6] <https://tools.ietf.org/id/draft-ietf-idr-flow-spec-v6-10.txt> +.. [Presentation] <https://docs.google.com/presentation/d/1ekQygUAG5yvQ3wWUyrw4Wcag0LgmbW1kV02IWcU4iUg/edit#slide=id.g378f0e1b5e_1_44> diff --git a/doc/user/frr-reload.rst b/doc/user/frr-reload.rst new file mode 100644 index 0000000..bd295db --- /dev/null +++ b/doc/user/frr-reload.rst @@ -0,0 +1,41 @@ +.. _frr-reload: + + +The frr-reload.py script +======================== + +The ``frr-reload.py`` script attempts to update the configuration of running +daemons. It takes as argument the path of the configuration file that we want +to apply. The script will attempt to retrieve the running configuration from +daemons, calculate the delta between that config and the intended one, and +execute the required sequence of vtysh commands to enforce the changes. + +Options +------- + +There are several options that control the behavior of ``frr-reload``: + +* ``--input INPUT``: uses the specified input file as the running configuration + instead of retrieving it from a ``show running-config`` in vtysh +* ``--reload``: applies the configuration delta to the daemons. Either this or + ``--test`` MUST be specified. +* ``--test``: only outputs the configuration delta, without enforcing it. + Either this or ``--reload`` MUST be specified. +* ``--debug``: enable debug messages +* ``--stdout``: print output to stdout +* ``--bindir BINDIR``: path to the vtysh executable +* ``--confdir CONFDIR``: path to the existing daemon config files +* ``--rundir RUNDIR``: path to a folder to be used to write the temporary files + needed by the script to do its job. The script should have write access to it +* ``--daemon DAEMON``: by default ``frr-reload.py`` assumes that we are using + integrated config and attempting to update the configuration for all daemons. + If this is not the case, e.g. each daemon has its individual config file, + then the delta can only be computed on a per-daemon basis. This option allows + the user to specify the daemon for which the config is intended. DAEMON + should be one of the keywords allowed in vtysh as an option for ``show + running-config``. +* ``--vty_socket VTY_SOCKET``: the socket to be used by vtysh to connect to the + running daemons. +* ``--overwrite``: overwrite the existing daemon config file with the new + config after the delta has been applied. The file name will be ``frr.conf`` + for integrate config, or ``DAEMON.conf`` when using per-daemon config files. diff --git a/doc/user/glossary.rst b/doc/user/glossary.rst new file mode 100644 index 0000000..6b49336 --- /dev/null +++ b/doc/user/glossary.rst @@ -0,0 +1,41 @@ +******** +Glossary +******** + +.. glossary:: + + distance-vector + A distance-vector routing protocol in data networks determines the best + route for data packets based on distance. Distance-vector routing + protocols measure the distance by the number of routers a packet has to + pass. Some distance-vector protocols also take into account network + latency and other factors that influence traffic on a given route. To + determine the best route across a network, routers on which a + distance-vector protocol is implemented exchange information with one + another, usually routing tables plus hop counts for destination networks + and possibly other traffic information. Distance-vector routing protocols + also require that a router informs its neighbours of network topology + changes periodically. [distance-vector-rp]_ + + link-state + Link-state algorithms (also known as shortest path first algorithms) + flood routing information to all nodes in the internetwork. Each router, + however, sends only the portion of the routing table that describes the + state of its own links. In link-state algorithms, each router builds a + picture of the entire network in its routing tables. Distance vector + algorithms (also known as Bellman-Ford algorithms) call for each router + to send all or some portion of its routing table, but only to its + neighbors. In essence, link-state algorithms send small updates + everywhere, while distance vector algorithms send larger updates only to + neighboring routers. Distance vector algorithms know only about their + neighbors. [link-state-rp]_ + + Bellman-Ford + The Bellman–Ford algorithm is an algorithm that computes shortest paths + from a single source vertex to all of the other vertices in a weighted + digraph. [bellman-ford]_ + + +.. [distance-vector-rp] https://en.wikipedia.org/wiki/Distance-vector_routing_protocol +.. [link-state-rp] https://en.wikipedia.org/wiki/Link-state_routing_protocol +.. [bellman-ford] https://en.wikipedia.org/wiki/Bellman-Ford_algorithm diff --git a/doc/user/grpc.rst b/doc/user/grpc.rst new file mode 100644 index 0000000..d6767fc --- /dev/null +++ b/doc/user/grpc.rst @@ -0,0 +1,66 @@ +.. _grpc: + +*************** +Northbound gRPC +*************** + +.. program:: configure + +*gRPC* provides a combined front end to all FRR daemons using the YANG +northbound. It is currently disabled by default due its experimental +stage, but it can be enabled with :option:`--enable-grpc` option in the +configure script. + + +.. _grpc-features: + +Northbound gRPC Features +======================== + +* Get/set configuration using JSON/XML/XPath encondings. +* Execute YANG RPC calls. +* Lock/unlock configuration. +* Create/edit/load/update/commit candidate configuration. +* List/get transactions. + + +.. note:: + + There is currently no support for YANG notifications. + + +.. note:: + + You can find more information on how to code programs to interact + with FRR by reading the gRPC Programming Language Bindings section + in the `developer's documentation + <http://docs.frrouting.org/projects/dev-guide/en/latest/grpc.html>`_. + + +.. _grpc-config: + +Daemon gRPC Configuration +========================= + +The *gRPC* module accepts the following run time option: + +- ``port``: the port to listen to (defaults to ``50051``). + + +.. note:: + + At the moment only localhost connections with no SSL/TLS are + supported. + + +To configure FRR daemons to listen to gRPC you need to append the +following parameter to the daemon's command line: ``-M grpc`` +(optionally ``-M grpc:PORT`` to specify listening port). + +To do that in production you need to edit the ``/etc/frr/daemons`` file +so the daemons get started with the command line argument. Example: + +:: + + # other daemons... + bfdd_options=" --daemon -A 127.0.0.1 -M grpc" diff --git a/doc/user/images/pathd_config.png b/doc/user/images/pathd_config.png Binary files differnew file mode 100644 index 0000000..b7be1a6 --- /dev/null +++ b/doc/user/images/pathd_config.png diff --git a/doc/user/images/pathd_general.png b/doc/user/images/pathd_general.png Binary files differnew file mode 100644 index 0000000..1da6d46 --- /dev/null +++ b/doc/user/images/pathd_general.png diff --git a/doc/user/images/pathd_initiated_multi.png b/doc/user/images/pathd_initiated_multi.png Binary files differnew file mode 100644 index 0000000..5c818e6 --- /dev/null +++ b/doc/user/images/pathd_initiated_multi.png diff --git a/doc/user/index.rst b/doc/user/index.rst new file mode 100644 index 0000000..4789677 --- /dev/null +++ b/doc/user/index.rst @@ -0,0 +1,103 @@ +FRRouting User Guide +==================== + +############ +Introduction +############ + +.. _introduction: +.. toctree:: + :maxdepth: 2 + + overview + installation + setup + +###### +Basics +###### + +.. _basics: +.. toctree:: + :maxdepth: 2 + + basic + extlog + vtysh + grpc + filter + routemap + affinitymap + ipv6 + kernel + snmp + scripting + nexthop_groups +.. modules + +######### +Protocols +######### + +.. _protocols: +.. toctree:: + :maxdepth: 2 + + zebra + bfd + bgp + babeld + fabricd + ldpd + eigrpd + evpn + isisd + nhrpd + ospfd + ospf6d + pathd + pim + pimv6 + pbr + ripd + ripngd + sharp + static + vnc + vrrp + bmp + watchfrr + mgmtd + +######## +Appendix +######## + +.. _appendix: +.. toctree:: + :maxdepth: 2 + + bugs + packet-dumps + glossary + frr-reload + +################ +Copyright notice +################ + +Copyright (c) 1996-2018 Kunihiro Ishiguro, et al. + +Permission is granted to make and distribute verbatim copies of this +manual provided the copyright notice and this permission notice are +preserved on all copies. + +Permission is granted to copy and distribute modified versions of this +manual under the conditions for verbatim copying, provided that the +entire resulting derived work is distributed under the terms of a +permission notice identical to this one. + +Permission is granted to copy and distribute translations of this manual +into another language, under the above conditions for modified versions, +except that this permission notice may be stated in a translation +approved by Kunihiro Ishiguro. diff --git a/doc/user/installation.rst b/doc/user/installation.rst new file mode 100644 index 0000000..24c6c22 --- /dev/null +++ b/doc/user/installation.rst @@ -0,0 +1,597 @@ +.. index:: + single: How to install FRR + single: Installing FRR + single: Building FRR + +.. _installation: + +Installation +============ + +This section covers the basics of building, installing and setting up FRR. + + +From Packages +------------- + +The project publishes packages for Red Hat, Centos, Debian and Ubuntu on the +`GitHub releases <https://github.com/FRRouting/frr/releases>`_. page. External +contributors offer packages for many other platforms including \*BSD, Alpine, +Gentoo, Docker, and others. There is currently no documentation on how to use +those but we hope to add it soon. + +From Snapcraft +-------------- + +In addition to traditional packages the project also builds and publishes +universal Snap images, available at https://snapcraft.io/frr. + +From Source +----------- + +Building FRR from source is the best way to ensure you have the latest features +and bug fixes. Details for each supported platform, including dependency +package listings, permissions, and other gotchas, are in the `developer's +documentation +<http://docs.frrouting.org/projects/dev-guide/en/latest/building.html>`_. This +section provides a brief overview on the process. + + +Getting the Source +^^^^^^^^^^^^^^^^^^ + +FRR's source is available on the project +`GitHub page <https://github.com/FRRouting/frr>`_. + +.. code-block:: shell + + git clone https://github.com/FRRouting/frr.git + +When building from Git there are several branches to choose from. The +``master`` branch is the primary development branch. It should be considered +unstable. Each release has its own branch named ``stable/X.X``, where ``X.X`` +is the release version. + +In addition, release tarballs are published on the GitHub releases page +`here <https://github.com/FRRouting/frr/releases>`_. + + +.. index:: + single: Configuration options + single: Options for configuring + single: Build options + single: Distribution configuration + single: Options to `./configure` + +.. _build-configuration: + +Build Configuration +^^^^^^^^^^^^^^^^^^^ + +FRR has an excellent configure script which automatically detects most host +configurations. There are several additional configure options to customize the +build to include or exclude specific features and dependencies. + +First, update the build system. Change into your FRR source directory and issue: + +.. code-block:: shell + + ./bootstrap.sh + +This will install any missing build scripts and update the Autotools +configuration. Once this is done you can move on to choosing your configuration +options from the list below. + +.. _frr-configuration: + +.. program:: configure + +.. option:: --enable-tcmalloc + + Enable the alternate malloc library. In some cases this is faster and more efficient, + in some cases it is not. + +.. option:: --disable-doc + + Do not build any documentation, including this one. + +.. option:: --enable-doc-html + + From the documentation build html docs as well in addition to the normal output. + +.. option:: --disable-zebra + + Do not build zebra daemon. This generally only be useful in a scenario where + you are building bgp as a standalone server. + +.. option:: --disable-ripd + + Do not build ripd. + +.. option:: --disable-ripngd + + Do not build ripngd. + +.. option:: --disable-ospfd + + Do not build ospfd. + +.. option:: --disable-ospf6d + + Do not build ospf6d. + +.. option:: --disable-bgpd + + Do not build bgpd. + +.. option:: --disable-ldpd + + Do not build ldpd. + +.. option:: --disable-nhrpd + + Do not build nhrpd. + +.. option:: --disable-eigrpd + + Do not build eigrpd. + +.. option:: --disable-babeld + + Do not build babeld. + +.. option:: --disable-watchfrr + + Do not build watchfrr. Watchfrr is used to integrate daemons into startup/shutdown + software available on your machine. This is needed for systemd integration, if you + disable watchfrr you cannot have any systemd integration. + +.. option:: --enable-werror + + Build with all warnings converted to errors as a compile option. This + is recommended for developers only. + +.. option:: --disable-pimd + + Turn off building of pimd. On some BSD platforms pimd will not build properly due + to lack of kernel support. + +.. option:: --disable-vrrpd + + Turn off building of vrrpd. Linux is required for vrrpd support; + other platforms are not supported. + +.. option:: --disable-pbrd + + Turn off building of pbrd. This daemon currently requires linux in order to function + properly. + +.. option:: --enable-sharpd + + Turn on building of sharpd. This daemon facilitates testing of FRR and can also + be used as a quick and easy route generator. + +.. option:: --disable-staticd + + Do not build staticd. This daemon is necessary if you want static routes. + +.. option:: --disable-bfdd + + Do not build bfdd. + +.. option:: --disable-bgp-announce + + Make *bgpd* which does not make bgp announcements at all. This + feature is good for using *bgpd* as a BGP announcement listener. + +.. option:: --disable-bgp-vnc + + Turn off bgpd's ability to use VNC. + +.. option:: --disable-bgp-bmp + + Turn off BGP BMP support + +.. option:: --enable-datacenter + + This option is deprecated as it is superseded by the `-F` (profile) command + line option which allows adjusting the setting at startup rather than + compile time. + + Enable system defaults to work as if in a Data Center. See defaults.h + for what is changed by this configure option. + +.. option:: --enable-snmp + + Enable SNMP support. By default, SNMP support is disabled. + +.. option:: --disable-ospfapi + + Disable support for OSPF-API, an API to interface directly with ospfd. + OSPF-API is enabled if --enable-opaque-lsa is set. + +.. option:: --disable-ospfclient + + Disable installation of the python ospfclient and building of the example + OSPF-API client. + +.. option:: --disable-isisd + + Do not build isisd. + +.. option:: --disable-fabricd + + Do not build fabricd. + +.. option:: --enable-isis-topology + + Enable IS-IS topology generator. + +.. option:: --enable-realms + + Enable the support of Linux Realms. Convert tag values from 1-255 into a + realm value when inserting into the Linux kernel. Then routing policy can be + assigned to the realm. See the tc man page. This option is currently not + compatible with the usage of nexthop groups in the linux kernel itself. + +.. option:: --disable-irdp + + Disable IRDP server support. This is enabled by default if we have + both `struct in_pktinfo` and `struct icmphdr` available to us. + +.. option:: --disable-rtadv + + Disable support IPV6 router advertisement in zebra. + +.. option:: --enable-gcc-rdynamic + + Pass the ``-rdynamic`` option to the linker driver. This is in most cases + necessary for getting usable backtraces. This option defaults to on if the + compiler is detected as gcc, but giving an explicit enable/disable is + suggested. + +.. option:: --disable-backtrace + + Controls backtrace support for the crash handlers. This is autodetected by + default. Using the switch will enforce the requested behaviour, failing with + an error if support is requested but not available. On BSD systems, this + needs libexecinfo, while on glibc support for this is part of libc itself. + +.. option:: --enable-dev-build + + Turn on some options for compiling FRR within a development environment in + mind. Specifically turn on -g3 -O0 for compiling options and add inclusion + of grammar sandbox. + +.. option:: --disable-snmp + + Build without SNMP support. + +.. option:: --disable-vtysh + + Build without VTYSH. + +.. option:: --enable-fpm + + Build with FPM module support. + +.. option:: --with-service-timeout=X + + Set timeout value for FRR service. The time of restarting or reloading FRR + service should not exceed this value. This number can be from 0-999. + Additionally if this parameter is not passed or setting X = 0, FRR will take + default value: 2 minutes. + +.. option:: --enable-numeric-version + + Alpine Linux does not allow non-numeric characters in the version string. + With this option, we provide a way to strip out these characters for APK dev + package builds. + +.. option:: --disable-version-build-config + + Remove the "configuerd with" field that has all of the build configuration + arguments when reporting the version string in `show version` command. + +.. option:: --with-pkg-extra-version=VER + + Add extra version field, for packagers/distributions + +.. option:: --with-pkg-git-version + + Add git information to MOTD and build version string + +.. option:: --enable-multipath=X + + Compile FRR with up to X way ECMP supported. This number can be from 0-999. + For backwards compatibility with older configure options when setting X = 0, + we will build FRR with 64 way ECMP. This is needed because there are + hardcoded arrays that FRR builds towards, so we need to know how big to + make these arrays at build time. Additionally if this parameter is + not passed in FRR will default to 16 ECMP. + +.. option:: --enable-shell-access + + Turn on the ability of FRR to access some shell options( telnet/ssh/bash/etc. ) + from vtysh itself. This option is considered extremely unsecure and should only + be considered for usage if you really really know what you are doing. This + option is deprecated and will be removed on Feb 1, 2024. + +.. option:: --enable-gcov + + Code coverage reports from gcov require adjustments to the C and LD flags. + With this option, gcov instrumentation is added to the build and coverage + reports are created during execution. The check-coverage make target is + also created to ease report uploading to codecov.io. The upload requires + the COMMIT (git hash) and TOKEN (codecov upload token) environment variables + be set. + +.. option:: --enable-config-rollbacks + + Build with configuration rollback support. Requires SQLite3. + +.. option:: --enable-confd=<dir> + + Build the ConfD northbound plugin. Look for the libconfd libs and headers + in `dir`. + +.. option:: --enable-sysrepo + + Build the Sysrepo northbound plugin. + +.. option:: --enable-grpc + + Enable the gRPC northbound plugin. + +.. option:: --enable-zeromq + + Enable the ZeroMQ handler. + +.. option:: --with-libpam + + Use libpam for PAM support in vtysh. + +.. option:: --enable-pcreposix + + Turn on the usage of PCRE Posix libs for regex functionality. + +.. option:: --enable-pcre2posix + + Turn on the usage of PCRE2 Posix libs for regex functionality. + + PCRE2 versions <= 10.31 work a bit differently. We suggest using at least + >= 10.36. + +.. option:: --enable-rpath + + Set hardcoded rpaths in the executable [default=yes]. + +.. option:: --enable-scripting + + Enable Lua scripting [default=no]. + +You may specify any combination of the above options to the configure +script. By default, the executables are placed in :file:`/usr/local/sbin` +and the configuration files in :file:`/usr/local/etc`. The :file:`/usr/local/` +installation prefix and other directories may be changed using the following +options to the configuration script. + +.. option:: --enable-ccls + + Enable the creation of a :file:`.ccls` file in the top level source + directory. + + Some development environments (e.g., LSP server within emacs, et al.) can + utilize :clicmd:`ccls` to provide highly sophisticated IDE features (e.g., + semantically accurate jump-to definition/reference, and even code + refactoring). The `--enable-ccls` causes :file:`configure` to generate a + configuration for the :clicmd:`ccls` command, based on the configured + FRR build environment. + +.. option:: --prefix <prefix> + + Install architecture-independent files in `prefix` [/usr/local]. + +.. option:: --sysconfdir <dir> + + Look for configuration files in `dir` [`prefix`/etc]. Note that sample + configuration files will be installed here. + +.. option:: --localstatedir <dir> + + Configure zebra to use `dir` for local state files, such as pid files and + unix sockets. + +.. option:: --with-scriptdir <dir> + + Look for Lua scripts in ``dir`` [``prefix``/etc/frr/scripts]. + +.. option:: --with-yangmodelsdir <dir> + + Look for YANG modules in `dir` [`prefix`/share/yang]. Note that the FRR + YANG modules will be installed here. + +.. option:: --with-vici-socket <path> + + Set StrongSWAN vici interface socket path [/var/run/charon.vici]. + +.. note:: + + The former ``--enable-systemd`` option does not exist anymore. Support for + systemd is now always available through built-in functions, without + depending on libsystemd. + +Python dependency, documentation and tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +FRR's documentation and basic unit tests heavily use code written in Python. +Additionally, FRR ships Python extensions written in C which are used during +its build process. + +To this extent, FRR needs the following: + +* an installation of CPython, preferably version 3.2 or newer (2.7 works but + is end of life and will stop working at some point.) +* development files (mostly headers) for that version of CPython +* an installation of `sphinx` for that version of CPython, to build the + documentation +* an installation of `pytest` for that version of CPython, to run the unit + tests + +The `sphinx` and `pytest` dependencies can be avoided by not building +documentation / not running ``make check``, but the CPython dependency is a +hard dependency of the FRR build process (for the `clippy` tool.) + +.. index:: + single: FRR Least-Privileges + single: FRR Privileges + +.. _least-privilege-support: + +Least-Privilege Support +""""""""""""""""""""""" + +Additionally, you may configure zebra to drop its elevated privileges +shortly after startup and switch to another user. The configure script will +automatically try to configure this support. There are three configure +options to control the behaviour of FRR daemons. + +.. option:: --enable-user <user> + + Switch to user `user shortly after startup, and run as user `user` in normal + operation. + +.. option:: --enable-group <user> + + Switch real and effective group to `group` shortly after startup. + +.. option:: --enable-vty-group <group> + + Create Unix Vty sockets (for use with vtysh) with group ownership set to + `group`. This allows one to create a separate group which is restricted to + accessing only the vty sockets, hence allowing one to delegate this group to + individual users, or to run vtysh setgid to this group. + +The default user and group which will be configured is 'frr' if no user or +group is specified. Note that this user or group requires write access to the +local state directory (see :option:`--localstatedir`) and requires at least +read access, and write access if you wish to allow daemons to write out their +configuration, to the configuration directory (see :option:`--sysconfdir`). + +On systems which have the 'libcap' capabilities manipulation library (currently +only Linux), FRR will retain only minimal capabilities required and will only +raise these capabilities for brief periods. On systems without libcap, FRR will +run as the user specified and only raise its UID to 0 for brief periods. + + +.. index:: + pair: building; Linux + pair: configuration; Linux + +Linux Notes +""""""""""" + +There are several options available only to GNU/Linux systems. If you use +GNU/Linux, make sure that the current kernel configuration is what you want. +FRR will run with any kernel configuration but some recommendations do exist. + +:makevar:`CONFIG_NETLINK` + Kernel/User Netlink socket. This enables an advanced interface between + the Linux kernel and *zebra* (:ref:`kernel-interface`). + +:makevar:`CONFIG_RTNETLINK` + This makes it possible to receive Netlink routing messages. If you specify + this option, *zebra* can detect routing information updates directly from + the kernel (:ref:`kernel-interface`). + +:makevar:`CONFIG_IP_MULTICAST` + This option enables IP multicast and should be specified when you use *ripd* + (:ref:`rip`) or *ospfd* (:ref:`ospfv2`) because these protocols use + multicast. + +Linux sysctl settings and kernel modules +```````````````````````````````````````` + +There are several kernel parameters that impact overall operation of FRR when +using Linux as a router. Generally these parameters should be set in a +sysctl related configuration file, e.g., :file:`/etc/sysctl.conf` on +Ubuntu based systems and a new file +:file:`/etc/sysctl.d/90-routing-sysctl.conf` on Centos based systems. +Additional kernel modules are also needed to support MPLS forwarding. + +:makevar:`IPv4 and IPv6 forwarding` + The following are set to enable IP forwarding in the kernel: + + .. code-block:: shell + + net.ipv4.conf.all.forwarding=1 + net.ipv6.conf.all.forwarding=1 + +:makevar:`MPLS forwarding` + Basic MPLS support was introduced in the kernel in version 4.1 and + additional capability was introduced in 4.3 and 4.5. + For some general information on Linux MPLS support, see + https://www.netdevconf.org/1.1/proceedings/slides/prabhu-mpls-tutorial.pdf. + The following modules should be loaded to support MPLS forwarding, + and are generally added to a configuration file such as + :file:`/etc/modules-load.d/modules.conf`: + + .. code-block:: shell + + # Load MPLS Kernel Modules + mpls_router + mpls_iptunnel + + The following is an example to enable MPLS forwarding in the + kernel, typically by editing :file:`/etc/sysctl.conf`: + + .. code-block:: shell + + # Enable MPLS Label processing on all interfaces + net.mpls.conf.eth0.input=1 + net.mpls.conf.eth1.input=1 + net.mpls.conf.eth2.input=1 + net.mpls.platform_labels=100000 + + Make sure to add a line equal to :file:`net.mpls.conf.<if>.input` for + each interface *'<if>'* used with MPLS and to set labels to an + appropriate value. + +:makevar:`VRF forwarding` + General information on Linux VRF support can be found in + https://www.kernel.org/doc/Documentation/networking/vrf.txt. + + Kernel support for VRFs was introduced in 4.3, but there are known issues + in versions up to 4.15 (for IPv4) and 5.0 (for IPv6). The FRR CI system + doesn't perform VRF tests on older kernel versions, and VRFs may not work + on them. If you experience issues with VRF support, you should upgrade your + kernel version. + + .. seealso:: :ref:`zebra-vrf` + +Building +^^^^^^^^ + +Once you have chosen your configure options, run the configure script and pass +the options you chose: + +.. code-block:: shell + + ./configure \ + --prefix=/usr \ + --localstatedir=/var/run/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-pimd \ + --enable-watchfrr \ + ... + +After configuring the software, you are ready to build and install it in your +system. + +.. code-block:: shell + + make && sudo make install + +If everything finishes successfully, FRR should be installed. You should now +skip to the section on :ref:`basic-setup`. diff --git a/doc/user/ipv6.rst b/doc/user/ipv6.rst new file mode 100644 index 0000000..4f01061 --- /dev/null +++ b/doc/user/ipv6.rst @@ -0,0 +1,223 @@ +.. _ipv6-support: + +************ +IPv6 Support +************ + +FRR fully supports IPv6 routing. As described so far, FRR supports RIPng, +OSPFv3, and BGP-4+. You can give IPv6 addresses to an interface and configure +static IPv6 routing information. FRR IPv6 also provides automatic address +configuration via a feature called ``address auto configuration``. To do it, +the router must send router advertisement messages to the all nodes that exist +on the network. + +Previous versions of FRR could be built without IPv6 support. This is +no longer possible. + +Router Advertisement +==================== + +.. clicmd:: show ipv6 nd ra-interfaces [vrf <VRFNAME|all>] + + Show configured route advertisement interfaces. VRF subcommand only + applicable for netns-based vrfs. + +.. clicmd:: ipv6 nd suppress-ra + + Don't send router advertisement messages. The ``no`` form of this command + enables sending RA messages. + +.. clicmd:: ipv6 nd prefix ipv6prefix [valid-lifetime] [preferred-lifetime] [off-link] [no-autoconfig] [router-address] + + Configuring the IPv6 prefix to include in router advertisements. Several prefix + specific optional parameters and flags may follow: + + - ``valid-lifetime``: the length of time in seconds during what the prefix is + valid for the purpose of on-link determination. Value ``infinite`` represents + infinity (i.e. a value of all one bits (``0xffffffff``)). + Range: ``(0-4294967295)`` Default: ``2592000`` + + - ``preferred-lifetime``: the length of time in seconds during what addresses + generated from the prefix remain preferred. Value ``infinite`` represents + infinity. + Range: ``(0-4294967295)`` Default: ``604800`` + + - ``off-link``: indicates that advertisement makes no statement about on-link or + off-link properties of the prefix. + Default: not set, i.e. this prefix can be used for on-link determination. + + - ``no-autoconfig``: indicates to hosts on the local link that the specified prefix + cannot be used for IPv6 autoconfiguration. + + Default: not set, i.e. prefix can be used for autoconfiguration. + + - ``router-address``: indicates to hosts on the local link that the specified + prefix contains a complete IP address by setting R flag. + + Default: not set, i.e. hosts do not assume a complete IP address is placed. + +.. clicmd:: ipv6 nd ra-interval [(1-1800)] + + The maximum time allowed between sending unsolicited multicast router + advertisements from the interface, in seconds. + Default: ``600`` + +.. clicmd:: ipv6 nd ra-interval [msec (70-1800000)] + + The maximum time allowed between sending unsolicited multicast router + advertisements from the interface, in milliseconds. + Default: ``600000`` + +.. clicmd:: ipv6 nd ra-fast-retrans + + RFC4861 states that consecutive RA packets should be sent no more + frequently than three seconds apart. FRR by default allows faster + transmissions of RA packets in order to speed convergence and + neighbor establishment, particularly for unnumbered peering. By + turning off ipv6 nd ra-fast-retrans, the implementation is + compliant with the RFC at the cost of slower convergence + and neighbor establishment. + Default: enabled + +.. clicmd:: ipv6 nd ra-retrans-interval [(0-4294967295)] + + The value to be placed in the retrans timer field of router advertisements + sent from the interface, in msec. Indicates the interval between router + advertisement retransmissions. Setting the value to zero indicates that + the value is unspecified by this router. Must be between zero or 4294967295 + msec. + Default: ``0`` + +.. clicmd:: ipv6 nd ra-hop-limit [(0-255)] + + The value to be placed in the hop count field of router advertisements sent + from the interface, in hops. Indicates the maximum diameter of the network. + Setting the value to zero indicates that the value is unspecified by this + router. Must be between zero or 255 hops. + Default: ``64`` + +.. clicmd:: ipv6 nd ra-lifetime [(0-9000)] + + The value to be placed in the Router Lifetime field of router advertisements + sent from the interface, in seconds. Indicates the usefulness of the router + as a default router on this interface. Setting the value to zero indicates + that the router should not be considered a default router on this interface. + Must be either zero or between value specified with ``ipv6 nd ra-interval`` + (or default) and 9000 seconds. + Default: ``1800`` + +.. clicmd:: ipv6 nd reachable-time [(1-3600000)] + + The value to be placed in the Reachable Time field in the Router + Advertisement messages sent by the router, in milliseconds. The configured + time enables the router to detect unavailable neighbors. The value zero + means unspecified (by this router). + Default: ``0`` + +.. clicmd:: ipv6 nd managed-config-flag + + Set/unset flag in IPv6 router advertisements which indicates to hosts that + they should use managed (stateful) protocol for addresses autoconfiguration + in addition to any addresses autoconfigured using stateless address + autoconfiguration. + Default: not set + +.. clicmd:: ipv6 nd other-config-flag + + Set/unset flag in IPv6 router advertisements which indicates to hosts that + they should use administered (stateful) protocol to obtain autoconfiguration + information other than addresses. + Default: not set + +.. clicmd:: ipv6 nd home-agent-config-flag + + Set/unset flag in IPv6 router advertisements which indicates to hosts that + the router acts as a Home Agent and includes a Home Agent Option. + Default: not set + + +.. clicmd:: ipv6 nd home-agent-preference [(0-65535)] + + The value to be placed in Home Agent Option, when Home Agent config flag is + set, which indicates to hosts Home Agent preference. The default value of 0 + stands for the lowest preference possible. + Default: ``0`` + +.. clicmd:: ipv6 nd home-agent-lifetime [(0-65520)] + + The value to be placed in Home Agent Option, when Home Agent config flag is set, + which indicates to hosts Home Agent Lifetime. The default value of 0 means to + place the current Router Lifetime value. + + Default: ``0`` + +.. clicmd:: ipv6 nd adv-interval-option + + Include an Advertisement Interval option which indicates to hosts the maximum time, + in milliseconds, between successive unsolicited Router Advertisements. + Default: not set + +.. clicmd:: ipv6 nd router-preference [(high|medium|low)] + + Set default router preference in IPv6 router advertisements per RFC4191. + Default: medium + +.. clicmd:: ipv6 nd mtu [(1-65535)] + + Include an MTU (type 5) option in each RA packet to assist the attached + hosts in proper interface configuration. The announced value is not verified + to be consistent with router interface MTU. + + Default: don't advertise any MTU option. + +.. clicmd:: ipv6 nd rdnss ipv6address [lifetime] + + Recursive DNS server address to advertise using the RDNSS (type 25) option + described in RFC8106. Can be specified more than once to advertise multiple + addresses. Note that hosts may choose to limit the number of RDNSS addresses + to track. + + Optional parameter: + + - ``lifetime``: the maximum time in seconds over which the specified address + may be used for domain name resolution. Value ``infinite`` represents + infinity (i.e. a value of all one bits (``0xffffffff``)). A value of 0 + indicates that the address must no longer be used. + Range: ``(0-4294967295)`` Default: ``3 * ra-interval`` + + Default: do not emit RDNSS option + +.. clicmd:: ipv6 nd dnssl domain-name-suffix [lifetime] + + Advertise DNS search list using the DNSSL (type 31) option described in + RFC8106. Specify more than once to advertise multiple domain name suffixes. + Host implementations may limit the number of honored search list entries. + + Optional parameter: + + - ``lifetime``: the maximum time in seconds over which the specified domain + suffix may be used in the course of name resolution. Value ``infinite`` + represents infinity (i.e. a value of all one bits (``0xffffffff``)). A + value of 0 indicates that the name suffix must no longer be used. + Range: ``(0-4294967295)`` Default: ``3 * ra-interval`` + + Default: do not emit DNSSL option + +Router Advertisement Configuration Example +========================================== +A small example: + +.. code-block:: frr + + interface eth0 + no ipv6 nd suppress-ra + ipv6 nd prefix 2001:0DB8:5009::/64 + + +.. seealso:: + + - :rfc:`2462` (IPv6 Stateless Address Autoconfiguration) + - :rfc:`4861` (Neighbor Discovery for IP Version 6 (IPv6)) + - :rfc:`6275` (Mobility Support in IPv6) + - :rfc:`4191` (Default Router Preferences and More-Specific Routes) + - :rfc:`8106` (IPv6 Router Advertisement Options for DNS Configuration) diff --git a/doc/user/isisd.rst b/doc/user/isisd.rst new file mode 100644 index 0000000..63c9213 --- /dev/null +++ b/doc/user/isisd.rst @@ -0,0 +1,857 @@ +.. _isis: + +**** +ISIS +**** + +:abbr:`ISIS (Intermediate System to Intermediate System)` is a routing protocol +which is described in :t:`ISO10589`, :rfc:`1195`, :rfc:`5308`. ISIS is an +:abbr:`IGP (Interior Gateway Protocol)`. Compared with :abbr:`RIP`, +:abbr:`ISIS` can provide scalable network support and faster convergence times +like :abbr:`OSPF`. ISIS is widely used in large networks such as :abbr:`ISP +(Internet Service Provider)` and carrier backbone networks. + +.. _configuring-isisd: + +Configuring isisd +================= + +There are no *isisd* specific options. Common options can be specified +(:ref:`common-invocation-options`) to *isisd*. *isisd* needs to acquire +interface information from *zebra* in order to function. Therefore *zebra* must +be running before invoking *isisd*. Also, if *zebra* is restarted then *isisd* +must be too. + +Like other daemons, *isisd* configuration is done in :abbr:`ISIS` specific +configuration file :file:`isisd.conf`. + +.. _isis-router: + +ISIS router +=========== + +To start the ISIS process you have to specify the ISIS router. As of this +writing, *isisd* does not support multiple ISIS processes. + +.. clicmd:: router isis WORD [vrf NAME] + + Enable or disable the ISIS process by specifying the ISIS domain with + 'WORD'. *isisd* does not yet support multiple ISIS processes but you must + specify the name of ISIS process. The ISIS process name 'WORD' is then used + for interface (see command :clicmd:`ip router isis WORD`). + +.. clicmd:: net XX.XXXX. ... .XXX.XX + + Set/Unset network entity title (NET) provided in ISO format. + +.. clicmd:: hostname dynamic + + Enable support for dynamic hostname. + +.. clicmd:: area-password [clear | md5] <password> + +.. clicmd:: domain-password [clear | md5] <password> + + Configure the authentication password for an area, respectively a domain, as + clear text or md5 one. + +.. clicmd:: attached-bit [receive ignore | send] + + Set attached bit for inter-area traffic: + + - receive + If LSP received with attached bit set, create default route to neighbor + - send + If L1|L2 router, set attached bit in LSP sent to L1 router + +.. clicmd:: log-adjacency-changes + + Log changes in adjacency state. + +.. clicmd:: log-pdu-drops + + Log any dropped PDUs. + +.. clicmd:: metric-style [narrow | transition | wide] + + Set old-style (ISO 10589) or new-style packet formats: + + - narrow + Use old style of TLVs with narrow metric + - transition + Send and accept both styles of TLVs during transition + - wide + Use new style of TLVs to carry wider metric. FRR uses this as a default value + +.. clicmd:: advertise-high-metrics + + Advertise high metric value on all interfaces to gracefully shift traffic off the router. Reference: :rfc:`3277` + + For narrow metrics, the high metric value is 63; for wide metrics, 16777215; for transition metrics, 62. + +.. clicmd:: set-overload-bit + + Set overload bit to avoid any transit traffic. + +.. clicmd:: set-overload-bit on-startup (0-86400) + + Set overload bit on startup for the specified duration, in seconds. Reference: :rfc:`3277` + +.. clicmd:: purge-originator + + Enable or disable :rfc:`6232` purge originator identification. + +.. clicmd:: lsp-mtu (128-4352) + + Configure the maximum size of generated LSPs, in bytes. + +.. clicmd:: advertise-passive-only + + Advertise prefixes of passive interfaces only. + +.. _isis-timer: + +ISIS Timer +========== + +.. clicmd:: lsp-gen-interval [level-1 | level-2] (1-120) + + Set minimum interval in seconds between regenerating same LSP, + globally, for an area (level-1) or a domain (level-2). + +.. clicmd:: lsp-refresh-interval [level-1 | level-2] (1-65235) + + Set LSP refresh interval in seconds, globally, for an area (level-1) or a + domain (level-2). + +.. clicmd:: max-lsp-lifetime [level-1 | level-2] (360-65535) + + Set LSP maximum LSP lifetime in seconds, globally, for an area (level-1) or + a domain (level-2). + +.. clicmd:: spf-interval [level-1 | level-2] (1-120) + + Set minimum interval between consecutive SPF calculations in seconds. + +.. _isis-fast-reroute: + +ISIS Fast-Reroute +================= + +Unless stated otherwise, commands in this section apply to all LFA +flavors (local LFA, Remote LFA and TI-LFA). + +.. clicmd:: spf prefix-priority [critical | high | medium] WORD + + Assign a priority to the prefixes that match the specified access-list. + + By default loopback prefixes have medium priority and non-loopback prefixes + have low priority. + +.. clicmd:: fast-reroute priority-limit [critical | high | medium] [level-1 | level-2] + + Limit LFA backup computation up to the specified prefix priority. + +.. clicmd:: fast-reroute lfa tiebreaker [downstream | lowest-backup-metric | node-protecting] index (1-255) [level-1 | level-2] + + Configure a tie-breaker for multiple local LFA backups. Lower indexes are + processed first. + +.. clicmd:: fast-reroute load-sharing disable [level-1 | level-2] + + Disable load sharing across multiple LFA backups. + +.. clicmd:: fast-reroute remote-lfa prefix-list [WORD] [level-1 | level-2] + + Configure a prefix-list to select eligible PQ nodes for remote LFA + backups (valid for all protected interfaces). + +.. clicmd:: redistribute <ipv4 | ipv6> table (1-65535) <level-1 | level-2> [metric (0-16777215)|route-map WORD] + + Redistribute routes from a given routing table into the given ISIS + level database. + +.. _isis-region: + +ISIS region +=========== + +.. clicmd:: is-type [level-1 | level-1-2 | level-2-only] + + Define the ISIS router behavior: + + - level-1 + Act as a station router only + - level-1-2 + Act as both a station router and an area router + - level-2-only + Act as an area router only + +.. _isis-interface: + +ISIS interface +============== + +.. _ip-router-isis-word: + +.. clicmd:: <ip|ipv6> router isis WORD + + Activate ISIS adjacency on this interface. Note that the name of ISIS + instance must be the same as the one used to configure the ISIS process (see + command :clicmd:`router isis WORD`). To enable IPv4, issue ``ip router isis + WORD``; to enable IPv6, issue ``ipv6 router isis WORD``. + +.. clicmd:: isis circuit-type [level-1 | level-1-2 | level-2] + + Configure circuit type for interface: + + - level-1 + Level-1 only adjacencies are formed + - level-1-2 + Level-1-2 adjacencies are formed + - level-2-only + Level-2 only adjacencies are formed + +.. clicmd:: isis csnp-interval (1-600) [level-1 | level-2] + + Set CSNP interval in seconds globally, for an area (level-1) or a domain + (level-2). + +.. clicmd:: isis hello padding + + Add padding to IS-IS hello packets. + +.. clicmd:: isis hello padding during-adjacency-formation + + Add padding to IS-IS hello packets during adjacency formation only. + +.. clicmd:: isis hello-interval (1-600) [level-1 | level-2] + + Set Hello interval in seconds globally, for an area (level-1) or a domain + (level-2). + +.. clicmd:: isis hello-multiplier (2-100) [level-1 | level-2] + + Set multiplier for Hello holding time globally, for an area (level-1) or a + domain (level-2). + +.. clicmd:: isis metric [(0-255) | (0-16777215)] [level-1 | level-2] + + Set default metric value globally, for an area (level-1) or a domain + (level-2). Max value depend if metric support narrow or wide value (see + command :clicmd:`metric-style [narrow | transition | wide]`). + +.. clicmd:: isis network point-to-point + + Set network type to 'Point-to-Point' (broadcast by default). + +.. clicmd:: isis passive + + Configure the passive mode for this interface. + +.. clicmd:: isis password [clear | md5] <password> + + Configure the authentication password (clear or encoded text) for the + interface. + +.. clicmd:: isis priority (0-127) [level-1 | level-2] + + Set priority for Designated Router election, globally, for the area + (level-1) or the domain (level-2). + +.. clicmd:: isis psnp-interval (1-120) [level-1 | level-2] + + Set PSNP interval in seconds globally, for an area (level-1) or a domain + (level-2). + +.. clicmd:: isis three-way-handshake + + Enable or disable :rfc:`5303` Three-Way Handshake for P2P adjacencies. + Three-Way Handshake is enabled by default. + +.. clicmd:: isis fast-reroute lfa [level-1 | level-2] + + Enable per-prefix local LFA fast reroute link protection. + +.. clicmd:: isis fast-reroute lfa [level-1 | level-2] exclude interface IFNAME + + Exclude an interface from the local LFA backup nexthop computation. + +.. clicmd:: isis fast-reroute remote-lfa tunnel mpls-ldp [level-1 | level-2] + + Enable per-prefix Remote LFA fast reroute link protection. Note that other + routers in the network need to be configured to accept LDP targeted hello + messages in order for RLFA to work. + +.. clicmd:: isis fast-reroute remote-lfa maximum-metric (1-16777215) [level-1 | level-2] + + Limit Remote LFA PQ node selection within the specified metric. + +.. clicmd:: isis fast-reroute ti-lfa [level-1|level-2] [node-protection [link-fallback]] + + Enable per-prefix TI-LFA fast reroute link or node protection. + When node protection is used, option link-fallback enables the computation and use of + link-protecting LFAs for destinations unprotected by node protection. + +.. _showing-isis-information: + +Showing ISIS information +======================== + +.. clicmd:: show isis [vrf <NAME|all>] summary [json] + + Show summary information about ISIS. + +.. clicmd:: show isis hostname + + Show information about ISIS node. + +.. clicmd:: show isis [vrf <NAME|all>] interface [detail] [IFNAME] [json] + + Show state and configuration of ISIS specified interface, or all interfaces + if no interface is given with or without details. + +.. clicmd:: show isis [vrf <NAME|all>] neighbor [detail] [SYSTEMID] [json] + + Show state and information of ISIS specified neighbor, or all neighbors if + no system id is given with or without details. + +.. clicmd:: show isis [vrf <NAME|all>] database [detail] [LSPID] [json] + + Show the ISIS database globally, for a specific LSP id without or with + details. + +.. clicmd:: show isis topology [level-1|level-2] [algorithm (128-255)] + + Show topology IS-IS paths to Intermediate Systems, globally, in area + (level-1) or domain (level-2). + +.. clicmd:: show isis route [level-1|level-2] [prefix-sid|backup] [algorithm (128-255)] + + Show the ISIS routing table, as determined by the most recent SPF + calculation. + +.. clicmd:: show isis fast-reroute summary [level-1|level-2] + + Show information about the number of prefixes having LFA protection, + and network-wide LFA coverage. + + +.. _isis-traffic-engineering: + +Traffic Engineering +=================== + +.. note:: + + IS-IS-TE supports RFC 5305 (base TE), RFC 6119 (IPv6) and RFC 7810 / 8570 + (Extended Metric) with or without Multi-Topology. All Traffic Engineering + information are stored in a database formally named TED. However, best + acccuracy is provided without Multi-Topology due to inconsistency of Traffic + Engineering Advertisement of 3rd party commercial routers when MT is enabled. + At this time, FRR offers partial support for some of the routing protocol + extensions that can be used with MPLS-TE. FRR does not currently support a + complete RSVP-TE solution. + +.. clicmd:: mpls-te on + + Enable Traffic Engineering LSP flooding. + +.. clicmd:: mpls-te router-address <A.B.C.D> + + Configure stable IP address for MPLS-TE. + +.. clicmd:: mpls-te router-address ipv6 <X:X::X:X> + + Configure stable IPv6 address for MPLS-TE. + +.. clicmd:: mpls-te export + + Export Traffic Engineering DataBase to other daemons through the ZAPI + Opaque Link State messages. + +.. clicmd:: show isis mpls-te interface + +.. clicmd:: show isis mpls-te interface INTERFACE + + Show MPLS Traffic Engineering parameters for all or specified interface. + +.. clicmd:: show isis mpls-te router + + Show Traffic Engineering router parameters. + +.. clicmd:: show isis [vrf <NAME|all>] mpls-te database [detail|json] + +.. clicmd:: show isis [vrf <NAME|all>] mpls-te database vertex [WORD] [detail|json] + +.. clicmd:: show isis [vrf <NAME|all>] mpls-te database edge [A.B.C.D|X:X::X:X] [detail|json] + +.. clicmd:: show isis [vrf <NAME|all>] mpls-te database subnet [A.B.C.D/M|X:X::X:X/M] [detail|json] + + Show Traffic Engineering Database + +.. seealso:: + + :ref:`ospf-traffic-engineering` + +.. _isis-segment-routing: + +Segment Routing +=============== + +This is an EXPERIMENTAL support of Segment Routing as per RFC8667 +for MPLS dataplane. It supports IPv4, IPv6 and ECMP and has been +tested against Cisco & Juniper routers. + +Known limitations: + - No support for level redistribution (L1 to L2 or L2 to L1) + - No support for binding SID + - No support for SRMS + - No support for SRLB + - Only one SRGB and default SPF Algorithm is supported + +.. clicmd:: segment-routing on + + Enable Segment Routing. + +.. clicmd:: segment-routing global-block (16-1048575) (16-1048575) [local-block (16-1048575) (16-1048575)] + + Set the Segment Routing Global Block i.e. the label range used by MPLS + to store label in the MPLS FIB for Prefix SID. Note that the block size + may not exceed 65535. Optionally sets also the Segment Routing Local Block. + The negative command always unsets both. + +.. clicmd:: segment-routing node-msd (1-16) + + Set the Maximum Stack Depth supported by the router. The value depend of the + MPLS dataplane. E.g. for Linux kernel, since version 4.13 the maximum value + is 32. + +.. clicmd:: segment-routing prefix <A.B.C.D/M|X:X::X:X/M> [algorithm (128-255)] <absolute (16-1048575)|index (0-65535) [no-php-flag|explicit-null] [n-flag-clear] + + prefix. The 'no-php-flag' means NO Penultimate Hop Popping that allows SR + node to request to its neighbor to not pop the label. The 'explicit-null' + flag allows SR node to request to its neighbor to send IP packet with the + EXPLICIT-NULL label. The 'n-flag-clear' option can be used to explicitly + clear the Node flag that is set by default for Prefix-SIDs associated to + loopback addresses. This option is necessary to configure Anycast-SIDs. + +.. clicmd:: show isis segment-routing node [algorithm (128-255)] + + Show detailed information about all learned Segment Routing Nodes. + +.. _isis-flex-algo: + +Flex-Algos (Flex-Algo) +====================== + +*isisd* supports some features of +`RFC 9350 <https://tools.ietf.org/html/rfc9350>`_ on an MPLS Segment-Routing +dataplane. The compatibility has been tested against Cisco. + +IS-IS uses by default the `Shortest-Path-First` algorithm that basically +calculates paths based on the shortest total metric to the destinations. +Flex-Algo allows new algorithms to run in parallel to compute paths in different +manners, based on metrics (IGP metric or a new type of metrics such as Traffic +Engineering (TE) metric and minimum delay...) and constraints. New metric types +are not yet implemented but constraints are already operational. Constraints can +restrict paths to links with specific affinities or avoid links with specific +affinities. Combinations of these are also possible. + +The administrator can configure up to 128 Flex-Algos in an IS-IS area. +To do so, it defines a set of Flex-Algo Definitions (FAD) which +have the following characteristics: + +- a numeric identifier (ID) between 128 and 255 inclusive +- a set of constraints (basically, include or exclude a certain given set of + links, designated by a admin-group) +- the calculation type (only the `Shortest-Path-First` is currently supported) +- the metric type (only the IGP inherited metric type is currently supported) +- some additional flags (not supported for the moment). + +A subset of routers advertises the Flex-Algo Definitions (FAD) to the other +routers within an area. In order to use a common set of FADs, each router runs a +FAD election process for each locally configured algorithm, using the following +rules: + +- If a locally configured FAD is not advertised to the area, the router does not + participate in the particular flex algorithm. +- If a given flex algorithm is running, the participation in this particular + flex algorithm stops when its advertisements are over. +- A router includes its own FAD in the election process if and only if it is + advertised to the other routers. +- If only one router advertises the FAD, the FAD is elected. +- If several FADs are advertised with different priorities, the one with the + highest priority value is selected. +- If there are multiple advertisements of the FAD with the same highest + priority, the FAD of the router with the highest IS-IS system-ID is + selected. + +Routers only use the specifications of the elected FAD regardless of the locally +configured definitions. If a router does not support one of the FAD +characteristics, it stops participating in the Flex-Algo. + +For each running Flex-Algo, the Segment-Routing SIDs must be +configured with values unique to the algorithm. It allows routers to identify +which flex algorithm they must use for a given packet. + +The following commands configure Flex-Algo at the 'router isis' configuration +level. Segment-Routing prefixes must be configured for the Flex-Algo. + +.. clicmd:: flexible-algorithm (128-255) + + Add a Flex-Algo Definition (FAD) and enter the FAD configuration + level. The algorithm ID value is in the range of 128 to 255 inclusive. + +.. clicmd:: no flexible-algorithm (128-255) + + Unconfigure a Flex-Algo Definition. + +.. clicmd:: affinity-map NAME bit-position (0-255) + + Add the specified 'affinity-map'. Affinity-map definitions are used in + FADs and in interfaces admin-group definition. + + Affinity-maps format in advertisement TLVs use the extended admin-group + format defined in the RFC7308 section 2.2. The extended admin-group uses a + 256 bits field. If an affinity-map is set, the bit at the extended + admin-group 'bit-position' is set 1, else it is set to 0. + +The following commands configure Flex-Algo at the 'router isis' and +'flexible-algorithm (128-255)' configuration level. + +.. clicmd:: advertise-definition + + Advertise the current FAD to other IS-IS routers by using specific IS-IS + TLVs. By default, the definition is is not shared with other routers. + + A router can advertise a FAD without participating in the Flex-Algo. + +.. clicmd:: priority (0-255) + + Set the specified 'priority' in the current FAD advertisements . + +.. clicmd:: metric-type [igp|te|delay] + + Set the 'metric-type' for the current FAD. 'igp' is + the default value and refers to the classic 'Shortest-Path-First' algorithm. + If the 'te' or the 'delay' metric is selected, the value is advertised but + the flex algorithm is disabled locally because these types are not currently + supported. + +.. clicmd:: no metric-type + + Reset the 'metric-type' to the default 'igp' metric. + +.. clicmd:: affinity exclude-any NAME + + Add the specified affinity to the list of exclude-any affinities. The + Flex-Algo will compute paths that exclude the segments with any of + the specified affinities. + +.. clicmd:: no affinity exclude-any NAME + + Remove the specified affinity to the list of exclude-any affinities. + +.. clicmd:: affinity include-all NAME + + Add the specified affinity to the list of include-all affinities. The + Flex-Algo will compute paths that include the segments with all + the specified affinities. + +.. clicmd:: no affinity include-all NAME + + Remove the specified affinity to the list of include-all affinities. + +.. clicmd:: affinity include-any NAME + + Add the specified affinity to the list of include-any affinities. The + Flex-Algo will compute paths that include the segments with any of + the specified affinities. + +.. clicmd:: no affinity include-any NAME + + Remove the specified affinity to the list of include-any affinities. + +The following commands configure Flex-Algo at the 'interface' configuration +level. + +.. clicmd:: isis affinity flex-algo NAME + + Add the specified affinity to the interface. + +.. clicmd:: no isis affinity flex-algo NAME + + Remove the specified affinity from the interface. + +The following command show Flex-Algo information: + +.. clicmd:: show isis flex-algo [(128-255)] + + Show information about the elected FADs + +'show isis route', 'show isis topology' and 'show isis segment-routing node' +includes an 'algorithm (128-255)' optional argument. See +:ref:`showing-isis-information` and :ref:`isis-segment-routing`. + +.. _isis-srv6: + +Segment Routing over IPv6 (SRv6) +================================ + +This feature enables extensions in IS-IS to support Segment Routing over IPv6 +data plane (SRv6) as per RFC 9352. + +.. clicmd:: segment-routing srv6 + + Enable Segment Routing over IPv6 data plane (SRv6). + +.. clicmd:: locator NAME + + Specify the SRv6 locator to use for SRv6. The locator must be configured in + Zebra. Once the locator is configured, IS-IS automatically allocates prefix + SID and adjacency SIDs, creates local SID entries in the data plane, and + advertises them in the IGP domain. + +.. clicmd:: interface NAME + + Specify the dummy interface used to install SRv6 SIDs in the Linux data plane. + The interface must be created manually. By default, the interface is 'sr0'. + The interface can be created using the iproute2 utility: + + .. code-block:: bash + + ip link add sr0 type dummy + ip link set sr0 up + +.. clicmd:: show isis segment-routing srv6 node + + Show detailed information about all learned SRv6 Nodes. + +Debugging ISIS +============== + +.. clicmd:: debug isis adj-packets + + IS-IS Adjacency related packets. + +.. clicmd:: debug isis checksum-errors + + IS-IS LSP checksum errors. + +.. clicmd:: debug isis events + + IS-IS Events. + +.. clicmd:: debug isis local-updates + + IS-IS local update packets. + +.. clicmd:: debug isis packet-dump + + IS-IS packet dump. + +.. clicmd:: debug isis protocol-errors + + IS-IS LSP protocol errors. + +.. clicmd:: debug isis route-events + + IS-IS Route related events. + +.. clicmd:: debug isis snp-packets + + IS-IS CSNP/PSNP packets. + +.. clicmd:: debug isis spf-events +.. clicmd:: debug isis spf-statistics +.. clicmd:: debug isis spf-triggers + + IS-IS Shortest Path First Events, Timing and Statistic Data and triggering + events. + +.. clicmd:: debug isis update-packets + + + Update related packets. + +.. clicmd:: debug isis te-events + + IS-IS Traffic Engineering events + +.. clicmd:: debug isis sr-events + + + IS-IS Segment Routing events. + +.. clicmd:: debug isis lfa + + + IS-IS LFA events. + +.. clicmd:: show debugging isis + + Print which ISIS debug level is activate. + +.. _isis-config-examples: + +ISIS Configuration Examples +=========================== + +A simple example, with MD5 authentication enabled: + +.. code-block:: frr + + ! + interface eth0 + ip router isis FOO + isis network point-to-point + isis circuit-type level-2-only + ! + router isis FOO + net 47.0023.0000.0000.0000.0000.0000.0000.1900.0004.00 + metric-style wide + is-type level-2-only + + +A Traffic Engineering configuration, with Inter-ASv2 support. + +First, the :file:`zebra.conf` part: + +.. code-block:: frr + + hostname HOSTNAME + password PASSWORD + log file /var/log/zebra.log + ! + interface eth0 + ip address 10.2.2.2/24 + link-params + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + admin-grp 0xab + ! + interface eth1 + ip address 10.1.1.1/24 + link-params + enable + metric 100 + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + neighbor 10.1.1.2 as 65000 + + +Then the :file:`isisd.conf` itself: + +.. code-block:: frr + + hostname HOSTNAME + password PASSWORD + log file /var/log/isisd.log + ! + ! + interface eth0 + ip router isis FOO + ! + interface eth1 + ip router isis FOO + ! + ! + router isis FOO + isis net 47.0023.0000.0000.0000.0000.0000.0000.1900.0004.00 + mpls-te on + mpls-te router-address 10.1.1.1 + ! + line vty + +A Segment Routing configuration, with IPv4, IPv6, SRGB and MSD configuration. + +.. code-block:: frr + + hostname HOSTNAME + password PASSWORD + log file /var/log/isisd.log + ! + ! + interface eth0 + ip router isis SR + isis network point-to-point + ! + interface eth1 + ip router isis SR + ! + ! + router isis SR + net 49.0000.0000.0000.0001.00 + is-type level-1 + topology ipv6-unicast + lsp-gen-interval 2 + segment-routing on + segment-routing node-msd 8 + segment-routing prefix 10.1.1.1/32 index 100 explicit-null + segment-routing prefix 2001:db8:1000::1/128 index 101 explicit-null + ! + +An SRv6 configuration: + +.. code-block:: frr + + hostname HOSTNAME + password PASSWORD + log file /var/log/isisd.log + ! + ! + interface eth0 + ipv6 router isis FOO + ip router isis FOO + isis hello-interval 5 + ! + interface eth1 + ip router isis FOO + ! + ! + router isis FOO + net 49.0001.1111.1111.1111.00 + is-type level-2-only + metric-style wide + segment-routing srv6 + locator loc1 + ! + line vty + + +.. _isis-vrf-config-examples: + +ISIS Vrf Configuration Examples +=============================== + +A simple vrf example: + +.. code-block:: frr + + ! + interface eth0 vrf RED + ip router isis FOO vrf RED + isis network point-to-point + isis circuit-type level-2-only + ! + router isis FOO vrf RED + net 47.0023.0000.0000.0000.0000.0000.0000.1900.0004.00 + metric-style wide + is-type level-2-only diff --git a/doc/user/kernel.rst b/doc/user/kernel.rst new file mode 100644 index 0000000..210ede7 --- /dev/null +++ b/doc/user/kernel.rst @@ -0,0 +1,36 @@ +.. _kernel-interface: + +**************** +Kernel Interface +**************** + +There are several different methods for reading kernel routing table +information, updating kernel routing tables, and for looking up interfaces. +FRR relies heavily on the Netlink (``man 7 netlink``) interface to +communicate with the Kernel. However, other interfaces are still used +in some parts of the code. + +- ioctl + This method is a very traditional way for reading or writing kernel + information. `ioctl` can be used for looking up interfaces and for + modifying interface addresses, flags, mtu settings and other types of + information. Also, `ioctl` can insert and delete kernel routing table + entries. It will soon be available on almost any platform which zebra + supports, but it is a little bit ugly thus far, so if a better method is + supported by the kernel, zebra will use that. + +- sysctl + This is a program that can lookup kernel information using MIB (Management + Information Base) syntax. Normally, it only provides a way of getting + information from the kernel. So one would usually want to change kernel + information using another method such as `ioctl`. + +- proc filesystem + This is a special filesystem mount that provides an easy way of getting + kernel information. + +- routing socket / Netlink + Netlink first appeard in Linux kernel 2.0. It makes asynchronous + communication between the kernel and FRR possible, similar to a routing + socket on BSD systems. Netlink communication is done by reading/writing + over Netlink socket. diff --git a/doc/user/ldpd.rst b/doc/user/ldpd.rst new file mode 100644 index 0000000..682443a --- /dev/null +++ b/doc/user/ldpd.rst @@ -0,0 +1,379 @@ +.. _ldp: + +*** +LDP +*** + +The *ldpd* daemon is a standardised protocol that permits exchanging MPLS label +information between MPLS devices. The LDP protocol creates peering between +devices, so as to exchange that label information. This information is stored in +MPLS table of *zebra*, and it injects that MPLS information in the underlying +system (Linux kernel or OpenBSD system for instance). +*ldpd* provides necessary options to create a Layer 2 VPN across MPLS network. +For instance, it is possible to interconnect several sites that share the same +broadcast domain. + +FRR implements LDP as described in :rfc:`5036`; other LDP standard are the +following ones: :rfc:`6720`, :rfc:`6667`, :rfc:`5919`, :rfc:`5561`, :rfc:`7552`, +:rfc:`4447`. +Because MPLS is already available, FRR also supports :rfc:`3031`. + +Running Ldpd +============ + +The *ldpd* daemon can be invoked with any of the common +options (:ref:`common-invocation-options`). + +.. option:: --ctl_socket + + This option allows you to override the path to the ldpd.sock file + used to control this daemon. If specified this option overrides + the -N option path addition. + +The *zebra* daemon must be running before *ldpd* is invoked. + +Configuration of *ldpd* is done in its configuration file +:file:`ldpd.conf`. + + +.. _understanding-ldp: + +Understanding LDP principles +============================ + +Let's first introduce some definitions that permit understand better the LDP +protocol: + +- `LSR` : Labeled Switch Router. Networking devices handling labels used to + forward traffic between and through them. + +- `LER` : Labeled Edge Router. A Labeled edge router is located at the edge of + an MPLS network, generally between an IP network and an MPLS network. + + +``LDP`` aims at sharing label information across devices. It tries to establish +peering with remote LDP capable devices, first by discovering using UDP port 646 +, then by peering using TCP port 646. Once the TCP session is established, the +label information is shared, through label advertisements. + +There are different methods to send label advertisement modes. The +implementation actually supports the following : Liberal Label Retention + +Downstream Unsolicited + Independent Control. +The other advertising modes are depicted below, and compared with the current +implementation. + +- Liberal label retention versus conservative mode + In liberal mode, every label sent by every LSR is stored in the MPLS table. + In conservative mode, only the label that was sent by the best next hop + (determined by the IGP metric) for that particular FEC is stored in the MPLS + table. + +- Independent LSP Control versus ordered LSP Control + MPLS has two ways of binding labels to FEC’s; either through ordered LSP + control, or independent LSP control. + Ordered LSP control only binds a label to a FEC if it is the egress LSR, or + the router received a label binding for a FEC from the next hop router. In + this mode, an MPLS router will create a label binding for each FEC and + distribute it to its neighbors so long as he has a entry in the RIB for the + destination. + In the other mode, label bindings are made without any dependencies on another + router advertising a label for a particular FEC. Each router makes it own + independent decision to create a label for each FEC. + By default IOS uses Independent LSP Control, while Juniper implements the + Ordered Control. Both modes are interoperable, the difference is that Ordered + Control prevent blackholing during the LDP convergence process, at cost of + slowing down the convergence itself + +- unsolicited downstream versus downstream on demand + Downstream on demand label distribution is where an LSR must explicitly + request that a label be sent from its downstream router for a particular FEC. + Unsolicited label distribution is where a label is sent from the downstream + router without the original router requesting it. + +.. _configuring-ldpd: + +.. _ldp-configuration: + +LDP Configuration +=================== + +.. clicmd:: mpls ldp + + Enable or disable LDP daemon + +.. clicmd:: router-id A.B.C.D + + The following command located under MPLS router node configures the MPLS + router-id of the local device. + +.. clicmd:: ordered-control + + Configure LDP Ordered Label Distribution Control. + +.. clicmd:: address-family [ipv4 | ipv6] + + Configure LDP for IPv4 or IPv6 address-family. Located under MPLS route node, + this subnode permits configuring the LDP neighbors. + +.. clicmd:: interface IFACE + + Located under MPLS address-family node, use this command to enable or disable + LDP discovery per interface. IFACE stands for the interface name where LDP is + enabled. By default it is disabled. Once this command executed, the + address-family interface node is configured. + +.. clicmd:: discovery transport-address A.B.C.D | A:B::C:D + + Located under mpls address-family interface node, use this command to set + the IPv4 or IPv6 transport-address used by the LDP protocol to talk on this + interface. + +.. clicmd:: ttl-security disable + + Located under the LDP address-family node, use this command to disable the + GTSM procedures described in RFC 6720 (for the IPv4 address-family) and + RFC 7552 (for the IPv6 address-family). + + Since GTSM is mandatory for LDPv6, the only effect of disabling GTSM for the + IPv6 address-family is that *ldpd* will not discard packets with a hop limit + below 255. This may be necessary to interoperate with older implementations. + Outgoing packets will still be sent using a hop limit of 255 for maximum + compatibility. + + If GTSM is enabled, multi-hop neighbors should have either GTSM disabled + individually or configured with an appropriate ttl-security hops distance. + +.. clicmd:: neighbor A.B.C.D password PASSWORD + + The following command located under MPLS router node configures the router + of a LDP device. This device, if found, will have to comply with the + configured password. PASSWORD is a clear text password wit its digest sent + through the network. + +.. clicmd:: neighbor A.B.C.D holdtime HOLDTIME + + The following command located under MPLS router node configures the holdtime + value in seconds of the LDP neighbor ID. Configuring it triggers a keepalive + mechanism. That value can be configured between 15 and 65535 seconds. After + this time of non response, the LDP established session will be considered as + set to down. By default, no holdtime is configured for the LDP devices. + +.. clicmd:: neighbor A.B.C.D ttl-security disable + + Located under the MPLS LDP node, use this command to override the global + configuration and enable/disable GTSM for the specified neighbor. + +.. clicmd:: neighbor A.B.C.D ttl-security hops (1-254) + + Located under the MPLS LDP node, use this command to set the maximum number + of hops the specified neighbor may be away. When GTSM is enabled for this + neighbor, incoming packets are required to have a TTL/hop limit of 256 + minus this value, ensuring they have not passed through more than the + expected number of hops. The default value is 1. + +.. clicmd:: discovery hello holdtime HOLDTIME + +.. clicmd:: discovery hello interval INTERVAL + + INTERVAL value ranges from 1 to 65535 seconds. Default value is 5 seconds. + This is the value between each hello timer message sent. + HOLDTIME value ranges from 1 to 65535 seconds. Default value is 15 seconds. + That value is added as a TLV in the LDP messages. + +.. clicmd:: dual-stack transport-connection prefer ipv4 + + When *ldpd* is configured for dual-stack operation, the transport connection + preference is IPv6 by default (as specified by :rfc:`7552`). On such + circumstances, *ldpd* will refuse to establish TCP connections over IPv4. + You can use above command to change the transport connection preference to + IPv4. In this case, it will be possible to distribute label mappings for + IPv6 FECs over TCPv4 connections. + +.. _show-ldp-information: + +Show LDP Information +==================== + +These commands dump various parts of *ldpd*. + +.. clicmd:: show mpls ldp neighbor [A.B.C.D] + + This command dumps the various neighbors discovered. Below example shows that + local machine has an operation neighbor with ID set to 1.1.1.1. + + :: + + west-vm# show mpls ldp neighbor + AF ID State Remote Address Uptime + ipv4 1.1.1.1 OPERATIONAL 1.1.1.1 00:01:37 + west-vm# + +.. clicmd:: show mpls ldp neighbor [A.B.C.D] capabilities + +.. clicmd:: show mpls ldp neighbor [A.B.C.D] detail + + Above commands dump other neighbor information. + +.. clicmd:: show mpls ldp discovery [detail] + +.. clicmd:: show mpls ldp ipv4 discovery [detail] + +.. clicmd:: show mpls ldp ipv6 discovery [detail] + + Above commands dump discovery information. + +.. clicmd:: show mpls ldp ipv4 interface + +.. clicmd:: show mpls ldp ipv6 interface + + Above command dumps the IPv4 or IPv6 interface per where LDP is enabled. + Below output illustrates what is dumped for IPv4. + + :: + + west-vm# show mpls ldp ipv4 interface + AF Interface State Uptime Hello Timers ac + ipv4 eth1 ACTIVE 00:08:35 5/15 0 + ipv4 eth3 ACTIVE 00:08:35 5/15 1 + + +.. clicmd:: show mpls ldp ipv4|ipv6 binding + + Above command dumps the binding obtained through MPLS exchanges with LDP. + + :: + + west-vm# show mpls ldp ipv4 binding + AF Destination Nexthop Local Label Remote Label In Use + ipv4 1.1.1.1/32 1.1.1.1 16 imp-null yes + ipv4 2.2.2.2/32 1.1.1.1 imp-null 16 no + ipv4 10.0.2.0/24 1.1.1.1 imp-null imp-null no + ipv4 10.115.0.0/24 1.1.1.1 imp-null 17 no + ipv4 10.135.0.0/24 1.1.1.1 imp-null imp-null no + ipv4 10.200.0.0/24 1.1.1.1 17 imp-null yes + west-vm# + + +LDP debugging commands +======================== + + +.. clicmd:: debug mpls ldp KIND + + Enable or disable debugging messages of a given kind. ``KIND`` can + be one of: + + - ``discovery`` + - ``errors`` + - ``event`` + - ``labels`` + - ``messages`` + - ``zebra`` + + +Sample configuration +==================== + +Below configuration gives a typical MPLS configuration of a device located in a +MPLS backbone. LDP is enabled on two interfaces and will attempt to peer with +two neighbors with router-id set to either 1.1.1.1 or 3.3.3.3. + +.. code-block:: frr + + mpls ldp + router-id 2.2.2.2 + neighbor 1.1.1.1 password test + neighbor 3.3.3.3 password test + ! + address-family ipv4 + discovery transport-address 2.2.2.2 + ! + interface eth1 + ! + interface eth3 + ! + exit-address-family + ! + + +Deploying LDP across a backbone generally is done in a full mesh configuration +topology. LDP is typically deployed with an IGP like OSPF, that helps discover +the remote IPs. Below example is an OSPF configuration extract that goes with +LDP configuration + +.. code-block:: frr + + router ospf + ospf router-id 2.2.2.2 + network 0.0.0.0/0 area 0 + ! + + +Below output shows the routing entry on the LER side. The OSPF routing entry +(10.200.0.0) is associated with Label entry (17), and shows that MPLS push action +that traffic to that destination will be applied. + +:: + + north-vm# show ip route + Codes: K - kernel route, C - connected, S - static, R - RIP, + O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, + T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, + F - PBR, + > - selected route, * - FIB route + + O>* 1.1.1.1/32 [110/120] via 10.115.0.1, eth2, label 16, 00:00:15 + O>* 2.2.2.2/32 [110/20] via 10.115.0.1, eth2, label implicit-null, 00:00:15 + O 3.3.3.3/32 [110/10] via 0.0.0.0, loopback1 onlink, 00:01:19 + C>* 3.3.3.3/32 is directly connected, loopback1, 00:01:29 + O>* 10.0.2.0/24 [110/11] via 10.115.0.1, eth2, label implicit-null, 00:00:15 + O 10.100.0.0/24 [110/10] is directly connected, eth1, 00:00:32 + C>* 10.100.0.0/24 is directly connected, eth1, 00:00:32 + O 10.115.0.0/24 [110/10] is directly connected, eth2, 00:00:25 + C>* 10.115.0.0/24 is directly connected, eth2, 00:00:32 + O>* 10.135.0.0/24 [110/110] via 10.115.0.1, eth2, label implicit-null, 00:00:15 + O>* 10.200.0.0/24 [110/210] via 10.115.0.1, eth2, label 17, 00:00:15 + north-vm# + + +Additional example demonstrating use of some miscellaneous config options: + +.. code-block:: frr + + interface eth0 + ! + interface eth1 + ! + interface lo + ! + mpls ldp + dual-stack cisco-interop + neighbor 10.0.1.5 password opensourcerouting + neighbor 172.16.0.1 password opensourcerouting + ! + address-family ipv4 + discovery transport-address 10.0.1.1 + label local advertise explicit-null + ! + interface eth0 + ! + interface eth1 + ! + ! + address-family ipv6 + discovery transport-address 2001:db8::1 + ! + interface eth1 + ! + ! + ! + l2vpn ENG type vpls + bridge br0 + member interface eth2 + ! + member pseudowire mpw0 + neighbor lsr-id 1.1.1.1 + pw-id 100 + ! + ! + diff --git a/doc/user/mgmtd.rst b/doc/user/mgmtd.rst new file mode 100644 index 0000000..42bc860 --- /dev/null +++ b/doc/user/mgmtd.rst @@ -0,0 +1,409 @@ +.. _mgmtd: + +************************* +MGMTd (Management Daemon) +************************* + +The FRR Management Daemon (from now on referred to as MGMTd) is a new +centralized entity representing the FRR Management Plane which can take +management requests from any kind of UI/Frontend entity (e.g. CLI, Netconf, +Restconf, Grpc etc.) over a new unified and common Frontend interface and +can help maintain configurational data or retrieve operational data from +any number of FRR managed entities/components that have been integrated +with the new FRR Centralised Management Framework. + +For organizing the management data to be owned by the FRR Management plane, +management data is stored in YANG in compliance with a pre-defined set +of YANG based schema. Data shall also be stored/retrieved in YANG format only. + +The MGMTd also acts as a separate computational entity for offloading much +of the management related computational overload involved in maintaining of +management data and processing of management requests, from individual +component daemons (which can otherwise be a signficant burden on the individual +components, affecting performance of its other functionalities). + +Lastly, the MGMTd works in-tandem with one (or more) MGMT Frontend +Clients and a bunch of MGMT Backend Clients to realize the entirety +of the FRR Management plane. Some of the advanatages of this new framework +are: + + 1. Consolidation and management of all Management data by a single entity. + 2. Better control over configuration validation, commit and rollback. + 3. Faster collection of configuration data (without needing to involve + individual component daemons). + 4. Offload computational burden of YANG data parsing and validations + of new configuration data being provisoned away from individual + component daemons + 5. Improve performance of individual component daemons while loading + huge configuration or retrieving huge operational dataset. + +The new FRR Management Daemon consists of the following sub-components: + - MGMT Frontend Interface + - MGMT Backend Interface + - MGMT Transaction Engine + +.. _mgmt_fe: + +MGMT Frontend Interface +======================= + +The MGMT Frontend Interface is a bunch of message-based APIs that lets +any UI/Frontend client to interact with the MGMT daemon to requests a +set of management operations on a specific datastore/database. +Following is a list of databases/datastores supported by the MGMT +Frontend Interface and MGMTd: + + - Candidate Database: + + - Consists of configuration data items only. + - Data can be edited anytime using SET_CONFIG API. + - Data can be retrieved anytime using GET_CONFIG/GET_DATA API. + + - Running Database: + + - Consists of configuration data items only. + - Data cannot be edited using SET_CONFIG API. + - Data can only be modified using COMMIT_CONFIG API after which un-committed + data from Candidate database will be first validated and applied to + individualBackend component(s). Only on successful validation and apply on + all individual components will the new data be copied over to the Running + database. + - Data can be retrieved anytime using GET_CONFIG/GET_DATA API. + + - Operational Database: + + - Consists of non-configurational data items. + - Data is not stored on MGMT daemon. Rather it will be need to be fetched + in real-time from the corresponding Backend component (if present). + - Data can be retrieved anytime using GET_DATA API. + +Frontend Clients connected to MGMTd via Frontend Interface can themselves have +multiple connections from one (or more) of its own remote clients. The MGMT +Frontend Interface supports reresenting each of the remote clients for a given +Frontend client(e.g. Netconf clients on a single Netconf server) as individual +Frontend Client Sessions. So a single connection from a single Frontend Client +can create more than one Frontend Client sessions. + +Following are some of the management operations supported: + - INIT_SESSION/CLOSE_SESSION: Create/Destroy a session. Rest of all the + operations are supported only in the context of a specific session. + - LOCK_DB/UNLOCK_DB: Lock/Unlock Management datastores/databases. + - GET_CONFIG/GET_DATA: Retrieve configurational/operational data from a + specific datastore/database. + - SET_CONFIG/DELETE_CONFIG: Add/Modify/Delete specific data in a specific + datastore/database. + - COMMIT_CONFIG: Validate and/or apply the uncommited set of configurations + from one configuration database to another. + - Currently committing configurations from Candidate to Running database + is only allowed, and not vice versa. + +The exact set of message-based APIs are represented as Google Protobuf +messages and can be found in the following file distributed with FRR codebase. + +.. code-block:: frr + + lib/mgmt.proto + +The MGMT daemon implements a MGMT Frontend Server that opens a UNIX +socket-based IPC channel on the following path to listen for incoming +connections from all possible Frontend clients: + +.. code-block:: frr + + /var/run/frr/mgmtd_fe.sock + +Each connection received from a Frontend client is managed and tracked +as a MGMT Frontend adapter by the MGMT Frontend Adapter sub-component +implemented by MGMTd. + +To facilitate faster development/integration of Frontend clients with +MGMT Frontend Interface, a C-based library has been developed. The API +specification of this library can be found at: + +.. code-block:: frr + + lib/mgmt_fe_client.h + +Following is a list of message types supported on the MGMT Frontend Interface: + - SESSION_REQ<Client-Connection-Id, Destroy> + - SESSION_REPLY<Client-Connection-Id, Destroy, Session-Id> + - LOCK_DB_REQ <Session-Id, Database-Id> + - LOCK_DB_REPLY <Session-Id, Database-Id> + - UNLOCK_DB_REQ <Session-Id, Database-Id> + - UNLOCK_DB_REPLY <Session-Id, Database-Id> + - GET_CONFIG_REQ <Session-Id, Database-Id, Base-Yang-Xpath> + - GET_CONFIG_REPLY <Session-Id, Database-Id, Base-Yang-Xpath, Yang-Data-Set> + - SET_CONFIG_REQ <Session-Id, Database-Id, Base-Yang-Xpath, Delete, ...> + - SET_CONFIG_REPLY <Session-Id, Database-id, Base-Yang-Xpath, ..., Status> + - COMMIT_CONFIG_REQ <Session-Id, Source-Db-Id, Dest-Db-Id> + - COMMIT_CONFIG_REPLY <Session-Id, Source-Db-id, Dest-Db-Id, Status> + - GET_DATA_REQ <Session-Id, Database-Id, Base-Yang-Xpath> + - GET_DATA_REPLY <Session-Id, Database-id, Base-Yang-Xpath, Yang-Data-Set> + - REGISTER_NOTIFY_REQ <Session-Id, Database-Id, Base-Yang-Xpath> + - DATA_NOTIFY_REQ <Database-Id, Base-Yang-Xpath, Yang-Data-Set> + +Please refer to the MGMT Frontend Client Developers Reference and Guide +(coming soon) for more details. + +MGMTD Backend Interface +======================= +The MGMT Backend Interface is a bunch of message-based APIs that can be +used by individual component daemons like BGPd, Staticd, Zebra to connect +with MGMTd and utilize the new FRR Management Framework to let any Frontend +clients to retrieve any operational data or manipulate any configuration data +owned by the individual daemon component. + +Like the MGMT Frontend Interface, the MGMT Backend Interface is is also +comprised of the following: + + - MGMT Backend Server (running on MGMT daemon) + - MGMT Backend Adapter (running on MGMT daemon) + - MGMT Backend client (running on Backend component daemons) + +The MGMT Backend Client and MGMT Backend Adapter sub-component communicates +using a specific set of message-based APIs. + +The exact set of message-based APIs are represented as Google Protobuf +messages and can be found in the following file distributed with FRR codebase. + +.. code-block:: frr + + lib/mgmt.proto + +The MGMT daemon implements a MGMT Backend Server that opens a UNIX +socket-based IPC channel on the following path to listen for incoming +connections from all possible Backend clients: + +.. code-block:: frr + + /var/run/frr/mgmtd_be.sock + +Each connection received from a Backend client is managed and tracked +as a MGMT Backend adapter by the MGMT Backend Adapter sub-component +implemented by MGMTd. + +To facilitate faster development/integration of Backend clients with +MGMTd, a C-based library has been developed. The API specification +of this library can be found at: + +.. code-block:: frr + + lib/mgmt_be_client.h + +Following is a list of message types supported on the MGMT Backend Interface: + + - SUBSCRIBE_REQ <Req-Id, Base-Yang-Xpath, Filter-Type> + - SUBSCRIBE_REPLY <Req-Id, Status> + - TXN_REQ <Txn-Id, Create> + - TXN_REPLY <Txn-Id, Status> + - CREATE_CFGDATA_REQ <Txn-Id, Req-Id, Batch-Id, ConfigDataContents> + - CREATE_CFGDATA_ERROR <Txn-Id, Req-Id, Batch-Id, Status> + - VALIDATE_CFGDATA_REQ <Txn-Id, Batch-Id> + - VALIDATE_CFGDATA_REPLY <Txn-Id, Batch-Id, Status, ErrorInfo> + - APPLY_CFGDATA_REQ <Txn-Id, Batch-Id> + - APPLY_CFGDATA_REPLY <Txn-Id, Batch-Id, Status, ErrorInfo> + - GET_OPERDATA_REQ <Txn-Id, Base-Yang-Xpath, Filter-Type> + - GET_OPERDATA_REPLY <Txn-Id, OperDataContents> + +Please refer to the MGMT Backend Client Developers Reference and Guide +(coming soon) for more details. + +MGMTD Transaction Engine +======================== + +The MGMT Transaction sub-component is the main brain of the MGMT daemon that +takes management requests from one (or more) Frontend Client translates +them into transactions and drives them to completion in co-oridination with +one (or more) Backend client daemons involved in the request. + +A transaction can be seen as a set of management procedures executed over +the Backend Interface with one (or more) individual Backend component +daemons, as a result of some management request initiated from a specific +Frontend client session. These group of operations on the Backend Interface +with one (or more) individual components involved should be executed without +taking any further management requests from other Frontend client sessions. +To maintain this kind of atomic behavior a lock needs to be acquired +(sometimes implicitly if not explicitly) by the corresponding Frontend client +session, on the various datastores/databases involved in the management request +being executed. The same datastores/databases need to be unlocked when all +the procedures have been executed and the transaction is being closed. + +Following are some of the transaction types supported by MGMT: + + - Configuration Transactions + + - Used to execute management operations like SET_CONFIG and COMMIT_CONFIG + that involve writing/over-writing the contents of Candidate and Running + databases. + - One (and only) can be created and be in-progress at any given time. + - Once initiated by a specific Frontend Client session and is still + in-progress, all subsequent SET_CONFIG and COMMIT_CONFIG operations + from other Frontend Client sessions will be rejected and responded + with failure. + - Requires acquiring write-lock on Candidate (and later Running) databases. + + - Show Transactions + + - Used to execute management operations like GET_CONFIG and GET_DATA + that involve only reading the contents of Candidate and Running + databases (and sometimes real-time retrieval of operational data + from individual component daemons). + - Multiple instance of this transaction type can be created and be + in-progress at any given time. + - However, when a configuration transaction is currently in-progress + show transaction can be initiated by any Frontend Client session. + - Requires acquiring read-lock on Candidate and/or Running databases. + - NOTE: Currently GET_DATA on Operational database is NOT supported. To + be added in a future time soon. + +MGMTD Configuration Rollback and Commit History +=============================================== + +The MGMT daemon maintains upto 10 last configuration commit buffers +and can rollback the contents of the Running Database to any of the +commit-ids maintained in the commit buffers. + +Once the number of commit buffers exceeds 10, the oldest commit +buffer is deleted to make space for the latest commit. Also on +rollback to a specific commit-id, buffer of all the later commits +are deleted from commit record. + +Configuration rollback is only allowed via VTYSH shell as of today +and is not possible through the MGMT Frontend interface. + +MGMT Configuration commands +=========================== + +.. clicmd:: mgmt set-config XPATH VALUE + + This command uses a SET_CONFIG request over the MGMT Frontend Interface + for the specified xpath with specific value. This command is used for + testing purpose only. But can be used to set configuration data from CLI + using SET_CONFIG operations. + +.. clicmd:: mgmt delete-config XPATH + + This command uses a SET_CONFIG request (with delete option) over the + MGMT Frontend Interface o delete the YANG data node at the given + xpath unless it is a key-leaf node(in which case it is not deleted). + +.. clicmd:: mgmt load-config FILE <merge|replace> + + This command loads configuration in JSON format from the filepath specified, + and merges or replaces the Candidate DB as per the option specified. + +.. clicmd:: mgmt save-config <candidate|running> FILE + + This command dumps the DB specified in the db-name into the file in JSON + format. This command in not supported for the Operational DB. + +.. clicmd:: mgmt commit abort + + This command will abort any configuration present on the Candidate but not + been applied to the Running DB. + +.. clicmd:: mgmt commit apply + + This command commits any uncommited changes in the Candidate DB to the + Running DB. + +.. clicmd:: mgmt commit check + + This command validates the configuration but does not apply them to the + Running DB. + +.. clicmd:: mgmt rollback commit-id WORD + + This command rolls back the Running Database contents to the state + corresponding to the commit-id specified. + +.. clicmd:: mgmt rollback last WORD + + This command rolls back the last specified number of recent commits. + + +MGMT Show commands +================== + +.. clicmd:: show mgmt backend-adapter all + + This command shows the backend adapter information and the clients/daemons + connected to the adapters. + +.. clicmd:: show mgmt backend-yang-xpath-registry + + This command shows which Backend adapters are registered for which YANG + data subtree(s). + +.. clicmd:: show mgmt frontend-adapter all [detail] + + This command shows the frontend adapter information and the clients + connected to the adapters. + +.. clicmd:: show mgmt transaction all + + Shows the list of transaction and bunch of information about the transaction. + +.. clicmd:: show mgmt get-config [candidate|running] XPATH + + This command uses the GET_CONFIG operation over the MGMT Frontend interface and + returns the xpaths and values of the nodes of the subtree pointed by the <xpath>. + +.. clicmd:: show mgmt get-data [candidate|operation|running] XPATH + + This command uses the GET_DATA operation over the MGMT Frontend interface and + returns the xpaths and values of the nodes of the subtree pointed by the <xpath>. + Currenlty supported values for 'candidate' and 'running' only + ('operational' shall be supported in future soon). + +.. clicmd:: show mgmt datastore-contents [candidate|operation|running] [xpath WORD] [file WORD] json|xml + + This command dumps the subtree pointed by the xpath in JSON or XML format. If filepath is + not present then the tree will be printed on the shell. + +.. clicmd:: show mgmt commit-history + + This command dumps details of upto last 10 commits handled by MGMTd. + + +MGMT Daemon debug commands +========================== + +The following debug commands enable debugging within the management daemon: + +.. clicmd:: [no] debug mgmt backend + + Enable[/Disable] debugging messages related to backend operations within the + management daemon. + +.. clicmd:: [no] debug mgmt datastore + + Enable[/Disable] debugging messages related to YANG datastore operations + within the management daemon. + +.. clicmd:: [no] debug mgmt frontend + + Enable[/Disable] debugging messages related to frontend operations within the + management daemon. + +.. clicmd:: [no] debug mgmt transaction + + Enable[/Disable] debugging messages related to transactions within the + management daemon. + + +MGMT Client debug commands +========================== + +The following debug commands enable debugging within the management front and +backend clients: + +.. clicmd:: [no] debug mgmt client backend + + Enable[/Disable] debugging messages related to backend operations inside the + backend mgmtd clients. + +.. clicmd:: [no] debug mgmt client frontend + + Enable[/Disable] debugging messages related to frontend operations inside the + frontend mgmtd clients. diff --git a/doc/user/nexthop_groups.rst b/doc/user/nexthop_groups.rst new file mode 100644 index 0000000..45f64ee --- /dev/null +++ b/doc/user/nexthop_groups.rst @@ -0,0 +1,29 @@ +.. _nexthop-groups: + +Nexthop Groups +============== + +Nexthop groups are a way to encapsulate ECMP information together. It's a +listing of ECMP nexthops used to forward packets. + +.. clicmd:: nexthop-group NAME + + Create a nexthop-group with an associated NAME. This will put you into a + sub-mode where you can specify individual nexthops. To exit this mode type + exit or end as per normal conventions for leaving a sub-mode. + +.. clicmd:: nexthop [A.B.C.D|X:X::X:XX] [interface [onlink]] [nexthop-vrf NAME] [label LABELS] + + Create a v4 or v6 nexthop. All normal rules for creating nexthops that you + are used to are allowed here. The syntax was intentionally kept the same as + creating nexthops as you would for static routes. + +.. clicmd:: resilient buckets (1-256) idle-timer (1-4294967295) unbalanced-timer (1-4294967295) + + Create a resilient Nexthop Group with the specified number of buckets, and + associated timers. Instead of using the normal kernel hashing methodology + this specifies that X buckets will be created for the nexthop group and + when a nexthop is lost the buckets forwarding that particular nexthop + will be automatically re-assigned. This cli command must be the first + command entered currently. Additionally this command only works with linux 5.19 + kernels or newer. diff --git a/doc/user/nhrpd.rst b/doc/user/nhrpd.rst new file mode 100644 index 0000000..54527a0 --- /dev/null +++ b/doc/user/nhrpd.rst @@ -0,0 +1,447 @@ +.. _nhrp: + +**** +NHRP +**** + +*nhrpd* is an implementation of the :abbr:`NHRP (Next Hop Routing Protocol)`. +NHRP is described in :rfc:`2332`. + +NHRP is used to improve the efficiency of routing computer network traffic over +:abbr:`NBMA (Non-Broadcast, Multiple Access)` networks. NHRP provides an +ARP-like solution that allows a system to dynamically learn the NBMA address of +the other systems that are part of that network, allowing these systems to +directly communicate without requiring traffic to use an intermediate hop. + +NHRP is a client-server protocol. The server side is called the :abbr:`NHS +(Next Hop Server)` or the hub, while a client is referred to as the :abbr:`NHC +(Next Hop Client)` or the spoke. When a node is configured as an NHC, it +registers its address with the NHS which keeps track of all registered spokes. +An NHC client can then query the addresses of other clients from NHS allowing +all spokes to communicate directly with each other. + +Cisco Dynamic Multipoint VPN (DMVPN) is based on NHRP, and |PACKAGE_NAME| nhrpd +implements this scenario. + +.. _routing-design: + +Routing Design +============== + +nhrpd never handles routing of prefixes itself. You need to run some +real routing protocol (e.g. BGP) to advertise routes over the tunnels. +What nhrpd does it establishes 'shortcut routes' that optimizes the +routing protocol to avoid going through extra nodes in NBMA GRE mesh. + +nhrpd does route NHRP domain addresses individually using per-host prefixes. +This is similar to Cisco FlexVPN; but in contrast to opennhrp which uses +a generic subnet route. + +To create NBMA GRE tunnel you might use the following (Linux terminal +commands): + +.. code-block:: console + + ip tunnel add gre1 mode gre key 42 ttl 64 + ip addr add 10.255.255.2/32 dev gre1 + ip link set gre1 up + + +Note that the IP-address is assigned as host prefix to gre1. nhrpd will +automatically create additional host routes pointing to gre1 when +a connection with these hosts is established. + +The gre1 subnet prefix should be announced by routing protocol from the +hub nodes (e.g. BGP 'network' announce). This allows the routing protocol +to decide which is the closest hub and determine the relay hub on prefix +basis when direct tunnel is not established. + +nhrpd will redistribute directly connected neighbors to zebra. Within +hub nodes, these routes should be internally redistributed using some +routing protocol (e.g. iBGP) to allow hubs to be able to relay all traffic. + +This can be achieved in hubs with the following bgp configuration (network +command defines the GRE subnet): + +.. code-block:: frr + + router bgp 65555 + address-family ipv4 unicast + network 172.16.0.0/16 + redistribute nhrp + exit-address-family + + +.. _configuring-nhrp: + +Configuring NHRP +================ + +.. clicmd:: ip nhrp holdtime (1-65000) + + Holdtime is the number of seconds that have to pass before stopping to + advertise an NHRP NBMA address as valid. It also controls how often NHRP + registration requests are sent. By default registrations are sent every one + third of the holdtime. + +.. clicmd:: ip nhrp map A.B.C.D|X:X::X:X A.B.C.D|local + + Map an IP address of a station to the station's NBMA address. + +.. clicmd:: ip nhrp network-id (1-4294967295) + + Enable NHRP on this interface and set the interface's network ID. The + network ID is used to allow creating multiple nhrp domains on a router when + multiple interfaces are configured on the router. Interfaces configured + with the same ID are part of the same logical NBMA network. The ID is a + local only parameter and is not sent to other NHRP nodes and so IDs on + different nodes do not need to match. When NHRP packets are received on an + interface they are assigned to the local NHRP domain for that interface. + +.. clicmd:: ip nhrp nhs A.B.C.D nbma A.B.C.D|FQDN + + Configure the Next Hop Server address and its NBMA address. + +.. clicmd:: ip nhrp nhs dynamic nbma A.B.C.D + + Configure the Next Hop Server to have a dynamic address and set its NBMA + address. + +.. clicmd:: ip nhrp registration no-unique + + Allow the client to not set the unique flag in the NHRP packets. This is + useful when a station has a dynamic IP address that could change over time. + +.. clicmd:: ip nhrp shortcut + + Enable shortcut (spoke-to-spoke) tunnels to allow NHC to talk to each others + directly after establishing a connection without going through the hub. + +.. clicmd:: ip nhrp mtu + + Configure NHRP advertised MTU. + + +.. _hub-functionality: + +Hub Functionality +================= + +In addition to routing nhrp redistributed host prefixes, the hub nodes +are also responsible to send NHRP Traffic Indication messages that +trigger creation of the shortcut tunnels. + +nhrpd sends Traffic Indication messages based on network traffic captured +using NFLOG. Typically you want to send Traffic Indications for network +traffic that is routed from gre1 back to gre1 in rate limited manner. +This can be achieved with the following iptables rule. + +.. code-block:: shell + + iptables -A FORWARD -i gre1 -o gre1 \\ + -m hashlimit --hashlimit-upto 4/minute --hashlimit-burst 1 \\ + --hashlimit-mode srcip,dstip --hashlimit-srcmask 24 --hashlimit-dstmask 24 \\ + --hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 --nflog-range 128 + + +You can fine tune the src/dstmask according to the prefix lengths you announce +internal, add additional IP range matches, or rate limitation if needed. +However, the above should be good in most cases. + +This kernel NFLOG target's nflog-group is configured in global nhrp config +with: + +.. clicmd:: nhrp nflog-group (1-65535) + +To start sending these traffic notices out from hubs, use the nhrp +per-interface directive: + +.. clicmd:: ip nhrp redirect + +This enable redirect replies on the NHS similar to ICMP redirects except this +is managed by the nhrp protocol. This setting allows spokes to communicate with +each others directly. + +.. _integration-with-ike: + +Integration with IKE +==================== + +nhrpd needs tight integration with IKE daemon for various reasons. +Currently only strongSwan is supported as IKE daemon. + +nhrpd connects to strongSwan using VICI protocol based on UNIX socket which +can be configured using the command below (default to /var/run/charon.vici). + +strongSwan currently needs few patches applied. Please check out the +original patches at: +https://git-old.alpinelinux.org/user/tteras/strongswan/ + +Actively maintained patches are also available at: +https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/main/strongswan + +.. _multicast-functionality: + +Multicast Functionality +======================= + +nhrpd can be configured to forward multicast packets, allowing routing +protocols that use multicast (such as OSPF) to be supported in the DMVPN +network. + +This support requires an iptables NFLOG rule to allow nhrpd to intercept +multicast packets. A second iptables rule is also usually used to drop the +original multicast packet. + + .. code-block:: shell + + iptables -A OUTPUT -d 224.0.0.0/24 -o gre1 -j NFLOG --nflog-group 2 + iptables -A OUTPUT -d 224.0.0.0/24 -o gre1 -j DROP + +.. clicmd:: nhrp multicast-nflog-group (1-65535) + + Sets the nflog group that nhrpd will listen on for multicast packets. This + value must match the nflog-group value set in the iptables rule. + +.. clicmd:: ip nhrp map multicast A.B.C.D|X:X::X:X A.B.C.D|dynamic + + Sends multicast packets to the specified NBMA address. If dynamic is + specified then destination NBMA address (or addresses) are learnt + dynamically. + +.. _nhrp-events: + +NHRP Events +=========== + +.. clicmd:: nhrp event socket SOCKET + + Configure the Unix path for the event socket. + +.. _show-nhrp: + +Show NHRP +========== + +.. clicmd:: show [ip|ipv6] nhrp cache [json] + + Dump the cache entries. + +.. clicmd:: show [ip|ipv6] nhrp opennhrp [json] + + Dump the cache entries with opennhrp format. + +.. clicmd:: show [ip|ipv6] nhrp nhs [json] + + Dump the hub context. + +.. clicmd:: show dmvpn [json] + + Dump the security contexts. + +Configuration Example +===================== + +.. figure:: ../figures/fig_dmvpn_topologies.png + :alt: image + + image + +IPSec configurration example +---------------------------- + +This changes required on all nodes as HUB and Spokes. + +ipsec.conf file + +.. code-block:: shell + + config setup + conn dmvpn + authby=secret + auto=add + keyexchange=ikev2 + ike=aes256-aes256-sha256-modp2048 + esp=aes256-aes256-sha256-modp2048 + dpdaction=clear + dpddelay=300s + left=%any + leftid=%any + right=%any + rightid=%any + leftprotoport=gre + rightprotoport=gre + type=transport + keyingtries=%forever + +ipsec.secrets file + +.. code-block:: shell + + %any : PSK "some_s3cret!" + + +HUB configuration example +------------------------- + +Creating gre interface + +.. code-block:: console + + ip tunnel add gre1 mode gre key 42 ttl 64 + ip addr add 10.0.0.254/32 dev gre1 + ip link set gre1 up + +Adding iptables rules to provide possibility shortcut tunnels and connect spokes directly + +.. code-block:: shell + + iptables -A FORWARD -i gre1 -o gre1 \\ + -m hashlimit --hashlimit-upto 4/minute --hashlimit-burst 1 \\ + --hashlimit-mode srcip,dstip --hashlimit-srcmask 24 --hashlimit-dstmask 24 \\ + --hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 --nflog-range 128 + +FRR config on HUB + +.. code-block:: frr + + nhrp nflog-group 1 + ! + interface gre1 + description DMVPN Tunnel Interface + ip address 10.0.0.254/32 + ip nhrp network-id 1 + ip nhrp redirect + ip nhrp registration no-unique + ip nhrp shortcut + tunnel protection vici profile dmvpn + tunnel source eth0 + ! + router bgp 65000 + bgp router-id 10.0.0.254 + no bgp ebgp-requires-policy + neighbor SPOKES peer-group + neighbor SPOKES disable-connected-check + neighbor 10.0.0.1 remote-as 65001 + neighbor 10.0.0.1 peer-group SPOKES + neighbor 10.0.0.2 remote-as 65002 + neighbor 10.0.0.2 peer-group SPOKES + neighbor 10.0.0.3 remote-as 65003 + neighbor 10.0.0.3 peer-group SPOKES + ! + address-family ipv4 unicast + network 172.16.0.0/24 + redistribute nhrp + exit-address-family + +Spoke1 configuration +-------------------- + +Creating gre interface + +.. code-block:: console + + ip tunnel add gre1 mode gre key 42 ttl 64 + ip addr add 10.0.0.1/32 dev gre1 + ip link set gre1 up + + +FRR config on Spoke1 + +.. code-block:: frr + + interface gre1 + description DMVPN Tunnel Interface + ip address 10.0.0.1/32 + ip nhrp network-id 1 + ip nhrp nhs dynamic nbma 198.51.100.1 + ip nhrp redirect + ip nhrp registration no-unique + ip nhrp shortcut + no link-detect + tunnel protection vici profile dmvpn + tunnel source eth0 + ! + router bgp 65001 + no bgp ebgp-requires-policy + neighbor 10.0.0.254 remote-as 65000 + neighbor 10.0.0.254 disable-connected-check + ! + address-family ipv4 unicast + network 172.16.1.0/24 + exit-address-family + + +Spoke2 configuration +-------------------- + +Creating gre interface + +.. code-block:: console + + ip tunnel add gre1 mode gre key 42 ttl 64 + ip addr add 10.0.0.1/32 dev gre1 + ip link set gre1 up + +FRR config on Spoke2 + +.. code-block:: frr + + interface gre1 + description DMVPN Tunnel Interface + ip address 10.0.0.2/32 + ip nhrp network-id 1 + ip nhrp nhs dynamic nbma 198.51.100.1 + ip nhrp redirect + ip nhrp registration no-unique + ip nhrp shortcut + no link-detect + tunnel protection vici profile dmvpn + tunnel source eth0 + ! + router bgp 65002 + no bgp ebgp-requires-policy + neighbor 10.0.0.254 remote-as 65000 + neighbor 10.0.0.254 disable-connected-check + ! + address-family ipv4 unicast + network 172.16.2.0/24 + exit-address-family + + +Spoke3 configuration +-------------------- + +Creating gre interface + +.. code-block:: console + + ip tunnel add gre1 mode gre key 42 ttl 64 + ip addr add 10.0.0.3/32 dev gre1 + ip link set gre1 up + +FRR config on Spoke3 + +.. code-block:: frr + + interface gre1 + description DMVPN Tunnel Interface + ip address 10.0.0.3/32 + ip nhrp network-id 1 + ip nhrp nhs dynamic nbma 198.51.100.1 + ip nhrp redirect + ip nhrp registration no-unique + ip nhrp shortcut + no link-detect + tunnel protection vici profile dmvpn + tunnel source eth0 + ! + router bgp 65003 + no bgp ebgp-requires-policy + neighbor 10.0.0.254 remote-as 65000 + neighbor 10.0.0.254 disable-connected-check + ! + address-family ipv4 unicast + network 172.16.3.0/24 + exit-address-family + diff --git a/doc/user/ospf6d.rst b/doc/user/ospf6d.rst new file mode 100644 index 0000000..2f4c956 --- /dev/null +++ b/doc/user/ospf6d.rst @@ -0,0 +1,879 @@ +.. _ospfv3: + +****** +OSPFv3 +****** + +*ospf6d* is a daemon support OSPF version 3 for IPv6 network. OSPF for IPv6 is +described in :rfc:`2740`. + +.. _ospf6-router: + +OSPF6 router +============ + +.. clicmd:: router ospf6 [vrf NAME] + +.. clicmd:: ospf6 router-id A.B.C.D + + Set router's Router-ID. + +.. clicmd:: timers throttle spf (0-600000) (0-600000) (0-600000) + + This command sets the initial `delay`, the `initial-holdtime` + and the `maximum-holdtime` between when SPF is calculated and the + event which triggered the calculation. The times are specified in + milliseconds and must be in the range of 0 to 600000 milliseconds. + + The `delay` specifies the minimum amount of time to delay SPF + calculation (hence it affects how long SPF calculation is delayed after + an event which occurs outside of the holdtime of any previous SPF + calculation, and also serves as a minimum holdtime). + + Consecutive SPF calculations will always be separated by at least + 'hold-time' milliseconds. The hold-time is adaptive and initially is + set to the `initial-holdtime` configured with the above command. + Events which occur within the holdtime of the previous SPF calculation + will cause the holdtime to be increased by `initial-holdtime`, bounded + by the `maximum-holdtime` configured with this command. If the adaptive + hold-time elapses without any SPF-triggering event occurring then + the current holdtime is reset to the `initial-holdtime`. + + .. code-block:: frr + + router ospf6 + timers throttle spf 200 400 10000 + + + In this example, the `delay` is set to 200ms, the initial holdtime is set + to 400ms and the `maximum holdtime` to 10s. Hence there will always be at + least 200ms between an event which requires SPF calculation and the actual + SPF calculation. Further consecutive SPF calculations will always be + separated by between 400ms to 10s, the hold-time increasing by 400ms each + time an SPF-triggering event occurs within the hold-time of the previous + SPF calculation. + +.. clicmd:: auto-cost reference-bandwidth COST + + + This sets the reference bandwidth for cost calculations, where this + bandwidth is considered equivalent to an OSPF cost of 1, specified in + Mbits/s. The default is 100Mbit/s (i.e. a link of bandwidth 100Mbit/s + or higher will have a cost of 1. Cost of lower bandwidth links will be + scaled with reference to this cost). + + This configuration setting MUST be consistent across all routers + within the OSPF domain. + +.. clicmd:: maximum-paths (1-64) + + Use this command to control the maximum number of parallel routes that + OSPFv3 can support. The default is 64. + +.. clicmd:: write-multiplier (1-100) + + Use this command to tune the amount of work done in the packet read and + write threads before relinquishing control. The parameter is the number + of packets to process before returning. The default value of this parameter + is 20. + +.. clicmd:: clear ipv6 ospf6 process [vrf NAME] + + This command clears up the database and routing tables and resets the + neighborship by restarting the interface state machine. This will be + helpful when there is a change in router-id and if user wants the router-id + change to take effect, user can use this cli instead of restarting the + ospf6d daemon. + +.. clicmd:: clear ipv6 ospf6 [vrf NAME] interface [IFNAME] + + This command restarts the interface state machine for all interfaces in the + VRF or only for the specific interface if ``IFNAME`` is specified. + +ASBR Summarisation Support in OSPFv3 +==================================== + + External routes in OSPFv3 are carried by type 5/7 LSA (external LSAs). + External LSAs are generated by ASBR (Autonomous System Boundary Router). + Large topology database requires a large amount of router memory, which + slows down all processes, including SPF calculations. + It is necessary to reduce the size of the OSPFv3 topology database, + especially in a large network. Summarising routes keeps the routing + tables smaller and easier to troubleshoot. + + External route summarization must be configured on ASBR. + Stub area do not allow ASBR because they don’t allow type 5 LSAs. + + An ASBR will inject a summary route into the OSPFv3 domain. + + Summary route will only be advertised if you have at least one subnet + that falls within the summary range. + + Users will be allowed an option in the CLI to not advertise range of + ipv6 prefixes as well. + + The configuration of ASBR Summarisation is supported using the CLI command + +.. clicmd:: summary-address X:X::X:X/M [tag (1-4294967295)] [{metric (0-16777215) | metric-type (1-2)}] + + This command will advertise a single External LSA on behalf of all the + prefixes falling under this range configured by the CLI. + The user is allowed to configure tag, metric and metric-type as well. + By default, tag is not configured, default metric as 20 and metric-type + as type-2 gets advertised. + A summary route is created when one or more specific routes are learned and + removed when no more specific route exist. + The summary route is also installed in the local system with Null0 as + next-hop to avoid leaking traffic. + +.. clicmd:: no summary-address X:X::X:X/M [tag (1-4294967295)] [{metric (0-16777215) | metric-type (1-2)}] + + This command can be used to remove the summarisation configuration. + This will flush the single External LSA if it was originated and advertise + the External LSAs for all the existing individual prefixes. + +.. clicmd:: summary-address X:X::X:X/M no-advertise + + This command can be used when user do not want to advertise a certain + range of prefixes using the no-advertise option. + This command when configured will flush all the existing external LSAs + falling under this range. + +.. clicmd:: no summary-address X:X::X:X/M no-advertise + + This command can be used to remove the previous configuration. + When configured, tt will resume originating external LSAs for all the prefixes + falling under the configured range. + +.. clicmd:: aggregation timer (5-1800) + + The summarisation command takes effect after the aggregation timer expires. + By default the value of this timer is 5 seconds. User can modify the time + after which the external LSAs should get originated using this command. + +.. clicmd:: no aggregation timer (5-1800) + + This command removes the timer configuration. It reverts back to default + 5 second timer. + +.. clicmd:: show ipv6 ospf6 summary-address [detail] [json] + + This command can be used to see all the summary-address related information. + When detail option is used, it shows all the prefixes falling under each + summary-configuration apart from other information. + +.. _ospf6-area: + +OSPF6 area +========== + +.. clicmd:: area A.B.C.D range X:X::X:X/M [<advertise|not-advertise|cost (0-16777215)>] + +.. clicmd:: area (0-4294967295) range X:X::X:X/M [<advertise|not-advertise|cost (0-16777215)>] + + Summarize a group of internal subnets into a single Inter-Area-Prefix LSA. + This command can only be used at the area boundary (ABR router). + + By default, the metric of the summary route is calculated as the highest + metric among the summarized routes. The `cost` option, however, can be used + to set an explicit metric. + + The `not-advertise` option, when present, prevents the summary route from + being advertised, effectively filtering the summarized routes. + +.. clicmd:: area A.B.C.D nssa [no-summary] [default-information-originate [metric-type (1-2)] [metric (0-16777214)]] + +.. clicmd:: area (0-4294967295) nssa [no-summary] [default-information-originate [metric-type (1-2)] [metric (0-16777214)]] + + Configure the area to be a NSSA (Not-So-Stubby Area). + + The following functionalities are implemented as per RFC 3101: + + 1. Advertising Type-7 LSA into NSSA area when external route is + redistributed into OSPFv3. + 2. Processing Type-7 LSA received from neighbor and installing route in the + route table. + 3. Support for NSSA ABR functionality which is generating Type-5 LSA when + backbone area is configured. Currently translation of Type-7 LSA to + Type-5 LSA is enabled by default. + 4. Support for NSSA Translator functionality when there are multiple NSSA + ABR in an area. + + An NSSA ABR can be configured with the `no-summary` option to prevent the + advertisement of summaries into the area. In that case, a single Type-3 LSA + containing a default route is originated into the NSSA. + + NSSA ABRs and ASBRs can be configured with `default-information-originate` + option to originate a Type-7 default route into the NSSA area. In the case + of NSSA ASBRs, the origination of the default route is conditioned to the + existence of a default route in the RIB that wasn't learned via the OSPF + protocol. + +.. clicmd:: area A.B.C.D nssa range X:X::X:X/M [<not-advertise|cost (0-16777215)>] + +.. clicmd:: area (0-4294967295) nssa range X:X::X:X/M [<not-advertise|cost (0-16777215)>] + + Summarize a group of external subnets into a single Type-7 LSA, which is + then translated to a Type-5 LSA and avertised to the backbone. + This command can only be used at the area boundary (NSSA ABR router). + + By default, the metric of the summary route is calculated as the highest + metric among the summarized routes. The `cost` option, however, can be used + to set an explicit metric. + + The `not-advertise` option, when present, prevents the summary route from + being advertised, effectively filtering the summarized routes. + +.. clicmd:: area A.B.C.D export-list NAME + +.. clicmd:: area (0-4294967295) export-list NAME + + Filter Type-3 summary-LSAs announced to other areas originated from intra- + area paths from specified area. + + .. code-block:: frr + + router ospf6 + area 0.0.0.10 export-list foo + ! + ipv6 access-list foo permit 2001:db8:1000::/64 + ipv6 access-list foo deny any + + With example above any intra-area paths from area 0.0.0.10 and from range + 2001:db8::/32 (for example 2001:db8:1::/64 and 2001:db8:2::/64) are announced + into other areas as Type-3 summary-LSA's, but any others (for example + 2001:200::/48) aren't. + + This command is only relevant if the router is an ABR for the specified + area. + +.. clicmd:: area A.B.C.D import-list NAME + +.. clicmd:: area (0-4294967295) import-list NAME + + Same as export-list, but it applies to paths announced into specified area + as Type-3 summary-LSAs. + +.. clicmd:: area A.B.C.D filter-list prefix NAME in + +.. clicmd:: area A.B.C.D filter-list prefix NAME out + +.. clicmd:: area (0-4294967295) filter-list prefix NAME in + +.. clicmd:: area (0-4294967295) filter-list prefix NAME out + + Filtering Type-3 summary-LSAs to/from area using prefix lists. This command + makes sense in ABR only. + +.. _ospf6-interface: + +OSPF6 interface +=============== + +.. clicmd:: ipv6 ospf6 area <A.B.C.D|(0-4294967295)> + + Enable OSPFv3 on the interface and add it to the specified area. + +.. clicmd:: ipv6 ospf6 cost COST + + Sets interface's output cost. Default value depends on the interface + bandwidth and on the auto-cost reference bandwidth. + +.. clicmd:: ipv6 ospf6 hello-interval HELLOINTERVAL + + Sets interface's Hello Interval. Default 10 + +.. clicmd:: ipv6 ospf6 dead-interval DEADINTERVAL + + Sets interface's Router Dead Interval. Default value is 40. + +.. clicmd:: ipv6 ospf6 graceful-restart hello-delay HELLODELAYINTERVAL + + Set the length of time during which Grace-LSAs are sent at 1-second intervals + while coming back up after an unplanned outage. During this time, no hello + packets are sent. + + A higher hello delay will increase the chance that all neighbors are notified + about the ongoing graceful restart before receiving a hello packet (which is + crucial for the graceful restart to succeed). The hello delay shouldn't be set + too high, however, otherwise the adjacencies might time out. As a best practice, + it's recommended to set the hello delay and hello interval with the same values. + The default value is 10 seconds. + +.. clicmd:: ipv6 ospf6 retransmit-interval RETRANSMITINTERVAL + + Sets interface's Rxmt Interval. Default value is 5. + +.. clicmd:: ipv6 ospf6 priority PRIORITY + + Sets interface's Router Priority. Default value is 1. + +.. clicmd:: ipv6 ospf6 transmit-delay TRANSMITDELAY + + Sets interface's Inf-Trans-Delay. Default value is 1. + +.. clicmd:: ipv6 ospf6 network (broadcast|point-to-point) + + Set explicitly network type for specified interface. + +OSPF6 route-map +=============== + +Usage of *ospfd6*'s route-map support. + +.. clicmd:: set metric [+|-](0-4294967295) + + Set a metric for matched route when sending announcement. Use plus (+) sign + to add a metric value to an existing metric. Use minus (-) sign to + substract a metric value from an existing metric. + +.. _redistribute-routes-to-ospf6: + +Redistribute routes to OSPF6 +============================ + +.. clicmd:: redistribute <babel|bgp|connected|isis|kernel|openfabric|ripng|sharp|static|table> [metric-type (1-2)] [metric (0-16777214)] [route-map WORD] + + Redistribute routes of the specified protocol or kind into OSPFv3, with the + metric type and metric set if specified, filtering the routes using the + given route-map if specified. + +.. clicmd:: default-information originate [{always|metric (0-16777214)|metric-type (1-2)|route-map WORD}] + + The command injects default route in the connected areas. The always + argument injects the default route regardless of it being present in the + router. Metric values and route-map can also be specified optionally. + +Graceful Restart +================ + +.. clicmd:: graceful-restart [grace-period (1-1800)] + + + Configure Graceful Restart (RFC 5187) restarting support. + When enabled, the default grace period is 120 seconds. + + To perform a graceful shutdown, the "graceful-restart prepare ipv6 ospf" + EXEC-level command needs to be issued before restarting the ospf6d daemon. + + When Graceful Restart is enabled and the ospf6d daemon crashes or is killed + abruptely (e.g. SIGKILL), it will attempt an unplanned Graceful Restart once + it restarts. + +.. clicmd:: graceful-restart helper enable [A.B.C.D] + + + Configure Graceful Restart (RFC 5187) helper support. + By default, helper support is disabled for all neighbours. + This config enables/disables helper support on this router + for all neighbours. + To enable/disable helper support for a specific + neighbour, the router-id (A.B.C.D) has to be specified. + +.. clicmd:: graceful-restart helper strict-lsa-checking + + + If 'strict-lsa-checking' is configured then the helper will + abort the Graceful Restart when a LSA change occurs which + affects the restarting router. + By default 'strict-lsa-checking' is enabled" + +.. clicmd:: graceful-restart helper supported-grace-time (10-1800) + + + Supports as HELPER for configured grace period. + +.. clicmd:: graceful-restart helper planned-only + + + It helps to support as HELPER only for planned + restarts. By default, it supports both planned and + unplanned outages. + +.. clicmd:: graceful-restart prepare ipv6 ospf + + + Initiate a graceful restart for all OSPFv3 instances configured with the + "graceful-restart" command. The ospf6d daemon should be restarted during + the instance-specific grace period, otherwise the graceful restart will fail. + + This is an EXEC-level command. + + +.. _Authentication-trailer: + +Authentication trailer support: +=============================== +IPv4 version of OSPF supports authentication as part of the base RFC. +When IPv6 version of OSPF was developed there was IPSec support for IPv6, +Hence OSPFv3(IPv6 version of OSPF) suggest to use IPSec as authentication +and encryption mechanism. IPSec supports authentication using AH header and +Encryption using ESP. + +There are few disadvantages of using IPSec with OSPFv3. + 1. If encryption is enabled for OSPFv3 packets, then its not + possible to give priority to control packets. + 2. IPSec has platform dependency and may not be supported + in all platforms. + 3. It is performance intensive. + 4. Its difficult to configure. + + +Some advantages of OSPFv3 authentication trailer feature. + 1. It provides replay protection via sequence number. + 2. It provides IPv6 source address protection. + 3. No platform dependency. + 4. Easy to implement and maintain. + + +This feature is support for ``RFC7166``. + +FRR supports MD5 and SHA256 internally and relays on openssl for other hash +algorithms. If user wants to use only MD5 and SHA256, no special action is +required. If user wants complete support of authentication trailer with all +hash algorithms follow below steps. + + +Installing Dependencies: +------------------------ + +.. code-block:: console + + sudo apt update + sudo apt-get install openssl + + +Compile: +-------- +Follow normal compilation as mentioned in the build page. If you want to +use all the hash algorithms then follow the steps mentioned in note before +compiling. + + +.. note:: + + If your platform supports ``openssl``, please make sure to add + ``--with-crypto=openssl`` to your configure options. + Default value is ``--with-crypto=internal`` + + +CLI Configuration: +------------------ +There are two ways in which authentication trailer can be configured for +OSPFv3. These commands are mutually exclusive, only one can be configured +at any time. + + 1. Using manual key configuration. + 2. Using keychain. + + +List of hash algorithms supported: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Without openssl: +++++++++++++++++ + ``MD5`` + ``HMAC-SHA-256`` + + +With openssl: ++++++++++++++ + ``MD5`` + ``HMAC-SHA-1`` + ``HMAC-SHA-256`` + ``HMAC-SHA-384`` + ``HMAC-SHA-512`` + + +Example configuration of manual key: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Without openssl: +++++++++++++++++ + +.. clicmd:: ipv6 ospf6 authentication key-id (1-65535) hash-algo <md5|hmac-sha-256> key WORD + +With openssl: ++++++++++++++ + +.. clicmd:: ipv6 ospf6 authentication key-id (1-65535) hash-algo <md5|hmac-sha-256|hmac-sha-1|hmac-sha-384|hmac-sha-512> key WORD + + +Example configuration of keychain: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: ipv6 ospf6 authentication keychain KEYCHAIN_NAME + + +Running configuration: +---------------------- + +Manual key: +^^^^^^^^^^^ + +.. code-block:: frr + + frr# show running-config + Building configuration... + + Current configuration: + ! + interface ens192 + ipv6 address 2001:DB8::2/64 + ipv6 ospf6 authentication key-id 10 hash-algo hmac-sha-256 key abhinay + +Keychain: +^^^^^^^^^ + +.. code-block:: frr + + frr# show running-config + Building configuration... + + Current configuration: + ! + interface ens192 + ipv6 address 2001:DB8::2/64 + ipv6 ospf6 authentication keychain abhinay + + +Example keychain config: +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: frr + + frr#show running-config + Building configuration... + + Current configuration: + ! + key chain abcd + key 100 + key-string password + cryptographic-algorithm sha1 + exit + key 200 + key-string password + cryptographic-algorithm sha256 + exit + ! + key chain pqr + key 300 + key-string password + cryptographic-algorithm sha384 + exit + key 400 + key-string password + cryptographic-algorithm sha384 + exit + ! + +Show commands: +-------------- +There is an interface show command that displays if authentication trailer +is enabled or not. json output is also supported. + +There is support for drop counters, which will help in debugging the feature. + +.. code-block:: frr + + frr# show ipv6 ospf6 interface ens192 + ens192 is up, type BROADCAST + Interface ID: 5 + Number of I/F scoped LSAs is 2 + 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] + 0 Pending LSAs for LSAck in Time 00:00:00 [thread off] + Authentication trailer is enabled with manual key ==> new info added + Packet drop Tx 0, Packet drop Rx 0 + + +OSPFv3 supports options in hello and database description packets hence +the presence of authentication trailer needs to be stored in OSPFv3 +neighbor info. Since RFC specifies that we need to handled sequence number +for every ospf6 packet type, sequence number recvd in authentication header +from the neighbor is stored in neighbor to validate the packet. +json output is also supported. + +.. code-block:: frr + + frr# show ipv6 ospf6 neighbor 2.2.2.2 detail + Neighbor 2.2.2.2%ens192 + Area 1 via interface ens192 (ifindex 3) + 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] + 0 Pending LSAs for LSAck in Time 00:00:00 [thread off] + Authentication header present ==> new info added + hello DBDesc LSReq LSUpd LSAck + Higher sequence no 0x0 0x0 0x0 0x0 0x0 + Lower sequence no 0x242E 0x1DC4 0x1DC3 0x23CC 0x1DDA + +Sent packet sequence number is maintained per ospf6 router for every packet +that is sent out of router, so sequence number is maintained per ospf6 process. + +.. code-block:: frr + + frr# show ipv6 ospf6 + OSPFv3 Routing Process (0) with Router-ID 2.2.2.2 + Number of areas in this router is 1 + Authentication Sequence number info + Higher sequence no 3, Lower sequence no 1656 + +Debug command: +-------------- +Below command can be used to enable ospfv3 authentication trailer +specific logs if you have to debug the feature. + +.. clicmd:: debug ospf6 authentication [<tx|rx>] + +Feature supports authentication trailer tx/rx drop counters for debugging, +which can be used to see if packets are getting dropped due to error in +processing authentication trailer information in OSPFv3 packet. +json output is also supported. + +.. code-block:: frr + + frr# show ipv6 ospf6 interface ens192 + ens192 is up, type BROADCAST + Interface ID: 5 + Number of I/F scoped LSAs is 2 + 0 Pending LSAs for LSUpdate in Time 00:00:00 [thread off] + 0 Pending LSAs for LSAck in Time 00:00:00 [thread off] + Authentication trailer is enabled with manual key + Packet drop Tx 0, Packet drop Rx 0 ==> new counters + +Clear command: +-------------- +Below command can be used to clear the tx/rx drop counters in interface. +Below command can be used to clear all ospfv3 interface or specific +interface by specifying the interface name. + +.. clicmd:: clear ipv6 ospf6 auth-counters interface [IFNAME] + + + +.. _showing-ospf6-information: + +Showing OSPF6 information +========================= + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] [json] + + Show information on a variety of general OSPFv3 and area state and + configuration information. JSON output can be obtained by appending 'json' + to the end of command. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] database [<detail|dump|internal>] [json] + + This command shows LSAs present in the LSDB. There are three view options. + These options helps in viewing all the parameters of the LSAs. JSON output + can be obtained by appending 'json' to the end of command. JSON option is + not applicable with 'dump' option. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] database <router|network|inter-prefix|inter-router|as-external|group-membership|type-7|link|intra-prefix> [json] + + These options filters out the LSA based on its type. The three views options + works here as well. JSON output can be obtained by appending 'json' to the + end of command. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] database adv-router A.B.C.D linkstate-id A.B.C.D [json] + + The LSAs additinally can also be filtered with the linkstate-id and + advertising-router fields. We can use the LSA type filter and views with + this command as well and visa-versa. JSON output can be obtained by + appending 'json' to the end of command. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] database self-originated [json] + + This command is used to filter the LSAs which are originated by the present + router. All the other filters are applicable here as well. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] interface [json] + + To see OSPF interface configuration like costs. JSON output can be + obtained by appending "json" in the end. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] neighbor [json] + + Shows state and chosen (Backup) DR of neighbor. JSON output can be + obtained by appending 'json' at the end. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] interface traffic [json] + + Shows counts of different packets that have been received and transmitted + by the interfaces. JSON output can be obtained by appending "json" at the + end. + +.. clicmd:: show ipv6 route ospf6 + + This command shows internal routing table. + +.. clicmd:: show ipv6 ospf6 zebra [json] + + Shows state about what is being redistributed between zebra and OSPF6. + JSON output can be obtained by appending "json" at the end. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] redistribute [json] + + Shows the routes which are redistributed by the router. JSON output can + be obtained by appending 'json' at the end. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] route [<intra-area|inter-area|external-1|external-2|X:X::X:X|X:X::X:X/M|detail|summary>] [json] + + This command displays the ospfv3 routing table as determined by the most + recent SPF calculations. Options are provided to view the different types + of routes. Other than the standard view there are two other options, detail + and summary. JSON output can be obtained by appending 'json' to the end of + command. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] route X:X::X:X/M match [detail] [json] + + The additional match option will match the given address to the destination + of the routes, and return the result accordingly. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] interface [IFNAME] prefix [detail|<X:X::X:X|X:X::X:X/M> [<match|detail>]] [json] + + This command shows the prefixes present in the interface routing table. + Interface name can also be given. JSON output can be obtained by appending + 'json' to the end of command. + +.. clicmd:: show ipv6 ospf6 [vrf <NAME|all>] spf tree [json] + + This commands shows the spf tree from the recent spf calculation with the + calling router as the root. If json is appended in the end, we can get the + tree in JSON format. Each area that the router belongs to has it's own + JSON object, with each router having "cost", "isLeafNode" and "children" as + arguments. + +.. clicmd:: show ipv6 ospf6 graceful-restart helper [detail] [json] + + This command shows the graceful-restart helper details including helper + configuration parameters. + +.. _ospf6-debugging: + +OSPFv3 Debugging +================ + +The following debug commands are supported: + +.. clicmd:: debug ospf6 abr + + Toggle OSPFv3 ABR debugging messages. + +.. clicmd:: debug ospf6 asbr + + Toggle OSPFv3 ASBR debugging messages. + +.. clicmd:: debug ospf6 border-routers {router-id [A.B.C.D] | area-id [A.B.C.D]} + + Toggle OSPFv3 border router debugging messages. This can be specified for a + router with specific Router-ID/Area-ID. + +.. clicmd:: debug ospf6 flooding + + Toggle OSPFv3 flooding debugging messages. + +.. clicmd:: debug ospf6 interface + + Toggle OSPFv3 interface related debugging messages. + +.. clicmd:: debug ospf6 lsa + + Toggle OSPFv3 Link State Advertisements debugging messages. + +.. clicmd:: debug ospf6 lsa aggregation + + Toggle OSPFv3 Link State Advertisements summarization debugging messages. + +.. clicmd:: debug ospf6 message + + Toggle OSPFv3 message exchange debugging messages. + +.. clicmd:: debug ospf6 neighbor + + Toggle OSPFv3 neighbor interaction debugging messages. + +.. clicmd:: debug ospf6 nssa + + Toggle OSPFv3 Not So Stubby Area (NSSA) debugging messages. + +.. clicmd:: debug ospf6 route + + Toggle OSPFv3 routes debugging messages. + +.. clicmd:: debug ospf6 spf + + Toggle OSPFv3 Shortest Path calculation debugging messages. + +.. clicmd:: debug ospf6 zebra + + Toggle OSPFv3 zebra interaction debugging messages. + +.. clicmd:: debug ospf6 graceful-restart + + Toggle OSPFv3 graceful-restart helper debugging messages. + +Sample configuration +==================== + +Example of ospf6d configured on one interface and area: + +.. code-block:: frr + + interface eth0 + ipv6 ospf6 area 0.0.0.0 + ipv6 ospf6 instance-id 0 + ! + router ospf6 + ospf6 router-id 212.17.55.53 + area 0.0.0.0 range 2001:770:105:2::/64 + ! + + +Larger example with policy and various options set: + + +.. code-block:: frr + + debug ospf6 neighbor state + ! + interface fxp0 + ipv6 ospf6 area 0.0.0.0 + ipv6 ospf6 cost 1 + ipv6 ospf6 hello-interval 10 + ipv6 ospf6 dead-interval 40 + ipv6 ospf6 retransmit-interval 5 + ipv6 ospf6 priority 0 + ipv6 ospf6 transmit-delay 1 + ipv6 ospf6 instance-id 0 + ! + interface lo0 + ipv6 ospf6 cost 1 + ipv6 ospf6 hello-interval 10 + ipv6 ospf6 dead-interval 40 + ipv6 ospf6 retransmit-interval 5 + ipv6 ospf6 priority 1 + ipv6 ospf6 transmit-delay 1 + ipv6 ospf6 instance-id 0 + ! + router ospf6 + router-id 255.1.1.1 + redistribute static route-map static-ospf6 + ! + access-list access4 permit 127.0.0.1/32 + ! + ipv6 access-list access6 permit 3ffe:501::/32 + ipv6 access-list access6 permit 2001:200::/48 + ipv6 access-list access6 permit ::1/128 + ! + ipv6 prefix-list test-prefix seq 1000 deny any + ! + route-map static-ospf6 permit 10 + match ipv6 address prefix-list test-prefix + set metric-type type-2 + set metric 2000 + ! + line vty + access-class access4 + ipv6 access-class access6 + exec-timeout 0 0 + ! diff --git a/doc/user/ospf_fundamentals.rst b/doc/user/ospf_fundamentals.rst new file mode 100644 index 0000000..c566059 --- /dev/null +++ b/doc/user/ospf_fundamentals.rst @@ -0,0 +1,558 @@ +.. _ospf-fundamentals: + +OSPF Fundamentals +================= + +.. index:: + pair: Link-state routing protocol; OSPF + pair: Distance-vector routing protocol; OSPF + + +:abbr:`OSPF` is, mostly, a link-state routing protocol. In contrast to +:term:`distance-vector` protocols, such as :abbr:`RIP` or :abbr:`BGP`, where +routers describe available `paths` (i.e. routes) to each other, in +:term:`link-state` protocols routers instead describe the state of their links +to their immediate neighbouring routers. + +.. index:: + single: Link State Announcement + single: Link State Advertisement + single: LSA flooding + single: Link State Database + + +Each router describes their link-state information in a message known as an +:abbr:`LSA (Link State Advertisement)`, which is then propagated through to all +other routers in a link-state routing domain, by a process called `flooding`. +Each router thus builds up an :abbr:`LSDB (Link State Database)` of all the +link-state messages. From this collection of LSAs in the LSDB, each router can +then calculate the shortest path to any other router, based on some common +metric, by using an algorithm such as +`Edsger Dijkstra's <http://www.cs.utexas.edu/users/EWD/>`_ +:abbr:`SPF (Shortest Path First)` algorithm. + +.. index:: + pair: Link-state routing protocol; advantages + +By describing connectivity of a network in this way, in terms of +routers and links rather than in terms of the paths through a network, +a link-state protocol can use less bandwidth and converge more quickly +than other protocols. A link-state protocol need distribute only one +link-state message throughout the link-state domain when a link on any +single given router changes state, in order for all routers to +reconverge on the best paths through the network. In contrast, distance +vector protocols can require a progression of different path update +messages from a series of different routers in order to converge. + +.. index:: + pair: Link-state routing protocol; disadvantages + +The disadvantage to a link-state protocol is that the process of +computing the best paths can be relatively intensive when compared to +distance-vector protocols, in which near to no computation need be done +other than (potentially) select between multiple routes. This overhead +is mostly negligible for modern embedded CPUs, even for networks with +thousands of nodes. The primary scaling overhead lies more in coping +with the ever greater frequency of LSA updates as the size of a +link-state area increases, in managing the :abbr:`LSDB` and required +flooding. + +This section aims to give a distilled, but accurate, description of the +more important workings of :abbr:`OSPF` which an administrator may need +to know to be able best configure and trouble-shoot :abbr:`OSPF`. + +OSPF Mechanisms +--------------- + +:abbr:`OSPF` defines a range of mechanisms, concerned with detecting, +describing and propagating state through a network. These mechanisms +will nearly all be covered in greater detail further on. They may be +broadly classed as: + + +.. index:: + pair: Hello protocol; OSPF + +The Hello Protocol +^^^^^^^^^^^^^^^^^^ + +The OSPF Hello protocol allows OSPF to quickly detect changes in two-way +reachability between routers on a link. OSPF can additionally avail of other +sources of reachability information, such as link-state information provided by +hardware, or through dedicated reachability protocols such as +:abbr:`BFD (Bidirectional Forwarding Detection)`. + +OSPF also uses the Hello protocol to propagate certain state between routers +sharing a link, for example: + +- Hello protocol configured state, such as the dead-interval. +- Router priority, for DR/BDR election. +- DR/BDR election results. +- Any optional capabilities supported by each router. + +The Hello protocol is comparatively trivial and will not be explored in more +detail. + + +.. index:: + pair: LSA; OSPF + +.. _ospf-lsas: + +LSAs +^^^^ + +At the heart of :abbr:`OSPF` are :abbr:`LSA (Link State Advertisement)` +messages. Despite the name, some :abbr:`LSA` s do not, strictly speaking, +describe link-state information. Common :abbr:`LSA` s describe information +such as: + +- Routers, in terms of their links. +- Networks, in terms of attached routers. +- Routes, external to a link-state domain: + + External Routes + Routes entirely external to :abbr:`OSPF`. Routers originating such + routes are known as :abbr:`ASBR (Autonomous-System Border Router)` + routers. + + Summary Routes + Routes which summarise routing information relating to OSPF areas + external to the OSPF link-state area at hand, originated by + :abbr:`ABR (Area Boundary Router)` routers. + +.. _ospf-lsa-flooding: + +LSA Flooding +"""""""""""" + +OSPF defines several related mechanisms, used to manage synchronisation of +:abbr:`LSDB` s between neighbours as neighbours form adjacencies and the +propagation, or `flooding` of new or updated :abbr:`LSA` s. + + +.. index:: + pair: Area; OSPF + +.. _ospf-areas: + +Areas +^^^^^ + +OSPF provides for the protocol to be broken up into multiple smaller and +independent link-state areas. Each area must be connected to a common backbone +area by an :abbr:`ABR (Area Boundary Router)`. These :abbr:`ABR` routers are +responsible for summarising the link-state routing information of an area into +`Summary LSAs`, possibly in a condensed (i.e. aggregated) form, and then +originating these summaries into all other areas the :abbr:`ABR` is connected +to. + +Note that only summaries and external routes are passed between areas. As +these describe *paths*, rather than any router link-states, routing between +areas hence is by :term:`distance-vector`, **not** link-state. + +OSPF LSAs +--------- + +The core objects in OSPF are :abbr:`LSA` s. Everything else in OSPF revolves +around detecting what to describe in LSAs, when to update them, how to flood +them throughout a network and how to calculate routes from them. + +There are a variety of different :abbr:`LSA` s, for purposes such as describing +actual link-state information, describing paths (i.e. routes), describing +bandwidth usage of links for :abbr:`TE (Traffic Engineering)` purposes, and +even arbitrary data by way of *Opaque* :abbr:`LSA` s. + +LSA Header +^^^^^^^^^^ + +All LSAs share a common header with the following information: + +- Type + + Different types of :abbr:`LSA` s describe different things in + :abbr:`OSPF`. Types include: + + - Router LSA + - Network LSA + - Network Summary LSA + - Router Summary LSA + - AS-External LSA + + The specifics of the different types of LSA are examined below. + +- Advertising Router + + The Router ID of the router originating the LSA. + +.. seealso:: + + :clicmd:`ospf router-id A.B.C.D`. + +- LSA ID + + The ID of the LSA, which is typically derived in some way from the + information the LSA describes, e.g. a Router LSA uses the Router ID as + the LSA ID, a Network LSA will have the IP address of the :abbr:`DR` + as its LSA ID. + + The combination of the Type, ID and Advertising Router ID must uniquely + identify the :abbr:`LSA`. There can however be multiple instances of + an LSA with the same Type, LSA ID and Advertising Router ID, see + :ref:`sequence number <ospf-lsa-sequence-number>`. + +- Age + + A number to allow stale :abbr:`LSA` s to, eventually, be purged by routers + from their :abbr:`LSDB` s. + + The value nominally is one of seconds. An age of 3600, i.e. 1 hour, is + called the `MaxAge`. MaxAge LSAs are ignored in routing + calculations. LSAs must be periodically refreshed by their Advertising + Router before reaching MaxAge if they are to remain valid. + + Routers may deliberately flood LSAs with the age artificially set to + 3600 to indicate an LSA is no longer valid. This is called + `flushing` of an LSA. + + It is not abnormal to see stale LSAs in the LSDB, this can occur where + a router has shutdown without flushing its LSA(s), e.g. where it has + become disconnected from the network. Such LSAs do little harm. + +.. _ospf-lsa-sequence-number: + +- Sequence Number + + A number used to distinguish newer instances of an LSA from older instances. + +Link-State LSAs +^^^^^^^^^^^^^^^ + +Of all the various kinds of :abbr:`LSA` s, just two types comprise the +actual link-state part of :abbr:`OSPF`, Router :abbr:`LSA` s and +Network :abbr:`LSA` s. These LSA types are absolutely core to the +protocol. + +Instances of these LSAs are specific to the link-state area in which +they are originated. Routes calculated from these two LSA types are +called `intra-area routes`. + +- Router LSA + + Each OSPF Router must originate a router :abbr:`LSA` to describe + itself. In it, the router lists each of its :abbr:`OSPF` enabled + interfaces, for the given link-state area, in terms of: + + Cost + The output cost of that interface, scaled inversely to some commonly known + reference value, :clicmd:`auto-cost reference-bandwidth (1-4294967)`. + + Link Type + Transit Network + + A link to a multi-access network, on which the router has at least one + Full adjacency with another router. + + :abbr:`PtP (Point-to-Point)` + A link to a single remote router, with a Full adjacency. No + :abbr:`DR (Designated Router)` is elected on such links; no network + LSA is originated for such a link. + + Stub + A link with no adjacent neighbours, or a host route. + + - Link ID and Data + + These values depend on the Link Type: + + +----------------+-----------------------------------+------------------------------------------+ + | Link Type | Link ID | Link Data | + +================+===================================+==========================================+ + | Transit | Link IP address of the :abbr:`DR` | Interface IP address | + +----------------+-----------------------------------+------------------------------------------+ + | Point-to-Point | Router ID of the remote router | Local interface IP address, or the | + | | | :abbr:`ifindex (MIB-II interface index)` | + | | | for unnumbered links | + +----------------+-----------------------------------+------------------------------------------+ + | Stub | IP address | Subnet Mask | + +----------------+-----------------------------------+------------------------------------------+ + + Links on a router may be listed multiple times in the Router LSA, e.g. a + :abbr:`PtP` interface on which OSPF is enabled must *always* be described + by a Stub link in the Router :abbr:`LSA`, in addition to being listed as + PtP link in the Router :abbr:`LSA` if the adjacency with the remote router + is Full. + + Stub links may also be used as a way to describe links on which OSPF is + *not* spoken, known as `passive interfaces`, see + :clicmd:`ip ospf passive [A.B.C.D]`. + +- Network LSA + + On multi-access links (e.g. ethernets, certain kinds of ATM and X.25 + configurations), routers elect a :abbr:`DR`. The :abbr:`DR` is + responsible for originating a Network :abbr:`LSA`, which helps reduce + the information needed to describe multi-access networks with multiple + routers attached. The :abbr:`DR` also acts as a hub for the flooding of + :abbr:`LSA` s on that link, thus reducing flooding overheads. + + The contents of the Network LSA describes the: + + - Subnet Mask + + As the :abbr:`LSA` ID of a Network LSA must be the IP address of the + :abbr:`DR`, the Subnet Mask together with the :abbr:`LSA` ID gives + you the network address. + + - Attached Routers + + Each router fully-adjacent with the :abbr:`DR` is listed in the LSA, + by their Router-ID. This allows the corresponding Router :abbr:`LSA` s to be + easily retrieved from the :abbr:`LSDB`. + +Summary of Link State LSAs: + ++-------------+----------------------------+--------------------------------------------+ +| LSA Type | LSA ID | LSA Data Describes | ++=============+============================+============================================+ +| Router LSA | Router ID | The :abbr:`OSPF` enabled links of the | +| | | router, within a specific link-state area. | ++-------------+----------------------------+--------------------------------------------+ +| Network LSA | The IP address of the | The subnet mask of the network and the | +| | :abbr:`DR` for the network | Router IDs of all routers on the network | ++-------------+----------------------------+--------------------------------------------+ + +With an LSDB composed of just these two types of :abbr:`LSA`, it is +possible to construct a directed graph of the connectivity between all +routers and networks in a given OSPF link-state area. So, not +surprisingly, when OSPF routers build updated routing tables, the first +stage of :abbr:`SPF` calculation concerns itself only with these two +LSA types. + +.. _ospf-link-state-lsa-examples: + +Link-State LSA Examples +^^^^^^^^^^^^^^^^^^^^^^^ + +The example below shows two :abbr:`LSA` s, both originated by the same router +(Router ID 192.168.0.49) and with the same :abbr:`LSA` ID (192.168.0.49), but +of different LSA types. + +The first LSA being the router LSA describing 192.168.0.49's links: 2 links +to multi-access networks with fully-adjacent neighbours (i.e. Transit +links) and 1 being a Stub link (no adjacent neighbours). + +The second LSA being a Network LSA, for which 192.168.0.49 is the +:abbr:`DR`, listing the Router IDs of 4 routers on that network which +are fully adjacent with 192.168.0.49. + +:: + + # show ip ospf database router 192.168.0.49 + + OSPF Router with ID (192.168.0.53) + + Router Link States (Area 0.0.0.0) + + LS age: 38 + Options: 0x2 : *|-|-|-|-|-|E|* + LS Flags: 0x6 + Flags: 0x2 : ASBR + LS Type: router-LSA + Link State ID: 192.168.0.49 + Advertising Router: 192.168.0.49 + LS Seq Number: 80000f90 + Checksum: 0x518b + Length: 60 + Number of Links: 3 + + Link connected to: a Transit Network + (Link ID) Designated Router address: 192.168.1.3 + (Link Data) Router Interface address: 192.168.1.3 + Number of TOS metrics: 0 + TOS 0 Metric: 10 + + Link connected to: a Transit Network + (Link ID) Designated Router address: 192.168.0.49 + (Link Data) Router Interface address: 192.168.0.49 + Number of TOS metrics: 0 + TOS 0 Metric: 10 + + Link connected to: Stub Network + (Link ID) Net: 192.168.3.190 + (Link Data) Network Mask: 255.255.255.255 + Number of TOS metrics: 0 + TOS 0 Metric: 39063 + # show ip ospf database network 192.168.0.49 + + OSPF Router with ID (192.168.0.53) + + Net Link States (Area 0.0.0.0) + + LS age: 285 + Options: 0x2 : *|-|-|-|-|-|E|* + LS Flags: 0x6 + LS Type: network-LSA + Link State ID: 192.168.0.49 (address of Designated Router) + Advertising Router: 192.168.0.49 + LS Seq Number: 80000074 + Checksum: 0x0103 + Length: 40 + Network Mask: /29 + Attached Router: 192.168.0.49 + Attached Router: 192.168.0.52 + Attached Router: 192.168.0.53 + Attached Router: 192.168.0.54 + + +Note that from one LSA, you can find the other. E.g. Given the +Network-LSA you have a list of Router IDs on that network, from which +you can then look up, in the local :abbr:`LSDB`, the matching Router +LSA. From that Router-LSA you may (potentially) find links to other +Transit networks and Routers IDs which can be used to lookup the +corresponding Router or Network LSA. And in that fashion, one can find +all the Routers and Networks reachable from that starting :abbr:`LSA`. + +Given the Router LSA instead, you have the IP address of the +:abbr:`DR` of any attached transit links. Network LSAs will have that IP +as their LSA ID, so you can then look up that Network LSA and from that +find all the attached routers on that link, leading potentially to more +links and Network and Router LSAs, etc. etc. + +From just the above two :abbr:`LSA` s, one can already see the +following partial topology: + +:: + + ------------------------ Network: ...... + | Designated Router IP: 192.168.1.3 + | + IP: 192.168.1.3 + (transit link) + (cost: 10) + Router ID: 192.168.0.49(stub)---------- IP: 192.168.3.190/32 + (cost: 10) (cost: 39063) + (transit link) + IP: 192.168.0.49 + | + | + ------------------------------ Network: 192.168.0.48/29 + | | | Designated Router IP: 192.168.0.49 + | | | + | | Router ID: 192.168.0.54 + | | + | Router ID: 192.168.0.53 + | + Router ID: 192.168.0.52 + + +Note the Router IDs, though they look like IP addresses and often are +IP addresses, are not strictly speaking IP addresses, nor need they be +reachable addresses (though, OSPF will calculate routes to Router IDs). + +External LSAs +^^^^^^^^^^^^^ + +External, or "Type 5", :abbr:`LSA` s describe routing information which is +entirely external to :abbr:`OSPF`, and is "injected" into +:abbr:`OSPF`. Such routing information may have come from another +routing protocol, such as RIP or BGP, they may represent static routes +or they may represent a default route. + +An :abbr:`OSPF` router which originates External :abbr:`LSA` s is known as an +:abbr:`ASBR (AS Boundary Router)`. Unlike the link-state :abbr:`LSA` s, and +most other :abbr:`LSA` s, which are flooded only within the area in +which they originate, External :abbr:`LSA` s are flooded through-out +the :abbr:`OSPF` network to all areas capable of carrying External +:abbr:`LSA` s (:ref:`ospf-areas`). + +Routes internal to OSPF (intra-area or inter-area) are always preferred +over external routes. + +The External :abbr:`LSA` describes the following: + +IP Network number + The IP Network number of the route is described by the :abbr:`LSA` ID field. + +IP Network Mask + The body of the External LSA describes the IP Network Mask of the route. + This, together with the :abbr:`LSA` ID, describes the prefix of the IP route + concerned. + +Metric + The cost of the External Route. This cost may be an OSPF cost (also known as + a "Type 1" metric), i.e. equivalent to the normal OSPF costs, or an + externally derived cost ("Type 2" metric) which is not comparable to OSPF + costs and always considered larger than any OSPF cost. Where there are both + Type 1 and 2 External routes for a route, the Type 1 is always preferred. + +Forwarding Address + The address of the router to forward packets to for the route. This may be, + and usually is, left as 0 to specify that the ASBR originating the External + :abbr:`LSA` should be used. There must be an internal OSPF route to the + forwarding address, for the forwarding address to be usable. + +Tag + An arbitrary 4-bytes of data, not interpreted by OSPF, which may carry + whatever information about the route which OSPF speakers desire. + +AS External LSA Example +^^^^^^^^^^^^^^^^^^^^^^^ + +To illustrate, below is an example of an External :abbr:`LSA` in the +:abbr:`LSDB` of an OSPF router. It describes a route to the IP prefix of +192.168.165.0/24, originated by the ASBR with Router-ID 192.168.0.49. The +metric of 20 is external to OSPF. The forwarding address is 0, so the route +should forward to the originating ASBR if selected. + +:: + + # show ip ospf database external 192.168.165.0 + LS age: 995 + Options: 0x2 : *|-|-|-|-|-|E|* + LS Flags: 0x9 + LS Type: AS-external-LSA + Link State ID: 192.168.165.0 (External Network Number) + Advertising Router: 192.168.0.49 + LS Seq Number: 800001d8 + Checksum: 0xea27 + Length: 36 + Network Mask: /24 + Metric Type: 2 (Larger than any link state path) + TOS: 0 + Metric: 20 + Forward Address: 0.0.0.0 + External Route Tag: 0 + + +We can add this to our partial topology from above, which now looks +like::: + + --------------------- Network: ...... + | Designated Router IP: 192.168.1.3 + | + IP: 192.168.1.3 /---- External route: 192.168.165.0/24 + (transit link) / Cost: 20 (External metric) + (cost: 10) / + Router ID: 192.168.0.49(stub)---------- IP: 192.168.3.190/32 + (cost: 10) (cost: 39063) + (transit link) + IP: 192.168.0.49 + | + | + ------------------------------ Network: 192.168.0.48/29 + | | | Designated Router IP: 192.168.0.49 + | | | + | | Router ID: 192.168.0.54 + | | + | Router ID: 192.168.0.53 + | + Router ID: 192.168.0.52 + + +Summary LSAs +^^^^^^^^^^^^ + +Summary LSAs are created by :abbr:`ABR` s to summarise the destinations +available within one area to other areas. These LSAs may describe IP networks, +potentially in aggregated form, or :abbr:`ASBR` routers. diff --git a/doc/user/ospfd.rst b/doc/user/ospfd.rst new file mode 100644 index 0000000..d61522b --- /dev/null +++ b/doc/user/ospfd.rst @@ -0,0 +1,1436 @@ +.. _ospfv2: + +****** +OSPFv2 +****** + +:abbr:`OSPF (Open Shortest Path First)` version 2 is a routing protocol which +is described in :rfc:`2328`. OSPF is an :abbr:`IGP (Interior Gateway +Protocol)`. Compared with :abbr:`RIP`, :abbr:`OSPF` can provide scalable +network support and faster convergence times. OSPF is widely used in large +networks such as :abbr:`ISP (Internet Service Provider)` backbone and +enterprise networks. + +.. include:: ospf_fundamentals.rst + +.. _configuring-ospfd: + +Configuring OSPF +================ + +*ospfd* accepts all :ref:`common-invocation-options`. + +.. option:: -n, --instance + + Specify the instance number for this invocation of *ospfd*. + +.. option:: -a, --apiserver + + Enable the OSPF API server. This is required to use ``ospfclient``. + +*ospfd* must acquire interface information from *zebra* in order to function. +Therefore *zebra* must be running before invoking *ospfd*. Also, if *zebra* is +restarted then *ospfd* must be too. + +Like other daemons, *ospfd* configuration is done in :abbr:`OSPF` specific +configuration file :file:`ospfd.conf` when the integrated config is not used. + +.. _ospf-multi-instance: + +Multi-instance Support +---------------------- + +OSPF supports multiple instances. Each instance is identified by a positive +nonzero integer that must be provided when adding configuration items specific +to that instance. Enabling instances is done with :file:`/etc/frr/daemons` in +the following manner: + +:: + + ... + ospfd=yes + ospfd_instances=1,5,6 + ... + +The ``ospfd_instances`` variable controls which instances are started and what +their IDs are. In this example, after starting FRR you should see the following +processes: + +.. code-block:: shell + + # ps -ef | grep "ospfd" + frr 11816 1 0 17:30 ? 00:00:00 /usr/lib/frr/ospfd --daemon -A 127.0.0.1 -n 1 + frr 11822 1 0 17:30 ? 00:00:00 /usr/lib/frr/ospfd --daemon -A 127.0.0.1 -n 2 + frr 11828 1 0 17:30 ? 00:00:00 /usr/lib/frr/ospfd --daemon -A 127.0.0.1 -n 3 + + +The instance number should be specified in the config when addressing a particular instance: + +.. code-block:: frr + + router ospf 5 + ospf router-id 1.2.3.4 + area 0.0.0.0 authentication message-digest + ... + +.. _ospf-router: + +Routers +------- + +To start OSPF process you have to specify the OSPF router. + +.. clicmd:: router ospf [{(1-65535)|vrf NAME}] + + + Enable or disable the OSPF process. + + Multiple instances don't support `vrf NAME`. + +.. clicmd:: ospf router-id A.B.C.D + + + This sets the router-ID of the OSPF process. The router-ID may be an IP + address of the router, but need not be - it can be any arbitrary 32bit + number. However it MUST be unique within the entire OSPF domain to the OSPF + speaker - bad things will happen if multiple OSPF speakers are configured + with the same router-ID! If one is not specified then *ospfd* will obtain a + router-ID automatically from *zebra*. + +.. clicmd:: ospf abr-type TYPE + + + `type` can be cisco|ibm|shortcut|standard. The "Cisco" and "IBM" types + are equivalent. + + The OSPF standard for ABR behaviour does not allow an ABR to consider + routes through non-backbone areas when its links to the backbone are + down, even when there are other ABRs in attached non-backbone areas + which still can reach the backbone - this restriction exists primarily + to ensure routing-loops are avoided. + + With the "Cisco" or "IBM" ABR type, the default in this release of FRR, this + restriction is lifted, allowing an ABR to consider summaries learned from + other ABRs through non-backbone areas, and hence route via non-backbone + areas as a last resort when, and only when, backbone links are down. + + Note that areas with fully-adjacent virtual-links are considered to be + "transit capable" and can always be used to route backbone traffic, and + hence are unaffected by this setting (:clicmd:`area A.B.C.D virtual-link A.B.C.D`). + + More information regarding the behaviour controlled by this command can + be found in :rfc:`3509`, and :t:`draft-ietf-ospf-shortcut-abr-02.txt`. + + Quote: "Though the definition of the :abbr:`ABR (Area Border Router)` + in the OSPF specification does not require a router with multiple + attached areas to have a backbone connection, it is actually + necessary to provide successful routing to the inter-area and + external destinations. If this requirement is not met, all traffic + destined for the areas not connected to such an ABR or out of the + OSPF domain, is dropped. This document describes alternative ABR + behaviors implemented in Cisco and IBM routers." + +.. clicmd:: ospf rfc1583compatibility + + + :rfc:`2328`, the successor to :rfc:`1583`, suggests according + to section G.2 (changes) in section 16.4 a change to the path + preference algorithm that prevents possible routing loops that were + possible in the old version of OSPFv2. More specifically it demands + that inter-area paths and intra-area backbone path are now of equal preference + but still both preferred to external paths. + + This command should NOT be set normally. + +.. clicmd:: log-adjacency-changes [detail] + + + Configures ospfd to log changes in adjacency. With the optional + detail argument, all changes in adjacency status are shown. Without detail, + only changes to full or regressions are shown. + +.. clicmd:: passive-interface default + + Make all interfaces that belong to this router passive by default. For the + description of passive interface look at :clicmd:`ip ospf passive [A.B.C.D]`. + Per-interface configuration takes precedence over the default value. + +.. clicmd:: timers throttle spf (0-600000) (0-600000) (0-600000) + + This command sets the initial `delay`, the `initial-holdtime` + and the `maximum-holdtime` between when SPF is calculated and the + event which triggered the calculation. The times are specified in + milliseconds and must be in the range of 0 to 600000 milliseconds. + + The `delay` specifies the minimum amount of time to delay SPF + calculation (hence it affects how long SPF calculation is delayed after + an event which occurs outside of the holdtime of any previous SPF + calculation, and also serves as a minimum holdtime). + + Consecutive SPF calculations will always be separated by at least + 'hold-time' milliseconds. The hold-time is adaptive and initially is + set to the `initial-holdtime` configured with the above command. + Events which occur within the holdtime of the previous SPF calculation + will cause the holdtime to be increased by `initial-holdtime`, bounded + by the `maximum-holdtime` configured with this command. If the adaptive + hold-time elapses without any SPF-triggering event occurring then + the current holdtime is reset to the `initial-holdtime`. The current + holdtime can be viewed with :clicmd:`show ip ospf`, where it is expressed as + a multiplier of the `initial-holdtime`. + + .. code-block:: frr + + router ospf + timers throttle spf 200 400 10000 + + + In this example, the `delay` is set to 200ms, the initial holdtime is set to + 400ms and the `maximum holdtime` to 10s. Hence there will always be at least + 200ms between an event which requires SPF calculation and the actual SPF + calculation. Further consecutive SPF calculations will always be separated + by between 400ms to 10s, the hold-time increasing by 400ms each time an + SPF-triggering event occurs within the hold-time of the previous SPF + calculation. + + This command supersedes the *timers spf* command in previous FRR + releases. + +.. clicmd:: max-metric router-lsa [on-startup (5-86400)|on-shutdown (5-100)] + +.. clicmd:: max-metric router-lsa administrative + + + This enables :rfc:`3137` support, where the OSPF process describes its + transit links in its router-LSA as having infinite distance so that other + routers will avoid calculating transit paths through the router while still + being able to reach networks through the router. + + This support may be enabled administratively (and indefinitely) or + conditionally. Conditional enabling of max-metric router-lsas can be for a + period of seconds after startup and/or for a period of seconds prior to + shutdown. + + Enabling this for a period after startup allows OSPF to converge fully first + without affecting any existing routes used by other routers, while still + allowing any connected stub links and/or redistributed routes to be + reachable. Enabling this for a period of time in advance of shutdown allows + the router to gracefully excuse itself from the OSPF domain. + + Enabling this feature administratively allows for administrative + intervention for whatever reason, for an indefinite period of time. Note + that if the configuration is written to file, this administrative form of + the stub-router command will also be written to file. If *ospfd* is + restarted later, the command will then take effect until manually + deconfigured. + + Configured state of this feature as well as current status, such as the + number of second remaining till on-startup or on-shutdown ends, can be + viewed with the :clicmd:`show ip ospf` command. + +.. clicmd:: auto-cost reference-bandwidth (1-4294967) + + + This sets the reference + bandwidth for cost calculations, where this bandwidth is considered + equivalent to an OSPF cost of 1, specified in Mbits/s. The default is + 100Mbit/s (i.e. a link of bandwidth 100Mbit/s or higher will have a + cost of 1. Cost of lower bandwidth links will be scaled with reference + to this cost). + + This configuration setting MUST be consistent across all routers within the + OSPF domain. + +.. clicmd:: network A.B.C.D/M area A.B.C.D + +.. clicmd:: network A.B.C.D/M area (0-4294967295) + + + + This command specifies the OSPF enabled interface(s). If the interface has + an address from range 192.168.1.0/24 then the command below enables ospf + on this interface so router can provide network information to the other + ospf routers via this interface. + + .. code-block:: frr + + router ospf + network 192.168.1.0/24 area 0.0.0.0 + + Prefix length in interface must be equal or bigger (i.e. smaller network) than + prefix length in network statement. For example statement above doesn't enable + ospf on interface with address 192.168.1.1/23, but it does on interface with + address 192.168.1.129/25. + + Note that the behavior when there is a peer address + defined on an interface changed after release 0.99.7. + Currently, if a peer prefix has been configured, + then we test whether the prefix in the network command contains + the destination prefix. Otherwise, we test whether the network command prefix + contains the local address prefix of the interface. + + It is also possible to enable OSPF on a per interface/subnet basis + using the interface command (:clicmd:`ip ospf area AREA [ADDR]`). + However, mixing both network commands (:clicmd:`network`) and interface + commands (:clicmd:`ip ospf`) on the same router is not supported. + +.. clicmd:: proactive-arp + + + This command enables or disables sending ARP requests to update neighbor + table entries. It speeds up convergence for /32 networks on a P2P + connection. + + This feature is enabled by default. + +.. clicmd:: clear ip ospf [(1-65535)] process + + This command can be used to clear the ospf process data structures. This + will clear the ospf neighborship as well and it will get re-established. + This will clear the LSDB too. This will be helpful when there is a change + in router-id and if user wants the router-id change to take effect, user can + use this cli instead of restarting the ospfd daemon. + +.. clicmd:: clear ip ospf [(1-65535)] neighbor + + This command can be used to clear the ospf neighbor data structures. This + will clear the ospf neighborship and it will get re-established. This + command can be used when the neighbor state get stuck at some state and + this can be used to recover it from that state. + +.. clicmd:: maximum-paths (1-64) + + Use this command to control the maximum number of equal cost paths to reach + a specific destination. The upper limit may differ if you change the value + of MULTIPATH_NUM during compilation. The default is MULTIPATH_NUM (64). + +.. clicmd:: write-multiplier (1-100) + + Use this command to tune the amount of work done in the packet read and + write threads before relinquishing control. The parameter is the number + of packets to process before returning. The defult value of this parameter + is 20. + +.. clicmd:: socket buffer <send | recv | all> (1-4000000000) + + This command controls the ospf instance's socket buffer sizes. The + 'no' form resets one or both values to the default. + +.. clicmd:: no socket-per-interface + + Ordinarily, ospfd uses a socket per interface for sending + packets. This command disables those per-interface sockets, and + causes ospfd to use a single socket per ospf instance for sending + and receiving packets. + +.. _ospf-area: + +Areas +----- + +.. clicmd:: area A.B.C.D range A.B.C.D/M [advertise [cost (0-16777215)]] + +.. clicmd:: area (0-4294967295) range A.B.C.D/M [advertise [cost (0-16777215)]] + + + + Summarize intra area paths from specified area into one Type-3 summary-LSA + announced to other areas. This command can be used only in ABR and ONLY + router-LSAs (Type-1) and network-LSAs (Type-2) (i.e. LSAs with scope area) can + be summarized. Type-5 AS-external-LSAs can't be summarized - their scope is AS. + + .. code-block:: frr + + router ospf + network 192.168.1.0/24 area 0.0.0.0 + network 10.0.0.0/8 area 0.0.0.10 + area 0.0.0.10 range 10.0.0.0/8 + + + With configuration above one Type-3 Summary-LSA with routing info 10.0.0.0/8 is + announced into backbone area if area 0.0.0.10 contains at least one intra-area + network (i.e. described with router or network LSA) from this range. + +.. clicmd:: area A.B.C.D range A.B.C.D/M not-advertise + +.. clicmd:: area (0-4294967295) range A.B.C.D/M not-advertise + + + Instead of summarizing intra area paths filter them - i.e. intra area paths from this + range are not advertised into other areas. + This command makes sense in ABR only. + +.. clicmd:: area A.B.C.D range A.B.C.D/M {substitute A.B.C.D/M|cost (0-16777215)} + +.. clicmd:: area (0-4294967295) range A.B.C.D/M {substitute A.B.C.D/M|cost (0-16777215)} + + + Substitute summarized prefix with another prefix. + + .. code-block:: frr + + router ospf + network 192.168.1.0/24 area 0.0.0.0 + network 10.0.0.0/8 area 0.0.0.10 + area 0.0.0.10 range 10.0.0.0/8 substitute 11.0.0.0/8 + + + One Type-3 summary-LSA with routing info 11.0.0.0/8 is announced into backbone area if + area 0.0.0.10 contains at least one intra-area network (i.e. described with router-LSA or + network-LSA) from range 10.0.0.0/8. + + By default, the metric of the summary route is calculated as the highest + metric among the summarized routes. The `cost` option, however, can be used + to set an explicit metric. + + This command makes sense in ABR only. + +.. clicmd:: area A.B.C.D virtual-link A.B.C.D + +.. clicmd:: area (0-4294967295) virtual-link A.B.C.D + + + +.. clicmd:: area A.B.C.D shortcut + +.. clicmd:: area (0-4294967295) shortcut + + + + Configure the area as Shortcut capable. See :rfc:`3509`. This requires + that the 'abr-type' be set to 'shortcut'. + +.. clicmd:: area A.B.C.D stub + +.. clicmd:: area (0-4294967295) stub + + + + Configure the area to be a stub area. That is, an area where no router + originates routes external to OSPF and hence an area where all external + routes are via the ABR(s). Hence, ABRs for such an area do not need + to pass AS-External LSAs (type-5s) or ASBR-Summary LSAs (type-4) into the + area. They need only pass Network-Summary (type-3) LSAs into such an area, + along with a default-route summary. + +.. clicmd:: area A.B.C.D stub no-summary + +.. clicmd:: area (0-4294967295) stub no-summary + + + + Prevents an *ospfd* ABR from injecting inter-area + summaries into the specified stub area. + +.. clicmd:: area A.B.C.D nssa + +.. clicmd:: area (0-4294967295) nssa + + Configure the area to be a NSSA (Not-So-Stubby Area). This is an area that + allows OSPF to import external routes into a stub area via a new LSA type + (type 7). An NSSA autonomous system boundary router (ASBR) will generate this + type of LSA. The area border router (ABR) translates the LSA type 7 into LSA + type 5, which is propagated into the OSPF domain. NSSA areas are defined in + RFC 3101. + +.. clicmd:: area A.B.C.D nssa suppress-fa + +.. clicmd:: area (0-4294967295) nssa suppress-fa + + Configure the router to set the forwarding address to 0.0.0.0 in all LSA type 5 + translated from LSA type 7. The router needs to be elected the translator of the + area for this command to take effect. This feature causes routers that are + configured not to advertise forwarding addresses into the backbone to direct + forwarded traffic to the NSSA ABR translator. + +.. clicmd:: area A.B.C.D nssa default-information-originate [metric-type (1-2)] [metric (0-16777214)] + +.. clicmd:: area (0-4294967295) nssa default-information-originate [metric-type (1-2)] [metric (0-16777214)] + + NSSA ABRs and ASBRs can be configured with the `default-information-originate` + option to originate a Type-7 default route into the NSSA area. In the case + of NSSA ASBRs, the origination of the default route is conditioned to the + existence of a default route in the RIB that wasn't learned via the OSPF + protocol. + +.. clicmd:: area A.B.C.D nssa range A.B.C.D/M [<not-advertise|cost (0-16777215)>] + +.. clicmd:: area (0-4294967295) nssa range A.B.C.D/M [<not-advertise|cost (0-16777215)>] + + Summarize a group of external subnets into a single Type-7 LSA, which is + then translated to a Type-5 LSA and avertised to the backbone. + This command can only be used at the area boundary (NSSA ABR router). + + By default, the metric of the summary route is calculated as the highest + metric among the summarized routes. The `cost` option, however, can be used + to set an explicit metric. + + The `not-advertise` option, when present, prevents the summary route from + being advertised, effectively filtering the summarized routes. + +.. clicmd:: area A.B.C.D default-cost (0-16777215) + + + Set the cost of default-summary LSAs announced to stubby areas. + +.. clicmd:: area A.B.C.D export-list NAME + +.. clicmd:: area (0-4294967295) export-list NAME + + + + Filter Type-3 summary-LSAs announced to other areas originated from intra- + area paths from specified area. + + .. code-block:: frr + + router ospf + network 192.168.1.0/24 area 0.0.0.0 + network 10.0.0.0/8 area 0.0.0.10 + area 0.0.0.10 export-list foo + ! + access-list foo permit 10.10.0.0/16 + access-list foo deny any + + With example above any intra-area paths from area 0.0.0.10 and from range + 10.10.0.0/16 (for example 10.10.1.0/24 and 10.10.2.128/30) are announced into + other areas as Type-3 summary-LSA's, but any others (for example 10.11.0.0/16 + or 10.128.30.16/30) aren't. + + This command is only relevant if the router is an ABR for the specified + area. + +.. clicmd:: area A.B.C.D import-list NAME + +.. clicmd:: area (0-4294967295) import-list NAME + + + + Same as export-list, but it applies to paths announced into specified area + as Type-3 summary-LSAs. + +.. clicmd:: area A.B.C.D filter-list prefix NAME in + +.. clicmd:: area A.B.C.D filter-list prefix NAME out + +.. clicmd:: area (0-4294967295) filter-list prefix NAME in + +.. clicmd:: area (0-4294967295) filter-list prefix NAME out + + + + + + Filtering Type-3 summary-LSAs to/from area using prefix lists. This command + makes sense in ABR only. + +.. clicmd:: area A.B.C.D authentication + +.. clicmd:: area (0-4294967295) authentication + + + + Specify that simple password authentication should be used for the given + area. + +.. clicmd:: area A.B.C.D authentication message-digest + +.. clicmd:: area (0-4294967295) authentication message-digest + + Specify that OSPF packets must be authenticated with MD5 HMACs within the + given area. Keying material must also be configured on a per-interface basis + (:clicmd:`ip ospf message-digest-key`). + + MD5 authentication may also be configured on a per-interface basis + (:clicmd:`ip ospf authentication message-digest`). Such per-interface + settings will override any per-area authentication setting. + +.. _ospf-interface: + +Interfaces +---------- + +.. clicmd:: ip ospf area AREA [ADDR] + + + Enable OSPF on the interface, optionally restricted to just the IP address + given by `ADDR`, putting it in the `AREA` area. If you have a lot of + interfaces, and/or a lot of subnets, then enabling OSPF via this command + instead of (:clicmd:`network A.B.C.D/M area A.B.C.D`) may result in a + slight performance improvement. + + Notice that, mixing both network commands (:clicmd:`network`) and interface + commands (:clicmd:`ip ospf`) on the same router is not supported. + If (:clicmd:`ip ospf`) is present, (:clicmd:`network`) commands will fail. + +.. clicmd:: ip ospf authentication-key AUTH_KEY + + + Set OSPF authentication key to a simple password. After setting `AUTH_KEY`, + all OSPF packets are authenticated. `AUTH_KEY` has length up to 8 chars. + + Simple text password authentication is insecure and deprecated in favour of + MD5 HMAC authentication. + +.. clicmd:: ip ospf authentication message-digest + + Specify that MD5 HMAC authentication must be used on this interface. MD5 + keying material must also be configured. Overrides any authentication + enabled on a per-area basis + (:clicmd:`area A.B.C.D authentication message-digest`) + + Note that OSPF MD5 authentication requires that time never go backwards + (correct time is NOT important, only that it never goes backwards), even + across resets, if ospfd is to be able to promptly reestablish adjacencies + with its neighbours after restarts/reboots. The host should have system time + be set at boot from an external or non-volatile source (e.g. battery backed + clock, NTP, etc.) or else the system clock should be periodically saved to + non-volatile storage and restored at boot if MD5 authentication is to be + expected to work reliably. + +.. clicmd:: ip ospf message-digest-key KEYID md5 KEY + + + Set OSPF authentication key to a cryptographic password. The cryptographic + algorithm is MD5. + + KEYID identifies secret key used to create the message digest. This ID is + part of the protocol and must be consistent across routers on a link. + + KEY is the actual message digest key, of up to 16 chars (larger strings will + be truncated), and is associated with the given KEYID. + +.. clicmd:: ip ospf authentication key-chain KEYCHAIN + + Specify that HMAC cryptographic authentication must be used on this interface + using a key chain. Overrides any authentication enabled on a per-area basis + (:clicmd:`area A.B.C.D authentication message-digest`). + + ``KEYCHAIN``: Specifies the name of the key chain that contains the authentication + key(s) and cryptographic algorithms to be used for OSPF authentication. The key chain + is a logical container that holds one or more authentication keys, + allowing for key rotation and management. + + Note that OSPF HMAC cryptographic authentication requires that time never go backwards + (correct time is NOT important, only that it never goes backwards), even + across resets, if ospfd is to be able to promptly reestablish adjacencies + with its neighbours after restarts/reboots. The host should have system time + be set at boot from an external or non-volatile source (e.g. battery backed + clock, NTP, etc.) or else the system clock should be periodically saved to + non-volatile storage and restored at boot if HMAC cryptographic authentication is to be + expected to work reliably. + + Example: + + .. code:: sh + + r1(config)#key chain temp + r1(config-keychain)#key 13 + r1(config-keychain-key)#key-string ospf + r1(config-keychain-key)#cryptographic-algorithm hmac-sha-256 + r1(config)#int eth0 + r1(config-if)#ip ospf authentication key-chain temp + r1(config-if)#ip ospf area 0 + +.. clicmd:: ip ospf cost (1-65535) + + + Set link cost for the specified interface. The cost value is set to + router-LSA's metric field and used for SPF calculation. + +.. clicmd:: ip ospf dead-interval (1-65535) + +.. clicmd:: ip ospf dead-interval minimal hello-multiplier (2-20) + + + Set number of seconds for RouterDeadInterval timer value used for Wait Timer + and Inactivity Timer. This value must be the same for all routers attached + to a common network. The default value is 40 seconds. + + If 'minimal' is specified instead, then the dead-interval is set to 1 second + and one must specify a hello-multiplier. The hello-multiplier specifies how + many Hellos to send per second, from 2 (every 500ms) to 20 (every 50ms). + Thus one can have 1s convergence time for OSPF. If this form is specified, + then the hello-interval advertised in Hello packets is set to 0 and the + hello-interval on received Hello packets is not checked, thus the + hello-multiplier need NOT be the same across multiple routers on a common + link. + +.. clicmd:: ip ospf hello-interval (1-65535) + + + Set number of seconds for HelloInterval timer value. Setting this value, + Hello packet will be sent every timer value seconds on the specified interface. + This value must be the same for all routers attached to a common network. + The default value is 10 seconds. + + This command has no effect if + :clicmd:`ip ospf dead-interval minimal hello-multiplier (2-20)` is also + specified for the interface. + +.. clicmd:: ip ospf graceful-restart hello-delay (1-1800) + + Set the length of time during which Grace-LSAs are sent at 1-second intervals + while coming back up after an unplanned outage. During this time, no hello + packets are sent. + + A higher hello delay will increase the chance that all neighbors are notified + about the ongoing graceful restart before receiving a hello packet (which is + crucial for the graceful restart to succeed). The hello delay shouldn't be set + too high, however, otherwise the adjacencies might time out. As a best practice, + it's recommended to set the hello delay and hello interval with the same values. + The default value is 10 seconds. + +.. clicmd:: ip ospf network (broadcast|non-broadcast|point-to-multipoint [delay-reflood]|point-to-point [dmvpn]) + + When configuring a point-to-point network on an interface and the interface + has a /32 address associated with then OSPF will treat the interface + as being `unnumbered`. If you are doing this you *must* set the + net.ipv4.conf.<interface name>.rp_filter value to 0. In order for + the ospf multicast packets to be delivered by the kernel. + + When used in a DMVPN network at a spoke, this OSPF will be configured in + point-to-point, but the HUB will be a point-to-multipoint. To make this + topology work, specify the optional 'dmvpn' parameter at the spoke. + + When the network is configured as point-to-multipoint and `delay-reflood` + is specified, LSAs received on the interface from neighbors on the + interface will not be flooded back out on the interface immediately. + Rather, they will be added to the neighbor's link state retransmission + list and only sent to the neighbor if the neighbor doesn't acknowledge + the LSA prior to the link state retransmission timer expiring. + + Set explicitly network type for specified interface. + +.. clicmd:: ip ospf priority (0-255) + + + Set RouterPriority integer value. The router with the highest priority will + be more eligible to become Designated Router. Setting the value to 0, makes + the router ineligible to become Designated Router. The default value is 1. + +.. clicmd:: ip ospf retransmit-interval (1-65535) + + + Set number of seconds for RxmtInterval timer value. This value is used when + retransmitting Database Description and Link State Request packets. The + default value is 5 seconds. + +.. clicmd:: ip ospf transmit-delay (1-65535) [A.B.C.D] + + + Set number of seconds for InfTransDelay value. LSAs' age should be + incremented by this value when transmitting. The default value is 1 second. + +.. clicmd:: ip ospf passive [A.B.C.D] + + Do not speak OSPF on the interface, but do advertise the interface as a stub + link in the router-:abbr:`LSA (Link State Advertisement)` for this router. + This allows one to advertise addresses on such connected interfaces without + having to originate AS-External/Type-5 LSAs (which have global flooding + scope) - as would occur if connected addresses were redistributed into + OSPF (:ref:`redistribute-routes-to-ospf`). This is the only way to + advertise non-OSPF links into stub areas. + +.. clicmd:: ip ospf prefix-suppression [A.B.C.D] + + Configure OSPF to not advertise the IPv4 prefix associated with the + OSPF interface. The associated IPv4 prefix will be omitted from an OSPF + router-LSA or advertised with a host mask in an OSPF network-LSA as + specified in RFC 6860, "Hiding Transit-Only Networks in OSPF". If an + optional IPv4 address is specified, the prefix suppression will apply + to the OSPF interface associated with the specified interface address. + +.. clicmd:: ip ospf area (A.B.C.D|(0-4294967295)) + + + Enable ospf on an interface and set associated area. + +OSPF route-map +============== + +Usage of *ospfd*'s route-map support. + +.. clicmd:: set metric [+|-](0-4294967295) + + Set a metric for matched route when sending announcement. Use plus (+) sign + to add a metric value to an existing metric. Use minus (-) sign to + substract a metric value from an existing metric. + +.. _redistribute-routes-to-ospf: + +Redistribution +-------------- + +.. _ospf-redistribute: + +.. clicmd:: redistribute <babel|bgp|connected|eigrp|isis|kernel|openfabric|ospf|rip|sharp|static|table> [metric-type (1-2)] [metric (0-16777214)] [route-map WORD] + + Redistribute routes of the specified protocol or kind into OSPF, with the + metric type and metric set if specified, filtering the routes using the + given route-map if specified. Redistributed routes may also be filtered + with distribute-lists, see + :ref:`ospf distribute-list configuration <ospf-distribute-list>`. + + Redistributed routes are distributed as into OSPF as Type-5 External LSAs + into links to areas that accept external routes, Type-7 External LSAs for + NSSA areas and are not redistributed at all into Stub areas, where external + routes are not permitted. + + Note that for connected routes, one may instead use the + :clicmd:`ip ospf passive [A.B.C.D]` configuration. + +.. clicmd:: default-information originate + +.. clicmd:: default-information originate metric (0-16777214) + +.. clicmd:: default-information originate metric (0-16777214) metric-type (1|2) + +.. clicmd:: default-information originate metric (0-16777214) metric-type (1|2) route-map WORD + +.. clicmd:: default-information originate always + +.. clicmd:: default-information originate always metric (0-16777214) + +.. clicmd:: default-information originate always metric (0-16777214) metric-type (1|2) + +.. clicmd:: default-information originate always metric (0-16777214) metric-type (1|2) route-map WORD + + + Originate an AS-External (type-5) LSA describing a default route into all + external-routing capable areas, of the specified metric and metric type. If + the 'always' keyword is given then the default is always advertised, even + when there is no default present in the routing table. + +.. _ospf-distribute-list: + +.. clicmd:: distribute-list NAME out <kernel|connected|static|rip|isis|bgp|eigrp|nhrp|table|vnc|babel|openfabric> + + Apply the access-list filter, NAME, to redistributed routes of the given + type before allowing the routes to be redistributed into OSPF + (:ref:`ospf redistribution <ospf-redistribute>`). + +.. clicmd:: default-metric (0-16777214) + + +.. clicmd:: distance (1-255) + + +.. clicmd:: distance ospf (intra-area|inter-area|external) (1-255) + + + +Graceful Restart +================ + +.. clicmd:: graceful-restart [grace-period (1-1800)] + + + Configure Graceful Restart (RFC 3623) restarting support. + When enabled, the default grace period is 120 seconds. + + To perform a graceful shutdown, the "graceful-restart prepare ip ospf" + EXEC-level command needs to be issued before restarting the ospfd daemon. + + When Graceful Restart is enabled and the ospfd daemon crashes or is killed + abruptely (e.g. SIGKILL), it will attempt an unplanned Graceful Restart once + it restarts. + +.. clicmd:: graceful-restart helper enable [A.B.C.D] + + + Configure Graceful Restart (RFC 3623) helper support. + By default, helper support is disabled for all neighbours. + This config enables/disables helper support on this router + for all neighbours. + To enable/disable helper support for a specific + neighbour, the router-id (A.B.C.D) has to be specified. + +.. clicmd:: graceful-restart helper strict-lsa-checking + + + If 'strict-lsa-checking' is configured then the helper will + abort the Graceful Restart when a LSA change occurs which + affects the restarting router. + By default 'strict-lsa-checking' is enabled" + +.. clicmd:: graceful-restart helper supported-grace-time + + + Supports as HELPER for configured grace period. + +.. clicmd:: graceful-restart helper planned-only + + + It helps to support as HELPER only for planned + restarts. By default, it supports both planned and + unplanned outages. + + +.. clicmd:: graceful-restart prepare ip ospf + + + Initiate a graceful restart for all OSPF instances configured with the + "graceful-restart" command. The ospfd daemon should be restarted during + the instance-specific grace period, otherwise the graceful restart will fail. + + This is an EXEC-level command. + + +.. _showing-ospf-information: + +Showing Information +=================== + +.. _show-ip-ospf: + +.. clicmd:: show ip ospf [vrf <NAME|all>] [json] + + Show information on a variety of general OSPF and area state and + configuration information. + +.. clicmd:: show ip ospf interface [INTERFACE] [json] + + Show state and configuration of OSPF the specified interface, or all + interfaces if no interface is given. + +.. clicmd:: show ip ospf neighbor [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] neighbor INTERFACE [json] + +.. clicmd:: show ip ospf neighbor detail [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] neighbor A.B.C.D [detail] [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] neighbor INTERFACE detail [json] + + Display lsa information of LSDB. + Json o/p of this command covers base route information + i.e all LSAs except opaque lsa info. + +.. clicmd:: show ip ospf [vrf <NAME|all>] database [self-originate] [json] + + Show the OSPF database summary. + +.. clicmd:: show ip ospf [vrf <NAME|all>] database max-age [json] + + Show all MaxAge LSAs present in the OSPF link-state database. + +.. clicmd:: show ip ospf [vrf <NAME|all>] database detail [LINK-STATE-ID] [adv-router A.B.C.D] [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] database detail [LINK-STATE-ID] [self-originate] [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (asbr-summary|external|network|router|summary|nssa-external|opaque-link|opaque-area|opaque-as) [LINK-STATE-ID] [adv-router A.B.C.D] [json] + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (asbr-summary|external|network|router|summary|nssa-external|opaque-link|opaque-area|opaque-as) [LINK-STATE-ID] [self-originate] [json] + + Show detailed information about the OSPF link-state database. + +.. clicmd:: show ip ospf route [detail] [json] + + Show the OSPF routing table, as determined by the most recent SPF + calculation. When detail option is used, it shows more information + to the CLI like advertising router ID for each route, etc. + +.. clicmd:: show ip ospf [vrf <NAME|all>] border-routers [json] + + Show the list of ABR and ASBR border routers summary learnt via + OSPFv2 Type-3 (Summary LSA) and Type-4 (Summary ASBR LSA). + User can get that information as JSON format when ``json`` keyword + at the end of cli is presented. + +.. clicmd:: show ip ospf graceful-restart helper [detail] [json] + + Displays the Graceful Restart Helper details including helper + config changes. + +.. _opaque-lsa: + +Opaque LSA +========== + +.. clicmd:: ospf opaque-lsa + +.. clicmd:: capability opaque + + + + *ospfd* supports Opaque LSA (:rfc:`5250`) as partial support for + MPLS Traffic Engineering LSAs. The opaque-lsa capability must be + enabled in the configuration. An alternate command could be + "mpls-te on" (:ref:`ospf-traffic-engineering`). Note that FRR + offers only partial support for some of the routing protocol + extensions that are used with MPLS-TE; it does not support a + complete RSVP-TE solution. + +.. clicmd:: ip ospf capability opaque [A.B.C.D] + + Enable or disable OSPF LSA database exchange and flooding on an interface. + The default is that opaque capability is enabled as long as the opaque + capability is enabled with the :clicmd:`capability opaque` command at the + OSPF instance level (using the command above). Note that disabling opaque + LSA support on an interface will impact the applications using opaque LSAs + if the opaque LSAs are not received on other flooding paths by all the + OSPF routers using those applications. For example, OSPF Graceful Restart + uses opaque-link LSAs and disabling support on an interface will disable + graceful restart signaling on that interface. + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) LINK-STATE-ID + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) LINK-STATE-ID adv-router ADV-ROUTER + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) adv-router ADV-ROUTER + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) LINK-STATE-ID self-originate + +.. clicmd:: show ip ospf [vrf <NAME|all>] database (opaque-link|opaque-area|opaque-external) self-originate + + Show Opaque LSA from the database. + +.. clicmd:: show ip ospf (1-65535) reachable-routers + +.. clicmd:: show ip ospf [vrf <NAME|all>] reachable-routers + + Show routing table of reachable routers. + +.. _ospf-traffic-engineering: + +Traffic Engineering +=================== + +.. note:: + + At this time, FRR offers partial support for some of the routing + protocol extensions that can be used with MPLS-TE. FRR does not + support a complete RSVP-TE solution currently. + +.. clicmd:: mpls-te on + + + Enable Traffic Engineering LSA flooding. + +.. clicmd:: mpls-te router-address <A.B.C.D> + + Configure stable IP address for MPLS-TE. This IP address is then advertise + in Opaque LSA Type-10 TLV=1 (TE) option 1 (Router-Address). + +.. clicmd:: mpls-te inter-as area <area-id>|as + + + Enable :rfc:`5392` support - Inter-AS TE v2 - to flood Traffic Engineering + parameters of Inter-AS link. 2 modes are supported: AREA and AS; LSA are + flood in AREA <area-id> with Opaque Type-10, respectively in AS with Opaque + Type-11. In all case, Opaque-LSA TLV=6. + +.. clicmd:: mpls-te export + + Export Traffic Engineering Data Base to other daemons through the ZAPI + Opaque Link State messages. + +.. clicmd:: show ip ospf mpls-te interface + +.. clicmd:: show ip ospf mpls-te interface INTERFACE + + Show MPLS Traffic Engineering parameters for all or specified interface. + +.. clicmd:: show ip ospf mpls-te router + + Show Traffic Engineering router parameters. + +.. clicmd:: show ip ospf mpls-te database [verbose|json] + +.. clicmd:: show ip ospf mpls-te database vertex [self-originate|adv-router ADV-ROUTER] [verbose|json] + +.. clicmd:: show ip ospf mpls-te database edge [A.B.C.D] [verbose|json] + +.. clicmd:: show ip ospf mpls-te database subnet [A.B.C.D/M] [verbose|json] + + Show Traffic Engineering Database + +.. _router-information: + +Router Information +================== + +.. clicmd:: router-info [as | area] + + + Enable Router Information (:rfc:`4970`) LSA advertisement with AS scope + (default) or Area scope flooding when area is specified. Old syntax + `router-info area <A.B.C.D>` is always supported but mark as deprecated + as the area ID is no more necessary. Indeed, router information support + multi-area and detect automatically the areas. + +.. clicmd:: pce address <A.B.C.D> + + +.. clicmd:: pce domain as (0-65535) + + +.. clicmd:: pce neighbor as (0-65535) + + +.. clicmd:: pce flag BITPATTERN + + +.. clicmd:: pce scope BITPATTERN + + + The commands are conform to :rfc:`5088` and allow OSPF router announce Path + Computation Element (PCE) capabilities through the Router Information (RI) + LSA. Router Information must be enable prior to this. The command set/unset + respectively the PCE IP address, Autonomous System (AS) numbers of + controlled domains, neighbor ASs, flag and scope. For flag and scope, please + refer to :rfc`5088` for the BITPATTERN recognition. Multiple 'pce neighbor' + command could be specified in order to specify all PCE neighbours. + +.. clicmd:: show ip ospf router-info + + Show Router Capabilities flag. + +.. clicmd:: show ip ospf router-info pce + + Show Router Capabilities PCE parameters. + +Segment Routing +=============== + +This is an EXPERIMENTAL support of Segment Routing as per `RFC 8665` for MPLS +dataplane. + +.. clicmd:: segment-routing on + + Enable Segment Routing. Even if this also activate routing information + support, it is preferable to also activate routing information, and set + accordingly the Area or AS flooding. + +.. clicmd:: segment-routing global-block (16-1048575) (16-1048575) [local-block (16-1048575) (16-1048575)] + + Set the Segment Routing Global Block i.e. the label range used by MPLS to + store label in the MPLS FIB for Prefix SID. Optionally also set the Local + Block, i.e. the label range used for Adjacency SID. The negative version + of the command always unsets both ranges. + +.. clicmd:: segment-routing node-msd (1-16) + + Fix the Maximum Stack Depth supported by the router. The value depend of the + MPLS dataplane. E.g. for Linux kernel, since version 4.13 it is 32. + +.. clicmd:: segment-routing prefix A.B.C.D/M [index (0-65535)|no-php-flag|explicit-null] + + prefix with /32 corresponding to a loopback interface are currently + supported. The 'no-php-flag' means NO Penultimate Hop Popping that allows SR + node to request to its neighbor to not pop the label. The 'explicit-null' means that + neighbor nodes must swap the incoming label by the MPLS Explicit Null label + before delivering the packet. + +.. clicmd:: show ip ospf database segment-routing <adv-router ADVROUTER|self-originate> [json] + + Show Segment Routing Data Base, all SR nodes, specific advertised router or + self router. Optional JSON output can be obtained by appending 'json' to the + end of the command. + +External Route Summarisation +============================ +This feature summarises originated external LSAs(Type-5 and Type-7). +Summary Route will be originated on-behalf of all matched external LSAs. + +.. clicmd:: summary-address A.B.C.D/M [tag (1-4294967295)] + + This command enable/disables summarisation for the configured address + range. Tag is the optional parameter. If tag configured Summary route + will be originated with the configured tag. + +.. clicmd:: summary-address A.B.C.D/M no-advertise + + This command to ensure not advertise the summary lsa for the matched + external LSAs. + +.. clicmd:: aggregation timer (5-1800) + + Configure aggregation delay timer interval. Summarisation starts only after + this delay timer expiry. By default, delay interval is 5 seconds. + + + The no form of the command resets the aggregation delay interval to default + value. + +.. clicmd:: show ip ospf [vrf <NAME|all>] summary-address [detail] [json] + + Show configuration for display all configured summary routes with + matching external LSA information. + +TI-LFA +====== + +Experimental support for Topology Independent LFA (Loop-Free Alternate), see +for example 'draft-bashandy-rtgwg-segment-routing-ti-lfa-05'. Note that +TI-LFA requires a proper Segment Routing configuration. + +.. clicmd:: fast-reroute ti-lfa [node-protection] + + Configured on the router level. Activates TI-LFA for all interfaces. + + Note that so far only P2P interfaces are supported. + +.. _debugging-ospf: + +Debugging OSPF +============== + +.. clicmd:: debug ospf [(1-65535)] bfd + + Enable or disable debugging for BFD events. This will show BFD integration + library messages and OSPF BFD integration messages that are mostly state + transitions and validation problems. + +.. clicmd:: debug ospf [(1-65535)] client-api + + Show debug information for the OSPF opaque data client API. + +.. clicmd:: debug ospf [(1-65535)] default-information + + Show debug information of default information + +.. clicmd:: debug ospf [(1-65535)] packet (hello|dd|ls-request|ls-update|ls-ack|all) (send|recv) [detail] + + + Dump Packet for debugging + +.. clicmd:: debug ospf [(1-65535)] ism [status|events|timers] + + + + Show debug information of Interface State Machine + +.. clicmd:: debug ospf [(1-65535)] nsm [status|events|timers] + + + + Show debug information of Network State Machine + +.. clicmd:: debug ospf [(1-65535)] event + + + Show debug information of OSPF event + +.. clicmd:: debug ospf [(1-65535)] nssa + + + Show debug information about Not So Stub Area + +.. clicmd:: debug ospf [(1-65535)] ldp-sync + + Show debug information about LDP-Sync + +.. clicmd:: debug ospf [(1-65535)] lsa [aggregate|flooding|generate|install|refresh] + + + + Show debug detail of Link State messages + +.. clicmd:: debug ospf [(1-65535)] sr + + Show debug information about Segment Routing + +.. clicmd:: debug ospf [(1-65535)] te + + + Show debug information about Traffic Engineering LSA + +.. clicmd:: debug ospf [(1-65535)] ti-lfa + + Show debug information about SR TI-LFA + +.. clicmd:: debug ospf [(1-65535)] zebra [interface|redistribute] + + + + Show debug information of ZEBRA API + +.. clicmd:: debug ospf [(1-65535)] graceful-restart + + + Enable/disable debug information for OSPF Graceful Restart Helper + +.. clicmd:: show debugging ospf + + + +Sample Configuration +==================== + +A simple example, with MD5 authentication enabled: + +.. code-block:: frr + + ! + interface bge0 + ip ospf authentication message-digest + ip ospf message-digest-key 1 md5 ABCDEFGHIJK + ! + router ospf + network 192.168.0.0/16 area 0.0.0.1 + area 0.0.0.1 authentication message-digest + + +An :abbr:`ABR` router, with MD5 authentication and performing summarisation +of networks between the areas: + +.. code-block:: frr + + ! + password ABCDEF + log file /var/log/frr/ospfd.log + service advanced-vty + ! + interface eth0 + ip ospf authentication message-digest + ip ospf message-digest-key 1 md5 ABCDEFGHIJK + ! + interface ppp0 + ip ospf passive + ! + interface br0 + ip ospf authentication message-digest + ip ospf message-digest-key 2 md5 XYZ12345 + ! + router ospf + ospf router-id 192.168.0.1 + redistribute connected + network 192.168.0.0/24 area 0.0.0.0 + network 10.0.0.0/16 area 0.0.0.0 + network 192.168.1.0/24 area 0.0.0.1 + area 0.0.0.0 authentication message-digest + area 0.0.0.0 range 10.0.0.0/16 + area 0.0.0.0 range 192.168.0.0/24 + area 0.0.0.1 authentication message-digest + area 0.0.0.1 range 10.2.0.0/16 + ! + + +A Traffic Engineering configuration, with Inter-ASv2 support. + +First, the :file:`zebra.conf` part: + +.. code-block:: frr + + interface eth0 + ip address 198.168.1.1/24 + link-params + enable + admin-grp 0xa1 + metric 100 + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + ! + interface eth1 + ip address 192.168.2.1/24 + link-params + enable + metric 10 + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + neighbor 192.168.2.2 as 65000 + hostname HOSTNAME + password PASSWORD + log file /var/log/zebra.log + ! + interface eth0 + ip address 198.168.1.1/24 + link-params + enable + admin-grp 0xa1 + metric 100 + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + ! + interface eth1 + ip address 192.168.2.1/24 + link-params + enable + metric 10 + max-bw 1.25e+07 + max-rsv-bw 1.25e+06 + unrsv-bw 0 1.25e+06 + unrsv-bw 1 1.25e+06 + unrsv-bw 2 1.25e+06 + unrsv-bw 3 1.25e+06 + unrsv-bw 4 1.25e+06 + unrsv-bw 5 1.25e+06 + unrsv-bw 6 1.25e+06 + unrsv-bw 7 1.25e+06 + neighbor 192.168.2.2 as 65000 + +Then the :file:`ospfd.conf` itself: + +.. code-block:: frr + + hostname HOSTNAME + password PASSWORD + log file /var/log/ospfd.log + ! + ! + interface eth0 + ip ospf hello-interval 60 + ip ospf dead-interval 240 + ! + interface eth1 + ip ospf hello-interval 60 + ip ospf dead-interval 240 + ! + ! + router ospf + ospf router-id 192.168.1.1 + network 192.168.0.0/16 area 1 + ospf opaque-lsa + mpls-te + mpls-te router-address 192.168.1.1 + mpls-te inter-as area 1 + ! + line vty + +A router information example with PCE advertisement: + +.. code-block:: frr + + ! + router ospf + ospf router-id 192.168.1.1 + network 192.168.0.0/16 area 1 + capability opaque + mpls-te + mpls-te router-address 192.168.1.1 + router-info area 0.0.0.1 + pce address 192.168.1.1 + pce flag 0x80 + pce domain as 65400 + pce neighbor as 65500 + pce neighbor as 65200 + pce scope 0x80 + ! diff --git a/doc/user/overview.rst b/doc/user/overview.rst new file mode 100644 index 0000000..a1cb0cc --- /dev/null +++ b/doc/user/overview.rst @@ -0,0 +1,572 @@ +.. _overview: + +******** +Overview +******** + +`FRR`_ is a fully featured, high performance, free software IP routing suite. + +FRR implements all standard routing protocols such as BGP, RIP, OSPF, IS-IS and +more (see :ref:`feature-matrix`), as well as many of their extensions. + +FRR is a high performance suite written primarily in C. It can easily handle +full Internet routing tables and is suitable for use on hardware ranging from +cheap SBCs to commercial grade routers. It is actively used in production by +hundreds of companies, universities, research labs and governments. + +FRR is distributed under GPLv2, with development modeled after the Linux +kernel. Anyone may contribute features, bug fixes, tools, documentation +updates, or anything else. + +FRR is a fork of `Quagga <http://www.quagga.net/>`_. + +.. _how-to-get-frr: + +How to get FRR +============== + +The official FRR website is located at |PACKAGE_URL| and contains further +information, as well as links to additional resources. + +Several distributions provide packages for FRR. Check your distribution's +repositories to find out if a suitable version is available. + +Up-to-date Debian & Redhat packages are available at https://deb.frrouting.org/ +& https://rpm.frrouting.org/ respectively. + +For instructions on installing from source, refer to the +`developer documentation <http://docs.frrouting.org/projects/dev-guide/en/latest/>`_. + + +.. _about-frr: + +About FRR +========= + +FRR provides IP routing services. Its role in a networking stack is to exchange +routing information with other routers, make routing and policy decisions, and +inform other layers of these decisions. In the most common scenario, FRR +installs routing decisions into the OS kernel, allowing the kernel networking +stack to make the corresponding forwarding decisions. + +In addition to dynamic routing FRR supports the full range of L3 configuration, +including static routes, addresses, router advertisements etc. It has some +light L2 functionality as well, but this is mostly left to the platform. This +makes it suitable for deployments ranging from small home networks with static +routes to Internet exchanges running full Internet tables. + +FRR runs on all modern \*NIX operating systems, including Linux and the BSDs. +Feature support varies by platform; see the :ref:`feature-matrix`. + +System Requirements +------------------- + +System resources needed by FRR are highly dependent on workload. Routing +software performance is particularly susceptible to external factors such as: + +* Kernel networking stack +* Physical NIC +* Peer behavior +* Routing information scale + +Because of these factors - especially the last one - it's difficult to lay out +resource requirements. + +To put this in perspective, FRR can be run on very low resource systems such as +SBCs, provided it is not stressed too much. If you want to set up 4 Raspberry +Pis to play with BGP or OSPF, it should work fine. If you ask a FRR to process +a complete internet routing table on a Raspberry Pi, you will be disappointed. +However, given enough resources, FRR ought to be capable of acting as a core IX +router. Such a use case requires at least 4gb of memory and a recent quad-core +server processor at a minimum. + +If you are new to networking, an important thing to remember is that FRR is +control plane software. It does not itself forward packets - it exchanges +information with peers about how to forward packets. Forwarding plane +performance largely depends on choice of NIC / ASIC. + + +System Architecture +------------------- + +.. index:: + pair: architecture; FRR + +Traditional routing software is made as a one process program which provides +all of the routing protocol functionalities. FRR takes a different approach. +FRR is a suite of daemons that work together to build the routing table. Each +major protocol is implemented in its own daemon, and these daemons talk to a +middleman daemon (*zebra*), which is responsible for coordinating routing +decisions and talking to the dataplane. + +This architecture allows for high resiliency, since an error, crash or exploit +in one protocol daemon will generally not affect the others. It is also +flexible and extensible since the modularity makes it easy to implement new +protocols and tie them into the suite. Additionally, each daemon implements a +plugin system allowing new functionality to be loaded at runtime. + +An illustration of the large scale architecture is given below. + +:: + + +----+ +----+ +-----+ +----+ +----+ +----+ +-----+ + |bgpd| |ripd| |ospfd| |ldpd| |pbrd| |pimd| |.....| + +----+ +----+ +-----+ +----+ +----+ +----+ +-----+ + | | | | | | | + +----v-------v--------v-------v-------v-------v--------v + | | + | Zebra | + | | + +------------------------------------------------------+ + | | | + | | | + +------v------+ +---------v--------+ +------v------+ + | | | | | | + | *NIX Kernel | | Remote dataplane | | ........... | + | | | | | | + +-------------+ +------------------+ +-------------+ + + +All of the FRR daemons can be managed through a single integrated user +interface shell called *vtysh*. *vtysh* connects to each daemon through a UNIX +domain socket and then works as a proxy for user input. In addition to a +unified frontend, *vtysh* also provides the ability to configure all the +daemons using a single configuration file through the integrated configuration +mode. This avoids the overhead of maintaining a separate configuration file for +each daemon. + +FRR is currently implementing a new internal configuration system based on YANG +data models. When this work is completed, FRR will be a fully programmable +routing stack. + + +.. index:: + pair: platforms; FRR + pair: operating systems; FRR + +.. _supported-platforms: + +Supported Platforms +------------------- + + +Currently FRR supports GNU/Linux and BSD. Porting FRR to other platforms is not +too difficult as platform dependent code should be mostly limited to the +*Zebra* daemon. Protocol daemons are largely platform independent. Please let +us know if you can get FRR to run on a platform which is not listed below: + +- GNU/Linux +- FreeBSD +- NetBSD +- OpenBSD + +Versions of these platforms that are older than around 2 years from the point +of their original release (in case of GNU/Linux, this is since the kernel's +release on https://kernel.org/) may need some work. Similarly, the following +platforms may work with some effort: + +- MacOS + +Recent versions of the following compilers are well tested: + +- GNU's GCC +- LLVM's Clang +- Intel's ICC + +.. _unsupported-platforms: + +Unsupported Platforms +--------------------- + +In General if the platform you are attempting to use is not listed above then +FRR does not support being run on that platform. The only caveat here is that +version 7.5 and before Solaris was supported in a limited fashion. + +.. _feature-matrix: + +Feature Matrix +^^^^^^^^^^^^^^ + +The following table lists all protocols cross-referenced to all operating +systems that have at least CI build tests. Note that for features, only +features with system dependencies are included here; if you don't see the +feature you're interested in, it should be supported on your platform. + +.. role:: mark + +.. comment - the :mark:`X` pieces mesh with a little bit of JavaScript and + CSS in _static/overrides.{js,css} respectively. The JS code looks at the + presence of the 'Y' 'N' '≥' '†' or 'CP' strings. This seemed to be the + best / least intrusive way of getting a nice table in HTML. The table + will look somewhat shoddy on other sphinx targets like PDF or info (but + should still be readable.) + ++-----------------------------------+----------------+--------------+------------+------------+ +| Daemon / Feature | Linux | OpenBSD | FreeBSD | NetBSD | ++===================================+================+==============+============+============+ +| **FRR Core** | | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| `zebra` | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| VRF | :mark:`≥4.8` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| MPLS | :mark:`≥4.5` | :mark:`Y` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `pbrd` (Policy Routing) | :mark:`Y` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| **WAN / Carrier protocols** | | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| `bgpd` (BGP) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| VRF / L3VPN | :mark:`≥4.8` | :mark:`CP` | :mark:`CP` | :mark:`CP` | +| | :mark:`†4.3` | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| EVPN | :mark:`≥4.18` | :mark:`CP` | :mark:`CP` | :mark:`CP` | +| | :mark:`†4.9` | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| VNC (Virtual Network Control) | :mark:`CP` | :mark:`CP` | :mark:`CP` | :mark:`CP` | ++-----------------------------------+----------------+--------------+------------+------------+ +| Flowspec | :mark:`CP` | :mark:`CP` | :mark:`CP` | :mark:`CP` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `ldpd` (LDP) | :mark:`≥4.5` | :mark:`Y` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| VPWS / PW | :mark:`N` | :mark:`≥5.8` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| VPLS | :mark:`N` | :mark:`≥5.8` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `nhrpd` (NHRP) | :mark:`Y` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| **Link-State Routing** | | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| `ospfd` (OSPFv2) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| Segment Routing | :mark:`≥4.12` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `ospf6d` (OSPFv3) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `isisd` (IS-IS) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| **Distance-Vector Routing** | | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| `ripd` (RIPv2) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `ripngd` (RIPng) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `babeld` (BABEL) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `eigrpd` (EIGRP) | :mark:`Y` | :mark:`Y` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| **Multicast Routing** | | | | | ++-----------------------------------+----------------+--------------+------------+------------+ +| `pimd` (PIM) | :mark:`≥4.19` | :mark:`N` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| SSM (Source Specific) | :mark:`Y` | :mark:`N` | :mark:`Y` | :mark:`Y` | ++-----------------------------------+----------------+--------------+------------+------------+ +| ASM (Any Source) | :mark:`Y` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| EVPN BUM Forwarding | :mark:`≥5.0` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ +| `vrrpd` (VRRP) | :mark:`≥5.1` | :mark:`N` | :mark:`N` | :mark:`N` | ++-----------------------------------+----------------+--------------+------------+------------+ + +The indicators have the following semantics: + +* :mark:`Y` - daemon/feature fully functional +* :mark:`≥X.X` - fully functional with kernel version X.X or newer +* :mark:`†X.X` - restricted functionality or impaired performance with kernel version X.X or newer +* :mark:`CP` - control plane only (i.e. BGP route server / route reflector) +* :mark:`N` - daemon/feature not supported by operating system + + +Known Kernel Issues +------------------- + +- Linux < 4.11 + + v6 Route Replacement - Linux kernels before 4.11 can cause issues with v6 + route deletion when you have ECMP routes installed into the kernel. This + especially becomes apparent if the route is being transformed from one ECMP + path to another. + + +.. index:: + pair: rfcs; FRR + +.. _supported-rfcs: + +Supported RFCs +-------------- + +FRR implements the following RFCs: + +.. note:: This list is incomplete. + +BGP +---- + +- :rfc:`1771` + :t:`A Border Gateway Protocol 4 (BGP-4). Y. Rekhter & T. Li. March 1995.` +- :rfc:`1965` + :t:`Autonomous System Confederations for BGP. P. Traina. June 1996.` +- :rfc:`1997` + :t:`BGP Communities Attribute. R. Chandra, P. Traina & T. Li. August 1996.` +- :rfc:`1998` + :t:`An Application of the BGP Community Attribute in Multi-home Routing. E. Chen, T. Bates. August 1996.` +- :rfc:`2385` + :t:`Protection of BGP Sessions via the TCP MD5 Signature Option. A. Heffernan. August 1998.` +- :rfc:`2439` + :t:`BGP Route Flap Damping. C. Villamizar, R. Chandra, R. Govindan. November 1998.` +- :rfc:`2545` + :t:`Use of BGP-4 Multiprotocol Extensions for IPv6 Inter-Domain Routing. P. Marques, F. Dupont. March 1999.` +- :rfc:`2796` + :t:`BGP Route Reflection An alternative to full mesh IBGP. T. Bates & R. Chandrasekeran. June 1996.` +- :rfc:`2842` + :t:`Capabilities Advertisement with BGP-4. R. Chandra, J. Scudder. May 2000.` +- :rfc:`2858` + :t:`Multiprotocol Extensions for BGP-4. T. Bates, Y. Rekhter, R. Chandra, D. Katz. June 2000.` +- :rfc:`2918` + :t:`Route Refresh Capability for BGP-4. E. Chen, September 2000.` +- :rfc:`3107` + :t:`Carrying Label Information in BGP-4. Y. Rekhter & E. Rosen. May 2001.` +- :rfc:`3765` + :t:`NOPEER Community for Border Gateway Protocol (BGP) Route Scope Control. G.Huston. April 2001.` +- :rfc:`4271` + :t:`A Border Gateway Protocol 4 (BGP-4). Updates RFC1771. Y. Rekhter, T. Li & S. Hares. January 2006.` +- :rfc:`4360` + :t:`BGP Extended Communities Attribute. S. Sangli, D. Tappan, Y. Rekhter. February 2006.` +- :rfc:`4364` + :t:`BGP/MPLS IP Virtual Private Networks (VPNs). Y. Rekhter. February 2006.` +- :rfc:`4456` + :t:`BGP Route Reflection An alternative to full mesh IBGP. T. Bates, E. Chen, R. Chandra. April 2006.` +- :rfc:`4486` + :t:`Subcodes for BGP Cease Notification Message. E. Chen, V. Gillet. April 2006.` +- :rfc:`4659` + :t:`BGP-MPLS IP Virtual Private Network (VPN) Extension for IPv6 VPN. J. De Clercq, D. Ooms, M. Carugi, F. Le Faucheur. September 2006.` +- :rfc:`4724` + :t:`Graceful Restart Mechanism for BGP. S. Sangli, E. Chen, R. Fernando, J. Scudder, Y. Rekhter. January 2007.` +- :rfc:`4760` + :t:`Multiprotocol Extensions for BGP-4. T. Bates, R. Chandra, D. Katz, Y. Rekhter. January 2007.` +- :rfc:`4893` + :t:`BGP Support for Four-octet AS Number Space. Q. Vohra, E. Chen May 2007.` +- :rfc:`5004` + :t:`Avoid BGP Best Path Transitions from One External to Another. E. Chen & S. Sangli. September 2007 (Partial support).` +- :rfc:`5065` + :t:`Autonomous System Confederations for BGP. P. Traina, D. McPherson, J. Scudder. August 2007.` +- :rfc:`5082` + :t:`The Generalized TTL Security Mechanism (GTSM). V. Gill, J. Heasley, D. Meyer, P. Savola, C. Pingnataro. October 2007.` +- :rfc:`5291` + :t:`Outbound Route Filtering Capability. E. Chen, Y. Rekhter. August 2008.` +- :rfc:`5292` + :t:`Address-Prefix-Based Outbound Route Filter for BGP-4. E. Chen, S. Sangli. August 2008.` +- :rfc:`5396` + :t:`Textual Representation of Autonomous System (AS) Numbers. G. Michaelson, G. Huston. December 2008.` +- :rfc:`5492` + :t:`Capabilities Advertisement with BGP-4. J. Scudder, R. Chandra. February 2009.` +- :rfc:`5575` + :t:`Dissemination of Flow Specification Rules. P. Marques, N. Sheth, R. Raszuk, B. Greene, J. Mauch, D. McPherson. August 2009.` +- :rfc:`5668` + :t:`4-Octet AS Specific BGP Extended Community. Y. Rekhter, S. Sangli, D. Tappan October 2009.` +- :rfc:`6286` + :t:`Autonomous-System-Wide Unique BGP Identifier for BGP-4. E. Chen, J. Yuan. June 2011.` +- :rfc:`6472` + :t:`Recommendation for Not Using AS_SET and AS_CONFED_SET in BGP. W. Kumari, K. Sriram. December 2011.` +- :rfc:`6608` + :t:`Subcodes for BGP Finite State Machine Error. J. Dong, M. Chen, Huawei Technologies, A. Suryanarayana, Cisco Systems. May 2012.` +- :rfc:`6810` + :t:`The Resource Public Key Infrastructure (RPKI) to Router Protocol. R. Bush, R. Austein. January 2013.` +- :rfc:`6811` + :t:`BGP Prefix Origin Validation. P. Mohapatra, J. Scudder, D. Ward, R. Bush, R. Austein. January 2013.` +- :rfc:`6938` + :t:`Deprecation of BGP Path Attributes: DPA, ADVERTISER, and RCID_PATH / CLUSTER_ID. J. Scudder. May 2013.` +- :rfc:`6996` + :t:`Autonomous System (AS) Reservation for Private Use. J. Mitchell. July 2013.` +- :rfc:`7196` + :t:`Making Route Flap Damping Usable. C. Pelsser, R. Bush, K. Patel, P. Mohapatra, O. Maennel. May 2014.` +- :rfc:`7300` + :t:`Reservation of Last Autonomous System (AS) Numbers. J. Haas, J. Mitchell. July 2014.` +- :rfc:`7313` + :t:`Enhanced Route Refresh Capability for BGP-4. K. Patel, E. Chen, B. Venkatachalapathy. July 2014.` +- :rfc:`7606` + :t:`Revised Error Handling for BGP UPDATE Messages. E. Chen, J. Scudder, P. Mohapatra, K. Patel. August 2015.` +- :rfc:`7607` + :t:`Codification of AS 0 Processing. W. Kumari, R. Bush, H. Schiller, K. Patel. August 2015.` +- :rfc:`7611` + :t:`BGP ACCEPT_OWN Community Attribute. J. Uttaro, P. Mohapatra, D. Smith, R. Raszuk, J. Scudder. August 2015.` +- :rfc:`7911` + :t:`Advertisement of Multiple Paths in BGP. D. Walton, A. Retana, E. Chen, J. Scudder. July 2016.` +- :rfc:`7947` + :t:`Internet Exchange BGP Route Server. E. Jasinska, N. Hilliard, R. Raszuk, N. Bakker. September 2016.` +- :rfc:`7999` + :t:`BLACKHOLE Community. T. King, C. Dietzel, J. Snijders, G. Doering, G. Hankins. October 2016.` +- :rfc:`8050` + :t:`Multi-Threaded Routing Toolkit (MRT) Routing Information Export Format with BGP Additional Path Extensions. C. Petrie, T. King. May 2017.` +- :rfc:`8092` + :t:`BGP Large Communities Attribute. J. Heitz, Ed., J. Snijders, Ed, K. Patel, I. Bagdonas, N. Hilliard. February 2017.` +- :rfc:`8093` + :t:`Deprecation of BGP Path Attribute Values 30, 31, 129, 241, 242, and 243. J. Snijders. February 2017.` +- :rfc:`8097` + :t:`BGP Prefix Origin Validation State Extended Community. P. Mohapatra, K. Patel, J. Scudder, D. Ward, R. Bush. March 2017.` +- :rfc:`8195` + :t:`Use of BGP Large Communities. J. Snijders, J. Heasley, M. Schmidt. June 2017.` +- :rfc:`8203` + :t:`BGP Administrative Shutdown Communication. J. Snijders, J. Heitz, J. Scudder. July 2017.` +- :rfc:`8212` + :t:`Default External BGP (EBGP) Route Propagation Behavior without Policies. J. Mauch, J. Snijders, G. Hankins. July 2017.` +- :rfc:`8277` + :t:`Using BGP to Bind MPLS Labels to Address Prefixes. E. Rosen. October 2017.` +- :rfc:`8538` + :t:`Notification Message Support for BGP Graceful Restart. K. Patel, R. Fernando, J. Scudder, J. Haas. March 2019.` +- :rfc:`8654` + :t:`Extended Message Support for BGP. R. Bush, K. Patel, D. Ward. October 2019.` +- :rfc:`9003` + :t:`Extended BGP Administrative Shutdown Communication. J. Snijders, J. Heitz, J. Scudder, A. Azimov. January 2021.` +- :rfc:`9012` + :t:`The BGP Tunnel Encapsulation Attribute. K. Patel, G. Van de Velde, S. Sangli, J. Scudder. April 2021.` +- :rfc:`9072` + :t:`Extended Optional Parameters Length for BGP OPEN Message. E. Chen, J. Scudder. July 2021.` +- :rfc:`9234` + :t:`Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages. A. Azimov, E. Bogomazov, R. Bush, K. Patel, K. Sriram. May 2022.` +- :rfc:`9384` + :t:`A BGP Cease NOTIFICATION Subcode for Bidirectional Forwarding Detection (BFD). J. Haas. March 2023.` + +OSPF +---- + +- :rfc:`2328` + :t:`OSPF Version 2. J. Moy. April 1998.` +- :rfc:`2370` + :t:`The OSPF Opaque LSA Option R. Coltun. July 1998.` +- :rfc:`3101` + :t:`The OSPF Not-So-Stubby Area (NSSA) Option P. Murphy. January 2003.` +- :rfc:`2740` + :t:`OSPF for IPv6. R. Coltun, D. Ferguson, J. Moy. December 1999.` +- :rfc:`3137` + :t:`OSPF Stub Router Advertisement, A. Retana, L. Nguyen, R. White, A. Zinin, D. McPherson. June 2001` + +ISIS +---- + +RIP +---- + +- :rfc:`1058` + :t:`Routing Information Protocol. C.L. Hedrick. Jun-01-1988.` +- :rfc:`2082` + :t:`RIP-2 MD5 Authentication. F. Baker, R. Atkinson. January 1997.` +- :rfc:`2453` + :t:`RIP Version 2. G. Malkin. November 1998.` +- :rfc:`2080` + :t:`RIPng for IPv6. G. Malkin, R. Minnear. January 1997.` + +PIM +---- + +BFD +---- +- :rfc:`5880` + :t:`Bidirectional Forwarding Detection (BFD), D. Katz, D. Ward. June 2010` +- :rfc:`5881` + :t:`Bidirectional Forwarding Detection (BFD) for IPv4 and IPv6 (Single Hop), D. Katz, D. Ward. June 2010` +- :rfc:`5882` + :t:`Generic Application of Bidirectional Forwarding Detection (BFD), D. Katz, D. Ward. June 2010` +- :rfc:`5883` + :t:`Bidirectional Forwarding Detection (BFD) for Multihop Paths, D. Katz, D. Ward. June 2010` + +MPLS +---- + +- :rfc:`2858` + :t:`Multiprotocol Extensions for BGP-4. T. Bates, Y. Rekhter, R. Chandra, D. Katz. June 2000.` +- :rfc:`4364` + :t:`BGP/MPLS IP Virtual Private Networks (VPNs). Y. Rekhter. Feb 2006.` +- :rfc:`4447` + :t:`Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP), L. Martini, E. Rosen, N. El-Aawar, T. Smith, and G. Heron. April 2006.` +- :rfc:`4659` + :t:`BGP-MPLS IP Virtual Private Network (VPN) Extension for IPv6 VPN. J. De Clercq, D. Ooms, M. Carugi, F. Le Faucheur. September 2006` +- :rfc:`4762` + :t:`Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling, M. Lasserre and V. Kompella. January 2007.` +- :rfc:`5036` + :t:`LDP Specification, L. Andersson, I. Minei, and B. Thomas. October 2007.` +- :rfc:`5561` + :t:`LDP Capabilities, B. Thomas, K. Raza, S. Aggarwal, R. Aggarwal, and JL. Le Roux. July 2009.` +- :rfc:`5918` + :t:`Label Distribution Protocol (LDP) 'Typed Wildcard' Forward Equivalence Class (FEC), R. Asati, I. Minei, and B. Thomas. August 2010.` +- :rfc:`5919` + :t:`Signaling LDP Label Advertisement Completion, R. Asati, P. Mohapatra, E. Chen, and B. Thomas. August 2010.` +- :rfc:`6667` + :t:`LDP 'Typed Wildcard' Forwarding Equivalence Class (FEC) for PWid and Generalized PWid FEC Elements, K. Raza, S. Boutros, and C. Pignataro. July 2012.` +- :rfc:`6720` + :t:`The Generalized TTL Security Mechanism (GTSM) for the Label Distribution Protocol (LDP), C. Pignataro and R. Asati. August 2012.` +- :rfc:`7552` + :t:`Updates to LDP for IPv6, R. Asati, C. Pignataro, K. Raza, V. Manral, and R. Papneja. June 2015.` + +VRRP +---- + +- :rfc:`3768` + :t:`Virtual Router Redundancy Protocol (VRRP). R. Hinden. April 2004.` +- :rfc:`5798` + :t:`Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 and IPv6. S. Nadas. June 2000.` + +SNMP +---- + +**When SNMP support is enabled, the following RFCs are also supported:** + +- :rfc:`1227` + :t:`SNMP MUX protocol and MIB. M.T. Rose. May-01-1991.` +- :rfc:`1657` + :t:`Definitions of Managed Objects for the Fourth Version of the Border + Gateway Protocol (BGP-4) using SMIv2. S. Willis, J. Burruss, J. Chu, Editor. + July 1994.` +- :rfc:`1724` + :t:`RIP Version 2 MIB Extension. G. Malkin & F. Baker. November 1994.` +- :rfc:`1850` + :t:`OSPF Version 2 Management Information Base. F. Baker, R. Coltun. + November 1995.` +- :rfc:`2741` + :t:`Agent Extensibility (AgentX) Protocol. M. Daniele, B. Wijnen. January 2000.` + + +.. index:: + pair: mailing lists; contact + +.. _mailing-lists: + +Mailing Lists +============= + +Italicized lists are private. + ++--------------------------------+------------------------------+ +| Topic | List | ++================================+==============================+ +| Development | dev@lists.frrouting.org | ++--------------------------------+------------------------------+ +| Users & Operators | frog@lists.frrouting.org | ++--------------------------------+------------------------------+ +| Announcements | announce@lists.frrouting.org | ++--------------------------------+------------------------------+ +| *Security* | security@lists.frrouting.org | ++--------------------------------+------------------------------+ +| *Technical Steering Committee* | tsc@lists.frrouting.org | ++--------------------------------+------------------------------+ + +The Development list is used to discuss and document general issues related to +project development and governance. The public `Slack`_ instance and weekly +technical meetings provide a higher bandwidth channel for discussions. The +results of such discussions are reflected in updates, as appropriate, to code +(i.e., merges), `GitHub issues`_ tracked issues, and for governance or process +changes, updates to the Development list and either this file or information +posted at `FRR`_. + + +Bug Reports +=========== + +For information on reporting bugs, please see :ref:`bug-reports`. + +.. _frr: https://frrouting.org +.. _github: https://github.com/frrouting/frr/ +.. _github issues: https://github.com/frrouting/frr/issues +.. _slack: https://frrouting.org/community diff --git a/doc/user/packet-dumps.rst b/doc/user/packet-dumps.rst new file mode 100644 index 0000000..6120c3b --- /dev/null +++ b/doc/user/packet-dumps.rst @@ -0,0 +1,221 @@ +.. _packet-binary-dump-format: + +Packet Binary Dump Format +========================= + +FRR can dump routing protocol packets into a file with a binary format. + +It seems to be better that we share the MRT's header format for +backward compatibility with MRT's dump logs. We should also define the +binary format excluding the header, because we must support both IP +v4 and v6 addresses as socket addresses and / or routing entries. + +In the last meeting, we discussed to have a version field in the +header. But Masaki told us that we can define new 'type' value rather +than having a 'version' field, and it seems to be better because we +don't need to change header format. + +Here is the common header format. This is same as that of MRT.:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Time | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Type | Subtype | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +If 'type' is PROTOCOL_BGP4MP_ET, the common header format will +contain an additional microsecond field (RFC6396 2011).:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Time | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Type | Subtype | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Length | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Microsecond | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_STATE_CHANGE, and +Address Family == IP (version 4):: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source AS number | Destination AS number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Interface Index | Address Family | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Old State | New State | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +Where State is the value defined in RFC1771. + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_STATE_CHANGE, +and Address Family == IP version 6:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source AS number | Destination AS number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Interface Index | Address Family | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Old State | New State | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_MESSAGE, +and Address Family == IP (version 4):: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source AS number | Destination AS number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Interface Index | Address Family | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | BGP Message Packet | + | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +Where BGP Message Packet is the whole contents of the +BGP4 message including header portion. + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_MESSAGE, +and Address Family == IP version 6:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source AS number | Destination AS number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Interface Index | Address Family | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Destination IP address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | BGP Message Packet | + | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_ENTRY, +and Address Family == IP (version 4):: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | View # | Status | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Time Last Change | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Address Family | SAFI | Next-Hop-Len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Next Hop Address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Prefix Length | Address Prefix [variable] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Attribute Length | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | BGP Attribute [variable length] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +If 'type' is PROTOCOL_BGP4MP, 'subtype' is BGP4MP_ENTRY, +and Address Family == IP version 6:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | View # | Status | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Time Last Change | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Address Family | SAFI | Next-Hop-Len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Next Hop Address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Next Hop Address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Next Hop Address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Next Hop Address (Cont'd) | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Prefix Length | Address Prefix [variable] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Address Prefix (cont'd) [variable] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Attribute Length | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | BGP Attribute [variable length] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +BGP4 Attribute must not contain MP_UNREACH_NLRI. If BGP Attribute has +MP_REACH_NLRI field, it must has zero length NLRI, e.g., MP_REACH_NLRI has only +Address Family, SAFI and next-hop values. + +If 'type' is PROTOCOL_BGP4MP and 'subtype' is BGP4MP_SNAPSHOT:: + + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | View # | File Name [variable] | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +The file specified in "File Name" contains all routing entries, +which are in the format of ``subtype == BGP4MP_ENTRY``. + +:: + + Constants: + + /\* type value \*/ + #define MSG_PROTOCOL_BGP4MP 16 + #define MSG_PROTOCOL_BGP4MP_ET 17 + /\* subtype value \*/ + #define BGP4MP_STATE_CHANGE 0 + #define BGP4MP_MESSAGE 1 + #define BGP4MP_ENTRY 2 + #define BGP4MP_SNAPSHOT 3 diff --git a/doc/user/pathd.rst b/doc/user/pathd.rst new file mode 100644 index 0000000..ec107fb --- /dev/null +++ b/doc/user/pathd.rst @@ -0,0 +1,622 @@ +.. _path: + +**** +PATH +**** + +:abbr:`PATH` is a daemon that handles the installation and deletion +of Segment Routing (SR) Policies. +Based on MPLS (This means that your OS of choice must support MPLS), +SR add a stack of MPLS labels to ingress packets so these +packets are egress through the desired path. + +.. image:: images/pathd_general.png + +The SR policies and Segment Lists can be configured either locally by means +of vtysh or centralized based on a SDN controller (ODL, Cisco, ...) +communicating using the PCEP protocol (:rfc:`5440`). + + +.. _starting-path: + +Configuration +============= + +Explicit Segment Lists +---------------------- + +This is the simplest way of configuration, no remote PCE is necessary. +In order to create a config that match the graphics used in this documentation, +we will create a segment list (SL) called SL1 with an element for each hop and +that element will be assigned a MPLS label. +Then the SL1 will be used in the policy ``example1``, please note also the +preference as in the case of multiple segment list it will be used with the +criteria of bigger number more preference. +Let see now the final configuration that match the graphics shown above. + + +.. code-block:: frr + + segment-routing + traffic-eng + segment-list SL1 + index 10 mpls label 16001 + index 20 mpls label 16002 + ! + policy color 1 endpoint 192.0.2.4 + name example1 + binding-sid 1111 + candidate-path preference 100 name CP1 explicit segment-list SL1 + + +Explicit Segment Lists and Traffic Engineering Database (TED) +------------------------------------------------------------- + +Sometimes is difficult to know the values of MPLS labels +(adjacency changes,...). +Based on the support of IS-IS or OSPF we can activate TED support what will +allow pathd to resolve MPLS based in different types of segments +(:rfc: `draft-ietf-spring-segment-routing-policy-07`). The supported types are +Type C (prefix and local interface), Type E (prefix and algorithm), +Type F (a pair of IP's). +So the configuration would change to this + +.. code-block:: frr + + segment-routing + traffic-eng + mpls-te on + mpls-te import ospfv2 + segment-list SL1 + index 10 nai prefix 10.1.2.1/32 iface 1 + index 20 nai adjacency 10.1.20.1 10.1.20.2 + ! + policy color 1 endpoint 192.0.2.4 + name example1 + binding-sid 1111 + candidate-path preference 100 name CP1 explicit segment-list SL1 + + +In this case no MPLS are provided but the pathd TED support will resolve the +configuration provided to corresponding MPLS labels. + +.. note:: + Please note the ``mpls-te`` configuration added that activate the TED + support and points to ``ospfv2`` so + the ospfv2 (:ref:`ospf-traffic-engineering`) daemon must be also + running and configure to export TED information. + +.. note:: + It would be the same for isis (:ref:`isis-traffic-engineering`) but in the + moment of writting it's not fully tested. + +Dynamic Segment Lists +--------------------- + +One of the useful options to configure is the creation of policies with +the dynamic option. In this case based on a given endpoint the SL will be +,first calculated, and then sended by means of PCEP protocol by the configured +PCE. + +.. code-block:: frr + + traffic-eng + ! + pcep + ! + pce PCE1 + address ip 192.0.2.10 + ! + pcc + peer PCE1 precedence 10 + ! + policy color 1 endpoint 192.0.2.4 + name example + binding-sid 1111 + candidate-path preference 100 name CP2 dynamic + +.. note:: + Please note the configuration for the remote pce which allows pathd to + connect to the given PCE and act as a PCC (PCEP Client) + +.. note:: + If the TED support feature is active, the data obtained from PCE will + be validated, so in a SL from PCEP/PCE the IP and MPLS will be checked + against local TED obtained and built from the igp configured in that + case. + +.. image:: images/pathd_config.png + +Pce Initiated +------------- + +We can step forward in the use of our controller not only by asking to +calculate paths to an endpoint but also to create the whole policies in the +controller and obtain those by means of the PCEP protocol. + + +.. code-block:: frr + + traffic-eng + ! + pcep + ! + pce PCE1 + address ip 192.0.2.10 + pce-initiated + ! + pce PCE2 + address ip 192.0.2.9 + pce-initiated + ! + pcc + peer PCE1 precedence 10 + peer PCE2 precedence 20 + ! + +.. note:: + Now there is no locally created policies in the config as they will + be obtain from the configured pce. + Please check command :clicmd:`show sr-te policy` in ``vtysh`` to see + the obtained policies. + +.. note:: + Another interesting command is :clicmd:`show mpls table` + to check the installed mpls configuration based in those obtained + policies. + +.. note:: + SR Policies could be a mix of local, remote obtained from PCE and + delegated to a PCE (but while testing Pce Initiated with Cisco PCE, + happens that controller sends PCE initiated delete commands to delete + the locally created configuration related to that PCE). + + +.. image:: images/pathd_initiated_multi.png + +Starting +======== + +Default configuration file for *pathd* is :file:`pathd.conf`. The typical +location of :file:`pathd.conf` is |INSTALL_PREFIX_ETC|/pathd.conf. + +If the user is using integrated config, then :file:`pathd.conf` need not be +present and the :file:`frr.conf` is read instead. + +.. program:: pathd + +:abbr:`PATH` supports all the common FRR daemon start options which are +documented elsewhere. + +PCEP Support +============ + +A pceplib is included in the frr source tree and build by default. + + +To start pathd with pcep support the extra parameter `-M pathd_pcep` should be +passed to the pathd daemon. + +An example of command line with pcep module could be this + +.. code-block:: frr + + pathd -u root -g root -f pathd.conf -z /tmp/zebra-demo1.sock --vty_socket=/var/run/demo1.vty -i /tmp/pathd-demo1.pid -M frr/modules/pathd_pcep.so --log file:/tmp/kk.txt + +Pathd Configuration +=================== + +Example: + +.. code-block:: frr + + debug pathd pcep basic + segment-routing + traffic-eng + mpls-te on + mpls-te import ospfv2 + segment-list SL1 + index 10 mpls label 16010 + index 20 mpls label 16030 + ! + segment-list SL2 + index 10 nai prefix 10.1.2.1/32 iface 1 + index 20 nai adjacency 10.1.20.1 10.1.20.2 + index 30 nai prefix 10.10.10.5/32 algorithm 0 + index 40 mpls label 18001 + ! + policy color 1 endpoint 192.0.2.1 + name default + binding-sid 4000 + candidate-path preference 100 name CP1 explicit segment-list SL1 + candidate-path preference 200 name CP2 dynamic + affinity include-any 0x000000FF + bandwidth 100000 + metric bound msd 16 required + metric te 10 + objective-function mcp required + ! + pcep + pce-config GROUP1 + source-address 192.0.2.1 + tcp-md5-auth secret + timer keep-alive 30 + ! + pce PCE1 + config GROUP1 + address ip 192.0.2.10 + ! + pce PCE2 + config GROUP1 + address ip 192.0.2.9 + ! + pcc + peer PCE1 precedence 10 + peer PCE2 precedence 20 + ! + ! + ! + ! + + +.. _path-commands: + +Configuration Commands +---------------------- + +.. clicmd:: segment-routing + + Configure segment routing. + +.. clicmd:: traffic-eng + + Configure segment routing traffic engineering. + +.. clicmd:: mpls-te <on|off> + + Activate/Deactivate use of internal Traffic Engineering Database + +.. clicmd:: mpls-te import <ospfv2|ospfv3|isis> + + Load data from the selected igp + +.. clicmd:: segment-list NAME + + Delete or start a segment list definition. + +.. clicmd:: index INDEX mpls label LABEL +.. clicmd:: index INDEX nai adjacency A.B.C.D A.B.C.D +.. clicmd:: index INDEX nai prefix A.B.C.D/M algorithm <0|1> +.. clicmd:: index INDEX nai prefix A.B.C.D/M iface (0-65535) + + Delete or specify a segment in a segment list definition. + + +.. clicmd:: policy color COLOR endpoint ENDPOINT + + Delete or start a policy definition. + + +.. clicmd:: name NAME + + Specify the policy name. + + +.. clicmd:: binding-sid LABEL + + Specify the policy SID. + + +.. clicmd:: candidate-path preference PREFERENCE name NAME explicit segment-list SEGMENT-LIST-NAME + + Delete or define an explicit candidate path. + + +.. clicmd:: candidate-path preference PREFERENCE name NAME dynamic + + Delete or start a dynamic candidate path definition. + + +.. clicmd:: affinity <exclude-any|include-any|include-all> BITPATTERN + + Delete or specify an affinity constraint for a dynamic candidate path. + + +.. clicmd:: bandwidth BANDWIDTH [required] + + Delete or specify a bandwidth constraint for a dynamic candidate path. + + +.. clicmd:: metric [bound] METRIC VALUE [required] + + Delete or specify a metric constraint for a dynamic candidate path. + + The possible metrics are: + - igp: IGP metric + - te: TE metric + - hc: Hop Counts + - abc: Aggregate bandwidth consumption + - mll: Load of the most loaded link + - igp: Cumulative IGP cost + - cte: Cumulative TE cost + - igp: P2MP IGP metric + - pte: P2MP TE metric + - phc: P2MP hop count metric + - msd: Segment-ID (SID) Depth + - pd: Path Delay metric + - pdv: Path Delay Variation metric + - pl: Path Loss metric + - ppd: P2MP Path Delay metric + - pdv: P2MP Path Delay variation metric + - ppl: P2MP Path Loss metric + - nap: Number of adaptations on a path + - nlp: Number of layers on a path + - dc: Domain Count metric + - bnc: Border Node Count metric + + +.. clicmd:: objective-function OBJFUN1 [required] + + Delete or specify a PCEP objective function constraint for a dynamic + candidate path. + + The possible functions are: + - mcp: Minimum Cost Path [RFC5541] + - mlp: Minimum Load Path [RFC5541] + - mbp: Maximum residual Bandwidth Path [RFC5541] + - mbc: Minimize aggregate Bandwidth Consumption [RFC5541] + - mll: Minimize the Load of the most loaded Link [RFC5541] + - mcc: Minimize the Cumulative Cost of a set of paths [RFC5541] + - spt: Shortest Path Tree [RFC8306] + - mct: Minimum Cost Tree [RFC8306] + - mplp: Minimum Packet Loss Path [RFC8233] + - mup: Maximum Under-Utilized Path [RFC8233] + - mrup: Maximum Reserved Under-Utilized Path [RFC8233] + - mtd: Minimize the number of Transit Domains [RFC8685] + - mbn: Minimize the number of Border Nodes [RFC8685] + - mctd: Minimize the number of Common Transit Domains [RFC8685] + - msl: Minimize the number of Shared Links [RFC8800] + - mss: Minimize the number of Shared SRLGs [RFC8800] + - msn: Minimize the number of Shared Nodes [RFC8800] + + +.. clicmd:: debug pathd pcep [basic|path|message|pceplib] + + Enable or disable debugging for the pcep module: + + - basic: Enable basic PCEP logging + - path: Log the path structures + - message: Log the PCEP messages + - pceplib: Enable pceplib logging + + +.. clicmd:: pcep + + Configure PCEP support. + + +.. clicmd:: pce-config NAME + + Define a shared PCE configuration that can be used in multiple PCE + declarations. + + +.. clicmd:: pce NAME + + Define or delete a PCE definition. + + +.. clicmd:: config WORD + + Select a shared configuration. If not defined, the default + configuration will be used. + + +.. clicmd:: address <ip A.B.C.D | ipv6 X:X::X:X> [port (1024-65535)] + + Define the address and port of the PCE. + + If not specified, the port is the standard PCEP port 4189. + + This should be specified in the PCC peer definition. + + +.. clicmd:: source-address [ip A.B.C.D | ipv6 X:X::X:X] [port PORT] + + Define the address and/or port of the PCC as seen by the PCE. + This can be used in a configuration group or a PCC peer declaration. + + If not specified, the source address will be the router identifier selected + by zebra, and the port will be the standard PCEP port 4189. + + This can be specified in either the PCC peer definition or in a + configuration group. + + +.. clicmd:: tcp-md5-auth WORD + + Enable TCP MD5 security with the given secret. + + This can be specified in either the PCC peer definition or in a + configuration group. + + +.. clicmd:: sr-draft07 + + Specify if a PCE only support segment routing draft 7, this flag will limit + the PCC behavior to this draft. + + This can be specified in either the PCC peer definition or in a + configuration group. + + +.. clicmd:: pce-initiated + + Specify if PCE-initiated LSP should be allowed for this PCE. + + This can be specified in either the PCC peer definition or in a + configuration group. + + +.. clicmd:: timer [keep-alive (1-63)] [min-peer-keep-alive (1-255)] [max-peer-keep-alive (1-255)] [dead-timer (4-255)] [min-peer-dead-timer (4-255)] [max-peer-dead-timer (4-255)] [pcep-request (1-120)] [session-timeout-interval (1-120)] [delegation-timeout (1-60)] + + Specify the PCEP timers. + + This can be specified in either the PCC peer definition or in a + configuration group. + + +.. clicmd:: pcc + + Disable or start the definition of a PCC. + + +.. clicmd:: msd (1-32) + + Specify the maximum SID depth in a PCC definition. + + +.. clicmd:: peer WORD [precedence (1-255)] + + Specify a peer and its precedence in a PCC definition. + +Debugging +--------- + +.. clicmd:: debug pathd policy + + Enable or disable Pathd policy information. + +Introspection Commands +---------------------- + +.. clicmd:: show sr-te policy [detail] + + Display the segment routing policies. + +.. code-block:: frr + + router# show sr-te policy + + Endpoint Color Name BSID Status + ------------------------------------------ + 192.0.2.1 1 default 4000 Active + + +.. code-block:: frr + + router# show sr-te policy detail + + Endpoint: 192.0.2.1 Color: 1 Name: LOW_DELAY BSID: 4000 Status: Active + Preference: 100 Name: cand1 Type: explicit Segment-List: sl1 Protocol-Origin: Local + * Preference: 200 Name: cand1 Type: dynamic Segment-List: 32453452 Protocol-Origin: PCEP + +The asterisk (*) marks the best, e.g. active, candidate path. Note that for segment-lists which are +retrieved via PCEP a random number based name is generated. + + +.. clicmd:: show sr-te pcep counters + + Display the counters from pceplib. + + +.. clicmd:: show sr-te pcep pce-config [NAME] + + Display a shared configuration. if no name is specified, the default + configuration will be displayed. + + +.. clicmd:: show sr-te pcep pcc + + Display PCC information. + + +.. clicmd:: show sr-te pcep session [NAME] + + Display the information of a PCEP session, if not name is specified all the + sessions will be displayed. + + +Utility Commands +---------------- + +.. clicmd:: clear sr-te pcep session [NAME] + + Reset the pcep session by disconnecting from the PCE and performing the + normal reconnection process. No configuration is changed. + + +Usage with BGP route-maps +========================= + +It is possible to steer traffic 'into' a segment routing policy for routes +learned through BGP using route-maps: + +.. code-block:: frr + + route-map SET_SR_POLICY permit 10 + set sr-te color 1 + ! + router bgp 1 + bgp router-id 192.0.2.2 + neighbor 192.0.2.1 remote-as 1 + neighbor 192.0.2.1 update-source lo + ! + address-family ipv4 unicast + neighbor 192.0.2.1 next-hop-self + neighbor 192.0.2.1 route-map SET_SR_POLICY in + redistribute static + exit-address-family + ! + ! + +In this case, the SR Policy with color `1` and endpoint `192.0.2.1` is selected. + + +Sample configuration +==================== + +.. code-block:: frr + + ! Default pathd configuration sample + ! + password frr + log stdout + + segment-routing + traffic-eng + segment-list test1 + index 10 mpls label 123 + index 20 mpls label 456 + ! + segment-list test2 + index 10 mpls label 321 + index 20 mpls label 654 + ! + policy color 1 endpoint 192.0.2.1 + name one + binding-sid 100 + candidate-path preference 100 name test1 explicit segment-list test1 + candidate-path preference 200 name test2 explicit segment-list test2 + ! + policy color 2 endpoint 192.0.2.2 + name two + binding-sid 101 + candidate-path preference 100 name def explicit segment-list test2 + candidate-path preference 200 name dyn dynamic + bandwidth 12345 + metric bound abc 16 required + metric te 10 + ! + ! + pcep + pcc-peer PCE1 + address ip 127.0.0.1 + sr-draft07 + ! + pcc + peer PCE1 + ! + ! + ! + diff --git a/doc/user/pbr.rst b/doc/user/pbr.rst new file mode 100644 index 0000000..7a4effd --- /dev/null +++ b/doc/user/pbr.rst @@ -0,0 +1,426 @@ +.. _pbr: + +*** +PBR +*** + +:abbr:`PBR` is Policy Based Routing, which means forwarding based on +packet fields other than solely the destination IP address. +This implementation currently works only on Linux. Note that some +functionality (VLAN matching, packet mangling) is not supported by +the default Linux kernel dataplane provider. + +.. _starting-pbr: + +Starting PBR +============ + +Default configuration file for *pbrd* is :file:`pbrd.conf`. The typical +location of :file:`pbrd.conf` is |INSTALL_PREFIX_ETC|/pbrd.conf. + +If FRR is using integrated config, then :file:`pbrd.conf` need not be +present and the :file:`frr.conf` is read instead. + +.. program:: pbrd + +:abbr:`PBR` supports all the common FRR daemon start options, which are +documented elsewhere. + +.. _nexthop-groups: + +PBR Nexthop Groups +================== + +A nexthop group is a list of ECMP nexthops used to forward packets +when a pbr-map is matched. +For details on specifying a nexthop group in the CLI, see +the nexthop-groups section. + +Showing Nexthop Group Information +--------------------------------- + +.. clicmd:: show pbr nexthop-groups [NAME] [json] + + Display information on a PBR nexthop-group. If ``NAME`` is omitted, all + nexthop groups are shown. Setting ``json`` will provide the same + information in an array of objects that adhere to the schema below: + + +-----------+----------------------------+---------+ + | Key | Description | Type | + +===========+============================+=========+ + | id | Unique ID | Integer | + +-----------+----------------------------+---------+ + | name | Name of this group | String | + +-----------+----------------------------+---------+ + | valid | Is this group well-formed? | Boolean | + +-----------+----------------------------+---------+ + | installed | ... and is it installed? | Boolean | + +-----------+----------------------------+---------+ + | nexthops | Nexthops within this group | Array | + +-----------+----------------------------+---------+ + + Each element within ``nexthops`` describes a single target within this + group, and its structure is described by the JSON below: + + +---------+------------------------------+---------+ + | Key | Description | Type | + +=========+==============================+=========+ + | nexthop | Name of this nexthop | String | + +---------+------------------------------+---------+ + | valid | Is this nexthop well-formed? | Boolean | + +---------+------------------------------+---------+ + +.. _pbr-maps: + +PBR Maps +======== + +PBR maps are a way to specify a set of rules that are applied to +packets received on individual interfaces. +If a received packet matches a rule, the rule's nexthop-group or +nexthop is used to forward it; any other actions +specified in the rule are also applied to the packet. + +.. clicmd:: pbr-map NAME seq (1-700) + + Create a pbr-map rule with map NAME and specified sequence number. + This command puts the CLI into a new submode for pbr-map rule specification. + To exit this submode, type ``exit`` or ``end``. + +.. clicmd:: match src-ip PREFIX + + Match the packet's source IP address. + + This command accepts both v4 and v6 prefixes. + +.. clicmd:: match dst-ip PREFIX + + Match the packet's destination IP address. + + This command accepts both v4 and v6 prefixes. + +.. clicmd:: match src-port (1-65535) + + Match the packet's UDP or TCP source port. + +.. clicmd:: match dst-port (1-65535) + + Match the packet's UDP or TCP destination port. + +.. clicmd:: match ip-protocol PROTOCOL + + Match the packet's IP protocol. + + Protocol names are queried from the protocols database (``/etc/protocols``; + see ``man 5 protocols`` and ``man 3 getprotobyname``). + +.. clicmd:: match mark (1-4294967295) + + Match the packet's meta-information mark. + The mark value is attached to the packet by the kernel/dataplane and + is platform-specific. + Currently, this field is supported only on linux and corresponds to + the underlying `ip rule .... fwmark XXXX` command. + +.. clicmd:: match dscp (DSCP|0-63) + + Match the packet's IP differentiated services code point (DSCP). + The specified DSCP may also be a standard name for a + differentiated service code point such as ``cs0`` or ``af11``. + + You may only specify one dscp per route map rule; to match on multiple + dscp values you will need to create several rules, one for each value. + +.. clicmd:: match ecn (0-3) + + Match the packet's IP explicit congestion notification (ECN) field. + +.. clicmd:: match pcp (0-7) + + Match the packet's 802.1Q Priority Code Point. + Zero is the default (nominally, "best effort"). + The Linux kernel dataplane provider does not currently support + matching PCPs, + so this field will be ignored unless other dataplane providers are used. + +.. clicmd:: match vlan (1-4094) + + Match the packet's VLAN (802.1Q) identifier. + Note that VLAN IDs 0 and 4095 are reserved. + The Linux kernel dataplane provider does not currently support + VLAN-matching facilities, + so this field will be ignored unless other dataplane providers are used. + +.. clicmd:: match vlan (tagged|untagged|untagged-or-zero) + + Match packets according to whether or not they have a VLAN tag. + Use `untagged-or-zero` to also match packets with either no VLAN tag + or with the reserved VLAN ID of 0 (indicating an untagged frame that + includes other 802.1Q fields). + The Linux kernel dataplane provider does not currently support + VLAN-matching facilities, + so this field will be ignored unless other dataplane providers are used. + +.. clicmd:: set nexthop-group NAME + + Action: + forward the packet using nexthop-group NAME. + +.. clicmd:: set nexthop [A.B.C.D|X:X::X:XX|blackhole] [interface] [nexthop-vrf NAME] + + Action: + forward the packet using the specified single nexthop. + If `blackhole`, packets will be sent to a blackhole route and dropped. + +.. clicmd:: set vrf unchanged|NAME + + Action: + If set to ``unchanged``, the rule will use the vrf table the interface + is in as its lookup. + If set to NAME, the rule will use that vrf table as its lookup. + + Not supported with NETNS VRF backend. + +.. clicmd:: set queue-id (1-65535) + + Action: + set the egress port queue identifier. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set pcp (0-7) + + Action: + set the 802.1Q priority code point (PCP). + A PCP of zero is the default (nominally, "best effort"). + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set vlan (1-4094) + + Action: + set the VLAN tag. Identifiers 0 and 4095 are reserved. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: strip vlan + + Action: + strip inner vlan tags. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + It is invalid to specify both a `strip` and `set vlan` action. + +.. clicmd:: set src-ip [A.B.C.D/M|X:X::X:X/M] + + Action: + Set the source IP address of matched packets, possibly using a mask `M`. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set dst-ip [A.B.C.D/M|X:X::X:X/M] + + Action: + set the destination IP address of matched packets, possibly using a mask + `M`. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set src-port (1-65535) + + Action: + set the source port of matched packets. Note that this action only makes + sense with layer 4 protocols that use ports, such as TCP, UDP, and SCTP. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set dst-port (1-65535) + + Action: + set the destination port of matched packets. Note that this action only + makes sense with layer 4 protocols that use ports, such as TCP, UDP, and + SCTP. + The Linux Kernel dataplane provider does not currently support + packet mangling, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set dscp DSCP + + Action: + set the differentiated services code point (DSCP) of matched packets. + The Linux Kernel dataplane provider does not currently support + this action, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: set ecn (0-3) + + Action: + set the explicit congestion notification (ECN) of matched packets. + The Linux Kernel dataplane provider does not currently support + this action, + so this field will be ignored unless another dataplane provider is used. + +.. clicmd:: show pbr map [NAME] [detail] [json] + + Display pbr maps either all or by ``NAME``. If ``detail`` is set, it will + give information about each rule's unique internal ID and some extra + debugging information about install state for the nexthop/nexthop group. + Setting ``json`` will provide the same information in an array of objects + that adher to the schema below: + + +----------+--------------------------------+---------+ + | Key | Description | Type | + +==========+================================+=========+ + | name | Map name | String | + +----------+--------------------------------+---------+ + | valid | Is the map well-formed? | Boolean | + +----------+--------------------------------+---------+ + | policies | Rules to match packets against | Array | + +----------+--------------------------------+---------+ + + Each element of the ``policies`` array is composed of a set of objects + representing the policies associated with this map. Each policy is + described below (not all fields are required): + + +-----------------+-------------------------------------------+---------+ + | Key | Description | Type | + +=================+===========================================+=========+ + | id | Unique ID | Integer | + +-----------------+-------------------------------------------+---------+ + | sequenceNumber | Order of this policy within the map | Integer | + +-----------------+-------------------------------------------+---------+ + | ruleNumber | Rule number to install into | Integer | + +-----------------+-------------------------------------------+---------+ + | vrfUnchanged | Use interface's VRF | Boolean | + +-----------------+-------------------------------------------+---------+ + | installed | Is this policy installed? | Boolean | + +-----------------+-------------------------------------------+---------+ + | installedReason | Why (or why not?) | String | + +-----------------+-------------------------------------------+---------+ + | matchSrc | Match packets with this source address | String | + +-----------------+-------------------------------------------+---------+ + | matchDst | ... or with this destination address | String | + +-----------------+-------------------------------------------+---------+ + | matchMark | ... or with this marker | Integer | + +-----------------+-------------------------------------------+---------+ + | vrfName | Associated VRF (if relevant) | String | + +-----------------+-------------------------------------------+---------+ + | nexthopGroup | This policy's nexthop group (if relevant) | Object | + +-----------------+-------------------------------------------+---------+ + + Finally, the ``nexthopGroup`` object above contains information FRR + knows about the configured nexthop for this policy: + + +---------------------+--------------------------------------+---------+ + | Key | Description | Type | + +=====================+======================================+=========+ + | tableId | Nexthop table ID | Integer | + +---------------------+--------------------------------------+---------+ + | name | Name of the nexthop group | String | + +---------------------+--------------------------------------+---------+ + | installed | Is this nexthop group installed? | Boolean | + +---------------------+--------------------------------------+---------+ + | installedInternally | Does FRR think NHG is installed? | Integer | + +---------------------+--------------------------------------+---------+ + + +.. index:: + pair: policy; PBR + +.. _pbr-policy: + +PBR Policy +========== + +After you have specified a PBR map, in order for it to be enabled, it must +be applied to an interface. This policy application to an interface +causes the policy to be installed into the kernel. + +.. clicmd:: pbr-policy NAME + + This command is available under interface sub-mode. + It enables the PBR map NAME on the interface. + +.. note:: + This command will not dynamically create PBR maps on sub-interfaces + (i.e. vlans), even if one is on the master. + Each sub-interface must have the PBR map enabled explicitly. + +.. clicmd:: show pbr interface [NAME] [json] + + Enumerates all interfaces which ``pbrd`` is keeping track of. Passing + ``json`` will return an array of interfaces; each returned interface will + adhere to the JSON schema below: + + +--------+----------------------------+---------+ + | Key | Description | Type | + +========+============================+=========+ + | name | Interface name | String | + +--------+----------------------------+---------+ + | index | Device Index | Integer | + +--------+----------------------------+---------+ + | policy | PBR map for this interface | String | + +--------+----------------------------+---------+ + | valid | Is the map well-formed? | Boolean | + +--------+----------------------------+---------+ + +.. clicmd:: pbr table range (10000-4294966272) (10000-4294966272) + + Set or unset the range used to assign numeric table IDs to new + nexthop-group tables. Existing tables will not be modified to fit in this + range, so this range should be configured before adding nexthop groups. + + .. seealso:: :ref:`pbr-details` + + +.. _pbr-debugs: + +PBR Debugs +=========== + +.. clicmd:: debug pbr events|map|nht|zebra + + Debug pbr in pbrd daemon. You must specify what types of debugs to turn on. + +.. _pbr-details: + +PBR Details +=========== + +Internally, a PBR map is translated into two separate constructs in the +Linux kernel. + + +The PBR map creates an `ip rule ...` that is inserted into the Linux +kernel that points to a table to use for forwarding once the rule matches. + + +The creation of a nexthop or nexthop-group is translated to a +table with a default route having the specified nexthop(s). + + +Sample configuration +==================== + +.. code-block:: frr + + nexthop-group TEST + nexthop 4.5.6.7 + nexthop 5.6.7.8 + ! + pbr-map BLUE seq 100 + match dst-ip 9.9.9.0/24 + match src-ip 10.10.10.0/24 + set nexthop-group TEST + ! + int swp1 + pbr-policy BLUE + + diff --git a/doc/user/pim.rst b/doc/user/pim.rst new file mode 100644 index 0000000..d70c3c0 --- /dev/null +++ b/doc/user/pim.rst @@ -0,0 +1,754 @@ +.. _pim: + +*** +PIM +*** + +PIM -- Protocol Independent Multicast + +*pimd* supports pim-sm as well as igmp v2 and v3. pim is +vrf aware and can work within the context of vrf's in order to +do S,G mrouting. Additionally PIM can be used in the EVPN underlay +network for optimizing forwarding of overlay BUM traffic. + +.. note:: + + On Linux for PIM-SM operation you *must* have kernel version 4.19 or greater. + To use PIM for EVPN BUM forwarding, kernels 5.0 or greater are required. + OpenBSD has no multicast support and FreeBSD, and NetBSD only + have support for SSM. + +.. _starting-and-stopping-pimd: + +Starting and Stopping pimd +========================== + +The default configuration file name of *pimd*'s is :file:`pimd.conf`. When +invoked *pimd* searches directory |INSTALL_PREFIX_ETC|. If +:file:`pimd.conf` is not there then next search current directory. + +*pimd* requires zebra for proper operation. Additionally *pimd* depends on +routing properly setup and working in the network that it is working on. + +:: + + # zebra -d + # pimd -d + + +Please note that *zebra* must be invoked before *pimd*. + +To stop *pimd* please use:: + + kill `cat /var/run/frr/pimd.pid` + +Certain signals have special meanings to *pimd*. + ++---------+---------------------------------------------------------------------+ +| Signal | Meaning | ++=========+=====================================================================+ +| SIGUSR1 | Rotate the *pimd* logfile | ++---------+---------------------------------------------------------------------+ +| SIGINT | *pimd* sweeps all installed PIM mroutes then terminates gracefully. | +| SIGTERM | | ++---------+---------------------------------------------------------------------+ + +*pimd* invocation options. Common options that can be specified +(:ref:`common-invocation-options`). + +.. clicmd:: ip pim rp A.B.C.D A.B.C.D/M + + In order to use pim, it is necessary to configure a RP for join messages to + be sent to. Currently the only methodology to do this is via static rp + commands. All routers in the pim network must agree on these values. The + first ip address is the RP's address and the second value is the matching + prefix of group ranges covered. This command is vrf aware, to configure for + a vrf, enter the vrf submode. + +.. clicmd:: ip pim rp keep-alive-timer (1-65535) + + Modify the time out value for a S,G flow from 1-65535 seconds at RP. + The normal keepalive period for the KAT(S,G) defaults to 210 seconds. + However, at the RP, the keepalive period must be at least the + Register_Suppression_Time, or the RP may time out the (S,G) state + before the next Null-Register arrives. Thus, the KAT(S,G) is set to + max(Keepalive_Period, RP_Keepalive_Period) when a Register-Stop is sent. + If choosing a value below 31 seconds be aware that some hardware platforms + cannot see data flowing in better than 30 second chunks. This command is + vrf aware, to configure for a vrf, enter the vrf submode. + +.. clicmd:: ip pim register-accept-list PLIST + + When pim receives a register packet the source of the packet will be compared + to the prefix-list specified, PLIST, and if a permit is received normal + processing continues. If a deny is returned for the source address of the + register packet a register stop message is sent to the source. + +.. clicmd:: ip pim spt-switchover infinity-and-beyond [prefix-list PLIST] + + On the last hop router if it is desired to not switch over to the SPT tree + configure this command. Optional parameter prefix-list can be use to control + which groups to switch or not switch. If a group is PERMIT as per the + PLIST, then the SPT switchover does not happen for it and if it is DENY, + then the SPT switchover happens. + This command is vrf aware, to configure for a vrf, + enter the vrf submode. + +.. clicmd:: ip pim ecmp + + If pim has the a choice of ECMP nexthops for a particular RPF, pim will + cause S,G flows to be spread out amongst the nexthops. If this command is + not specified then the first nexthop found will be used. This command is vrf + aware, to configure for a vrf, enter the vrf submode. + +.. clicmd:: ip pim ecmp rebalance + + If pim is using ECMP and an interface goes down, cause pim to rebalance all + S,G flows across the remaining nexthops. If this command is not configured + pim only modifies those S,G flows that were using the interface that went + down. This command is vrf aware, to configure for a vrf, enter the vrf + submode. + +.. clicmd:: ip pim join-prune-interval (1-65535) + + Modify the join/prune interval that pim uses to the new value. Time is + specified in seconds. This command is vrf aware, to configure for a vrf, + enter the vrf submode. The default time is 60 seconds. If you enter + a value smaller than 60 seconds be aware that this can and will affect + convergence at scale. + +.. clicmd:: ip pim keep-alive-timer (1-65535) + + Modify the time out value for a S,G flow from 1-65535 seconds. If choosing + a value below 31 seconds be aware that some hardware platforms cannot see data + flowing in better than 30 second chunks. This command is vrf aware, to + configure for a vrf, enter the vrf submode. + +.. clicmd:: ip pim packets (1-255) + + When processing packets from a neighbor process the number of packets + incoming at one time before moving on to the next task. The default value is + 3 packets. This command is only useful at scale when you can possibly have + a large number of pim control packets flowing. This command is vrf aware, to + configure for a vrf, enter the vrf submode. + +.. clicmd:: ip pim register-suppress-time (1-65535) + + Modify the time that pim will register suppress a FHR will send register + notifications to the kernel. This command is vrf aware, to configure for a + vrf, enter the vrf submode. + +.. clicmd:: ip pim send-v6-secondary + + When sending pim hello packets tell pim to send any v6 secondary addresses + on the interface. This information is used to allow pim to use v6 nexthops + in it's decision for RPF lookup. This command is vrf aware, to configure for + a vrf, enter the vrf submode. + +.. clicmd:: ip pim ssm prefix-list WORD + + Specify a range of group addresses via a prefix-list that forces pim to + never do SM over. This command is vrf aware, to configure for a vrf, enter + the vrf submode. + +.. clicmd:: ip multicast rpf-lookup-mode WORD + + Modify how PIM does RPF lookups in the zebra routing table. You can use + these choices: + + longer-prefix + Lookup the RPF in both tables using the longer prefix as a match + + lower-distance + Lookup the RPF in both tables using the lower distance as a match + + mrib-only + Lookup in the Multicast RIB only + + mrib-then-urib + Lookup in the Multicast RIB then the Unicast Rib, returning first found. + This is the default value for lookup if this command is not entered + + urib-only + Lookup in the Unicast Rib only. + +.. clicmd:: ip igmp generate-query-once [version (2-3)] + + Generate IGMP query (v2/v3) on user requirement. This will not depend on + the existing IGMP general query timer.If no version is provided in the cli, + the default will be the igmp version enabled on that interface. + +.. clicmd:: ip igmp watermark-warn (1-65535) + + Configure watermark warning generation for an igmp group limit. Generates + warning once the configured group limit is reached while adding new groups. + 'no' form of the command disables the warning generation. This command is + vrf aware. To configure per vrf, enter vrf submode. + +.. _pim-interface-configuration: + +PIM Interface Configuration +=========================== + +PIM interface commands allow you to configure an interface as either a Receiver +or a interface that you would like to form pim neighbors on. If the interface +is in a vrf, enter the interface command with the vrf keyword at the end. + +.. clicmd:: ip pim active-active + + Turn on pim active-active configuration for a Vxlan interface. This + command will not do anything if you do not have the underlying ability + of a mlag implementation. + +.. clicmd:: ip pim bsm + + Tell pim that we would like to use this interface to process bootstrap + messages. This is enabled by default. 'no' form of this command is used to + restrict bsm messages on this interface. + +.. clicmd:: ip pim unicast-bsm + + Tell pim that we would like to allow interface to process unicast bootstrap + messages. This is enabled by default. 'no' form of this command is used to + restrict processing of unicast bsm messages on this interface. + +.. clicmd:: ip pim drpriority (1-4294967295) + + Set the DR Priority for the interface. This command is useful to allow the + user to influence what node becomes the DR for a lan segment. + +.. clicmd:: ip pim hello (1-65535) (1-65535) + + Set the pim hello and hold interval for a interface. + +.. clicmd:: ip pim + + Tell pim that we would like to use this interface to form pim neighbors + over. Please note that this command does not enable the reception of IGMP + reports on the interface. Refer to the next `ip igmp` command for IGMP + management. + +.. clicmd:: ip pim use-source A.B.C.D + + If you have multiple addresses configured on a particular interface + and would like pim to use a specific source address associated with + that interface. + +.. clicmd:: ip pim passive + + Disable sending and receiving pim control packets on the interface. + +.. clicmd:: ip igmp + + Tell pim to receive IGMP reports and Query on this interface. The default + version is v3. This command is useful on a LHR. + +.. clicmd:: ip igmp join A.B.C.D [A.B.C.D] + + Join multicast group or source-group on an interface. + +.. clicmd:: ip igmp query-interval (1-65535) + + Set the IGMP query interval that PIM will use. + +.. clicmd:: ip igmp query-max-response-time (1-65535) + + Set the IGMP query response timeout value. If an report is not returned in + the specified time we will assume the S,G or \*,G has timed out. + +.. clicmd:: ip igmp version (2-3) + + Set the IGMP version used on this interface. The default value is 3. + +.. clicmd:: ip multicast boundary oil WORD + + Set a pim multicast boundary, based upon the WORD prefix-list. If a pim join + or IGMP report is received on this interface and the Group is denied by the + prefix-list, PIM will ignore the join or report. + +.. clicmd:: ip igmp last-member-query-count (1-255) + + Set the IGMP last member query count. The default value is 2. 'no' form of + this command is used to to configure back to the default value. + +.. clicmd:: ip igmp last-member-query-interval (1-65535) + + Set the IGMP last member query interval in deciseconds. The default value is + 10 deciseconds. 'no' form of this command is used to to configure back to the + default value. + +.. clicmd:: ip mroute INTERFACE A.B.C.D [A.B.C.D] + + Set a static multicast route for a traffic coming on the current interface to + be forwarded on the given interface if the traffic matches the group address + and optionally the source address. + + +.. seealso:: + + :ref:`bfd-pim-peer-config` + + +.. _pim-multicast-rib: + +PIM Multicast RIB +================= + +In order to influence Multicast RPF lookup, it is possible to insert +into zebra routes for the Multicast RIB. These routes are only +used for RPF lookup and will not be used by zebra for insertion +into the kernel *or* for normal rib processing. As such it is +possible to create weird states with these commands. Use with +caution. Most of the time this will not be necessary. + +.. clicmd:: ip mroute A.B.C.D/M A.B.C.D (1-255) + + Insert into the Multicast Rib Route A.B.C.D/M with specified nexthop. The + distance can be specified as well if desired. + +.. clicmd:: ip mroute A.B.C.D/M INTERFACE (1-255) + + Insert into the Multicast Rib Route A.B.C.D/M using the specified INTERFACE. + The distance can be specified as well if desired. + +.. _msdp-configuration: + +Multicast Source Discovery Protocol (MSDP) Configuration +======================================================== + +MSDP can be setup in different ways: + +* MSDP meshed-group: where all peers are connected with each other creating + a fully meshed network. SAs (source active) messages are not forwarded in + this mode because the origin is able to send SAs to all members. + + This setup is commonly used with anycast. + +* MSDP peering: when there is one or more peers that are not fully meshed. SAs + may be forwarded depending on the result of filtering and RPF checks. + + This setup is commonly consistent with BGP peerings (for RPF checks). + +* MSDP default peer: there is only one peer and all SAs will be forwarded + there. + +.. note:: + + MSDP default peer and SA filtering is not implemented. + + +Commands available for MSDP: + +.. clicmd:: ip msdp timers (1-65535) (1-65535) [(1-65535)] + + Configure global MSDP timers. + + First value is the keep-alive interval. This configures the interval in + seconds between keep-alive messages. The default value is 60 seconds. It + should be less than the remote hold time. + + Second value is the hold-time. This configures the interval in seconds before + closing a non responding connection. The default value is 75. This value + should be greater than the remote keep alive time. + + Third value is the connection retry interval and it is optional. This + configures the interval between connection attempts. The default value + is 30 seconds. + +.. clicmd:: ip msdp mesh-group WORD member A.B.C.D + + Create or update a mesh group to include the specified MSDP peer. + +.. clicmd:: ip msdp mesh-group WORD source A.B.C.D + + Create or update a mesh group to set the source address used to connect to + peers. + +.. clicmd:: ip msdp peer A.B.C.D source A.B.C.D + + Create a regular MSDP session with peer using the specified source address. + + +.. _show-pim-information: + +Show PIM Information +==================== + +All PIM show commands are vrf aware and typically allow you to insert a +specified vrf command if information is desired about a specific vrf. If no +vrf is specified then the default vrf is assumed. Finally the special keyword +'all' allows you to look at all vrfs for the command. Naming a vrf 'all' will +cause great confusion. + +.. clicmd:: show ip igmp interface + + Display IGMP interface information. + +.. clicmd:: show ip igmp [vrf NAME] join [json] + + Display IGMP static join information for a specific vrf. + +.. index:: show ip igmp [vrf NAME$vrf_name] groups [INTERFACE$ifname [GROUP$grp_str]] [detail] [json$json] +.. clicmd:: show ip igmp [vrf NAME$vrf_name] groups [INTERFACE$ifname [GROUP$grp_str]] [detail] [json$json] + + Display IGMP static join information for all the vrfs present. + +.. index:: show ip igmp vrf all groups [GROUP$grp_str] [detail$detail] [json$json] +.. clicmd:: show ip igmp vrf all groups [GROUP$grp_str] [detail$detail] [json$json] + + Display IGMP groups information. + +.. clicmd:: show ip igmp groups retransmissions + + Display IGMP group retransmission information. + +.. clicmd:: show ip igmp [vrf NAME] sources [json] + + Display IGMP sources information. + +.. clicmd:: show ip igmp sources retransmissions + + Display IGMP source retransmission information. + +.. clicmd:: show ip igmp statistics + + Display IGMP statistics information. + +.. clicmd:: show ip multicast + + Display various information about the interfaces used in this pim instance. + +.. clicmd:: show ip mroute [vrf NAME] [A.B.C.D [A.B.C.D]] [fill] [json] + + Display information about installed into the kernel S,G mroutes. If + one address is specified we assume it is the Group we are interested + in displaying data on. If the second address is specified then it is + Source Group. The keyword `fill` says to fill in all assumed data + for test/data gathering purposes. + +.. clicmd:: show ip mroute [vrf NAME] count [json] + + Display information about installed into the kernel S,G mroutes and in + addition display data about packet flow for the mroutes for a specific + vrf. + +.. clicmd:: show ip mroute vrf all count [json] + + Display information about installed into the kernel S,G mroutes and in + addition display data about packet flow for the mroutes for all vrfs. + +.. clicmd:: show ip mroute [vrf NAME] summary [json] + + Display total number of S,G mroutes and number of S,G mroutes installed + into the kernel for a specific vrf. + +.. clicmd:: show ip mroute vrf all summary [json] + + Display total number of S,G mroutes and number of S,G mroutes + installed into the kernel for all vrfs. + +.. clicmd:: show ip msdp mesh-group + + Display the configured mesh-groups, the local address associated with each + mesh-group, the peer members included in each mesh-group, and their status. + +.. clicmd:: show ip msdp peer + + Display information about the MSDP peers. That includes the peer address, + the local address used to establish the connection to the peer, the + connection status, and the number of active sources. + +.. clicmd:: show ip pim assert + + Display information about asserts in the PIM system for S,G mroutes. + This command does not show S,G Channel states that in a NOINFO state. + +.. clicmd:: show ip pim assert-internal + + Display internal assert state for S,G mroutes + +.. clicmd:: show ip pim assert-metric + + Display metric information about assert state for S,G mroutes + +.. clicmd:: show ip pim assert-winner-metric + + Display winner metric for assert state for S,G mroutes + +.. clicmd:: show ip pim group-type + + Display SSM group ranges. + +.. clicmd:: show ip pim interface + + Display information about interfaces PIM is using. + +.. clicmd:: show ip pim mlag [vrf NAME|all] interface [detail|WORD] [json] + + Display mlag interface information. + +.. clicmd:: show ip pim join + + Display information about PIM joins received. If one address is specified + then we assume it is the Group we are interested in displaying data on. + If the second address is specified then it is Source Group. + +.. clicmd:: show ip pim local-membership + + Display information about PIM interface local-membership. + +.. clicmd:: show ip pim mlag summary [json] + + Display mlag information state that PIM is keeping track of. + +.. clicmd:: show ip pim neighbor + + Display information about PIM neighbors. + +.. clicmd:: show ip pim nexthop + + Display information about pim nexthops that are being used. + +.. clicmd:: show ip pim nexthop-lookup + + Display information about a S,G pair and how the RPF would be chosen. This + is especially useful if there are ECMP's available from the RPF lookup. + +.. clicmd:: show ip pim [vrf NAME] rp-info [A.B.C.D/M] [json] + + Display information about RP's that are configured on this router. + + You can filter the output by specifying an arbitrary group range. + + .. code-block:: frr + + # show ip pim rp-info + RP address group/prefix-list OIF I am RP Source Group-Type + 192.168.10.123 225.0.0.0/24 eth2 yes Static ASM + 192.168.10.123 239.0.0.0/8 eth2 yes Static ASM + 192.168.10.123 239.4.0.0/24 eth2 yes Static SSM + + # show ip pim rp-info 239.4.0.0/25 + RP address group/prefix-list OIF I am RP Source Group-Type + 192.168.10.123 239.0.0.0/8 eth2 yes Static ASM + 192.168.10.123 239.4.0.0/24 eth2 yes Static SSM + +.. clicmd:: show ip pim rpf + + Display information about currently being used S,G's and their RPF lookup + information. Additionally display some statistics about what has been + happening on the router. + +.. clicmd:: show ip pim secondary + + Display information about an interface and all the secondary addresses + associated with it. + +.. clicmd:: show ip pim state + + Display information about known S,G's and incoming interface as well as the + OIL and how they were chosen. + +.. clicmd:: show ip pim [vrf NAME] upstream [A.B.C.D [A.B.C.D]] [json] + + Display upstream information about a S,G mroute. Allow the user to + specify sub Source and Groups that we are only interested in. + +.. clicmd:: show ip pim upstream-join-desired + + Display upstream information for S,G's and if we desire to + join the multicast tree + +.. clicmd:: show ip pim upstream-rpf + + Display upstream information for S,G's and the RPF data associated with them. + +.. clicmd:: show ip pim [vrf NAME] mlag upstream [A.B.C.D [A.B.C.D]] [json] + + Display upstream entries that are synced across MLAG switches. + Allow the user to specify sub Source and Groups address filters. + +.. clicmd:: show ip pim mlag summary + + Display PIM MLAG (multi-chassis link aggregation) session status and + control message statistics. + +.. clicmd:: show ip pim bsr + + Display current bsr, its uptime and last received bsm age. + +.. clicmd:: show ip pim bsrp-info + + Display group-to-rp mappings received from E-BSR. + +.. clicmd:: show ip pim bsm-database + + Display all fragments of stored bootstrap message in user readable format. + +.. clicmd:: mtrace A.B.C.D [A.B.C.D] + + Display multicast traceroute towards source, optionally for particular group. + +.. clicmd:: show ip multicast count [vrf NAME] [json] + + Display multicast data packets count per interface for a vrf. + +.. clicmd:: show ip multicast count vrf all [json] + + Display multicast data packets count per interface for all vrf. + + +.. seealso:: + + :ref:`multicast-rib-commands` + + +PIM Debug Commands +================== + +The debugging subsystem for PIM behaves in accordance with how FRR handles +debugging. You can specify debugging at the enable CLI mode as well as the +configure CLI mode. If you specify debug commands in the configuration cli +mode, the debug commands can be persistent across restarts of the FRR pimd if +the config was written out. + +.. clicmd:: debug igmp + + This turns on debugging for IGMP protocol activity. + +.. clicmd:: debug mtrace + + This turns on debugging for mtrace protocol activity. + +.. clicmd:: debug mroute + + This turns on debugging for PIM interaction with kernel MFC cache. + +.. clicmd:: debug pim events + + This turns on debugging for PIM system events. Especially timers. + +.. clicmd:: debug pim nht + + This turns on debugging for PIM nexthop tracking. It will display + information about RPF lookups and information about when a nexthop changes. + +.. clicmd:: debug pim nht detail + + This turns on debugging for PIM nexthop in detail. This is not enabled + by default. + +.. clicmd:: debug pim packet-dump + + This turns on an extraordinary amount of data. Each pim packet sent and + received is dumped for debugging purposes. This should be considered a + developer only command. + +.. clicmd:: debug pim packets + + This turns on information about packet generation for sending and about + packet handling from a received packet. + +.. clicmd:: debug pim trace + + This traces pim code and how it is running. + +.. clicmd:: debug pim bsm + + This turns on debugging for BSR message processing. + +.. clicmd:: debug pim zebra + + This gathers data about events from zebra that come up through the ZAPI. + +PIM Clear Commands +================== +Clear commands reset various variables. + +.. clicmd:: clear ip interfaces + + Reset interfaces. + +.. clicmd:: clear ip igmp interfaces + + Reset IGMP interfaces. + +.. clicmd:: clear ip mroute + + Reset multicast routes. + +.. clicmd:: clear ip mroute [vrf NAME] count + + When this command is issued, reset the counts of data shown for + packet count, byte count and wrong interface to 0 and start count + up from this spot. + +.. clicmd:: clear ip pim interfaces + + Reset PIM interfaces. + +.. clicmd:: clear ip pim oil + + Rescan PIM OIL (output interface list). + +.. clicmd:: clear ip pim [vrf NAME] bsr-data + + This command will clear the BSM scope data struct. This command also + removes the next hop tracking for the bsr and resets the upstreams + for the dynamically learnt RPs. + +PIM EVPN configuration +====================== +To use PIM in the underlay for overlay BUM forwarding associate a multicast +group with the L2 VNI. The actual configuration is based on your distribution. +Here is an ifupdown2 example:: + + auto vx-10100 + iface vx-10100 + vxlan-id 10100 + bridge-access 100 + vxlan-local-tunnelip 27.0.0.11 + vxlan-mcastgrp 239.1.1.100 + +.. note:: + + PIM will see the ``vxlan-mcastgrp`` configuration and auto configure state + to properly forward BUM traffic. + +PIM also needs to be configured in the underlay to allow the BUM MDT to be +setup. This is existing PIM configuration: + +- Enable pim on the underlay L3 interface via the "ip pim" command. +- Configure RPs for the BUM multicast group range. +- Ensure the PIM is enabled on the lo of the VTEPs and the RP. + + +Sample configuration +==================== + +.. code-block:: frr + + debug igmp + debug pim + debug pim zebra + + ! You may want to enable ssmpingd for troubleshooting + ! See http://www.venaas.no/multicast/ssmping/ + ! + ip ssmpingd 1.1.1.1 + ip ssmpingd 2.2.2.2 + + ! HINTS: + ! - Enable "ip pim ssm" on the interface directly attached to the + ! multicast source host (if this is the first-hop router) + ! - Enable "ip pim ssm" on pim-routers-facing interfaces + ! - Enable "ip igmp" on IGMPv3-hosts-facing interfaces + ! - In order to inject IGMPv3 local membership information in the + ! PIM protocol state, enable both "ip pim ssm" and "ip igmp" on + ! the same interface; otherwise PIM won't advertise + ! IGMPv3-learned membership to other PIM routers + + interface eth0 + ip pim ssm + ip igmp + diff --git a/doc/user/pimv6.rst b/doc/user/pimv6.rst new file mode 100644 index 0000000..8569390 --- /dev/null +++ b/doc/user/pimv6.rst @@ -0,0 +1,502 @@ +.. _pimv6: + +***** +PIMv6 +***** + +PIMv6 -- Protocol Independent Multicast for IPv6 + +*pim6d* supports pim-sm as well as MLD v1 and v2. PIMv6 is +vrf aware and can work within the context of vrf's in order to +do S,G mrouting. + +.. _starting-and-stopping-pim6d: + +Starting and Stopping pim6d +=========================== + +The default configuration file name of *pim6d*'s is :file:`pim6d.conf`. When +invoked *pim6d* searches directory |INSTALL_PREFIX_ETC|. If +:file:`pim6d.conf` is not there then next search current directory. + +*pim6d* requires zebra for proper operation. Additionally *pim6d* depends on +routing properly setup and working in the network that it is working on. + +:: + + # zebra -d + # pim6d -d + + +Please note that *zebra* must be invoked before *pim6d*. + +To stop *pim6d* please use:: + + kill `cat /var/run/frr/pim6d.pid` + +Certain signals have special meanings to *pim6d*. + ++---------+---------------------------------------------------------------------+ +| Signal | Meaning | ++=========+=====================================================================+ +| SIGUSR1 | Rotate the *pim6d* logfile | ++---------+---------------------------------------------------------------------+ +| SIGINT | *pim6d* sweeps all installed PIM mroutes then terminates gracefully.| +| SIGTERM | | ++---------+---------------------------------------------------------------------+ + +*pim6d* invocation options. Common options that can be specified +(:ref:`common-invocation-options`). + +.. clicmd:: ipv6 pim rp X:X::X:X Y:Y::Y:Y/M + + In order to use pimv6, it is necessary to configure a RP for join messages to + be sent to. Currently the only methodology to do this is via static rp + commands. All routers in the pimv6 network must agree on these values. The + first ipv6 address is the RP's address and the second value is the matching + prefix of group ranges covered. This command is vrf aware, to configure for + a vrf, enter the vrf submode. + +.. clicmd:: ipv6 pim rp X:X::X:X prefix-list WORD + + This CLI helps in configuring RP address for a range of groups specified + by the prefix-list. + +.. clicmd:: ipv6 pim rp keep-alive-timer (1-65535) + + Modify the time out value for a S,G flow from 1-65535 seconds at RP. + The normal keepalive period for the KAT(S,G) defaults to 210 seconds. + However, at the RP, the keepalive period must be at least the + Register_Suppression_Time, or the RP may time out the (S,G) state + before the next Null-Register arrives. Thus, the KAT(S,G) is set to + max(Keepalive_Period, RP_Keepalive_Period) when a Register-Stop is sent. + If choosing a value below 31 seconds be aware that some hardware platforms + cannot see data flowing in better than 30 second chunks. This command is + vrf aware, to configure for a vrf, enter the vrf submode. + +.. clicmd:: ipv6 pim spt-switchover infinity-and-beyond [prefix-list PLIST] + + On the last hop router if it is desired to not switch over to the SPT tree + configure this command. Optional parameter prefix-list can be use to control + which groups to switch or not switch. If a group is PERMIT as per the + PLIST, then the SPT switchover does not happen for it and if it is DENY, + then the SPT switchover happens. + This command is vrf aware, to configure for a vrf, + enter the vrf submode. + +.. clicmd:: ipv6 pim join-prune-interval (1-65535) + + Modify the join/prune interval that pim uses to the new value. Time is + specified in seconds. This command is vrf aware, to configure for a vrf, + enter the vrf submode. The default time is 60 seconds. If you enter + a value smaller than 60 seconds be aware that this can and will affect + convergence at scale. + +.. clicmd:: ipv6 pim keep-alive-timer (1-65535) + + Modify the time out value for a S,G flow from 1-65535 seconds. If choosing + a value below 31 seconds be aware that some hardware platforms cannot see data + flowing in better than 30 second chunks. This command is vrf aware, to + configure for a vrf, enter the vrf submode. + +.. clicmd:: ipv6 pim packets (1-255) + + When processing packets from a neighbor process the number of packets + incoming at one time before moving on to the next task. The default value is + 3 packets. This command is only useful at scale when you can possibly have + a large number of pim control packets flowing. This command is vrf aware, to + configure for a vrf, enter the vrf submode. + +.. clicmd:: ipv6 pim register-suppress-time (1-65535) + + Modify the time that pim will register suppress a FHR will send register + notifications to the kernel. This command is vrf aware, to configure for a + vrf, enter the vrf submode. + +.. clicmd:: ipv6 ssmpingd [X:X::X:X] + + Enable ipv6 ssmpingd configuration. A network level management tool + to check whether one can receive multicast packets via SSM from host. + The host target given to ssmping must run the ssmpingd daemon which listens + for IPv4 and IPv6 unicast requests. When it receives one, it responds to a + well known SSM multicast group which ssmping just have joined. + +.. _pimv6-interface-configuration: + +PIMv6 Interface Configuration +============================= + +PIMv6 interface commands allow you to configure an interface as either a Receiver +or a interface that you would like to form pimv6 neighbors on. If the interface +is in a vrf, enter the interface command with the vrf keyword at the end. + +.. clicmd:: ipv6 pim active-active + + Turn on pim active-active configuration for a Vxlan interface. This + command will not do anything if you do not have the underlying ability + of a mlag implementation. + +.. clicmd:: ipv6 pim drpriority (1-4294967295) + + Set the DR Priority for the interface. This command is useful to allow the + user to influence what node becomes the DR for a lan segment. + +.. clicmd:: ipv6 pim hello (1-65535) (1-65535) + + Set the pim hello and hold interval for a interface. + +.. clicmd:: ipv6 pim + + Tell pim that we would like to use this interface to form pim neighbors + over. Please note that this command does not enable the reception of MLD + reports on the interface. Refer to the next ``ipv6 mld`` command for MLD + management. + +.. clicmd:: ipv6 pim use-source X:X::X:X + + If you have multiple addresses configured on a particular interface + and would like pim to use a specific source address associated with + that interface. + +.. clicmd:: ipv6 pim passive + + Disable sending and receiving pim control packets on the interface. + +.. clicmd:: ipv6 pim bsm + + Tell pim that we would like to use this interface to process bootstrap + messages. This is enabled by default. 'no' form of this command is used to + restrict bsm messages on this interface. + +.. clicmd:: ipv6 pim unicast-bsm + + Tell pim that we would like to allow interface to process unicast bootstrap + messages. This is enabled by default. 'no' form of this command is used to + restrict processing of unicast bsm messages on this interface. + +.. clicmd:: ipv6 mld + + Tell pim to receive MLD reports and Query on this interface. The default + version is v2. This command is useful on a LHR. + +.. clicmd:: ipv6 mld join X:X::X:X [Y:Y::Y:Y] + + Join multicast group or source-group on an interface. + +.. clicmd:: ipv6 mld query-interval (1-65535) + + Set the MLD query interval that PIM will use. + +.. clicmd:: ipv6 mld query-max-response-time (1-65535) + + Set the MLD query response timeout value. If an report is not returned in + the specified time we will assume the S,G or \*,G has timed out. + +.. clicmd:: ipv6 mld version (1-2) + + Set the MLD version used on this interface. The default value is 2. + +.. clicmd:: ipv6 multicast boundary oil WORD + + Set a PIMv6 multicast boundary, based upon the WORD prefix-list. If a PIMv6 + join or MLD report is received on this interface and the Group is denied by + the prefix-list, PIMv6 will ignore the join or report. + +.. clicmd:: ipv6 mld last-member-query-count (1-255) + + Set the MLD last member query count. The default value is 2. 'no' form of + this command is used to configure back to the default value. + +.. clicmd:: ipv6 mld last-member-query-interval (1-65535) + + Set the MLD last member query interval in deciseconds. The default value is + 10 deciseconds. 'no' form of this command is used to to configure back to the + default value. + +.. clicmd:: ipv6 mroute INTERFACE X:X::X:X [Y:Y::Y:Y] + + Set a static multicast route for a traffic coming on the current interface to + be forwarded on the given interface if the traffic matches the group address + and optionally the source address. + +.. _show-pimv6-information: + +Show PIMv6 Information +====================== + +All PIMv6 show commands are vrf aware and typically allow you to insert a +specified vrf command if information is desired about a specific vrf. If no +vrf is specified then the default vrf is assumed. Finally the special keyword +'all' allows you to look at all vrfs for the command. Naming a vrf 'all' will +cause great confusion. + +PIM protocol state +------------------ + +.. clicmd:: show ipv6 pim [vrf NAME] group-type [json] + + Display SSM group ranges. + +.. clicmd:: show ipv6 pim interface + + Display information about interfaces PIM is using. + +.. clicmd:: show ipv6 pim [vrf NAME] join [X:X::X:X [X:X::X:X]] [json] +.. clicmd:: show ipv6 pim vrf all join [json] + + Display information about PIM joins received. If one address is specified + then we assume it is the Group we are interested in displaying data on. + If the second address is specified then it is Source Group. + +.. clicmd:: show ipv6 pim [vrf NAME] local-membership [json] + + Display information about PIM interface local-membership. + +.. clicmd:: show ipv6 pim [vrf NAME] neighbor [detail|WORD] [json] +.. clicmd:: show ipv6 pim vrf all neighbor [detail|WORD] [json] + + Display information about PIM neighbors. + +.. clicmd:: show ipv6 pim [vrf NAME] nexthop + + Display information about pim nexthops that are being used. + +.. clicmd:: show ipv6 pim [vrf NAME] nexthop-lookup X:X::X:X X:X::X:X + + Display information about a S,G pair and how the RPF would be chosen. This + is especially useful if there are ECMP's available from the RPF lookup. + +.. clicmd:: show ipv6 pim [vrf NAME] rp-info [json] +.. clicmd:: show ipv6 pim vrf all rp-info [json] + + Display information about RP's that are configured on this router. + +.. clicmd:: show ipv6 pim [vrf NAME] rpf [json] +.. clicmd:: show ipv6 pim vrf all rpf [json] + + Display information about currently being used S,G's and their RPF lookup + information. Additionally display some statistics about what has been + happening on the router. + +.. clicmd:: show ipv6 pim [vrf NAME] secondary + + Display information about an interface and all the secondary addresses + associated with it. + +.. clicmd:: show ipv6 pim [vrf NAME] state [X:X::X:X [X:X::X:X]] [json] +.. clicmd:: show ipv6 pim vrf all state [X:X::X:X [X:X::X:X]] [json] + + Display information about known S,G's and incoming interface as well as the + OIL and how they were chosen. + +.. clicmd:: show ipv6 pim [vrf NAME] upstream [X:X::X:X [Y:Y::Y:Y]] [json] +.. clicmd:: show ipv6 pim vrf all upstream [json] + + Display upstream information about a S,G mroute. Allow the user to + specify sub Source and Groups that we are interested in. + +.. clicmd:: show ipv6 pim [vrf NAME] upstream-join-desired [json] + + Display upstream information for S,G's and if we desire to + join the multicast tree + +.. clicmd:: show ipv6 pim [vrf NAME] upstream-rpf [json] + + Display upstream information for S,G's and the RPF data associated with them. + +.. clicmd:: show ipv6 pim [vrf NAME] interface traffic [WORD] [json] + + Display information about the number of PIM protocol packets sent/received + on an interface. + +MLD state +--------- + +.. clicmd:: show ipv6 mld [vrf NAME] interface [IFNAME] [detail|json] + + Display per-interface MLD state, elected querier and related timers. Use + the ``detail`` or ``json`` options for further information (the JSON output + always contains all details.) + +.. clicmd:: show ipv6 mld [vrf NAME] statistics [interface IFNAME] [json] + + Display packet and error counters for MLD interfaces. All counters are + packet counters (not bytes) and wrap at 64 bit. In some rare cases, + malformed received MLD reports may be partially processed and counted on + multiple counters. + +.. clicmd:: show ipv6 mld [vrf NAME] joins [{interface IFNAME|groups X:X::X:X/M|sources X:X::X:X/M|detail}] [json] + + Display joined groups tracked by MLD. ``interface``, ``groups`` and + ``sources`` options may be used to limit output to a subset (note ``sources`` + refers to the multicast traffic sender, not the host that joined to receive + the traffic.) + + The ``detail`` option also reports which hosts have joined (subscribed) to + particular ``S,G``. This information is only available for MLDv2 hosts with + a MLDv2 querier. MLDv1 joins are recorded as "untracked" and shown in the + ``NonTrkSeen`` output column. + +.. clicmd:: show ipv6 mld [vrf NAME] groups [json] + + Display MLD group information. + +General multicast routing state +------------------------------- + +.. clicmd:: show ipv6 multicast + + Display various information about the interfaces used in this pim instance. + +.. clicmd:: show ipv6 multicast count [vrf NAME] [json] + + Display multicast data packets count per interface for a vrf. + +.. clicmd:: show ipv6 multicast count vrf all [json] + + Display multicast data packets count per interface for all vrf. + +.. clicmd:: show ipv6 mroute [vrf NAME] [X:X::X:X [X:X::X:X]] [fill] [json] + + Display information about installed into the kernel S,G mroutes. If + one address is specified we assume it is the Group we are interested + in displaying data on. If the second address is specified then it is + Source Group. The keyword ``fill`` says to fill in all assumed data + for test/data gathering purposes. + +.. clicmd:: show ipv6 mroute [vrf NAME] count [json] + + Display information about installed into the kernel S,G mroutes and in + addition display data about packet flow for the mroutes for a specific + vrf. + +.. clicmd:: show ipv6 mroute vrf all count [json] + + Display information about installed into the kernel S,G mroutes and in + addition display data about packet flow for the mroutes for all vrfs. + +.. clicmd:: show ipv6 mroute [vrf NAME] summary [json] + + Display total number of S,G mroutes and number of S,G mroutes installed + into the kernel for a specific vrf. + +.. clicmd:: show ipv6 mroute vrf all summary [json] + + Display total number of S,G mroutes and number of S,G mroutes + installed into the kernel for all vrfs. + +.. clicmd:: show ipv6 pim bsr + + Display current bsr, its uptime and last received bsm age. + +.. clicmd:: show ipv6 pim bsrp-info + + Display group-to-rp mappings received from E-BSR. + +.. clicmd:: show ipv6 pim bsm-database + + Display all fragments of stored bootstrap message in user readable format. + +PIMv6 Clear Commands +==================== + +Clear commands reset various variables. + +.. clicmd:: clear ipv6 mroute + + Reset multicast routes. + +.. clicmd:: clear ipv6 mroute [vrf NAME] count + + When this command is issued, reset the counts of data shown for + packet count, byte count and wrong interface to 0 and start count + up from this spot. + +.. clicmd:: clear ipv6 pim interfaces + + Reset PIMv6 interfaces. + +.. clicmd:: clear ipv6 pim [vrf NAME] interface traffic + + When this command is issued, resets the information about the + number of PIM protocol packets sent/received on an interface. + +.. clicmd:: clear ipv6 pim oil + + Rescan PIMv6 OIL (output interface list). + +.. clicmd:: clear ipv6 pim [vrf NAME] bsr-data + + This command will clear the BSM scope data struct. This command also + removes the next hop tracking for the bsr and resets the upstreams + for the dynamically learnt RPs. + +PIMv6 Debug Commands +==================== + +The debugging subsystem for PIMv6 behaves in accordance with how FRR handles +debugging. You can specify debugging at the enable CLI mode as well as the +configure CLI mode. If you specify debug commands in the configuration cli +mode, the debug commands can be persistent across restarts of the FRR pim6d if +the config was written out. + +.. clicmd:: debug mld + + This turns on debugging for MLD protocol activity. + +.. clicmd:: debug pimv6 events + + This turns on debugging for PIMv6 system events. Especially timers. + +.. clicmd:: debug pimv6 nht + + This turns on debugging for PIMv6 nexthop tracking. It will display + information about RPF lookups and information about when a nexthop changes. + +.. clicmd:: debug pimv6 nht detail + + This turns on debugging for PIMv6 nexthop in detail. This is not enabled + by default. + +.. clicmd:: debug pimv6 packet-dump + + This turns on an extraordinary amount of data. Each pim packet sent and + received is dumped for debugging purposes. This should be considered a + developer only command. + +.. clicmd:: debug pimv6 packets + + This turns on information about packet generation for sending and about + packet handling from a received packet. + +.. clicmd:: debug pimv6 trace + + This traces pim code and how it is running. + +.. clicmd:: debug pimv6 zebra + + This gathers data about events from zebra that come up through the ZAPI. + +.. clicmd:: debug mroute6 + + This turns on debugging for PIMv6 interaction with kernel MFC cache. + +.. clicmd:: debug mroute6 detail + + This turns on detailed debugging for PIMv6 interaction with kernel MFC cache. + +.. clicmd:: debug mld events + + This turns on debugging for MLD system events. + +.. clicmd:: debug mld packets + + This turns on information about MLD protocol packets handling. + +.. clicmd:: debug mld trace [detail] + + This traces mld code and how it is running. + +.. clicmd:: debug pimv6 bsm + + This turns on debugging for BSR message processing. diff --git a/doc/user/ripd.rst b/doc/user/ripd.rst new file mode 100644 index 0000000..f9c7724 --- /dev/null +++ b/doc/user/ripd.rst @@ -0,0 +1,560 @@ +.. _rip: + +*** +RIP +*** + +RIP -- Routing Information Protocol is widely deployed interior gateway +protocol. RIP was developed in the 1970s at Xerox Labs as part of the +XNS routing protocol. RIP is a :term:`distance-vector` protocol and is +based on the :term:`Bellman-Ford` algorithms. As a distance-vector +protocol, RIP router send updates to its neighbors periodically, thus +allowing the convergence to a known topology. In each update, the +distance to any given network will be broadcast to its neighboring +router. + +*ripd* supports RIP version 2 as described in RFC2453 and RIP +version 1 as described in RFC1058. + +.. _starting-and-stopping-ripd: + +Starting and Stopping ripd +========================== + +The default configuration file name of *ripd*'s is :file:`ripd.conf`. When +invocation *ripd* searches directory |INSTALL_PREFIX_ETC|. If :file:`ripd.conf` +is not there next search current directory. + +RIP uses UDP port 520 to send and receive RIP packets. So the user must have +the capability to bind the port, generally this means that the user must have +superuser privileges. RIP protocol requires interface information maintained by +*zebra* daemon. So running *zebra* is mandatory to run *ripd*. Thus minimum +sequence for running RIP is like below: + +:: + + # zebra -d + # ripd -d + + +Please note that *zebra* must be invoked before *ripd*. + +To stop *ripd*. Please use:: + + kill `cat /var/run/frr/ripd.pid` + +Certain signals have special meanings to *ripd*. + + +-------------+------------------------------------------------------+ + | Signal | Action | + +=============+======================================================+ + | ``SIGHUP`` | Reload configuration file :file:`ripd.conf`. | + | | All configurations are reset. All routes learned | + | | so far are cleared and removed from routing table. | + +-------------+------------------------------------------------------+ + | ``SIGUSR1`` | Rotate the *ripd* logfile. | + +-------------+------------------------------------------------------+ + | ``SIGINT`` | | + | ``SIGTERM`` | Sweep all installed routes and gracefully terminate. | + +-------------+------------------------------------------------------+ + +*ripd* invocation options. Common options that can be specified +(:ref:`common-invocation-options`). + + +.. _rip-netmask: + +RIP netmask +----------- + +The netmask features of *ripd* support both version 1 and version 2 of RIP. +Version 1 of RIP originally contained no netmask information. In RIP version 1, +network classes were originally used to determine the size of the netmask. +Class A networks use 8 bits of mask, Class B networks use 16 bits of masks, +while Class C networks use 24 bits of mask. Today, the most widely used method +of a network mask is assigned to the packet on the basis of the interface that +received the packet. Version 2 of RIP supports a variable length subnet mask +(VLSM). By extending the subnet mask, the mask can be divided and reused. Each +subnet can be used for different purposes such as large to middle size LANs and +WAN links. FRR *ripd* does not support the non-sequential netmasks that are +included in RIP Version 2. + +In a case of similar information with the same prefix and metric, the old +information will be suppressed. Ripd does not currently support equal cost +multipath routing. + +.. _rip-configuration: + +RIP Configuration +================= + +.. clicmd:: router rip [vrf NAME] + + The `router rip` command is necessary to enable RIP. To disable RIP, use the + `no router rip` command. RIP must be enabled before carrying out any of the + RIP commands. + +.. clicmd:: network NETWORK + + + Set the RIP enable interface by NETWORK. The interfaces which have addresses + matching with NETWORK are enabled. + + This group of commands either enables or disables RIP interfaces between + certain numbers of a specified network address. For example, if the network + for 10.0.0.0/24 is RIP enabled, this would result in all the addresses from + 10.0.0.0 to 10.0.0.255 being enabled for RIP. The `no network` command will + disable RIP for the specified network. + +.. clicmd:: network IFNAME + + + Set a RIP enabled interface by IFNAME. Both the sending and + receiving of RIP packets will be enabled on the port specified in the + `network ifname` command. The `no network ifname` command will disable + RIP on the specified interface. + +.. clicmd:: neighbor A.B.C.D + + + Specify a RIP neighbor to send updates to. This is required when a neighbor + is connected via a network that does not support multicast, or when it is + desired to statically define a neighbor. RIP updates will be sent via unicast + to each neighbour. Neighbour updates are in addition to any multicast updates + sent when an interface is not in passive mode (see the `passive-interface` + command). RIP will continue to process updates received from both the + neighbor and any received via multicast. The `no neighbor a.b.c.d` command + will disable the RIP neighbor. + + Below is very simple RIP configuration. Interface `eth0` and interface which + address match to `10.0.0.0/8` are RIP enabled. + + .. code-block:: frr + + ! + router rip + network 10.0.0.0/8 + network eth0 + ! + + +.. clicmd:: passive-interface (IFNAME|default) + + + This command sets the specified interface to passive mode. On passive mode + interface, all receiving packets are processed as normal and ripd does not + send either multicast or unicast RIP packets except to RIP neighbors + specified with `neighbor` command. The interface may be specified as + `default` to make ripd default to passive on all interfaces. + + The default is to be passive on all interfaces. + +.. clicmd:: ip split-horizon [poisoned-reverse] + + + Control split-horizon on the interface. Default is `ip split-horizon`. If + you don't perform split-horizon on the interface, please specify `no ip + split-horizon`. + + If `poisoned-reverse` is also set, the router sends the poisoned routes + with highest metric back to the sending router. + +.. clicmd:: allow-ecmp [1-MULTIPATH_NUM] + + Control how many ECMP paths RIP can inject for the same prefix. If specified + without a number, a maximum is taken (compiled with ``--enable-multipath``). + +.. _rip-version-control: + +RIP Version Control +=================== + +RIP can be configured to send either Version 1 or Version 2 packets. The +default is to send RIPv2 while accepting both RIPv1 and RIPv2 (and replying +with packets of the appropriate version for REQUESTS / triggered updates). The +version to receive and send can be specified globally, and further overridden on +a per-interface basis if needs be for send and receive separately (see below). + +It is important to note that RIPv1 cannot be authenticated. Further, if RIPv1 +is enabled then RIP will reply to REQUEST packets, sending the state of its RIP +routing table to any remote routers that ask on demand. For a more detailed +discussion on the security implications of RIPv1 see :ref:`rip-authentication`. + +.. clicmd:: version VERSION + + Set RIP version to accept for reads and send. VERSION can be either + ``1`` or ``2``. + + Disabling RIPv1 by specifying version 2 is STRONGLY encouraged, + :ref:`rip-authentication`. This may become the default in a future release. + + Default: Send Version 2, and accept either version. + +.. clicmd:: ip rip send version VERSION + + VERSION can be ``1``, ``2``, or ``1 2``. + + This interface command overrides the global rip version setting, and selects + which version of RIP to send packets with, for this interface specifically. + Choice of RIP Version 1, RIP Version 2, or both versions. In the latter + case, where ``1 2`` is specified, packets will be both broadcast and + multicast. + + Default: Send packets according to the global version (version 2) + +.. clicmd:: ip rip receive version VERSION + + VERSION can be ``1``, ``2``, or ``1 2``. + + This interface command overrides the global rip version setting, and selects + which versions of RIP packets will be accepted on this interface. Choice of + RIP Version 1, RIP Version 2, or both. + + Default: Accept packets according to the global setting (both 1 and 2). + + +.. _how-to-announce-rip-route: + +How to Announce RIP route +========================= + +.. clicmd:: redistribute <babel|bgp|connected|eigrp|isis|kernel|openfabric|ospf|sharp|static|table> [metric (0-16)] [route-map WORD] + + Redistribute routes from other sources into RIP. + +If you want to specify RIP only static routes: + +.. clicmd:: default-information originate + +.. clicmd:: route A.B.C.D/M + + + This command is specific to FRR. The `route` command makes a static route + only inside RIP. This command should be used only by advanced users who are + particularly knowledgeable about the RIP protocol. In most cases, we + recommend creating a static route in FRR and redistributing it in RIP using + `redistribute static`. + +.. _filtering-rip-routes: + +Filtering RIP Routes +==================== + +RIP routes can be filtered by a distribute-list. + +.. clicmd:: distribute-list [prefix] LIST <in|out> IFNAME + + You can apply access lists to the interface with a `distribute-list` command. + If prefix is specified LIST is a prefix-list. If prefix is not specified + then LIST is the access list name. `in` specifies packets being received, + and `out` specifies outgoing packets. Finally if an interface is specified + it will be applied against a specific interface. + + The `distribute-list` command can be used to filter the RIP path. + `distribute-list` can apply access-lists to a chosen interface. First, one + should specify the access-list. Next, the name of the access-list is used in + the distribute-list command. For example, in the following configuration + ``eth0`` will permit only the paths that match the route 10.0.0.0/8 + + .. code-block:: frr + + ! + router rip + distribute-list private in eth0 + ! + access-list private permit 10 10.0.0.0/8 + access-list private deny any + ! + + + `distribute-list` can be applied to both incoming and outgoing data. + +.. _rip-metric-manipulation: + +RIP Metric Manipulation +======================= + +RIP metric is a value for distance for the network. Usually +*ripd* increment the metric when the network information is +received. Redistributed routes' metric is set to 1. + +.. clicmd:: default-metric (1-16) + + + This command modifies the default metric value for redistributed routes. + The default value is 1. This command does not affect connected route even if + it is redistributed by *redistribute connected*. To modify connected route's + metric value, please use ``redistribute connected metric`` or *route-map*. + *offset-list* also affects connected routes. + +.. clicmd:: offset-list ACCESS-LIST (in|out) + +.. clicmd:: offset-list ACCESS-LIST (in|out) IFNAME + + +.. _rip-distance: + +RIP distance +============ + +Distance value is used in zebra daemon. Default RIP distance is 120. + +.. clicmd:: distance (1-255) + + + Set default RIP distance to specified value. + +.. clicmd:: distance (1-255) A.B.C.D/M + + + Set default RIP distance to specified value when the route's source IP + address matches the specified prefix. + +.. clicmd:: distance (1-255) A.B.C.D/M ACCESS-LIST + + + Set default RIP distance to specified value when the route's source IP + address matches the specified prefix and the specified access-list. + +.. _rip-route-map: + +RIP route-map +============= + +Usage of *ripd*'s route-map support. + +Optional argument route-map MAP_NAME can be added to each `redistribute` +statement. + +.. code-block:: frr + + redistribute static [route-map MAP_NAME] + redistribute connected [route-map MAP_NAME] + ..... + + +Cisco applies route-map _before_ routes will exported to rip route table. In +current FRR's test implementation, *ripd* applies route-map after routes are +listed in the route table and before routes will be announced to an interface +(something like output filter). I think it is not so clear, but it is draft and +it may be changed at future. + +Route-map statement (:ref:`route-map`) is needed to use route-map +functionality. + +.. clicmd:: match interface WORD + + This command match to incoming interface. Notation of this match is + different from Cisco. Cisco uses a list of interfaces - NAME1 NAME2 ... + NAMEN. Ripd allows only one name (maybe will change in the future). Next - + Cisco means interface which includes next-hop of routes (it is somewhat + similar to "ip next-hop" statement). Ripd means interface where this route + will be sent. This difference is because "next-hop" of same routes which + sends to different interfaces must be different. Maybe it'd be better to + made new matches - say "match interface-out NAME" or something like that. + +.. clicmd:: match ip address WORD + +.. clicmd:: match ip address prefix-list WORD + + Match if route destination is permitted by access-list. + +.. clicmd:: match ip next-hop WORD + +.. clicmd:: match ip next-hop prefix-list WORD + + Match if route next-hop (meaning next-hop listed in the rip route-table as + displayed by "show ip rip") is permitted by access-list. + +.. clicmd:: match metric (0-4294967295) + + This command match to the metric value of RIP updates. For other protocol + compatibility metric range is shown as (0-4294967295). But for RIP protocol + only the value range (0-16) make sense. + +.. clicmd:: set ip next-hop A.B.C.D + + This command set next hop value in RIPv2 protocol. This command does not + affect RIPv1 because there is no next hop field in the packet. + +.. clicmd:: set metric (0-4294967295) + + Set a metric for matched route when sending announcement. The metric value + range is very large for compatibility with other protocols. For RIP, valid + metric values are from 1 to 16. + +.. _rip-authentication: + +RIP Authentication +================== + +RIPv2 allows packets to be authenticated via either an insecure plain +text password, included with the packet, or via a more secure MD5 based +:abbr:`HMAC (keyed-Hashing for Message AuthentiCation)`, +RIPv1 can not be authenticated at all, thus when authentication is +configured `ripd` will discard routing updates received via RIPv1 +packets. + +However, unless RIPv1 reception is disabled entirely, +:ref:`rip-version-control`, RIPv1 REQUEST packets which are received, +which query the router for routing information, will still be honoured +by `ripd`, and `ripd` WILL reply to such packets. This allows +`ripd` to honour such REQUESTs (which sometimes is used by old +equipment and very simple devices to bootstrap their default route), +while still providing security for route updates which are received. + +In short: Enabling authentication prevents routes being updated by +unauthenticated remote routers, but still can allow routes (I.e. the +entire RIP routing table) to be queried remotely, potentially by anyone +on the internet, via RIPv1. + +To prevent such unauthenticated querying of routes disable RIPv1, +:ref:`rip-version-control`. + +.. clicmd:: ip rip authentication mode md5 + + + Set the interface with RIPv2 MD5 authentication. + +.. clicmd:: ip rip authentication mode text + + + Set the interface with RIPv2 simple password authentication. + +.. clicmd:: ip rip authentication string STRING + + + RIP version 2 has simple text authentication. This command sets + authentication string. The string must be shorter than 16 characters. + +.. clicmd:: ip rip authentication key-chain KEY-CHAIN + + + Specify Keyed MD5 chain. + + .. code-block:: frr + + ! + key chain test + key 1 + key-string test + ! + interface eth1 + ip rip authentication mode md5 + ip rip authentication key-chain test + ! + + +.. _rip-timers: + +RIP Timers +========== + +.. clicmd:: timers basic UPDATE TIMEOUT GARBAGE + + + RIP protocol has several timers. User can configure those timers' values + by `timers basic` command. + + The default settings for the timers are as follows: + + - The update timer is 30 seconds. Every update timer seconds, the RIP + process is awakened to send an unsolicited Response message containing + the complete routing table to all neighboring RIP routers. + - The timeout timer is 180 seconds. Upon expiration of the timeout, the + route is no longer valid; however, it is retained in the routing table + for a short time so that neighbors can be notified that the route has + been dropped. + - The garbage collect timer is 120 seconds. Upon expiration of the + garbage-collection timer, the route is finally removed from the routing + table. + + The ``timers basic`` command allows the the default values of the timers + listed above to be changed. + + +.. _show-rip-information: + +Show RIP Information +==================== + +To display RIP routes. + +.. clicmd:: show ip rip [vrf NAME] + + Show RIP routes. + +The command displays all RIP routes. For routes that are received +through RIP, this command will display the time the packet was sent and +the tag information. This command will also display this information +for routes redistributed into RIP. + +.. clicmd:: show ip rip [vrf NAME] status + + The command displays current RIP status. It includes RIP timer, + filtering, version, RIP enabled interface and RIP peer information. + +:: + + ripd> **show ip rip status** + Routing Protocol is "rip" + Sending updates every 30 seconds with +/-50%, next due in 35 seconds + Timeout after 180 seconds, garbage collect after 120 seconds + Outgoing update filter list for all interface is not set + Incoming update filter list for all interface is not set + Default redistribution metric is 1 + Redistributing: kernel connected + Default version control: send version 2, receive version 2 + Interface Send Recv + Routing for Networks: + eth0 + eth1 + 1.1.1.1 + 203.181.89.241 + Routing Information Sources: + Gateway BadPackets BadRoutes Distance Last Update + + +RIP Debug Commands +================== + +Debug for RIP protocol. + +.. clicmd:: debug rip events + + Shows RIP events. Sending and receiving packets, timers, and changes in + interfaces are events shown with *ripd*. + +.. clicmd:: debug rip packet + + Shows display detailed information about the RIP packets. The origin and + port number of the packet as well as a packet dump is shown. + +.. clicmd:: debug rip zebra + + This command will show the communication between *ripd* and *zebra*. The + main information will include addition and deletion of paths to the kernel + and the sending and receiving of interface information. + +.. clicmd:: show debugging rip + + Shows all information currently set for ripd debug. + + +Sample configuration +==================== + +.. code-block:: frr + + + debug rip events + debug rip packet + + router rip + network 11.0.0.0/8 + network eth0 + route 10.0.0.0/8 + distribute-list private-only in eth0 + + access-list private-only permit 10.0.0.0/8 + access-list private-only deny any diff --git a/doc/user/ripngd.rst b/doc/user/ripngd.rst new file mode 100644 index 0000000..1e78294 --- /dev/null +++ b/doc/user/ripngd.rst @@ -0,0 +1,156 @@ +.. _ripng: + +***** +RIPng +***** + +*ripngd* supports the RIPng protocol as described in :rfc:`2080`. It's an IPv6 +reincarnation of the RIP protocol. + +.. _invoking-ripngd: + +Invoking ripngd +=============== + +There are no `ripngd` specific invocation options. Common options can be +specified (:ref:`common-invocation-options`). + +.. _ripngd-configuration: + +ripngd Configuration +==================== + +Currently ripngd supports the following commands: + +.. clicmd:: router ripng [vrf NAME] + + Enable RIPng. + +.. clicmd:: network NETWORK + + Set RIPng enabled interface by NETWORK. + +.. clicmd:: network IFNAME + + Set RIPng enabled interface by IFNAME. + +.. clicmd:: route NETWORK + + Set RIPng static routing announcement of NETWORK. + +.. clicmd:: allow-ecmp [1-MULTIPATH_NUM] + + Control how many ECMP paths RIPng can inject for the same prefix. If specified + without a number, a maximum is taken (compiled with ``--enable-multipath``). + +.. _ripngd-terminal-mode-commands: + +ripngd Terminal Mode Commands +============================= + +.. clicmd:: show ipv6 ripng [vrf NAME] status + +.. clicmd:: show debugging ripng + +.. clicmd:: debug ripng events + +.. clicmd:: debug ripng packet + +.. clicmd:: debug ripng zebra + + +ripngd Filtering Commands +========================= + +RIPng routes can be filtered by a distribute-list. + +.. clicmd:: distribute-list [prefix] LIST <in|out> IFNAME + + You can apply access lists to the interface with a `distribute-list` command. + If prefix is specified LIST is a prefix-list. If prefix is not specified + then LIST is the access list name. `in` specifies packets being received, + and `out` specifies outgoing packets. Finally if an interface is specified + it will be applied against a specific interface. + + The ``distribute-list`` command can be used to filter the RIPNG path. + ``distribute-list`` can apply access-lists to a chosen interface. First, one + should specify the access-list. Next, the name of the access-list is used in + the distribute-list command. For example, in the following configuration + ``eth0`` will permit only the paths that match the route 10.0.0.0/8 + + .. code-block:: frr + + ! + router ripng + distribute-list private in eth0 + ! + access-list private permit 10 10.0.0.0/8 + access-list private deny any + ! + + + `distribute-list` can be applied to both incoming and outgoing data. + + +.. _ripng-route-map: + +RIPng route-map +=============== + +Usage of *ripngd*'s route-map support. + +Route-map statement (:ref:`route-map`) is needed to use route-map +functionality. + +.. clicmd:: match interface WORD + + This command match to incoming interface. Notation of this match is + different from Cisco. Cisco uses a list of interfaces - NAME1 NAME2 ... + NAMEN. Ripngd allows only one name (maybe will change in the future). Next - + Cisco means interface which includes next-hop of routes (it is somewhat + similar to "ipv6 next-hop" statement). Ripngd means interface where this route + will be sent. This difference is because "next-hop" of same routes which + sends to different interfaces must be different. + +.. clicmd:: match ipv6 address WORD + +.. clicmd:: match ipv6 address prefix-list WORD + + Match if route destination is permitted by access-list/prefix-list. + +.. clicmd:: match metric (0-4294967295) + + This command match to the metric value of RIPng updates. For other protocol + compatibility metric range is shown as (0-4294967295). But for RIPng protocol + only the value range (0-16) make sense. + +.. clicmd:: set ipv6 next-hop local IPV6_ADDRESS + + Set the link-local IPv6 nexthop address. + +.. clicmd:: set metric (1-16) + + Set a metric for matched route when sending announcement. The metric value + range is very large for compatibility with other protocols. For RIPng, valid + metric values are from 1 to 16. + +.. clicmd:: set tag (1-4294967295) + + Set a tag on the matched route. + + +Sample configuration +==================== + +.. code-block:: frr + + debug ripng events + debug ripng packet + + router ripng + network sit1 + route 3ffe:506::0/32 + distribute-list local-only out sit1 + + ipv6 access-list local-only permit 3ffe:506::0/32 + ipv6 access-list local-only deny any diff --git a/doc/user/routemap.rst b/doc/user/routemap.rst new file mode 100644 index 0000000..3d43e74 --- /dev/null +++ b/doc/user/routemap.rst @@ -0,0 +1,426 @@ +.. _route-map: + +********** +Route Maps +********** + +Route maps provide a means to both filter and/or apply actions to route, hence +allowing policy to be applied to routes. + +For a route reflector to apply a ``route-map`` to reflected routes, be sure to +include ``bgp route-reflector allow-outbound-policy`` in ``router bgp`` mode. + +Route maps are an ordered list of route map entries. Each entry may specify up +to four distinct sets of clauses: + +.. glossary:: + + Matching Conditions + A route-map entry may, optionally, specify one or more conditions which + must be matched if the entry is to be considered further, as governed by + the Match Policy. If a route-map entry does not explicitly specify any + matching conditions, then it always matches. + + Set Actions + A route-map entry may, optionally, specify one or more Set Actions to set + or modify attributes of the route. + + Matching Policy + This specifies the policy implied if the :term:`Matching Conditions` are + met or not met, and which actions of the route-map are to be taken, if + any. The two possibilities are: + + - :dfn:`permit`: If the entry matches, then carry out the + :term:`Set Actions`. Then finish processing the route-map, permitting + the route, unless an :term:`Exit Policy` action indicates otherwise. + + - :dfn:`deny`: If the entry matches, then finish processing the route-map and + deny the route (return `deny`). + + The `Matching Policy` is specified as part of the command which defines + the ordered entry in the route-map. See below. + + Call Action + Call to another route-map, after any :term:`Set Actions` have been + carried out. If the route-map called returns `deny` then processing of + the route-map finishes and the route is denied, regardless of the + :term:`Matching Policy` or the :term:`Exit Policy`. If the called + route-map returns `permit`, then :term:`Matching Policy` and :term:`Exit + Policy` govern further behaviour, as normal. + + Exit Policy + An entry may, optionally, specify an alternative :dfn:`Exit Policy` to + take if the entry matched, rather than the normal policy of exiting the + route-map and permitting the route. The two possibilities are: + + - :dfn:`next`: Continue on with processing of the route-map entries. + + - :dfn:`goto N`: Jump ahead to the first route-map entry whose order in + the route-map is >= N. Jumping to a previous entry is not permitted. + +The default action of a route-map, if no entries match, is to deny. I.e. a +route-map essentially has as its last entry an empty *deny* entry, which +matches all routes. To change this behaviour, one must specify an empty +*permit* entry as the last entry in the route-map. + +To summarise the above: + ++--------+--------+----------+ +| | Match | No Match | ++========+========+==========+ +| Permit | action | cont | ++--------+--------+----------+ +| Deny | deny | cont | ++--------+--------+----------+ + +action + - Apply *set* statements + - If *call* is present, call given route-map. If that returns a ``deny``, + finish processing and return ``deny``. + - If *Exit Policy* is *next*, goto next route-map entry + - If *Exit Policy* is *goto*, goto first entry whose order in the + list is >= the given order. + - Finish processing the route-map and permit the route. + +deny + The route is denied by the route-map (return ``deny``). + +cont + goto next route-map entry + +.. _route-map-show-command: + +.. clicmd:: show route-map [WORD] [json] + + Display data about each daemons knowledge of individual route-maps. + If WORD is supplied narrow choice to that particular route-map. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. _route-map-clear-counter-command: + +.. clicmd:: clear route-map counter [WORD] + + Clear counters that are being stored about the route-map utilization + so that subsuquent show commands will indicate since the last clear. + If WORD is specified clear just that particular route-map's counters. + +.. _route-map-command: + +Route Map Command +================= + +.. clicmd:: route-map ROUTE-MAP-NAME (permit|deny) ORDER + + Configure the `order`'th entry in `route-map-name` with ``Match Policy`` of + either *permit* or *deny*. + +.. _route-map-match-command: + +Route Map Match Command +======================= + +.. clicmd:: match ip address ACCESS_LIST + + Matches the specified `access_list` + +.. clicmd:: match ip address prefix-list PREFIX_LIST + + Matches the specified `PREFIX_LIST` + +.. clicmd:: match ip address prefix-len 0-32 + + Matches the specified `prefix-len`. This is a Zebra specific command. + +.. clicmd:: match ipv6 address ACCESS_LIST + + Matches the specified `access_list` + +.. clicmd:: match ipv6 address prefix-list PREFIX_LIST + + Matches the specified `PREFIX_LIST` + +.. clicmd:: match ipv6 address prefix-len 0-128 + + Matches the specified `prefix-len`. This is a Zebra specific command. + +.. clicmd:: match ip next-hop ACCESS_LIST + + Match the next-hop according to the given access-list. + +.. clicmd:: match ip next-hop address IPV4_ADDR + + This is a BGP specific match command. Matches the specified `ipv4_addr`. + +.. clicmd:: match ip next-hop prefix-list PREFIX_LIST + + Match the next-hop according to the given prefix-list. + +.. clicmd:: match ipv6 next-hop ACCESS_LIST + + Match the next-hop according to the given access-list. + +.. clicmd:: match ipv6 next-hop address IPV6_ADDR + + This is a BGP specific match command. Matches the specified `ipv6_addr`. + +.. clicmd:: match ipv6 next-hop prefix-list PREFIX_LIST + + Match the next-hop according to the given prefix-list. + +.. clicmd:: match as-path AS_PATH + + Matches the specified `as_path`. + +.. clicmd:: match metric METRIC + + Matches the specified `metric`. + +.. clicmd:: match tag TAG + + Matches the specified tag value associated with the route. This tag value + can be in the range of (1-4294967295). + +.. clicmd:: match local-preference METRIC + + Matches the specified `local-preference`. + +.. clicmd:: match community COMMUNITY_LIST + + Matches the specified `community_list` + +.. clicmd:: match peer IPV4_ADDR + + This is a BGP specific match command. Matches the peer ip address + if the neighbor was specified in this manner. + +.. clicmd:: match peer IPV6_ADDR + + This is a BGP specific match command. Matches the peer ipv6 + address if the neighbor was specified in this manner. + +.. clicmd:: match peer INTERFACE_NAME + + This is a BGP specific match command. Matches the peer + interface name specified if the neighbor was specified + in this manner. + +.. clicmd:: match peer PEER_GROUP_NAME + + This is a BGP specific match command. Matches the peer + group name specified for the peer in question. + +.. clicmd:: match source-protocol PROTOCOL_NAME + + This is a ZEBRA and BGP specific match command. Matches the + originating protocol specified. + +.. clicmd:: match source-instance NUMBER + + This is a ZEBRA specific match command. The number is a range from (0-255). + Matches the originating protocols instance specified. + +.. clicmd:: match evpn route-type ROUTE_TYPE_NAME + + This is a BGP EVPN specific match command. It matches to EVPN route-type + from type-1 (EAD route-type) to type-5 (Prefix route-type). + User can provide in an integral form (1-5) or string form of route-type + (i.e ead, macip, multicast, es, prefix). + +.. clicmd:: match evpn vni NUMBER + + This is a BGP EVPN specific match command which matches to EVPN VNI id. + The number is a range from (1-6777215). + +.. _route-map-set-command: + +Route Map Set Command +===================== + +.. program:: configure + +.. clicmd:: set tag TAG + + Set a tag on the matched route. This tag value can be from (1-4294967295). + Additionally if you have compiled with the :option:`--enable-realms` + configure option. Tag values from (1-255) are sent to the Linux kernel as a + realm value. Then route policy can be applied. See the tc man page. As + a note realms cannot currently be used with the installation of nexthops + as nexthop groups in the linux kernel. + +.. clicmd:: set ip next-hop IPV4_ADDRESS + + Set the BGP nexthop address to the specified IPV4_ADDRESS. For both + incoming and outgoing route-maps. + +.. clicmd:: set ip next-hop peer-address + + Set the BGP nexthop address to the address of the peer. For an incoming + route-map this means the ip address of our peer is used. For an outgoing + route-map this means the ip address of our self is used to establish the + peering with our neighbor. + +.. clicmd:: set ip next-hop unchanged + + Set the route-map as unchanged. Pass the route-map through without + changing it's value. + +.. clicmd:: set ipv6 next-hop peer-address + + Set the BGP nexthop address to the address of the peer. For an incoming + route-map this means the ipv6 address of our peer is used. For an outgoing + route-map this means the ip address of our self is used to establish the + peering with our neighbor. + +.. clicmd:: set ipv6 next-hop prefer-global + + For Incoming and Import Route-maps if we receive a v6 global and v6 LL + address for the route, then prefer to use the global address as the nexthop. + +.. clicmd:: set ipv6 next-hop global IPV6_ADDRESS + + Set the next-hop to the specified IPV6_ADDRESS for both incoming and + outgoing route-maps. + +.. clicmd:: set local-preference LOCAL_PREF + + Set the BGP local preference to `local_pref`. + +.. clicmd:: set local-preference +LOCAL_PREF + + Add the BGP local preference to an existing `local_pref`. + +.. clicmd:: set local-preference -LOCAL_PREF + + Subtract the BGP local preference from an existing `local_pref`. + +.. clicmd:: set distance (1-255) + + Set the Administrative distance to use for the route. + This is only locally significant and will not be dispersed to peers. + +.. clicmd:: set weight WEIGHT + + Set the route's weight. + +.. clicmd:: set metric <[+|-](1-4294967295)|rtt|+rtt|-rtt> + + Set the route metric. When used with BGP, set the BGP attribute MED to a + specific value. Use `+`/`-` to add or subtract the specified value to/from + the existing/MED. Use `rtt` to set the MED to the round trip time or + `+rtt`/`-rtt` to add/subtract the round trip time to/from the MED. + +.. clicmd:: set min-metric <(0-4294967295)> + + Set the minimum meric for the route. + +.. clicmd:: set max-metric <(0-4294967295)> + + Set the maximum meric for the route. + +.. clicmd:: set aigp-metric <igp-metric|(1-4294967295)> + + Set the BGP attribute AIGP to a specific value. If ``igp-metric`` is specified, + then the value is taken from the IGP protocol, otherwise an arbitrary value. + +.. clicmd:: set as-path prepend AS_PATH + + Set the BGP AS path to prepend. + +.. clicmd:: set as-path exclude AS-NUMBER... + + Drop AS-NUMBER from the BGP AS path. + +.. clicmd:: set community COMMUNITY + + Set the BGP community attribute. + +.. clicmd:: set extended-comm-list <EXTCOMMUNITY_LIST_NAME> delete + + Set BGP extended community list for deletion. + +.. clicmd:: set ipv6 next-hop local IPV6_ADDRESS + + Set the BGP-4+ link local IPv6 nexthop address. + +.. clicmd:: set origin ORIGIN <egp|igp|incomplete> + + Set BGP route origin. + +.. clicmd:: set table (1-4294967295) + + Set the BGP table to a given table identifier + +.. clicmd:: set sr-te color (1-4294967295) + + Set the color of a SR-TE Policy to be applied to a learned route. The SR-TE + Policy is uniquely determined by the color and the BGP nexthop. + +.. clicmd:: set l3vpn next-hop encapsulation gre + + Accept L3VPN traffic over GRE encapsulation. + +.. _route-map-call-command: + +Route Map Call Command +====================== + +.. clicmd:: call NAME + + Call route-map `name`. If it returns deny, deny the route and + finish processing the route-map. + + +.. _route-map-exit-action-command: + +Route Map Exit Action Command +============================= + +.. clicmd:: on-match next + +.. clicmd:: continue + + Proceed on to the next entry in the route-map. + +.. clicmd:: on-match goto N + +.. clicmd:: continue N + + Proceed processing the route-map at the first entry whose order is >= N + + +.. _route-map-optimization-command: + +Route Map Optimization Command +============================== + +.. clicmd:: route-map ROUTE-MAP-NAME optimization + + Enable route-map processing optimization for `route-map-name`. + The optimization is enabled by default. + Instead of sequentially passing through all the route-map indexes + until a match is found, the search for the best-match index will be + based on a look-up in a prefix-tree. A per-route-map prefix-tree + will be constructed for this purpose. The prefix-tree will compose + of all the prefixes in all the prefix-lists that are included in the + match rule of all the sequences of a route-map. + + +Route Map Examples +================== + +A simple example of a route-map: + +.. code-block:: frr + + route-map test permit 10 + match ip address 10 + set local-preference 200 + + +This means that if a route matches ip access-list number 10 it's +local-preference value is set to 200. + +See :ref:`bgp-configuration-examples` for examples of more sophisticated +usage of route-maps, including of the ``call`` action. + diff --git a/doc/user/routeserver.rst b/doc/user/routeserver.rst new file mode 100644 index 0000000..969cd17 --- /dev/null +++ b/doc/user/routeserver.rst @@ -0,0 +1,542 @@ +.. _configuring-frr-as-a-route-server: + +Configuring FRR as a Route Server +================================= + +The purpose of a Route Server is to centralize the peerings between BGP +speakers. For example if we have an exchange point scenario with four BGP +speakers, each of which maintaining a BGP peering with the other three +(:ref:`fig-topologies-full`), we can convert it into a centralized scenario where +each of the four establishes a single BGP peering against the Route Server +(:ref:`fig-topologies-rs`). + +We will first describe briefly the Route Server model implemented by FRR. +We will explain the commands that have been added for configuring that +model. And finally we will show a full example of FRR configured as Route +Server. + +.. _description-of-the-route-server-model: + +Description of the Route Server model +------------------------------------- + +First we are going to describe the normal processing that BGP announcements +suffer inside a standard BGP speaker, as shown in :ref:`fig-normal-processing`, +it consists of three steps: + +- When an announcement is received from some peer, the `In` filters configured + for that peer are applied to the announcement. These filters can reject the + announcement, accept it unmodified, or accept it with some of its attributes + modified. + +- The announcements that pass the `In` filters go into the Best Path Selection + process, where they are compared to other announcements referred to the same + destination that have been received from different peers (in case such other + announcements exist). For each different destination, the announcement which + is selected as the best is inserted into the BGP speaker's Loc-RIB. + +- The routes which are inserted in the Loc-RIB are considered for announcement + to all the peers (except the one from which the route came). This is done by + passing the routes in the Loc-RIB through the `Out` filters corresponding to + each peer. These filters can reject the route, accept it unmodified, or + accept it with some of its attributes modified. Those routes which are + accepted by the `Out` filters of a peer are announced to that peer. + +.. _fig-normal-processing: + +.. figure:: ../figures/fig-normal-processing.png + :alt: Normal announcement processing + :align: center + + Announcement processing inside a 'normal' BGP speaker + +.. _fig-topologies-full: + +.. figure:: ../figures/fig_topologies_full.png + :alt: Full Mesh BGP Topology + :align: center + + Full Mesh + +.. _fig-topologies-rs: + +.. figure:: ../figures/fig_topologies_rs.png + :alt: Route Server BGP Topology + :align: center + + Route server and clients + +Of course we want that the routing tables obtained in each of the routers are +the same when using the route server than when not. But as a consequence of +having a single BGP peering (against the route server), the BGP speakers can no +longer distinguish from/to which peer each announce comes/goes. + +.. _filter-delegation: + +This means that the routers connected to the route server are not able to apply +by themselves the same input/output filters as in the full mesh scenario, so +they have to delegate those functions to the route server. + +Even more, the 'best path' selection must be also performed inside the route +server on behalf of its clients. The reason is that if, after applying the +filters of the announcer and the (potential) receiver, the route server decides +to send to some client two or more different announcements referred to the same +destination, the client will only retain the last one, considering it as an +implicit withdrawal of the previous announcements for the same destination. +This is the expected behavior of a BGP speaker as defined in :rfc:`1771`, +and even though there are some proposals of mechanisms that permit multiple +paths for the same destination to be sent through a single BGP peering, none +are currently supported by most existing BGP implementations. + +As a consequence a route server must maintain additional information and +perform additional tasks for a RS-client that those necessary for common BGP +peerings. Essentially a route server must: + +.. _route-server-tasks: + +- Maintain a separated Routing Information Base (Loc-RIB) + for each peer configured as RS-client, containing the routes + selected as a result of the 'Best Path Selection' process + that is performed on behalf of that RS-client. + +- Whenever it receives an announcement from a RS-client, + it must consider it for the Loc-RIBs of the other RS-clients. + + - This means that for each of them the route server must pass the + announcement through the appropriate `Out` filter of the + announcer. + + - Then through the appropriate `In` filter of the potential receiver. + + - Only if the announcement is accepted by both filters it will be passed + to the 'Best Path Selection' process. + + - Finally, it might go into the Loc-RIB of the receiver. + +When we talk about the 'appropriate' filter, both the announcer and the +receiver of the route must be taken into account. Suppose that the route server +receives an announcement from client A, and the route server is considering it +for the Loc-RIB of client B. The filters that should be applied are the same +that would be used in the full mesh scenario, i.e., first the `Out` filter of +router A for announcements going to router B, and then the `In` filter of +router B for announcements coming from router A. + +We call 'Export Policy' of a RS-client to the set of `Out` filters that the +client would use if there was no route server. The same applies for the 'Import +Policy' of a RS-client and the set of `In` filters of the client if there was +no route server. + +It is also common to demand from a route server that it does not modify some +BGP attributes (next-hop, as-path and MED) that are usually modified by +standard BGP speakers before announcing a route. + +The announcement processing model implemented by FRR is shown in +:ref:`fig-rs-processing`. The figure shows a mixture of RS-clients (B, C and D) +with normal BGP peers (A). There are some details that worth additional +comments: + +- Announcements coming from a normal BGP peer are also considered for the + Loc-RIBs of all the RS-clients. But logically they do not pass through any + export policy. + +- Those peers that are configured as RS-clients do not receive any announce + from the `Main` Loc-RIB. + +- Apart from import and export policies, `In` and `Out` filters can also be set + for RS-clients. `In` filters might be useful when the route server has also + normal BGP peers. On the other hand, `Out` filters for RS-clients are + probably unnecessary, but we decided not to remove them as they do not hurt + anybody (they can always be left empty). + +.. _fig-rs-processing: +.. figure:: ../figures/fig-rs-processing.png + :align: center + :alt: Route Server Processing Model + + Announcement processing model implemented by the Route Server + +.. _commands-for-configuring-a-route-server: + +Commands for configuring a Route Server +--------------------------------------- + +Now we will describe the commands that have been added to frr +in order to support the route server features. + +.. clicmd:: neighbor PEER-GROUP route-server-client + +.. clicmd:: neighbor A.B.C.D route-server-client + +.. clicmd:: neighbor X:X::X:X route-server-client + + This command configures the peer given by `peer`, `A.B.C.D` or `X:X::X:X` as + an RS-client. + + Actually this command is not new, it already existed in standard FRR. It + enables the transparent mode for the specified peer. This means that some + BGP attributes (as-path, next-hop and MED) of the routes announced to that + peer are not modified. + + With the route server patch, this command, apart from setting the + transparent mode, creates a new Loc-RIB dedicated to the specified peer + (those named `Loc-RIB for X` in :ref:`fig-rs-processing`.). Starting from + that moment, every announcement received by the route server will be also + considered for the new Loc-RIB. + +.. clicmd:: neigbor A.B.C.D|X.X::X.X|peer-group route-map WORD in|out + + This set of commands can be used to specify the route-map that represents + the Import or Export policy of a peer which is configured as a RS-client + (with the previous command). + +.. clicmd:: match peer A.B.C.D|X:X::X:X + + This is a new *match* statement for use in route-maps, enabling them to + describe import/export policies. As we said before, an import/export policy + represents a set of input/output filters of the RS-client. This statement + makes possible that a single route-map represents the full set of filters + that a BGP speaker would use for its different peers in a non-RS scenario. + + The *match peer* statement has different semantics whether it is used inside + an import or an export route-map. In the first case the statement matches if + the address of the peer who sends the announce is the same that the address + specified by {A.B.C.D|X:X::X:X}. For export route-maps it matches when + {A.B.C.D|X:X::X:X} is the address of the RS-Client into whose Loc-RIB the + announce is going to be inserted (how the same export policy is applied + before different Loc-RIBs is shown in :ref:`fig-rs-processing`.). + +.. clicmd:: call WORD + + This command (also used inside a route-map) jumps into a different + route-map, whose name is specified by `WORD`. When the called + route-map finishes, depending on its result the original route-map + continues or not. Apart from being useful for making import/export + route-maps easier to write, this command can also be used inside + any normal (in or out) route-map. + +.. _example-of-route-server-configuration: + +Example of Route Server Configuration +------------------------------------- + +Finally we are going to show how to configure a FRR daemon to act as a +Route Server. For this purpose we are going to present a scenario without +route server, and then we will show how to use the configurations of the BGP +routers to generate the configuration of the route server. + +All the configuration files shown in this section have been taken +from scenarios which were tested using the VNUML tool +`http://www.dit.upm.es/vnuml,VNUML <http://www.dit.upm.es/vnuml,VNUML>`_. + +.. _configuration-of-the-bgp-routers-without-route-server: + +Configuration of the BGP routers without Route Server +----------------------------------------------------- + +We will suppose that our initial scenario is an exchange point with three +BGP capable routers, named RA, RB and RC. Each of the BGP speakers generates +some routes (with the `network` command), and establishes BGP peerings +against the other two routers. These peerings have In and Out route-maps +configured, named like 'PEER-X-IN' or 'PEER-X-OUT'. For example the +configuration file for router RA could be the following: + +.. code-block:: frr + + #Configuration for router 'RA' + ! + hostname RA + password **** + ! + router bgp 65001 + no bgp default ipv4-unicast + neighbor 2001:0DB8::B remote-as 65002 + neighbor 2001:0DB8::C remote-as 65003 + ! + address-family ipv6 + network 2001:0DB8:AAAA:1::/64 + network 2001:0DB8:AAAA:2::/64 + network 2001:0DB8:0000:1::/64 + network 2001:0DB8:0000:2::/64 + neighbor 2001:0DB8::B activate + neighbor 2001:0DB8::B soft-reconfiguration inbound + neighbor 2001:0DB8::B route-map PEER-B-IN in + neighbor 2001:0DB8::B route-map PEER-B-OUT out + neighbor 2001:0DB8::C activate + neighbor 2001:0DB8::C soft-reconfiguration inbound + neighbor 2001:0DB8::C route-map PEER-C-IN in + neighbor 2001:0DB8::C route-map PEER-C-OUT out + exit-address-family + ! + ipv6 prefix-list COMMON-PREFIXES seq 5 permit 2001:0DB8:0000::/48 ge 64 le 64 + ipv6 prefix-list COMMON-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-A-PREFIXES seq 5 permit 2001:0DB8:AAAA::/48 ge 64 le 64 + ipv6 prefix-list PEER-A-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-B-PREFIXES seq 5 permit 2001:0DB8:BBBB::/48 ge 64 le 64 + ipv6 prefix-list PEER-B-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-C-PREFIXES seq 5 permit 2001:0DB8:CCCC::/48 ge 64 le 64 + ipv6 prefix-list PEER-C-PREFIXES seq 10 deny any + ! + route-map PEER-B-IN permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set metric 100 + route-map PEER-B-IN permit 20 + match ipv6 address prefix-list PEER-B-PREFIXES + set community 65001:11111 + ! + route-map PEER-C-IN permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set metric 200 + route-map PEER-C-IN permit 20 + match ipv6 address prefix-list PEER-C-PREFIXES + set community 65001:22222 + ! + route-map PEER-B-OUT permit 10 + match ipv6 address prefix-list PEER-A-PREFIXES + ! + route-map PEER-C-OUT permit 10 + match ipv6 address prefix-list PEER-A-PREFIXES + ! + line vty + ! + + +.. _configuration-of-the-bgp-routers-with-route-server: + +Configuration of the BGP routers with Route Server +-------------------------------------------------- + +To convert the initial scenario into one with route server, first we must +modify the configuration of routers RA, RB and RC. Now they must not peer +between them, but only with the route server. For example, RA's +configuration would turn into: + +.. code-block:: frr + + # Configuration for router 'RA' + ! + hostname RA + password **** + ! + router bgp 65001 + no bgp default ipv4-unicast + neighbor 2001:0DB8::FFFF remote-as 65000 + ! + address-family ipv6 + network 2001:0DB8:AAAA:1::/64 + network 2001:0DB8:AAAA:2::/64 + network 2001:0DB8:0000:1::/64 + network 2001:0DB8:0000:2::/64 + + neighbor 2001:0DB8::FFFF activate + neighbor 2001:0DB8::FFFF soft-reconfiguration inbound + exit-address-family + ! + line vty + ! + + +Which is logically much simpler than its initial configuration, as it now +maintains only one BGP peering and all the filters (route-maps) have +disappeared. + +.. _configuration-of-the-route-server-itself: + +Configuration of the Route Server itself +---------------------------------------- + +As we said when we described the functions of a route server +(:ref:`description-of-the-route-server-model`), it is in charge of all the +route filtering. To achieve that, the In and Out filters from the RA, RB and RC +configurations must be converted into Import and Export policies in the route +server. + +This is a fragment of the route server configuration (we only show +the policies for client RA): + +.. code-block:: frr + + # Configuration for Route Server ('RS') + ! + hostname RS + password ix + ! + router bgp 65000 view RS + no bgp default ipv4-unicast + neighbor 2001:0DB8::A remote-as 65001 + neighbor 2001:0DB8::B remote-as 65002 + neighbor 2001:0DB8::C remote-as 65003 + ! + address-family ipv6 + neighbor 2001:0DB8::A activate + neighbor 2001:0DB8::A route-server-client + neighbor 2001:0DB8::A route-map RSCLIENT-A-IMPORT in + neighbor 2001:0DB8::A route-map RSCLIENT-A-EXPORT out + neighbor 2001:0DB8::A soft-reconfiguration inbound + + neighbor 2001:0DB8::B activate + neighbor 2001:0DB8::B route-server-client + neighbor 2001:0DB8::B route-map RSCLIENT-B-IMPORT in + neighbor 2001:0DB8::B route-map RSCLIENT-B-EXPORT out + neighbor 2001:0DB8::B soft-reconfiguration inbound + + neighbor 2001:0DB8::C activate + neighbor 2001:0DB8::C route-server-client + neighbor 2001:0DB8::C route-map RSCLIENT-C-IMPORT in + neighbor 2001:0DB8::C route-map RSCLIENT-C-EXPORT out + neighbor 2001:0DB8::C soft-reconfiguration inbound + exit-address-family + ! + ipv6 prefix-list COMMON-PREFIXES seq 5 permit 2001:0DB8:0000::/48 ge 64 le 64 + ipv6 prefix-list COMMON-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-A-PREFIXES seq 5 permit 2001:0DB8:AAAA::/48 ge 64 le 64 + ipv6 prefix-list PEER-A-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-B-PREFIXES seq 5 permit 2001:0DB8:BBBB::/48 ge 64 le 64 + ipv6 prefix-list PEER-B-PREFIXES seq 10 deny any + ! + ipv6 prefix-list PEER-C-PREFIXES seq 5 permit 2001:0DB8:CCCC::/48 ge 64 le 64 + ipv6 prefix-list PEER-C-PREFIXES seq 10 deny any + ! + route-map RSCLIENT-A-IMPORT permit 10 + match peer 2001:0DB8::B + call A-IMPORT-FROM-B + route-map RSCLIENT-A-IMPORT permit 20 + match peer 2001:0DB8::C + call A-IMPORT-FROM-C + ! + route-map A-IMPORT-FROM-B permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set metric 100 + route-map A-IMPORT-FROM-B permit 20 + match ipv6 address prefix-list PEER-B-PREFIXES + set community 65001:11111 + ! + route-map A-IMPORT-FROM-C permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set metric 200 + route-map A-IMPORT-FROM-C permit 20 + match ipv6 address prefix-list PEER-C-PREFIXES + set community 65001:22222 + ! + route-map RSCLIENT-A-EXPORT permit 10 + match peer 2001:0DB8::B + match ipv6 address prefix-list PEER-A-PREFIXES + route-map RSCLIENT-A-EXPORT permit 20 + match peer 2001:0DB8::C + match ipv6 address prefix-list PEER-A-PREFIXES + ! + ... + ... + ... + + +If you compare the initial configuration of RA with the route server +configuration above, you can see how easy it is to generate the Import and +Export policies for RA from the In and Out route-maps of RA's original +configuration. + +When there was no route server, RA maintained two peerings, one with RB and +another with RC. Each of this peerings had an In route-map configured. To +build the Import route-map for client RA in the route server, simply add +route-map entries following this scheme: + +:: + + route-map <NAME> permit 10 + match peer <Peer Address> + call <In Route-Map for this Peer> + route-map <NAME> permit 20 + match peer <Another Peer Address> + call <In Route-Map for this Peer> + + +This is exactly the process that has been followed to generate the route-map +RSCLIENT-A-IMPORT. The route-maps that are called inside it (A-IMPORT-FROM-B +and A-IMPORT-FROM-C) are exactly the same than the In route-maps from the +original configuration of RA (PEER-B-IN and PEER-C-IN), only the name is +different. + +The same could have been done to create the Export policy for RA (route-map +RSCLIENT-A-EXPORT), but in this case the original Out route-maps where so +simple that we decided not to use the `call WORD` commands, and we +integrated all in a single route-map (RSCLIENT-A-EXPORT). + +The Import and Export policies for RB and RC are not shown, but +the process would be identical. + +Further considerations about Import and Export route-maps +--------------------------------------------------------- + +The current version of the route server patch only allows to specify a +route-map for import and export policies, while in a standard BGP speaker +apart from route-maps there are other tools for performing input and output +filtering (access-lists, community-lists, ...). But this does not represent +any limitation, as all kinds of filters can be included in import/export +route-maps. For example suppose that in the non-route-server scenario peer +RA had the following filters configured for input from peer B: + +.. code-block:: frr + + neighbor 2001:0DB8::B prefix-list LIST-1 in + neighbor 2001:0DB8::B filter-list LIST-2 in + neighbor 2001:0DB8::B route-map PEER-B-IN in + ... + ... + route-map PEER-B-IN permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set local-preference 100 + route-map PEER-B-IN permit 20 + match ipv6 address prefix-list PEER-B-PREFIXES + set community 65001:11111 + + +It is possible to write a single route-map which is equivalent to the three +filters (the community-list, the prefix-list and the route-map). That route-map +can then be used inside the Import policy in the route server. Lets see how to +do it: + +.. code-block:: frr + + neighbor 2001:0DB8::A route-map RSCLIENT-A-IMPORT in + ... + ! + ... + route-map RSCLIENT-A-IMPORT permit 10 + match peer 2001:0DB8::B + call A-IMPORT-FROM-B + ... + ... + ! + route-map A-IMPORT-FROM-B permit 1 + match ipv6 address prefix-list LIST-1 + match as-path LIST-2 + on-match goto 10 + route-map A-IMPORT-FROM-B deny 2 + route-map A-IMPORT-FROM-B permit 10 + match ipv6 address prefix-list COMMON-PREFIXES + set local-preference 100 + route-map A-IMPORT-FROM-B permit 20 + match ipv6 address prefix-list PEER-B-PREFIXES + set community 65001:11111 + ! + ... + ... + + +The route-map A-IMPORT-FROM-B is equivalent to the three filters (LIST-1, +LIST-2 and PEER-B-IN). The first entry of route-map A-IMPORT-FROM-B (sequence +number 1) matches if and only if both the prefix-list LIST-1 and the +filter-list LIST-2 match. If that happens, due to the 'on-match goto 10' +statement the next route-map entry to be processed will be number 10, and as of +that point route-map A-IMPORT-FROM-B is identical to PEER-B-IN. If the first +entry does not match, `on-match goto 10`' will be ignored and the next +processed entry will be number 2, which will deny the route. + +Thus, the result is the same that with the three original filters, i.e., if +either LIST-1 or LIST-2 rejects the route, it does not reach the route-map +PEER-B-IN. In case both LIST-1 and LIST-2 accept the route, it passes to +PEER-B-IN, which can reject, accept or modify the route. diff --git a/doc/user/rpki.rst b/doc/user/rpki.rst new file mode 100644 index 0000000..4053536 --- /dev/null +++ b/doc/user/rpki.rst @@ -0,0 +1,284 @@ +.. _prefix-origin-validation-using-rpki: + +Prefix Origin Validation Using RPKI +=================================== + +Prefix Origin Validation allows BGP routers to verify if the origin AS of an IP +prefix is legitimate to announce this IP prefix. The required attestation +objects are stored in the Resource Public Key Infrastructure (:abbr:`RPKI`). +However, RPKI-enabled routers do not store cryptographic data itself but only +validation information. The validation of the cryptographic data (so called +Route Origin Authorization, or short :abbr:`ROA`, objects) will be performed by +trusted cache servers. The RPKI/RTR protocol defines a standard mechanism to +maintain the exchange of the prefix/origin AS mapping between the cache server +and routers. In combination with a BGP Prefix Origin Validation scheme a +router is able to verify received BGP updates without suffering from +cryptographic complexity. + +The RPKI/RTR protocol is defined in :rfc:`6810` and the validation scheme in +:rfc:`6811`. The current version of Prefix Origin Validation in FRR implements +both RFCs. + +For a more detailed but still easy-to-read background, we suggest: + +- [Securing-BGP]_ +- [Resource-Certification]_ + +.. _features-of-the-current-implementation: + +Features of the Current Implementation +-------------------------------------- + +In a nutshell, the current implementation provides the following features + +- The BGP router can connect to one or more RPKI cache servers to receive + validated prefix to origin AS mappings. Advanced failover can be implemented + by server sockets with different preference values. +- If no connection to an RPKI cache server can be established after a + pre-defined timeout, the router will process routes without prefix origin + validation. It still will try to establish a connection to an RPKI cache + server in the background. +- By default, enabling RPKI does not change best path selection. In particular, + invalid prefixes will still be considered during best path selection. + However, the router can be configured to ignore all invalid prefixes. +- Route maps can be configured to match a specific RPKI validation state. This + allows the creation of local policies, which handle BGP routes based on the + outcome of the Prefix Origin Validation. +- Updates from the RPKI cache servers are directly applied and path selection + is updated accordingly. (Soft reconfiguration **must** be enabled for this + to work). + + +.. _enabling-rpki: + +Enabling RPKI +------------- + +You must install ``frr-rpki-rtrlib`` additional package for RPKI support, +otherwise ``bgpd`` daemon won't startup. + +.. clicmd:: rpki + + This command enables the RPKI configuration mode. Most commands that start + with *rpki* can only be used in this mode. + + When it is used in a telnet session, leaving of this mode cause rpki to be + initialized. + + Executing this command alone does not activate prefix validation. You need + to configure at least one reachable cache server. See section + :ref:`configuring-rpki-rtr-cache-servers` for configuring a cache server. + +Remember to add ``-M rpki`` to the variable ``bgpd_options`` in +:file:`/etc/frr/daemons` , like so:: + + bgpd_options=" -A 127.0.0.1 -M rpki" + +instead of the default setting:: + + bgpd_options=" -A 127.0.0.1" + +Otherwise you will encounter an error when trying to enter RPKI +configuration mode due to the ``rpki`` module not being loaded when the BGP +daemon is initialized. + +Examples of the error:: + + router(config)# debug rpki + % [BGP] Unknown command: debug rpki + + router(config)# rpki + % [BGP] Unknown command: rpki + +Note that the RPKI commands will be available in vtysh when running +``find rpki`` regardless of whether the module is loaded. + +.. _configuring-rpki-rtr-cache-servers: + +Configuring RPKI/RTR Cache Servers +---------------------------------- + +The following commands are independent of a specific cache server. + +.. clicmd:: rpki polling_period (1-3600) + + Set the number of seconds the router waits until the router asks the cache + again for updated data. + + The default value is 300 seconds. + +.. clicmd:: rpki expire_interval (600-172800) + + Set the number of seconds the router waits until the router expires the cache. + + The default value is 7200 seconds. + +.. clicmd:: rpki retry_interval (1-7200) + + Set the number of seconds the router waits until retrying to connect to the + cache server. + + The default value is 600 seconds. + +.. clicmd:: rpki cache (A.B.C.D|WORD) PORT [SSH_USERNAME] [SSH_PRIVKEY_PATH] [KNOWN_HOSTS_PATH] [source A.B.C.D] preference (1-255) + + + Add a cache server to the socket. By default, the connection between router + and cache server is based on plain TCP. Protecting the connection between + router and cache server by SSH is optional. Deleting a socket removes the + associated cache server and terminates the existing connection. + + A.B.C.D|WORD + Address of the cache server. + + PORT + Port number to connect to the cache server + + SSH_USERNAME + SSH username to establish an SSH connection to the cache server. + + SSH_PRIVKEY_PATH + Local path that includes the private key file of the router. + + KNOWN_HOSTS_PATH + Local path that includes the known hosts file. The default value depends + on the configuration of the operating system environment, usually + :file:`~/.ssh/known_hosts`. + + source A.B.C.D + Source address of the RPKI connection to access cache server. + + +.. _validating-bgp-updates: + +Validating BGP Updates +---------------------- + +.. clicmd:: match rpki notfound|invalid|valid + + + Create a clause for a route map to match prefixes with the specified RPKI + state. + + In the following example, the router prefers valid routes over invalid + prefixes because invalid routes have a lower local preference. + + .. code-block:: frr + + ! Allow for invalid routes in route selection process + route bgp 60001 + ! + ! Set local preference of invalid prefixes to 10 + route-map rpki permit 10 + match rpki invalid + set local-preference 10 + ! + ! Set local preference of valid prefixes to 500 + route-map rpki permit 500 + match rpki valid + set local-preference 500 + +.. clicmd:: match rpki-extcommunity notfound|invalid|valid + + Create a clause for a route map to match prefixes with the specified RPKI + state, that is derived from the Origin Validation State extended community + attribute (OVS). OVS extended community is non-transitive and is exchanged + only between iBGP peers. + +.. _debugging: + +Debugging +--------- + +.. clicmd:: debug rpki + + + Enable or disable debugging output for RPKI. + +.. _displaying-rpki: + +Displaying RPKI +--------------- + +.. clicmd:: show rpki prefix <A.B.C.D/M|X:X::X:X/M> [(1-4294967295)] [json] + + Display validated prefixes received from the cache servers filtered + by the specified prefix. + +.. clicmd:: show rpki as-number ASN [json] + + Display validated prefixes received from the cache servers filtered + by ASN. + +.. clicmd:: show rpki prefix-table [json] + + Display all validated prefix to origin AS mappings/records which have been + received from the cache servers and stored in the router. Based on this data, + the router validates BGP Updates. + +.. clicmd:: show rpki cache-server [json] + + Display all configured cache servers, whether active or not. + +.. clicmd:: show rpki cache-connection [json] + + Display all cache connections, and show which is connected or not. + +.. clicmd:: show bgp [afi] [safi] <A.B.C.D|A.B.C.D/M|X:X::X:X|X:X::X:X/M> rpki <valid|invalid|notfound> + + Display for the specified prefix or address the bgp paths that match the given rpki state. + +.. clicmd:: show bgp [afi] [safi] rpki <valid|invalid|notfound> + + Display all prefixes that match the given rpki state. + +RPKI Configuration Example +-------------------------- + +.. code-block:: frr + + hostname bgpd1 + password zebra + ! log stdout + debug bgp updates + debug bgp keepalives + debug rpki + ! + rpki + rpki polling_period 1000 + rpki timeout 10 + ! SSH Example: + rpki cache example.com source 141.22.28.223 22 rtr-ssh ./ssh_key/id_rsa ./ssh_key/id_rsa.pub preference 1 + ! TCP Example: + rpki cache rpki-validator.realmv6.org 8282 preference 2 + exit + ! + router bgp 60001 + bgp router-id 141.22.28.223 + network 192.168.0.0/16 + neighbor 123.123.123.0 remote-as 60002 + neighbor 123.123.123.0 route-map rpki in + neighbor 123.123.123.0 update-source 141.22.28.223 + ! + address-family ipv6 + neighbor 123.123.123.0 activate + neighbor 123.123.123.0 route-map rpki in + exit-address-family + ! + route-map rpki permit 10 + match rpki invalid + set local-preference 10 + ! + route-map rpki permit 20 + match rpki notfound + set local-preference 20 + ! + route-map rpki permit 30 + match rpki valid + set local-preference 30 + ! + route-map rpki permit 40 + ! + +.. [Securing-BGP] Geoff Huston, Randy Bush: Securing BGP, In: The Internet Protocol Journal, Volume 14, No. 2, 2011. <https://www.cisco.com/c/dam/en_us/about/ac123/ac147/archived_issues/ipj_14-2/ipj_14-2.pdf> +.. [Resource-Certification] Geoff Huston: Resource Certification, In: The Internet Protocol Journal, Volume 12, No.1, 2009. <https://www.cisco.com/c/dam/en_us/about/ac123/ac147/archived_issues/ipj_12-1/ipj_12-1.pdf> diff --git a/doc/user/scripting.rst b/doc/user/scripting.rst new file mode 100644 index 0000000..42855de --- /dev/null +++ b/doc/user/scripting.rst @@ -0,0 +1,85 @@ +.. _scripting-user: + +********* +Scripting +********* + +The behavior of FRR may be extended or customized using its built-in scripting +capabilities. The scripting language is Lua 5.3. This guide assumes Lua +knowledge. For more information on Lua, consult the Lua 5.3 reference manual, or +*Programming in Lua* (note that the free version covers only Lua 5.0). + +https://www.lua.org/manual/5.3/ + +http://www.lua.org/pil/contents.html + +Scripting +========= + +.. seealso:: Developer docs for scripting + +How to use +---------- + +1. Identify the Lua function name. See :ref:`lua-hook-calls`. + +2. Write the Lua script + +3. Configure FRR to use the Lua script + +In order to use scripting, FRR must be built with ``--enable-scripting``. + +.. note:: + + Scripts are typically loaded just-in-time. This means you can change the + contents of a script that is in use without restarting FRR. Not all + scripting locations may behave this way; refer to the documentation for the + particular location. + + +Example: on_rib_process_dplane_results +-------------------------------------- + +This example shows how to write a Lua script that logs changes when a route is +added. + +First, identify the Lua hook call to attach a Lua function to: this will be the +name of the Lua function. In this case, since the hook call is +`on_rib_process_dplane_results`: + +.. code-block:: lua + + function on_rib_process_dplane_results(ctx) + log.info(ctx.rinfo.zd_dest.network) + return {} + + +The documentation for :ref:`on-rib-process-dplane-results` tells us its +arguments. Here, the destination prefix for a route is being logged out. + +Scripts live in :file:`/etc/frr/scripts/` by default. This is configurable at +compile time via ``--with-scriptdir``. It may be overridden at runtime with the +``--scriptdir`` daemon option. + +The documentation for :ref:`on-rib-process-dplane-results` indicates that the +``script`` command should be used to set the script. Assuming that the above +function was created in :file:`/etc/frr/scripts/my_dplane_script.lua`, the +following vtysh command sets the script for the hook call: + +.. code-block:: console + + script on_rib_process_dplane_results my_dplane_script + + +After the script is set, when the hook call is hit, FRR will look for a +*on_rib_process_dplane_results* function in +:file:`/etc/frr/scripts/my_dplane_script.lua` and run it with the ``ctx`` object +as its argument. + + +.. _lua-hook-calls: + +Available Lua hook calls +======================== + +:ref:`on-rib-process-dplane-results` diff --git a/doc/user/setup.rst b/doc/user/setup.rst new file mode 100644 index 0000000..372494f --- /dev/null +++ b/doc/user/setup.rst @@ -0,0 +1,325 @@ +.. _basic-setup: + +Basic Setup +============ + +After installing FRR, some basic configuration must be completed before it is +ready to use. + +Crash logs +---------- + +If any daemon should crash for some reason (segmentation fault, assertion +failure, etc.), it will attempt to write a backtrace to a file located in +:file:`/var/tmp/frr/<daemon>[-<instance>].<pid>/crashlog`. This feature is +not affected by any configuration options. + +The crashlog file's directory also contains files corresponding to per-thread +message buffers in files named +:file:`/var/tmp/frr/<daemon>[-<instance>].<pid>/logbuf.<tid>`. In case of a +crash, these may contain unwritten buffered log messages. To show the contents +of these buffers, pipe their contents through ``tr '\0' '\n'``. A blank line +marks the end of valid unwritten data (it will generally be followed by +garbled, older log messages since the buffer is not cleared.) + +.. _daemons-configuration-file: + +Daemons Configuration File +-------------------------- +After a fresh install, starting FRR will do nothing. This is because daemons +must be explicitly enabled by editing a file in your configuration directory. +This file is usually located at :file:`/etc/frr/daemons` and determines which +daemons are activated when issuing a service start / stop command via init or +systemd. The file initially looks like this: + +:: + + zebra=no + bgpd=no + ospfd=no + ospf6d=no + ripd=no + ripngd=no + isisd=no + pimd=no + ldpd=no + nhrpd=no + eigrpd=no + babeld=no + sharpd=no + staticd=no + pbrd=no + bfdd=no + fabricd=no + + # + # If this option is set the /etc/init.d/frr script automatically loads + # the config via "vtysh -b" when the servers are started. + # Check /etc/pam.d/frr if you intend to use "vtysh"! + # + vtysh_enable=yes + zebra_options=" -s 90000000 --daemon -A 127.0.0.1" + bgpd_options=" --daemon -A 127.0.0.1" + ospfd_options=" --daemon -A 127.0.0.1" + ospf6d_options=" --daemon -A ::1" + ripd_options=" --daemon -A 127.0.0.1" + ripngd_options=" --daemon -A ::1" + isisd_options=" --daemon -A 127.0.0.1" + pimd_options=" --daemon -A 127.0.0.1" + ldpd_options=" --daemon -A 127.0.0.1" + nhrpd_options=" --daemon -A 127.0.0.1" + eigrpd_options=" --daemon -A 127.0.0.1" + babeld_options=" --daemon -A 127.0.0.1" + sharpd_options=" --daemon -A 127.0.0.1" + staticd_options=" --daemon -A 127.0.0.1" + pbrd_options=" --daemon -A 127.0.0.1" + bfdd_options=" --daemon -A 127.0.0.1" + fabricd_options=" --daemon -A 127.0.0.1" + + #MAX_FDS=1024 + # The list of daemons to watch is automatically generated by the init script. + #watchfrr_options="" + + # for debugging purposes, you can specify a "wrap" command to start instead + # of starting the daemon directly, e.g. to use valgrind on ospfd: + # ospfd_wrap="/usr/bin/valgrind" + # or you can use "all_wrap" for all daemons, e.g. to use perf record: + # all_wrap="/usr/bin/perf record --call-graph -" + # the normal daemon command is added to this at the end. + +Breaking this file down: + +:: + + bgpd=yes + +To enable a particular daemon, simply change the corresponding 'no' to 'yes'. +Subsequent service restarts should start the daemon. + +:: + + vtysh_enable=yes + +As the comment says, this causes :ref:`VTYSH <vty-shell>` to apply +configuration when starting the daemons. This is useful for a variety of +reasons touched on in the VTYSH documentation and should generally be enabled. + +:: + + MAX_FDS=1024 + +This allows the operator to control the number of open file descriptors +each daemon is allowed to start with. The current assumed value on +most operating systems is 1024. If the operator plans to run bgp with +several thousands of peers then this is where we would modify FRR to +allow this to happen. + +:: + + FRR_NO_ROOT="yes" + +This option allows you to run FRR as a non-root user. Use this option +only when you know what you are doing since most of the daemons +in FRR will not be able to run under a regular user. This option +is useful for example when you run FRR in a container with a designated +user instead of root. + +:: + + zebra_options=" -s 90000000 --daemon -A 127.0.0.1" + bgpd_options=" --daemon -A 127.0.0.1" + ... + +The next set of lines controls what options are passed to daemons when started +from the service script. Usually daemons will have ``--daemon`` and ``-A +<address>`` specified in order to daemonize and listen for VTY commands on a +particular address. + +The remaining file content regarding `watchfrr_options` and `*_wrap` settings +should not normally be needed; refer to the comments in case they are. + +Services +-------- +FRR daemons have their own terminal interface or VTY. After installation, it's +a good idea to setup each daemon's port number to connect to them. To do this +add the following entries to :file:`/etc/services`. + +:: + + zebrasrv 2600/tcp # zebra service + zebra 2601/tcp # zebra vty + ripd 2602/tcp # RIPd vty + ripngd 2603/tcp # RIPngd vty + ospfd 2604/tcp # OSPFd vty + bgpd 2605/tcp # BGPd vty + ospf6d 2606/tcp # OSPF6d vty + ospfapi 2607/tcp # ospfapi + isisd 2608/tcp # ISISd vty + babeld 2609/tcp # BABELd vty + nhrpd 2610/tcp # nhrpd vty + pimd 2611/tcp # PIMd vty + ldpd 2612/tcp # LDPd vty + eigprd 2613/tcp # EIGRPd vty + bfdd 2617/tcp # bfdd vty + fabricd 2618/tcp # fabricd vty + vrrpd 2619/tcp # vrrpd vty + + +If you use a FreeBSD newer than 2.2.8, the above entries are already added to +:file:`/etc/services` so there is no need to add it. If you specify a port +number when starting the daemon, these entries may not be needed. + +You may need to make changes to the config files in |INSTALL_PREFIX_ETC|. + +Systemd +------- +Although not installed when installing from source, FRR provides a service file +for use with ``systemd``. It is located in :file:`tools/frr.service` in the Git +repository. If ``systemctl status frr.service`` indicates that the FRR service +is not found, copy the service file from the Git repository into your preferred +location. A good place is usually ``/etc/systemd/system/``. + +After issuing a ``systemctl daemon-reload``, you should be able to start the +FRR service via ``systemctl start frr``. If this fails, or no daemons are +started. check the ``journalctl`` logs for an indication of what went wrong. + +Operations +---------- + +This section covers a few common operational tasks and how to perform them. + +Interactive Shell +^^^^^^^^^^^^^^^^^ +FRR offers an IOS-like interactive shell called ``vtysh`` where a user can run +individual configuration or show commands. To get into this shell, issue the +``vtysh`` command from either a privilege user (root, or with sudo) or a user +account that is part of the ``frrvty`` group. +e.g. + +.. code-block:: console + + root@ub18:~# vtysh + + Hello, this is FRRouting (version 8.1-dev). + Copyright 1996-2005 Kunihiro Ishiguro, et al. + + ub18# + +.. note:: + The default install location for vtysh is /usr/bin/vtysh + + +Restarting +^^^^^^^^^^ + +Restarting kills all running FRR daemons and starts them again. Any unsaved +configuration will be lost. + +.. code-block:: console + + service frr restart + +.. note:: + + Alternatively, you can invoke the init script directly:: + + /etc/init.d/frr restart + + Or, if using systemd:: + + systemctl restart frr + +Reloading +^^^^^^^^^ + +Reloading applies the differential between on-disk configuration and the +current effective configuration of running FRR processes. This includes +starting daemons that were previously stopped and any changes made to +individual or unified daemon configuration files. + +.. code-block:: console + + service frr reload + +.. note:: + + Alternatively, you can invoke the init script directly:: + + /etc/init.d/frr reload + + Or, if using systemd:: + + systemctl reload frr + +See :ref:`FRR-RELOAD <frr-reload>` for more about the `frr-reload.py` script. + + +Starting a new daemon +^^^^^^^^^^^^^^^^^^^^^ + +Suppose *bgpd* and *zebra* are running, and you wish to start *pimd*. In +``/etc/frr/daemons`` make the following change: + +.. code-block:: diff + + - pimd=no + + pimd=yes + +Then perform a reload. + +Currently there is no way to stop or restart an individual daemon. This is +because FRR's monitoring program cannot currently distinguish between a crashed +/ killed daemon versus one that has been intentionally stopped or restarted. +The closest that can be achieved is to remove all configuration for the daemon, +and set its line in ``/etc/frr/daemons`` to ``=no``. Once this is done, the +daemon will be stopped the next time FRR is restarted. + + +Network Namespaces +^^^^^^^^^^^^^^^^^^ + +It is possible to run FRR in different network namespaces so it can be +further compartmentalized (e.g. confining to a smaller subset network). +The network namespace configuration can be used in the default FRR +configuration pathspace or it can be used in a different pathspace +(`-N/--pathspace`). + +To use FRR network namespace in the default pathspace you should add +or uncomment the ``watchfrr_options`` line in ``/etc/frr/daemons``: + +.. code-block:: diff + + - #watchfrr_options="--netns" + + watchfrr_options="--netns=<network-namespace-name>" + +If you want to use a different pathspace with the network namespace +(the recommended way) you should add/uncomment the ``watchfrr_options`` +line in ``/etc/frr/<namespace>/daemons``: + +.. code-block:: diff + + - #watchfrr_options="--netns" + + #watchfrr_options="--netns=<network-namespace-name>" + + + + # `--netns` argument is optional and if not provided it will + + # default to the pathspace name. + + watchfrr_options="--netns" + +To start FRR in the new pathspace+network namespace the initialization script +should be called with an extra parameter: + + +.. code:: + + /etc/init.d/frr start <pathspace-name> + + +.. note:: + + Some Linux distributions might not use the default init script + shipped with FRR, in that case you might want to try running the + bundled script in ``/usr/lib/frr/frrinit.sh``. + + On systemd you might create different units or parameterize the + existing one. See the man page: + https://www.freedesktop.org/software/systemd/man/systemd.unit.html diff --git a/doc/user/sharp.rst b/doc/user/sharp.rst new file mode 100644 index 0000000..3e73a59 --- /dev/null +++ b/doc/user/sharp.rst @@ -0,0 +1,305 @@ +.. _sharp: + +***** +SHARP +***** + +:abbr:`SHARP (Super Happy Advanced Routing Process)` is a daemon that provides +miscellaneous functionality used for testing FRR and creating proof-of-concept +labs. + +.. _starting-sharp: + +Starting SHARP +============== + +Default configuration file for *sharpd* is :file:`sharpd.conf`. The typical +location of :file:`sharpd.conf` is |INSTALL_PREFIX_ETC|/sharpd.conf. + +If the user is using integrated config, then :file:`sharpd.conf` need not be +present and the :file:`frr.conf` is read instead. + +.. program:: sharpd + +:abbr:`SHARP` supports all the common FRR daemon start options which are +documented elsewhere. + +.. _using-sharp: + +Using SHARP +=========== + +All sharp commands are under the enable node and preceded by the ``sharp`` +keyword. At present, no sharp commands will be preserved in the config. + +.. clicmd:: sharp install routes A.B.C.D <nexthop <E.F.G.H|X:X::X:X>|nexthop-group NAME> (1-1000000) [instance (0-255)] [repeat (2-1000)] [opaque WORD] + + Install up to 1,000,000 (one million) /32 routes starting at ``A.B.C.D`` + with specified nexthop ``E.F.G.H`` or ``X:X::X:X``. The nexthop is + a ``NEXTHOP_TYPE_IPV4`` or ``NEXTHOP_TYPE_IPV6`` and must be reachable + to be installed into the kernel. Alternatively a nexthop-group NAME + can be specified and used as the nexthops. The routes are installed into + zebra as ``ZEBRA_ROUTE_SHARP`` and can be used as part of a normal route + redistribution. Route installation time is noted in the debug + log. When zebra successfully installs a route into the kernel and SHARP + receives success notifications for all routes this is logged as well. + Instance (0-255) if specified causes the routes to be installed in a different + instance. If repeat is used then we will install/uninstall the routes the + number of times specified. If the keyword opaque is specified then the + next word is sent down to zebra as part of the route installation. + +.. clicmd:: sharp remove routes A.B.C.D (1-1000000) + + Remove up to 1,000,000 (one million) /32 routes starting at ``A.B.C.D``. The + routes are removed from zebra. Route deletion start is noted in the debug + log and when all routes have been successfully deleted the debug log will be + updated with this information as well. + +.. clicmd:: sharp data route + + Allow end user doing route install and deletion to get timing information + from the vty or vtysh instead of having to read the log file. This command + is informational only and you should look at sharp_vty.c for explanation + of the output as that it may change. + +.. clicmd:: sharp label <ipv4|ipv6> vrf NAME label (0-1000000) + + Install a label into the kernel that causes the specified vrf NAME table to + be used for pop and forward operations when the specified label is seen. + +.. clicmd:: sharp watch <nexthop <A.B.C.D|X:X::X:X>|import <A.B.C.D/M:X:X::X:X/M> [connected] + + Instruct zebra to monitor and notify sharp when the specified nexthop is + changed. The notification from zebra is written into the debug log. + The nexthop or import choice chooses the type of nexthop we are asking + zebra to watch for us. This choice affects zebra's decision on what + matches. Connected tells zebra whether or not that we want the route + matched against to be a static or connected route for the nexthop keyword, + for the import keyword connected means exact match. The no form of + the command obviously turns this watching off. + +.. clicmd:: sharp data nexthop + + Allow end user to dump associated data with the nexthop tracking that + may have been turned on. + +.. clicmd:: sharp watch [vrf NAME] redistribute ROUTETYPE + + Allow end user to monitor redistributed routes of ROUTETYPE + origin. + +.. clicmd:: sharp lsp [update] (0-100000) nexthop-group NAME [prefix A.B.C.D/M TYPE [instance (0-255)]] + + Install an LSP using the specified in-label, with nexthops as + listed in nexthop-group ``NAME``. If ``update`` is included, the + update path is used. The LSP is installed as type ZEBRA_LSP_SHARP. + If ``prefix`` is specified, an existing route with type ``TYPE`` + (and optional ``instance`` id) will be updated to use the LSP. + +.. clicmd:: sharp remove lsp (0-100000) nexthop-group NAME [prefix A.B.C.D/M TYPE [instance (0-255)]] + + Remove a SHARPD LSP that uses the specified in-label, where the + nexthops are specified in nexthop-group ``NAME``. If ``prefix`` is + specified, remove label bindings from the route of type ``TYPE`` + also. + +.. clicmd:: sharp send opaque type (1-255) (1-1000) + + Send opaque ZAPI messages with subtype ``type``. Sharpd will send + a stream of messages if the count is greater than one. + +.. clicmd:: sharp send opaque unicast type (1-255) PROTOCOL [{instance (0-1000) | session (1-1000)}] (1-1000) + + Send unicast opaque ZAPI messages with subtype ``type``. The + protocol, instance, and session_id identify a single target zapi + client. Sharpd will send a stream of messages if the count is + greater than one. + +.. clicmd:: sharp send opaque <reg | unreg> PROTOCOL [{instance (0-1000) | session (1-1000)}] type (1-1000) + + Send opaque ZAPI registration and unregistration messages for a + single subtype. The messages must specify a protocol daemon by + name, and can include optional zapi ``instance`` and ``session`` + values. + +.. clicmd:: sharp create session (1-1024) + + Create an additional zapi client session for testing, using the + specified session id. + +.. clicmd:: sharp remove session (1-1024) + + Remove a test zapi client session that was created with the + specified session id. + +.. clicmd:: sharp neigh discover [vrf NAME] <A.B.C.D|X:X::X:X> IFNAME + + Send an ARP/NDP request to trigger the addition of a neighbor in the ARP + table. + +.. clicmd:: sharp import-te + + Import Traffic Engineering Database produced by OSPF or IS-IS. + +.. clicmd:: show sharp ted [verbose|json] + +.. clicmd:: show sharp ted [<vertex [A.B.C.D]|edge [A.B.C.D]|subnet [A.B.C.D/M]>] [verbose|json] + + Show imported Traffic Engineering Data Base + +.. clicmd:: show sharp cspf source <A.B.C.D|X:X:X:X> destination <A.B.C.D|X:X:X:X> <metric|te-metric|delay> (0-16777215) [rsv-bw (0-7) BANDWIDTH] + + Show the result of a call to the Constraint Shortest Path First (CSPF) + algorithm that allows to compute a path between a source and a + destination under various constraints. Standard Metric, TE Metric, Delay + and Bandwidth are supported constraints. Prior to use this function, it is + necessary to import a Traffic Engineering Database with `sharp import-te` + command (see above). + +.. clicmd:: sharp install seg6-routes [vrf NAME] <A.B.C.D|X:X::X:X> nexthop-seg6 X:X::X:X encap X:X::X:X (1-1000000) + + This command installs a route for SRv6 Transit behavior (on Linux it is + known as seg6 route). The count, destination, vrf, etc. have the same + meaning as in the ``sharp install routes`` command. With this command, + sharpd will request zebra to configure seg6 route via ZEBRA_ROUTE_ADD + ZAPI. As in the following example. + +:: + + router# sharp install seg6-routes 1::A nexthop-seg6 2001::2 encap A:: 1 + router# sharp install seg6-routes 1::B nexthop-seg6 2001::2 encap B:: 1 + + router# show ipv6 route + D>* 1::A/128 [150/0] via 2001::2, dum0, seg6 a::, weight 1, 00:00:01 + D>* 1::B/128 [150/0] via 2001::2, dum0, seg6 b::, weight 1, 00:00:01 + + bash# ip -6 route list + 1::A encap seg6 mode encap segs 1 [ a:: ] via 2001::2 dev dum0 proto 194 metric 20 pref medium + 1::B encap seg6 mode encap segs 1 [ b:: ] via 2001::2 dev dum0 proto 194 metric 20 pref medium + +.. clicmd:: sharp install seg6local-routes [vrf NAME] X:X::X:X nexthop-seg6local NAME ACTION ARGS.. (1-1000000) + + This command installs a route for SRv6 Endpoint behavior (on Linux it is + known as seg6local route). The count, destination, vrf, etc. have the same + meaning as in the ``sharp install routes`` command. With this command, + sharpd will request zebra to configure seg6local route via ZEBRA_ROUTE_ADD + ZAPI. As in the following example. + + There are many End Functions defined in SRv6, which have been standardized + in RFC 8986. The current implementation supports End, End.X, End.T, End.DX4, + End.DT6 and End.DT46, which can be configured as follows. + +:: + + router# sharp install seg6local-routes 1::1 nexthop-seg6local dum0 End 1 + router# sharp install seg6local-routes 1::2 nexthop-seg6local dum0 End_X 2001::1 1 + router# sharp install seg6local-routes 1::3 nexthop-seg6local dum0 End_T 10 1 + router# sharp install seg6local-routes 1::4 nexthop-seg6local dum0 End_DX4 10.0.0.1 1 + router# sharp install seg6local-routes 1::5 nexthop-seg6local dum0 End_DT6 10 1 + router# sharp install seg6local-routes 1::6 nexthop-seg6local dum0 End_DT46 10 1 + + router# show ipv6 route + D>* 1::1/128 [150/0] is directly connected, dum0, seg6local End USP, weight 1, 00:00:05 + D>* 1::2/128 [150/0] is directly connected, dum0, seg6local End.X nh6 2001::1, weight 1, 00:00:05 + D>* 1::3/128 [150/0] is directly connected, dum0, seg6local End.T table 10, weight 1, 00:00:05 + D>* 1::4/128 [150/0] is directly connected, dum0, seg6local End.DX4 nh4 10.0.0.1, weight 1, 00:00:05 + D>* 1::5/128 [150/0] is directly connected, dum0, seg6local End.DT6 table 10, weight 1, 00:00:05 + D>* 1::6/128 [150/0] is directly connected, dum0, seg6local End.DT46 table 10, weight 1, 00:00:05 + + bash# ip -6 route + 1::1 encap seg6local action End dev dum0 proto 194 metric 20 pref medium + 1::2 encap seg6local action End.X nh6 2001::1 dev dum0 proto 194 metric 20 pref medium + 1::3 encap seg6local action End.T table 10 dev dum0 proto 194 metric 20 pref medium + 1::4 encap seg6local action End.DX4 nh4 10.0.0.1 dev dum0 proto 194 metric 20 pref medium + 1::5 encap seg6local action End.DT6 table 10 dev dum0 proto 194 metric 20 pref medium + 1::6 encap seg6local action End.DT46 table 10 dev dum0 proto 194 metric 20 pref medium + +.. clicmd:: show sharp segment-routing srv6 + + This command shows us what SRv6 locator chunk, sharp is holding as zclient. + An SRv6 locator is defined for each SRv6 router, and a single locator may + be shared by multiple protocols. + + In the FRRouting implementation, the Locator chunk get request is executed + by a routing protocol daemon such as sharpd or bgpd, And then Zebra + allocates a Locator Chunk, which is a subset of the Locator Prefix, and + notifies the requesting protocol daemon of this information. + + This command example shows how the locator chunk of sharpd itself is + allocated. + +:: + + router# show segment-routing srv6 locator + Locator: + Name ID 2 2001:db8:2:2::/64 Up + + router# show sharp segment-routing srv6 + Locator loc1 has 1 prefix chunks + 2001:db8:1:1::/64 + +.. clicmd:: sharp srv6-manager get-locator-chunk + + This command requests the SRv6 locator to allocate a locator chunk via ZAPI. + This chunk can be owned by the protocol daemon, and the chunk obtained by + sharpd will not be used by the SRv6 mechanism of another routing protocol. + + Since this request is made asynchronously, it can be issued before the SRv6 + locator is configured on the zebra side, and as soon as it is ready on the + zebra side, sharpd can check the allocated locator chunk via zapi. + +:: + + router# show segment-routing srv6 locator loc1 detail + Name: loc1 + Prefix: 2001:db8:1:1::/64 + Chunks: + - prefix: 2001:db8:1:1::/64, owner: system + + router# show sharp segment-routing srv6 + (nothing) + + router# sharp srv6-manager get-locator-chunk loc1 + + router# show segment-routing srv6 locator loc1 detail + Name: loc1 + Prefix: 2001:db8:1:1::/64 + Chunks: + - prefix: 2001:db8:1:1::/64, owner: sharp + + router# show sharp segment-routing srv6 + Locator loc1 has 1 prefix chunks + 2001:db8:1:1::/64 + +.. clicmd:: sharp srv6-manager release-locator-chunk + + This command releases a locator chunk that has already been allocated by + ZAPI. The freed chunk will have its owner returned to the system and will + be available to another protocol daemon. + +:: + + router# show segment-routing srv6 locator loc1 detail + Name: loc1 + Prefix: 2001:db8:1:1::/64 + Chunks: + - prefix: 2001:db8:1:1::/64, owner: sharp + + router# show sharp segment-routing srv6 + Locator loc1 has 1 prefix chunks + 2001:db8:1:1::/64 + + router# sharp srv6-manager release-locator-chunk loc1 + + router# show segment-routing srv6 locator loc1 detail + Name: loc1 + Prefix: 2001:db8:1:1::/64 + Chunks: + - prefix: 2001:db8:1:1::/64, owner: system + + router# show sharp segment-routing srv6 + (nothing) + +.. clicmd:: sharp interface IFNAME protodown + + Set an interface protodown. diff --git a/doc/user/snmp.rst b/doc/user/snmp.rst new file mode 100644 index 0000000..3c2d11a --- /dev/null +++ b/doc/user/snmp.rst @@ -0,0 +1,264 @@ +.. _snmp-support: + +************ +SNMP Support +************ + +:abbr:`SNMP (Simple Network Managing Protocol)` is a widely implemented feature +for collecting network information from router and/or host. FRR itself does +not support SNMP agent (server daemon) functionality but is able to connect to +a SNMP agent using the the AgentX protocol (:rfc:`2741`) and make the +routing protocol MIBs available through it. + +Note that SNMP Support needs to be enabled at compile-time and loaded as module +on daemon startup. Refer to :ref:`loadable-module-support` on the latter. If +you do not start the daemons with snmp module support snmp will not work +properly. + +.. _getting-and-installing-an-snmp-agent: + +Getting and installing an SNMP agent +==================================== + +The supported SNMP agent is AgentX. We recommend to use +the latest version of `net-snmp` which was formerly known as `ucd-snmp`. It is +free and open software and available at `http://www.net-snmp.org/ <http://www.net-snmp.org/>`_ +and as binary package for most Linux distributions. + +.. _net-smtp-configuration: + +NET-SNMP configuration +====================== + +Routers with a heavy amount of routes (e.g. BGP full table) might experience +problems with a hanging vtysh from time to time, 100% CPU on the snmpd or +even crashes of the frr daemon(s) due to stalls within AgentX. Once snmp +agents connects they start receiving a heavy amount of SNMP data (all the +routes) which cannot be handled quick enough. It's recommended (by several +vendors as well) to exclude these OID's unless you really need them, which +can be achieved by amending the default view from SNMP + +:file:`/etc/snmp/snmpd.conf`: + +:: + + # This is the default view + view all included .1 80 + # Remove ipRouteTable from view + view all excluded .1.3.6.1.2.1.4.21 + # Remove ipNetToMediaTable from view + view all excluded .1.3.6.1.2.1.4.22 + # Remove ipNetToPhysicalPhysAddress from view + view all excluded .1.3.6.1.2.1.4.35 + # Remove ipCidrRouteTable from view + view all excluded .1.3.6.1.2.1.4.24 + # Optionally protect SNMP private/secret values + view all excluded .1.3.6.1.6.3.15 + view all excluded .1.3.6.1.6.3.16 + view all excluded .1.3.6.1.6.3.18 + # Optionally allow SNMP public info (sysName, location, etc) + view system included .iso.org.dod.internet.mgmt.mib-2.system + + +.. _agentx-configuration: + +AgentX configuration +==================== + +.. program:: configure + +To enable AgentX protocol support, FRR must have been build with the +:option:`--enable-snmp` or `--enable-snmp=agentx` option. Both the +master SNMP agent (snmpd) and each of the FRR daemons must be configured. In +:file:`/etc/snmp/snmpd.conf`, the ``master agentx`` directive should be added. +In each of the FRR daemons, ``agentx`` command will enable AgentX support. + +:file:`/etc/snmp/zebra.conf`: + +:: + + # + # example access restrictions setup + # + com2sec readonly default public + group MyROGroup v1 readonly + view all included .1 80 + access MyROGroup "" any noauth exact all none none + # + # enable master agent for AgentX subagents + # + master agentx + +:file:`/etc/frr/ospfd.conf:` + + .. code-block:: frr + + ! ... the rest of ospfd.conf has been omitted for clarity ... + ! + agentx + ! + + +Upon successful connection, you should get something like this in the log of +each FRR daemons: + +:: + + 2012/05/25 11:39:08 ZEBRA: snmp[info]: NET-SNMP version 5.4.3 AgentX subagent connected + + +Then, you can use the following command to check everything works as expected: + +:: + + # snmpwalk -c public -v1 localhost .1.3.6.1.2.1.14.1.1 + OSPF-MIB::ospfRouterId.0 = IpAddress: 192.168.42.109 + [...] + +An example below is how to query SNMP for BGP: + + .. code-block:: shell + + $ # BGP4-MIB (https://www.circitor.fr/Mibs/Mib/B/BGP4-MIB.mib) + $ snmpwalk -c public -v2c -On -Ln localhost .1.3.6.1.2.1.15 + + $ # BGP4V2-MIB (http://www.circitor.fr/Mibs/Mib/B/BGP4V2-MIB.mib) + $ # Information about the peers (bgp4V2PeerTable): + $ snmpwalk -c public -v2c -On -Ln localhost .1.3.6.1.3.5.1.1.2 + ... + .1.3.6.1.3.5.1.1.2.1.1.1.1.192.168.10.124 = Gauge32: 0 + .1.3.6.1.3.5.1.1.2.1.1.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Gauge32: 0 + .1.3.6.1.3.5.1.1.2.1.2.1.1.192.168.10.124 = INTEGER: 1 + .1.3.6.1.3.5.1.1.2.1.2.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = INTEGER: 2 + .1.3.6.1.3.5.1.1.2.1.3.1.1.192.168.10.124 = Hex-STRING: C0 A8 0A 11 + .1.3.6.1.3.5.1.1.2.1.3.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Hex-STRING: 2A 02 47 80 0A BC 00 00 00 00 00 00 00 00 00 01 + .1.3.6.1.3.5.1.1.2.1.4.1.1.192.168.10.124 = INTEGER: 1 + .1.3.6.1.3.5.1.1.2.1.4.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = INTEGER: 2 + .1.3.6.1.3.5.1.1.2.1.5.1.1.192.168.10.124 = Hex-STRING: C0 A8 0A 7C + .1.3.6.1.3.5.1.1.2.1.5.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Hex-STRING: 2A 02 47 80 0A BC 00 00 00 00 00 00 00 00 00 02 + .1.3.6.1.3.5.1.1.2.1.6.1.1.192.168.10.124 = Gauge32: 179 + .1.3.6.1.3.5.1.1.2.1.6.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Gauge32: 179 + .1.3.6.1.3.5.1.1.2.1.7.1.1.192.168.10.124 = Gauge32: 65002 + .1.3.6.1.3.5.1.1.2.1.7.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Gauge32: 65002 + .1.3.6.1.3.5.1.1.2.1.8.1.1.192.168.10.124 = Hex-STRING: C0 A8 0A 11 + .1.3.6.1.3.5.1.1.2.1.8.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Hex-STRING: C0 A8 0A 11 + .1.3.6.1.3.5.1.1.2.1.9.1.1.192.168.10.124 = Gauge32: 41894 + .1.3.6.1.3.5.1.1.2.1.9.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Gauge32: 39960 + .1.3.6.1.3.5.1.1.2.1.10.1.1.192.168.10.124 = Gauge32: 65001 + .1.3.6.1.3.5.1.1.2.1.10.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Gauge32: 65001 + .1.3.6.1.3.5.1.1.2.1.11.1.1.192.168.10.124 = Hex-STRING: C8 C8 C8 CA + .1.3.6.1.3.5.1.1.2.1.11.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = Hex-STRING: C8 C8 C8 CA + .1.3.6.1.3.5.1.1.2.1.12.1.1.192.168.10.124 = INTEGER: 2 + .1.3.6.1.3.5.1.1.2.1.12.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = INTEGER: 2 + .1.3.6.1.3.5.1.1.2.1.13.1.1.192.168.10.124 = INTEGER: 6 + .1.3.6.1.3.5.1.1.2.1.13.1.2.42.2.71.128.10.188.0.0.0.0.0.0.0.0.0.2 = INTEGER: 6 + + $ # Information about the BGP table (bgp4V2NlriTable): + $ snmpwalk -c public -v2c -On -Ln localhost .1.3.6.1.3.5.1.1.9 + ... + .1.3.6.1.3.5.1.1.9.1.1.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.1.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.1.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.1.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.2.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.2.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.2.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 2 + .1.3.6.1.3.5.1.1.9.1.2.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 2 + .1.3.6.1.3.5.1.1.9.1.3.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.3.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.3.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.3.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.4.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.4.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.4.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 2 + .1.3.6.1.3.5.1.1.9.1.4.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 2 + .1.3.6.1.3.5.1.1.9.1.5.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Hex-STRING: 0A 00 00 00 + .1.3.6.1.3.5.1.1.9.1.5.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Hex-STRING: 0A 00 00 02 + .1.3.6.1.3.5.1.1.9.1.5.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 20 01 0D B8 00 00 00 00 00 00 00 00 00 00 00 01 + .1.3.6.1.3.5.1.1.9.1.5.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 20 01 0D B8 00 01 00 00 00 00 00 00 00 00 00 00 + .1.3.6.1.3.5.1.1.9.1.6.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 31 + .1.3.6.1.3.5.1.1.9.1.6.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 32 + .1.3.6.1.3.5.1.1.9.1.6.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 128 + .1.3.6.1.3.5.1.1.9.1.6.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 56 + .1.3.6.1.3.5.1.1.9.1.7.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.7.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.7.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.7.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.8.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.8.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.8.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.8.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.9.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.9.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 3 + .1.3.6.1.3.5.1.1.9.1.9.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.9.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 3 + .1.3.6.1.3.5.1.1.9.1.10.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.10.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.10.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 4 + .1.3.6.1.3.5.1.1.9.1.10.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 4 + .1.3.6.1.3.5.1.1.9.1.11.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Hex-STRING: C0 A8 0C 01 + .1.3.6.1.3.5.1.1.9.1.11.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Hex-STRING: C0 A8 0C 01 + .1.3.6.1.3.5.1.1.9.1.11.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: FE 80 00 00 00 00 00 00 30 39 84 FF FE 9A 24 2B + .1.3.6.1.3.5.1.1.9.1.11.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: FE 80 00 00 00 00 00 00 30 39 84 FF FE 9A 24 2B + .1.3.6.1.3.5.1.1.9.1.14.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 0 + .1.3.6.1.3.5.1.1.9.1.14.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 0 + .1.3.6.1.3.5.1.1.9.1.14.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + .1.3.6.1.3.5.1.1.9.1.14.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + .1.3.6.1.3.5.1.1.9.1.15.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.15.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.15.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.15.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + .1.3.6.1.3.5.1.1.9.1.16.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.16.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.16.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + .1.3.6.1.3.5.1.1.9.1.16.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 1 + 1.3.6.1.3.5.1.1.9.1.17.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 1 + 1.3.6.1.3.5.1.1.9.1.17.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 2 + 1.3.6.1.3.5.1.1.9.1.17.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 1 + 1.3.6.1.3.5.1.1.9.1.17.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 2 + 1.3.6.1.3.5.1.1.9.1.18.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.18.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.18.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.18.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.19.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.19.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.19.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.19.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = INTEGER: 0 + 1.3.6.1.3.5.1.1.9.1.20.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 0 + 1.3.6.1.3.5.1.1.9.1.20.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 0 + 1.3.6.1.3.5.1.1.9.1.20.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + 1.3.6.1.3.5.1.1.9.1.20.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 0 + 1.3.6.1.3.5.1.1.9.1.21.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Hex-STRING: 00 00 00 00 + 1.3.6.1.3.5.1.1.9.1.21.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Hex-STRING: 00 00 00 00 + 1.3.6.1.3.5.1.1.9.1.21.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 00 00 00 00 + 1.3.6.1.3.5.1.1.9.1.21.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 00 00 00 00 + 1.3.6.1.3.5.1.1.9.1.22.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Gauge32: 1 + 1.3.6.1.3.5.1.1.9.1.22.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Gauge32: 1 + .1.3.6.1.3.5.1.1.9.1.22.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 1 + .1.3.6.1.3.5.1.1.9.1.22.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Gauge32: 1 + .1.3.6.1.3.5.1.1.9.1.24.1.1.1.1.10.0.0.0.31.1.192.168.12.1.1 = Hex-STRING: 02 01 FD E9 + .1.3.6.1.3.5.1.1.9.1.24.1.1.1.1.10.0.0.2.32.1.192.168.12.1.1 = Hex-STRING: 02 01 FD E9 + .1.3.6.1.3.5.1.1.9.1.24.1.2.1.2.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.1.128.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 02 01 FD E9 + .1.3.6.1.3.5.1.1.9.1.24.1.2.1.2.32.1.13.184.0.1.0.0.0.0.0.0.0.0.0.0.56.2.32.1.13.184.0.0.0.0.0.0.0.0.0.18.0.1.1 = Hex-STRING: 02 01 FD E9 + + +The AgentX protocol can be transported over a Unix socket or using TCP or UDP. +It usually defaults to a Unix socket and depends on how NetSNMP was built. If +need to configure FRR to use another transport, you can configure it through +:file:`/etc/snmp/frr.conf`: + +:: + + [snmpd] + # Use a remote master agent + agentXSocket tcp:192.168.15.12:705 + + +Here is the syntax for using AgentX: + +.. clicmd:: agentx + + Once enabled, it can't be unconfigured. Only removing from the daemons file + the keyword ``agentx`` takes an effect. + +.. include:: snmptrap.rst diff --git a/doc/user/snmptrap.rst b/doc/user/snmptrap.rst new file mode 100644 index 0000000..7e306b7 --- /dev/null +++ b/doc/user/snmptrap.rst @@ -0,0 +1,201 @@ +Handling SNMP Traps +=================== + +To handle snmp traps make sure your snmp setup of frr works correctly as +described in the frr documentation in :ref:`snmp-support`. + +The BGP4 mib will send traps on peer up/down events. These should be visible in +your snmp logs with a message similar to: + +:: + + snmpd[13733]: Got trap from peer on fd 14 + +To react on these traps they should be handled by a trapsink. Configure your +trapsink by adding the following lines to :file:`/etc/snmpd/snmpd.conf`: + +:: + + # send traps to the snmptrapd on localhost + trapsink localhost + + +This will send all traps to an snmptrapd running on localhost. You can of +course also use a dedicated management station to catch traps. Configure the +snmptrapd daemon by adding the following line to +:file:`/etc/snmpd/snmptrapd.conf`: + +:: + + traphandle .1.3.6.1.4.1.3317.1.2.2 /etc/snmp/snmptrap_handle.sh + + +This will use the bash script :file:`/etc/snmp/snmptrap_handle.sh` to handle +the BGP4 traps. To add traps for other protocol daemons, lookup their +appropriate OID from their mib. (For additional information about which traps +are supported by your mib, lookup the mib on +`http://www.oidview.com/mibs/detail.html <http://www.oidview.com/mibs/detail.html>`_). + +Make sure *snmptrapd* is started. + +The snmptrap_handle.sh script I personally use for handling BGP4 traps is +below. You can of course do all sorts of things when handling traps, like sound +a siren, have your display flash, etc., be creative ;). + +.. code-block:: shell + + #!/bin/bash + + # routers name + ROUTER=`hostname -s` + + #email address use to sent out notification + EMAILADDR="john@doe.com" + #email address used (allongside above) where warnings should be sent + EMAILADDR_WARN="sms-john@doe.com" + + # type of notification + TYPE="Notice" + + # local snmp community for getting AS belonging to peer + COMMUNITY="<community>" + + # if a peer address is in $WARN_PEERS a warning should be sent + WARN_PEERS="192.0.2.1" + + # get stdin + INPUT=`cat -` + + # get some vars from stdin + uptime=`echo $INPUT | cut -d' ' -f5` + peer=`echo $INPUT | cut -d' ' -f8 | sed -e 's/SNMPv2-SMI::mib-2.15.3.1.14.//g'` + peerstate=`echo $INPUT | cut -d' ' -f13` + errorcode=`echo $INPUT | cut -d' ' -f9 | sed -e 's/\\"//g'` + suberrorcode=`echo $INPUT | cut -d' ' -f10 | sed -e 's/\\"//g'` + remoteas=`snmpget -v2c -c $COMMUNITY localhost SNMPv2-SMI::mib-2.15.3.1.9.$peer | cut -d' ' -f4` + + WHOISINFO=`whois -h whois.ripe.net " -r AS$remoteas" | egrep '(as-name|descr)'` + asname=`echo "$WHOISINFO" | grep "^as-name:" | sed -e 's/^as-name://g' -e 's/ //g' -e 's/^ //g' | uniq` + asdescr=`echo "$WHOISINFO" | grep "^descr:" | sed -e 's/^descr://g' -e 's/ //g' -e 's/^ //g' | uniq` + + # if peer address is in $WARN_PEER, the email should also + # be sent to $EMAILADDR_WARN + for ip in $WARN_PEERS; do + if [ "x$ip" == "x$peer" ]; then + EMAILADDR="$EMAILADDR,$EMAILADDR_WARN" + TYPE="WARNING" + break + fi + done + + # convert peer state + case "$peerstate" in + 1) peerstate="Idle" ;; + 2) peerstate="Connect" ;; + 3) peerstate="Active" ;; + 4) peerstate="Opensent" ;; + 5) peerstate="Openconfirm" ;; + 6) peerstate="Established" ;; + *) peerstate="Unknown" ;; + esac + + # get textual messages for errors + case "$errorcode" in + 00) + error="No error" + suberror="" + ;; + 01) + error="Message Header Error" + case "$suberrorcode" in + 01) suberror="Connection Not Synchronized" ;; + 02) suberror="Bad Message Length" ;; + 03) suberror="Bad Message Type" ;; + *) suberror="Unknown" ;; + esac + ;; + 02) + error="OPEN Message Error" + case "$suberrorcode" in + 01) suberror="Unsupported Version Number" ;; + 02) suberror="Bad Peer AS" ;; + 03) suberror="Bad BGP Identifier" ;; + 04) suberror="Unsupported Optional Parameter" ;; + 05) suberror="Authentication Failure" ;; + 06) suberror="Unacceptable Hold Time" ;; + *) suberror="Unknown" ;; + esac + ;; + 03) + error="UPDATE Message Error" + case "$suberrorcode" in + 01) suberror="Malformed Attribute List" ;; + 02) suberror="Unrecognized Well-known Attribute" ;; + 03) suberror="Missing Well-known Attribute" ;; + 04) suberror="Attribute Flags Error" ;; + 05) suberror="Attribute Length Error" ;; + 06) suberror="Invalid ORIGIN Attribute" ;; + 07) suberror="AS Routing Loop" ;; + 08) suberror="Invalid NEXT_HOP Attribute" ;; + 09) suberror="Optional Attribute Error" ;; + 10) suberror="Invalid Network Field" ;; + 11) suberror="Malformed AS_PATH" ;; + *) suberror="Unknown" ;; + esac + ;; + 04) + error="Hold Timer Expired" + suberror="" + ;; + 05) + error="Finite State Machine Error" + suberror="" + ;; + 06) + error="Cease" + case "$suberrorcode" in + 01) suberror="Maximum Number of Prefixes Reached" ;; + 02) suberror="Administrative Shutdown" ;; + 03) suberror="Peer De-configured" ;; + 04) suberror="Administrative Reset" ;; + 05) suberror="Connection Rejected" ;; + 06) suberror="Other Configuration Change" ;; + 07) suberror="Connection Collision Resolution" ;; + 08) suberror="Out of Resources" ;; + 09) suberror="MAX" ;; + *) suberror="Unknown" ;; + esac + ;; + *) + error="Unknown" + suberror="" + ;; + esac + + # create textual message from errorcodes + if [ "x$suberror" == "x" ]; then + NOTIFY="$errorcode ($error)" + else + NOTIFY="$errorcode/$suberrorcode ($error/$suberror)" + fi + + # form a decent subject + SUBJECT="$TYPE: $ROUTER [bgp] $peer is $peerstate: $NOTIFY" + # create the email body + MAIL=`cat << EOF + BGP notification on router $ROUTER. + + Peer: $peer + AS: $remoteas + New state: $peerstate + Notification: $NOTIFY + + Info: + $asname + $asdescr + + Snmpd uptime: $uptime + EOF` + + # mail the notification + echo "$MAIL" | mail -s "$SUBJECT" $EMAILADDR diff --git a/doc/user/static.rst b/doc/user/static.rst new file mode 100644 index 0000000..d405276 --- /dev/null +++ b/doc/user/static.rst @@ -0,0 +1,186 @@ +.. _static: + +****** +STATIC +****** + +:abbr:`STATIC` is a daemon that handles the installation and deletion +of static routes. + +.. _starting-static: + +Starting STATIC +=============== + +Default configuration file for *staticd* is :file:`staticd.conf`. The typical +location of :file:`staticd.conf` is |INSTALL_PREFIX_ETC|/staticd.conf. + +If the user is using integrated config, then :file:`staticd.conf` need not be +present and the :file:`frr.conf` is read instead. + +If the user has not fully upgraded to using the staticd.conf and still has +a non-integrated config with zebra.conf holding the static routes, *staticd* +will read in the :file:`zebrad.conf` as a backup. + +.. program:: staticd + +:abbr:`STATIC` supports all the common FRR daemon start options which are +documented elsewhere. + +.. _static-route-commands: + +Static Route Commands +===================== + +Static routing is a very fundamental feature of routing technology. It defines +a static prefix and gateway, with several possible forms. + +.. clicmd:: ip route NETWORK GATEWAY [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ip route NETWORK IFNAME [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ip route NETWORK GATEWAY IFNAME [DISTANCE] [onlink] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ip route NETWORK (Null0|blackhole|reject) [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ipv6 route NETWORK [from SRCPREFIX] GATEWAY [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ipv6 route NETWORK [from SRCPREFIX] IFNAME [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ipv6 route NETWORK [from SRCPREFIX] GATEWAY IFNAME [DISTANCE] [onlink] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + +.. clicmd:: ipv6 route NETWORK [from SRCPREFIX] (Null0|blackhole|reject) [DISTANCE] [table TABLENO] [nexthop-vrf VRFNAME] [vrf VRFNAME] + + NETWORK is destination prefix with a valid v4 or v6 network based upon + initial form of the command. + + GATEWAY is the IP address to use as next-hop for the prefix. Currently, it must match + the v4 or v6 route type specified at the start of the command. + + IFNAME is the name of the interface to use as next-hop. If only IFNAME is specified + (without GATEWAY), a connected route will be created. + + When both IFNAME and GATEWAY are specified together, it binds the route to the specified + interface. In this case, it is also possible to specify ``onlink`` to force the kernel + to consider the next-hop as "on link" on the given interface. + + Alternatively, the gateway can be specified as ``Null0`` or ``blackhole`` to create a blackhole + route that drops all traffic. It can also be specified as ``reject`` to create an unreachable + route that rejects traffic with ICMP "Destination Unreachable" messages. + + TABLENO is an optional parameter for namespaces that allows you to create the + route in a specified table associated with the vrf namespace. ``table`` will + be rejected if you are not using namespace based vrfs. + + ``vrf`` VRFNAME allows you to create the route in a specified vrf. + + ``nexthop-vrf`` VRFNAME allows you to create a leaked route with a nexthop in the + specified VRFNAME. ``nexthop-vrf`` cannot be currently used with namespace based vrfs. + + The IPv6 variant allows the installation of a static source-specific route + with the SRCPREFIX sub command. These routes are currently supported + on Linux operating systems only, and perform AND matching on packet's + destination and source addresses in the kernel's forwarding path. Note + that destination longest-prefix match is "more important" than source + LPM, e.g. ``2001:db8:1::/64 from 2001:db8::/48`` will win over + ``2001:db8::/48 from 2001:db8:1::/64`` if both match. + +.. _multiple-route-command: + +Multiple nexthop static route +============================= + +To create multiple nexthops to the same NETWORK (also known as a multipath route), just reenter the same +network statement with different nexthop information. + +.. code-block:: frr + + ip route 10.0.0.1/32 10.0.0.2 + ip route 10.0.0.1/32 10.0.0.3 + ip route 10.0.0.1/32 eth0 + + +If there is no route to 10.0.0.2 and 10.0.0.3, and interface eth0 +is reachable, then the last route is installed into the kernel. + +If zebra has been compiled with multipath support, and both 10.0.0.2 and +10.0.0.3 are reachable, zebra will install a multipath route via both +nexthops, if the platform supports this. + +:: + + router> show ip route + S> 10.0.0.1/32 [1/0] via 10.0.0.2 inactive + via 10.0.0.3 inactive + * is directly connected, eth0 + + +.. code-block:: frr + + ip route 10.0.0.0/8 10.0.0.2 + ip route 10.0.0.0/8 10.0.0.3 + ip route 10.0.0.0/8 null0 255 + + +This will install a multipath route via the specified next-hops if they are +reachable, as well as a high-distance blackhole route, which can be useful to +prevent traffic destined for a prefix to match less-specific routes (e.g. +default) should the specified gateways not be reachable. E.g.: + +:: + + router> show ip route 10.0.0.0/8 + Routing entry for 10.0.0.0/8 + Known via "static", distance 1, metric 0 + 10.0.0.2 inactive + 10.0.0.3 inactive + + Routing entry for 10.0.0.0/8 + Known via "static", distance 255, metric 0 + directly connected, Null0 + +Also, if the user wants to configure a static route for a specific VRF, then +a specific VRF configuration mode is available. After entering into that mode +with :clicmd:`vrf VRF` the user can enter the same route command as before, +but this time, the route command will apply to the VRF. + +.. code-block:: frr + + # case with VRF + configure + vrf r1-cust1 + ip route 10.0.0.0/24 10.0.0.2 + exit-vrf + + +SR-TE Route Commands +==================== + +It is possible to specify a route using a SR-TE policy configured in Zebra. + +e.g. to use the SR-TE policy with endpoint 6.6.6.6 and color 123 to reach the +network 9.9.9.9/24: + +.. code-block:: frr + + ip route 9.9.9.9/24 6.6.6.6 color 123 + +SRv6 Route Commands +==================== + +It is possible to specify a static route for ipv6 prefixes using an SRv6 +`segments` instruction. The `/` separator can be used to specify +multiple segments instructions. + +.. code-block:: frr + + ipv6 route X:X::X:X <X:X::X:X|nexthop> segments U:U::U:U/Y:Y::Y:Y/Z:Z::Z:Z + + +:: + + router(config)# ipv6 route 2005::1/64 ens3 segments 2001:db8:aaaa::7/2002::4/2002::3/2002::2 + + router# show ipv6 route + [..] + S>* 2005::/64 [1/0] is directly connected, ens3, seg6 2001:db8:aaaa::7,2002::4,2002::3,2002::2, weight 1, 00:00:06 diff --git a/doc/user/subdir.am b/doc/user/subdir.am new file mode 100644 index 0000000..4879f7f --- /dev/null +++ b/doc/user/subdir.am @@ -0,0 +1,133 @@ +# +# doc/user +# + +user_RSTFILES = \ + doc/user/affinitymap.rst \ + doc/user/babeld.rst \ + doc/user/ldpd.rst \ + doc/user/basic.rst \ + doc/user/bgp.rst \ + doc/user/bmp.rst \ + doc/user/bugs.rst \ + doc/user/conf.py \ + doc/user/eigrpd.rst \ + doc/user/evpn.rst \ + doc/user/extlog.rst \ + doc/user/fabricd.rst \ + doc/user/filter.rst \ + doc/user/frr-reload.rst \ + doc/user/glossary.rst \ + doc/user/grpc.rst \ + doc/user/index.rst \ + doc/user/installation.rst \ + doc/user/ipv6.rst \ + doc/user/isisd.rst \ + doc/user/kernel.rst \ + doc/user/nexthop_groups.rst \ + doc/user/nhrpd.rst \ + doc/user/ospf6d.rst \ + doc/user/ospfd.rst \ + doc/user/ospf_fundamentals.rst \ + doc/user/overview.rst \ + doc/user/packet-dumps.rst \ + doc/user/pathd.rst \ + doc/user/pim.rst \ + doc/user/pimv6.rst \ + doc/user/ripd.rst \ + doc/user/pbr.rst \ + doc/user/ripngd.rst \ + doc/user/routemap.rst \ + doc/user/routeserver.rst \ + doc/user/rpki.rst \ + doc/user/scripting.rst \ + doc/user/setup.rst \ + doc/user/sharp.rst \ + doc/user/snmp.rst \ + doc/user/snmptrap.rst \ + doc/user/static.rst \ + doc/user/vnc.rst \ + doc/user/vrrp.rst \ + doc/user/vtysh.rst \ + doc/user/zebra.rst \ + doc/user/bfd.rst \ + doc/user/flowspec.rst \ + doc/user/watchfrr.rst \ + doc/user/wecmp_linkbw.rst \ + doc/user/mgmtd.rst \ + # end + +EXTRA_DIST += \ + $(user_RSTFILES) \ + doc/user/Useful_Sysctl_Settings.md \ + doc/user/_static/overrides.css \ + doc/user/_static/overrides.js \ + # end + +USERBUILD = doc/user/_build +$(USERBUILD)/.doctrees/environment.pickle: $(user_RSTFILES) + +# +# automake integration (things that should be built in "all") +# + +if DOC +nodist_noinst_DATA += $(USERBUILD)/texinfo/frr.info +endif +if DOC_HTML +nodist_noinst_DATA += $(USERBUILD)/html/.buildinfo +endif + +# +# standard targets +# + +.PHONY: info html pdf +info: $(USERBUILD)/texinfo/frr.info +html: $(USERBUILD)/html/.buildinfo +pdf: $(USERBUILD)/latexpdf + +# +# hook-ins for clean / install / doc +# + +.PHONY: clean-userdocs +clean-local: clean-userdocs +clean-userdocs: + -rm -rf "$(USERBUILD)" + +# INSTALL_INFO=install-info +.PHONY: install-info uninstall-info install-html uninstall-html + +install-info: $(USERBUILD)/texinfo/frr.info + $(MKDIR_P) "$(DESTDIR)$(infodir)" + $(INSTALL_DATA) "$<" "$(DESTDIR)$(infodir)" + [ -z "${DESTDIR}" ] && $(INSTALL_INFO) --info-dir="$(DESTDIR)$(infodir)" "$<" || true +uninstall-info: $(USERBUILD)/texinfo/frr.info + -rm -f "$(DESTDIR)$(infodir)/$<" + [ -z "${DESTDIR}" ] && $(INSTALL_INFO) --delete --info-dir="$(DESTDIR)$(infodir)" "$<" || true + +install-html: $(USERBUILD)/html/.buildinfo + $(MKDIR_P) "$(DESTDIR)$(htmldir)" + cp -r "$(USERBUILD)/html" "$(DESTDIR)$(htmldir)" +uninstall-html: + -rm -rf "$(DESTDIR)$(htmldir)/html" + +.PHONY: install-data-local uninstall-local +if DOC +DOC_INFO=info +TARGET_INSTALL_INFO=install-info +TARGET_UNINSTALL_INFO=uninstall-info +endif +if DOC_HTML +DOC_HTML=html +TARGET_INSTALL_HTML=install-html +TARGET_UNINSTALL_HTML=uninstall-html +endif + +# leave the comments in, this was causing weird reordering issues in automake +install-data-local: $(TARGET_INSTALL_INFO) $(TARGET_INSTALL_HTML) +# +uninstall-local: $(TARGET_UNINSTALL_INFO) $(TARGET_UNINSTALL_HTML) +# +doc: $(DOC_INFO) $(DOC_HTML) diff --git a/doc/user/vnc.rst b/doc/user/vnc.rst new file mode 100644 index 0000000..4ff27c6 --- /dev/null +++ b/doc/user/vnc.rst @@ -0,0 +1,1393 @@ +.. _vnc-and-vnc-gw: + +************** +VNC and VNC-GW +************** + +This chapter describes how to use :abbr:`VNC (Virtual Network Control)` +services, including :abbr:`NVA (Network Virtualization Authority)` and +:abbr:`VNC-GW (VNC Gateway)` functions. Background information on NVAs, +:abbr:`NVE (Network Virtualization Edge)` s, :abbr:`UN (Underlay Network)` s, +and :abbr:`VN (Virtual Network)` is available from the +`IETF <https://datatracker.ietf.org/wg/nvo3>`_. :abbr:`VNC-GW (VNC Gateway)` s +support the import/export of routing information between VNC and :abbr:`CE +(customer edge)` routers operating within a VN. Both IP/Layer 3 (L3) VNs, and +IP with Ethernet/Layer 2 (L2) VNs are supported. + +BGP, with IP VPNs and Tunnel Encapsulation, is used to distribute VN +information between NVAs. BGP based IP VPN support is defined in :rfc:`4364`, +and :rfc:`4659`. Encapsulation information is provided via the Tunnel +Encapsulation Attribute, :rfc:`5512`. + +The protocol that is used to communicate routing and Ethernet / Layer 2 (L2) +forwarding information between NVAs and NVEs is referred to as the Remote +Forwarder Protocol (RFP). `OpenFlow` is an example RFP. Specific RFP +implementations may choose to implement either a `hard-state` or `soft-state` +prefix and address registration model. To support a `soft-state` refresh model, +a `lifetime` in seconds is associated with all registrations and responses. + +The chapter also provides sample configurations for basic example scenarios. + +.. _configuring-vnc: + +Configuring VNC +=============== + +Virtual Network Control (:abbr:`VNC`) service configuration commands appear in +the `router bgp` section of the BGPD configuration file +(:ref:`bgp-configuration-examples`). The commands are broken down into the +following areas: + +- :dfn:`General VNC` configuration applies to general VNC operation and is + primarily used to control the method used to advertise tunnel information. + +- :dfn:`Remote Forwarder Protocol (RFP)` configuration relates to the protocol + used between NVAs and NVEs. + +- :dfn:`VNC Defaults` provides default parameters for registered NVEs. + +- :dfn:`VNC NVE Group` provides for configuration of a specific set of + registered NVEs and overrides default parameters. + +- :dfn:`Redistribution` and :dfn:`Export` control VNC-GW operation, i.e., the + import/export of routing information between VNC and customer edge routers + (:abbr:`CE` s) operating within a VN. + + +.. _general-vnc-configuration: + +General VNC Configuration +------------------------- + +.. _rfp-related-configuration: + +RFP Related Configuration +------------------------- + +The protocol that is used to communicate routing and Ethernet / L2 forwarding +information between NVAs and NVEs is referred to as the Remote Forwarder +Protocol (RFP). Currently, only a simple example RFP is included in FRR. +Developers may use this example as a starting point to integrate FRR with an +RFP of their choosing, e.g., `OpenFlow`. The example code includes the +following sample configuration: + +.. clicmd:: rfp example-config-value VALUE + +This is a simple example configuration parameter included as part of the RFP +example code. VALUE must be in the range of 0 to 4294967295. + +.. _vnc-defaults-configuration: + +VNC Defaults Configuration +-------------------------- + +The VNC Defaults section allows the user to specify default values for +configuration parameters for all registered NVEs. +Default values are overridden by :ref:`vnc-nve-group-configuration`. + +.. clicmd:: vnc defaults + +Enter VNC configuration mode for specifying VNC default behaviors. Use +`exit-vnc` to leave VNC configuration mode. `vnc defaults` is optional. + +.. code-block:: frr + + vnc defaults + ... various VNC defaults + exit-vnc + + +These are the statements that can appear between ``vnc defaults`` and +``exit-vnc``. Documentation for these statements is given in +:ref:`vnc-nve-group-configuration`. + +- :clicmd:`rt import RT-LIST` +- :clicmd:`rt export RT-LIST` +- :clicmd:`rt both RT-LIST` +- :clicmd:`rd ROUTE-DISTINGUISHER` +- :clicmd:`l2rd NVE-ID-VALUE` +- :clicmd:`response-lifetime LIFETIME|infinite` +- :clicmd:`export bgp|zebra route-map MAP-NAME` +- :clicmd:`export bgp|zebra no route-map` + +.. clicmd:: exit-vnc + + Exit VNC configuration mode. + +.. _vnc-nve-group-configuration: + +VNC NVE Group Configuration +--------------------------- + +A NVE Group corresponds to a specific set of NVEs. A Client NVE is +assigned to an NVE Group based on whether there is a match for either +its virtual or underlay network address against the VN and/or UN address +prefixes specified in the NVE Group definition. When an NVE Group +definition specifies both VN and UN address prefixes, then an NVE must +match both prefixes in order to be assigned to the NVE Group. In the +event that multiple NVE Groups match based on VN and/or UN addresses, +the NVE is assigned to the first NVE Group listed in the configuration. +If an NVE is not assigned to an NVE Group, its messages will be ignored. + +Configuration values specified for an NVE group apply to all +member NVEs and override configuration values specified in the VNC +Defaults section. + +**At least one `nve-group` is mandatory for useful VNC operation.** + +.. clicmd:: vnc nve-group NAME + + Enter VNC configuration mode for defining the NVE group `name`. + Use `exit` or `exit-vnc` to exit group configuration mode. + + .. code-block:: frr + + vnc nve-group group1 + ... configuration commands + exit-vnc + + +The following statements are valid in an NVE group definition: + +.. clicmd:: l2rd NVE-ID-VALUE + + Set the value used to distinguish NVEs connected to the same physical + Ethernet segment (i.e., at the same location) [#]_. + + The nve-id subfield may be specified as either a literal value in the range + 1-255, or it may be specified as `auto:vn`, which means to use the + least-significant octet of the originating NVE's VN address. + +.. clicmd:: prefix vn|un A.B.C.D/M|X:X::X:X/M + + Specify the matching prefix for this NVE group by either virtual-network + address (`vn`) or underlay-network address (`un`). Either or both + virtual-network and underlay-network prefixes may be specified. Subsequent + virtual-network or underlay-network values within a `vnc nve-group` + `exit-vnc` block override their respective previous values. + + These prefixes are used only for determining assignments of NVEs to NVE + Groups. + +.. clicmd:: rd ROUTE-DISTINGUISHER + + Specify the route distinguisher for routes advertised via BGP + VPNs. The route distinguisher must be in one of these forms: + + - ``IPv4-address:two-byte-integer`` + - ``four-byte-autonomous-system-number:two-byte-integer`` + - ``two-byte-autonomous-system-number:four-byte-integer`` + - ``auto:vn:two-byte-integer`` + + Routes originated by NVEs in the NVE group will use the group's specified + `route-distinguisher` when they are advertised via BGP. If the `auto` form + is specified, it means that a matching NVE has its RD set to + ``rd_type=IP=1:IPv4-address=VN-address:two-byte-integer``, for IPv4 VN + addresses and + ``rd_type=IP=1:IPv4-address=Last-four-bytes-of-VN-address:two-byte-integer``, + for IPv6 VN addresses. + + If the NVE group definition does not specify a `route-distinguisher`, then + the default `route-distinguisher` is used. If neither a group nor a default + `route-distinguisher` is configured, then the advertised RD is set to + ``two-byte-autonomous-system-number=0:four-byte-integer=0``. + +.. clicmd:: response-lifetime LIFETIME|infinite + + Specify the response lifetime, in seconds, to be included in RFP response + messages sent to NVEs. If the value 'infinite' is given, an infinite + lifetime will be used. + + Note that this parameter is not the same as the lifetime supplied by NVEs in + RFP registration messages. This parameter does not affect the lifetime value + attached to routes sent by this server via BGP. + + If the NVE group definition does not specify a `response-lifetime`, the + default `response-lifetime` will be used. If neither a group nor a default + `response-lifetime` is configured, the value 3600 will be used. The maximum + response lifetime is 2147483647. + +.. clicmd:: rt export RT-LIST + +.. clicmd:: rt import RT-LIST + +.. clicmd:: rt both RT-LIST + + Specify route target import and export lists. `rt-list` is a + space-separated list of route targets, each element of which is + in one of the following forms: + + - ``IPv4-address:two-byte-integer`` + - ``four-byte-autonomous-system-number:two-byte-integer`` + - ``two-byte-autonomous-system-number:four-byte-integer`` + + The first form, `rt export`, specifies an `export rt-list`. The `export + rt-list` will be attached to routes originated by NVEs in the NVE group + when they are advertised via BGP. If the NVE group definition does not + specify an `export rt-list`, then the default `export rt-list` is used. + If neither a group nor a default `export rt-list` is configured, then no + RT list will be sent; in turn, these routes will probably not be + processed by receiving NVAs. + + The second form, `rt import` specifies an `import rt-list`, which is a + filter for incoming routes. In order to be made available to NVEs in the + group, incoming BGP VPN routes must have RT lists that have at least one + route target in common with the group's `import rt-list`. + + If the NVE group definition does not specify an import filter, then the + default `import rt-list` is used. If neither a group nor a default + `import rt-list` is configured, there can be no RT intersections when + receiving BGP routes and therefore no incoming BGP routes will be + processed for the group. + + The third, `rt both`, is a shorthand way of specifying both lists + simultaneously, and is equivalent to `rt export `rt-list`` followed by + `rt import `rt-list``. + +.. clicmd:: export bgp|zebra route-map MAP-NAME + + Specify that the named route-map should be applied to routes being exported + to bgp or zebra. This parameter is used in conjunction with + :ref:`configuring-export-of-routes-to-other-routing-protocols`. This item + is optional. + +.. clicmd:: export bgp|zebra no route-map + + Specify that no route-map should be applied to routes being exported to bgp + or zebra. This parameter is used in conjunction with + :ref:`configuring-export-of-routes-to-other-routing-protocols`. This item + is optional. + +.. clicmd:: export bgp|zebra ipv4|ipv6 prefix-list LIST-NAME + + Specify that the named prefix-list filter should be applied to routes being + exported to bgp or zebra. Prefix-lists for ipv4 and ipv6 are independent of + each other. This parameter is used in conjunction with + :ref:`configuring-export-of-routes-to-other-routing-protocols`. This item + is optional. + +.. clicmd:: export bgp|zebra no ipv4|ipv6 prefix-list + + Specify that no prefix-list filter should be applied to routes being + exported to bgp or zebra. This parameter is used in conjunction with + :ref:`configuring-export-of-routes-to-other-routing-protocols`. This item + is optional. + +.. _vnc-l2-group-configuration: + +VNC L2 Group Configuration +-------------------------- + +The route targets advertised with prefixes and addresses registered by an NVE +are determined based on the NVE's associated VNC NVE Group Configuration, +:ref:`vnc-nve-group-configuration`. Layer 2 (L2) Groups are used to override +the route targets for an NVE's Ethernet registrations based on the Logical +Network Identifier and label value. A Logical Network Identifier is used to +uniquely identify a logical Ethernet segment and is conceptually similar to the +Ethernet Segment Identifier defined in :rfc:`7432`. Both the Logical Network +Identifier and Label are passed to VNC via RFP prefix and address registration. + +Note that a corresponding NVE group configuration must be present, and that +other NVE associated configuration information, notably RD, is not impacted by +L2 Group Configuration. + +.. clicmd:: vnc l2-group NAME + + Enter VNC configuration mode for defining the L2 group `name`. + Use `exit` or `exit-vnc` to exit group configuration mode. + + .. code-block:: frr + + vnc l2-group group1 + ... configuration commands + exit-vnc + + + + Delete the L2 group named `name`. + +The following statements are valid in a L2 group definition: + +.. clicmd:: logical-network-id VALUE + + Define the Logical Network Identifier with a value in the range of + 0-4294967295 that identifies the logical Ethernet segment. + +.. clicmd:: labels LABEL-LIST + + + Add or remove labels associated with the group. `label-list` is a + space separated list of label values in the range of 0-1048575. + +.. clicmd:: rt import RT-TARGET + +.. clicmd:: rt export RT-TARGET + +.. clicmd:: rt both RT-TARGET + + Specify the route target import and export value associated with the group. + A complete definition of these parameters is given above, + :ref:`vnc-nve-group-configuration`. + +.. _configuring-redistribution-of-routes-from-other-routing-protocols: + +Configuring Redistribution of Routes from Other Routing Protocols +----------------------------------------------------------------- + +Routes from other protocols (including BGP) can be provided to VNC (both for +RFP and for redistribution via BGP) from three sources: the zebra kernel +routing process; directly from the main (default) unicast BGP RIB; or directly +from a designated BGP unicast exterior routing RIB instance. + +The protocol named in the `vnc redistribute` command indicates the route +source: `bgp-direct` routes come directly from the main (default) unicast BGP +RIB and are available for RFP and are redistributed via BGP; +`bgp-direct-to-nve-groups` routes come directly from a designated BGP unicast +routing RIB and are made available only to RFP; and routes from other protocols +come from the zebra kernel routing process. +Note that the zebra process does not need to be active if +only `bgp-direct` or `bgp-direct-to-nve-groups` routes are used. + +zebra routes +^^^^^^^^^^^^ + +Routes originating from protocols other than BGP must be obtained +via the zebra routing process. +Redistribution of these routes into VNC does not support policy mechanisms +such as prefix-lists or route-maps. + +bgp-direct routes +^^^^^^^^^^^^^^^^^ + +`bgp-direct` redistribution supports policy via +prefix lists and route-maps. This policy is applied to incoming +original unicast routes before the redistribution translations +(described below) are performed. + +Redistribution of `bgp-direct` routes is performed in one of three +possible modes: `plain`, `nve-group`, or `resolve-nve`. +The default mode is `plain`. +These modes indicate the kind of translations applied to routes before +they are added to the VNC RIB. + +In `plain` mode, the route's next hop is unchanged and the RD is set +based on the next hop. +For `bgp-direct` redistribution, the following translations are performed: + +- The VN address is set to the original unicast route's next hop address. +- The UN address is NOT set. (VN->UN mapping will occur via + ENCAP route or attribute, based on `vnc advertise-un-method` + setting, generated by the RFP registration of the actual NVE) +- The RD is set to as if auto:vn:0 were specified (i.e., + `rd_type=IP=1`:`IPv4-address=VN-address`:`two-byte-integer=0`) +- The RT list is included in the extended community list copied from the + original unicast route (i.e., it must be set in the original unicast route). + +In `nve-group` mode, routes are registered with VNC as if they came from an NVE +in the nve-group designated in the `vnc redistribute nve-group` command. The +following translations are performed: + +- The next hop/VN address is set to the VN prefix configured for the + redistribute nve-group. +- The UN address is set to the UN prefix configured for the redistribute + nve-group. +- The RD is set to the RD configured for the redistribute nve-group. +- The RT list is set to the RT list configured for the redistribute nve-group. + If `bgp-direct` routes are being redistributed, any extended communities + present in the original unicast route will also be included. + +In `resolve-nve` mode, the next hop of the original BGP route is typically the +address of an NVE connected router (CE) connected by one or more NVEs. +Each of the connected NVEs will register, via RFP, a VNC host route to the CE. +This mode may be though of as a mechanism to proxy RFP registrations of BGP +unicast routes on behalf of registering NVEs. + +Multiple copies of the BGP route, one per matching NVE host route, will be +added to VNC. In other words, for a given BGP unicast route, each instance of +a RFP-registered host route to the unicast route's next hop will result in an +instance of an imported VNC route. Each such imported VNC route will have a +prefix equal to the original BGP unicast route's prefix, and a next hop equal +to the next hop of the matching RFP-registered host route. If there is no +RFP-registered host route to the next hop of the BGP unicast route, no +corresponding VNC route will be imported. + +The following translations are applied: + +- The Next Hop is set to the next hop of the NVE route (i.e., the + VN address of the NVE). + +- The extended community list in the new route is set to the + union of: + +- Any extended communities in the original BGP route + + - Any extended communities in the NVE route + - An added route-origin extended community with the next hop of the + original BGP route + is added to the new route. + The value of the local administrator field defaults 5226 but may + be configured by the user via the `roo-ec-local-admin` parameter. + +- The Tunnel Encapsulation attribute is set to the value of the Tunnel + Encapsulation attribute of the NVE route, if any. + + +bgp-direct-to-nve-groups routes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Unicast routes from the main or a designated instance of BGP may be +redistributed to VNC as bgp-direct-to-nve-groups routes. These routes are NOT +announced via BGP, but they are made available for local RFP lookup in response +to queries from NVEs. + +A non-main/default BGP instance is configured using the +`router bgp AS view NAME` command as described elsewhere in this document. + +In order for a route in the unicast BGP RIB to be made available to a querying +NVE, there must already be, available to that NVE, an (interior) VNC route +matching the next hop address of the unicast route. When the unicast route is +provided to the NVE, its next hop is replaced by the next hop of the +corresponding NVE. If there are multiple longest-prefix-match VNC routes, the +unicast route will be replicated for each. + +There is currently no policy (prefix-list or route-map) support for +`bgp-direct-to-nve-groups` routes. + +Redistribution Command Syntax +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. clicmd:: vnc redistribute ipv4|ipv6 bgp|bgp-direct|ipv6 bgp-direct-to-nve-groups|connected|kernel|ospf|rip|static + +.. clicmd:: vnc redistribute ipv4|ipv6 bgp-direct-to-nve-groups view VIEWNAME + + + Import (or do not import) prefixes from another routing protocols. Specify + both the address family to import (`ipv4` or `ipv6`) and the protocol + (`bgp`, `bgp-direct`, `bgp-direct-to-nve-groups`, `connected`, `kernel`, + `ospf`, `rip`, or `static`). Repeat this statement as needed for each + combination of address family and routing protocol. Prefixes from protocol + `bgp-direct` are imported from unicast BGP in the same bgpd process. + Prefixes from all other protocols (including `bgp`) are imported via the + `zebra` kernel routing process. + +.. clicmd:: vnc redistribute mode plain|nve-group|resolve-nve + + Redistribute routes from other protocols into VNC using the specified mode. + Not all combinations of modes and protocols are supported. + +.. clicmd:: vnc redistribute nve-group GROUP-NAME + + + When using `nve-group` mode, assign (or do not assign) the NVE group + `group-name` to routes redistributed from another routing protocol. + `group-name` must be configured using `vnc nve-group`. + + The VN and UN prefixes of the nve-group must both be configured, and each + prefix must be specified as a full-length (/32 for IPv4, /128 for IPv6) + prefix. + +.. clicmd:: vnc redistribute lifetime LIFETIME|infinite + + Assign a registration lifetime, either `lifetime` seconds or `infinite`, to + prefixes redistributed from other routing protocols as if they had been + received via RFP registration messages from an NVE. `lifetime` can be any + integer between 1 and 4294967295, inclusive. + +.. clicmd:: vnc redistribute resolve-nve roo-ec-local-admin 0-65536 + + Assign a value to the local-administrator subfield used in the + Route Origin extended community that is assigned to routes exported + under the `resolve-nve` mode. The default value is `5226`. + +The following four `prefix-list` and `route-map` commands may be specified +in the context of an nve-group or not. If they are specified in the context +of an nve-group, they apply only if the redistribution mode is `nve-group`, +and then only for routes being redistributed from `bgp-direct`. If they are +specified outside the context of an nve-group, then they apply only for +redistribution modes `plain` and `resolve-nve`, and then only for routes +being redistributed from `bgp-direct`. + +.. clicmd:: vnc redistribute bgp-direct (ipv4|ipv6) prefix-list LIST-NAME + + When redistributing `bgp-direct` routes, + specifies that the named prefix-list should be applied. + +.. clicmd:: vnc redistribute bgp-direct no (ipv4|ipv6) prefix-list + + When redistributing `bgp-direct` routes, + specifies that no prefix-list should be applied. + +.. clicmd:: vnc redistribute bgp-direct route-map MAP-NAME + + When redistributing `bgp-direct` routes, + specifies that the named route-map should be applied. + +.. clicmd:: vnc redistribute bgp-direct no route-map + + When redistributing `bgp-direct` routes, + specifies that no route-map should be applied. + +.. _configuring-export-of-routes-to-other-routing-protocols: + +Configuring Export of Routes to Other Routing Protocols +------------------------------------------------------- + +Routes from VNC (both for RFP and for redistribution via BGP) can be provided +to other protocols, either via zebra or directly to BGP. + +It is important to note that when exporting routes to other protocols, the +downstream protocol must also be configured to import the routes. For example, +when VNC routes are exported to unicast BGP, the BGP configuration must include +a corresponding `redistribute vnc-direct` statement. + +.. clicmd:: export bgp|zebra mode none|group-nve|registering-nve|ce + + Specify how routes should be exported to bgp or zebra. If the mode is + `none`, routes are not exported. If the mode is `group-nve`, routes are + exported according to nve-group or vrf-policy group configuration + (:ref:`vnc-nve-group-configuration`): if a group is configured to allow + export, then each prefix visible to the group is exported with next hops set + to the currently-registered NVEs. If the mode is `registering-nve`, then all + VNC routes are exported with their original next hops. If the mode is `ce`, + only VNC routes that have an NVE connected CE Router encoded in a Route + Origin Extended Community are exported. This extended community must have an + administrative value that matches the configured `roo-ec-local-admin` value. + The next hop of the exported route is set to the encoded NVE connected CE + Router. + + The default for both bgp and zebra is mode `none`. + +.. clicmd:: vnc export bgp|zebra group-nve group GROUP-NAME + +.. clicmd:: vnc export bgp|zebra group-nve no group GROUP-NAME + + When export mode is `group-nve`, export (or do not export) prefixes from the + specified nve-group or vrf-policy group to unicast BGP or to zebra. Repeat + this statement as needed for each nve-group to be exported. Each VNC prefix + that is exported will result in N exported routes to the prefix, each with a + next hop corresponding to one of the N NVEs currently associated with the + nve-group. + +Some commands have a special meaning under certain export modes. + +:clicmd:`export bgp|zebra ipv4|ipv6 prefix-list LIST-NAME` + When export mode is `ce` or `registering-nve`, + specifies that the named prefix-list should be applied to routes + being exported to bgp or zebra. + Prefix-lists for ipv4 and ipv6 are independent of each other. + +:clicmd:`export bgp|zebra no ipv4|ipv6 prefix-list` + When export mode is `ce` or `registering-nve`, + specifies that no prefix-list should be applied to routes + being exported to bgp or zebra. + +:clicmd:`export bgp|zebra route-map MAP-NAME` + When export mode is `ce` or `registering-nve`, specifies that the named + route-map should be applied to routes being exported to bgp or zebra. + +:clicmd:`export bgp|zebra no route-map` + When export mode is `ce` or `registering-nve`, specifies that no route-map + should be applied to routes being exported to bgp or zebra. + + When the export mode is `group-nve`, policy for exported routes is specified + per-NVE-group or vrf-policy group inside a `nve-group` `RFG-NAME` block via + the following commands(:ref:`vnc-nve-group-configuration`): + +:clicmd:`export bgp|zebra route-map MAP-NAME` + This command is valid inside a `nve-group` `RFG-NAME` block. It specifies + that the named route-map should be applied to routes being exported to bgp + or zebra. + +:clicmd:`export bgp|zebra no route-map` + This command is valid inside a `nve-group` `RFG-NAME` block. It specifies + that no route-map should be applied to routes being exported to bgp or + zebra. + +:clicmd:`export bgp|zebra ipv4|ipv6 prefix-list LIST-NAME` + This command is valid inside a `nve-group` `RFG-NAME` block. It specifies + that the named prefix-list filter should be applied to routes being exported + to bgp or zebra. Prefix-lists for ipv4 and ipv6 are independent of each + other. + +:clicmd:`export bgp|zebra no ipv4|ipv6 prefix-list` + This command is valid inside a `nve-group` `RFG-NAME` block. It specifies + that no prefix-list filter should be applied to routes being exported to + bgp or zebra. + +.. _manual-address-control: + +Manual Address Control +====================== + +The commands in this section can be used to augment normal dynamic VNC. The +`add vnc` commands can be used to manually add IP prefix or Ethernet MAC +address forwarding information. The `clear vnc` commands can be used to remove +manually and dynamically added information. + +.. clicmd:: add vnc prefix (A.B.C.D/M|X:X::X:X/M) vn (A.B.C.D|X:X::X:X) un (A.B.C.D|X:X::X:X) [cost (0-255)] [lifetime (infinite|(1-4294967295))] [local-next-hop (A.B.C.D|X:X::X:X) [local-cost (0-255)]] + + Register an IP prefix on behalf of the NVE identified by the VN and UN + addresses. The `cost` parameter provides the administrative preference of + the forwarding information for remote advertisement. If omitted, it defaults + to 255 (lowest preference). The `lifetime` parameter identifies the period, + in seconds, that the information remains valid. If omitted, it defaults to + `infinite`. The optional `local-next-hop` parameter is used to configure a + nexthop to be used by an NVE to reach the prefix via a locally connected CE + router. This information remains local to the NVA, i.e., not passed to other + NVAs, and is only passed to registered NVEs. When specified, it is also + possible to provide a `local-cost` parameter to provide a forwarding + preference. If omitted, it defaults to 255 (lowest preference). + +.. clicmd:: add vnc mac xx:xx:xx:xx:xx:xx virtual-network-identifier (1-4294967295) vn (A.B.C.D|X:X::X:X) un (A.B.C.D|X:X::X:X) [prefix (A.B.C.D/M|X:X::X:X/M)] [cost (0-255)] [lifetime (infinite|(1-4294967295))] + + Register a MAC address for a logical Ethernet (L2VPN) on behalf of the NVE + identified by the VN and UN addresses. The optional `prefix` parameter is to + support enable IP address mediation for the given prefix. The `cost` + parameter provides the administrative preference of the forwarding + information. If omitted, it defaults to 255. The `lifetime` parameter + identifies the period, in seconds, that the information remains valid. If + omitted, it defaults to `infinite`. + +.. clicmd:: clear vnc prefix (\*|A.B.C.D/M|X:X::X:X/M) (\*|[(vn|un) (A.B.C.D|X:X::X:X|\*) [(un|vn) (A.B.C.D|X:X::X:X|\*)] [mac xx:xx:xx:xx:xx:xx] [local-next-hop (A.B.C.D|X:X::X:X)]) + + Delete the information identified by prefix, VN address, and UN address. + Any or all of these parameters may be wildcarded to (potentially) match more + than one registration. The optional `mac` parameter specifies a layer-2 MAC + address that must match the registration(s) to be deleted. The optional + `local-next-hop` parameter is used to delete specific local nexthop + information. + +.. clicmd:: clear vnc mac (\*|xx:xx:xx:xx:xx:xx) virtual-network-identifier (\*|(1-4294967295)) (\*|[(vn|un) (A.B.C.D|X:X::X:X|\*) [(un|vn) (A.B.C.D|X:X::X:X|\*)] [prefix (\*|A.B.C.D/M|X:X::X:X/M)]) + + Delete mac forwarding information. Any or all of these parameters may be + wildcarded to (potentially) match more than one registration. The default + value for the `prefix` parameter is the wildcard value `*`. + +.. clicmd:: clear vnc nve (\*|((vn|un) (A.B.C.D|X:X::X:X) [(un|vn) (A.B.C.D|X:X::X:X)])) + + Delete prefixes associated with the NVE specified by the given VN and UN + addresses. It is permissible to specify only one of VN or UN, in which case + any matching registration will be deleted. It is also permissible to specify + `*` in lieu of any VN or UN address, in which case all registrations will + match. + +.. _other-vnc-related-commands: + +Other VNC-Related Commands +========================== + +Note: VNC-Related configuration can be obtained via the `show +running-configuration` command when in `enable` mode. + +The following commands are used to clear and display Virtual Network Control +related information: + +.. clicmd:: clear vnc counters + + Reset the counter values stored by the NVA. Counter + values can be seen using the `show vnc` commands listed above. This + command is only available in `enable` mode. + +.. clicmd:: show vnc summary + + Print counter values and other general information + about the NVA. Counter values can be reset + using the `clear vnc counters` command listed below. + +.. clicmd:: show vnc nves + +.. clicmd:: show vnc nves vn|un ADDRESS + + Display the NVA's current clients. Specifying `address` limits the output to + the NVEs whose addresses match `address`. The time since the NVA last + communicated with the NVE, per-NVE summary counters and each NVE's addresses + will be displayed. + +.. clicmd:: show vnc queries + +.. clicmd:: show vnc queries PREFIX + + Display active Query information. Queries remain valid for the default + Response Lifetime (:ref:`vnc-defaults-configuration`) or NVE-group Response + Lifetime (:ref:`vnc-nve-group-configuration`). Specifying `prefix` limits + the output to Query Targets that fall within `prefix`. + + Query information is provided for each querying NVE, and includes the Query + Target and the time remaining before the information is removed. + +.. clicmd:: show vnc registrations [all|local|remote|holddown|imported] + +.. clicmd:: show vnc registrations [all|local|remote|holddown|imported] PREFIX + + Display local, remote, holddown, and/or imported registration information. + Local registrations are routes received via RFP, which are present in the + NVA Registrations Cache. Remote registrations are routes received via BGP + (VPN SAFIs), which are present in the NVE-group import tables. Holddown + registrations are local and remote routes that have been withdrawn but whose + holddown timeouts have not yet elapsed. Imported information represents + routes that are imported into NVA and are made available to querying NVEs. + Depending on configuration, imported routes may also be advertised via BGP. + Specifying `prefix` limits the output to the registered prefixes that fall + within `prefix`. + + Registration information includes the registered prefix, the registering NVE + addresses, the registered administrative cost, the registration lifetime and + the time since the information was registered or, in the case of Holddown + registrations, the amount of time remaining before the information is + removed. + +.. clicmd:: show vnc responses [active|removed] + +.. clicmd:: show vnc responses [active|removed] PREFIX + + Display all, active and/or removed response information which are + present in the NVA Responses Cache. Responses remain valid for the + default Response Lifetime (:ref:`vnc-defaults-configuration`) or + NVE-group Response Lifetime (:ref:`vnc-nve-group-configuration`.) + When Removal Responses are enabled (:ref:`general-vnc-configuration`), + such responses are listed for the Response Lifetime. Specifying + `prefix` limits the output to the addresses that fall within + `prefix`. + + Response information is provided for each querying NVE, and includes + the response prefix, the prefix-associated registering NVE addresses, + the administrative cost, the provided response lifetime and the time + remaining before the information is to be removed or will become inactive. + +.. clicmd:: show memory vnc + + Print the number of memory items allocated by the NVA. + +.. _example-vnc-and-vnc-gw-configurations: + +Example VNC and VNC-GW Configurations +===================================== + +.. _vnc-mesh-nva-config: + +Mesh NVA Configuration +---------------------- + +This example includes three NVAs, nine NVEs, and two NVE groups. Note that +while not shown, a single physical device may support multiple logical NVEs. +:ref:`vnc-fig-vnc-mesh` shows ``code NVA-1`` (192.168.1.100), ``NVA 2`` +(192.168.1.101), and ``NVA 3`` (192.168.1.102), which are connected in a full +mesh. Each is a member of the autonomous system 64512. Each NVA provides VNC +services to three NVE clients in the 172.16.0.0/16 virtual-network address +range. The 172.16.0.0/16 address range is partitioned into two NVE groups, +``group1`` (172.16.0.0/17) and ``group2`` (172.16.128.0/17). + +Each NVE belongs to either NVE group ``group1`` or NVE group ``group2``. The +NVEs ``NVE 1``, ``NVE 2``, ``NVE 4``, ``NVE 7``, and ``NVE 8`` are members of +the NVE group ``group1``. The NVEs ``NVE 3``, ``NVE 5``, ``NVE 6``, and ``NVE +9`` are members of the NVE group ``group2``. + +Each NVA advertises NVE underlay-network IP addresses using the +Tunnel Encapsulation Attribute. + +.. _vnc-fig-vnc-mesh: + +.. figure:: ../figures/fig-vnc-mesh.png + :align: center + :alt: Three-way Mesh + + A three-way full mesh with three NVEs per NVA. + +:file:`bgpd.conf` for ``NVA 1`` (192.168.1.100): + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.100 + + neighbor 192.168.1.101 remote-as 64512 + neighbor 192.168.1.102 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.101 activate + neighbor 192.168.1.102 activate + exit-address-family + + vnc defaults + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + + vnc nve-group group1 + prefix vn 172.16.0.0/17 + rt both 1000:1 + exit-vnc + + vnc nve-group group2 + prefix vn 172.16.128.0/17 + rt both 1000:2 + exit-vnc + + exit + +:file:`bgpd.conf` for ``NVA 2`` (192.168.1.101): + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.101 + + neighbor 192.168.1.100 remote-as 64512 + neighbor 192.168.1.102 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + neighbor 192.168.1.102 activate + exit-address-family + + vnc nve-group group1 + prefix vn 172.16.0.0/17 + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + exit + +:file:`bgpd.conf` for ``NVA 3`` (192.168.1.102): + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.102 + + neighbor 192.168.1.101 remote-as 64512 + neighbor 192.168.1.102 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + neighbor 192.168.1.101 activate + exit-address-family + + vnc defaults + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + + vnc nve-group group1 + prefix vn 172.16.128.0/17 + exit-vnc + exit + + +Mesh NVA and VNC-GW Configuration +--------------------------------- + +This example includes two NVAs, each with two associated NVEs, and two VNC-GWs, +each supporting two CE routers physically attached to the four NVEs. Note that +this example is showing a more complex configuration where VNC-GW is separated +from normal NVA functions; it is equally possible to simplify the configuration +and combine NVA and VNC-GW functions in a single FRR instance. + +.. _vnc-fig-vnc-gw: +.. figure:: ../figures/fig-vnc-gw.png + :align: center + :alt: FRR VNC Gateway + + Meshed NVEs and VNC-GWs + +As shown in :ref:`vnc-fig-vnc-gw`, NVAs and VNC-GWs are connected in a full iBGP +mesh. The VNC-GWs each have two CEs configured as route-reflector clients. +Each client provides BGP updates with unicast routes that the VNC-GW reflects +to the other client. The VNC-GW also imports these unicast routes into VPN +routes to be shared with the other VNC-GW and the two NVAs. This route +importation is controlled with the ``vnc redistribute`` statements shown in the +configuration. Similarly, registrations sent by NVEs via RFP to the NVAs are +exported by the VNC-GWs to the route-reflector clients as unicast routes. RFP +registrations exported this way have a next-hop address of the CE behind the +connected (registering) NVE. Exporting VNC routes as IPv4 unicast is enabled +with the ``vnc export`` command below. + +The configuration for ``VNC-GW 1`` is shown below. + +.. code-block:: frr + + router bgp 64512 + bgp router-id 192.168.1.101 + bgp cluster-id 1.2.3.4 + neighbor 192.168.1.102 remote-as 64512 + neighbor 192.168.1.103 remote-as 64512 + neighbor 192.168.1.104 remote-as 64512 + neighbor 172.16.1.2 remote-as 64512 + neighbor 172.16.2.2 remote-as 64512 + ! + address-family ipv4 unicast + redistribute vnc-direct + no neighbor 192.168.1.102 activate + no neighbor 192.168.1.103 activate + no neighbor 192.168.1.104 activate + neighbor 172.16.1.2 route-reflector-client + neighbor 172.16.2.2 route-reflector-client + exit-address-family + ! + address-family ipv4 vpn + neighbor 192.168.1.102 activate + neighbor 192.168.1.103 activate + neighbor 192.168.1.104 activate + exit-address-family + vnc export bgp mode ce + vnc redistribute mode resolve-nve + vnc redistribute ipv4 bgp-direct + exit + +Note that in the VNC-GW configuration, the neighboring VNC-GW and NVAs each +have a statement disabling the IPv4 unicast address family. IPv4 unicast is on +by default and this prevents the other VNC-GW and NVAs from learning unicast +routes advertised by the route-reflector clients. + +Configuration for ``NVA 2``: + +.. code-block:: frr + + router bgp 64512 + bgp router-id 192.168.1.104 + neighbor 192.168.1.101 remote-as 64512 + neighbor 192.168.1.102 remote-as 64512 + neighbor 192.168.1.103 remote-as 64512 + ! + address-family ipv4 unicast + no neighbor 192.168.1.101 activate + no neighbor 192.168.1.102 activate + no neighbor 192.168.1.103 activate + exit-address-family + ! + address-family ipv4 vpn + neighbor 192.168.1.101 activate + neighbor 192.168.1.102 activate + neighbor 192.168.1.103 activate + exit-address-family + ! + vnc defaults + response-lifetime 3600 + exit-vnc + vnc nve-group nve1 + prefix vn 172.16.1.1/32 + response-lifetime 3600 + rt both 1000:1 1000:2 + exit-vnc + vnc nve-group nve2 + prefix vn 172.16.2.1/32 + response-lifetime 3600 + rt both 1000:1 1000:2 + exit-vnc + exit + +.. TBD make this its own example: +.. +.. @float Figure,fig:fig-vnc-gw-rr +.. @center @image{fig-vnc-gw-rr,400pt,,FRR VNC Gateway with RR} +.. @end float +.. An NVA can also import unicast routes from BGP without advertising the +.. imported routes as VPN routes. Such imported routes, while not +.. distributed to other NVAs or VNC-GWs, are are available to NVEs via +.. RFP query messages sent to the NVA. @ref{fig:fig-vnc-gw-rr} +.. shows an example topology where unicast routes are imported into NVAs +.. from a Route Reflector. (@pxref{Route Reflector} for route reflector +.. configuration details.) The following three lines can be added to the +.. ``NVA 1`` and ``NVA 2`` configurations to import routes into VNC +.. for local VNC use: +.. +.. @verbatim +.. neighbor 192.168.1.105 remote-as 64512 +.. vnc redistribute mode plain +.. vnc redistribute ipv4 bgp-direct-to-nve-groups +.. @end verbatim + +.. _vnc-with-frr-route-reflector-config: + +VNC with FRR Route Reflector Configuration +------------------------------------------ + +A route reflector eliminates the need for a fully meshed NVA network by acting +as the hub between NVAs. :ref:`vnc-fig-vnc-frr-route-reflector` shows BGP +route reflector ``BGP Route Reflector 1`` (192.168.1.100) as a route reflector +for NVAs ``NVA 2``(192.168.1.101) and ``NVA 3`` (192.168.1.102). + +.. _vnc-fig-vnc-frr-route-reflector: +.. figure:: ../figures/fig-vnc-frr-route-reflector.png + :align: center + :alt: FRR Route Reflector + + Two NVAs and a BGP Route Reflector + +``NVA 2`` and ``NVA 3`` advertise NVE underlay-network IP addresses using the +Tunnel Encapsulation Attribute. ``BGP Route Reflector 1`` ``reflects'' +advertisements from ``NVA 2`` to ``NVA 3`` and vice versa. + +As in the example of :ref:`vnc-mesh-nva-config`, there are two NVE groups. The +172.16.0.0/16 address range is partitioned into two NVE groups, ``group1`` +(172.16.0.0/17) and ``group2`` (172.16.128.0/17). The NVE ``NVE 4``, ``NVE +7``, and ``NVE 8`` are members of the NVE group ``group1``. The NVEs ``NVE +5``, ``NVE 6``, and ``NVE 9`` are members of the NVE group ``group2``. + +:file:`bgpd.conf` for ``BGP Route Reflector 1`` on 192.168.1.100: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.100 + + neighbor 192.168.1.101 remote-as 64512 + neighbor 192.168.1.101 port 7179 + neighbor 192.168.1.101 description iBGP-client-192-168-1-101 + + neighbor 192.168.1.102 remote-as 64512 + neighbor 192.168.1.102 port 7179 + neighbor 192.168.1.102 description iBGP-client-192-168-1-102 + + address-family ipv4 unicast + neighbor 192.168.1.101 route-reflector-client + neighbor 192.168.1.102 route-reflector-client + exit-address-family + + address-family ipv4 vpn + neighbor 192.168.1.101 activate + neighbor 192.168.1.102 activate + + neighbor 192.168.1.101 route-reflector-client + neighbor 192.168.1.102 route-reflector-client + exit-address-family + + exit + +:file:`bgpd.conf` for ``NVA 2`` on 192.168.1.101: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.101 + + neighbor 192.168.1.100 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + exit-address-family + + vnc nve-group group1 + prefix vn 172.16.0.0/17 + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + exit + +:file:`bgpd.conf` for ``NVA 2`` on 192.168.1.102: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.102 + + neighbor 192.168.1.100 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + exit-address-family + + vnc defaults + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + + vnc nve-group group1 + prefix vn 172.16.128.0/17 + exit-vnc + exit + +While not shown, an NVA can also be configured as a route reflector. + +.. _vnc-with-commercial-route-reflector-config: + +VNC with Commercial Route Reflector Configuration +------------------------------------------------- + +This example is identical to :ref:`vnc-with-frr-route-reflector-config` +with the exception that the route reflector is a commercial router. Only the +VNC-relevant configuration is provided. + +.. figure:: ../figures/fig-vnc-commercial-route-reflector.png + :align: center + :alt: Commercial Route Reflector + + Two NVAs with a commercial route reflector + +:file:`bgpd.conf` for BGP route reflector ``Commercial Router`` on 192.168.1.104::: + + version 8.5R1.13; + routing-options { + rib inet.0 { + static { + route 172.16.0.0/16 next-hop 192.168.1.104; + } + } + autonomous-system 64512; + resolution { + rib inet.3 { + resolution-ribs inet.0; + } + rib bgp.l3vpn.0 { + resolution-ribs inet.0; + } + } + } + protocols { + bgp { + advertise-inactive; + family inet { + labeled-unicast; + } + group 1 { + type internal; + advertise-inactive; + advertise-peer-as; + import h; + family inet { + unicast; + } + family inet-vpn { + unicast; + } + cluster 192.168.1.104; + neighbor 192.168.1.101; + neighbor 192.168.1.102; + } + } + } + policy-options { + policy-statement h { + from protocol bgp; + then { + as-path-prepend 64512; + accept; + } + } + } + +:file:`bgpd.conf` for ``NVA 2`` on 192.168.1.101: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.101 + + neighbor 192.168.1.100 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + exit-address-family + + vnc nve-group group1 + prefix vn 172.16.0.0/17 + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + exit + +:file:`bgpd.conf` for ``NVA 3`` on 192.168.1.102: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.102 + + neighbor 192.168.1.100 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + exit-address-family + + vnc defaults + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + + vnc nve-group group1 + prefix vn 172.16.128.0/17 + exit-vnc + exit + +VNC with Redundant Route Reflectors Configuration +------------------------------------------------- + +This example combines the previous two +(:ref:`vnc-with-frr-route-reflector-config` and +:ref:`vnc-with-commercial-route-reflector-config`) into a redundant route +reflector configuration. BGP route reflectors ``BGP Route Reflector 1`` and +``Commercial Router`` are the route reflectors for NVAs ``NVA 2`` and ``NVA +3``. The two NVAs have connections to both route reflectors. + +.. figure:: ../figures/fig-vnc-redundant-route-reflectors.png + :align: center + :alt: Redundant Route Reflectors + + FRR-based NVA with redundant route reflectors + +:file:`bgpd.conf` for ``BPGD Route Reflector 1`` on 192.168.1.100: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.100 + bgp cluster-id 192.168.1.100 + + neighbor 192.168.1.104 remote-as 64512 + + neighbor 192.168.1.101 remote-as 64512 + neighbor 192.168.1.101 description iBGP-client-192-168-1-101 + neighbor 192.168.1.101 route-reflector-client + + neighbor 192.168.1.102 remote-as 64512 + neighbor 192.168.1.102 description iBGP-client-192-168-1-102 + neighbor 192.168.1.102 route-reflector-client + + address-family ipv4 vpn + neighbor 192.168.1.101 activate + neighbor 192.168.1.102 activate + neighbor 192.168.1.104 activate + + neighbor 192.168.1.101 route-reflector-client + neighbor 192.168.1.102 route-reflector-client + exit-address-family + exit + +:file:`bgpd.conf` for ``NVA 2`` on 192.168.1.101: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.101 + + neighbor 192.168.1.100 remote-as 64512 + neighbor 192.168.1.104 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + neighbor 192.168.1.104 activate + exit-address-family + + vnc nve-group group1 + prefix vn 172.16.0.0/17 + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + exit + +:file:`bgpd.conf` for ``NVA 3`` on 192.168.1.102: + +.. code-block:: frr + + router bgp 64512 + + bgp router-id 192.168.1.102 + + neighbor 192.168.1.100 remote-as 64512 + neighbor 192.168.1.104 remote-as 64512 + + address-family ipv4 vpn + neighbor 192.168.1.100 activate + neighbor 192.168.1.104 activate + exit-address-family + + vnc defaults + rd 64512:1 + response-lifetime 200 + rt both 1000:1 1000:2 + exit-vnc + + vnc nve-group group1 + prefix vn 172.16.128.0/17 + exit-vnc + exit + +:file:`bgpd.conf` for the Commercial Router route reflector on 192.168.1.104::: + + routing-options { + rib inet.0 { + static { + route 172.16.0.0/16 next-hop 192.168.1.104; + } + } + autonomous-system 64512; + resolution { + rib inet.3 { + resolution-ribs inet.0; + } + rib bgp.l3vpn.0 { + resolution-ribs inet.0; + } + } + } + protocols { + bgp { + advertise-inactive; + family inet { + labeled-unicast; + } + group 1 { + type internal; + advertise-inactive; + advertise-peer-as; + import h; + family inet { + unicast; + } + family inet-vpn { + unicast; + } + cluster 192.168.1.104; + neighbor 192.168.1.101; + neighbor 192.168.1.102; + } + + group 2 { + type internal; + advertise-inactive; + advertise-peer-as; + import h; + family inet { + unicast; + } + family inet-vpn { + unicast; + } + neighbor 192.168.1.100; + } + + } + } + policy-options { + policy-statement h { + from protocol bgp; + then { + as-path-prepend 64512; + accept; + } + } + } + +.. [#] The nve-id is carried in the route distinguisher. It is the second octet + of the eight-octet route distinguisher generated for Ethernet / L2 + advertisements. The first octet is a constant 0xFF, and the third + through eighth octets are set to the L2 + ethernet address being advertised. + diff --git a/doc/user/vrrp.rst b/doc/user/vrrp.rst new file mode 100644 index 0000000..ef3aebe --- /dev/null +++ b/doc/user/vrrp.rst @@ -0,0 +1,573 @@ +.. _vrrp: + +**** +VRRP +**** + +:abbr:`VRRP` stands for Virtual Router Redundancy Protocol. This protocol is +used to allow multiple backup routers on the same segment to take over +operation of each others' IP addresses if the primary router fails. This is +typically used to provide fault-tolerant gateways to hosts on the segment. + +FRR implements VRRPv2 (:rfc:`3768`) and VRRPv3 (:rfc:`5798`). For VRRPv2, no +authentication methods are supported; these are deprecated in the VRRPv2 +specification as they do not provide any additional security over the base +protocol. + +.. note:: + + - VRRP is supported on Linux 5.1+ + - VRRP does not implement Accept_Mode + +.. _vrrp-starting: + +Starting VRRP +============= + +The configuration file for *vrrpd* is :file:`vrrpd.conf`. The typical location +of :file:`vrrpd.conf` is |INSTALL_PREFIX_ETC|/vrrpd.conf. + +If using integrated config, then :file:`vrrpd.conf` need not be present and +:file:`frr.conf` is read instead. + +.. program:: vrrpd + +:abbr:`VRRP` supports all the common FRR daemon start options which are +documented elsewhere. + +.. _vrrp-protocol-overview: + +Protocol Overview +================= + +From :rfc:`5798`: + + VRRP specifies an election protocol that dynamically assigns responsibility + for a virtual router to one of the VRRP routers on a LAN. The VRRP router + controlling the IPv4 or IPv6 address(es) associated with a virtual router is + called the Master, and it forwards packets sent to these IPv4 or IPv6 + addresses. VRRP Master routers are configured with virtual IPv4 or IPv6 + addresses, and VRRP Backup routers infer the address family of the virtual + addresses being carried based on the transport protocol. Within a VRRP + router, the virtual routers in each of the IPv4 and IPv6 address families + are a domain unto themselves and do not overlap. The election process + provides dynamic failover in the forwarding responsibility should the Master + become unavailable. For IPv4, the advantage gained from using VRRP is a + higher-availability default path without requiring configuration of dynamic + routing or router discovery protocols on every end-host. For IPv6, the + advantage gained from using VRRP for IPv6 is a quicker switchover to Backup + routers than can be obtained with standard IPv6 Neighbor Discovery + mechanisms. + +VRRP accomplishes these goals primarily by using a virtual MAC address shared +between the physical routers participating in a VRRP virtual router. This +reduces churn in the neighbor tables of hosts and downstream switches and makes +router failover theoretically transparent to these devices. + +FRR implements the election protocol and handles changing the operating system +interface configuration in response to protocol state changes. + +As a consequence of the shared virtual MAC requirement, VRRP is currently +supported only on Linux, as Linux is the only operating system that provides +the necessary features in its network stack to make implementing this protocol +feasible. + +When a VRRP router is acting as the Master router, FRR allows the interface(s) +with the backed-up IP addresses to remain up and functional. When the router +transitions to Backup state, these interfaces are set into ``protodown`` mode. +This is an interface mode that is functionally equivalent to ``NO-CARRIER``. +Physical drivers typically use this state indication to drop traffic on an +interface. In the case of VRRP, the interfaces in question are macvlan devices, +which are virtual interfaces. Since the IP addresses managed by VRRP are on +these interfaces, this has the same effect as removing these addresses from the +interface, but is implemented as a state flag. + +.. _vrrp-configuration: + +Configuring VRRP +================ + +VRRP is configured on a per-interface basis, with some global defaults +accessible outside the interface context. + +.. _vrrp-system-configuration: + +System Configuration +-------------------- + +FRR's VRRP implementation uses Linux macvlan devices to to implement the shared +virtual MAC feature of the protocol. Currently, it does not create those system +interfaces - they must be configured outside of FRR before VRRP can be enabled +on them. + +Each interface on which VRRP will be enabled must have at least one macvlan +device configured with the virtual MAC and placed in the proper operation mode. +The addresses backed up by VRRP are assigned to these interfaces. + +Suppose you have an interface ``eth0`` with the following configuration: + +.. code-block:: console + + $ ip addr show eth0 + 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether 02:17:45:00:aa:aa brd ff:ff:ff:ff:ff:ff + inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0 + valid_lft 72532sec preferred_lft 72532sec + inet6 fe80::17:45ff:fe00:aaaa/64 scope link + valid_lft forever preferred_lft forever + +Suppose that the IPv4 and IPv6 addresses you want to back up are ``10.0.2.16`` +and ``2001:db8::370:7334``, and that they will be managed by the virtual router +with id ``5``. A macvlan device with the appropriate MAC address must be created +before VRRP can begin to operate. + +If you are using ``ifupdown2``, the configuration is as follows: + +.. code-block:: console + + iface eth0 + ... + vrrp 5 10.0.2.16/24 2001:0db8::0370:7334/64 + +Applying this configuration with ``ifreload -a`` will create the appropriate +macvlan device. If you are using ``iproute2``, the equivalent configuration is: + +.. code-block:: console + + ip link add vrrp4-2-1 link eth0 addrgenmode random type macvlan mode bridge + ip link set dev vrrp4-2-1 address 00:00:5e:00:01:05 + ip addr add 10.0.2.16/24 dev vrrp4-2-1 + ip link set dev vrrp4-2-1 up + + ip link add vrrp6-2-1 link eth0 addrgenmode random type macvlan mode bridge + ip link set dev vrrp6-2-1 address 00:00:5e:00:02:05 + ip addr add 2001:db8::370:7334/64 dev vrrp6-2-1 + ip link set dev vrrp6-2-1 up + +In either case, the created interfaces will look like this: + +.. code-block:: console + + $ ip addr show vrrp4-2-1 + 5: vrrp4-2-1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 + link/ether 00:00:5e:00:01:05 brd ff:ff:ff:ff:ff:ff + inet 10.0.2.16/24 scope global vrrp4-2-1 + valid_lft forever preferred_lft forever + inet6 fe80::dc56:d11a:e69d:ea72/64 scope link stable-privacy + valid_lft forever preferred_lft forever + + $ ip addr show vrrp6-2-1 + 8: vrrp6-2-1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 + link/ether 00:00:5e:00:02:05 brd ff:ff:ff:ff:ff:ff + inet6 2001:db8::370:7334/64 scope global + valid_lft forever preferred_lft forever + inet6 fe80::f8b7:c9dd:a1e8:9844/64 scope link stable-privacy + valid_lft forever preferred_lft forever + +Using ``vrrp4-2-1`` as an example, a few things to note about this interface: + +- It is slaved to ``eth0``; any packets transmitted on this interface will + egress via ``eth0`` +- Its MAC address is set to the VRRP IPv4 virtual MAC specified by the RFC for + :abbr:`VRID (Virtual Router ID)` ``5`` +- The :abbr:`VIP (Virtual IP)` address ``10.0.2.16`` must not be present on + the parent interface ``eth0``. +- The link local address on the interface is not derived from the interface + MAC + +First to note is that packets transmitted on this interface will egress via +``eth0``, but with their Ethernet source MAC set to the VRRP virtual MAC. This +is how FRR's VRRP implementation accomplishes the virtual MAC requirement on +real hardware. + +Ingress traffic is a more complicated matter. Macvlan devices have multiple +operating modes that change how ingress traffic is handled. Of relevance to +FRR's implementation are the ``bridge`` and ``private`` modes. In ``private`` +mode, any ingress traffic on ``eth0`` (in our example) with a source MAC +address equal to the MAC address on any of ``eth0``'s macvlan devices will be +placed *only* on that macvlan device. This curious behavior is undesirable, +since FRR's implementation of VRRP needs to be able to receive advertisements +from neighbors while in Backup mode - i.e., while its macvlan devices are in +``protodown on``. If the macvlan devices are instead set to ``bridge`` mode, +all ingress traffic shows up on all interfaces - including ``eth0`` - +regardless of source MAC or any other factor. Consequently, macvlans used by +FRR for VRRP must be set to ``bridge`` mode or the protocol will not function +correctly. + +As for the MAC address assigned to this interface, the last byte of the address +holds the :abbr:`VRID (Virtual Router Identifier)`, in this case ``0x05``. The +second to last byte is ``0x01``, as specified by the RFC for IPv4 operation. +The IPv6 MAC address is be identical except that the second to last byte is +defined to be ``0x02``. Two things to note from this arrangement: + +1. There can only be up to 255 unique Virtual Routers on an interface (only 1 + byte is available for the VRID) +2. IPv4 and IPv6 addresses must be assigned to different macvlan devices, + because they have different MAC addresses + +Finally, take note of the generated IPv6 link local address on the interface. +For interfaces on which VRRP will operate in IPv6 mode, this link local +*cannot* be derived using the usual EUI-64 method. This is because VRRP +advertisements are sent from the link local address of this interface, and VRRP +uses the source address of received advertisements as part of its election +algorithm. If the IPv6 link local of a router is equivalent to the IPv6 link +local in a received advertisement, this can cause both routers to assume the +Master role (very bad). ``ifupdown`` knows to set the ``addrgenmode`` of the +interface properly, but when using ``iproute2`` to create the macvlan devices, +you must be careful to manually specify ``addrgenmode random``. + +A brief note on the Backup state +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It is worth noting here that an alternate choice for the implementation of the +Backup state, such as removing all the IP addresses assigned to the macvlan +device or deleting their local routes instead of setting the device into +``protodown on``, would allow the protocol to function regardless of whether +the macvlan device(s) are set to ``private`` or ``bridge`` mode. Indeed, the +strange behavior of the kernel macvlan driver in ``private`` mode, whereby it +performs what may be thought of as a sort of interface-level layer 2 "NAT" +based on source MAC, can be traced back to a patch clearly designed to +accommodate a VRRP implementation from a different vendor. However, the +``protodown`` based implementation allows for a configuration model in which +FRR does not dynamically manage the addresses assigned on a system, but instead +just manages interface state. Such a scenario was in mind when this protocol +implementation was initially built, which is why the other choices are not +currently present. Since support for placing macvlan devices into ``protodown`` +was not added to Linux until version 5.1, this also explains the relatively +restrictive kernel versioning requirement. + +In the future other methods of implementing Backup state may be added along +with a configuration knob to choose between them. + +.. _vrrp-interface-configuration: + +Interface Configuration +----------------------- + +Continuing with the example from the previous section, we assume the macvlan +interfaces have been properly configured with the proper MAC addresses and the +IPvX addresses assigned. + +In FRR, a possible VRRPv3 configuration for this interface is: + +.. code-block:: frr + + interface eth0 + vrrp 5 version 3 + vrrp 5 priority 200 + vrrp 5 advertisement-interval 1500 + vrrp 5 ip 10.0.2.16 + vrrp 5 ipv6 2001:0db8::0370:7334 + +VRRP will activate as soon as the first IPvX address configuration line is +encountered. If you do not want this behavior, use the :clicmd:`vrrp (1-255) +shutdown` command, and apply the ``no`` form when you are ready to activate +VRRP. + +At this point executing ``show vrrp`` will display the following: + +.. code-block:: console + + ubuntu-bionic# show vrrp + + Virtual Router ID 5 + Protocol Version 3 + Autoconfigured Yes + Shutdown No + Interface eth0 + VRRP interface (v4) vrrp4-2-5 + VRRP interface (v6) vrrp6-2-5 + Primary IP (v4) 10.0.2.15 + Primary IP (v6) fe80::9b91:7155:bf6a:d386 + Virtual MAC (v4) 00:00:5e:00:01:05 + Virtual MAC (v6) 00:00:5e:00:02:05 + Status (v4) Master + Status (v6) Master + Priority 200 + Effective Priority (v4) 200 + Effective Priority (v6) 200 + Preempt Mode Yes + Accept Mode Yes + Advertisement Interval 1500 ms + Master Advertisement Interval (v4) 1000 ms + Master Advertisement Interval (v6) 1000 ms + Advertisements Tx (v4) 14 + Advertisements Tx (v6) 14 + Advertisements Rx (v4) 0 + Advertisements Rx (v6) 0 + Gratuitous ARP Tx (v4) 1 + Neigh. Adverts Tx (v6) 1 + State transitions (v4) 2 + State transitions (v6) 2 + Skew Time (v4) 210 ms + Skew Time (v6) 210 ms + Master Down Interval (v4) 3210 ms + Master Down Interval (v6) 3210 ms + IPv4 Addresses 1 + .................................. 10.0.2.16 + IPv6 Addresses 1 + .................................. 2001:db8::370:7334 + +At this point, VRRP has sent gratuitous ARP requests for the IPv4 address, +Unsolicited Neighbor Advertisements for the IPv6 address, and has asked Zebra +to send Router Advertisements on its behalf. It is also transmitting VRRPv3 +advertisements on the macvlan interfaces. + +The Primary IP fields are of some interest, as the behavior may be +counterintuitive. These fields show the source address used for VRRP +advertisements. Although VRRPv3 advertisements are always transmitted on the +macvlan interfaces, in the IPv4 case the source address is set to the primary +IPv4 address on the base interface, ``eth0`` in this case. This is a protocol +requirement, and IPv4 VRRP will not function unless the base interface has an +IPv4 address assigned. In the IPv6 case the link local of the macvlan interface +is used. + +If any misconfiguration errors are detected, VRRP for the misconfigured address +family will not come up and the configuration issue will be logged to FRR's +configured logging destination. + +Per the RFC, IPv4 and IPv6 virtual routers are independent of each other. For +instance, it is possible for the IPv4 router to be in Backup state while the +IPv6 router is in Master state; or for either to be completely inoperative +while the other is operative, etc. Instances sharing the same base interface +and VRID are shown together in the show output for conceptual convenience. + +To complete your VRRP deployment, configure other routers on the segment with +the exact same system and FRR configuration as shown above. Provided each +router receives the others' VRRP advertisements, the Master election protocol +will run, one Master will be elected, and the other routers will place their +macvlan interfaces into ``protodown on`` until Master fails or priority values +are changed to favor another router. + +Switching the protocol version to VRRPv2 is accomplished simply by changing +``version 3`` to ``version 2`` in the VRID configuration line. Note that VRRPv2 +does not support IPv6, so any IPv6 configuration will be rejected by FRR when +using VRRPv2. + +.. note:: + + All VRRP routers initially start in Backup state, and wait for the + calculated Master Down Interval to pass before they assume Master status. + This prevents downstream neighbor table churn if another router is already + Master with higher priority, meaning this box will ultimately assume Backup + status once the first advertisement is received. However, if the calculated + Master Down Interval is high and this router is configured such that it will + ultimately assume Master status, then it will take a while for this to + happen. This is a known issue. + + +All interface configuration commands are documented below. + +.. clicmd:: vrrp (1-255) [version (2-3)] + + Create a VRRP router with the specified VRID on the interface. Optionally + specify the protocol version. If the protocol version is not specified, the + default is VRRPv3. + +.. clicmd:: vrrp (1-255) advertisement-interval (10-40950) + + Set the advertisement interval. This is the interval at which VRRP + advertisements will be sent. Values are given in milliseconds, but must be + multiples of 10, as VRRP itself uses centiseconds. + +.. clicmd:: vrrp (1-255) ip A.B.C.D + + Add an IPv4 address to the router. This address must already be configured + on the appropriate macvlan device. Adding an IP address to the router will + implicitly activate the router; see :clicmd:`[no] vrrp (1-255) shutdown` to + override this behavior. + +.. clicmd:: vrrp (1-255) ipv6 X:X::X:X + + Add an IPv6 address to the router. This address must already be configured + on the appropriate macvlan device. Adding an IP address to the router will + implicitly activate the router; see :clicmd:`[no] vrrp (1-255) shutdown` to + override this behavior. + + This command will fail if the protocol version is set to VRRPv2, as VRRPv2 + does not support IPv6. + +.. clicmd:: vrrp (1-255) preempt + + Toggle preempt mode. When enabled, preemption allows Backup routers with + higher priority to take over Master status from the existing Master. Enabled + by default. + +.. clicmd:: vrrp (1-255) checksum-with-ipv4-pseudoheader + + Specify whether VRRPv3 checksum should involve IPv4 pseudoheader. This + command should not affect VRRPv2 and IPv6. Enabled by default. + +.. clicmd:: vrrp (1-255) priority (1-254) + + Set the router priority. The router with the highest priority is elected as + the Master. If all routers in the VRRP virtual router are configured with + the same priority, the router with the highest primary IP address is elected + as the Master. Priority value 255 is reserved for the acting Master router. + +.. clicmd:: vrrp (1-255) shutdown + + Place the router into administrative shutdown. VRRP will not activate for + this router until this command is removed with the ``no`` form. + +.. _vrrp-global-configuration: + +Global Configuration +-------------------- + +Show commands, global defaults and debugging configuration commands. + +.. clicmd:: show vrrp [interface INTERFACE] [(1-255)] [json] + + Shows VRRP status for some or all configured VRRP routers. Specifying an + interface will only show routers configured on that interface. Specifying a + VRID will only show routers with that VRID. Specifying ``json`` will dump + each router state in a JSON array. + +.. clicmd:: debug vrrp [{protocol|autoconfigure|packets|sockets|ndisc|arp|zebra}] + + Toggle debugging logs for VRRP components. + If no component is specified, debugging for all components are turned on/off. + + protocol + Logs state changes, election protocol decisions, and interface status + changes. + + autoconfigure + Logs actions taken by the autoconfiguration procedures. See + :ref:`vrrp-autoconfiguration`. + + packets + Logs details of ingress and egress packets. Includes packet decodes and + hex dumps. + + sockets + Logs details of socket configuration and initialization. + + ndisc + Logs actions taken by the Neighbor Discovery component of VRRP. + + arp + Logs actions taken by the ARP component of VRRP. + + zebra + Logs communications with Zebra. + +.. clicmd:: vrrp default <advertisement-interval (1-4096)|preempt|priority (1-254)|checksum-with-ipv4-pseudoheader|shutdown> + + Configure defaults for new VRRP routers. These values will not affect + already configured VRRP routers, but will be applied to newly configured + ones. + +.. _vrrp-autoconfiguration: + +Autoconfiguration +----------------- + +In light of the complicated configuration required on the base system before +VRRP can be enabled, FRR has the ability to automatically configure VRRP +sessions by inspecting the interfaces present on the system. Since it is quite +unlikely that macvlan devices with VRRP virtual MACs will exist on systems not +using VRRP, this can be a convenient shortcut to automatically generate FRR +configuration. + +After configuring the interfaces as described in +:ref:`vrrp-system-configuration`, and configuring any defaults you may want, +execute the following command: + +.. clicmd:: vrrp autoconfigure [version (2-3)] + + Generates VRRP configuration based on the interface configuration on the + base system. If the protocol version is not specified, the default is VRRPv3. + Any existing interfaces that are configured properly for VRRP - + i.e. have the correct MAC address, link local address (when required), IPv4 + and IPv6 addresses - are used to create a VRRP router on their parent + interfaces, with VRRP IPvX addresses taken from the addresses assigned to + the macvlan devices. The generated configuration appears in the output of + ``show run``, which can then be modified as needed and written to the config + file. The ``version`` parameter controls the protocol version; if using + VRRPv2, keep in mind that IPv6 is not supported and will not be configured. + +The following configuration is then generated for you: + +.. code-block:: frr + + interface eth0 + vrrp 5 + vrrp 5 ip 10.0.2.16 + vrrp 5 ipv6 2001:db8::370:7334 + + +VRRP is automatically activated. Global defaults, if set, are applied. + +You can then edit this configuration with **vtysh** as needed, and commit it by +writing to the configuration file. + + +Troubleshooting +--------------- + +My virtual routers are not seeing each others' advertisements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Check: + +- Is your kernel at least 5.1? +- Did you set the macvlan devices to ``bridge`` mode? +- If using IPv4 virtual addresses, does the parent of the macvlan devices have + an IPv4 address? +- If using IPv6 virtual addresses, is ``addrgenmode`` correctly set to + ``random`` and not the default ``eui64``? +- Is a firewall (``iptables``) or policy (``ip rule``) dropping multicast + traffic? +- Do you have unusual ``sysctls`` enabled that could affect the operation of + multicast traffic? +- Are you running in ESXi? See below. + + +My master router is not forwarding traffic +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There's several possible causes here. If you're sure your configuration is +otherwise correct, the following sysctl likely needs to be turned on: + +.. code-block:: console + + sysctl -w net.ipv4.conf.eth0.ignore_routes_with_linkdown=1 + +Without this setting, it's possible to create topologies in which virtual +routers holding mastership status will not forward traffic. + +Issue reference: https://github.com/FRRouting/frr/issues/7391 + + +My router is running in ESXi and VRRP isn't working +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +By default, ESXi traffic security settings don't allow traffic to egress a VNIC +that does not have the MAC address assigned to the VNIC. This breaks VRRP, +since virtual MACs are the basis of the protocol. + +On ESXi before 6.7, you need to enable Promiscuous Mode in the ESXi settings. +This is a significant security issue in some deployments so make sure you +understand what you're doing. On 6.7 and later, you can use the MAC Learning +feature instead, explained `here +<https://www.virtuallyghetto.com/2018/04/native-mac-learning-in-vsphere-6-7-removes-the-need-for-promiscuous-mode-for-nested-esxi.html>`_. + +Issue reference: https://github.com/FRRouting/frr/issues/5386 + + +My router cannot interoperate with branded routers / L3 switches +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +FRR includes a pseudoheader when calculating VRRPv3 checksums by default, +regardless of whether it's IPv4 or IPv6. + +Some vendors have different interpretations of `VRRPv3 RFC 5798 #5.2.8 +<https://www.rfc-editor.org/rfc/rfc5798.html#section-5.2.8>`_. In such cases, +their checksums are calculated with a pseudoheader only when it comes to IPv6. + +You need to disable ``checksum-with-ipv4-pseudoheader`` so that FRR computes and +accepts such checksums. + +Issue reference: https://github.com/FRRouting/frr/issues/9951 diff --git a/doc/user/vtysh.rst b/doc/user/vtysh.rst new file mode 100644 index 0000000..adbdf34 --- /dev/null +++ b/doc/user/vtysh.rst @@ -0,0 +1,226 @@ +.. _vty-shell: + +********* +VTY shell +********* + +.. program:: configure + +*vtysh* provides a combined frontend to all FRR daemons in a single combined +session. It is enabled by default at build time, but can be disabled through +the :option:`--disable-vtysh` option to the configure script. + +*vtysh* has a configuration file, :file:`vtysh.conf`. The location of that +file cannot be changed from |INSTALL_PREFIX_ETC| since it contains options +controlling authentication behavior. This file will also not be written by +configuration-save commands, it is intended to be updated manually by an +administrator with an external editor. + +.. warning:: + + This also means the ``hostname``, ``domainname``, and ``banner motd`` commands + (which do have effect for vtysh) need to be manually updated + in :file:`vtysh.conf`. + + +.. clicmd:: copy FILENAME running-config + + Process and load a configuration file manually; each line in the + file is read and processed as if it were being typed (or piped) to + vtysh. + + +Live logs +========= + +.. clicmd:: terminal monitor [DAEMON] + + Receive and display log messages. + + It is not currently possible to change the minimum message priority (fixed + to debug) or output formatting. These will likely be made configurable in + the future. + + Log messages are received asynchronously and may be printed both during + command execution as well as while on the prompt. They are printed to + stderr, unlike regular CLI output which is printed to stdout. The intent is + that stdin/stdout might be driven by some script while log messages are + visible on stderr. If stdout and stderr are the same file, the prompt and + pending input will be cleared and reprinted appropriately. + + .. note:: + + If ``vtysh`` cannot keep up, some log messages may be lost. The daemons + do **not** wait for, get blocked by, or buffer messages for ``vtysh``. + + +Pager usage +=========== + +*vtysh* can call an external paging program (e.g. *more* or *less*) to +paginate long output from commands. This feature used to be enabled by +default but is now controlled by the ``VTYSH_PAGER`` environment variable +and the :clicmd:`terminal paginate` command: + +.. envvar:: VTYSH_PAGER + + If set, the ``VTYSH_PAGER`` environment variable causes *vtysh* to pipe + output from commands through the given command. Note that this happens + regardless of the length of the output. As such, standard pager behavior + (particularly waiting at the end of output) tends to be annoying to the + user. Using ``less -EFX`` is recommended for a better user experience. + + If this environment variable is unset, *vtysh* defaults to not using any + pager. + + This variable should be set by the user according to their preferences, + in their :file:`~/.profile` file. + +.. clicmd:: terminal paginate + + Enables/disables vtysh output pagination. This command is intended to + be placed in :file:`vtysh.conf` to set a system-wide default. If this + is enabled but ``VTYSH_PAGER`` is not set, the system default pager + (likely ``more`` or ``/usr/bin/pager``) will be used. + + +Permissions and setup requirements +================================== + +*vtysh* connects to running daemons through Unix sockets located in +|INSTALL_PREFIX_STATE|. Running vtysh thus requires access to that directory, +plus membership in the |INSTALL_VTY_GROUP| group (which is the group that the +daemons will change ownership of their sockets to). + +To restrict access to FRR configuration, make sure no unauthorized users are +members of the |INSTALL_VTY_GROUP| group. + +.. warning:: + + VTYSH implements a CLI option ``-u, --user`` that disallows entering the + characters "en" on the command line, which ideally restricts access to + configuration commands. However, VTYSH was never designed to be a privilege + broker and is not built using secure coding practices. No guarantees of + security are provided for this option and under no circumstances should this + option be used to provide any semblance of security or read-only access to + FRR. + +PAM support (experimental) +-------------------------- + +vtysh has working (but rather useless) PAM support. It will perform an +"authenticate" PAM call using |PACKAGE_NAME| as service name. No other +(accounting, session, password change) calls will be performed by vtysh. + +Users using vtysh still need to have appropriate access to the daemons' VTY +sockets, usually by being member of the |INSTALL_VTY_GROUP| group. If they +have this membership, PAM support is useless since they can connect to daemons +and issue commands using some other tool. Alternatively, the *vtysh* binary +could be made SGID (set group ID) to the |INSTALL_VTY_GROUP| group. + +.. warning:: + + No security guarantees are made for this configuration. + + +.. clicmd:: username USERNAME nopassword + + If PAM support is enabled at build-time, this command allows disabling the + use of PAM on a per-user basis. If vtysh finds that an user is trying to + use vtysh and a "nopassword" entry is found, no calls to PAM will be made + at all. + + +.. _integrated-configuration-mode: + +Integrated configuration mode +============================= + +Integrated configuration mode uses a single configuration file, +:file:`frr.conf`, for all daemons. This replaces the individual files like +:file:`zebra.conf` or :file:`bgpd.conf`. + +:file:`frr.conf` is located in |INSTALL_PREFIX_ETC|. All daemons check for the +existence of this file at startup, and if it exists will not load their +individual configuration files. Instead, ``vtysh -b`` must be invoked to +process :file:`frr.conf` and apply its settings to the individual daemons. + +.. warning:: + + *vtysh -b* must also be executed after restarting any daemon. + + +Configuration saving, file ownership and permissions +---------------------------------------------------- + +The :file:`frr.conf` file is not written by any of the daemons; instead *vtysh* +contains the necessary logic to collect configuration from all of the daemons, +combine it and write it out. + +.. warning:: + + Daemons must be running for *vtysh* to be able to collect their + configuration. Any configuration from non-running daemons is permanently + lost after doing a configuration save. + +Since the *vtysh* command may be running as ordinary user on the system, +configuration writes will be tried through *watchfrr*, using the ``write +integrated`` command internally. Since *watchfrr* is running as superuser, +*vtysh* is able to ensure correct ownership and permissions on +:file:`frr.conf`. + +If *watchfrr* is not running or the configuration write fails, *vtysh* will +attempt to directly write to the file. This is likely to fail if running as +unprivileged user; alternatively it may leave the file with incorrect owner or +permissions. + +Writing the configuration can be triggered directly by invoking *vtysh -w*. +This may be useful for scripting. Note this command should be run as either the +superuser or the FRR user. + +We recommend you do not mix the use of the two types of files. + +.. clicmd:: service integrated-vtysh-config + + + Control whether integrated :file:`frr.conf` file is written when + 'write file' is issued. + + These commands need to be placed in :file:`vtysh.conf` to have any effect. + Note that since :file:`vtysh.conf` is not written by FRR itself, they + therefore need to be manually placed in that file. + + This command has 3 states: + + + service integrated-vtysh-config + *vtysh* will always write :file:`frr.conf`. + + + no service integrated-vtysh-config + *vtysh* will never write :file:`frr.conf`; instead it will ask + daemons to write their individual configuration files. + + Neither option present (default) + *vtysh* will check whether :file:`frr.conf` exists. If it does, + configuration writes will update that file. Otherwise, writes are performed + through the individual daemons. + + This command is primarily intended for packaging/distribution purposes, to + preset one of the two operating modes and ensure consistent operation across + installations. + +.. clicmd:: write integrated + + Unconditionally (regardless of ``service integrated-vtysh-config`` setting) + write out integrated :file:`frr.conf` file through *watchfrr*. If *watchfrr* + is not running, this command is unavailable. + +.. warning:: + + Configuration changes made while some daemon is not running will be + invisible to that daemon. The daemon will start up with its saved + configuration (either in its individual configuration file, or in + :file:`frr.conf`). This is particularly troublesome for route-maps and + prefix lists, which would otherwise be synchronized between daemons. + diff --git a/doc/user/watchfrr.rst b/doc/user/watchfrr.rst new file mode 100644 index 0000000..3a19e3c --- /dev/null +++ b/doc/user/watchfrr.rst @@ -0,0 +1,28 @@ +.. _watchfrr: + +******** +WATCHFRR +******** + +:abbr:`WATCHFRR` is a daemon that handles failed daemon processes and +intelligently restarts them as needed. + +Starting WATCHFRR +================= + +WATCHFRR is started as per normal systemd startup and typically does not +require end users management. + +WATCHFRR commands +================= + +.. clicmd:: show watchfrr + + Give status information about the state of the different daemons being + watched by WATCHFRR + +.. clicmd:: watchfrr ignore DAEMON + + Tell WATCHFRR to ignore a particular DAEMON if it goes unresponsive. + This is particularly useful when you are a developer and need to debug + a working system, without watchfrr pulling the rug out from under you. diff --git a/doc/user/wecmp_linkbw.rst b/doc/user/wecmp_linkbw.rst new file mode 100644 index 0000000..4df6559 --- /dev/null +++ b/doc/user/wecmp_linkbw.rst @@ -0,0 +1,297 @@ +.. _wecmp_linkbw: + +Weighted ECMP using BGP link bandwidth +====================================== + +.. _features-of-wecmp-linkbw: + +Overview +-------- + +In normal equal cost multipath (ECMP), the route to a destination has +multiple next hops and traffic is expected to be equally distributed +across these next hops. In practice, flow-based hashing is used so that +all traffic associated with a particular flow uses the same next hop, +and by extension, the same path across the network. + +Weighted ECMP using BGP link bandwidth introduces support for network-wide +unequal cost multipathing (UCMP) to an IP destination. The unequal cost +load balancing is implemented by the forwarding plane based on the weights +associated with the next hops of the IP prefix. These weights are computed +based on the bandwidths of the corresponding multipaths which are encoded +in the ``BGP link bandwidth extended community`` as specified in +[Draft-IETF-idr-link-bandwidth]_. Exchange of an appropriate BGP link +bandwidth value for a prefix across the network results in network-wide +unequal cost multipathing. + +One of the primary use cases of this capability is in the data center when +a service (represented by its anycast IP) has an unequal set of resources +across the regions (e.g., PODs) of the data center and the network itself +provides the load balancing function instead of an external load balancer. +Refer to [Draft-IETF-mohanty-bess-ebgp-dmz]_ and :rfc:`7938` for details +on this use case. This use case is applicable in a pure L3 network as +well as in a EVPN network. + +The traditional use case for BGP link bandwidth to load balance traffic +to the exit routers in the AS based on the bandwidth of their external +eBGP peering links is also supported. + + +Design Principles +----------------- + +Next hop weight computation and usage +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +As described, in UCMP, there is a weight associated with each next hop of an +IP prefix, and traffic is expected to be distributed across the next hops in +proportion to their weight. The weight of a next hop is a simple factoring +of the bandwidth of the corresponding path against the total bandwidth of +all multipaths, mapped to the range 1 to 100. What happens if not all the +paths in the multipath set have link bandwidth associated with them? In such +a case, in adherence to [Draft-IETF-idr-link-bandwidth]_, the behavior +reverts to standard ECMP among all the multipaths, with the link bandwidth +being effectively ignored. + +Note that there is no change to either the BGP best path selection algorithm +or to the multipath computation algorithm; the mapping of link bandwidth to +weight happens at the time of installation of the route in the RIB. + +If data forwarding is implemented by means of the Linux kernel, the next hop’s +weight is used in the hash calculation. The kernel uses the Hash threshold +algorithm and use of the next hop weight is built into it; next hops need +not be expanded to achieve UCMP. UCMP for IPv4 is available in older Linux +kernels too, while UCMP for IPv6 is available from the 4.16 kernel onwards. + +If data forwarding is realized in hardware, common implementations expand +the next hops (i.e., they are repeated) in the ECMP container in proportion +to their weight. For example, if the weights associated with 3 next hops for +a particular route are 50, 25 and 25 and the ECMP container has a size of 16 +next hops, the first next hop will be repeated 8 times and the other 2 next +hops repeated 4 times each. Other implementations are also possible. + +Unequal cost multipath across a network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For the use cases listed above, it is not sufficient to support UCMP on just +one router (e.g., egress router), or individually, on multiple routers; UCMP +must be deployed across the entire network. This is achieved by employing the +BGP link-bandwidth extended community. + +At the router which originates the BGP link bandwidth, there has to be user +configuration to trigger it, which is described below. Receiving routers +would use the received link bandwidth from their downstream routers to +determine the next hop weight as described in the earlier section. Further, +if the received link bandwidth is a transitive attribute, it would be +propagated to eBGP peers, with the additional change that if the next hop +is set to oneself, the cumulative link bandwidth of all downstream paths +is propagated to other routers. In this manner, the entire network will +know how to distribute traffic to an anycast service across the network. + +The BGP link-bandwidth extended community is encoded in bytes-per-second. +In the use case where UCMP must be based on the number of paths, a reference +bandwidth of 1 Mbps is used. So, for example, if there are 4 equal cost paths +to an anycast IP, the encoded bandwidth in the extended community will be +500,000. The actual value itself doesn’t matter as long as all routers +originating the link-bandwidth are doing it in the same way. + + +Configuration Guide +------------------- + +The configuration for weighted ECMP using BGP link bandwidth requires +one essential step - using a route-map to inject the link bandwidth +extended community. An additional option is provided to control the +processing of received link bandwidth. + +Injecting link bandwidth into the network +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +At the "entry point" router that is injecting the prefix to which weighted +load balancing must be performed, a route-map must be configured to +attach the link bandwidth extended community. + +For the use case of providing weighted load balancing for an anycast service, +this configuration will typically need to be applied at the TOR or Leaf +router that is connected to servers which provide the anycast service and +the bandwidth would be based on the number of multipaths for the destination. + +For the use case of load balancing to the exit router, the exit router should +be configured with the route map specifying the a bandwidth value that +corresponds to the bandwidth of the link connecting to its eBGP peer in the +adjoining AS. In addition, the link bandwidth extended community must be +explicitly configured to be non-transitive. + +The complete syntax of the route-map set command can be found at +:ref:`bgp-extended-communities-in-route-map` + +This route-map is supported only at two attachment points: +(a) the outbound route-map attached to a peer or peer-group, per address-family +(b) the EVPN advertise route-map used to inject IPv4 or IPv6 unicast routes +into EVPN as type-5 routes. + +Since the link bandwidth origination is done by using a route-map, it can +be constrained to certain prefixes (e.g., only for anycast services) or it +can be generated for all prefixes. Further, when the route-map is used in +the neighbor context, the link bandwidth usage can be constrained to certain +peers only. + +A sample configuration is shown below and illustrates link bandwidth +advertisement towards the "SPINE" peer-group for anycast IPs in the +range 192.168.x.x + +.. code-block:: frr + + ip prefix-list anycast_ip seq 10 permit 192.168.0.0/16 le 32 + route-map anycast_ip permit 10 + match ip address prefix-list anycast_ip + set extcommunity bandwidth num-multipaths + route-map anycast_ip permit 20 + ! + router bgp 65001 + neighbor SPINE peer-group + neighbor SPINE remote-as external + neighbor 172.16.35.1 peer-group SPINE + neighbor 172.16.36.1 peer-group SPINE + ! + address-family ipv4 unicast + network 110.0.0.1/32 + network 192.168.44.1/32 + neighbor SPINE route-map anycast_ip out + exit-address-family + ! + + +Controlling link bandwidth processing on the receiver +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There is no configuration necessary to process received link bandwidth and +translate it into the weight associated with the corresponding next hop; +that happens by default. If some of the multipaths do not have the link +bandwidth extended community, the default behavior is to revert to normal +ECMP as recommended in [Draft-IETF-idr-link-bandwidth]_. + +The operator can change these behaviors with the following configuration: + +.. clicmd:: bgp bestpath bandwidth <ignore | skip-missing | default-weight-for-missing> + +The different options imply behavior as follows: + +- ignore: Ignore link bandwidth completely for route installation + (i.e., do regular ECMP, not weighted) +- skip-missing: Skip paths without link bandwidth and do UCMP among + the others (if at least some paths have link-bandwidth) +- default-weight-for-missing: Assign a low default weight (value 1) + to paths not having link bandwidth + +This configuration is per BGP instance similar to other BGP route-selection +controls; it operates on both IPv4-unicast and IPv6-unicast routes in that +instance. In an EVPN network, this configuration (if required) should be +implemented in the tenant VRF and is again applicable for IPv4-unicast and +IPv6-unicast, including the ones sourced from EVPN type-5 routes. + +A sample snippet of FRR configuration on a receiver to skip paths without +link bandwidth and do weighted ECMP among the other paths (if some of them +have link bandwidth) is as shown below. + +.. code-block:: frr + + router bgp 65021 + bgp bestpath as-path multipath-relax + bgp bestpath bandwidth skip-missing + neighbor LEAF peer-group + neighbor LEAF remote-as external + neighbor 172.16.35.2 peer-group LEAF + neighbor 172.16.36.2 peer-group LEAF + ! + address-family ipv4 unicast + network 130.0.0.1/32 + exit-address-family + ! + + +Stopping the propagation of the link bandwidth outside a domain +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The link bandwidth extended community will get automatically propagated +with the prefix to EBGP peers, if it is encoded as a transitive attribute +by the originator. If this propagation has to be stopped outside of a +particular domain (e.g., stopped from being propagated to routers outside +of the data center core network), the mechanism available is to disable +the advertisement of all BGP extended communities on the specific peering/s. +In other words, the propagation cannot be blocked just for the link bandwidth +extended community. The configuration to disable all extended communities +can be applied to a peer or peer-group (per address-family). + +Of course, the other common way to stop the propagation of the link bandwidth +outside the domain is to block the prefixes themselves from being advertised +and possibly, announce only an aggregate route. This would be quite common +in a EVPN network. + +BGP link bandwidth and UCMP monitoring & troubleshooting +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Existing operational commands to display the BGP routing table for a specific +prefix will show the link bandwidth extended community also, if present. + +An example of an IPv4-unicast route received with the link bandwidth +attribute from two peers is shown below: + +.. code-block:: frr + + CLI# show bgp ipv4 unicast 192.168.10.1/32 + BGP routing table entry for 192.168.10.1/32 + Paths: (2 available, best #2, table default) + Advertised to non peer-group peers: + l1(swp1) l2(swp2) l3(swp3) l4(swp4) + 65002 + fe80::202:ff:fe00:1b from l2(swp2) (110.0.0.2) + (fe80::202:ff:fe00:1b) (used) + Origin IGP, metric 0, valid, external, multipath, bestpath-from-AS 65002 + Extended Community: LB:65002:125000000 (1000.000 Mbps) + Last update: Thu Feb 20 18:34:16 2020 + + 65001 + fe80::202:ff:fe00:15 from l1(swp1) (110.0.0.1) + (fe80::202:ff:fe00:15) (used) + Origin IGP, metric 0, valid, external, multipath, bestpath-from-AS 65001, best (Older Path) + Extended Community: LB:65001:62500000 (500.000 Mbps) + Last update: Thu Feb 20 18:22:34 2020 + +The weights associated with the next hops of a route can be seen by querying +the RIB for a specific route. + +For example, the next hop weights corresponding to the link bandwidths in the +above example is illustrated below: + +.. code-block:: frr + + spine1# show ip route 192.168.10.1/32 + Routing entry for 192.168.10.1/32 + Known via "bgp", distance 20, metric 0, best + Last update 00:00:32 ago + * fe80::202:ff:fe00:1b, via swp2, weight 66 + * fe80::202:ff:fe00:15, via swp1, weight 33 + +For troubleshooting, existing debug logs ``debug bgp updates``, +``debug bgp bestpath <prefix>``, ``debug bgp zebra`` and +``debug zebra kernel`` can be used. + +A debug log snippet when ``debug bgp zebra`` is enabled and a route is +installed by BGP in the RIB with next hop weights is shown below: + +.. code-block:: frr + + 2020-02-29T06:26:19.927754+00:00 leaf1 bgpd[5459]: bgp_zebra_announce: p=192.168.150.1/32, bgp_is_valid_label: 0 + 2020-02-29T06:26:19.928096+00:00 leaf1 bgpd[5459]: Tx route add VRF 33 192.168.150.1/32 metric 0 tag 0 count 2 + 2020-02-29T06:26:19.928289+00:00 leaf1 bgpd[5459]: nhop [1]: 110.0.0.6 if 35 VRF 33 wt 50 RMAC 0a:11:2f:7d:35:20 + 2020-02-29T06:26:19.928479+00:00 leaf1 bgpd[5459]: nhop [2]: 110.0.0.5 if 35 VRF 33 wt 50 RMAC 32:1e:32:a3:6c:bf + 2020-02-29T06:26:19.928668+00:00 leaf1 bgpd[5459]: bgp_zebra_announce: 192.168.150.1/32: announcing to zebra (recursion NOT set) + + +References +---------- + +.. [Draft-IETF-idr-link-bandwidth] <https://tools.ietf.org/html/draft-ietf-idr-link-bandwidth> +.. [Draft-IETF-mohanty-bess-ebgp-dmz] <https://tools.ietf.org/html/draft-mohanty-bess-ebgp-dmz> + diff --git a/doc/user/zebra.rst b/doc/user/zebra.rst new file mode 100644 index 0000000..ba6e3bf --- /dev/null +++ b/doc/user/zebra.rst @@ -0,0 +1,1895 @@ +.. _zebra: + +***** +Zebra +***** + +*zebra* is an IP routing manager. It provides kernel routing +table updates, interface lookups, and redistribution of routes between +different routing protocols. + +.. _invoking-zebra: + +Invoking zebra +============== + +Besides the common invocation options (:ref:`common-invocation-options`), the +*zebra* specific invocation options are listed below. + +.. program:: zebra + +.. option:: -b, --batch + + Runs in batch mode. *zebra* parses configuration file and terminates + immediately. + +.. option:: -K TIME, --graceful_restart TIME + + If this option is specified, the graceful restart time is TIME seconds. + Zebra, when started, will read in routes. Those routes that Zebra + identifies that it was the originator of will be swept in TIME seconds. + If no time is specified then we will sweep those routes immediately. + Under the \*BSD's, there is no way to properly store the originating + route and the route types in this case will show up as a static route + with an admin distance of 255. + +.. option:: -r, --retain + + When program terminates, do not flush routes installed by *zebra* from the + kernel. + +.. option:: -e X, --ecmp X + + Run zebra with a limited ecmp ability compared to what it is compiled to. + If you are running zebra on hardware limited functionality you can + force zebra to limit the maximum ecmp allowed to X. This number + is bounded by what you compiled FRR with as the maximum number. + +.. option:: -n, --vrfwnetns + + When *Zebra* starts with this option, the VRF backend is based on Linux + network namespaces. That implies that all network namespaces discovered by + ZEBRA will create an associated VRF. The other daemons will operate on the VRF + VRF defined by *Zebra*, as usual. + + .. seealso:: :ref:`zebra-vrf` + +.. option:: -z <path_to_socket>, --socket <path_to_socket> + + If this option is supplied on the cli, the path to the zebra + control socket(zapi), is used. This option overrides a -N <namespace> + option if handed to it on the cli. + +.. option:: --v6-rr-semantics + + The linux kernel is receiving the ability to use the same route + replacement semantics for v6 that v4 uses. If you are using a + kernel that supports this functionality then run *Zebra* with this + option and we will use Route Replace Semantics instead of delete + than add. + +.. option:: --routing-table <tableno> + + Specify which kernel routing table *Zebra* should communicate with. + If this option is not specified the default table (RT_TABLE_MAIN) is + used. + +.. option:: --asic-offload=[notify_on_offload|notify_on_ack] + + The linux kernel has the ability to use asic-offload ( see switchdev + development ). When the operator knows that FRR will be working in + this way, allow them to specify this with FRR. At this point this + code only supports asynchronous notification of the offload state. + In other words the initial ACK received for linux kernel installation + does not give zebra any data about what the state of the offload + is. This option takes the optional parameters notify_on_offload + or notify_on_ack. This signals to zebra to notify upper level + protocols about route installation/update on ack received from + the linux kernel or from offload notification. + + +.. option:: -s <SIZE>, --nl-bufsize <SIZE> + + Allow zebra to modify the default receive buffer size to SIZE + in bytes. Under \*BSD only the -s option is available. + +.. option:: --v6-with-v4-nexthops + + Signal to zebra that v6 routes with v4 nexthops are accepted + by the underlying dataplane. This will be communicated to + the upper level daemons that can install v6 routes with v4 + nexthops. + +.. _interface-commands: + +Configuration Addresses behaviour +================================= + +At startup, *Zebra* will first discover the underlying networking objects +from the operating system. This includes interfaces, addresses of +interfaces, static routes, etc. Then, it will read the configuration +file, including its own interface addresses, static routes, etc. All this +information comprises the operational context from *Zebra*. But +configuration context from *Zebra* will remain the same as the one from +:file:`zebra.conf` config file. As an example, executing the following +:clicmd:`show running-config` will reflect what was in :file:`zebra.conf`. +In a similar way, networking objects that are configured outside of the +*Zebra* like *iproute2* will not impact the configuration context from +*Zebra*. This behaviour permits you to continue saving your own config +file, and decide what is really to be pushed on the config file, and what +is dependent on the underlying system. +Note that inversely, from *Zebra*, you will not be able to delete networking +objects that were previously configured outside of *Zebra*. + + +Interface Commands +================== + +.. _standard-commands: + +Standard Commands +----------------- + + +.. clicmd:: interface IFNAME + + +.. clicmd:: interface IFNAME vrf VRF + + +.. clicmd:: shutdown + + + Up or down the current interface. + + +.. clicmd:: ip address ADDRESS/PREFIX + +.. clicmd:: ipv6 address ADDRESS/PREFIX + + + + Set the IPv4 or IPv6 address/prefix for the interface. + + +.. clicmd:: ip address LOCAL-ADDR peer PEER-ADDR/PREFIX + + + Configure an IPv4 Point-to-Point address on the interface. (The concept of + PtP addressing does not exist for IPv6.) + + ``local-addr`` has no subnet mask since the local side in PtP addressing is + always a single (/32) address. ``peer-addr/prefix`` can be an arbitrary subnet + behind the other end of the link (or even on the link in Point-to-Multipoint + setups), though generally /32s are used. + + +.. clicmd:: description DESCRIPTION ... + + Set description for the interface. + + +.. clicmd:: mpls <enable|disable> + + Choose mpls kernel processing value on the interface, for linux. Interfaces + configured with mpls will not automatically turn on if mpls kernel modules do not + happen to be loaded. This command will fail on 3.X linux kernels and does not + work on non-linux systems at all. 'enable' and 'disable' will respectively turn + on and off mpls on the given interface. + +.. clicmd:: multicast + + + Enable or disable multicast flag for the interface. + + +.. clicmd:: bandwidth (1-10000000) + + + Set bandwidth value of the interface in kilobits/sec. This is for + calculating OSPF cost. This command does not affect the actual device + configuration. + + +.. clicmd:: link-detect + + + Enable or disable link-detect on platforms which support this. Currently only + Linux, and only where network interface drivers support reporting + link-state via the ``IFF_RUNNING`` flag. + + In FRR, link-detect is on by default. + +.. _link-parameters-commands: + +Link Parameters Commands +------------------------ + +.. note:: + + At this time, FRR offers partial support for some of the routing + protocol extensions that can be used with MPLS-TE. FRR does not + support a complete RSVP-TE solution currently. + +.. clicmd:: link-params + + + Enter into the link parameters sub node. At least 'enable' must be + set to activate the link parameters, and consequently routing + information that could be used as part of Traffic Engineering on + this interface. MPLS-TE must be enable at the OSPF + (:ref:`ospf-traffic-engineering`) or ISIS + (:ref:`isis-traffic-engineering`) router level in complement to + this. + + Under link parameter statement, the following commands set the different TE values: + +.. clicmd:: enable + + Enable link parameters for this interface. + +.. clicmd:: metric (0-4294967295) + +.. clicmd:: max-bw BANDWIDTH + +.. clicmd:: max-rsv-bw BANDWIDTH + +.. clicmd:: unrsv-bw (0-7) BANDWIDTH + + These commands specifies the Traffic Engineering parameters of the interface + in conformity to RFC3630 (OSPF) or RFC5305 (ISIS). There are respectively + the TE Metric (different from the OSPF or ISIS metric), Maximum Bandwidth + (interface speed by default), Maximum Reservable Bandwidth, Unreserved + Bandwidth for each 0-7 priority and Admin Group (ISIS) or Resource + Class/Color (OSPF). + + Note that BANDWIDTH is specified in IEEE floating point format and express + in Bytes/second. + +.. clicmd:: admin-grp 0x(0-FFFFFFFF) + + This commands configures the Traffic Engineering Admin-Group of the interface + as specified in RFC3630 (OSPF) or RFC5305 (ISIS). Admin-group is also known + as Resource Class/Color in the OSPF protocol. + +.. clicmd:: [no] affinity AFFINITY-MAP-NAME + + This commands configures the Traffic Engineering Admin-Group of the + interface using the affinity-map definitions (:ref:`affinity-map`). + Multiple AFFINITY-MAP-NAME can be specified at the same time. Affinity-map + names are added or removed if ``no`` is present. It means that specifying one + value does not override the full list. + + ``admin-grp`` and ``affinity`` commands provide two ways of setting + admin-groups. They cannot be both set on the same interface. + +.. clicmd:: [no] affinity-mode [extended|standard|both] + + This commands configures which admin-group format is set by the affinity + command. ``extended`` Admin-Group is the default and uses the RFC7308 format. + ``standard`` mode uses the standard admin-group format that is defined by + RFC3630, RFC5305 and RFC5329. When the ``standard`` mode is set, + affinity-maps with bit-positions higher than 31 cannot be applied to the + interface. The ``both`` mode allows setting standard and extended admin-group + on the link at the same time. In this case, the bit-positions 0 to 31 are + the same on standard and extended admin-groups. + + Note that extended admin-groups are only supported by IS-IS for the moment. + +.. clicmd:: delay (0-16777215) [min (0-16777215) | max (0-16777215)] + +.. clicmd:: delay-variation (0-16777215) + +.. clicmd:: packet-loss PERCENTAGE + +.. clicmd:: res-bw BANDWIDTH + +.. clicmd:: ava-bw BANDWIDTH + +.. clicmd:: use-bw BANDWIDTH + + These command specifies additional Traffic Engineering parameters of the + interface in conformity to draft-ietf-ospf-te-metrics-extension-05.txt and + draft-ietf-isis-te-metrics-extension-03.txt. There are respectively the + delay, jitter, loss, available bandwidth, reservable bandwidth and utilized + bandwidth. + + Note that BANDWIDTH is specified in IEEE floating point format and express + in Bytes/second. Delays and delay variation are express in micro-second + (µs). Loss is specified in PERCENTAGE ranging from 0 to 50.331642% by step + of 0.000003. + +.. clicmd:: neighbor <A.B.C.D> as (0-65535) + + Specifies the remote ASBR IP address and Autonomous System (AS) number + for InterASv2 link in OSPF (RFC5392). Note that this option is not yet + supported for ISIS (RFC5316). + +Global Commands +------------------------ + +.. clicmd:: zebra protodown reason-bit (0-31) + + This command is only supported for linux and a kernel > 5.1. + Change reason-bit frr uses for setting protodown. We default to 7, but + if another userspace app ever conflicts with this, you can change it here. + The descriptor for this bit should exist in :file:`/etc/iproute2/protodown_reasons.d/` + to display with :clicmd:`ip -d link show`. + +Nexthop Tracking +================ + +Nexthop tracking doesn't resolve nexthops via the default route by default. +Allowing this might be useful when e.g. you want to allow BGP to peer across +the default route. + +.. clicmd:: zebra nexthop-group keep (1-3600) + + Set the time that zebra will keep a created and installed nexthop group + before removing it from the system if the nexthop group is no longer + being used. The default time is 180 seconds. + +.. clicmd:: ip nht resolve-via-default + + Allow IPv4 nexthop tracking to resolve via the default route. This parameter + is configured per-VRF, so the command is also available in the VRF subnode. + + This is enabled by default for a traditional profile. + +.. clicmd:: ipv6 nht resolve-via-default + + Allow IPv6 nexthop tracking to resolve via the default route. This parameter + is configured per-VRF, so the command is also available in the VRF subnode. + + This is enabled by default for a traditional profile. + +.. clicmd:: show ip nht [vrf NAME] [A.B.C.D|X:X::X:X] [mrib] [json] + + Show nexthop tracking status for address resolution. If vrf is not specified + then display the default vrf. If ``all`` is specified show all vrf address + resolution output. If an ipv4 or ipv6 address is not specified then display + all addresses tracked, else display the requested address. The mrib keyword + indicates that the operator wants to see the multicast rib address resolution + table. An alternative form of the command is ``show ip import-check`` and this + form of the command is deprecated at this point in time. + User can get that information as JSON string when ``json`` key word + at the end of cli is presented. + +.. clicmd:: show ip nht route-map [vrf <NAME|all>] [json] + + This command displays route-map attach point to nexthop tracking and + displays list of protocol with its applied route-map. + When zebra considers sending NHT resoultion, the nofification only + sent to appropriate client protocol only after applying route-map filter. + User can get that information as JSON format when ``json`` keyword + at the end of cli is presented. + +PBR dataplane programming +========================= + +Some dataplanes require the PBR nexthop to be resolved into a SMAC, DMAC and +outgoing interface + +.. clicmd:: pbr nexthop-resolve + + Resolve PBR nexthop via ip neigh tracking + +.. _administrative-distance: + +Administrative Distance +======================= + +Administrative distance allows FRR to make decisions about what routes +should be installed in the rib based upon the originating protocol. +The lowest Admin Distance is the route selected. This is purely a +subjective decision about ordering and care has been taken to choose +the same distances that other routing suites have chosen. + ++------------+-----------+ +| Protocol | Distance | ++------------+-----------+ +| System | 0 | ++------------+-----------+ +| Kernel | 0 | ++------------+-----------+ +| Connect | 0 | ++------------+-----------+ +| Static | 1 | ++------------+-----------+ +| NHRP | 10 | ++------------+-----------+ +| EBGP | 20 | ++------------+-----------+ +| EIGRP | 90 | ++------------+-----------+ +| BABEL | 100 | ++------------+-----------+ +| OSPF | 110 | ++------------+-----------+ +| ISIS | 115 | ++------------+-----------+ +| OPENFABRIC | 115 | ++------------+-----------+ +| RIP | 120 | ++------------+-----------+ +| Table | 150 | ++------------+-----------+ +| SHARP | 150 | ++------------+-----------+ +| IBGP | 200 | ++------------+-----------+ +| PBR | 200 | ++------------+-----------+ + +An admin distance of 255 indicates to Zebra that the route should not be +installed into the Data Plane. Additionally routes with an admin distance +of 255 will not be redistributed. + +Zebra does treat Kernel routes as special case for the purposes of Admin +Distance. Upon learning about a route that is not originated by FRR +we read the metric value as a uint32_t. The top byte of the value +is interpreted as the Administrative Distance and the low three bytes +are read in as the metric. This special case is to facilitate VRF +default routes. + +.. code-block:: shell + + $ # Set administrative distance to 255 for Zebra + $ ip route add 192.0.2.0/24 metric $(( 2**32 - 2**24 )) dev lo + $ vtysh -c 'show ip route 192.0.2.0/24 json' | jq '."192.0.2.0/24"[] | (.distance, .metric)' + 255 + 0 + $ # Set administrative distance to 192 for Zebra + $ ip route add 192.0.2.0/24 metric $(( 2**31 + 2**30 )) dev lo + $ vtysh -c 'show ip route 192.0.2.0/24 json' | jq '."192.0.2.0/24"[] | (.distance, .metric)' + 192 + 0 + $ # Set administrative distance to 128, and metric 100 for Zebra + $ ip route add 192.0.2.0/24 metric $(( 2**31 + 100 )) dev lo + $ vtysh -c 'show ip route 192.0.2.0/24 json' | jq '."192.0.2.0/24"[] | (.distance, .metric)' + 128 + 100 + +Route Replace Semantics +======================= + +When using the Linux Kernel as a forwarding plane, routes are installed +with a metric of 20 to the kernel. Please note that the kernel's metric +value bears no resemblence to FRR's RIB metric or admin distance. It +merely is a way for the Linux Kernel to decide which route to use if it +has multiple routes for the same prefix from multiple sources. An example +here would be if someone else was running another routing suite besides +FRR at the same time, the kernel must choose what route to use to forward +on. FRR choose the value of 20 because of two reasons. FRR wanted a +value small enough to be chosen but large enough that the operator could +allow route prioritization by the kernel when multiple routing suites are +being run and FRR wanted to take advantage of Route Replace semantics that +the linux kernel offers. In order for Route Replacement semantics to +work FRR must use the same metric when issuing the replace command. +Currently FRR only supports Route Replace semantics using the Linux +Kernel. + +.. _zebra-vrf: + +Virtual Routing and Forwarding +============================== + +FRR supports :abbr:`VRF (Virtual Routing and Forwarding)`. VRF is a way to +separate networking contexts on the same machine. Those networking contexts are +associated with separate interfaces, thus making it possible to associate one +interface with a specific VRF. + +VRF can be used, for example, when instantiating per enterprise networking +services, without having to instantiate the physical host machine or the +routing management daemons for each enterprise. As a result, interfaces are +separate for each set of VRF, and routing daemons can have their own context +for each VRF. + +This conceptual view introduces the *Default VRF* case. If the user does not +configure any specific VRF, then by default, FRR uses the *Default VRF*. The +name "default" is used to refer to this VRF in various CLI commands and YANG +models. It is possible to change that name by passing the ``-o`` option to all +daemons, for example, one can use ``-o vrf0`` to change the name to "vrf0". +The easiest way to pass the same option to all daemons is to use the +``frr_global_options`` variable in the +:ref:`Daemons Configuration File <daemons-configuration-file>`. + +Configuring VRF networking contexts can be done in various ways on FRR. The VRF +interfaces can be configured by entering in interface configuration mode +:clicmd:`interface IFNAME vrf VRF`. + +A VRF backend mode is chosen when running *Zebra*. + +If no option is chosen, then the *Linux VRF* implementation as references in +https://www.kernel.org/doc/Documentation/networking/vrf.txt will be mapped over +the *Zebra* VRF. The routing table associated to that VRF is a Linux table +identifier located in the same *Linux network namespace* where *Zebra* started. +Please note when using the *Linux VRF* routing table it is expected that a +default Kernel route will be installed that has a metric as outlined in the +www.kernel.org doc above. The Linux Kernel does table lookup via a combination +of rule application of the rule table and then route lookup of the specified +table. If no route match is found then the next applicable rule is applied +to find the next route table to use to look for a route match. As such if +your VRF table does not have a default blackhole route with a high metric +VRF route lookup will leave the table specified by the VRF, which is undesirable. + +If the :option:`-n` option is chosen, then the *Linux network namespace* will +be mapped over the *Zebra* VRF. That implies that *Zebra* is able to configure +several *Linux network namespaces*. The routing table associated to that VRF +is the whole routing tables located in that namespace. For instance, this mode +matches OpenStack Network Namespaces. It matches also OpenFastPath. The default +behavior remains Linux VRF which is supported by the Linux kernel community, +see https://www.kernel.org/doc/Documentation/networking/vrf.txt. + +Because of that difference, there are some subtle differences when running some +commands in relationship to VRF. Here is an extract of some of those commands: + +.. clicmd:: vrf VRF + + This command is available on configuration mode. By default, above command + permits accessing the VRF configuration mode. This mode is available for + both VRFs. It is to be noted that *Zebra* does not create Linux VRF. + The network administrator can however decide to provision this command in + configuration file to provide more clarity about the intended configuration. + +.. clicmd:: netns NAMESPACE + + This command is based on VRF configuration mode. This command is available + when *Zebra* is run in :option:`-n` mode. This command reflects which *Linux + network namespace* is to be mapped with *Zebra* VRF. It is to be noted that + *Zebra* creates and detects added/suppressed VRFs from the Linux environment + (in fact, those managed with iproute2). The network administrator can however + decide to provision this command in configuration file to provide more clarity + about the intended configuration. + +.. clicmd:: show ip route vrf VRF + + The show command permits dumping the routing table associated to the VRF. If + *Zebra* is launched with default settings, this will be the ``TABLENO`` of + the VRF configured on the kernel, thanks to information provided in + https://www.kernel.org/doc/Documentation/networking/vrf.txt. If *Zebra* is + launched with :option:`-n` option, this will be the default routing table of + the *Linux network namespace* ``VRF``. + +.. clicmd:: show ip route vrf VRF table TABLENO + + The show command is only available with :option:`-n` option. This command + will dump the routing table ``TABLENO`` of the *Linux network namespace* + ``VRF``. + +.. clicmd:: show ip route vrf VRF tables + + This command will dump the routing tables within the vrf scope. If ``vrf all`` + is executed, all routing tables will be dumped. + +.. clicmd:: show <ip|ipv6> route summary [vrf VRF] [table TABLENO] [prefix] + + This command will dump a summary output of the specified VRF and TABLENO + combination. If neither VRF or TABLENO is specified FRR defaults to + the default vrf and default table. If prefix is specified dump the + number of prefix routes. + +.. _zebra-table-allocation: + +Table Allocation +================ + +Some services like BGP flowspec allocate routing tables to perform policy +routing based on netfilter criteria and IP rules. In order to avoid +conflicts between VRF allocated routing tables and those services, Zebra +proposes to define a chunk of routing tables to use by other services. + +Allocation configuration can be done like below, with the range of the +chunk of routing tables to be used by the given service. + +.. clicmd:: ip table range <STARTTABLENO> <ENDTABLENO> + +.. _zebra-ecmp: + +ECMP +==== + +FRR supports ECMP as part of normal operations and is generally compiled +with a limit of 64 way ECMP. This of course can be modified via configure +options on compilation if the end operator desires to do so. Individual +protocols each have their own way of dictating ECMP policy and their +respective documentation should be read. + +ECMP can be inspected in zebra by doing a ``show ip route X`` command. + +.. code-block:: shell + + eva# show ip route 4.4.4.4/32 + Codes: K - kernel route, C - connected, S - static, R - RIP, + O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, + T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, + F - PBR, f - OpenFabric, + > - selected route, * - FIB route, q - queued, r - rejected, b - backup + t - trapped, o - offload failure + + D>* 4.4.4.4/32 [150/0] via 192.168.161.1, enp39s0, weight 1, 00:00:02 + * via 192.168.161.2, enp39s0, weight 1, 00:00:02 + * via 192.168.161.3, enp39s0, weight 1, 00:00:02 + * via 192.168.161.4, enp39s0, weight 1, 00:00:02 + * via 192.168.161.5, enp39s0, weight 1, 00:00:02 + * via 192.168.161.6, enp39s0, weight 1, 00:00:02 + * via 192.168.161.7, enp39s0, weight 1, 00:00:02 + * via 192.168.161.8, enp39s0, weight 1, 00:00:02 + * via 192.168.161.9, enp39s0, weight 1, 00:00:02 + * via 192.168.161.10, enp39s0, weight 1, 00:00:02 + * via 192.168.161.11, enp39s0, weight 1, 00:00:02 + * via 192.168.161.12, enp39s0, weight 1, 00:00:02 + * via 192.168.161.13, enp39s0, weight 1, 00:00:02 + * via 192.168.161.14, enp39s0, weight 1, 00:00:02 + * via 192.168.161.15, enp39s0, weight 1, 00:00:02 + * via 192.168.161.16, enp39s0, weight 1, 00:00:02 + +In this example we have 16 way ecmp for the 4.4.4.4/32 route. The ``*`` character +tells us that the route is installed in the Data Plane, or FIB. + +If you are using the Linux kernel as a Data Plane, this can be inspected +via a ``ip route show X`` command: + +.. code-block:: shell + + sharpd@eva ~/f/doc(ecmp_doc_change)> ip route show 4.4.4.4/32 + 4.4.4.4 nhid 185483868 proto sharp metric 20 + nexthop via 192.168.161.1 dev enp39s0 weight 1 + nexthop via 192.168.161.10 dev enp39s0 weight 1 + nexthop via 192.168.161.11 dev enp39s0 weight 1 + nexthop via 192.168.161.12 dev enp39s0 weight 1 + nexthop via 192.168.161.13 dev enp39s0 weight 1 + nexthop via 192.168.161.14 dev enp39s0 weight 1 + nexthop via 192.168.161.15 dev enp39s0 weight 1 + nexthop via 192.168.161.16 dev enp39s0 weight 1 + nexthop via 192.168.161.2 dev enp39s0 weight 1 + nexthop via 192.168.161.3 dev enp39s0 weight 1 + nexthop via 192.168.161.4 dev enp39s0 weight 1 + nexthop via 192.168.161.5 dev enp39s0 weight 1 + nexthop via 192.168.161.6 dev enp39s0 weight 1 + nexthop via 192.168.161.7 dev enp39s0 weight 1 + nexthop via 192.168.161.8 dev enp39s0 weight 1 + nexthop via 192.168.161.9 dev enp39s0 weight 1 + +Once installed into the FIB, FRR currently has little control over what +nexthops are chosen to forward packets on. Currently the Linux kernel +has a ``fib_multipath_hash_policy`` sysctl which dictates how the hashing +algorithm is used to forward packets. + +.. _zebra-svd: + +Single Vxlan Device Support +=========================== + +FRR supports configuring VLAN-to-VNI mappings for EVPN-VXLAN, +when working with the Linux kernel. In this new way, the mapping of a VLAN +to a VNI is configured against a container VXLAN interface which is referred +to as a ‘Single VXLAN device (SVD)’. Multiple VLAN to VNI mappings can be +configured against the same SVD. This allows for a significant scaling of +the number of VNIs since a separate VXLAN interface is no longer required +for each VNI. Sample configuration of SVD with VLAN to VNI mappings is shown +below. + +If you are using the Linux kernel as a Data Plane, this can be configured +via `ip link`, `bridge link` and `bridge vlan` commands: + +.. code-block:: shell + + # linux shell + ip link add dev bridge type bridge + ip link set dev bridge type bridge vlan_filtering 1 + ip link add dev vxlan0 type vxlan external + ip link set dev vxlan0 master bridge + bridge link set dev vxlan0 vlan_tunnel on + bridge vlan add dev vxlan0 vid 100 + bridge vlan add dev vxlan0 vid 100 tunnel_info id 100 + bridge vlan tunnelshow + port vlan ids tunnel id + bridge None + vxlan0 100 100 + +.. clicmd:: show evpn access-vlan [IFNAME VLAN-ID | detail] [json] + + Show information for EVPN Access VLANs. + + :: + + VLAN SVI L2-VNI VXLAN-IF # Members + bridge.20 vlan20 20 vxlan0 0 + bridge.10 vlan10 0 vxlan0 0 + +.. _zebra-mpls: + +MPLS Commands +============= + +You can configure static mpls entries in zebra. Basically, handling MPLS +consists of popping, swapping or pushing labels to IP packets. + +MPLS Acronyms +------------- + +:abbr:`LSR (Labeled Switch Router)` + Networking devices handling labels used to forward traffic between and through + them. + +:abbr:`LER (Labeled Edge Router)` + A Labeled edge router is located at the edge of an MPLS network, generally + between an IP network and an MPLS network. + +MPLS Push Action +---------------- + +The push action is generally used for LER devices, which want to encapsulate +all traffic for a wished destination into an MPLS label. This action is stored +in routing entry, and can be configured like a route: + +.. clicmd:: ip route NETWORK MASK GATEWAY|INTERFACE label LABEL + + NETWORK and MASK stand for the IP prefix entry to be added as static + route entry. + GATEWAY is the gateway IP address to reach, in order to reach the prefix. + INTERFACE is the interface behind which the prefix is located. + LABEL is the MPLS label to use to reach the prefix abovementioned. + + You can check that the static entry is stored in the zebra RIB database, by + looking at the presence of the entry. + + :: + + zebra(configure)# ip route 1.1.1.1/32 10.0.1.1 label 777 + zebra# show ip route + Codes: K - kernel route, C - connected, S - static, R - RIP, + O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, + T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, + F - PBR, + > - selected route, * - FIB route + + S>* 1.1.1.1/32 [1/0] via 10.0.1.1, r2-eth0, label 777, 00:39:42 + +MPLS Swap and Pop Action +------------------------ + +The swap action is generally used for LSR devices, which swap a packet with a +label, with an other label. The Pop action is used on LER devices, at the +termination of the MPLS traffic; this is used to remove MPLS header. + +.. clicmd:: mpls lsp INCOMING_LABEL GATEWAY OUTGOING_LABEL|explicit-null|implicit-null + + INCOMING_LABEL and OUTGOING_LABEL are MPLS labels with values ranging from 16 + to 1048575. + GATEWAY is the gateway IP address where to send MPLS packet. + The outgoing label can either be a value or have an explicit-null label header. This + specific header can be read by IP devices. The incoming label can also be removed; in + that case the implicit-null keyword is used, and the outgoing packet emitted is an IP + packet without MPLS header. + +You can check that the MPLS actions are stored in the zebra MPLS table, by looking at the +presence of the entry. + +.. clicmd:: show mpls table + +:: + + zebra(configure)# mpls lsp 18 10.125.0.2 implicit-null + zebra(configure)# mpls lsp 19 10.125.0.2 20 + zebra(configure)# mpls lsp 21 10.125.0.2 explicit-null + zebra# show mpls table + Inbound Outbound + Label Type Nexthop Label + -------- ------- --------------- -------- + 18 Static 10.125.0.2 implicit-null + 19 Static 10.125.0.2 20 + 21 Static 10.125.0.2 IPv4 Explicit Null + + +Allocated label chunks table can be dumped using the command + +.. clicmd:: show debugging label-table + +:: + + zebra# show debugging label-table + Proto ospf: [300/350] + Proto srte: [500/500] + Proto isis: [1200/1300] + Proto ospf: [20000/21000] + Proto isis: [22000/23000] + +.. _zebra-srv6: + +Segment-Routing IPv6 +==================== + +Segment-Routing is source routing paradigm that allows +network operator to encode network intent into the packets. +SRv6 is an implementation of Segment-Routing +with application of IPv6 and segment-routing-header. + +All routing daemon can use the Segment-Routing base +framework implemented on zebra to use SRv6 routing mechanism. +In that case, user must configure initial srv6 setting on +FRR's cli or frr.conf or zebra.conf. This section shows how +to configure SRv6 on FRR. Of course SRv6 can be used as standalone, +and this section also helps that case. + +.. clicmd:: show segment-routing srv6 locator [json] + + This command dump SRv6-locator configured on zebra. SRv6-locator is used + to route to the node before performing the SRv6-function. and that works as + aggregation of SRv6-function's IDs. Following console log shows two + SRv6-locators loc1 and loc2. All locators are identified by unique IPv6 + prefix. User can get that information as JSON string when ``json`` key word + at the end of cli is presented. + +:: + + router# sh segment-routing srv6 locator + Locator: + Name ID Prefix Status + -------------------- ------- ------------------------ ------- + loc1 1 2001:db8:1:1::/64 Up + loc2 2 2001:db8:2:2::/64 Up + +.. clicmd:: show segment-routing srv6 locator NAME detail [json] + + As shown in the example, by specifying the name of the locator, you + can see the detailed information for each locator. Locator can be + represented by a single IPv6 prefix, but SRv6 is designed to share this + Locator among multiple Routing Protocols. For this purpose, zebra divides + the IPv6 prefix block that makes the Locator unique into multiple chunks, + and manages the ownership of each chunk. + + For example, loc1 has system as its owner. For example, loc1 is owned by + system, which means that it is not yet proprietary to any routing protocol. + For example, loc2 has sharp as its owner. This means that the shaprd for + function development holds the owner of the chunk of this locator, and no + other routing protocol will use this area. + +:: + + router# show segment-routing srv6 locator loc1 detail + Name: loc1 + Prefix: 2001:db8:1:1::/64 + Chunks: + - prefix: 2001:db8:1:1::/64, owner: system + + router# show segment-routing srv6 locator loc2 detail + Name: loc2 + Prefix: 2001:db8:2:2::/64 + Chunks: + - prefix: 2001:db8:2:2::/64, owner: sharp + +.. clicmd:: segment-routing + + Move from configure mode to segment-routing node. + +.. clicmd:: srv6 + + Move from segment-routing node to srv6 node. + +.. clicmd:: locators + + Move from srv6 node to locator node. In this locator node, user can + configure detailed settings such as the actual srv6 locator. + +.. clicmd:: locator NAME + + Create a new locator. If the name of an existing locator is specified, + move to specified locator's configuration node to change the settings it. + +.. clicmd:: prefix X:X::X:X/M [func-bits (0-64)] [block-len 40] [node-len 24] + + Set the ipv6 prefix block of the locator. SRv6 locator is defined by + RFC8986. The actual routing protocol specifies the locator and allocates a + SID to be used by each routing protocol. This SID is included in the locator + as an IPv6 prefix. + + Following example console log shows the typical configuration of SRv6 + data-plane. After a new SRv6 locator, named loc1, is created, loc1's prefix + is configured as ``2001:db8:1:1::/64``. If user or some routing daemon + allocates new SID on this locator, new SID will allocated in range of this + prefix. For example, if some routing daemon creates new SID on locator + (``2001:db8:1:1::/64``), Then new SID will be ``2001:db8:1:1:7::/80``, + ``2001:db8:1:1:8::/80``, and so on. Each locator has default SID that is + SRv6 local function "End". Usually default SID is allocated as + ``PREFIX:1::``. (``PREFIX`` is locator's prefix) For example, if user + configure the locator's prefix as ``2001:db8:1:1::/64``, then default SID + will be ``2001:db8:1:1:1::``) + + This command takes three optional parameters: ``func-bits``, ``block-len`` + and ``node-len``. These parameters allow users to set the format for the SIDs + allocated from the SRv6 Locator. SID Format is defined in RFC 8986. + + According to RFC 8986, an SRv6 SID consists of BLOCK:NODE:FUNCTION:ARGUMENT, + where BLOCK is the SRv6 SID block (i.e., the IPv6 prefix allocated for SRv6 + SIDs by the operator), NODE is the identifier of the parent node instantiating + the SID, FUNCTION identifies the local behavior associated to the SID and + ARGUMENT encodes additional information used to process the behavior. + BLOCK and NODE make up the SRv6 Locator. + + The function bits range is 16bits by default. If operator want to change + function bits range, they can configure with ``func-bits`` + option. + + The ``block-len`` and ``node-len`` parameters allow the user to configure the + length of the SRv6 SID block and SRv6 SID node, respectively. Both the lengths + are expressed in bits. + + ``block-len``, ``node-len`` and ``func-bits`` may be any value as long as + ``block-len+node-len = locator-len`` and ``block-len+node-len+func-bits <= 128``. + + When both ``block-len`` and ``node-len`` are omitted, the following default + values are used: ``block-len = 24``, ``node-len = prefix-len-24``. + + If only one parameter is omitted, the other parameter is derived from the first. + +:: + + router# configure terminal + router(config)# segment-routinig + router(config-sr)# srv6 + router(config-srv6)# locators + router(config-srv6-locs)# locator loc1 + router(config-srv6-loc)# prefix 2001:db8:1:1::/64 + + router(config-srv6-loc)# show run + ... + segment-routing + srv6 + locators + locator loc1 + prefix 2001:db8:1:1::/64 + ! + ... + +.. clicmd:: behavior usid + + Specify the SRv6 locator as a Micro-segment (uSID) locator. When a locator is + specified as a uSID locator, all the SRv6 SIDs allocated from the locator by the routing + protocols are bound to the SRv6 uSID behaviors. For example, if you configure BGP to use + a locator specified as a uSID locator, BGP instantiates and advertises SRv6 uSID behaviors + (e.g., ``uDT4`` / ``uDT6`` / ``uDT46``) instead of classic SRv6 behaviors + (e.g., ``End.DT4`` / ``End.DT6`` / ``End.DT46``). + +:: + + router# configure terminal + router(config)# segment-routinig + router(config-sr)# srv6 + router(config-srv6)# locators + router(config-srv6-locators)# locator loc1 + router(config-srv6-locator)# prefix fc00:0:1::/48 block-len 32 node-len 16 func-bits 16 + router(config-srv6-locator)# behavior usid + + router(config-srv6-locator)# show run + ... + segment-routing + srv6 + locators + locator loc1 + prefix fc00:0:1::/48 + behavior usid + ! + ... + +.. _multicast-rib-commands: + +Multicast RIB Commands +====================== + +The Multicast RIB provides a separate table of unicast destinations which +is used for Multicast Reverse Path Forwarding decisions. It is used with +a multicast source's IP address, hence contains not multicast group +addresses but unicast addresses. + +This table is fully separate from the default unicast table. However, +RPF lookup can include the unicast table. + +WARNING: RPF lookup results are non-responsive in this version of FRR, +i.e. multicast routing does not actively react to changes in underlying +unicast topology! + +.. clicmd:: ip multicast rpf-lookup-mode MODE + + + MODE sets the method used to perform RPF lookups. Supported modes: + + urib-only + Performs the lookup on the Unicast RIB. The Multicast RIB is never used. + + mrib-only + Performs the lookup on the Multicast RIB. The Unicast RIB is never used. + + mrib-then-urib + Tries to perform the lookup on the Multicast RIB. If any route is found, + that route is used. Otherwise, the Unicast RIB is tried. + + lower-distance + Performs a lookup on the Multicast RIB and Unicast RIB each. The result + with the lower administrative distance is used; if they're equal, the + Multicast RIB takes precedence. + + longer-prefix + Performs a lookup on the Multicast RIB and Unicast RIB each. The result + with the longer prefix length is used; if they're equal, the + Multicast RIB takes precedence. + + The ``mrib-then-urib`` setting is the default behavior if nothing is + configured. If this is the desired behavior, it should be explicitly + configured to make the configuration immune against possible changes in + what the default behavior is. + +.. warning:: + + Unreachable routes do not receive special treatment and do not cause + fallback to a second lookup. + +.. clicmd:: show [ip|ipv6] rpf ADDR + + Performs a Multicast RPF lookup, as configured with ``ip multicast + rpf-lookup-mode MODE``. ADDR specifies the multicast source address to look + up. + + :: + + > show ip rpf 192.0.2.1 + Routing entry for 192.0.2.0/24 using Unicast RIB + Known via "kernel", distance 0, metric 0, best + * 198.51.100.1, via eth0 + + + Indicates that a multicast source lookup for 192.0.2.1 would use an + Unicast RIB entry for 192.0.2.0/24 with a gateway of 198.51.100.1. + +.. clicmd:: show [ip|ipv6] rpf + + Prints the entire Multicast RIB. Note that this is independent of the + configured RPF lookup mode, the Multicast RIB may be printed yet not + used at all. + +.. clicmd:: ip mroute PREFIX NEXTHOP [DISTANCE] + + + Adds a static route entry to the Multicast RIB. This performs exactly as the + ``ip route`` command, except that it inserts the route in the Multicast RIB + instead of the Unicast RIB. + +.. _zebra-route-filtering: + +zebra Route Filtering +===================== + +Zebra supports :dfn:`prefix-list` s and :ref:`route-map` s to match routes +received from other FRR components. The permit/deny facilities provided by +these commands can be used to filter which routes zebra will install in the +kernel. + +.. clicmd:: ip protocol PROTOCOL route-map ROUTEMAP + + Apply a route-map filter to routes for the specified protocol. PROTOCOL can + be: + + - any, + - babel, + - bgp, + - connected, + - eigrp, + - isis, + - kernel, + - nhrp, + - openfabric, + - ospf, + - ospf6, + - rip, + - sharp, + - static, + - ripng, + - table, + - vnc. + + If you choose any as the option that will cause all protocols that are sending + routes to zebra. You can specify a :dfn:`ip protocol PROTOCOL route-map ROUTEMAP` + on a per vrf basis, by entering this command under vrf mode for the vrf you + want to apply the route-map against. + +.. clicmd:: set src ADDRESS + + Within a route-map, set the preferred source address for matching routes + when installing in the kernel. + + +The following creates a prefix-list that matches all addresses, a route-map +that sets the preferred source address, and applies the route-map to all +*rip* routes. + +.. code-block:: frr + + ip prefix-list ANY permit 0.0.0.0/0 le 32 + route-map RM1 permit 10 + match ip address prefix-list ANY + set src 10.0.0.1 + + ip protocol rip route-map RM1 + +IPv6 example for OSPFv3. + +.. code-block:: frr + + ipv6 prefix-list ANY seq 10 permit any + route-map RM6 permit 10 + match ipv6 address prefix-list ANY + set src 2001:db8:425:1000::3 + + ipv6 protocol ospf6 route-map RM6 + + +.. note:: + + For both IPv4 and IPv6, the IP address has to exist on some interface when + the route is getting installed into the system. Otherwise, kernel rejects + the route. To solve the problem of disappearing IPv6 addresses when the + interface goes down, use ``net.ipv6.conf.all.keep_addr_on_down`` + :ref:`sysctl option <zebra-sysctl>`. + +.. clicmd:: zebra route-map delay-timer (0-600) + + Set the delay before any route-maps are processed in zebra. The + default time for this is 5 seconds. + +.. _zebra-fib-push-interface: + +zebra FIB push interface +======================== + +Zebra supports a 'FIB push' interface that allows an external +component to learn the forwarding information computed by the FRR +routing suite. This is a loadable module that needs to be enabled +at startup as described in :ref:`loadable-module-support`. + +In FRR, the Routing Information Base (RIB) resides inside +zebra. Routing protocols communicate their best routes to zebra, and +zebra computes the best route across protocols for each prefix. This +latter information makes up the Forwarding Information Base +(FIB). Zebra feeds the FIB to the kernel, which allows the IP stack in +the kernel to forward packets according to the routes computed by +FRR. The kernel FIB is updated in an OS-specific way. For example, +the ``Netlink`` interface is used on Linux, and route sockets are +used on FreeBSD. + +The FIB push interface aims to provide a cross-platform mechanism to +support scenarios where the router has a forwarding path that is +distinct from the kernel, commonly a hardware-based fast path. In +these cases, the FIB needs to be maintained reliably in the fast path +as well. We refer to the component that programs the forwarding plane +(directly or indirectly) as the Forwarding Plane Manager or FPM. + +.. program:: configure + +The relevant zebra code kicks in when zebra is configured with the +:option:`--enable-fpm` flag and started with the module (``-M fpm`` +or ``-M dplane_fpm_nl``). + +.. note:: + + The ``fpm`` implementation attempts to connect to ``127.0.0.1`` port ``2620`` + by default without configurations. The ``dplane_fpm_nl`` only attempts to + connect to a server if configured. + +Zebra periodically attempts to connect to the well-known FPM port (``2620``). +Once the connection is up, zebra starts sending messages containing routes +over the socket to the FPM. Zebra sends a complete copy of the forwarding +table to the FPM, including routes that it may have picked up from the kernel. +The existing interaction of zebra with the kernel remains unchanged -- that +is, the kernel continues to receive FIB updates as before. + +The default FPM message format is netlink, however it can be controlled +with the module load-time option. The modules accept the following options: + +- ``fpm``: ``netlink`` and ``protobuf``. +- ``dplane_fpm_nl``: none, it only implements netlink. + +The zebra FPM interface uses replace semantics. That is, if a 'route +add' message for a prefix is followed by another 'route add' message, +the information in the second message is complete by itself, and +replaces the information sent in the first message. + +If the connection to the FPM goes down for some reason, zebra sends +the FPM a complete copy of the forwarding table(s) when it reconnects. + +For more details on the implementation, please read the developer's manual FPM +section. + +FPM Commands +============ + +``fpm`` implementation +---------------------- + +.. clicmd:: fpm connection ip A.B.C.D port (1-65535) + + Configure ``zebra`` to connect to a different FPM server than the default of + ``127.0.0.1:2620`` + +.. clicmd:: show zebra fpm stats + + Shows the FPM statistics. + + Sample output: + + :: + + Counter Total Last 10 secs + + connect_calls 3 2 + connect_no_sock 0 0 + read_cb_calls 2 2 + write_cb_calls 2 0 + write_calls 1 0 + partial_writes 0 0 + max_writes_hit 0 0 + t_write_yields 0 0 + nop_deletes_skipped 6 0 + route_adds 5 0 + route_dels 0 0 + updates_triggered 11 0 + redundant_triggers 0 0 + dests_del_after_update 0 0 + t_conn_down_starts 0 0 + t_conn_down_dests_processed 0 0 + t_conn_down_yields 0 0 + t_conn_down_finishes 0 0 + t_conn_up_starts 1 0 + t_conn_up_dests_processed 11 0 + t_conn_up_yields 0 0 + t_conn_up_aborts 0 0 + t_conn_up_finishes 1 0 + + +.. clicmd:: clear zebra fpm stats + + Reset statistics related to the zebra code that interacts with the + optional Forwarding Plane Manager (FPM) component. + + +``dplane_fpm_nl`` implementation +-------------------------------- + +.. clicmd:: fpm address <A.B.C.D|X:X::X:X> [port (1-65535)] + + Configures the FPM server address. Once configured ``zebra`` will attempt + to connect to it immediately. + + The ``no`` form disables FPM entirely. ``zebra`` will close any current + connections and will not attempt to connect to it anymore. + +.. clicmd:: fpm use-next-hop-groups + + Use the new netlink messages ``RTM_NEWNEXTHOP`` / ``RTM_DELNEXTHOP`` to + group repeated route next hop information. + + The ``no`` form uses the old known FPM behavior of including next hop + information in the route (e.g. ``RTM_NEWROUTE``) messages. + +.. clicmd:: fpm use-route-replace + + Use the netlink ``NLM_F_REPLACE`` flag for updating routes instead of + two different messages to update a route + (``RTM_DELROUTE`` + ``RTM_NEWROUTE``). + +.. clicmd:: show fpm counters [json] + + Show the FPM statistics (plain text or JSON formatted). + + Sample output: + + :: + + FPM counters + ============ + Input bytes: 0 + Output bytes: 308 + Output buffer current size: 0 + Output buffer peak size: 308 + Connection closes: 0 + Connection errors: 0 + Data plane items processed: 0 + Data plane items enqueued: 0 + Data plane items queue peak: 0 + Buffer full hits: 0 + User FPM configurations: 1 + User FPM disable requests: 0 + + +.. clicmd:: clear fpm counters + + Reset statistics related to the zebra code that interacts with the + optional Forwarding Plane Manager (FPM) component. + + +.. _zebra-dplane: + +Dataplane Commands +================== + +The zebra dataplane subsystem provides a framework for FIB +programming. Zebra uses the dataplane to program the local kernel as +it makes changes to objects such as IP routes, MPLS LSPs, and +interface IP addresses. The dataplane runs in its own pthread, in +order to off-load work from the main zebra pthread. + + +.. clicmd:: show zebra dplane [detailed] + + Display statistics about the updates and events passing through the + dataplane subsystem. + + +.. clicmd:: show zebra dplane providers + + Display information about the running dataplane plugins that are + providing updates to a FIB. By default, the local kernel plugin is + present. + + +.. clicmd:: zebra dplane limit [NUMBER] + + Configure the limit on the number of pending updates that are + waiting to be processed by the dataplane pthread. + + +DPDK dataplane +============== + +The zebra DPDK subsystem programs the dataplane via rte_XXX APIs. +This module needs be compiled in via "--enable-dp-dpdk=yes" +and enabled at start up time via the zebra daemon option "-M dplane_dpdk". + +To program the PBR rules as rte_flows you additionally need to configure +"pbr nexthop-resolve". This is used to expland the PBR actions into the +{SMAC, DMAC, outgoing port} needed by rte_flow. + + +.. clicmd:: show dplane dpdk port [detail] + + Displays the mapping table between zebra interfaces and DPDK port-ids. + Sample output: + + :: + Port Device IfName IfIndex sw,domain,port + + 0 0000:03:00.0 p0 4 0000:03:00.0,0,65535 + 1 0000:03:00.0 pf0hpf 6 0000:03:00.0,0,4095 + 2 0000:03:00.0 pf0vf0 15 0000:03:00.0,0,4096 + 3 0000:03:00.0 pf0vf1 16 0000:03:00.0,0,4097 + 4 0000:03:00.1 p1 5 0000:03:00.1,1,65535 + 5 0000:03:00.1 pf1hpf 7 0000:03:00.1,1,20479 + +.. clicmd:: show dplane dpdk pbr flows + Displays the DPDK stats per-PBR entry. + Sample output: + + :: + Rules if pf0vf0 + Seq 1 pri 300 + SRC Match 77.0.0.8/32 + DST Match 88.0.0.8/32 + Tableid: 10000 + Action: nh: 45.0.0.250 intf: p0 + Action: mac: 00:00:5e:00:01:fa + DPDK flow: installed 0x40 + DPDK flow stats: packets 13 bytes 1586 + +.. clicmd:: show dplane dpdk counters + Displays the ZAPI message handler counters + + Sample output: + + :: + Ignored updates: 0 + PBR rule adds: 1 + PBR rule dels: 0 + + +zebra Terminal Mode Commands +============================ + +.. clicmd:: show ip route + + Display current routes which zebra holds in its database. + +:: + + Router# show ip route + Codes: K - kernel route, C - connected, S - static, R - RIP, + B - BGP * - FIB route. + + K* 0.0.0.0/0 203.181.89.241 + S 0.0.0.0/0 203.181.89.1 + C* 127.0.0.0/8 lo + C* 203.181.89.240/28 eth0 + + +.. clicmd:: show ipv6 route + +.. clicmd:: show [ip|ipv6] route [PREFIX] [nexthop-group] + + Display detailed information about a route. If [nexthop-group] is + included, it will display the nexthop group ID the route is using as well. + +.. clicmd:: show interface [NAME] [{vrf VRF|brief}] [json] + +.. clicmd:: show interface [NAME] [{vrf all|brief}] [json] + +.. clicmd:: show interface [NAME] [{vrf VRF|brief}] [nexthop-group] + +.. clicmd:: show interface [NAME] [{vrf all|brief}] [nexthop-group] + + Display interface information. If no extra information is added, it will + dump information on all interfaces. If [NAME] is specified, it will display + detailed information about that single interface. If [nexthop-group] is + specified, it will display nexthop groups pointing out that interface. + + If the ``json`` option is specified, output is displayed in JSON format. + +.. clicmd:: show ip prefix-list [NAME] + +.. clicmd:: show route-map [NAME] + +.. clicmd:: show ip protocol + +.. clicmd:: show ip forward + + Display whether the host's IP forwarding function is enabled or not. + Almost any UNIX kernel can be configured with IP forwarding disabled. + If so, the box can't work as a router. + +.. clicmd:: show ipv6 forward + + Display whether the host's IP v6 forwarding is enabled or not. + +.. clicmd:: show ip neigh + + Display the ip neighbor table + +.. clicmd:: show pbr rule + + Display the pbr rule table with resolved nexthops + +.. clicmd:: show zebra + + Display various statistics related to the installation and deletion + of routes, neighbor updates, and LSP's into the kernel. In addition + show various zebra state that is useful when debugging an operator's + setup. + +.. clicmd:: show zebra client [summary] + + Display statistics about clients that are connected to zebra. This is + useful for debugging and seeing how much data is being passed between + zebra and it's clients. If the summary form of the command is chosen + a table is displayed with shortened information. + +.. clicmd:: show zebra router table summary + + Display summarized data about tables created, their afi/safi/tableid + and how many routes each table contains. Please note this is the + total number of route nodes in the table. Which will be higher than + the actual number of routes that are held. + +.. clicmd:: show nexthop-group rib [ID] [vrf NAME] [singleton [ip|ip6]] [type] [json] + + Display nexthop groups created by zebra. The [vrf NAME] option + is only meaningful if you have started zebra with the --vrfwnetns + option as that nexthop groups are per namespace in linux. + If you specify singleton you would like to see the singleton + nexthop groups that do have an afi. [type] allows you to filter those + only coming from a specific NHG type (protocol). + +.. clicmd:: show <ip|ipv6> zebra route dump [<vrf> VRFNAME] + + It dumps all the routes from RIB with detailed information including + internal flags, status etc. This is defined as a hidden command. + + +Router-id +========= + +Many routing protocols require a router-id to be configured. To have a +consistent router-id across all daemons, the following commands are available +to configure and display the router-id: + +.. clicmd:: [ip] router-id A.B.C.D + + Allow entering of the router-id. This command also works under the + vrf subnode, to allow router-id's per vrf. + +.. clicmd:: [ip] router-id A.B.C.D vrf NAME + + Configure the router-id of this router from the configure NODE. + A show run of this command will display the router-id command + under the vrf sub node. This command is deprecated and will + be removed at some point in time in the future. + +.. clicmd:: show [ip] router-id [vrf NAME] + + Display the user configured router-id. + +For protocols requiring an IPv6 router-id, the following commands are available: + +.. clicmd:: ipv6 router-id X:X::X:X + + Configure the IPv6 router-id of this router. Like its IPv4 counterpart, + this command works under the vrf subnode, to allow router-id's per vrf. + +.. clicmd:: show ipv6 router-id [vrf NAME] + + Display the user configured IPv6 router-id. + +.. _zebra-sysctl: + +sysctl settings +=============== + +The linux kernel has a variety of sysctl's that affect it's operation as a router. This +section is meant to act as a starting point for those sysctl's that must be used in +order to provide FRR with smooth operation as a router. This section is not meant +as the full documentation for sysctl's. The operator must use the sysctl documentation +with the linux kernel for that. The following link has helpful references to many relevant +sysctl values: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt + +Expected sysctl settings +------------------------ + +.. option:: net.ipv4.ip_forward = 1 + + This global option allows the linux kernel to forward (route) ipv4 packets incoming from one + interface to an outgoing interface. If this is set to 0, the system will not route transit + ipv4 packets, i.e. packets that are not sent to/from a process running on the local system. + +.. option:: net.ipv4.conf.{all,default,<interface>}.forwarding = 1 + + The linux kernel can selectively enable forwarding (routing) of ipv4 packets on a per + interface basis. The forwarding check in the kernel dataplane occurs against the ingress + Layer 3 interface, i.e. if the ingress L3 interface has forwarding set to 0, packets will not + be routed. + +.. option:: net.ipv6.conf.{all,default,<interface>}.forwarding = 1 + + This per interface option allows the linux kernel to forward (route) transit ipv6 packets + i.e. incoming from one Layer 3 interface to an outgoing Layer 3 interface. + The forwarding check in the kernel dataplane occurs against the ingress Layer 3 interface, + i.e. if the ingress L3 interface has forwarding set to 0, packets will not be routed. + +.. option:: net.ipv6.conf.all.keep_addr_on_down = 1 + + When an interface is taken down, do not remove the v6 addresses associated with the interface. + This option is recommended because this is the default behavior for v4 as well. + +.. option:: net.ipv6.route.skip_notify_on_dev_down = 1 + + When an interface is taken down, the linux kernel will not notify, via netlink, about routes + that used that interface being removed from the FIB. This option is recommended because this + is the default behavior for v4 as well. + +Optional sysctl settings +------------------------ + +.. option:: net.ipv4.conf.{all,default,<interface>}.bc_forwarding = 0 + + This per interface option allows the linux kernel to optionally allow Directed Broadcast + (i.e. Routed Broadcast or Subnet Broadcast) packets to be routed onto the connected network + segment where the subnet exists. + If the local router receives a routed packet destined for a broadcast address of a connected + subnet, setting bc_forwarding to 1 on the interface with the target subnet assigned to it will + allow non locally-generated packets to be routed via the broadcast route. + If bc_forwarding is set to 0, routed packets destined for a broadcast route will be dropped. + e.g. + Host1 (SIP:192.0.2.10, DIP:10.0.0.255) -> (eth0:192.0.2.1/24) Router1 (eth1:10.0.0.1/24) -> BC + If net.ipv4.conf.{all,default,<interface>}.bc_forwarding=1, then Router1 will forward each + packet destined to 10.0.0.255 onto the eth1 interface with a broadcast DMAC (ff:ff:ff:ff:ff:ff). + +.. option:: net.ipv4.conf.{all,default,<interface>}.arp_accept = 1 + + This per interface option allows the linux kernel to optionally skip the creation of ARP + entries upon the receipt of a Gratuitous ARP (GARP) frame carrying an IP that is not already + present in the ARP cache. Setting arp_accept to 0 on an interface will ensure NEW ARP entries + are not created due to the arrival of a GARP frame. + Note: This does not impact how the kernel reacts to GARP frames that carry a "known" IP + (that is already in the ARP cache) -- an existing ARP entry will always be updated + when a GARP for that IP is received. + +.. option:: net.ipv4.conf.{all,default,<interface>}.arp_ignore = 0 + + This per interface option allows the linux kernel to control what conditions must be met in + order for an ARP reply to be sent in response to an ARP request targeting a local IP address. + When arp_ignore is set to 0, the kernel will send ARP replies in response to any ARP Request + with a Target-IP matching a local address. + When arp_ignore is set to 1, the kernel will send ARP replies if the Target-IP in the ARP + Request matches an IP address on the interface the Request arrived at. + When arp_ignore is set to 2, the kernel will send ARP replies only if the Target-IP matches an + IP address on the interface where the Request arrived AND the Sender-IP falls within the subnet + assigned to the local IP/interface. + +.. option:: net.ipv4.conf.{all,default,<interface>}.arp_notify = 1 + + This per interface option allows the linux kernel to decide whether to send a Gratuitious ARP + (GARP) frame when the Layer 3 interface comes UP. + When arp_notify is set to 0, no GARP is sent. + When arp_notify is set to 1, a GARP is sent when the interface comes UP. + +.. option:: net.ipv6.conf.{all,default,<interface>}.ndisc_notify = 1 + + This per interface option allows the linux kernel to decide whether to send an Unsolicited + Neighbor Advertisement (U-NA) frame when the Layer 3 interface comes UP. + When ndisc_notify is set to 0, no U-NA is sent. + When ndisc_notify is set to 1, a U-NA is sent when the interface comes UP. + +Useful sysctl settings +---------------------- + +.. option:: net.ipv6.conf.all.use_oif_addrs_only = 1 + + When enabled, the candidate source addresses for destinations routed via this interface are + restricted to the set of addresses configured on this interface (RFC 6724 section 4). If + an operator has hundreds of IP addresses per interface this solves the latency problem. + +Debugging +========= + +.. clicmd:: debug zebra mpls [detailed] + + MPLS-related events and information. + +.. clicmd:: debug zebra events + + Zebra events + +.. clicmd:: debug zebra nht [detailed] + + Nexthop-tracking / reachability information + +.. clicmd:: debug zebra vxlan + + VxLAN (EVPN) events + +.. clicmd:: debug zebra pseudowires + + Pseudowire events. + +.. clicmd:: debug zebra packet [<recv|send>] [detail] + + ZAPI message and packet details + +.. clicmd:: debug zebra kernel + + Kernel / OS events. + +.. clicmd:: debug zebra kernel msgdump [<recv|send>] + + Raw OS (netlink) message details. + +.. clicmd:: debug zebra rib [detailed] + + RIB events. + +.. clicmd:: debug zebra fpm + + FPM (forwarding-plane manager) events. + +.. clicmd:: debug zebra dplane [detailed] + + Dataplane / FIB events. + +.. clicmd:: debug zebra pbr + + PBR (policy-based routing) events. + +.. clicmd:: debug zebra mlag + + MLAG events. + +.. clicmd:: debug zebra evpn mh <es|mac|neigh|nh> + + EVPN multi-hop events. + +.. clicmd:: debug zebra nexthop [detail] + + Nexthop and nexthop-group events. + +Scripting +========= + +.. clicmd:: zebra on-rib-process script SCRIPT + + Set a Lua script for :ref:`on-rib-process-dplane-results` hook call. + SCRIPT is the basename of the script, without ``.lua``. + +Data structures +--------------- + +.. _const-struct-zebra-dplane-ctx: + +const struct zebra_dplane_ctx +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + * integer zd_op + * integer zd_status + * integer zd_provider + * integer zd_vrf_id + * integer zd_table_id + * integer zd_ifname + * integer zd_ifindex + * table rinfo (if zd_op is DPLANE_OP_ROUTE*, DPLANE_NH_*) + + * prefix zd_dest + * prefix zd_src + * integer zd_afi + * integer zd_safi + * integer zd_type + * integer zd_old_type + * integer zd_tag + * integer zd_old_tag + * integer zd_metric + * integer zd_old_metric + * integer zd_instance + * integer zd_old_instance + * integer zd_distance + * integer zd_old_distance + * integer zd_mtu + * integer zd_nexthop_mtu + * table nhe + + * integer id + * integer old_id + * integer afi + * integer vrf_id + * integer type + * nexthop_group ng + * nh_grp + * integer nh_grp_count + + * integer zd_nhg_id + * nexthop_group zd_ng + * nexthop_group backup_ng + * nexthop_group zd_old_ng + * nexthop_group old_backup_ng + + * integer label (if zd_op is DPLANE_OP_LSP_*) + * table pw (if zd_op is DPLANE_OP_PW_*) + + * integer type + * integer af + * integer status + * integer flags + * integer local_label + * integer remote_label + + * table macinfo (if zd_op is DPLANE_OP_MAC_*) + + * integer vid + * integer br_ifindex + * ethaddr mac + * integer vtep_ip + * integer is_sticky + * integer nhg_id + * integer update_flags + + * table rule (if zd_op is DPLANE_OP_RULE_*) + + * integer sock + * integer unique + * integer seq + * string ifname + * integer priority + * integer old_priority + * integer table + * integer old_table + * integer filter_bm + * integer old_filter_bm + * integer fwmark + * integer old_fwmark + * integer dsfield + * integer old_dsfield + * integer ip_proto + * integer old_ip_proto + * prefix src_ip + * prefix old_src_ip + * prefix dst_ip + * prefix old_dst_ip + + * table iptable (if zd_op is DPLANE_OP_IPTABLE_*) + + * integer sock + * integer vrf_id + * integer unique + * integer type + * integer filter_bm + * integer fwmark + * integer action + * integer pkt_len_min + * integer pkt_len_max + * integer tcp_flags + * integer dscp_value + * integer fragment + * integer protocol + * integer nb_interface + * integer flow_label + * integer family + * string ipset_name + + * table ipset (if zd_op is DPLANE_OP_IPSET_*) + * integer sock + * integer vrf_id + * integer unique + * integer type + * integer family + * string ipset_name + + * table neigh (if zd_op is DPLANE_OP_NEIGH_*) + + * ipaddr ip_addr + * table link + + * ethaddr mac + * ipaddr ip_addr + + * integer flags + * integer state + * integer update_flags + + * table br_port (if zd_op is DPLANE_OP_BR_PORT_UPDATE) + + * integer sph_filter_cnt + * integer flags + * integer backup_nhg_id + + * table neightable (if zd_op is DPLANE_OP_NEIGH_TABLE_UPDATE) + + * integer family + * integer app_probes + * integer ucast_probes + * integer mcast_probes + + * table gre (if zd_op is DPLANE_OP_GRE_SET)** + + * integer link_ifindex + * integer mtu + + +.. _const-struct-nh-grp: + +const struct nh_grp +^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + * integer id + * integer weight + + +.. _zebra-hook-calls: + +Zebra Hook calls +---------------- + +.. _on-rib-process-dplane-results: + +on_rib_process_dplane_results +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Called when RIB processes dataplane events. +Set script location with the ``zebra on-rib-process script SCRIPT`` command. + +**Arguments** + +* :ref:`const struct zebra_dplane_ctx<const-struct-zebra-dplane-ctx>` ctx + + +.. code-block:: lua + + function on_rib_process_dplane_results(ctx) + log.info(ctx.rinfo.zd_dest.network) + return {} |
