diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-27 04:24:31 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-27 04:24:31 +0000 |
| commit | acb594b1d825c6e12369cebb941968ec08c840ce (patch) | |
| tree | d544788908e7353a4f117e2991f15f4236a0c963 /doc/developer | |
| parent | Adding upstream version 9.1. (diff) | |
| download | frr-acb594b1d825c6e12369cebb941968ec08c840ce.tar.xz frr-acb594b1d825c6e12369cebb941968ec08c840ce.zip | |
Adding upstream version 10.0.upstream/10.0
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'doc/developer')
49 files changed, 1061 insertions, 587 deletions
diff --git a/doc/developer/.readthedocs.yaml b/doc/developer/.readthedocs.yaml index 113672f..90ee5c7 100644 --- a/doc/developer/.readthedocs.yaml +++ b/doc/developer/.readthedocs.yaml @@ -6,6 +6,12 @@ build: os: ubuntu-22.04 tools: python: "3.11" + apt_packages: + - graphviz + +python: + install: + - requirements: doc/developer/requirements.txt # Build documentation in the docs/ directory with Sphinx sphinx: diff --git a/doc/developer/bmp.rst b/doc/developer/bmp.rst new file mode 100644 index 0000000..1c0e4b0 --- /dev/null +++ b/doc/developer/bmp.rst @@ -0,0 +1,49 @@ +.. _bmp: + +*** +BMP +*** + +RFC 7854 +======== +Missing features (non exhaustive): + - Per-Peer Header + + - Peer Type Flag + - Peer Distingsher + + - Peer Up + + - Reason codes (according to TODO comments in code) + +Peer Type Flag and Peer Distinguisher can be implemented easily using RFC 9069's base code. + +RFC 9069 +======== +Everything that isn't listed here is implemented and should be working. +Missing features (should be exhaustive): + +- Per-Peer Header + + - Timestamp + + - set to 0 + - value is now saved `struct bgp_path_info -> locrib_uptime` + - needs testing + +- Peer Up/Down + + - VRF/Table Name TLV + + - code for TLV exists + - need better RFC understanding + +- Peer Down Only + + - Reason code (bc not supported in RFC 7854 either) + +- Statistics Report + + - Stat Type = 8: (64-bit Gauge) Number of routes in Loc-RIB. + - Stat Type = 10: Number of routes in per-AFI/SAFI Loc-RIB. The value is + structured as: 2-byte AFI, 1-byte SAFI, followed by a 64-bit Gauge. diff --git a/doc/developer/building-docker.rst b/doc/developer/building-docker.rst index 9d42784..644e02b 100644 --- a/doc/developer/building-docker.rst +++ b/doc/developer/building-docker.rst @@ -14,7 +14,7 @@ source-built FRR on the following base platforms: The following platform images are used to support Travis CI and can also be used to reproduce topotest failures when the docker host is Ubuntu -(tested on 18.04 and 20.04): +(tested on 20.04 and 22.04): * Ubuntu 20.04 * Ubuntu 22.04 @@ -139,12 +139,12 @@ Build image (from project root directory):: Running Full Topotest:: - docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \ + docker run --init -it --privileged --name frr-ubuntu20 -v /lib/modules:/lib/modules \ frr-ubuntu20:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile' Extract results from the above run into `run-results` dir and analyze:: - tests/topotest/analyze.py -C frr -Ar run-results + tests/topotests/analyze.py -C frr-ubuntu20 -Ar run-results Start the container:: @@ -176,12 +176,12 @@ Build image (from project root directory):: Running Full Topotest:: - docker run --init -it --privileged --name frr -v /lib/modules:/lib/modules \ + docker run --init -it --privileged --name frr-ubuntu22 -v /lib/modules:/lib/modules \ frr-ubuntu22:latest bash -c 'cd ~/frr/tests/topotests ; sudo pytest -nauto --dist=loadfile' Extract results from the above run into `run-results` dir and analyze:: - tests/topotest/analyze.py -C frr -Ar run-results + tests/topotests/analyze.py -C frr-ubuntu22 -Ar run-results Start the container:: diff --git a/doc/developer/building-frr-for-archlinux.rst b/doc/developer/building-frr-for-archlinux.rst index 406d22d..8b0df21 100644 --- a/doc/developer/building-frr-for-archlinux.rst +++ b/doc/developer/building-frr-for-archlinux.rst @@ -11,18 +11,12 @@ Installing Dependencies git autoconf automake libtool make cmake pcre readline texinfo \ pkg-config pam json-c bison flex python-pytest \ c-ares python python2-ipaddress python-sphinx \ - net-snmp perl libcap libelf libunwind + net-snmp perl libcap libelf libunwind protobuf-c .. include:: building-libunwind-note.rst .. include:: building-libyang.rst -Protobuf -^^^^^^^^ - -.. code-block:: console - - sudo pacman -S protobuf-c ZeroMQ ^^^^^^ diff --git a/doc/developer/building-frr-for-centos6.rst b/doc/developer/building-frr-for-centos6.rst index 233d089..3531162 100644 --- a/doc/developer/building-frr-for-centos6.rst +++ b/doc/developer/building-frr-for-centos6.rst @@ -124,7 +124,7 @@ Install libyang and its dependencies: sudo yum install pcre-devel doxygen cmake git clone https://github.com/CESNET/libyang.git cd libyang - git checkout 090926a89d59a3c4000719505d563aaf6ac60f2 + git checkout v2.1.128 mkdir build ; cd build cmake -DENABLE_LYD_PRIV=ON -DCMAKE_INSTALL_PREFIX:PATH=/usr -D CMAKE_BUILD_TYPE:String="Release" .. make build-rpm @@ -161,10 +161,8 @@ an example.) ./configure \ --bindir=/usr/bin \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --libdir=/usr/lib/frr \ --libexecdir=/usr/lib/frr \ - --localstatedir=/var/run/frr \ --with-moduledir=/usr/lib/frr/modules \ --disable-pimd \ --enable-snmp=agentx \ diff --git a/doc/developer/building-frr-for-centos7.rst b/doc/developer/building-frr-for-centos7.rst index e6da830..eabf515 100644 --- a/doc/developer/building-frr-for-centos7.rst +++ b/doc/developer/building-frr-for-centos7.rst @@ -58,10 +58,8 @@ an example.) ./configure \ --bindir=/usr/bin \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --libdir=/usr/lib/frr \ --libexecdir=/usr/lib/frr \ - --localstatedir=/var/run/frr \ --with-moduledir=/usr/lib/frr/modules \ --enable-snmp=agentx \ --enable-multipath=64 \ diff --git a/doc/developer/building-frr-for-centos8.rst b/doc/developer/building-frr-for-centos8.rst index 6d18e7b..2d514ea 100644 --- a/doc/developer/building-frr-for-centos8.rst +++ b/doc/developer/building-frr-for-centos8.rst @@ -52,10 +52,8 @@ an example.) ./configure \ --bindir=/usr/bin \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --libdir=/usr/lib/frr \ --libexecdir=/usr/lib/frr \ - --localstatedir=/var/run/frr \ --with-moduledir=/usr/lib/frr/modules \ --enable-snmp=agentx \ --enable-multipath=64 \ diff --git a/doc/developer/building-frr-for-debian12.rst b/doc/developer/building-frr-for-debian12.rst index ca882ee..06bc18c 100644 --- a/doc/developer/building-frr-for-debian12.rst +++ b/doc/developer/building-frr-for-debian12.rst @@ -47,9 +47,9 @@ an example.) cd frr ./bootstrap.sh ./configure \ - --localstatedir=/var/opt/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --enable-multipath=64 \ --enable-user=frr \ --enable-group=frr \ diff --git a/doc/developer/building-frr-for-debian8.rst b/doc/developer/building-frr-for-debian8.rst index 7071cb6..fe4eeea 100644 --- a/doc/developer/building-frr-for-debian8.rst +++ b/doc/developer/building-frr-for-debian8.rst @@ -57,9 +57,9 @@ an example.) cd frr ./bootstrap.sh ./configure \ - --localstatedir=/var/run/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --enable-multipath=64 \ --enable-user=frr \ --enable-group=frr \ @@ -118,9 +118,9 @@ Troubleshooting The local state directory must exist and have the correct permissions applied for the frrouting daemons to start. In the above ./configure -example the local state directory is set to /var/run/frr -(--localstatedir=/var/run/frr) Debian considers /var/run/frr to be -temporary and this is removed after a reboot. +example the local state directory is set to ``/var`` such that ``/var/run/frr`` +is used. Debian considers ``/var/run/frr`` to be temporary and this is removed +after a reboot. When using a different local state directory you need to create the new directory and change the ownership to the frr user, for example: diff --git a/doc/developer/building-frr-for-debian9.rst b/doc/developer/building-frr-for-debian9.rst index 1b2f1b9..a590cf7 100644 --- a/doc/developer/building-frr-for-debian9.rst +++ b/doc/developer/building-frr-for-debian9.rst @@ -47,9 +47,9 @@ an example.) cd frr ./bootstrap.sh ./configure \ - --localstatedir=/var/opt/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --enable-multipath=64 \ --enable-user=frr \ --enable-group=frr \ diff --git a/doc/developer/building-frr-for-freebsd10.rst b/doc/developer/building-frr-for-freebsd10.rst index 707f1e7..beefb59 100644 --- a/doc/developer/building-frr-for-freebsd10.rst +++ b/doc/developer/building-frr-for-freebsd10.rst @@ -60,9 +60,9 @@ an example) export LDFLAGS="-L/usr/local/lib" export CPPFLAGS="-I/usr/local/include" ./configure \ - --sysconfdir=/usr/local/etc/frr \ + --sysconfdir=/usr/local/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --prefix=/usr/local \ --enable-multipath=64 \ --enable-user=frr \ diff --git a/doc/developer/building-frr-for-freebsd11.rst b/doc/developer/building-frr-for-freebsd11.rst index af0b72b..7c8fb83 100644 --- a/doc/developer/building-frr-for-freebsd11.rst +++ b/doc/developer/building-frr-for-freebsd11.rst @@ -65,9 +65,9 @@ an example) setenv CPPFLAGS -I/usr/local/include ln -s /usr/local/bin/sphinx-build-3.6 /usr/local/bin/sphinx-build ./configure \ - --sysconfdir=/usr/local/etc/frr \ + --sysconfdir=/usr/local/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --prefix=/usr/local \ --enable-multipath=64 \ --enable-user=frr \ diff --git a/doc/developer/building-frr-for-freebsd13.rst b/doc/developer/building-frr-for-freebsd13.rst index 0bc8277..86506a9 100644 --- a/doc/developer/building-frr-for-freebsd13.rst +++ b/doc/developer/building-frr-for-freebsd13.rst @@ -52,9 +52,9 @@ an example) ./bootstrap.sh export MAKE=gmake LDFLAGS=-L/usr/local/lib CPPFLAGS=-I/usr/local/include ./configure \ - --sysconfdir=/usr/local/etc/frr \ + --sysconfdir=/usr/local/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --prefix=/usr/local \ --enable-multipath=64 \ --enable-user=frr \ diff --git a/doc/developer/building-frr-for-freebsd14.rst b/doc/developer/building-frr-for-freebsd14.rst new file mode 100644 index 0000000..b3fd37a --- /dev/null +++ b/doc/developer/building-frr-for-freebsd14.rst @@ -0,0 +1,122 @@ +FreeBSD 14 +========== + +FreeBSD 14 restrictions: +------------------------ + +- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel + (4.5 or higher). LDP can be built, but may have limited use without + MPLS +- PIM for IPv6 is not currently supported on ``FreeBSD``. + +Install required packages +------------------------- + +Add packages: (Allow the install of the package management tool if this +is first package install and asked) + +.. code-block:: shell + + pkg install autoconf automake bison c-ares git gmake json-c libtool \ + libunwind libyang2 pkgconf protobuf-c py39-pytest py39-sphinx texinfo + +.. include:: building-libunwind-note.rst + +Get FRR, compile it and install it (from Git) +--------------------------------------------- + +**This assumes you want to build and install FRR from source and not using any +packages** + +Add frr group and user +^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + pw groupadd frr -g 101 + pw groupadd frrvty -g 102 + pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \ + -d /usr/local/etc/frr -s /usr/sbin/nologin + + +Download Source, configure and compile it +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +(You may prefer different options on configure statement. These are just +an example) + +.. code-block:: shell + + git clone https://github.com/frrouting/frr.git frr + cd frr + ./bootstrap.sh + export MAKE=gmake LDFLAGS=-L/usr/local/lib CPPFLAGS=-I/usr/local/include + ./configure \ + --sysconfdir=/usr/local/etc \ + --localstatedir=/var \ + --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ + --prefix=/usr/local \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + gmake + gmake check + sudo gmake install + +Create empty FRR configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: shell + + sudo mkdir /usr/local/etc/frr + +For integrated config file: + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/frr.conf + +For individual config files: + +.. note:: Integrated config is preferred to individual config. + +.. code-block:: shell + + sudo touch /usr/local/etc/frr/babeld.conf + sudo touch /usr/local/etc/frr/bfdd.conf + sudo touch /usr/local/etc/frr/bgpd.conf + sudo touch /usr/local/etc/frr/eigrpd.conf + sudo touch /usr/local/etc/frr/isisd.conf + sudo touch /usr/local/etc/frr/ldpd.conf + sudo touch /usr/local/etc/frr/nhrpd.conf + sudo touch /usr/local/etc/frr/ospf6d.conf + sudo touch /usr/local/etc/frr/ospfd.conf + sudo touch /usr/local/etc/frr/pbrd.conf + sudo touch /usr/local/etc/frr/pimd.conf + sudo touch /usr/local/etc/frr/ripd.conf + sudo touch /usr/local/etc/frr/ripngd.conf + sudo touch /usr/local/etc/frr/staticd.conf + sudo touch /usr/local/etc/frr/zebra.conf + sudo chown -R frr:frr /usr/local/etc/frr/ + sudo touch /usr/local/etc/frr/vtysh.conf + sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf + sudo chmod 640 /usr/local/etc/frr/*.conf + +Enable IP & IPv6 forwarding +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add the following lines to the end of ``/etc/sysctl.conf``: + +:: + + # Routing: We need to forward packets + net.inet.ip.forwarding=1 + net.inet6.ip6.forwarding=1 + +**Reboot** or use ``sysctl`` to apply the same config to the running system. diff --git a/doc/developer/building-frr-for-freebsd9.rst b/doc/developer/building-frr-for-freebsd9.rst index 3033287..9f9073d 100644 --- a/doc/developer/building-frr-for-freebsd9.rst +++ b/doc/developer/building-frr-for-freebsd9.rst @@ -70,9 +70,9 @@ an example) export LDFLAGS="-L/usr/local/lib" export CPPFLAGS="-I/usr/local/include" ./configure \ - --sysconfdir=/usr/local/etc/frr \ + --sysconfdir=/usr/local/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --prefix=/usr/local \ --enable-multipath=64 \ --enable-user=frr \ diff --git a/doc/developer/building-frr-for-netbsd6.rst b/doc/developer/building-frr-for-netbsd6.rst index 8958862..77c0e00 100644 --- a/doc/developer/building-frr-for-netbsd6.rst +++ b/doc/developer/building-frr-for-netbsd6.rst @@ -64,9 +64,9 @@ an example) export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib" export CPPFLAGS="-I/usr/pkg/include" ./configure \ - --sysconfdir=/usr/pkg/etc/frr \ + --sysconfdir=/usr/pkg/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --enable-multipath=64 \ --enable-user=frr \ --enable-group=frr \ diff --git a/doc/developer/building-frr-for-netbsd7.rst b/doc/developer/building-frr-for-netbsd7.rst index e751ba3..abb04a0 100644 --- a/doc/developer/building-frr-for-netbsd7.rst +++ b/doc/developer/building-frr-for-netbsd7.rst @@ -55,9 +55,9 @@ an example) export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib" export CPPFLAGS="-I/usr/pkg/include" ./configure \ - --sysconfdir=/usr/pkg/etc/frr \ + --sysconfdir=/usr/pkg/etc \ + --localstatedir=/var \ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \ - --localstatedir=/var/run/frr \ --enable-multipath=64 \ --enable-user=frr \ --enable-group=frr \ diff --git a/doc/developer/building-frr-for-openbsd6.rst b/doc/developer/building-frr-for-openbsd6.rst index 00bc2e5..6d7f346 100644 --- a/doc/developer/building-frr-for-openbsd6.rst +++ b/doc/developer/building-frr-for-openbsd6.rst @@ -71,8 +71,8 @@ an example) export LDFLAGS="-L/usr/local/lib" export CPPFLAGS="-I/usr/local/include" ./configure \ - --sysconfdir=/etc/frr \ - --localstatedir=/var/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --enable-multipath=64 \ --enable-user=_frr \ --enable-group=_frr \ diff --git a/doc/developer/building-frr-for-opensuse.rst b/doc/developer/building-frr-for-opensuse.rst index 3ff445b..6e9913d 100644 --- a/doc/developer/building-frr-for-opensuse.rst +++ b/doc/developer/building-frr-for-opensuse.rst @@ -13,11 +13,13 @@ Installing Dependencies zypper in git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig libjson-c-devel\ pam-devel python3-pytest bison flex c-ares-devel python3-devel\ - python3-Sphinx perl patch libcap-devel libyang-devel \ + python3-Sphinx perl patch libcap-devel \ libelf-devel libunwind-devel protobuf-c .. include:: building-libunwind-note.rst +.. include:: building-libyang.rst + Building & Installing FRR ------------------------- diff --git a/doc/developer/building-frr-for-ubuntu1404.rst b/doc/developer/building-frr-for-ubuntu1404.rst index cc6c3c0..dd3f98a 100644 --- a/doc/developer/building-frr-for-ubuntu1404.rst +++ b/doc/developer/building-frr-for-ubuntu1404.rst @@ -14,16 +14,11 @@ Installing Dependencies git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ libc-ares-dev python3-dev python3-sphinx install-info build-essential \ + protobuf-c-compiler libprotobuf-c-dev \ libsnmp-dev perl libcap-dev libelf-dev .. include:: building-libyang.rst -Protobuf -^^^^^^^^ - -.. code-block:: console - - sudo apt-get install protobuf-c-compiler libprotobuf-c-dev Building & Installing FRR ------------------------- diff --git a/doc/developer/building-frr-for-ubuntu1604.rst b/doc/developer/building-frr-for-ubuntu1604.rst index e5c2389..f3b6aa0 100644 --- a/doc/developer/building-frr-for-ubuntu1604.rst +++ b/doc/developer/building-frr-for-ubuntu1604.rst @@ -19,12 +19,6 @@ Installing Dependencies .. include:: building-libyang.rst -Protobuf -^^^^^^^^ - -.. code-block:: console - - sudo apt-get install protobuf-c-compiler libprotobuf-c-dev Building & Installing FRR ------------------------- diff --git a/doc/developer/building-frr-for-ubuntu1804.rst b/doc/developer/building-frr-for-ubuntu1804.rst index fcfd94e..b4880e2 100644 --- a/doc/developer/building-frr-for-ubuntu1804.rst +++ b/doc/developer/building-frr-for-ubuntu1804.rst @@ -15,18 +15,13 @@ Installing Dependencies pkg-config libpam0g-dev libjson-c-dev bison flex \ libc-ares-dev python3-dev python3-sphinx \ install-info build-essential libsnmp-dev perl libcap-dev \ + protobuf-c-compiler libprotobuf-c-dev \ libelf-dev libunwind-dev .. include:: building-libunwind-note.rst .. include:: building-libyang.rst -Protobuf -^^^^^^^^ - -.. code-block:: console - - sudo apt-get install protobuf-c-compiler libprotobuf-c-dev ZeroMQ ^^^^^^ diff --git a/doc/developer/building-frr-for-ubuntu2004.rst b/doc/developer/building-frr-for-ubuntu2004.rst index fdfc25d..3db97c4 100644 --- a/doc/developer/building-frr-for-ubuntu2004.rst +++ b/doc/developer/building-frr-for-ubuntu2004.rst @@ -15,34 +15,33 @@ Installing Dependencies pkg-config libpam0g-dev libjson-c-dev bison flex \ libc-ares-dev python3-dev python3-sphinx \ install-info build-essential libsnmp-dev perl \ - libcap-dev python2 libelf-dev libunwind-dev + protobuf-c-compiler libprotobuf-c-dev \ + libcap-dev libelf-dev libunwind-dev .. include:: building-libunwind-note.rst -Note that Ubuntu 20 no longer installs python 2.x, so it must be -installed explicitly. Ensure that your system has a symlink named -``/usr/bin/python`` pointing at ``/usr/bin/python3``. +.. include:: building-libyang.rst -In addition, ``pip`` for python2 must be installed if you wish to run -the FRR topotests. That version of ``pip`` is not available from the -ubuntu apt repositories; in order to install it: +GRPC +^^^^ +If GRPC is enabled using ``--enable-grpc`` the following packages should be +installed. -.. code-block:: shell +.. code-block:: console - curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py - sudo python2 ./get-pip.py + sudo apt-get install libgrpc++-dev protobuf-compiler-grpc - # And verify the installation - pip2 --version -.. include:: building-libyang.rst +Config Rollbacks +^^^^^^^^^^^^^^^^ -Protobuf -^^^^^^^^ +If config rollbacks are enabled using ``--enable-config-rollbacks`` +the sqlite3 developer package also should be installed. .. code-block:: console - sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + sudo apt install libsqlite3-dev + ZeroMQ ^^^^^^ diff --git a/doc/developer/building-frr-for-ubuntu2204.rst b/doc/developer/building-frr-for-ubuntu2204.rst index 97bdf88..c898c3c 100644 --- a/doc/developer/building-frr-for-ubuntu2204.rst +++ b/doc/developer/building-frr-for-ubuntu2204.rst @@ -15,40 +15,21 @@ Installing Dependencies pkg-config libpam0g-dev libjson-c-dev bison flex \ libc-ares-dev python3-dev python3-sphinx \ install-info build-essential libsnmp-dev perl \ - libcap-dev python2 libelf-dev libunwind-dev \ - libyang2 libyang2-dev + libcap-dev libelf-dev libunwind-dev \ + protobuf-c-compiler libprotobuf-c-dev .. include:: building-libunwind-note.rst -Note that Ubuntu >= 20 no longer installs python 2.x, so it must be -installed explicitly. Ensure that your system has a symlink named -``/usr/bin/python`` pointing at ``/usr/bin/python3``. +.. include:: building-libyang.rst -.. code-block:: shell - - sudo ln -s /usr/bin/python3 /usr/bin/python - python --version - -In addition, ``pip`` for python2 must be installed if you wish to run -the FRR topotests. That version of ``pip`` is not available from the -ubuntu apt repositories; in order to install it: - -.. code-block:: shell - - curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py - sudo python2 ./get-pip.py - - # And verify the installation - pip2 --version - - -Protobuf -^^^^^^^^ -This is optional +GRPC +^^^^ +If GRPC is enabled using ``--enable-grpc`` the following packages should be +installed. .. code-block:: console - sudo apt-get install protobuf-c-compiler libprotobuf-c-dev + sudo apt-get install libgrpc++-dev protobuf-compiler-grpc Config Rollbacks diff --git a/doc/developer/building-libyang.rst b/doc/developer/building-libyang.rst index c36cd34..a46c793 100644 --- a/doc/developer/building-libyang.rst +++ b/doc/developer/building-libyang.rst @@ -10,11 +10,11 @@ The FRR project builds some binary ``libyang`` packages. RPM packages are at our `RPM repository <https://rpm.frrouting.org>`_. DEB packages are available as CI artifacts `here -<https://ci1.netdef.org/browse/LIBYANG-LIBYANGV2/latestSuccessful/artifact>`_. +<https://ci1.netdef.org/browse/LIBYANG-LIBYANG21/latestSuccessful/artifact>`_. .. warning:: - ``libyang`` version 2.0.0 or newer is required to build FRR. + ``libyang`` version 2.1.128 or newer is required to build FRR. .. note:: @@ -39,7 +39,7 @@ DEB packages are available as CI artifacts `here git clone https://github.com/CESNET/libyang.git cd libyang - git checkout v2.0.0 + git checkout v2.1.128 mkdir build; cd build cmake -D CMAKE_INSTALL_PREFIX:PATH=/usr \ -D CMAKE_BUILD_TYPE:String="Release" .. diff --git a/doc/developer/building.rst b/doc/developer/building.rst index 8ca0c13..6762604 100644 --- a/doc/developer/building.rst +++ b/doc/developer/building.rst @@ -21,6 +21,7 @@ Building FRR building-frr-for-freebsd10 building-frr-for-freebsd11 building-frr-for-freebsd13 + building-frr-for-freebsd14 building-frr-for-netbsd6 building-frr-for-netbsd7 building-frr-for-openbsd6 diff --git a/doc/developer/cross-compiling.rst b/doc/developer/cross-compiling.rst index 3bf78f7..af99262 100644 --- a/doc/developer/cross-compiling.rst +++ b/doc/developer/cross-compiling.rst @@ -239,9 +239,9 @@ the last thing to actually build is FRR itself: --host=${HOST_ARCH} \ --with-sysroot=/usr/${HOST_ARCH} \ --with-clippy=./build-clippy/lib/clippy \ - --sysconfdir=/etc/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir="\${prefix}/lib/frr" \ - --localstatedir=/var/run/frr \ --prefix=/usr \ --enable-user=frr \ --enable-group=frr \ diff --git a/doc/developer/frr-release-procedure.rst b/doc/developer/frr-release-procedure.rst index 9378637..9dbc9b4 100644 --- a/doc/developer/frr-release-procedure.rst +++ b/doc/developer/frr-release-procedure.rst @@ -13,6 +13,13 @@ Stage 1 - Preparation Note: use ``tools/release_notes.py`` to help draft release notes changelog + .. code-block:: console + + ./tools/release_notes.py -b dev/9.1 -t frr-9.0.1 + + dev/9.1 is the branch to be renamed to stable/9.1, and frr-9.0.1 in this + example is the latest tag from which to generate the logs. + #. Checkout the existing ``dev/<version>`` branch. .. code-block:: console diff --git a/doc/developer/include-compile.rst b/doc/developer/include-compile.rst index b98d237..49fd6c8 100644 --- a/doc/developer/include-compile.rst +++ b/doc/developer/include-compile.rst @@ -14,8 +14,8 @@ obtained by running ``./configure -h``. The options shown below are examples. --sbindir=\${prefix}/lib/frr \ --libdir=\${prefix}/lib/frr \ --libexecdir=\${prefix}/lib/frr \ - --localstatedir=/var/run/frr \ - --sysconfdir=/etc/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --with-moduledir=\${prefix}/lib/frr/modules \ --enable-configfile-mask=0640 \ --enable-logfile-mask=0640 \ diff --git a/doc/developer/mgmtd-dev.rst b/doc/developer/mgmtd-dev.rst index 9839aa8..2404ffe 100644 --- a/doc/developer/mgmtd-dev.rst +++ b/doc/developer/mgmtd-dev.rst @@ -12,7 +12,7 @@ MGMTD Development ================= Overview -^^^^^^^^ +-------- ``mgmtd`` (Management Daemon) is a new centralized management daemon for FRR. @@ -33,18 +33,64 @@ each daemon. ``mgmtd`` currently provides the CLI interface for each daemon that has been converted to it, but in the future RESTCONF and NETCONF servers can easily be added as *front-ends* to mgmtd to support those protocols as well. +Conversion Status +^^^^^^^^^^^^^^^^^ + +Fully Converted To MGMTD +"""""""""""""""""""""""" + +- lib/distribute +- lib/filter +- lib/if_rmap +- lib/routemap +- lib/affinitymap +- lib/if +- lib/vrf +- ripd +- ripngd +- staticd +- zebra (* - partial) + +Converted To Northbound +""""""""""""""""""""""" +- bfdd +- pathd +- pbrd +- pimd + +Converted To Northbound With Issues +""""""""""""""""""""""""""""""""""" +- eigrp +- isisd + +Unconverted +""""""""""" +- babel +- bgpd +- ldpd +- lib/event +- lib/keychain +- lib/log_vty +- lib/nexthop_group +- lib/zlog_5424_cli +- nhrpd +- ospfd +- ospf6d +- pceplib +- qdb +- sharpd +- vrrpd Converting A Daemon to MGMTD -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------------------- -A daemon must first be transitioned to the new *northbound* interface if that -has not already been done (see `this northbound conversion documentation -<https://github.com/opensourcerouting/frr/wiki/Retrofitting-Configuration-Commands>`_ -for how to do this). Once this is done a few simple steps are all that is -required move the daemon over to ``mgmtd`` control. +A daemon must first be transitioned to the new :ref:`northbound` interface if that +has not already been done (see :ref:`nb-retrofit` for how to do this). Once this +is done a few simple steps are all that is required move the daemon over to +``mgmtd`` control. Overview of Changes -------------------- +^^^^^^^^^^^^^^^^^^^ Adding support for a *northbound* converted daemon involves very little work. It requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support. @@ -52,64 +98,126 @@ requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support. Front-End Interface: -1. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.c``) -2. Add YANG module description into array defined in ``mgmtd/mgmt_main.c`` -3. Add CLI handler file[s] to ``mgmtd/subdir.am`` (e.g., ``staticd/static_vty.c``) -4. [if needed] Exclude (#ifndef) non-configuration CLI handlers from CLI source - file (e.g., inside ``staticd/static_vty.c``) +#. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.yang.c``). + +#. Add CLI handler file[s] to ``mgmtd/subdir.am``. The `subdir.am` variable to + use is indicated in the next 2 steps. + + #. [if needed] Exclude (:code:`#ifndef`) non-configuration CLI handlers from + CLI source file (e.g., inside :file:`staticd/static_vty.c`) and add the + file to :code:`nodist_mgmtd_libmgmt_be_nb_la_SOURCES` in + :file:`mgmtd/subdir.am`. + + #. [otherwise] Remove CLI handler file from _SOURCES variable in the daemon + :file:`subdir.am` file (e.g in :file:`staticd/subdir.am`) and add to + :code:`mgmtd_libmgmtd_a_SOURCES` in :file:`mgmtd/subdir.am`. + +#. In order to have mgmtd try and load existing per-daemon config files, add + the daemon to the :code:`mgmt_daemons` array in :file:`lib/vty.c`. With the + official release of the mgmtd code FRR is no longer supporting per daemon log + files but it will take a while before all of the topotest is converted. + +#. In the daemon's :code:`struct frr_daemon_info` (i.e., inside it's + :code:`FRR_DAEMON_INFO()`) set the `.flags` bit `FRR_NO_SPLIT_CONFIG`. This + will keep the daemon from trying to read it's per-daemon config file as mgmtd + will now be doing this. + +#. Add the daemon's YANG module description[s] into the array + :code:`mgmt_yang_modules` defined in :file:`mgmtd/mgmt_main.c` (see + :ref:`mgmtd-config-write`). Make sure that all YANG modules that the daemon + uses are present in the mgmtd list. To find this list look in the daemon's + equivalent yang module array variable. + +#. Initialize the CLI handlers inside :code:`mgmt_vty_init` in :file:`mgmtd/mgmt_vty.c`. + +#. Direct ``vtysh`` to send CLI commands to ``mgmtd`` by modifying + ``vtysh/vtysh.h``. At the top of this file each daemon has a bit + ``#define``'d (e.g., ``#define VTYSH_STATICD 0x08000``) below this there are + groupings, replace all the uses of the daemons bit with ``VTYSH_MGMTD`` + instead so that the CLI commands get properly routed to ``mgmtd`` rather than + the daemon now. + + #. Remove initialization (and installation) of library CLI routines. These will + correspond with the VTYSH removals from the last step i.e.,: + + - change access_list_init() to access_list_init_new(false) and remove from + VTYSH_ACL_CONFIG (leave in VTYSH_ACL_SHOW). + - remove if_cmd_init_default() => remove from VTYSH_INTERFACE_SUBSET + - remove if_cmd_init() => remove from VTYSH_INTERFACE_SUBSET + - change route_map_init() to route_map_init_new(false) and remove from + VTYSH_ROUTE_MAP_CONFIG (leave in VTYSH_ROUTE_MAP_SHOW). + - remove vrf_cmd_init(NULL) => remove from VTYSH_INTERFACE_SUBSET + ... Back-End Interface: -5. Add XPATHs mappings to a couple arrays to direct ``mgmtd`` at your daemon in - ``mgmtd/mgmt_be_adapter.c`` - +#. In the daemon's main file initialize the BE client library. You add a global + `struct mgmt_be_client *mgmt_be_client` near the daemons `event_loop *master` + variable. Then where the daemon used to initialize it's CLI/VTY code replace + that with the client initialization by calling `mgmt_be_client_create`. + Likewise in the daemon's sigint cleanup code, operational walks should be + canceled with a call to `nb_oper_cancel_all_walks`, and then the BE client + should be destroyed with a call to `mgmt_be_client_destroy` and to be safe + NULL out the global `mgmt_be_client` variable. + +#. In ``mgmtd/mgmt_be_adapter.c`` add xpath prefix mappings to a one or both + mapping arrays (``be_client_config_xpaths`` and ``be_client_oper_xpaths``) to + direct ``mgmtd`` to send config and oper-state requests to your daemon. NOTE: + make sure to include library supported xpaths prefixes as well (e.g., + "/frr-interface:lib"). A good way to figure these paths out are to look in + each of the YANG modules that the daemon uses and include each of their paths + in the array. Add YANG and CLI into MGMTD ---------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ As an example here is the addition made to ``mgmtd/subdir.am`` for adding ``staticd`` support. .. code-block:: make - if STATICD - nodist_mgmtd_mgmtd_SOURCES += \ + if STATICD + nodist_mgmtd_mgmtd_SOURCES += \ yang/frr-staticd.yang.c \ yang/frr-bfdd.yang.c \ # end - nodist_mgmtd_libmgmt_be_nb_la_SOURCES += staticd/static_vty.c - endif + nodist_mgmtd_libmgmt_be_nb_la_SOURCES += staticd/static_vty.c + endif An here is the addition to the modules array in ``mgmtd/mgmt_main.c``: .. code-block:: c - static const struct frr_yang_module_info *const mgmt_yang_modules[] = { + #ifdef HAVE_STATICD + extern const struct frr_yang_module_info frr_staticd_info; + #endif + + static const struct frr_yang_module_info *const mgmt_yang_modules[] = { &frr_filter_info, ... - #ifdef HAVE_STATICD - &(struct frr_yang_module_info){.name = "frr-staticd", - .ignore_cbs = true}, - #endif - } + #ifdef HAVE_STATICD + &frr_staticd_info, + #endif + } -CLI Handlers ------------- +CLI Config and Show Handlers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The daemon's CLI handlers for configuration (which having been converted to -*northbound* now simply generate YANG changes) will be linked directly into +The daemon's CLI handlers for configuration (which having been converted to the +:ref:`northbound` now simply generate YANG changes) will be linked directly into ``mgmtd``. If the operational and debug CLI commands are kept in files separate from the daemon's configuration CLI commands then no extra work is required. Otherwise some CPP #ifndef's will be required. -Currently ``mgmtd`` supports configuration CLI but not operational -state so if both types of CLI handlers are present in a single file (e.g. a -``xxx_vty.c`` or ``xxx_cli.c`` file ) then #ifndef will be used to exclude these -non-configuration CLI handlers from ``mgmtd``. The same goes for *debug* CLI -handlers. For example: +``mgmtd`` supports both config and operational state. However, many +daemons have not had their operational state CLI commands converted over to the +new YANG based methods. If that is the case and if both types of CLI handlers +are present in a single file (e.g. a ``xxx_vty.c`` or ``xxx_cli.c`` file) then +:code:`#ifndef` will need to be used to exclude the non-config CLI handlers from +``mgmtd``. The same goes for unconverted *debug* CLI handlers. For example: .. code-block:: c @@ -121,7 +229,7 @@ handlers. For example: } #ifndef INCLUDE_MGMTD_CMDDEFS_ONLY - DEFPY(daemon_show_oepr, daemon_show_oepr_cmd, + DEFPY(daemon_show_oper, daemon_show_oper_cmd, "show daemon oper [all]" ... { @@ -140,83 +248,136 @@ handlers. For example: } +.. _mgmtd-config-write: + +CLI Config Write Handlers (:code:`cli_show`) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To support writing out the CLI configuration file the northbound API defines a +2 callbacks (:code:`cli_show` and :code:`cli_show_end`). Pointers to these +callbacks used to live side-by-side in a daemons :code:`struct frr_yang_module_info`, +with the daemons back-end configuration and operational state callbacks +(normally in a file named `<daemon>_nb.c`). + +However, these 2 functionalities need to be split up now. The *frontend* config +writing callbacks (:code:`cli_show`) should now be linked into ``mgmtd`` while +the *backend* config and oper-state callbacks (e.g., :code:`create`, +:code:`modify`, etc) should continue to be linked into the daemon. + +So you will need to define 2 :code:`struct frr_yang_module_info` arrays. + +#. The existing array remains in the same place in the daemon, but with all the + :code:`cli_show` handlers removed. + +#. The removed :code:`cli_show` handlers should be added to a new + :code:`struct frr_yang_module_info` array. This second array should be + included in the same file that includes that actual function pointed to by + the the :code:`cli_show` callbacks (i.e., the file is compiled into + ``mgmtd``). + + This new :code:`struct frr_yang_module_info` array is the one to be included + in mgmtd in `mgmt_yang_modules` inside ``mgmtd/mgmt_main.c``. + +Back-End Client Connection +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In order for your daemon to communicate with mgmtd you need to initialize the +backend client library. You normally do this where you used to initialize your +CLI/VTY code. + +.. code-block:: c + + ... + struct event_loop *master; -Add Back-End XPATH mappings ---------------------------- + static struct mgmt_be_client *mgmt_be_client; + ... + + int main(int argc, char **argv) + { + ... + rip_init(); + rip_if_init(); + mgmt_be_client = mgmt_be_client_create("ripd", NULL, 0, master); + +Likewise the client should be cleaned up in the daemon cleanup routine. + +.. code-block:: c + + /* SIGINT handler. */ + static void sigint(void) + { + zlog_notice("Terminating on signal"); + ... + nb_oper_cancel_all_walks(); + mgmt_be_client_destroy(mgmt_be_client); + mgmt_be_client = NULL; + + +Back-End XPATH mappings +^^^^^^^^^^^^^^^^^^^^^^^ In order for ``mgmtd`` to direct configuration to your daemon you need to add some XPATH mappings to ``mgmtd/mgmt_be_adapter.c``. These XPATHs determine which configuration changes get sent over the *back-end* interface to your daemon. +There are 2 arrays to update the first for config support and the second for +operational state. -Below are the strings added for staticd support: +Below are the strings added for staticd config support: .. code-block:: c - static const struct mgmt_be_xpath_map_init mgmt_xpath_map_init[] = { - { - .xpath_regexp = "/frr-vrf:lib/*", - .subscr_info = - { - #if HAVE_STATICD - [MGMTD_BE_CLIENT_ID_STATICD] = - MGMT_SUBSCR_VALIDATE_CFG | - MGMT_SUBSCR_NOTIFY_CFG, - #endif - }, - }, - ... - { - .xpath_regexp = - "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*", - .subscr_info = - { - #if HAVE_STATICD - [MGMTD_BE_CLIENT_ID_STATICD] = - MGMT_SUBSCR_VALIDATE_CFG | - MGMT_SUBSCR_NOTIFY_CFG, - #endif - }, - }, - }; - - #if HAVE_STATICD - static struct mgmt_be_client_xpath staticd_xpaths[] = { - { - .xpath = "/frr-vrf:lib/*", - .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG, - }, - ... - { - .xpath = - "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*", - .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG, - }, - }; - #endif - - static struct mgmt_be_client_xpath_map - mgmt_client_xpaths[MGMTD_BE_CLIENT_ID_MAX] = { - #ifdef HAVE_STATICD - [MGMTD_BE_CLIENT_ID_STATICD] = {staticd_xpaths, - array_size(staticd_xpaths)}, - #endif - }; + #if HAVE_STATICD + static const char *const staticd_xpaths[] = { + "/frr-vrf:lib", + "/frr-interface:lib", + "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd", + NULL, + }; + #endif + + static const char *const *be_client_xpaths[MGMTD_BE_CLIENT_ID_MAX] = { + #ifdef HAVE_STATICD + [MGMTD_BE_CLIENT_ID_STATICD] = staticd_xpaths, + #endif + }; +Below are the strings added for zebra operational state support (note zebra is +not conditionalized b/c it should always be present): + +.. code-block:: c + + static const char *const zebra_oper_xpaths[] = { + "/frr-interface:lib/interface", + "/frr-vrf:lib/vrf/frr-zebra:zebra", + "/frr-zebra:zebra", + NULL, + }; + + static const char *const *be_client_oper_xpaths[MGMTD_BE_CLIENT_ID_MAX] = { + [MGMTD_BE_CLIENT_ID_ZEBRA] = zebra_oper_xpaths, + }; MGMTD Internals -^^^^^^^^^^^^^^^ +--------------- This section will describe the internal functioning of ``mgmtd``, for now a couple diagrams are included to aide in source code perusal. -The client side of a CLI change +The client side of a CLI configuration change .. figure:: ../figures/cli-change-client.svg :align: center -The server (mgmtd) side of a CLI change +The server (mgmtd) side of a CLI configuration change .. figure:: ../figures/cli-change-mgmtd.svg :align: center + + +The client and server sides of oper-state query + +.. figure:: ../figures/cli-oper-state.svg + :align: center diff --git a/doc/developer/northbound/advanced-topics.rst b/doc/developer/northbound/advanced-topics.rst index bee29a9..edfc10b 100644 --- a/doc/developer/northbound/advanced-topics.rst +++ b/doc/developer/northbound/advanced-topics.rst @@ -1,3 +1,11 @@ +Advanced Topics +=============== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Auto-generated CLI commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -34,13 +42,16 @@ CLI on a separate program The flexible design of the northbound architecture opens the door to move the CLI to a separate program in the long-term future. Some -advantages of doing so would be: \* Treat the CLI as just another -northbound client, instead of having CLI commands embedded in the -binaries of all FRR daemons. \* Improved robustness: bugs in CLI -commands (e.g. null-pointer dereferences) or in the CLI code itself -wouldn’t affect the FRR daemons. \* Foster innovation by allowing other -CLI programs to be implemented, possibly using higher level programming -languages. +advantages of doing so would be: + +* Treat the CLI as just another northbound client, instead of having CLI + commands embedded in the binaries of all FRR daemons. + +* Improved robustness: bugs in CLI commands (e.g. null-pointer dereferences) or + in the CLI code itself wouldn’t affect the FRR daemons. + +* Foster innovation by allowing other CLI programs to be implemented, possibly + using higher level programming languages. The problem, however, is that the northbound retrofitting process will convert only the CLI configuration commands and EXEC commands in a first @@ -232,40 +243,42 @@ vtysh support As explained in the [[Transactional CLI]] page, all commands introduced by the transactional CLI are not yet available in *vtysh*. This needs to be addressed in the short term future. Some challenges for doing that -work include: \* How to display configurations (running, candidates and -rollbacks) in a more clever way? The implementation of the -``show running-config`` command in *vtysh* is not something that should -be followed as an example. A better idea would be to fetch the desired -configuration from all daemons (encoded in JSON for example), merge them -all into a single ``lyd_node`` variable and then display the combined -configurations from this variable (the configuration merges would -transparently take care of combining the shared configuration objects). -In order to be able to manipulate the JSON configurations, *vtysh* will -need to load the YANG modules from all daemons at startup (this might -have a minimal impact on startup time). The only issue with this -approach is that the ``cli_show()`` callbacks from all daemons are -embedded in their binaries and thus not accessible externally. It might -be necessary to compile these callbacks on a separate shared library so -that they are accessible to *vtysh* too. Other than that, displaying the -combined configurations in the JSON/XML formats should be -straightforward. \* With the current design, transaction IDs are -per-daemon and not global across all FRR daemons. This means that the -same transaction ID can represent different transactions on different -daemons. Given this observation, how to implement the -``rollback configuration`` command in *vtysh*? The easy solution would -be to add a ``daemon WORD`` argument to specify the context of the -rollback, but per-daemon rollbacks would certainly be confusing and -convoluted to end users. A better idea would be to attack the root of -the problem: change configuration transactions to be global instead of -being per-daemon. This involves a bigger change in the northbound -architecture, and would have implications on how transactions are stored -in the SQL database (daemon-specific and shared configuration objects -would need to have their own tables or columns). \* Loading -configuration files in the JSON or XML formats will be tricky, as -*vtysh* will need to know which sections of the configuration should be -sent to which daemons. *vtysh* will either need to fetch the YANG -modules implemented by all daemons at runtime or obtain this information -at compile-time somehow. +work include: + +* How to display configurations (running, candidates and rollbacks) in a more + clever way? The implementation of the ``show running-config`` command in + *vtysh* is not something that should be followed as an example. A better idea + would be to fetch the desired configuration from all daemons (encoded in JSON + for example), merge them all into a single ``lyd_node`` variable and then + display the combined configurations from this variable (the configuration + merges would transparently take care of combining the shared configuration + objects). In order to be able to manipulate the JSON configurations, *vtysh* + will need to load the YANG modules from all daemons at startup (this might + have a minimal impact on startup time). The only issue with this approach is + that the ``cli_show()`` callbacks from all daemons are embedded in their + binaries and thus not accessible externally. It might be necessary to compile + these callbacks on a separate shared library so that they are accessible to + *vtysh* too. Other than that, displaying the combined configurations in the + JSON/XML formats should be straightforward. + +* With the current design, transaction IDs are per-daemon and not global across + all FRR daemons. This means that the same transaction ID can represent + different transactions on different daemons. Given this observation, how to + implement the ``rollback configuration`` command in *vtysh*? The easy solution + would be to add a ``daemon WORD`` argument to specify the context of the + rollback, but per-daemon rollbacks would certainly be confusing and convoluted + to end users. A better idea would be to attack the root of the problem: change + configuration transactions to be global instead of being per-daemon. This + involves a bigger change in the northbound architecture, and would have + implications on how transactions are stored in the SQL database + (daemon-specific and shared configuration objects would need to have their own + tables or columns). + +* Loading configuration files in the JSON or XML formats will be tricky, as + *vtysh* will need to know which sections of the configuration should be sent + to which daemons. *vtysh* will either need to fetch the YANG modules + implemented by all daemons at runtime or obtain this information at + compile-time somehow. Detecting type mismatches at compile-time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/developer/northbound/architecture.rst b/doc/developer/northbound/architecture.rst index e571971..5fd89c3 100644 --- a/doc/developer/northbound/architecture.rst +++ b/doc/developer/northbound/architecture.rst @@ -1,3 +1,6 @@ +Architecture +============ + Introduction ------------ @@ -42,30 +45,34 @@ and `CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__. In addition to being management-protocol independent, some other -advantages of using YANG in FRR are listed below: \* Have a formal -contract between FRR and application developers (management clients). A -management client that has access to the FRR YANG models knows about all -existing configuration options available for use. This information can -be used to auto-generate user-friendly interfaces like Web-UIs, custom -CLIs and even code bindings for several different programming languages. -Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for -example, it’s possible to generate Python class hierarchies from YANG -models and use these classes to instantiate objects that mirror the -structure of the YANG modules and can be serialized/deserialized using -different encoding formats. \* Support different encoding formats for -instance data. Currently only JSON and XML are supported, but -`GPB <https://developers.google.com/protocol-buffers/>`__ and -`CBOR <http://cbor.io/>`__ are other viable options in the long term. -Additional encoding formats can be implemented in the *libyang* library -for optimal performance, or externally by translating data to/from one -of the supported formats (with a performance penalty). \* Have a formal -mechanism to introduce backward-incompatible changes based on `semantic -versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the -YANG standard, which allows backward-compatible module updates only). \* -Provide seamless support to the industry-standard NETCONF/RESTCONF -protocols as alternative management APIs. If FRR configuration/state -data is modeled using YANG, supporting YANG-based protocols like NETCONF -and RESTCONF is much easier. +advantages of using YANG in FRR are listed below: + +* Have a formal contract between FRR and application developers (management + clients). A management client that has access to the FRR YANG models knows + about all existing configuration options available for use. This information + can be used to auto-generate user-friendly interfaces like Web-UIs, custom + CLIs and even code bindings for several different programming languages. Using + `PyangBind <https://github.com/robshakir/pyangbind>`__, for example, it’s + possible to generate Python class hierarchies from YANG models and use these + classes to instantiate objects that mirror the structure of the YANG modules + and can be serialized/deserialized using different encoding formats. + +* Support different encoding formats for instance data. Currently only JSON and + XML are supported, but `GPB + <https://developers.google.com/protocol-buffers/>`__ and `CBOR + <http://cbor.io/>`__ are other viable options in the long term. Additional + encoding formats can be implemented in the *libyang* library for optimal + performance, or externally by translating data to/from one of the supported + formats (with a performance penalty). + +* Have a formal mechanism to introduce backward-incompatible changes based on + `semantic versioning <http://www.openconfig.net/docs/semver/>`__ (not part of + the YANG standard, which allows backward-compatible module updates only). + +* Provide seamless support to the industry-standard NETCONF/RESTCONF protocols + as alternative management APIs. If FRR configuration/state data is modeled + using YANG, supporting YANG-based protocols like NETCONF and RESTCONF is much + easier. As important as shifting to a model-driven management paradigm, the new northbound architecture also introduces the concept of configuration diff --git a/doc/developer/northbound/demos.rst b/doc/developer/northbound/demos.rst index 876bd25..8a0f6ad 100644 --- a/doc/developer/northbound/demos.rst +++ b/doc/developer/northbound/demos.rst @@ -1,3 +1,6 @@ +Demos +===== + Transactional CLI ----------------- diff --git a/doc/developer/northbound/links.rst b/doc/developer/northbound/links.rst index e80374c..6cec176 100644 --- a/doc/developer/northbound/links.rst +++ b/doc/developer/northbound/links.rst @@ -1,3 +1,6 @@ +Links +===== + RFCs ~~~~ diff --git a/doc/developer/northbound/northbound.rst b/doc/developer/northbound/northbound.rst index 7dddf06..c5f4e2f 100644 --- a/doc/developer/northbound/northbound.rst +++ b/doc/developer/northbound/northbound.rst @@ -11,11 +11,11 @@ Northbound API transactional-cli retrofitting-configuration-commands operational-data-rpcs-and-notifications - plugins-sysrepo advanced-topics yang-tools yang-module-translator demos links + plugins-sysrepo ppr-basic-test-topology ppr-mpls-basic-test-topology diff --git a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst index 554bc17..5cb70ca 100644 --- a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst +++ b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst @@ -1,3 +1,11 @@ +Operational Data, RPCs and Notifications +======================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Operational data ~~~~~~~~~~~~~~~~ @@ -330,10 +338,12 @@ CLI can take too long, potentially long enough to the point of triggering some protocol timeouts and bringing sessions down. To avoid this kind of problem, northbound clients are encouraged to do -one of the following: \* Create a separate pthread for handling requests -to fetch operational data. \* Iterate over YANG lists and leaf-lists -asynchronously, returning a maximum number of elements per time instead -of returning all elements in one shot. +one of the following: + +* Create a separate pthread for handling requests to fetch operational data. + +* Iterate over YANG lists and leaf-lists asynchronously, returning a maximum + number of elements per time instead of returning all elements in one shot. In order to handle both cases correctly, the ``get_next`` callbacks need to use locks to prevent the YANG lists from being modified while they diff --git a/doc/developer/northbound/plugins-sysrepo.rst b/doc/developer/northbound/plugins-sysrepo.rst index 186c3a0..0cfdb82 100644 --- a/doc/developer/northbound/plugins-sysrepo.rst +++ b/doc/developer/northbound/plugins-sysrepo.rst @@ -1,137 +1,193 @@ +Plugins Sysrepo +=============== + Installation ------------ Required dependencies ^^^^^^^^^^^^^^^^^^^^^ +Install FRR build required dependencies, check `Building FRR +<https://docs.frrouting.org/projects/dev-guide/en/latest/building.html>`_ document for specific platform required packages. +Below are debian systems required packages: -:: +.. code-block:: console - # apt-get install git cmake build-essential bison flex libpcre3-dev libev-dev \ - libavl-dev libprotobuf-c-dev protobuf-c-compiler libcmocka0 \ - libcmocka-dev doxygen libssl-dev libssl-dev libssh-dev + sudo apt-get install git autoconf automake libtool make \ + libprotobuf-c-dev protobuf-c-compiler build-essential \ + python3-dev python3-pytest python3-sphinx libjson-c-dev \ + libelf-dev libreadline-dev cmake libcap-dev bison flex \ + pkg-config texinfo gdb libgrpc-dev python3-grpc-tools libpcre2-dev libyang ^^^^^^^ -:: +.. note:: - # apt-get install libyang0.16 libyang-dev + FRR requires version 2.1.128 or newer, in this document we will + be compiling and installing libyang version 2.1.148. -Sysrepo -^^^^^^^ +.. code-block:: console -:: + git clone https://github.com/CESNET/libyang.git + cd libyang + git checkout v2.1.148 + mkdir build; cd build + cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DCMAKE_BUILD_TYPE:String="Release" .. + make + sudo make install - $ git clone https://github.com/sysrepo/sysrepo.git - $ cd sysrepo/ - $ mkdir build; cd build - $ cmake -DCMAKE_BUILD_TYPE=Release -DGEN_LANGUAGE_BINDINGS=OFF .. && make - # make install +Sysrepo +^^^^^^^ -libnetconf2 -^^^^^^^^^^^ +.. note:: -:: + The following code block assumes you have installed libyang v2.1.148, if you have + libyang v2.1.128 change sysrepo version to 2.2.105. - $ git clone https://github.com/CESNET/libnetconf2.git - $ cd libnetconf2/ - $ mkdir build; cd build - $ cmake .. && make - # make install +.. code-block:: console -netopeer2 -^^^^^^^^^ + git clone https://github.com/sysrepo/sysrepo.git + cd sysrepo/ + git checkout v2.2.150 + mkdir build; cd build + cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DCMAKE_BUILD_TYPE:String="Release" .. + make + sudo make install -:: +Verify that sysrepo is installed correctly: - $ git clone https://github.com/CESNET/Netopeer2.git - $ cd Netopeer2 - $ cd server - $ mkdir build; cd build - $ cmake .. && make - # make install +.. code-block:: console -**Note:** If ``make install`` fails as it can’t find -``libsysrepo.so.0.7``, then run ``ldconfig`` and try again as it might -not have updated the lib search path + sudo sysrepoctl -l FRR ^^^ -Build and install FRR using the ``--enable-sysrepo`` configure-time -option. +Follow the steps of `Building FRR +<https://docs.frrouting.org/projects/dev-guide/en/latest/building.html>`_ + + +Make sure to use ``--enable-sysrepo`` configure-time option while building FRR. + +Below is an example of frr configure-time options, your options +might vary, however in order to allow sysrepo plugin you have +to keep ``--enable-sysrepo`` option: + +.. code-block:: console + + ./bootstrap.sh + ./configure \ + --localstatedir=/var/opt/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --enable-sysrepo \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + Initialization -------------- -Install the FRR YANG modules in the Sysrepo datastore: +Install FRR YANG modules in Sysrepo datastore: -:: +.. code-block:: console - # sysrepoctl --install /usr/local/share/yang/ietf-interfaces@2018-01-09.yang - # sysrepoctl --install /usr/local/share/yang/frr-vrf.yang - # sysrepoctl --install /usr/local/share/yang/frr-interface.yang - # sysrepoctl --install /usr/local/share/yang/frr-route-types.yang - # sysrepoctl --install /usr/local/share/yang/frr-filter.yang - # sysrepoctl --install /usr/local/share/yang/frr-route-map.yang - # sysrepoctl --install /usr/local/share/yang/frr-isisd.yang - # sysrepoctl --install /usr/local/share/yang/frr-ripd.yang - # sysrepoctl --install /usr/local/share/yang/frr-ripngd.yang - # sysrepoctl -c frr-vrf --owner frr --group frr - # sysrepoctl -c frr-interface --owner frr --group frr - # sysrepoctl -c frr-route-types --owner frr --group frr - # sysrepoctl -c frr-filter --owner frr --group frr - # sysrepoctl -c frr-route-map --owner frr --group frr - # sysrepoctl -c frr-isisd --owner frr --group frr - # sysrepoctl -c frr-ripd --owner frr --group frr - # sysrepoctl -c frr-ripngd --owner frr --group frr + cd frr/yang/ + sudo sysrepoctl -i ./ietf/ietf-interfaces.yang -o frr -g frr + sudo sysrepoctl -i frr-vrf.yang -o frr -g frr + sudo sysrepoctl -i frr-interface.yang -o frr -g frr + sudo sysrepoctl -i frr-route-types.yang -o frr -g frr + sudo sysrepoctl -i frr-filter.yang -o frr -g frr + sudo sysrepoctl -i frr-route-map.yang -o frr -g frr + sudo sysrepoctl -i frr-isisd.yang -o frr -g frr + sudo sysrepoctl -i frr-bfdd.yang -o frr -g frr + sudo sysrepoctl -i ./ietf/ietf-routing-types.yang -o frr -g frr + sudo sysrepoctl -i frr-nexthop.yang -o frr -g frr + sudo sysrepoctl -i frr-if-rmap.yang -o frr -g frr + sudo sysrepoctl -i frr-ripd.yang -o frr -g frr + sudo sysrepoctl -i frr-ripngd.yang -o frr -g frr + sudo sysrepoctl -i frr-affinity-map.yang -o frr -g frr + sudo sysrepoctl -i ./ietf/frr-deviations-ietf-interfaces.yang -o frr -g frr -Start netopeer2-server: -:: +Start FRR daemons with sysrepo plugin: - # netopeer2-server -d & +.. code-block:: console -Start the FRR daemons with the sysrepo module: + sudo /usr/lib/frr/isisd -M sysrepo --log stdout -:: - - # isisd -M sysrepo --log=stdout +Any daemon running with ``-M sysrepo`` will subscribe to its frr yang moduels +on sysrepo and you be able to configure it by editing module configuration on sysrepo. Managing the configuration -------------------------- -The following NETCONF scripts can be used to show and edit the FRR -configuration: -https://github.com/rzalamena/ietf-hackathon-brazil-201907/tree/master/netconf-scripts +Testing +^^^^^^^ -Example: +To test FRR intergartion with sysrepo, ``sysrepocfg`` tool can be used +to edit frr configuration on sysrepo -:: +Example: - # ./netconf-edit.py 127.0.0.1 - # ./netconf-get-config.py 127.0.0.1 - <?xml version="1.0" encoding="UTF-8"?><data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><isis xmlns="http://frrouting.org/yang/isisd"><instance><area-tag>testnet</area-tag><is-type>level-1</is-type></instance></isis></data> +Edit sysrepo running datastore configuration for the desiged frr module: -.. +.. code-block:: console - NOTE: the ncclient library needs to be installed first: - ``apt install -y python3-ncclient`` + sudo sysrepocfg -E nano -d running -m frr-isisd -f json -The *sysrepocfg* tool can also be used to show/edit the FRR -configuration. Example: +Paste the following json configuration: -:: +.. code-block:: console - # sysrepocfg --format=json --import=frr-isisd.json --datastore=running frr-isisd - # sysrepocfg --format=json --export --datastore=running frr-isisd { "frr-isisd:isis": { "instance": [ { "area-tag": "testnet", + "vrf": "default", "is-type": "level-1" } ] } } + +Exit and save config to the same file. + +After that, this configuration should get reflected to vtysh: + +.. code-block:: console + + show run + Building configuration... + + Current configuration: + ! + frr version 9.2-dev-MyOwnFRRVersion + frr defaults traditional + hostname bullseye + ! + router isis testnet + is-type level-1 + exit + ! + end + +NETCONF +^^^^^^^ + +To manage sysrepo configuration through netconf +you can use `netopeer2 <https://github.com/CESNET/netopeer2>`_ as a netfconf server that can +be easily integrated with sysrepo. diff --git a/doc/developer/northbound/ppr-basic-test-topology.rst b/doc/developer/northbound/ppr-basic-test-topology.rst index a680ed7..4929c9b 100644 --- a/doc/developer/northbound/ppr-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic +=============== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst index cedb795..aceec5f 100644 --- a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic MPLS +==================== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/retrofitting-configuration-commands.rst b/doc/developer/northbound/retrofitting-configuration-commands.rst index b407246..d328be9 100644 --- a/doc/developer/northbound/retrofitting-configuration-commands.rst +++ b/doc/developer/northbound/retrofitting-configuration-commands.rst @@ -1,5 +1,16 @@ + +.. _nb-retrofit: + Retrofitting Configuration Commands ------------------------------------ +=================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 + +Retrofitting process +-------------------- This page explains how to convert existing CLI configuration commands to the new northbound model. This documentation is meant to be the primary @@ -7,9 +18,6 @@ reference for developers working on the northbound retrofitting process. We’ll show several examples taken from the ripd northbound conversion to illustrate some concepts described herein. -Retrofitting process --------------------- - Step 1: writing a YANG module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -30,34 +38,41 @@ possible to facilitate the process of writing module translators using the [[YANG module translator]]. As an example, the frr-ripd YANG module incorporated several parts of the IETF RIP YANG module. The repositories below contain big collections of YANG models that might be used as a -reference: \* https://github.com/YangModels/yang \* -https://github.com/openconfig/public +reference: + +* https://github.com/YangModels/yang + +* https://github.com/openconfig/public When writing a YANG module, it’s highly recommended to follow the guidelines from `RFC 6087 <https://tools.ietf.org/html/rfc6087>`__. In general, most commands should be modeled fairly easy. Here are a few -guidelines specific to authors of FRR YANG models: \* Use -presence-containers or lists to model commands that change the CLI node -(e.g. ``router rip``, ``interface eth0``). This way, if the -presence-container or list entry is removed, all configuration options -below them are removed automatically (exactly like the CLI behaves when -a configuration object is removed using a *no* command). This -recommendation is orthogonal to the `YANG authoring guidelines for -OpenConfig -models <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ -where the use of presence containers is discouraged. OpenConfig YANG -models however were not designed to replicate the behavior of legacy CLI -commands. \* When using YANG lists, be careful to identify what should -be the key leaves. In the ``offset-list WORD <in|out> (0-16) IFNAME`` -command, for example, both the direction (``<in|out>``) and the -interface name should be the keys of the list. This can be only known by -analyzing the data structures used to store the commands. \* For -clarity, use non-presence containers to group leaves that are associated -to the same configuration command (as we’ll see later, this also -facilitate the process of writing ``cli_show`` callbacks). \* YANG -leaves of type *enumeration* should define explicitly the value of each -*enum* option based on the value used in the FRR source code. \* Default -values should be taken from the source code whenever they exist. +guidelines specific to authors of FRR YANG models: + +* Use presence-containers or lists to model commands that change the CLI node + (e.g. ``router rip``, ``interface eth0``). This way, if the presence-container + or list entry is removed, all configuration options below them are removed + automatically (exactly like the CLI behaves when a configuration object is + removed using a *no* command). This recommendation is orthogonal to the `YANG + authoring guidelines for OpenConfig models + <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ + where the use of presence containers is discouraged. OpenConfig YANG models + however were not designed to replicate the behavior of legacy CLI commands. + +* When using YANG lists, be careful to identify what should be the key leaves. + In the ``offset-list WORD <in|out> (0-16) IFNAME`` command, for example, both + the direction (``<in|out>``) and the interface name should be the keys of the + list. This can be only known by analyzing the data structures used to store + the commands. + +* For clarity, use non-presence containers to group leaves that are associated + to the same configuration command (as we’ll see later, this also facilitate + the process of writing ``cli_show`` callbacks). + +* YANG leaves of type *enumeration* should define explicitly the value of each + *enum* option based on the value used in the FRR source code. + +* Default values should be taken from the source code whenever they exist. Some commands are more difficult to model and demand the use of more advanced YANG constructs like *choice*, *when* and *must* statements. @@ -726,15 +741,17 @@ Configuration options are edited individually ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Several CLI commands edit multiple configuration options at the same -time. Some examples taken from ripd: \* -``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` - -*/frr-ripd:ripd/instance/timers/flush-interval* - -*/frr-ripd:ripd/instance/timers/holddown-interval* - -*/frr-ripd:ripd/instance/timers/update-interval* \* -``distance (1-255) A.B.C.D/M [WORD]`` - -*/frr-ripd:ripd/instance/distance/source/prefix* - -*/frr-ripd:ripd/instance/distance/source/distance* - -*/frr-ripd:ripd/instance/distance/source/access-list* +time. Some examples taken from ripd: + +* ``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` + * */frr-ripd:ripd/instance/timers/flush-interval* + * */frr-ripd:ripd/instance/timers/holddown-interval* + * */frr-ripd:ripd/instance/timers/update-interval* + +* ``distance (1-255) A.B.C.D/M [WORD]`` + * */frr-ripd:ripd/instance/distance/source/prefix* + * */frr-ripd:ripd/instance/distance/source/distance* + * */frr-ripd:ripd/instance/distance/source/access-list* In the new northbound model, there’s one or more separate callbacks for each configuration option. This usually has implications when converting @@ -965,34 +982,30 @@ future. For libfrr commands, it’s not possible to centralize all commands in a single file because the *extract.pl* script from *vtysh* treats commands differently depending on the file in which they are defined (e.g. DEFUNs -from *lib/routemap.c* are installed using the ``VTYSH_RMAP`` constant, +from *lib/routemap.c* are installed using the ``VTYSH_RMAP_SHOW`` constant, which identifies the daemons that support route-maps). In this case, the CLI commands should be rewritten but maintained in the same file. Since all CLI configuration commands from FRR will need to be rewritten, this is an excellent opportunity to rework this part of the code to make the commands easier to maintain and extend. These are the three main -recommendations: 1. Always use DEFPY instead of DEFUN to improve code -readability. 2. Always try to join multiple DEFUNs into a single DEFPY -whenever possible. As an example, there’s no need to have both -``distance (1-255) A.B.C.D/M`` and ``distance (1-255) A.B.C.D/M WORD`` -when a single ``distance (1-255) A.B.C.D/M [WORD]`` would suffice. 3. -When necessary, create a separate DEFPY for ``no`` commands so that part -of the configuration command can be made optional for convenience. -Example: -``no timers basic [(5-2147483647) (5-2147483647) (5-2147483647)]``. In -this example, everything after ``no timers basic`` is ignored by FRR, so -it makes sense to accept ``no timers basic`` as a valid command. But it -also makes sense to accept all parameters -(``no timers basic (5-2147483647) (5-2147483647) (5-2147483647)``) to -make it easier to remove the command just by prefixing a “no” to it. +recommendations: + +#. Always use DEFPY instead of DEFUN to improve code readability +#. Always try to join multiple DEFUNs into a single DEFPY whenever possible. As + an example, there’s no need to have both ``distance (1-255) A.B.C.D/M`` and + ``distance (1-255) A.B.C.D/M WORD`` when a single ``distance (1-255) + A.B.C.D/M [WORD]`` would suffice. +#. When making a negative form of a command, put ``[no]`` in the positive form + and use ``![...]`` to mark portions of the command that should be optional + only in the ``no`` version. To rewrite a CLI command as a dumb wrapper around the northbound callbacks, use the ``nb_cli_cfg_change()`` function. This function accepts as a parameter an array of ``cli_config_change`` structures that specify the changes that need to performed on the candidate configuration. Here’s the declaration of this structure (taken from the -*lib/northbound_cli.h* file): +``lib/northbound_cli.h`` file): .. code:: c @@ -1005,7 +1018,7 @@ configuration. Here’s the declaration of this structure (taken from the /* * Operation to apply (either NB_OP_CREATE, NB_OP_MODIFY or - * NB_OP_DELETE). + * NB_OP_DESTROY). */ enum nb_operation operation; @@ -1034,16 +1047,23 @@ changing the candidate configuration. the northbound callbacks are not involved). Other important details to keep in mind while rewriting the CLI -commands: \* ``nb_cli_cfg_change()`` returns CLI errors codes -(e.g. ``CMD_SUCCESS``, ``CMD_WARNING``), so the return value of this -function can be used as the return value of CLI commands. \* Calls to -``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted to -calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t -necessary anymore and can be removed: ``VTY_DECLVAR_CONTEXT``, -``VTY_DECLVAR_CONTEXT_SUB``, ``VTY_GET_CONTEXT`` and -``VTY_CHECK_CONTEXT``. The ``nb_cli_cfg_change()`` functions uses the -``VTY_CHECK_XPATH`` macro to check if the data node being edited still -exists before doing anything else. +commands: + +* ``nb_cli_cfg_change()`` returns CLI errors codes (e.g. ``CMD_SUCCESS``, + ``CMD_WARNING``), so the return value of this function can be used as the + return value of CLI commands. + +* Calls to ``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted + to calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t + necessary anymore and can be removed: + + * ``VTY_DECLVAR_CONTEXT`` + * ``VTY_DECLVAR_CONTEXT_SUB`` + * ``VTY_GET_CONTEXT`` + * ``VTY_CHECK_CONTEXT``. + + The ``nb_cli_cfg_change()`` functions uses the ``VTY_CHECK_XPATH`` macro to + check if the data node being edited still exists before doing anything else. The examples below provide additional details about how the conversion should be done. @@ -1205,7 +1225,7 @@ This example shows how to create a list entry: }, { .xpath = "./access-list", - .operation = acl ? NB_OP_MODIFY : NB_OP_DELETE, + .operation = acl ? NB_OP_MODIFY : NB_OP_DESTROY, .value = acl, }, }; @@ -1242,7 +1262,7 @@ When deleting a list entry, all non-key leaves can be ignored: struct cli_config_change changes[] = { { .xpath = ".", - .operation = NB_OP_DELETE, + .operation = NB_OP_DESTROY, }, }; @@ -1785,10 +1805,13 @@ Implementation of the ``cli_show`` callback: } This is the most complex ``cli_show`` callback we have in ripd. Its -complexity comes from the following: \* The -``ip rip authentication mode ...`` command changes two YANG leaves at -the same time. \* Part of the command should be hidden when the -``show_defaults`` parameter is set to false. +complexity comes from the following: + +* The ``ip rip authentication mode ...`` command changes two YANG leaves at the + same time. + +* Part of the command should be hidden when the ``show_defaults`` parameter is + set to false. This is the behavior we want to implement: @@ -1838,19 +1861,27 @@ As mentioned in the fourth step, the northbound retrofitting process can happen gradually over time, since both “old” and “new” commands can coexist without problems. Once all commands from a given daemon were converted, we can proceed to the consolidation step, which consists of -the following: \* Remove the vty configuration lock, which is enabled by -default in all daemons. Now multiple users should be able to edit the -configuration concurrently, using either shared or private candidate -configurations. \* Reference commit: -`57dccdb1 <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. -\* Stop using the qobj infrastructure to keep track of configuration -objects. This is not necessary anymore, the northbound uses a similar -mechanism to keep track of YANG data nodes in the candidate -configuration. \* Reference commit: -`4e6d63ce <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. -\* Make the daemon SIGHUP handler re-read the configuration file (and -ensure it’s not doing anything other than that). \* Reference commit: -`5e57edb4 <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. +the following: + +* Remove the vty configuration lock, which is enabled by default in all daemons. + Now multiple users should be able to edit the configuration concurrently, + using either shared or private candidate configurations. + +* Reference commit: `57dccdb1 + <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. + +* Stop using the qobj infrastructure to keep track of configuration objects. + This is not necessary anymore, the northbound uses a similar mechanism to keep + track of YANG data nodes in the candidate configuration. + +* Reference commit: `4e6d63ce + <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. + +* Make the daemon SIGHUP handler re-read the configuration file (and ensure it’s + not doing anything other than that). + +* Reference commit: `5e57edb4 + <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. Final Considerations -------------------- diff --git a/doc/developer/northbound/transactional-cli.rst b/doc/developer/northbound/transactional-cli.rst index 439bb6a..5c495d3 100644 --- a/doc/developer/northbound/transactional-cli.rst +++ b/doc/developer/northbound/transactional-cli.rst @@ -1,25 +1,10 @@ -Table of Contents ------------------ - -- `Introduction <#introduction>`__ -- `Configuration modes <#config-modes>`__ -- `New commands <#retrofitting-process>`__ - - - `commit check <#cmd1>`__ - - `commit <#cmd2>`__ - - `discard <#cmd3>`__ - - `configuration database max-transactions <#cmd4>`__ - - `configuration load <#cmd5>`__ - - `rollback configuration <#cmd6>`__ - - `show configuration candidate <#cmd7>`__ - - `show configuration compare <#cmd8>`__ - - `show configuration running <#cmd9>`__ - - `show configuration transaction <#cmd10>`__ - - `show yang module <#cmd11>`__ - - `show yang module-translator <#cmd12>`__ - - `update <#cmd13>`__ - - `yang module-translator load <#cmd14>`__ - - `yang module-translator unload <#cmd15>`__ +Transactional CLI +================= + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ~~~~~~~~~~~~ @@ -70,18 +55,21 @@ Configuration modes ~~~~~~~~~~~~~~~~~~~ When using the transactional CLI (``--tcli``), FRR supports three -different forms of the ``configure`` command: \* ``configure terminal``: -in this mode, a single candidate configuration is shared by all users. -This means that one user might delete a configuration object that’s -being edited by another user, in which case the CLI will detect and -report the problem. If one user issues the ``commit`` command, all -changes done by all users are committed. \* ``configure private``: users -have a private candidate configuration that is edited separately from -the other users. The ``commit`` command commits only the changes done by -the user. \* ``configure exclusive``: similar to ``configure private``, -but also locks the running configuration to prevent other users from -changing it. The configuration lock is released when the user exits the -configuration mode. +different forms of the ``configure`` command: + +* ``configure terminal``: in this mode, a single candidate configuration is + shared by all users. This means that one user might delete a configuration + object that’s being edited by another user, in which case the CLI will detect + and report the problem. If one user issues the ``commit`` command, all changes + done by all users are committed. + +* ``configure private``: users have a private candidate configuration that is + edited separately from the other users. The ``commit`` command commits only + the changes done by the user. + +* ``configure exclusive``: similar to ``configure private``, but also locks the + running configuration to prevent other users from changing it. The + configuration lock is released when the user exits the configuration mode. When using ``configure terminal`` or ``configure private``, the candidate configuration being edited might become outdated if another @@ -112,12 +100,14 @@ Check if the candidate configuration is valid or not. Commit the changes done in the candidate configuration into the running configuration. -Options: \* ``force``: commit even if the candidate configuration is -outdated. It’s usually a better option to use the ``update`` command -instead. \* ``comment LINE...``: assign a comment to the configuration -transaction. This comment is displayed when viewing the recorded -transactions in the output of the ``show configuration transaction`` -command. +Options: + +* ``force``: commit even if the candidate configuration is outdated. It’s + usually a better option to use the ``update`` command instead. + +* ``comment LINE...``: assign a comment to the configuration transaction. This + comment is displayed when viewing the recorded transactions in the output of + the ``show configuration transaction`` command. ``discard`` ''''''''''' @@ -140,10 +130,13 @@ respectively. It’s also possible to load a configuration from a previous transaction by specifying the desired transaction ID (``(1-4294967296)``). -Options: \* ``translate WORD``: translate the JSON/XML configuration -file using the YANG module translator. \* ``replace``: replace the -candidate by the loaded configuration. The default is to merge the -loaded configuration into the candidate configuration. +Options: + +* ``translate WORD``: translate the JSON/XML configuration file using the YANG + module translator. + +* ``replace``: replace the candidate by the loaded configuration. The default is + to merge the loaded configuration into the candidate configuration. ``rollback configuration (1-4294967296)`` ''''''''''''''''''''''''''''''''''''''''' @@ -156,39 +149,42 @@ identified by its transaction ID (``(1-4294967296)``). Show the candidate configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show only the changes done in the candidate -configuration. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show only the changes done in the candidate configuration. ``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the difference between two different configurations. -Options: \* ``json``: show the configuration differences in the JSON -format. \* ``xml``: show the configuration differences in the XML -format. \* ``translate WORD``: translate the JSON/XML output using the -YANG module translator. +Options: + +* ``json``: show the configuration differences in the JSON format. +* ``xml``: show the configuration differences in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. ``show configuration running [<json|xml> [translate WORD]] [with-defaults]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the running configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. +Options: - NOTE: ``show configuration running`` shows only the running - configuration as known by the northbound layer. Configuration - commands not converted to the new northbound model will not be - displayed. To show the full running configuration, the legacy - ``show running-config`` command must be used. +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. + +NOTE: ``show configuration running`` shows only the running +configuration as known by the northbound layer. Configuration +commands not converted to the new northbound model will not be +displayed. To show the full running configuration, the legacy +``show running-config`` command must be used. ``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]`` ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -199,12 +195,13 @@ configuration associated to the previously committed transaction. When a transaction ID is not given, show all recorded transactions in the rollback log. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show changes compared to the previous -transaction. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show changes compared to the previous transaction. ``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -212,11 +209,14 @@ transaction. When a YANG module is not given, show all loaded YANG modules. Otherwise, show detailed information about the given module. -Options: \* ``module-translator WORD``: change the context to modules -loaded by the specified YANG module translator. \* ``summary``: display -summary information about the module. \* ``tree``: display module in the -tree (RFC 8340) format. \* ``yang``: display module in the YANG format. -\* ``yin``: display module in the YIN format. +Options: + +* ``module-translator WORD``: change the context to modules loaded by the + specified YANG module translator. +* ``summary``: display summary information about the module. +* ``tree``: display module in the tree (RFC 8340) format. +* ``yang``: display module in the YANG format. +* ``yin``: display module in the YIN format. ``show yang module-translator`` ''''''''''''''''''''''''''''''' diff --git a/doc/developer/northbound/yang-module-translator.rst b/doc/developer/northbound/yang-module-translator.rst index aa527ce..17ae160 100644 --- a/doc/developer/northbound/yang-module-translator.rst +++ b/doc/developer/northbound/yang-module-translator.rst @@ -1,11 +1,10 @@ -Table of Contents ------------------ +YANG Module Translation +======================= -- `Introduction <#introduction>`__ -- `Deviation Modules <#deviation-modules>`__ -- `Translation Tables <#translation-tables>`__ -- `CLI Demonstration <#cli-demonstration>`__ -- `Implementation Details <#implementation-details>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ------------ @@ -421,15 +420,20 @@ this shortcoming and make it possible to create more powerful YANG module translators. YANG module translators can be evaluated based on the following metrics: -\* Translation potential: is it possible to make complex translations, -taking several variables into account? \* Complexity: measure of how -easy or hard it is to write a module translator. \* Speed: measure of -how fast the translation can be achieved. Translation speed is of -fundamental importance, especially for operational data. \* Robustness: -can the translator be checked for inconsistencies at load time? A module -translator based on scripts wouldn’t fare well on this metric. \* -Round-trip conversions: can the translated data be translated back to -the original format without information loss? + +* Translation potential: is it possible to make complex translations, taking + several variables into account? + +* Complexity: measure of how easy or hard it is to write a module translator. + +* Speed: measure of how fast the translation can be achieved. Translation speed + is of fundamental importance, especially for operational data. + +* Robustness: can the translator be checked for inconsistencies at load time? A + module translator based on scripts wouldn’t fare well on this metric. + +* Round-trip conversions: can the translated data be translated back to the + original format without information loss? CLI Demonstration ----------------- diff --git a/doc/developer/northbound/yang-tools.rst b/doc/developer/northbound/yang-tools.rst index 346efca..fb5a287 100644 --- a/doc/developer/northbound/yang-tools.rst +++ b/doc/developer/northbound/yang-tools.rst @@ -1,5 +1,5 @@ Yang Tools -~~~~~~~~~~ +========== Here's some information about various tools for working with yang models. @@ -83,17 +83,19 @@ Indent a YANG file: --keep-comments -f yang --yang-canonical \ module.yang -o module.yang -Generate skeleton instance data: \* XML: +Generate skeleton instance data: -.. code:: sh +* XML: + + .. code:: sh $ pyang -p <yang-search-path> \ -f sample-xml-skeleton --sample-xml-skeleton-defaults \ module.yang [augmented-module1.yang ...] -o module.xml -- JSON: +* JSON: -.. code:: sh + .. code:: sh $ pyang -p <yang-search-path> \ -f jsonxsl module.yang -o module.xsl diff --git a/doc/developer/process-architecture.rst b/doc/developer/process-architecture.rst index 06ee6a3..85126ca 100644 --- a/doc/developer/process-architecture.rst +++ b/doc/developer/process-architecture.rst @@ -87,7 +87,7 @@ are given by integer macros in :file:`frrevent.h` and are: ``EVENT_EXECUTE`` Just before a task is run its type is changed to this. This is used to show - ``X`` as the type in the output of :clicmd:`show thread cpu`. + ``X`` as the type in the output of :clicmd:`show event cpu`. The programmer never has to work with these types explicitly. Each type of task is created and queued via special-purpose functions (actually macros, but @@ -238,16 +238,16 @@ proceeding. In addition, the existing commands to show statistics and other information for tasks within the event driven model have been expanded to handle multiple -pthreads; running :clicmd:`show thread cpu` will display the usual event +pthreads; running :clicmd:`show event cpu` will display the usual event breakdown, but it will do so for each pthread running in the program. For example, :ref:`bgpd` runs a dedicated I/O pthread and shows the following -output for :clicmd:`show thread cpu`: +output for :clicmd:`show event cpu`: :: - frr# show thread cpu + frr# show event cpu - Thread statistics for bgpd: + Event statistics for bgpd: Showing statistics for pthread main ------------------------------------ diff --git a/doc/developer/rcu.rst b/doc/developer/rcu.rst index 4fd5658..2335e8f 100644 --- a/doc/developer/rcu.rst +++ b/doc/developer/rcu.rst @@ -232,6 +232,15 @@ Internals that case, either all of the library's threads must be registered for RCU, or the code must instead pass a (non-RCU) copy of the data to the library. +.. c:function:: int frr_pthread_non_controlled_startup(pthread_t thread, const char *name, const char *os_name) + + If a pthread is started outside the control of normal pthreads in frr + then frr_pthread_non_controlled_startup should be called. This will + properly setup both the pthread with rcu usage as well as some data + structures pertaining to the name of the pthread. This is especially + important if the pthread created ends up calling back into FRR and + one of the various zlog_XXX functions is called. + .. c:function:: void rcu_shutdown(void) Stop the RCU sweeper thread and make sure all cleanup has finished. diff --git a/doc/developer/requirements.txt b/doc/developer/requirements.txt new file mode 100644 index 0000000..483a4e9 --- /dev/null +++ b/doc/developer/requirements.txt @@ -0,0 +1 @@ +sphinx_rtd_theme diff --git a/doc/developer/scripting.rst b/doc/developer/scripting.rst index 202f003..7a43314 100644 --- a/doc/developer/scripting.rst +++ b/doc/developer/scripting.rst @@ -488,12 +488,6 @@ match *exactly*. In the above example, we defined encoders/decoders for a value of ``struct prefix *``, but not ``struct prefix`` or ``const struct prefix *``. -``const`` values are a special case. We want to use them in our Lua scripts -but not modify them, so creating a decoder for them would be meaningless. -But we still need a decoder for the type of value so that the compiler will be -satisfied. -For that, use ``lua_decode_noop``: - .. code-block:: diff #define DECODE_ARGS_WITH_STATE(L, value) \ diff --git a/doc/developer/subdir.am b/doc/developer/subdir.am index 0deb0f5..652ee4e 100644 --- a/doc/developer/subdir.am +++ b/doc/developer/subdir.am @@ -5,6 +5,7 @@ dev_RSTFILES = \ doc/developer/bgp-typecodes.rst \ doc/developer/bgpd.rst \ + doc/developer/bmp.rst \ doc/developer/building-frr-for-alpine.rst \ doc/developer/building-frr-for-archlinux.rst \ doc/developer/building-frr-for-centos6.rst \ diff --git a/doc/developer/topotests.rst b/doc/developer/topotests.rst index b8f213b..9b9058b 100644 --- a/doc/developer/topotests.rst +++ b/doc/developer/topotests.rst @@ -8,14 +8,15 @@ Topotests is a suite of topology tests for FRR built on top of micronet. Installation and Setup ---------------------- -Topotests run under python3. Additionally, for ExaBGP (which is used -in some of the BGP tests) an older python2 version (and the python2 -version of ``pip``) must be installed. +Topotests run under python3. -Tested with Ubuntu 20.04,Ubuntu 18.04, and Debian 11. +Tested with Ubuntu 22.04,Ubuntu 20.04, and Debian 12. -Instructions are the same for all setups (i.e. ExaBGP is only used for -BGP tests). +Python protobuf version < 4 is required b/c python protobuf >= 4 requires a +protoc >= 3.19, and older package versions are shipped by in the above distros. + +Instructions are the same for all setups. However, ExaBGP is only used for +BGP tests. Tshark is only required if you enable any packet captures on test runs. @@ -35,15 +36,23 @@ Installing Topotest Requirements tshark \ valgrind python3 -m pip install wheel - python3 -m pip install 'pytest>=6.2.4' - python3 -m pip install 'pytest-xdist>=2.3.0' + python3 -m pip install 'pytest>=6.2.4' 'pytest-xdist>=2.3.0' python3 -m pip install 'scapy>=2.4.5' python3 -m pip install xmltodict - # Use python2 pip to install older ExaBGP - python2 -m pip install 'exabgp<4.0.0' + python3 -m pip install git+https://github.com/Exa-Networks/exabgp@0659057837cd6c6351579e9f0fa47e9fb7de7311 useradd -d /var/run/exabgp/ -s /bin/false exabgp - # To enable the gRPC topotest install: +The version of protobuf package that is installed on your system will determine +which versions of the python protobuf packages you need to install. + +.. code:: shell + # - Either - For protobuf version <= 3.12 + python3 -m pip install 'protobuf<4' + + # - OR- for protobuf version >= 3.21 + python3 -m pip install 'protobuf>=4' + + # To enable the gRPC topotest also install: python3 -m pip install grpcio grpcio-tools @@ -116,9 +125,9 @@ If you prefer to manually build FRR, then use the following suggested config: ./configure \ --prefix=/usr \ - --localstatedir=/var/run/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir=/usr/lib/frr \ - --sysconfdir=/etc/frr \ --enable-vtysh \ --enable-pimd \ --enable-pim6d \ @@ -230,8 +239,8 @@ the number of the test we are interested in along with ``--errmsg`` option. ~/frr/tests/topotests# ./analyze.py -Ar run-save -T0 --errmsg bgp_multiview_topo1/test_bgp_multiview_topo1.py::test_bgp_converge: AssertionError: BGP did not converge: - IPv4 Unicast Summary (VIEW 1): - BGP router identifier 172.30.1.1, local AS number 100 vrf-id -1 + IPv4 Unicast Summary: + BGP router identifier 172.30.1.1, local AS number 100 VIEW 1 vrf-id -1 BGP table version 1 RIB entries 1, using 184 bytes of memory Peers 3, using 2169 KiB of memory @@ -266,8 +275,8 @@ select the first failed test case. > assert False, "BGP did not converge:\n%s" % bgpStatus E AssertionError: BGP did not converge: E - E IPv4 Unicast Summary (VIEW 1): - E BGP router identifier 172.30.1.1, local AS number 100 vrf-id -1 + E IPv4 Unicast Summary: + E BGP router identifier 172.30.1.1, local AS number 100 VIEW 1 vrf-id -1 [...] E Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc E 172.16.1.1 4 65001 0 0 0 0 0 never Connect 0 N/A @@ -386,8 +395,9 @@ for ``master`` branch: ./bootstrap.sh ./configure \ --enable-address-sanitizer \ - --prefix=/usr/lib/frr --sysconfdir=/etc/frr \ - --localstatedir=/var/run/frr \ + --prefix=/usr/lib/frr \ + --sysconfdir=/etc \ + --localstatedir=/var \ --sbindir=/usr/lib/frr --bindir=/usr/lib/frr \ --with-moduledir=/usr/lib/frr/modules \ --enable-multipath=0 --enable-rtadv \ @@ -559,6 +569,8 @@ Here's an example of launching ``vtysh`` on routers ``rt1`` and ``rt2``. sudo -E pytest --vtysh=rt1,rt2 all-protocol-startup +.. _debug_with_gdb: + Debugging with GDB """""""""""""""""" @@ -583,6 +595,12 @@ Here's an example of launching ``zebra`` and ``bgpd`` inside ``gdb`` on router --gdb-breakpoints=nb_config_diff \ all-protocol-startup +Finally, for Emacs users, you can specify ``--gdb-use-emacs``. When specified +the first router and daemon to be launched in gdb will be launched and run with +Emacs gdb functionality by using `emacsclient --eval` commands. This provides an +IDE debugging experience for Emacs users. This functionality works best when +using password-less sudo. + Reporting Memleaks with FRR Memory Statistics """"""""""""""""""""""""""""""""""""""""""""" @@ -631,16 +649,26 @@ Detecting Memleaks with Valgrind """""""""""""""""""""""""""""""" Topotest can automatically launch all daemons with ``valgrind`` to check for -memleaks. This is enabled by specifying 1 or 2 CLI arguments. -``--valgrind-memleaks`` will enable general memleak detection, and -``--valgrind-extra`` enables extra functionality including generating a -suppression file. The suppression file ``tools/valgrind.supp`` is used when -memleak detection is enabled. +memleaks. This is enabled by specifying 1 to 3 CLI arguments. +``--valgrind-memleaks`` enables memleak detection. ``--valgrind-extra`` enables +extra functionality including generating a suppression file. The suppression +file ``tools/valgrind.supp`` is used when memleak detection is enabled. Finally, +``--valgrind-leak-kinds=KINDS`` can be used to modify what types of links are +reported. This corresponds to valgrind's ``--show-link-kinds`` arg. The value is +either ``all`` or a comma-separated list of types: +``definite,indirect,possible,reachable``. The default is ``definite,possible``. .. code:: shell sudo -E pytest --valgrind-memleaks all-protocol-startup +.. note:: GDB can be used in conjection with valgrind. + + When you enable ``--valgrind-memleaks`` and you also launch various daemons + under GDB (debug_with_gdb_) topotest will connect the two utilities using + ``--vgdb-error=0`` and attaching to a ``vgdb`` process. This is very + useful for debugging bugs with use of uninitialized errors, et al. + Collecting Performance Data using perf(1) """"""""""""""""""""""""""""""""""""""""" @@ -662,6 +690,28 @@ during the config_timing test. To specify different arguments for ``perf record``, one can use the ``--perf-options`` this will replace the ``-g`` used by default. +Running Daemons under RR Debug (``rr record``) +"""""""""""""""""""""""""""""""""""""""""""""" + +Topotest can automatically launch any daemon under ``rr(1)`` to collect +execution state. The daemon is run in the foreground with ``rr record``. + +The execution state will be saved in the router specific directory +(in a `rr` subdir that rr creates) under the test's run directoy. + +Here's an example of collecting ``rr`` execution state from ``mgmtd`` on router +``r1`` during the ``config_timing`` test. + +.. code:: console + + $ sudo -E pytest --rr-routers=r1 --rr-daemons=mgmtd config_timing + ... + $ find /tmp/topotests/ -name '*perf.data*' + /tmp/topotests/config_timing.test_config_timing/r1/perf.data + +To specify additional arguments for ``rr record``, one can use the +``--rr-options``. + .. _topotests_docker: Running Tests with Docker |