diff options
Diffstat (limited to 'doc/developer/northbound')
| -rw-r--r-- | doc/developer/northbound/advanced-topics.rst | 95 | ||||
| -rw-r--r-- | doc/developer/northbound/architecture.rst | 55 | ||||
| -rw-r--r-- | doc/developer/northbound/demos.rst | 3 | ||||
| -rw-r--r-- | doc/developer/northbound/links.rst | 3 | ||||
| -rw-r--r-- | doc/developer/northbound/northbound.rst | 2 | ||||
| -rw-r--r-- | doc/developer/northbound/operational-data-rpcs-and-notifications.rst | 18 | ||||
| -rw-r--r-- | doc/developer/northbound/plugins-sysrepo.rst | 212 | ||||
| -rw-r--r-- | doc/developer/northbound/ppr-basic-test-topology.rst | 17 | ||||
| -rw-r--r-- | doc/developer/northbound/ppr-mpls-basic-test-topology.rst | 17 | ||||
| -rw-r--r-- | doc/developer/northbound/retrofitting-configuration-commands.rst | 197 | ||||
| -rw-r--r-- | doc/developer/northbound/transactional-cli.rst | 150 | ||||
| -rw-r--r-- | doc/developer/northbound/yang-module-translator.rst | 36 | ||||
| -rw-r--r-- | doc/developer/northbound/yang-tools.rst | 12 |
13 files changed, 468 insertions, 349 deletions
diff --git a/doc/developer/northbound/advanced-topics.rst b/doc/developer/northbound/advanced-topics.rst index bee29a9..edfc10b 100644 --- a/doc/developer/northbound/advanced-topics.rst +++ b/doc/developer/northbound/advanced-topics.rst @@ -1,3 +1,11 @@ +Advanced Topics +=============== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Auto-generated CLI commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -34,13 +42,16 @@ CLI on a separate program The flexible design of the northbound architecture opens the door to move the CLI to a separate program in the long-term future. Some -advantages of doing so would be: \* Treat the CLI as just another -northbound client, instead of having CLI commands embedded in the -binaries of all FRR daemons. \* Improved robustness: bugs in CLI -commands (e.g. null-pointer dereferences) or in the CLI code itself -wouldn’t affect the FRR daemons. \* Foster innovation by allowing other -CLI programs to be implemented, possibly using higher level programming -languages. +advantages of doing so would be: + +* Treat the CLI as just another northbound client, instead of having CLI + commands embedded in the binaries of all FRR daemons. + +* Improved robustness: bugs in CLI commands (e.g. null-pointer dereferences) or + in the CLI code itself wouldn’t affect the FRR daemons. + +* Foster innovation by allowing other CLI programs to be implemented, possibly + using higher level programming languages. The problem, however, is that the northbound retrofitting process will convert only the CLI configuration commands and EXEC commands in a first @@ -232,40 +243,42 @@ vtysh support As explained in the [[Transactional CLI]] page, all commands introduced by the transactional CLI are not yet available in *vtysh*. This needs to be addressed in the short term future. Some challenges for doing that -work include: \* How to display configurations (running, candidates and -rollbacks) in a more clever way? The implementation of the -``show running-config`` command in *vtysh* is not something that should -be followed as an example. A better idea would be to fetch the desired -configuration from all daemons (encoded in JSON for example), merge them -all into a single ``lyd_node`` variable and then display the combined -configurations from this variable (the configuration merges would -transparently take care of combining the shared configuration objects). -In order to be able to manipulate the JSON configurations, *vtysh* will -need to load the YANG modules from all daemons at startup (this might -have a minimal impact on startup time). The only issue with this -approach is that the ``cli_show()`` callbacks from all daemons are -embedded in their binaries and thus not accessible externally. It might -be necessary to compile these callbacks on a separate shared library so -that they are accessible to *vtysh* too. Other than that, displaying the -combined configurations in the JSON/XML formats should be -straightforward. \* With the current design, transaction IDs are -per-daemon and not global across all FRR daemons. This means that the -same transaction ID can represent different transactions on different -daemons. Given this observation, how to implement the -``rollback configuration`` command in *vtysh*? The easy solution would -be to add a ``daemon WORD`` argument to specify the context of the -rollback, but per-daemon rollbacks would certainly be confusing and -convoluted to end users. A better idea would be to attack the root of -the problem: change configuration transactions to be global instead of -being per-daemon. This involves a bigger change in the northbound -architecture, and would have implications on how transactions are stored -in the SQL database (daemon-specific and shared configuration objects -would need to have their own tables or columns). \* Loading -configuration files in the JSON or XML formats will be tricky, as -*vtysh* will need to know which sections of the configuration should be -sent to which daemons. *vtysh* will either need to fetch the YANG -modules implemented by all daemons at runtime or obtain this information -at compile-time somehow. +work include: + +* How to display configurations (running, candidates and rollbacks) in a more + clever way? The implementation of the ``show running-config`` command in + *vtysh* is not something that should be followed as an example. A better idea + would be to fetch the desired configuration from all daemons (encoded in JSON + for example), merge them all into a single ``lyd_node`` variable and then + display the combined configurations from this variable (the configuration + merges would transparently take care of combining the shared configuration + objects). In order to be able to manipulate the JSON configurations, *vtysh* + will need to load the YANG modules from all daemons at startup (this might + have a minimal impact on startup time). The only issue with this approach is + that the ``cli_show()`` callbacks from all daemons are embedded in their + binaries and thus not accessible externally. It might be necessary to compile + these callbacks on a separate shared library so that they are accessible to + *vtysh* too. Other than that, displaying the combined configurations in the + JSON/XML formats should be straightforward. + +* With the current design, transaction IDs are per-daemon and not global across + all FRR daemons. This means that the same transaction ID can represent + different transactions on different daemons. Given this observation, how to + implement the ``rollback configuration`` command in *vtysh*? The easy solution + would be to add a ``daemon WORD`` argument to specify the context of the + rollback, but per-daemon rollbacks would certainly be confusing and convoluted + to end users. A better idea would be to attack the root of the problem: change + configuration transactions to be global instead of being per-daemon. This + involves a bigger change in the northbound architecture, and would have + implications on how transactions are stored in the SQL database + (daemon-specific and shared configuration objects would need to have their own + tables or columns). + +* Loading configuration files in the JSON or XML formats will be tricky, as + *vtysh* will need to know which sections of the configuration should be sent + to which daemons. *vtysh* will either need to fetch the YANG modules + implemented by all daemons at runtime or obtain this information at + compile-time somehow. Detecting type mismatches at compile-time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/developer/northbound/architecture.rst b/doc/developer/northbound/architecture.rst index e571971..5fd89c3 100644 --- a/doc/developer/northbound/architecture.rst +++ b/doc/developer/northbound/architecture.rst @@ -1,3 +1,6 @@ +Architecture +============ + Introduction ------------ @@ -42,30 +45,34 @@ and `CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__. In addition to being management-protocol independent, some other -advantages of using YANG in FRR are listed below: \* Have a formal -contract between FRR and application developers (management clients). A -management client that has access to the FRR YANG models knows about all -existing configuration options available for use. This information can -be used to auto-generate user-friendly interfaces like Web-UIs, custom -CLIs and even code bindings for several different programming languages. -Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for -example, it’s possible to generate Python class hierarchies from YANG -models and use these classes to instantiate objects that mirror the -structure of the YANG modules and can be serialized/deserialized using -different encoding formats. \* Support different encoding formats for -instance data. Currently only JSON and XML are supported, but -`GPB <https://developers.google.com/protocol-buffers/>`__ and -`CBOR <http://cbor.io/>`__ are other viable options in the long term. -Additional encoding formats can be implemented in the *libyang* library -for optimal performance, or externally by translating data to/from one -of the supported formats (with a performance penalty). \* Have a formal -mechanism to introduce backward-incompatible changes based on `semantic -versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the -YANG standard, which allows backward-compatible module updates only). \* -Provide seamless support to the industry-standard NETCONF/RESTCONF -protocols as alternative management APIs. If FRR configuration/state -data is modeled using YANG, supporting YANG-based protocols like NETCONF -and RESTCONF is much easier. +advantages of using YANG in FRR are listed below: + +* Have a formal contract between FRR and application developers (management + clients). A management client that has access to the FRR YANG models knows + about all existing configuration options available for use. This information + can be used to auto-generate user-friendly interfaces like Web-UIs, custom + CLIs and even code bindings for several different programming languages. Using + `PyangBind <https://github.com/robshakir/pyangbind>`__, for example, it’s + possible to generate Python class hierarchies from YANG models and use these + classes to instantiate objects that mirror the structure of the YANG modules + and can be serialized/deserialized using different encoding formats. + +* Support different encoding formats for instance data. Currently only JSON and + XML are supported, but `GPB + <https://developers.google.com/protocol-buffers/>`__ and `CBOR + <http://cbor.io/>`__ are other viable options in the long term. Additional + encoding formats can be implemented in the *libyang* library for optimal + performance, or externally by translating data to/from one of the supported + formats (with a performance penalty). + +* Have a formal mechanism to introduce backward-incompatible changes based on + `semantic versioning <http://www.openconfig.net/docs/semver/>`__ (not part of + the YANG standard, which allows backward-compatible module updates only). + +* Provide seamless support to the industry-standard NETCONF/RESTCONF protocols + as alternative management APIs. If FRR configuration/state data is modeled + using YANG, supporting YANG-based protocols like NETCONF and RESTCONF is much + easier. As important as shifting to a model-driven management paradigm, the new northbound architecture also introduces the concept of configuration diff --git a/doc/developer/northbound/demos.rst b/doc/developer/northbound/demos.rst index 876bd25..8a0f6ad 100644 --- a/doc/developer/northbound/demos.rst +++ b/doc/developer/northbound/demos.rst @@ -1,3 +1,6 @@ +Demos +===== + Transactional CLI ----------------- diff --git a/doc/developer/northbound/links.rst b/doc/developer/northbound/links.rst index e80374c..6cec176 100644 --- a/doc/developer/northbound/links.rst +++ b/doc/developer/northbound/links.rst @@ -1,3 +1,6 @@ +Links +===== + RFCs ~~~~ diff --git a/doc/developer/northbound/northbound.rst b/doc/developer/northbound/northbound.rst index 7dddf06..c5f4e2f 100644 --- a/doc/developer/northbound/northbound.rst +++ b/doc/developer/northbound/northbound.rst @@ -11,11 +11,11 @@ Northbound API transactional-cli retrofitting-configuration-commands operational-data-rpcs-and-notifications - plugins-sysrepo advanced-topics yang-tools yang-module-translator demos links + plugins-sysrepo ppr-basic-test-topology ppr-mpls-basic-test-topology diff --git a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst index 554bc17..5cb70ca 100644 --- a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst +++ b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst @@ -1,3 +1,11 @@ +Operational Data, RPCs and Notifications +======================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Operational data ~~~~~~~~~~~~~~~~ @@ -330,10 +338,12 @@ CLI can take too long, potentially long enough to the point of triggering some protocol timeouts and bringing sessions down. To avoid this kind of problem, northbound clients are encouraged to do -one of the following: \* Create a separate pthread for handling requests -to fetch operational data. \* Iterate over YANG lists and leaf-lists -asynchronously, returning a maximum number of elements per time instead -of returning all elements in one shot. +one of the following: + +* Create a separate pthread for handling requests to fetch operational data. + +* Iterate over YANG lists and leaf-lists asynchronously, returning a maximum + number of elements per time instead of returning all elements in one shot. In order to handle both cases correctly, the ``get_next`` callbacks need to use locks to prevent the YANG lists from being modified while they diff --git a/doc/developer/northbound/plugins-sysrepo.rst b/doc/developer/northbound/plugins-sysrepo.rst index 186c3a0..0cfdb82 100644 --- a/doc/developer/northbound/plugins-sysrepo.rst +++ b/doc/developer/northbound/plugins-sysrepo.rst @@ -1,137 +1,193 @@ +Plugins Sysrepo +=============== + Installation ------------ Required dependencies ^^^^^^^^^^^^^^^^^^^^^ +Install FRR build required dependencies, check `Building FRR +<https://docs.frrouting.org/projects/dev-guide/en/latest/building.html>`_ document for specific platform required packages. +Below are debian systems required packages: -:: +.. code-block:: console - # apt-get install git cmake build-essential bison flex libpcre3-dev libev-dev \ - libavl-dev libprotobuf-c-dev protobuf-c-compiler libcmocka0 \ - libcmocka-dev doxygen libssl-dev libssl-dev libssh-dev + sudo apt-get install git autoconf automake libtool make \ + libprotobuf-c-dev protobuf-c-compiler build-essential \ + python3-dev python3-pytest python3-sphinx libjson-c-dev \ + libelf-dev libreadline-dev cmake libcap-dev bison flex \ + pkg-config texinfo gdb libgrpc-dev python3-grpc-tools libpcre2-dev libyang ^^^^^^^ -:: +.. note:: - # apt-get install libyang0.16 libyang-dev + FRR requires version 2.1.128 or newer, in this document we will + be compiling and installing libyang version 2.1.148. -Sysrepo -^^^^^^^ +.. code-block:: console -:: + git clone https://github.com/CESNET/libyang.git + cd libyang + git checkout v2.1.148 + mkdir build; cd build + cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DCMAKE_BUILD_TYPE:String="Release" .. + make + sudo make install - $ git clone https://github.com/sysrepo/sysrepo.git - $ cd sysrepo/ - $ mkdir build; cd build - $ cmake -DCMAKE_BUILD_TYPE=Release -DGEN_LANGUAGE_BINDINGS=OFF .. && make - # make install +Sysrepo +^^^^^^^ -libnetconf2 -^^^^^^^^^^^ +.. note:: -:: + The following code block assumes you have installed libyang v2.1.148, if you have + libyang v2.1.128 change sysrepo version to 2.2.105. - $ git clone https://github.com/CESNET/libnetconf2.git - $ cd libnetconf2/ - $ mkdir build; cd build - $ cmake .. && make - # make install +.. code-block:: console -netopeer2 -^^^^^^^^^ + git clone https://github.com/sysrepo/sysrepo.git + cd sysrepo/ + git checkout v2.2.150 + mkdir build; cd build + cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DCMAKE_BUILD_TYPE:String="Release" .. + make + sudo make install -:: +Verify that sysrepo is installed correctly: - $ git clone https://github.com/CESNET/Netopeer2.git - $ cd Netopeer2 - $ cd server - $ mkdir build; cd build - $ cmake .. && make - # make install +.. code-block:: console -**Note:** If ``make install`` fails as it can’t find -``libsysrepo.so.0.7``, then run ``ldconfig`` and try again as it might -not have updated the lib search path + sudo sysrepoctl -l FRR ^^^ -Build and install FRR using the ``--enable-sysrepo`` configure-time -option. +Follow the steps of `Building FRR +<https://docs.frrouting.org/projects/dev-guide/en/latest/building.html>`_ + + +Make sure to use ``--enable-sysrepo`` configure-time option while building FRR. + +Below is an example of frr configure-time options, your options +might vary, however in order to allow sysrepo plugin you have +to keep ``--enable-sysrepo`` option: + +.. code-block:: console + + ./bootstrap.sh + ./configure \ + --localstatedir=/var/opt/frr \ + --sbindir=/usr/lib/frr \ + --sysconfdir=/etc/frr \ + --enable-multipath=64 \ + --enable-user=frr \ + --enable-group=frr \ + --enable-vty-group=frrvty \ + --enable-configfile-mask=0640 \ + --enable-logfile-mask=0640 \ + --enable-fpm \ + --enable-sysrepo \ + --with-pkg-git-version \ + --with-pkg-extra-version=-MyOwnFRRVersion + make + make check + sudo make install + Initialization -------------- -Install the FRR YANG modules in the Sysrepo datastore: +Install FRR YANG modules in Sysrepo datastore: -:: +.. code-block:: console - # sysrepoctl --install /usr/local/share/yang/ietf-interfaces@2018-01-09.yang - # sysrepoctl --install /usr/local/share/yang/frr-vrf.yang - # sysrepoctl --install /usr/local/share/yang/frr-interface.yang - # sysrepoctl --install /usr/local/share/yang/frr-route-types.yang - # sysrepoctl --install /usr/local/share/yang/frr-filter.yang - # sysrepoctl --install /usr/local/share/yang/frr-route-map.yang - # sysrepoctl --install /usr/local/share/yang/frr-isisd.yang - # sysrepoctl --install /usr/local/share/yang/frr-ripd.yang - # sysrepoctl --install /usr/local/share/yang/frr-ripngd.yang - # sysrepoctl -c frr-vrf --owner frr --group frr - # sysrepoctl -c frr-interface --owner frr --group frr - # sysrepoctl -c frr-route-types --owner frr --group frr - # sysrepoctl -c frr-filter --owner frr --group frr - # sysrepoctl -c frr-route-map --owner frr --group frr - # sysrepoctl -c frr-isisd --owner frr --group frr - # sysrepoctl -c frr-ripd --owner frr --group frr - # sysrepoctl -c frr-ripngd --owner frr --group frr + cd frr/yang/ + sudo sysrepoctl -i ./ietf/ietf-interfaces.yang -o frr -g frr + sudo sysrepoctl -i frr-vrf.yang -o frr -g frr + sudo sysrepoctl -i frr-interface.yang -o frr -g frr + sudo sysrepoctl -i frr-route-types.yang -o frr -g frr + sudo sysrepoctl -i frr-filter.yang -o frr -g frr + sudo sysrepoctl -i frr-route-map.yang -o frr -g frr + sudo sysrepoctl -i frr-isisd.yang -o frr -g frr + sudo sysrepoctl -i frr-bfdd.yang -o frr -g frr + sudo sysrepoctl -i ./ietf/ietf-routing-types.yang -o frr -g frr + sudo sysrepoctl -i frr-nexthop.yang -o frr -g frr + sudo sysrepoctl -i frr-if-rmap.yang -o frr -g frr + sudo sysrepoctl -i frr-ripd.yang -o frr -g frr + sudo sysrepoctl -i frr-ripngd.yang -o frr -g frr + sudo sysrepoctl -i frr-affinity-map.yang -o frr -g frr + sudo sysrepoctl -i ./ietf/frr-deviations-ietf-interfaces.yang -o frr -g frr -Start netopeer2-server: -:: +Start FRR daemons with sysrepo plugin: - # netopeer2-server -d & +.. code-block:: console -Start the FRR daemons with the sysrepo module: + sudo /usr/lib/frr/isisd -M sysrepo --log stdout -:: - - # isisd -M sysrepo --log=stdout +Any daemon running with ``-M sysrepo`` will subscribe to its frr yang moduels +on sysrepo and you be able to configure it by editing module configuration on sysrepo. Managing the configuration -------------------------- -The following NETCONF scripts can be used to show and edit the FRR -configuration: -https://github.com/rzalamena/ietf-hackathon-brazil-201907/tree/master/netconf-scripts +Testing +^^^^^^^ -Example: +To test FRR intergartion with sysrepo, ``sysrepocfg`` tool can be used +to edit frr configuration on sysrepo -:: +Example: - # ./netconf-edit.py 127.0.0.1 - # ./netconf-get-config.py 127.0.0.1 - <?xml version="1.0" encoding="UTF-8"?><data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><isis xmlns="http://frrouting.org/yang/isisd"><instance><area-tag>testnet</area-tag><is-type>level-1</is-type></instance></isis></data> +Edit sysrepo running datastore configuration for the desiged frr module: -.. +.. code-block:: console - NOTE: the ncclient library needs to be installed first: - ``apt install -y python3-ncclient`` + sudo sysrepocfg -E nano -d running -m frr-isisd -f json -The *sysrepocfg* tool can also be used to show/edit the FRR -configuration. Example: +Paste the following json configuration: -:: +.. code-block:: console - # sysrepocfg --format=json --import=frr-isisd.json --datastore=running frr-isisd - # sysrepocfg --format=json --export --datastore=running frr-isisd { "frr-isisd:isis": { "instance": [ { "area-tag": "testnet", + "vrf": "default", "is-type": "level-1" } ] } } + +Exit and save config to the same file. + +After that, this configuration should get reflected to vtysh: + +.. code-block:: console + + show run + Building configuration... + + Current configuration: + ! + frr version 9.2-dev-MyOwnFRRVersion + frr defaults traditional + hostname bullseye + ! + router isis testnet + is-type level-1 + exit + ! + end + +NETCONF +^^^^^^^ + +To manage sysrepo configuration through netconf +you can use `netopeer2 <https://github.com/CESNET/netopeer2>`_ as a netfconf server that can +be easily integrated with sysrepo. diff --git a/doc/developer/northbound/ppr-basic-test-topology.rst b/doc/developer/northbound/ppr-basic-test-topology.rst index a680ed7..4929c9b 100644 --- a/doc/developer/northbound/ppr-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic +=============== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst index cedb795..aceec5f 100644 --- a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic MPLS +==================== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/retrofitting-configuration-commands.rst b/doc/developer/northbound/retrofitting-configuration-commands.rst index b407246..d328be9 100644 --- a/doc/developer/northbound/retrofitting-configuration-commands.rst +++ b/doc/developer/northbound/retrofitting-configuration-commands.rst @@ -1,5 +1,16 @@ + +.. _nb-retrofit: + Retrofitting Configuration Commands ------------------------------------ +=================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 + +Retrofitting process +-------------------- This page explains how to convert existing CLI configuration commands to the new northbound model. This documentation is meant to be the primary @@ -7,9 +18,6 @@ reference for developers working on the northbound retrofitting process. We’ll show several examples taken from the ripd northbound conversion to illustrate some concepts described herein. -Retrofitting process --------------------- - Step 1: writing a YANG module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -30,34 +38,41 @@ possible to facilitate the process of writing module translators using the [[YANG module translator]]. As an example, the frr-ripd YANG module incorporated several parts of the IETF RIP YANG module. The repositories below contain big collections of YANG models that might be used as a -reference: \* https://github.com/YangModels/yang \* -https://github.com/openconfig/public +reference: + +* https://github.com/YangModels/yang + +* https://github.com/openconfig/public When writing a YANG module, it’s highly recommended to follow the guidelines from `RFC 6087 <https://tools.ietf.org/html/rfc6087>`__. In general, most commands should be modeled fairly easy. Here are a few -guidelines specific to authors of FRR YANG models: \* Use -presence-containers or lists to model commands that change the CLI node -(e.g. ``router rip``, ``interface eth0``). This way, if the -presence-container or list entry is removed, all configuration options -below them are removed automatically (exactly like the CLI behaves when -a configuration object is removed using a *no* command). This -recommendation is orthogonal to the `YANG authoring guidelines for -OpenConfig -models <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ -where the use of presence containers is discouraged. OpenConfig YANG -models however were not designed to replicate the behavior of legacy CLI -commands. \* When using YANG lists, be careful to identify what should -be the key leaves. In the ``offset-list WORD <in|out> (0-16) IFNAME`` -command, for example, both the direction (``<in|out>``) and the -interface name should be the keys of the list. This can be only known by -analyzing the data structures used to store the commands. \* For -clarity, use non-presence containers to group leaves that are associated -to the same configuration command (as we’ll see later, this also -facilitate the process of writing ``cli_show`` callbacks). \* YANG -leaves of type *enumeration* should define explicitly the value of each -*enum* option based on the value used in the FRR source code. \* Default -values should be taken from the source code whenever they exist. +guidelines specific to authors of FRR YANG models: + +* Use presence-containers or lists to model commands that change the CLI node + (e.g. ``router rip``, ``interface eth0``). This way, if the presence-container + or list entry is removed, all configuration options below them are removed + automatically (exactly like the CLI behaves when a configuration object is + removed using a *no* command). This recommendation is orthogonal to the `YANG + authoring guidelines for OpenConfig models + <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ + where the use of presence containers is discouraged. OpenConfig YANG models + however were not designed to replicate the behavior of legacy CLI commands. + +* When using YANG lists, be careful to identify what should be the key leaves. + In the ``offset-list WORD <in|out> (0-16) IFNAME`` command, for example, both + the direction (``<in|out>``) and the interface name should be the keys of the + list. This can be only known by analyzing the data structures used to store + the commands. + +* For clarity, use non-presence containers to group leaves that are associated + to the same configuration command (as we’ll see later, this also facilitate + the process of writing ``cli_show`` callbacks). + +* YANG leaves of type *enumeration* should define explicitly the value of each + *enum* option based on the value used in the FRR source code. + +* Default values should be taken from the source code whenever they exist. Some commands are more difficult to model and demand the use of more advanced YANG constructs like *choice*, *when* and *must* statements. @@ -726,15 +741,17 @@ Configuration options are edited individually ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Several CLI commands edit multiple configuration options at the same -time. Some examples taken from ripd: \* -``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` - -*/frr-ripd:ripd/instance/timers/flush-interval* - -*/frr-ripd:ripd/instance/timers/holddown-interval* - -*/frr-ripd:ripd/instance/timers/update-interval* \* -``distance (1-255) A.B.C.D/M [WORD]`` - -*/frr-ripd:ripd/instance/distance/source/prefix* - -*/frr-ripd:ripd/instance/distance/source/distance* - -*/frr-ripd:ripd/instance/distance/source/access-list* +time. Some examples taken from ripd: + +* ``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` + * */frr-ripd:ripd/instance/timers/flush-interval* + * */frr-ripd:ripd/instance/timers/holddown-interval* + * */frr-ripd:ripd/instance/timers/update-interval* + +* ``distance (1-255) A.B.C.D/M [WORD]`` + * */frr-ripd:ripd/instance/distance/source/prefix* + * */frr-ripd:ripd/instance/distance/source/distance* + * */frr-ripd:ripd/instance/distance/source/access-list* In the new northbound model, there’s one or more separate callbacks for each configuration option. This usually has implications when converting @@ -965,34 +982,30 @@ future. For libfrr commands, it’s not possible to centralize all commands in a single file because the *extract.pl* script from *vtysh* treats commands differently depending on the file in which they are defined (e.g. DEFUNs -from *lib/routemap.c* are installed using the ``VTYSH_RMAP`` constant, +from *lib/routemap.c* are installed using the ``VTYSH_RMAP_SHOW`` constant, which identifies the daemons that support route-maps). In this case, the CLI commands should be rewritten but maintained in the same file. Since all CLI configuration commands from FRR will need to be rewritten, this is an excellent opportunity to rework this part of the code to make the commands easier to maintain and extend. These are the three main -recommendations: 1. Always use DEFPY instead of DEFUN to improve code -readability. 2. Always try to join multiple DEFUNs into a single DEFPY -whenever possible. As an example, there’s no need to have both -``distance (1-255) A.B.C.D/M`` and ``distance (1-255) A.B.C.D/M WORD`` -when a single ``distance (1-255) A.B.C.D/M [WORD]`` would suffice. 3. -When necessary, create a separate DEFPY for ``no`` commands so that part -of the configuration command can be made optional for convenience. -Example: -``no timers basic [(5-2147483647) (5-2147483647) (5-2147483647)]``. In -this example, everything after ``no timers basic`` is ignored by FRR, so -it makes sense to accept ``no timers basic`` as a valid command. But it -also makes sense to accept all parameters -(``no timers basic (5-2147483647) (5-2147483647) (5-2147483647)``) to -make it easier to remove the command just by prefixing a “no” to it. +recommendations: + +#. Always use DEFPY instead of DEFUN to improve code readability +#. Always try to join multiple DEFUNs into a single DEFPY whenever possible. As + an example, there’s no need to have both ``distance (1-255) A.B.C.D/M`` and + ``distance (1-255) A.B.C.D/M WORD`` when a single ``distance (1-255) + A.B.C.D/M [WORD]`` would suffice. +#. When making a negative form of a command, put ``[no]`` in the positive form + and use ``![...]`` to mark portions of the command that should be optional + only in the ``no`` version. To rewrite a CLI command as a dumb wrapper around the northbound callbacks, use the ``nb_cli_cfg_change()`` function. This function accepts as a parameter an array of ``cli_config_change`` structures that specify the changes that need to performed on the candidate configuration. Here’s the declaration of this structure (taken from the -*lib/northbound_cli.h* file): +``lib/northbound_cli.h`` file): .. code:: c @@ -1005,7 +1018,7 @@ configuration. Here’s the declaration of this structure (taken from the /* * Operation to apply (either NB_OP_CREATE, NB_OP_MODIFY or - * NB_OP_DELETE). + * NB_OP_DESTROY). */ enum nb_operation operation; @@ -1034,16 +1047,23 @@ changing the candidate configuration. the northbound callbacks are not involved). Other important details to keep in mind while rewriting the CLI -commands: \* ``nb_cli_cfg_change()`` returns CLI errors codes -(e.g. ``CMD_SUCCESS``, ``CMD_WARNING``), so the return value of this -function can be used as the return value of CLI commands. \* Calls to -``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted to -calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t -necessary anymore and can be removed: ``VTY_DECLVAR_CONTEXT``, -``VTY_DECLVAR_CONTEXT_SUB``, ``VTY_GET_CONTEXT`` and -``VTY_CHECK_CONTEXT``. The ``nb_cli_cfg_change()`` functions uses the -``VTY_CHECK_XPATH`` macro to check if the data node being edited still -exists before doing anything else. +commands: + +* ``nb_cli_cfg_change()`` returns CLI errors codes (e.g. ``CMD_SUCCESS``, + ``CMD_WARNING``), so the return value of this function can be used as the + return value of CLI commands. + +* Calls to ``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted + to calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t + necessary anymore and can be removed: + + * ``VTY_DECLVAR_CONTEXT`` + * ``VTY_DECLVAR_CONTEXT_SUB`` + * ``VTY_GET_CONTEXT`` + * ``VTY_CHECK_CONTEXT``. + + The ``nb_cli_cfg_change()`` functions uses the ``VTY_CHECK_XPATH`` macro to + check if the data node being edited still exists before doing anything else. The examples below provide additional details about how the conversion should be done. @@ -1205,7 +1225,7 @@ This example shows how to create a list entry: }, { .xpath = "./access-list", - .operation = acl ? NB_OP_MODIFY : NB_OP_DELETE, + .operation = acl ? NB_OP_MODIFY : NB_OP_DESTROY, .value = acl, }, }; @@ -1242,7 +1262,7 @@ When deleting a list entry, all non-key leaves can be ignored: struct cli_config_change changes[] = { { .xpath = ".", - .operation = NB_OP_DELETE, + .operation = NB_OP_DESTROY, }, }; @@ -1785,10 +1805,13 @@ Implementation of the ``cli_show`` callback: } This is the most complex ``cli_show`` callback we have in ripd. Its -complexity comes from the following: \* The -``ip rip authentication mode ...`` command changes two YANG leaves at -the same time. \* Part of the command should be hidden when the -``show_defaults`` parameter is set to false. +complexity comes from the following: + +* The ``ip rip authentication mode ...`` command changes two YANG leaves at the + same time. + +* Part of the command should be hidden when the ``show_defaults`` parameter is + set to false. This is the behavior we want to implement: @@ -1838,19 +1861,27 @@ As mentioned in the fourth step, the northbound retrofitting process can happen gradually over time, since both “old” and “new” commands can coexist without problems. Once all commands from a given daemon were converted, we can proceed to the consolidation step, which consists of -the following: \* Remove the vty configuration lock, which is enabled by -default in all daemons. Now multiple users should be able to edit the -configuration concurrently, using either shared or private candidate -configurations. \* Reference commit: -`57dccdb1 <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. -\* Stop using the qobj infrastructure to keep track of configuration -objects. This is not necessary anymore, the northbound uses a similar -mechanism to keep track of YANG data nodes in the candidate -configuration. \* Reference commit: -`4e6d63ce <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. -\* Make the daemon SIGHUP handler re-read the configuration file (and -ensure it’s not doing anything other than that). \* Reference commit: -`5e57edb4 <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. +the following: + +* Remove the vty configuration lock, which is enabled by default in all daemons. + Now multiple users should be able to edit the configuration concurrently, + using either shared or private candidate configurations. + +* Reference commit: `57dccdb1 + <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. + +* Stop using the qobj infrastructure to keep track of configuration objects. + This is not necessary anymore, the northbound uses a similar mechanism to keep + track of YANG data nodes in the candidate configuration. + +* Reference commit: `4e6d63ce + <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. + +* Make the daemon SIGHUP handler re-read the configuration file (and ensure it’s + not doing anything other than that). + +* Reference commit: `5e57edb4 + <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. Final Considerations -------------------- diff --git a/doc/developer/northbound/transactional-cli.rst b/doc/developer/northbound/transactional-cli.rst index 439bb6a..5c495d3 100644 --- a/doc/developer/northbound/transactional-cli.rst +++ b/doc/developer/northbound/transactional-cli.rst @@ -1,25 +1,10 @@ -Table of Contents ------------------ - -- `Introduction <#introduction>`__ -- `Configuration modes <#config-modes>`__ -- `New commands <#retrofitting-process>`__ - - - `commit check <#cmd1>`__ - - `commit <#cmd2>`__ - - `discard <#cmd3>`__ - - `configuration database max-transactions <#cmd4>`__ - - `configuration load <#cmd5>`__ - - `rollback configuration <#cmd6>`__ - - `show configuration candidate <#cmd7>`__ - - `show configuration compare <#cmd8>`__ - - `show configuration running <#cmd9>`__ - - `show configuration transaction <#cmd10>`__ - - `show yang module <#cmd11>`__ - - `show yang module-translator <#cmd12>`__ - - `update <#cmd13>`__ - - `yang module-translator load <#cmd14>`__ - - `yang module-translator unload <#cmd15>`__ +Transactional CLI +================= + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ~~~~~~~~~~~~ @@ -70,18 +55,21 @@ Configuration modes ~~~~~~~~~~~~~~~~~~~ When using the transactional CLI (``--tcli``), FRR supports three -different forms of the ``configure`` command: \* ``configure terminal``: -in this mode, a single candidate configuration is shared by all users. -This means that one user might delete a configuration object that’s -being edited by another user, in which case the CLI will detect and -report the problem. If one user issues the ``commit`` command, all -changes done by all users are committed. \* ``configure private``: users -have a private candidate configuration that is edited separately from -the other users. The ``commit`` command commits only the changes done by -the user. \* ``configure exclusive``: similar to ``configure private``, -but also locks the running configuration to prevent other users from -changing it. The configuration lock is released when the user exits the -configuration mode. +different forms of the ``configure`` command: + +* ``configure terminal``: in this mode, a single candidate configuration is + shared by all users. This means that one user might delete a configuration + object that’s being edited by another user, in which case the CLI will detect + and report the problem. If one user issues the ``commit`` command, all changes + done by all users are committed. + +* ``configure private``: users have a private candidate configuration that is + edited separately from the other users. The ``commit`` command commits only + the changes done by the user. + +* ``configure exclusive``: similar to ``configure private``, but also locks the + running configuration to prevent other users from changing it. The + configuration lock is released when the user exits the configuration mode. When using ``configure terminal`` or ``configure private``, the candidate configuration being edited might become outdated if another @@ -112,12 +100,14 @@ Check if the candidate configuration is valid or not. Commit the changes done in the candidate configuration into the running configuration. -Options: \* ``force``: commit even if the candidate configuration is -outdated. It’s usually a better option to use the ``update`` command -instead. \* ``comment LINE...``: assign a comment to the configuration -transaction. This comment is displayed when viewing the recorded -transactions in the output of the ``show configuration transaction`` -command. +Options: + +* ``force``: commit even if the candidate configuration is outdated. It’s + usually a better option to use the ``update`` command instead. + +* ``comment LINE...``: assign a comment to the configuration transaction. This + comment is displayed when viewing the recorded transactions in the output of + the ``show configuration transaction`` command. ``discard`` ''''''''''' @@ -140,10 +130,13 @@ respectively. It’s also possible to load a configuration from a previous transaction by specifying the desired transaction ID (``(1-4294967296)``). -Options: \* ``translate WORD``: translate the JSON/XML configuration -file using the YANG module translator. \* ``replace``: replace the -candidate by the loaded configuration. The default is to merge the -loaded configuration into the candidate configuration. +Options: + +* ``translate WORD``: translate the JSON/XML configuration file using the YANG + module translator. + +* ``replace``: replace the candidate by the loaded configuration. The default is + to merge the loaded configuration into the candidate configuration. ``rollback configuration (1-4294967296)`` ''''''''''''''''''''''''''''''''''''''''' @@ -156,39 +149,42 @@ identified by its transaction ID (``(1-4294967296)``). Show the candidate configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show only the changes done in the candidate -configuration. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show only the changes done in the candidate configuration. ``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the difference between two different configurations. -Options: \* ``json``: show the configuration differences in the JSON -format. \* ``xml``: show the configuration differences in the XML -format. \* ``translate WORD``: translate the JSON/XML output using the -YANG module translator. +Options: + +* ``json``: show the configuration differences in the JSON format. +* ``xml``: show the configuration differences in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. ``show configuration running [<json|xml> [translate WORD]] [with-defaults]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the running configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. +Options: - NOTE: ``show configuration running`` shows only the running - configuration as known by the northbound layer. Configuration - commands not converted to the new northbound model will not be - displayed. To show the full running configuration, the legacy - ``show running-config`` command must be used. +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. + +NOTE: ``show configuration running`` shows only the running +configuration as known by the northbound layer. Configuration +commands not converted to the new northbound model will not be +displayed. To show the full running configuration, the legacy +``show running-config`` command must be used. ``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]`` ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -199,12 +195,13 @@ configuration associated to the previously committed transaction. When a transaction ID is not given, show all recorded transactions in the rollback log. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show changes compared to the previous -transaction. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show changes compared to the previous transaction. ``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -212,11 +209,14 @@ transaction. When a YANG module is not given, show all loaded YANG modules. Otherwise, show detailed information about the given module. -Options: \* ``module-translator WORD``: change the context to modules -loaded by the specified YANG module translator. \* ``summary``: display -summary information about the module. \* ``tree``: display module in the -tree (RFC 8340) format. \* ``yang``: display module in the YANG format. -\* ``yin``: display module in the YIN format. +Options: + +* ``module-translator WORD``: change the context to modules loaded by the + specified YANG module translator. +* ``summary``: display summary information about the module. +* ``tree``: display module in the tree (RFC 8340) format. +* ``yang``: display module in the YANG format. +* ``yin``: display module in the YIN format. ``show yang module-translator`` ''''''''''''''''''''''''''''''' diff --git a/doc/developer/northbound/yang-module-translator.rst b/doc/developer/northbound/yang-module-translator.rst index aa527ce..17ae160 100644 --- a/doc/developer/northbound/yang-module-translator.rst +++ b/doc/developer/northbound/yang-module-translator.rst @@ -1,11 +1,10 @@ -Table of Contents ------------------ +YANG Module Translation +======================= -- `Introduction <#introduction>`__ -- `Deviation Modules <#deviation-modules>`__ -- `Translation Tables <#translation-tables>`__ -- `CLI Demonstration <#cli-demonstration>`__ -- `Implementation Details <#implementation-details>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ------------ @@ -421,15 +420,20 @@ this shortcoming and make it possible to create more powerful YANG module translators. YANG module translators can be evaluated based on the following metrics: -\* Translation potential: is it possible to make complex translations, -taking several variables into account? \* Complexity: measure of how -easy or hard it is to write a module translator. \* Speed: measure of -how fast the translation can be achieved. Translation speed is of -fundamental importance, especially for operational data. \* Robustness: -can the translator be checked for inconsistencies at load time? A module -translator based on scripts wouldn’t fare well on this metric. \* -Round-trip conversions: can the translated data be translated back to -the original format without information loss? + +* Translation potential: is it possible to make complex translations, taking + several variables into account? + +* Complexity: measure of how easy or hard it is to write a module translator. + +* Speed: measure of how fast the translation can be achieved. Translation speed + is of fundamental importance, especially for operational data. + +* Robustness: can the translator be checked for inconsistencies at load time? A + module translator based on scripts wouldn’t fare well on this metric. + +* Round-trip conversions: can the translated data be translated back to the + original format without information loss? CLI Demonstration ----------------- diff --git a/doc/developer/northbound/yang-tools.rst b/doc/developer/northbound/yang-tools.rst index 346efca..fb5a287 100644 --- a/doc/developer/northbound/yang-tools.rst +++ b/doc/developer/northbound/yang-tools.rst @@ -1,5 +1,5 @@ Yang Tools -~~~~~~~~~~ +========== Here's some information about various tools for working with yang models. @@ -83,17 +83,19 @@ Indent a YANG file: --keep-comments -f yang --yang-canonical \ module.yang -o module.yang -Generate skeleton instance data: \* XML: +Generate skeleton instance data: -.. code:: sh +* XML: + + .. code:: sh $ pyang -p <yang-search-path> \ -f sample-xml-skeleton --sample-xml-skeleton-defaults \ module.yang [augmented-module1.yang ...] -o module.xml -- JSON: +* JSON: -.. code:: sh + .. code:: sh $ pyang -p <yang-search-path> \ -f jsonxsl module.yang -o module.xsl |