diff options
Diffstat (limited to 'doc/developer/northbound/retrofitting-configuration-commands.rst')
| -rw-r--r-- | doc/developer/northbound/retrofitting-configuration-commands.rst | 1897 |
1 files changed, 1897 insertions, 0 deletions
diff --git a/doc/developer/northbound/retrofitting-configuration-commands.rst b/doc/developer/northbound/retrofitting-configuration-commands.rst new file mode 100644 index 0000000..b407246 --- /dev/null +++ b/doc/developer/northbound/retrofitting-configuration-commands.rst @@ -0,0 +1,1897 @@ +Retrofitting Configuration Commands +----------------------------------- + +This page explains how to convert existing CLI configuration commands to +the new northbound model. This documentation is meant to be the primary +reference for developers working on the northbound retrofitting process. +We’ll show several examples taken from the ripd northbound conversion to +illustrate some concepts described herein. + +Retrofitting process +-------------------- + +Step 1: writing a YANG module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first step is to write a YANG module that models faithfully the +commands that are going to be converted. As explained in the +[[Architecture]] page, the goal is to introduce the new YANG-based +Northbound API without introducing backward incompatible changes in the +CLI. The northbound retrofitting process should be completely +transparent to FRR users. + +The developer is free to choose whether to write a full YANG module or a +partial YANG module and increment it gradually. For developers who lack +experience with YANG it’s probably a better idea to model one command at +time. + +It’s recommended to reuse definitions from standard YANG models whenever +possible to facilitate the process of writing module translators using +the [[YANG module translator]]. As an example, the frr-ripd YANG module +incorporated several parts of the IETF RIP YANG module. The repositories +below contain big collections of YANG models that might be used as a +reference: \* https://github.com/YangModels/yang \* +https://github.com/openconfig/public + +When writing a YANG module, it’s highly recommended to follow the +guidelines from `RFC 6087 <https://tools.ietf.org/html/rfc6087>`__. In +general, most commands should be modeled fairly easy. Here are a few +guidelines specific to authors of FRR YANG models: \* Use +presence-containers or lists to model commands that change the CLI node +(e.g. ``router rip``, ``interface eth0``). This way, if the +presence-container or list entry is removed, all configuration options +below them are removed automatically (exactly like the CLI behaves when +a configuration object is removed using a *no* command). This +recommendation is orthogonal to the `YANG authoring guidelines for +OpenConfig +models <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ +where the use of presence containers is discouraged. OpenConfig YANG +models however were not designed to replicate the behavior of legacy CLI +commands. \* When using YANG lists, be careful to identify what should +be the key leaves. In the ``offset-list WORD <in|out> (0-16) IFNAME`` +command, for example, both the direction (``<in|out>``) and the +interface name should be the keys of the list. This can be only known by +analyzing the data structures used to store the commands. \* For +clarity, use non-presence containers to group leaves that are associated +to the same configuration command (as we’ll see later, this also +facilitate the process of writing ``cli_show`` callbacks). \* YANG +leaves of type *enumeration* should define explicitly the value of each +*enum* option based on the value used in the FRR source code. \* Default +values should be taken from the source code whenever they exist. + +Some commands are more difficult to model and demand the use of more +advanced YANG constructs like *choice*, *when* and *must* statements. +**One key requirement is that it should be impossible to load an invalid +JSON/XML configuration to FRR**. The YANG modules should model exactly +what the CLI accepts in the form of commands, and all restrictions +imposed by the CLI should be defined in the YANG models whenever +possible. As we’ll see later, not all constraints can be expressed using +the YANG language and sometimes we’ll need to resort to code-level +validation in the northbound callbacks. + + Tip: the :doc:`yang-tools` page details several tools and commands that + might be useful when writing a YANG module, like validating YANG + files, indenting YANG files, validating instance data, etc. + +In the example YANG snippet below, we can see the use of the *must* +statement that prevents ripd from redistributing RIP routes into itself. +Although ripd CLI doesn’t allow the operator to enter *redistribute rip* +under *router rip*, we don’t have the same protection when configuring +ripd using other northbound interfaces (e.g. NETCONF). So without this +constraint it would be possible to feed an invalid configuration to ripd +(i.e. a bug). + +.. code:: yang + + list redistribute { + key "protocol"; + description + "Redistributes routes learned from other routing protocols."; + leaf protocol { + type frr-route-types:frr-route-types-v4; + description + "Routing protocol."; + must '. != "rip"'; + } + [snip] + } + +In the example below, we use the YANG *choice* statement to ensure that +either the ``password`` leaf or the ``key-chain`` leaf is configured, +but not both. This is in accordance to the sanity checks performed by +the *ip rip authentication* commands. + +.. code:: yang + + choice authentication-data { + description + "Choose whether to use a simple password or a key-chain."; + leaf authentication-password { + type string { + length "1..16"; + } + description + "Authentication string."; + } + leaf authentication-key-chain { + type string; + description + "Key-chain name."; + } + } + +Once finished, the new YANG model should be put into the FRR *yang/* top +level directory. This will ensure it will be installed automatically by +``make install``. It’s also encouraged (but not required) to put sample +configurations under *yang/examples/* using either JSON or XML files. + +Step 2: generate skeleton northbound callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the *gen_northbound_callbacks* tool to generate skeleton callbacks +for the YANG module. Example: + +.. code:: sh + + $ tools/gen_northbound_callbacks frr-ripd > ripd/rip_northbound.c + +The tool will look for the given module in the ``YANG_MODELS_PATH`` +directory defined during the installation. For each schema node of the +YANG module, the tool will generate skeleton callbacks based on the +properties of the node. Example: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance + */ + static int ripd_instance_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + /* TODO: implement me. */ + return NB_OK; + } + + static int ripd_instance_delete(enum nb_event event, + const struct lyd_node *dnode) + { + /* TODO: implement me. */ + return NB_OK; + } + + /* + * XPath: /frr-ripd:ripd/instance/allow-ecmp + */ + static int ripd_instance_allow_ecmp_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + /* TODO: implement me. */ + return NB_OK; + } + + [snip] + + const struct frr_yang_module_info frr_ripd_info = { + .name = "frr-ripd", + .nodes = { + { + .xpath = "/frr-ripd:ripd/instance", + .cbs.create = ripd_instance_create, + .cbs.delete = ripd_instance_delete, + }, + { + .xpath = "/frr-ripd:ripd/instance/allow-ecmp", + .cbs.modify = ripd_instance_allow_ecmp_modify, + }, + [snip] + { + .xpath = "/frr-ripd:ripd/state/routes/route", + .cbs.get_next = ripd_state_routes_route_get_next, + .cbs.get_keys = ripd_state_routes_route_get_keys, + .cbs.lookup_entry = ripd_state_routes_route_lookup_entry, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/prefix", + .cbs.get_elem = ripd_state_routes_route_prefix_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/next-hop", + .cbs.get_elem = ripd_state_routes_route_next_hop_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/interface", + .cbs.get_elem = ripd_state_routes_route_interface_get_elem, + }, + { + .xpath = "/frr-ripd:ripd/state/routes/route/metric", + .cbs.get_elem = ripd_state_routes_route_metric_get_elem, + }, + { + .xpath = "/frr-ripd:clear-rip-route", + .cbs.rpc = clear_rip_route_rpc, + }, + [snip] + +After the C source file is generated, it’s necessary to add a copyright +header on it and indent the code using ``clang-format``. + +Step 3: update the *frr_yang_module_info* array of all relevant daemons +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We must inform the northbound about which daemons will implement the new +YANG module. This is done by updating the ``frr_daemon_info`` structure +of these daemons, with help of the ``FRR_DAEMON_INFO`` macro. + +When a YANG module is specific to a single daemon, like the frr-ripd +module, then only the corresponding daemon should be updated. When the +YANG module is related to a subset of libfrr (e.g. route-maps), then all +FRR daemons that make use of that subset must be updated. + +Example: + +.. code:: c + + static const struct frr_yang_module_info *ripd_yang_modules[] = { + &frr_interface_info, + &frr_ripd_info, + }; + + FRR_DAEMON_INFO(ripd, RIP, .vty_port = RIP_VTY_PORT, + [snip] + .yang_modules = ripd_yang_modules, + .n_yang_modules = array_size(ripd_yang_modules), ) + +Step 4: implement the northbound configuration callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Implementing the northbound configuration callbacks consists mostly of +copying code from the corresponding CLI commands and make the required +adaptations. + +It’s recommended to convert one command or a small group of related +commands per commit. Small commits are preferred to facilitate the +review process. Both “old” and “new” command can coexist without +problems, so the retrofitting process can happen gradually over time. + +The configuration callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These are the four main northbound configuration callbacks, as defined +in the ``lib/northbound.h`` file: + +.. code:: c + + /* + * Configuration callback. + * + * A presence container, list entry, leaf-list entry or leaf of type + * empty has been created. + * + * For presence-containers and list entries, the callback is supposed to + * initialize the default values of its children (if any) from the YANG + * models. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being created. + * + * resource + * Pointer to store resource(s) allocated during the NB_EV_PREPARE + * phase. The same pointer can be used during the NB_EV_ABORT and + * NB_EV_APPLY phases to either release or make use of the allocated + * resource(s). It's set to NULL when the event is NB_EV_VALIDATE. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_RESOURCE when the callback failed to allocate a resource. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*create)(enum nb_event event, const struct lyd_node *dnode, + union nb_resource *resource); + + /* + * Configuration callback. + * + * The value of a leaf has been modified. + * + * List keys don't need to implement this callback. When a list key is + * modified, the northbound treats this as if the list was deleted and a + * new one created with the updated key value. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being modified + * + * resource + * Pointer to store resource(s) allocated during the NB_EV_PREPARE + * phase. The same pointer can be used during the NB_EV_ABORT and + * NB_EV_APPLY phases to either release or make use of the allocated + * resource(s). It's set to NULL when the event is NB_EV_VALIDATE. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_RESOURCE when the callback failed to allocate a resource. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*modify)(enum nb_event event, const struct lyd_node *dnode, + union nb_resource *resource); + + /* + * Configuration callback. + * + * A presence container, list entry, leaf-list entry or optional leaf + * has been deleted. + * + * The callback is supposed to delete the entire configuration object, + * including its children when they exist. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being deleted. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*delete)(enum nb_event event, const struct lyd_node *dnode); + + /* + * Configuration callback. + * + * A list entry or leaf-list entry has been moved. Only applicable when + * the "ordered-by user" statement is present. + * + * event + * The transaction phase. Refer to the documentation comments of + * nb_event for more details. + * + * dnode + * libyang data node that is being moved. + * + * Returns: + * - NB_OK on success. + * - NB_ERR_VALIDATION when a validation error occurred. + * - NB_ERR_INCONSISTENCY when an inconsistency was detected. + * - NB_ERR for other errors. + */ + int (*move)(enum nb_event event, const struct lyd_node *dnode); + +Since skeleton northbound callbacks are generated automatically by the +*gen_northbound_callbacks* tool, the developer doesn’t need to worry +about which callbacks need to be implemented. + + NOTE: once a daemon starts, it reads its YANG modules and validates + that all required northbound callbacks were implemented. If any + northbound callback is missing, an error is logged and the program + exists. + +Transaction phases +^^^^^^^^^^^^^^^^^^ + +Configuration transactions and their phases were described in detail in +the [[Architecture]] page. Here’s the definition of the ``nb_event`` +enumeration as defined in the *lib/northbound.h* file: + +.. code:: c + + /* Northbound events. */ + enum nb_event { + /* + * The configuration callback is supposed to verify that the changes are + * valid and can be applied. + */ + NB_EV_VALIDATE, + + /* + * The configuration callback is supposed to prepare all resources + * required to apply the changes. + */ + NB_EV_PREPARE, + + /* + * Transaction has failed, the configuration callback needs to release + * all resources previously allocated. + */ + NB_EV_ABORT, + + /* + * The configuration changes need to be applied. The changes can't be + * rejected at this point (errors are logged and ignored). + */ + NB_EV_APPLY, + }; + +When converting a CLI command, we must identify all error-prone +operations and perform them in the ``NB_EV_PREPARE`` phase of the +northbound callbacks. When the operation in question involves the +allocation of a specific resource (e.g. file descriptors), we can store +the allocated resource in the ``resource`` variable given to the +callback. This way the allocated resource can be obtained in the other +phases of the transaction using the same parameter. + +Here’s the ``create`` northbound callback associated to the +``router rip`` command: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance + */ + static int ripd_instance_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + int socket; + + switch (event) { + case NB_EV_VALIDATE: + break; + case NB_EV_PREPARE: + socket = rip_create_socket(); + if (socket < 0) + return NB_ERR_RESOURCE; + resource->fd = socket; + break; + case NB_EV_ABORT: + socket = resource->fd; + close(socket); + break; + case NB_EV_APPLY: + socket = resource->fd; + rip_create(socket); + break; + } + + return NB_OK; + } + +Note that the socket creation is an error-prone operation since it +depends on the underlying operating system, so the socket must be +created during the ``NB_EV_PREPARE`` phase and stored in +``resource->fd``. This socket is then either closed or used depending on +the outcome of the preparation phase of the whole transaction. + +During the ``NB_EV_VALIDATE`` phase, the northbound callbacks must +validate if the intended changes are valid. As an example, FRR doesn’t +allow the operator to deconfigure active interfaces: + +.. code:: c + + static int lib_interface_delete(enum nb_event event, + const struct lyd_node *dnode) + { + struct interface *ifp; + + ifp = yang_dnode_get_entry(dnode); + + switch (event) { + case NB_EV_VALIDATE: + if (CHECK_FLAG(ifp->status, ZEBRA_INTERFACE_ACTIVE)) { + zlog_warn("%s: only inactive interfaces can be deleted", + __func__); + return NB_ERR_VALIDATION; + } + break; + case NB_EV_PREPARE: + case NB_EV_ABORT: + break; + case NB_EV_APPLY: + if_delete(ifp); + break; + } + + return NB_OK; + } + +Note however that it’s preferred to use YANG to model the validation +constraints whenever possible. Code-level validations should be used +only to validate constraints that can’t be modeled using the YANG +language. + +Most callbacks don’t need to perform any validations nor perform any +error-prone operations, so in these cases we can use the following +pattern to return early if ``event`` is different than ``NB_EV_APPLY``: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/distance/default + */ + static int ripd_instance_distance_default_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + if (event != NB_EV_APPLY) + return NB_OK; + + rip->distance = yang_dnode_get_uint8(dnode, NULL); + + return NB_OK; + } + +During development it’s recommend to use the *debug northbound* command +to debug configuration transactions and see what callbacks are being +called. Example: + +:: + + ripd# conf t + ripd(config)# debug northbound + ripd(config)# router rip + ripd(config-router)# allow-ecmp + ripd(config-router)# network eth0 + ripd(config-router)# redistribute ospf metric 2 + ripd(config-router)# commit + % Configuration committed successfully. + + ripd(config-router)# + +Now the ripd log: + +:: + + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [validate] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [prepare] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [modify] xpath [/frr-ripd:ripd/instance/allow-ecmp] value [true] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance/interface[.='eth0']] value [eth0] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [create] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(none)] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [modify] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']/metric] value [2] + 2018/09/23 12:43:59 RIP: northbound callback: event [apply] op [apply_finish] xpath [/frr-ripd:ripd/instance/redistribute[protocol='ospf']] value [(null)] + +Getting the data +^^^^^^^^^^^^^^^^ + +One parameter that is common to all northbound configuration callbacks +is the ``dnode`` parameter. This is a libyang data node structure that +contains information relative to the configuration change that is being +performed. For ``create`` callbacks, it contains the configuration node +that is being added. For ``delete`` callbacks, it contains the +configuration node that is being deleted. For ``modify`` callbacks, it +contains the configuration node that is being modified. + +In order to get the actual data value out of the ``dnode`` variable, we +need to use the ``yang_dnode_get_*()`` wrappers documented in +*lib/yang_wrappers.h*. + +The advantage of passing a ``dnode`` structure to the northbound +callbacks is that the whole candidate being committed is made available, +so the callbacks can obtain values from other portions of the +configuration if necessary. This can be done by providing an xpath +expression to the second parameter of the ``yang_dnode_get_*()`` +wrappers to specify the element we want to get. The example below shows +a callback that gets the values of two leaves that are part of the same +list entry: + +.. code:: c + + static int + ripd_instance_redistribute_metric_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + int type; + uint8_t metric; + + if (event != NB_EV_APPLY) + return NB_OK; + + type = yang_dnode_get_enum(dnode, "../protocol"); + metric = yang_dnode_get_uint8(dnode, NULL); + + rip->route_map[type].metric_config = true; + rip->route_map[type].metric = metric; + rip_redistribute_conf_update(type); + + return NB_OK; + } + +.. + + NOTE: if the wrong ``yang_dnode_get_*()`` wrapper is used, the code + will log an error and abort. An example would be using + ``yang_dnode_get_enum()`` to get the value of a boolean data node. + +No need to check if the configuration value has changed +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A common pattern in CLI commands is this: + +.. code:: c + + DEFUN (...) + { + [snip] + if (new_value == old_value) + return CMD_SUCCESS; + [snip] + } + +Several commands need to check if the new value entered by the user is +the same as the one currently configured. Then, if yes, ignore the +command since nothing was changed. + +The northbound callbacks on the other hand don’t need to perform this +check since they act on effective configuration changes. Using the CLI +as an example, if the operator enters the same command multiple times, +the northbound layer will detect that nothing has changed in the +configuration and will avoid calling the northbound callbacks +unnecessarily. + +In some cases, however, it might be desirable to check for +inconsistencies and notify the northbound when that happens: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/interface + */ + static int ripd_instance_interface_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + const char *ifname; + + if (event != NB_EV_APPLY) + return NB_OK; + + ifname = yang_dnode_get_string(dnode, NULL); + + return rip_enable_if_add(ifname); + } + +.. code:: c + + /* Add interface to rip_enable_if. */ + int rip_enable_if_add(const char *ifname) + { + int ret; + + ret = rip_enable_if_lookup(ifname); + if (ret >= 0) + return NB_ERR_INCONSISTENCY; + + vector_set(rip_enable_interface, + XSTRDUP(MTYPE_RIP_INTERFACE_STRING, ifname)); + + rip_enable_apply_all(); /* TODOVJ */ + + return NB_OK; + } + +In the example above, the ``rip_enable_if_add()`` function should never +return ``NB_ERR_INCONSISTENCY`` in normal conditions. This is because +the northbound layer guarantees that the same interface will never be +added more than once (except when it’s removed and re-added again). But +to be on the safe side it’s probably wise to check for internal +inconsistencies to ensure everything is working as expected. + +Default values +^^^^^^^^^^^^^^ + +Whenever creating a new presence-container or list entry, it’s usually +necessary to initialize certain variables to their default values. FRR +most of the time uses special constants for that purpose +(e.g. ``RIP_DEFAULT_METRIC_DEFAULT``, ``DFLT_BGP_HOLDTIME``, etc). Now +that we have YANG models, we want to fetch the default values from these +models instead. This will allow us to changes default values smoothly +without needing to touch the code. Better yet, it will allow users to +create YANG deviations to define custom default values easily. + +To fetch default values from the loaded YANG models, use the +``yang_get_default_*()`` wrapper functions +(e.g. ``yang_get_default_bool()``) documented in *lib/yang_wrappers.h*. + +Example: + +.. code:: c + + int rip_create(int socket) + { + rip = XCALLOC(MTYPE_RIP, sizeof(struct rip)); + + /* Set initial values. */ + rip->ecmp = yang_get_default_bool("%s/allow-ecmp", RIP_INSTANCE); + rip->default_metric = + yang_get_default_uint8("%s/default-metric", RIP_INSTANCE); + [snip] + } + +Configuration options are edited individually +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Several CLI commands edit multiple configuration options at the same +time. Some examples taken from ripd: \* +``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` - +*/frr-ripd:ripd/instance/timers/flush-interval* - +*/frr-ripd:ripd/instance/timers/holddown-interval* - +*/frr-ripd:ripd/instance/timers/update-interval* \* +``distance (1-255) A.B.C.D/M [WORD]`` - +*/frr-ripd:ripd/instance/distance/source/prefix* - +*/frr-ripd:ripd/instance/distance/source/distance* - +*/frr-ripd:ripd/instance/distance/source/access-list* + +In the new northbound model, there’s one or more separate callbacks for +each configuration option. This usually has implications when converting +code from CLI commands to the northbound commands. An example of this is +the following commit from ripd: +`7cf2f2eaf <https://github.com/opensourcerouting/frr/commit/7cf2f2eaf43ef5df294625d1ab4c708db8293510>`__. +The ``rip_distance_set()`` and ``rip_distance_unset()`` functions were +torn apart and their code split into a few different callbacks. + +For lists and presence-containers, it’s possible to use the +``yang_dnode_set_entry()`` function to attach user data to a libyang +data node, and then retrieve this value in the other callbacks (for the +same node or any of its children) using the ``yang_dnode_get_entry()`` +function. Example: + +.. code:: c + + static int ripd_instance_distance_source_create(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + struct prefix_ipv4 prefix; + struct route_node *rn; + + if (event != NB_EV_APPLY) + return NB_OK; + + yang_dnode_get_ipv4p(&prefix, dnode, "./prefix"); + + /* Get RIP distance node. */ + rn = route_node_get(rip_distance_table, (struct prefix *)&prefix); + rn->info = rip_distance_new(); + yang_dnode_set_entry(dnode, rn); + + return NB_OK; + } + +.. code:: c + + static int + ripd_instance_distance_source_distance_modify(enum nb_event event, + const struct lyd_node *dnode, + union nb_resource *resource) + { + struct route_node *rn; + uint8_t distance; + struct rip_distance *rdistance; + + if (event != NB_EV_APPLY) + return NB_OK; + + /* Set distance value. */ + rn = yang_dnode_get_entry(dnode); + distance = yang_dnode_get_uint8(dnode, NULL); + rdistance = rn->info; + rdistance->distance = distance; + + return NB_OK; + } + +Commands that edit multiple configuration options at the same time can +also use the ``apply_finish`` optional callback, documented as follows +in the *lib/northbound.h* file: + +.. code:: c + + /* + * Optional configuration callback for YANG lists and containers. + * + * The 'apply_finish' callbacks are called after all other callbacks + * during the apply phase (NB_EV_APPLY). These callbacks are called only + * under one of the following two cases: + * * The container or a list entry has been created; + * * Any change is made within the descendants of the list entry or + * container (e.g. a child leaf was modified, created or deleted). + * + * This callback is useful in the cases where a single event should be + * triggered regardless if the container or list entry was changed once + * or multiple times. + * + * dnode + * libyang data node from the YANG list or container. + */ + void (*apply_finish)(const struct lyd_node *dnode); + +Here’s an example of how this callback can be used: + +.. code:: c + + /* + * XPath: /frr-ripd:ripd/instance/timers/ + */ + static void ripd_instance_timers_apply_finish(const struct lyd_node *dnode) + { + /* Reset update timer thread. */ + rip_event(RIP_UPDATE_EVENT, 0); + } + +.. code:: c + + { + .xpath = "/frr-ripd:ripd/instance/timers", + .cbs.apply_finish = ripd_instance_timers_apply_finish, + .cbs.cli_show = cli_show_rip_timers, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/flush-interval", + .cbs.modify = ripd_instance_timers_flush_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/holddown-interval", + .cbs.modify = ripd_instance_timers_holddown_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/update-interval", + .cbs.modify = ripd_instance_timers_update_interval_modify, + }, + +In this example, we want to call the ``rip_event()`` function only once +regardless if all RIP timers were modified or only one of them. Without +the ``apply_finish`` callback we’d need to call ``rip_event()`` in the +``modify`` callback of each timer (a YANG leaf), resulting in redundant +call to the ``rip_event()`` function if multiple timers are changed at +once. + +Bonus: libyang user types +^^^^^^^^^^^^^^^^^^^^^^^^^ + +When writing YANG modules, it’s advisable to create derived types for +data types that are used on multiple places (e.g. MAC addresses, IS-IS +networks, etc). Here’s how `RFC +7950 <https://tools.ietf.org/html/rfc7950#page-25>`__ defines derived +types: > YANG can define derived types from base types using the +“typedef” > statement. A base type can be either a built-in type or a +derived > type, allowing a hierarchy of derived types. > > A derived +type can be used as the argument for the “type” statement. > > YANG +Example: > > typedef percent { > type uint8 { > range “0 .. 100”; > } > +} > > leaf completed { > type percent; > } + +Derived types are essentially built-in types with imposed restrictions. +As an example, the ``ipv4-address`` derived type from IETF is defined +using the ``string`` built-in type with a ``pattern`` constraint (a +regular expression): + +:: + + typedef ipv4-address { + type string { + pattern + '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}' + + '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])' + + '(%[\p{N}\p{L}]+)?'; + } + description + "The ipv4-address type represents an IPv4 address in + dotted-quad notation. The IPv4 address may include a zone + index, separated by a % sign. + + The zone index is used to disambiguate identical address + values. For link-local addresses, the zone index will + typically be the interface index number or the name of an + interface. If the zone index is not present, the default + zone of the device will be used. + + The canonical format for the zone index is the numerical + format"; + } + +Sometimes, however, it’s desirable to have a binary representation of +the derived type that is different from the associated built-in type. +Taking the ``ipv4-address`` example above, it would be more convenient +to manipulate this YANG type using ``in_addr`` structures instead of +strings. libyang allow us to do that using the user types plugin: +https://netopeer.liberouter.org/doc/libyang/master/howtoschemaplugins.html#usertypes + +Here’s how the the ``ipv4-address`` derived type is implemented in FRR +(*yang/libyang_plugins/frr_user_types.c*): + +.. code:: c + + static int ipv4_address_store_clb(const char *type_name, const char *value_str, + lyd_val *value, char **err_msg) + { + value->ptr = malloc(sizeof(struct in_addr)); + if (!value->ptr) + return 1; + + if (inet_pton(AF_INET, value_str, value->ptr) != 1) { + free(value->ptr); + return 1; + } + + return 0; + } + +.. code:: c + + struct lytype_plugin_list frr_user_types[] = { + {"ietf-inet-types", "2013-07-15", "ipv4-address", + ipv4_address_store_clb, free}, + {"ietf-inet-types", "2013-07-15", "ipv4-address-no-zone", + ipv4_address_store_clb, free}, + [snip] + {NULL, NULL, NULL, NULL, NULL} /* terminating item */ + }; + +Now, in addition to the string representation of the data value, libyang +will also store the data in the binary format we specified (an +``in_addr`` structure). + +Whenever a new derived type is implemented in FRR, it’s also recommended +to write new wrappers in the *lib/yang_wrappers.c* file +(e.g. ``yang_dnode_get_ipv4()``, ``yang_get_default_ipv4()``, etc). + +Step 5: rewrite the CLI commands as dumb wrappers around the northbound callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once the northbound callbacks are implemented, we need to rewrite the +associated CLI commands on top of the northbound layer. This is the +easiest part of the retrofitting process. + +For protocol daemons, it’s recommended to put all CLI commands on a +separate C file (e.g. *ripd/rip_cli.c*). This helps to keep the code +more clean by separating the main protocol code from the user interface. +It should also help when moving the CLI to a separate program in the +future. + +For libfrr commands, it’s not possible to centralize all commands in a +single file because the *extract.pl* script from *vtysh* treats commands +differently depending on the file in which they are defined (e.g. DEFUNs +from *lib/routemap.c* are installed using the ``VTYSH_RMAP`` constant, +which identifies the daemons that support route-maps). In this case, the +CLI commands should be rewritten but maintained in the same file. + +Since all CLI configuration commands from FRR will need to be rewritten, +this is an excellent opportunity to rework this part of the code to make +the commands easier to maintain and extend. These are the three main +recommendations: 1. Always use DEFPY instead of DEFUN to improve code +readability. 2. Always try to join multiple DEFUNs into a single DEFPY +whenever possible. As an example, there’s no need to have both +``distance (1-255) A.B.C.D/M`` and ``distance (1-255) A.B.C.D/M WORD`` +when a single ``distance (1-255) A.B.C.D/M [WORD]`` would suffice. 3. +When necessary, create a separate DEFPY for ``no`` commands so that part +of the configuration command can be made optional for convenience. +Example: +``no timers basic [(5-2147483647) (5-2147483647) (5-2147483647)]``. In +this example, everything after ``no timers basic`` is ignored by FRR, so +it makes sense to accept ``no timers basic`` as a valid command. But it +also makes sense to accept all parameters +(``no timers basic (5-2147483647) (5-2147483647) (5-2147483647)``) to +make it easier to remove the command just by prefixing a “no” to it. + +To rewrite a CLI command as a dumb wrapper around the northbound +callbacks, use the ``nb_cli_cfg_change()`` function. This function +accepts as a parameter an array of ``cli_config_change`` structures that +specify the changes that need to performed on the candidate +configuration. Here’s the declaration of this structure (taken from the +*lib/northbound_cli.h* file): + +.. code:: c + + struct cli_config_change { + /* + * XPath (absolute or relative) of the configuration option being + * edited. + */ + char xpath[XPATH_MAXLEN]; + + /* + * Operation to apply (either NB_OP_CREATE, NB_OP_MODIFY or + * NB_OP_DELETE). + */ + enum nb_operation operation; + + /* + * New value of the configuration option. Should be NULL for typeless + * YANG data (e.g. presence-containers). For convenience, NULL can also + * be used to restore a leaf to its default value. + */ + const char *value; + }; + +The ``nb_cli_cfg_change()`` function positions the CLI command on top on +top of the northbound layer. Instead of changing the running +configuration directly, this function changes the candidate +configuration instead, as described in the [[Transactional CLI]] page. +When the transactional CLI is not in use (i.e. the default mode), then +``nb_cli_cfg_change()`` performs an implicit ``commit`` operation after +changing the candidate configuration. + + NOTE: the ``nb_cli_cfg_change()`` function clones the candidate + configuration before actually editing it. This way, if any error + happens during the editing, the original candidate is restored to + avoid inconsistencies. Either all changes from the configuration + command are performed successfully or none are. It’s like a + mini-transaction but happening on the candidate configuration (thus + the northbound callbacks are not involved). + +Other important details to keep in mind while rewriting the CLI +commands: \* ``nb_cli_cfg_change()`` returns CLI errors codes +(e.g. ``CMD_SUCCESS``, ``CMD_WARNING``), so the return value of this +function can be used as the return value of CLI commands. \* Calls to +``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted to +calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t +necessary anymore and can be removed: ``VTY_DECLVAR_CONTEXT``, +``VTY_DECLVAR_CONTEXT_SUB``, ``VTY_GET_CONTEXT`` and +``VTY_CHECK_CONTEXT``. The ``nb_cli_cfg_change()`` functions uses the +``VTY_CHECK_XPATH`` macro to check if the data node being edited still +exists before doing anything else. + +The examples below provide additional details about how the conversion +should be done. + +Example 1 +^^^^^^^^^ + +In this first example, the *router rip* command becomes a dumb wrapper +around the ``ripd_instance_create()`` callback. Note that we don’t need +to check if the ``/frr-ripd:ripd/instance`` data path already exists +before trying to create it. The northbound will detect when this +presence-container already exists and do nothing. The +``VTY_PUSH_XPATH()`` macro is used to change the vty node and set the +context for other commands under *router rip*. + +.. code:: c + + DEFPY_NOSH (router_rip, + router_rip_cmd, + "router rip", + "Enable a routing process\n" + "Routing Information Protocol (RIP)\n") + { + int ret; + + struct cli_config_change changes[] = { + { + .xpath = "/frr-ripd:ripd/instance", + .operation = NB_OP_CREATE, + .value = NULL, + }, + }; + + ret = nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + if (ret == CMD_SUCCESS) + VTY_PUSH_XPATH(RIP_NODE, changes[0].xpath); + + return ret; + } + +Example 2 +^^^^^^^^^ + +Here we can see the use of relative xpaths (starting with ``./``), which +are more convenient that absolute xpaths (which would be +``/frr-ripd:ripd/instance/default-metric`` in this example). This is +possible because the use of ``VTY_PUSH_XPATH()`` in the *router rip* +command set the vty base xpath to ``/frr-ripd:ripd/instance``. + +.. code:: c + + DEFPY (rip_default_metric, + rip_default_metric_cmd, + "default-metric (1-16)", + "Set a metric of redistribute routes\n" + "Default metric\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./default-metric", + .operation = NB_OP_MODIFY, + .value = default_metric_str, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +In the command below we the ``value`` to NULL to indicate that we want +to set this leaf to its default value. This is better than hardcoding +the default value because the default might change in the future. Also, +users might define custom defaults by using YANG deviations, so it’s +better to write code that works correctly regardless of the default +values defined in the YANG models. + +.. code:: c + + DEFPY (no_rip_default_metric, + no_rip_default_metric_cmd, + "no default-metric [(1-16)]", + NO_STR + "Set a metric of redistribute routes\n" + "Default metric\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./default-metric", + .operation = NB_OP_MODIFY, + .value = NULL, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +Example 3 +^^^^^^^^^ + +This example shows how one command can change multiple leaves at the +same time. + +.. code:: c + + DEFPY (rip_timers, + rip_timers_cmd, + "timers basic (5-2147483647)$update (5-2147483647)$timeout (5-2147483647)$garbage", + "Adjust routing timers\n" + "Basic routing protocol update timers\n" + "Routing table update timer value in second. Default is 30.\n" + "Routing information timeout timer. Default is 180.\n" + "Garbage collection timer. Default is 120.\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./timers/update-interval", + .operation = NB_OP_MODIFY, + .value = update_str, + }, + { + .xpath = "./timers/holddown-interval", + .operation = NB_OP_MODIFY, + .value = timeout_str, + }, + { + .xpath = "./timers/flush-interval", + .operation = NB_OP_MODIFY, + .value = garbage_str, + }, + }; + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +Example 4 +^^^^^^^^^ + +This example shows how to create a list entry: + +.. code:: c + + DEFPY (rip_distance_source, + rip_distance_source_cmd, + "distance (1-255) A.B.C.D/M$prefix [WORD$acl]", + "Administrative distance\n" + "Distance value\n" + "IP source prefix\n" + "Access list name\n") + { + char xpath_list[XPATH_MAXLEN]; + struct cli_config_change changes[] = { + { + .xpath = ".", + .operation = NB_OP_CREATE, + }, + { + .xpath = "./distance", + .operation = NB_OP_MODIFY, + .value = distance_str, + }, + { + .xpath = "./access-list", + .operation = acl ? NB_OP_MODIFY : NB_OP_DELETE, + .value = acl, + }, + }; + + snprintf(xpath_list, sizeof(xpath_list), "./distance/source[prefix='%s']", + prefix_str); + + return nb_cli_cfg_change(vty, xpath_list, changes, array_size(changes)); + } + +The ``xpath_list`` variable is used to hold the xpath that identifies +the list entry. The keys of the list entry should be embedded in this +xpath and don’t need to be part of the array of configuration changes. +All entries from the ``changes`` array use relative xpaths which are +based on the xpath of the list entry. + +The ``access-list`` optional leaf can be either modified or deleted +depending whether the optional *WORD* parameter is present or not. + +When deleting a list entry, all non-key leaves can be ignored: + +.. code:: c + + DEFPY (no_rip_distance_source, + no_rip_distance_source_cmd, + "no distance (1-255) A.B.C.D/M$prefix [WORD$acl]", + NO_STR + "Administrative distance\n" + "Distance value\n" + "IP source prefix\n" + "Access list name\n") + { + char xpath_list[XPATH_MAXLEN]; + struct cli_config_change changes[] = { + { + .xpath = ".", + .operation = NB_OP_DELETE, + }, + }; + + snprintf(xpath_list, sizeof(xpath_list), "./distance/source[prefix='%s']", + prefix_str); + + return nb_cli_cfg_change(vty, xpath_list, changes, 1); + } + +Example 5 +^^^^^^^^^ + +This example shows a DEFPY statement that performs two validations +before calling ``nb_cli_cfg_change()``: + +.. code:: c + + DEFPY (ip_rip_authentication_string, + ip_rip_authentication_string_cmd, + "ip rip authentication string LINE$password", + IP_STR + "Routing Information Protocol\n" + "Authentication control\n" + "Authentication string\n" + "Authentication string\n") + { + struct cli_config_change changes[] = { + { + .xpath = "./frr-ripd:rip/authentication/password", + .operation = NB_OP_MODIFY, + .value = password, + }, + }; + + if (strlen(password) > 16) { + vty_out(vty, + "%% RIPv2 authentication string must be shorter than 16\n"); + return CMD_WARNING_CONFIG_FAILED; + } + + if (yang_dnode_exists(vty->candidate_config->dnode, "%s%s", + VTY_GET_XPATH, + "/frr-ripd:rip/authentication/key-chain")) { + vty_out(vty, "%% key-chain configuration exists\n"); + return CMD_WARNING_CONFIG_FAILED; + } + + return nb_cli_cfg_change(vty, NULL, changes, array_size(changes)); + } + +These two validations are not strictly necessary since the configuration +change is validated using libyang afterwards. The issue with the libyang +validation is that the error messages from libyang are too verbose: + +:: + + ripd# conf t + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication string XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + % Failed to edit candidate configuration. + + Value "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" does not satisfy the constraint "1..16" (range, length, or pattern). + Failed to create node "authentication-password" as a child of "rip". + YANG path: /frr-interface:lib/interface[name='eth0'][vrf='Default-IP-Routing-Table']/frr-ripd:rip/authentication-password + +On the other hand, the original error message from ripd is much cleaner: + +:: + + ripd# conf t + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication string XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + % RIPv2 authentication string must be shorter than 16 + +The second validation is a bit more complex. If we try to create the +``authentication/password`` leaf when the ``authentication/key-chain`` +leaf already exists (both are under a YANG *choice* statement), libyang +will automatically delete the ``authentication/key-chain`` and create +``authentication/password`` on its place. This is different from the +original ripd behavior where the *ip rip authentication key-chain* +command must be removed before configuring the *ip rip authentication +string* command. + +In the spirit of not introducing any backward-incompatible changes in +the CLI, converted commands should retain some of their validation +checks to preserve their original behavior. + +Step 6: implement the ``cli_show`` callbacks +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The traditional method used by FRR to display the running configuration +consists of looping through all CLI nodes all call their ``func`` +callbacks one by one, which in turn read the configuration from internal +variables and dump them to the terminal in the form of CLI commands. + +The problem with this approach is twofold. First, since the callbacks +read the configuration from internal variables, they can’t display +anything other than the running configuration. Second, they don’t have +the ability to display default values when requested by the user +(e.g. *show configuration candidate with-defaults*). + +The new northbound architecture solves these problems by introducing a +new callback: ``cli_show``. Here’s the signature of this function (taken +from the *lib/northbound.h* file): + +.. code:: c + + /* + * Optional callback to show the CLI command associated to the given + * YANG data node. + * + * vty + * the vty terminal to dump the configuration to + * + * dnode + * libyang data node that should be shown in the form of a CLI + * command + * + * show_defaults + * specify whether to display default configuration values or not. + * This parameter can be ignored most of the time since the + * northbound doesn't call this callback for default leaves or + * non-presence containers that contain only default child nodes. + * The exception are commands associated to multiple configuration + * options, in which case it might be desirable to hide one or more + * parts of the command when this parameter is set to false. + */ + void (*cli_show)(struct vty *vty, struct lyd_node *dnode, + bool show_defaults); + +One of the main differences to the old CLI ``func`` callbacks is that +the ``cli_show`` callbacks are associated to YANG data paths and not to +CLI nodes. This means we can define one separate callback for each CLI +command, making the code more modular and easier to maintain (among +other advantages that will be more clear later). For enhanced code +readability, it’s recommended to position the ``cli_show`` callbacks +immediately after their associated command definitions (DEFPYs). + +The ``cli_show`` callbacks are used by the ``nb_cli_show_config_cmds()`` +function to display configurations stored inside ``nb_config`` +structures. The configuration being displayed can be anything from the +running configuration (*show configuration running*), a candidate +configuration (*show configuration candidate*) or a rollback +configuration (*show configuration transaction (1-4294967296)*). The +``nb_cli_show_config_cmds()`` function works by iterating over all data +nodes from the given configuration and calling the ``cli_show`` callback +for the nodes where it’s defined. If a list has dozens of entries, the +``cli_show`` callback associated to this list will be called multiple +times with the ``dnode`` parameter pointing to different list entries on +each iteration. + +For backward compatibility with the *show running-config* command, we +can’t get rid of the CLI ``func`` callbacks at this point in time. +However, we can make the CLI ``func`` callbacks call the corresponding +``cli_show`` callbacks to avoid code duplication. The +``nb_cli_show_dnode_cmds()`` function can be used for that purpose. Once +the CLI retrofitting process finishes for all FRR daemons, we can remove +the legacy CLI ``func`` callbacks and turn *show running-config* into a +shorthand for *show configuration running*. + +Regarding displaying configuration with default values, this is +something that is taken care of by the ``nb_cli_show_config_cmds()`` +function itself. When the *show configuration* command is used without +the *with-defaults* option, ``nb_cli_show_config_cmds()`` will skip +calling ``cli_show`` callbacks for data nodes that contain only default +values (e.g. default leaves or non-presence containers that contain only +default child nodes). There are however some exceptional cases where the +implementer of the ``cli_show`` callback should take into consideration +if default values should be displayed or not. This and other concepts +will be explained in more detail in the examples below. + +.. _example-1-1: + +Example 1 +^^^^^^^^^ + +Command: ``default-metric (1-16)`` + +YANG representation: + +.. code:: yang + + leaf default-metric { + type uint8 { + range "1..16"; + } + default "1"; + description + "Default metric of redistributed routes."; + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance/default-metric", + .cbs.modify = ripd_instance_default_metric_modify, + + .cbs.cli_show = cli_show_rip_default_metric, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_default_metric(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " default-metric %s\n", + yang_dnode_get_string(dnode, NULL)); + } + +In this first example, the *default-metric* command was modeled using a +YANG leaf, and we added a new ``cli_show`` callback attached to the YANG +path of this leaf. + +The callback makes use of the ``yang_dnode_get_string()`` function to +obtain the string value of the configuration option. The following would +also be possible: + +.. code:: c + + vty_out(vty, " default-metric %u\n", + yang_dnode_get_uint8(dnode, NULL)); + +Both options are possible because libyang stores both a binary +representation and a textual representation of all values stored in a +data node (``lyd_node``). For simplicity, it’s recommended to always use +``yang_dnode_get_string()`` in the ``cli_show`` callbacks. + +.. _example-2-1: + +Example 2 +^^^^^^^^^ + +Command: ``router rip`` + +YANG representation: + +.. code:: yang + + container instance { + presence "Present if the RIP protocol is enabled."; + description + "RIP routing instance."; + [snip] + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance", + .cbs.create = ripd_instance_create, + .cbs.delete = ripd_instance_delete, + + .cbs.cli_show = cli_show_router_rip, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_router_rip(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, "!\n"); + vty_out(vty, "router rip\n"); + } + +In this example, the ``cli_show`` callback doesn’t need to obtain any +value from the ``dnode`` parameter since presence-containers don’t hold +any data (apart from their child nodes, but they have their own +``cli_show`` callbacks). + +.. _example-3-1: + +Example 3 +^^^^^^^^^ + +Command: ``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` + +YANG representation: + +.. code:: yang + + container timers { + description + "Settings of basic timers"; + leaf flush-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "120"; + description + "Interval before a route is flushed from the routing + table."; + } + leaf holddown-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "180"; + description + "Interval before better routes are released."; + } + leaf update-interval { + type uint32 { + range "5..2147483647"; + } + units "seconds"; + default "30"; + description + "Interval at which RIP updates are sent."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + + .xpath = "/frr-ripd:ripd/instance/timers", + + .cbs.cli_show = cli_show_rip_timers, + + }, + + { + .xpath = "/frr-ripd:ripd/instance/timers/flush-interval", + .cbs.modify = ripd_instance_timers_flush_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/holddown-interval", + .cbs.modify = ripd_instance_timers_holddown_interval_modify, + }, + { + .xpath = "/frr-ripd:ripd/instance/timers/update-interval", + .cbs.modify = ripd_instance_timers_update_interval_modify, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_timers(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " timers basic %s %s %s\n", + yang_dnode_get_string(dnode, "./update-interval"), + yang_dnode_get_string(dnode, "./holddown-interval"), + yang_dnode_get_string(dnode, "./flush-interval")); + } + +This command is a bit different since it changes three leaves at the +same time. This means we need to have a single ``cli_show`` callback in +order to display the three leaves together in the same line. + +The new ``cli_show_rip_timers()`` callback was added attached to the +*timers* non-presence container that groups the three leaves. Without +the *timers* non-presence container we’d need to display the *timers +basic* command inside the ``cli_show_router_rip()`` callback, which +would break our requirement of having a separate ``cli_show`` callback +for each configuration command. + +.. _example-4-1: + +Example 4 +^^^^^^^^^ + +Command: +``redistribute <kernel|connected|static|ospf|isis|bgp|eigrp|nhrp|table|vnc|babel|sharp> [{metric (0-16)|route-map WORD}]`` + +YANG representation: + +.. code:: yang + + list redistribute { + key "protocol"; + description + "Redistributes routes learned from other routing protocols."; + leaf protocol { + type frr-route-types:frr-route-types-v4; + description + "Routing protocol."; + must '. != "rip"'; + } + leaf route-map { + type string { + length "1..max"; + } + description + "Applies the conditions of the specified route-map to + routes that are redistributed into the RIP routing + instance."; + } + leaf metric { + type uint8 { + range "0..16"; + } + description + "Metric used for the redistributed route. If a metric is + not specified, the metric configured with the + default-metric attribute in RIP router configuration is + used. If the default-metric attribute has not been + configured, the default metric for redistributed routes + is 0."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + { + .xpath = "/frr-ripd:ripd/instance/redistribute", + .cbs.create = ripd_instance_redistribute_create, + .cbs.delete = ripd_instance_redistribute_delete, + + .cbs.cli_show = cli_show_rip_redistribute, + }, + { + .xpath = "/frr-ripd:ripd/instance/redistribute/route-map", + .cbs.modify = ripd_instance_redistribute_route_map_modify, + .cbs.delete = ripd_instance_redistribute_route_map_delete, + }, + { + .xpath = "/frr-ripd:ripd/instance/redistribute/metric", + .cbs.modify = ripd_instance_redistribute_metric_modify, + .cbs.delete = ripd_instance_redistribute_metric_delete, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_rip_redistribute(struct vty *vty, struct lyd_node *dnode, + bool show_defaults) + { + vty_out(vty, " redistribute %s", + yang_dnode_get_string(dnode, "./protocol")); + if (yang_dnode_exists(dnode, "./metric")) + vty_out(vty, " metric %s", + yang_dnode_get_string(dnode, "./metric")); + if (yang_dnode_exists(dnode, "./route-map")) + vty_out(vty, " route-map %s", + yang_dnode_get_string(dnode, "./route-map")); + vty_out(vty, "\n"); + } + +Similar to the previous example, the *redistribute* command changes +several leaves at the same time, and we need a single callback to +display all leaves in a single line in accordance to the CLI command. In +this case, the leaves are already grouped by a YANG list so there’s no +need to add a non-presence container. The new ``cli_show`` callback was +attached to the YANG path of the list. + +It’s also worth noting the use of the ``yang_dnode_exists()`` function +to check if optional leaves exist in the configuration before displaying +them. + +.. _example-5-1: + +Example 5 +^^^^^^^^^ + +Command: +``ip rip authentication mode <md5 [auth-length <rfc|old-ripd>]|text>`` + +YANG representation: + +.. code:: yang + + container authentication-scheme { + description + "Specify the authentication scheme for the RIP interface"; + leaf mode { + type enumeration { + [snip] + } + default "none"; + description + "Specify the authentication mode."; + } + leaf md5-auth-length { + when "../mode = 'md5'"; + type enumeration { + [snip] + } + default "20"; + description + "MD5 authentication data length."; + } + } + +Placement of the ``cli_show`` callback: + +.. code:: diff + + + { + + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme", + + .cbs.cli_show = cli_show_ip_rip_authentication_scheme, + }, + { + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme/mode", + .cbs.modify = lib_interface_rip_authentication_scheme_mode_modify, + }, + { + .xpath = "/frr-interface:lib/interface/frr-ripd:rip/authentication-scheme/md5-auth-length", + .cbs.modify = lib_interface_rip_authentication_scheme_md5_auth_length_modify, + .cbs.delete = lib_interface_rip_authentication_scheme_md5_auth_length_delete, + }, + +Implementation of the ``cli_show`` callback: + +.. code:: c + + void cli_show_ip_rip_authentication_scheme(struct vty *vty, + struct lyd_node *dnode, + bool show_defaults) + { + switch (yang_dnode_get_enum(dnode, "./mode")) { + case RIP_NO_AUTH: + vty_out(vty, " no ip rip authentication mode\n"); + break; + case RIP_AUTH_SIMPLE_PASSWORD: + vty_out(vty, " ip rip authentication mode text\n"); + break; + case RIP_AUTH_MD5: + vty_out(vty, " ip rip authentication mode md5"); + if (show_defaults + || !yang_dnode_is_default(dnode, "./md5-auth-length")) { + if (yang_dnode_get_enum(dnode, "./md5-auth-length") + == RIP_AUTH_MD5_SIZE) + vty_out(vty, " auth-length rfc"); + else + vty_out(vty, " auth-length old-ripd"); + } + vty_out(vty, "\n"); + break; + } + } + +This is the most complex ``cli_show`` callback we have in ripd. Its +complexity comes from the following: \* The +``ip rip authentication mode ...`` command changes two YANG leaves at +the same time. \* Part of the command should be hidden when the +``show_defaults`` parameter is set to false. + +This is the behavior we want to implement: + +:: + + ripd(config)# interface eth0 + ripd(config-if)# ip rip authentication mode md5 + ripd(config-if)# + ripd(config-if)# show configuration candidate + Configuration: + ! + [snip] + ! + interface eth0 + ip rip authentication mode md5 + ! + end + ripd(config-if)# + ripd(config-if)# show configuration candidate with-defaults + Configuration: + ! + [snip] + ! + interface eth0 + [snip] + ip rip authentication mode md5 auth-length old-ripd + ! + end + +Note that ``auth-length old-ripd`` should be hidden unless the +configuration is shown using the *with-defaults* option. This is why the +``cli_show_ip_rip_authentication_scheme()`` callback needs to consult +the value of the *show_defaults* parameter. It’s expected that only a +very small minority of all ``cli_show`` callbacks will need to consult +the *show_defaults* parameter (there’s a chance this might be the only +case!) + +In the case of the *timers basic* command seen before, we need to +display the value of all leaves even if only one of them has a value +different from the default. Hence the ``cli_show_rip_timers()`` callback +was able to completely ignore the *show_defaults* parameter. + +Step 7: consolidation +~~~~~~~~~~~~~~~~~~~~~ + +As mentioned in the fourth step, the northbound retrofitting process can +happen gradually over time, since both “old” and “new” commands can +coexist without problems. Once all commands from a given daemon were +converted, we can proceed to the consolidation step, which consists of +the following: \* Remove the vty configuration lock, which is enabled by +default in all daemons. Now multiple users should be able to edit the +configuration concurrently, using either shared or private candidate +configurations. \* Reference commit: +`57dccdb1 <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. +\* Stop using the qobj infrastructure to keep track of configuration +objects. This is not necessary anymore, the northbound uses a similar +mechanism to keep track of YANG data nodes in the candidate +configuration. \* Reference commit: +`4e6d63ce <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. +\* Make the daemon SIGHUP handler re-read the configuration file (and +ensure it’s not doing anything other than that). \* Reference commit: +`5e57edb4 <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. + +Final Considerations +-------------------- + +Testing +~~~~~~~ + +Converting CLI commands to the new northbound model can be a complicated +task for beginners, but the more commands one converts, the easier it +gets. It’s highly recommended to perform as much testing as possible on +the converted commands to reduce the likelihood of introducing +regressions. Tools like topotests, ANVL and the `CLI +fuzzer <https://github.com/rwestphal/frr-cli-fuzzer>`__ can be used to +catch hidden bugs that might be present. As usual, it’s also recommended +to use valgrind and static code analyzers to catch other types of +problems like memory leaks. + +Amount of work +~~~~~~~~~~~~~~ + +The output below gives a rough estimate of the total number of +configuration commands that need to be converted per daemon: + +.. code:: sh + + $ for dir in lib zebra bgpd ospfd ospf6d isisd ripd ripngd eigrpd pimd pbrd ldpd nhrpd babeld ; do echo -n "$dir: " && cd $dir && grep -ERn "DEFUN|DEFPY" * | grep -Ev "clippy|show|clear" | wc -l && cd ..; done + lib: 302 + zebra: 181 + bgpd: 569 + ospfd: 198 + ospf6d: 99 + isisd: 126 + ripd: 64 + ripngd: 44 + eigrpd: 58 + pimd: 113 + pbrd: 9 + ldpd: 46 + nhrpd: 24 + babeld: 28 + +As it can be seen, the northbound retrofitting process will demand a lot +of work from FRR developers and should take months to complete. Everyone +is welcome to collaborate! |