Merging upstream version 10.0.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 253 insertions, 92 deletions

diff --git a/doc/developer/mgmtd-dev.rst b/doc/developer/mgmtd-dev.rst
index 9839aa8..2404ffe 100644
--- a/doc/developer/mgmtd-dev.rst
+++ b/doc/developer/mgmtd-dev.rst
@@ -12,7 +12,7 @@ MGMTD Development
 =================
 
 Overview
-^^^^^^^^
+--------
 
 ``mgmtd`` (Management Daemon) is a new centralized management daemon for FRR.
 
@@ -33,18 +33,64 @@ each daemon. ``mgmtd`` currently provides the CLI interface for each daemon that
 has been converted to it, but in the future RESTCONF and NETCONF servers can
 easily be added as *front-ends* to mgmtd to support those protocols as well.
 
+Conversion Status
+^^^^^^^^^^^^^^^^^
+
+Fully Converted To MGMTD
+""""""""""""""""""""""""
+
+- lib/distribute
+- lib/filter
+- lib/if_rmap
+- lib/routemap
+- lib/affinitymap
+- lib/if
+- lib/vrf
+- ripd
+- ripngd
+- staticd
+- zebra (* - partial)
+
+Converted To Northbound
+"""""""""""""""""""""""
+- bfdd
+- pathd
+- pbrd
+- pimd
+
+Converted To Northbound With Issues
+"""""""""""""""""""""""""""""""""""
+- eigrp
+- isisd
+
+Unconverted
+"""""""""""
+- babel
+- bgpd
+- ldpd
+- lib/event
+- lib/keychain
+- lib/log_vty
+- lib/nexthop_group
+- lib/zlog_5424_cli
+- nhrpd
+- ospfd
+- ospf6d
+- pceplib
+- qdb
+- sharpd
+- vrrpd
 
 Converting A Daemon to MGMTD
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------------------------
 
-A daemon must first be transitioned to the new *northbound* interface if that
-has not already been done (see `this northbound conversion documentation
-<https://github.com/opensourcerouting/frr/wiki/Retrofitting-Configuration-Commands>`_
-for how to do this). Once this is done a few simple steps are all that is
-required move the daemon over to ``mgmtd`` control.
+A daemon must first be transitioned to the new :ref:`northbound` interface if that
+has not already been done (see :ref:`nb-retrofit` for how to do this). Once this
+is done a few simple steps are all that is required move the daemon over to
+``mgmtd`` control.
 
 Overview of Changes
--------------------
+^^^^^^^^^^^^^^^^^^^
 
 Adding support for a *northbound* converted daemon involves very little work. It
 requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support.
@@ -52,64 +98,126 @@ requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support.
 
 Front-End Interface:
 
-1. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.c``)
-2. Add YANG module description into array defined in ``mgmtd/mgmt_main.c``
-3. Add CLI handler file[s] to ``mgmtd/subdir.am`` (e.g., ``staticd/static_vty.c``)
-4. [if needed] Exclude (#ifndef) non-configuration CLI handlers from CLI source
-   file (e.g., inside ``staticd/static_vty.c``)
+#. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.yang.c``).
+
+#. Add CLI handler file[s] to ``mgmtd/subdir.am``. The `subdir.am` variable to
+   use is indicated in the next 2 steps.
+
+   #. [if needed] Exclude (:code:`#ifndef`) non-configuration CLI handlers from
+      CLI source file (e.g., inside :file:`staticd/static_vty.c`) and add the
+      file to :code:`nodist_mgmtd_libmgmt_be_nb_la_SOURCES` in
+      :file:`mgmtd/subdir.am`.
+
+   #. [otherwise] Remove CLI handler file from _SOURCES variable in the daemon
+      :file:`subdir.am` file (e.g in :file:`staticd/subdir.am`) and add to
+      :code:`mgmtd_libmgmtd_a_SOURCES` in :file:`mgmtd/subdir.am`.
+
+#. In order to have mgmtd try and load existing per-daemon config files, add
+   the daemon to the :code:`mgmt_daemons` array in :file:`lib/vty.c`. With the
+   official release of the mgmtd code FRR is no longer supporting per daemon log
+   files but it will take a while before all of the topotest is converted.
+
+#. In the daemon's :code:`struct frr_daemon_info` (i.e., inside it's
+   :code:`FRR_DAEMON_INFO()`) set the `.flags` bit `FRR_NO_SPLIT_CONFIG`. This
+   will keep the daemon from trying to read it's per-daemon config file as mgmtd
+   will now be doing this.
+
+#. Add the daemon's YANG module description[s] into the array
+   :code:`mgmt_yang_modules` defined in :file:`mgmtd/mgmt_main.c` (see
+   :ref:`mgmtd-config-write`). Make sure that all YANG modules that the daemon
+   uses are present in the mgmtd list. To find this list look in the daemon's
+   equivalent yang module array variable.
+
+#. Initialize the CLI handlers inside :code:`mgmt_vty_init` in :file:`mgmtd/mgmt_vty.c`.
+
+#. Direct ``vtysh`` to send CLI commands to ``mgmtd`` by modifying
+   ``vtysh/vtysh.h``. At the top of this file each daemon has a bit
+   ``#define``'d (e.g., ``#define VTYSH_STATICD 0x08000``) below this there are
+   groupings, replace all the uses of the daemons bit with ``VTYSH_MGMTD``
+   instead so that the CLI commands get properly routed to ``mgmtd`` rather than
+   the daemon now.
+
+ #. Remove initialization (and installation) of library CLI routines. These will
+    correspond with the VTYSH removals from the last step i.e.,:
+
+    - change access_list_init() to access_list_init_new(false) and remove from
+      VTYSH_ACL_CONFIG (leave in VTYSH_ACL_SHOW).
+    - remove if_cmd_init_default() => remove from VTYSH_INTERFACE_SUBSET
+    - remove if_cmd_init() => remove from VTYSH_INTERFACE_SUBSET
+    - change route_map_init() to route_map_init_new(false) and remove from
+      VTYSH_ROUTE_MAP_CONFIG (leave in VTYSH_ROUTE_MAP_SHOW).
+    - remove vrf_cmd_init(NULL)  => remove from VTYSH_INTERFACE_SUBSET
+    ...
 
 Back-End Interface:
 
-5. Add XPATHs mappings to a couple arrays to direct ``mgmtd`` at your daemon in
-   ``mgmtd/mgmt_be_adapter.c``
-
+#. In the daemon's main file initialize the BE client library. You add a global
+   `struct mgmt_be_client *mgmt_be_client` near the daemons `event_loop *master`
+   variable. Then where the daemon used to initialize it's CLI/VTY code replace
+   that with the client initialization by calling `mgmt_be_client_create`.
+   Likewise in the daemon's sigint cleanup code, operational walks should be
+   canceled with a call to `nb_oper_cancel_all_walks`, and then the BE client
+   should be destroyed with a call to `mgmt_be_client_destroy` and to be safe
+   NULL out the global `mgmt_be_client` variable.
+
+#. In ``mgmtd/mgmt_be_adapter.c`` add xpath prefix mappings to a one or both
+   mapping arrays (``be_client_config_xpaths`` and ``be_client_oper_xpaths``) to
+   direct ``mgmtd`` to send config and oper-state requests to your daemon. NOTE:
+   make sure to include library supported xpaths prefixes as well (e.g.,
+   "/frr-interface:lib"). A good way to figure these paths out are to look in
+   each of the YANG modules that the daemon uses and include each of their paths
+   in the array.
 
 Add YANG and CLI into MGMTD
----------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 As an example here is the addition made to ``mgmtd/subdir.am`` for adding
 ``staticd`` support.
 
 .. code-block:: make
 
-   if STATICD
-   nodist_mgmtd_mgmtd_SOURCES += \
+    if STATICD
+    nodist_mgmtd_mgmtd_SOURCES += \
         yang/frr-staticd.yang.c \
         yang/frr-bfdd.yang.c \
         # end
-   nodist_mgmtd_libmgmt_be_nb_la_SOURCES += staticd/static_vty.c
-   endif
+    nodist_mgmtd_libmgmt_be_nb_la_SOURCES += staticd/static_vty.c
+    endif
 
 An here is the addition to the modules array in ``mgmtd/mgmt_main.c``:
 
 .. code-block:: c
 
-   static const struct frr_yang_module_info *const mgmt_yang_modules[] = {
+    #ifdef HAVE_STATICD
+    extern const struct frr_yang_module_info frr_staticd_info;
+    #endif
+
+    static const struct frr_yang_module_info *const mgmt_yang_modules[] = {
            &frr_filter_info,
            ...
-   #ifdef HAVE_STATICD
-           &(struct frr_yang_module_info){.name = "frr-staticd",
-				         .ignore_cbs = true},
-   #endif
-   }
+    #ifdef HAVE_STATICD
+           &frr_staticd_info,
+    #endif
+    }
 
 
-CLI Handlers
-------------
+CLI Config and Show Handlers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The daemon's CLI handlers for configuration (which having been converted to
-*northbound* now simply generate YANG changes) will be linked directly into
+The daemon's CLI handlers for configuration (which having been converted to the
+:ref:`northbound` now simply generate YANG changes) will be linked directly into
 ``mgmtd``.
 
 If the operational and debug CLI commands are kept in files separate from the
 daemon's configuration CLI commands then no extra work is required. Otherwise some
 CPP #ifndef's will be required.
 
-Currently ``mgmtd`` supports configuration CLI but not operational
-state so if both types of CLI handlers are present in a single file (e.g. a
-``xxx_vty.c`` or ``xxx_cli.c`` file ) then #ifndef will be used to exclude these
-non-configuration CLI handlers from ``mgmtd``. The same goes for *debug* CLI
-handlers. For example:
+``mgmtd`` supports both config and operational state. However, many
+daemons have not had their operational state CLI commands converted over to the
+new YANG based methods. If that is the case and if both types of CLI handlers
+are present in a single file (e.g. a ``xxx_vty.c`` or ``xxx_cli.c`` file) then
+:code:`#ifndef` will need to be used to exclude the non-config CLI handlers from
+``mgmtd``. The same goes for unconverted *debug* CLI handlers. For example:
 
 .. code-block:: c
 
@@ -121,7 +229,7 @@ handlers. For example:
   }
 
   #ifndef INCLUDE_MGMTD_CMDDEFS_ONLY
-  DEFPY(daemon_show_oepr, daemon_show_oepr_cmd,
+  DEFPY(daemon_show_oper, daemon_show_oper_cmd,
         "show daemon oper [all]"
         ...
   {
@@ -140,83 +248,136 @@ handlers. For example:
 
   }
 
+.. _mgmtd-config-write:
+
+CLI Config Write Handlers (:code:`cli_show`)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To support writing out the CLI configuration file the northbound API defines a
+2 callbacks (:code:`cli_show` and :code:`cli_show_end`). Pointers to these
+callbacks used to live side-by-side in a daemons :code:`struct frr_yang_module_info`,
+with the daemons back-end configuration and operational state callbacks
+(normally in a file named `<daemon>_nb.c`).
+
+However, these 2 functionalities need to be split up now. The *frontend* config
+writing callbacks (:code:`cli_show`) should now be linked into ``mgmtd`` while
+the *backend* config and oper-state callbacks (e.g., :code:`create`,
+:code:`modify`, etc) should continue to be linked into the daemon.
+
+So you will need to define 2 :code:`struct frr_yang_module_info` arrays.
+
+#. The existing array remains in the same place in the daemon, but with all the
+   :code:`cli_show` handlers removed.
+
+#. The removed :code:`cli_show` handlers should be added to a new
+   :code:`struct frr_yang_module_info` array. This second array should be
+   included in the same file that includes that actual function pointed to by
+   the the :code:`cli_show` callbacks (i.e., the file is compiled into
+   ``mgmtd``).
+
+   This new :code:`struct frr_yang_module_info` array is the one to be included
+   in mgmtd in `mgmt_yang_modules` inside ``mgmtd/mgmt_main.c``.
+
+Back-End Client Connection
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order for your daemon to communicate with mgmtd you need to initialize the
+backend client library. You normally do this where you used to initialize your
+CLI/VTY code.
+
+.. code-block:: c
+
+    ...
+    struct event_loop *master;
 
-Add Back-End XPATH mappings
----------------------------
+    static struct mgmt_be_client *mgmt_be_client;
+    ...
+
+    int main(int argc, char **argv)
+    {
+        ...
+        rip_init();
+	rip_if_init();
+	mgmt_be_client = mgmt_be_client_create("ripd", NULL, 0, master);
+
+Likewise the client should be cleaned up in the daemon cleanup routine.
+
+.. code-block:: c
+
+    /* SIGINT handler. */
+    static void sigint(void)
+    {
+            zlog_notice("Terminating on signal");
+            ...
+            nb_oper_cancel_all_walks();
+            mgmt_be_client_destroy(mgmt_be_client);
+            mgmt_be_client = NULL;
+
+
+Back-End XPATH mappings
+^^^^^^^^^^^^^^^^^^^^^^^
 
 In order for ``mgmtd`` to direct configuration to your daemon you need to add
 some XPATH mappings to ``mgmtd/mgmt_be_adapter.c``. These XPATHs determine which
 configuration changes get sent over the *back-end* interface to your daemon.
+There are 2 arrays to update the first for config support and the second for
+operational state.
 
-Below are the strings added for staticd support:
+Below are the strings added for staticd config support:
 
 .. code-block:: c
 
-   static const struct mgmt_be_xpath_map_init mgmt_xpath_map_init[] = {
-       {
-           .xpath_regexp = "/frr-vrf:lib/*",
-           .subscr_info =
-               {
-   #if HAVE_STATICD
-                   [MGMTD_BE_CLIENT_ID_STATICD] =
-                       MGMT_SUBSCR_VALIDATE_CFG |
-                       MGMT_SUBSCR_NOTIFY_CFG,
-   #endif
-               },
-       },
-       ...
-       {
-           .xpath_regexp =
-               "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*",
-           .subscr_info =
-               {
-   #if HAVE_STATICD
-                   [MGMTD_BE_CLIENT_ID_STATICD] =
-                       MGMT_SUBSCR_VALIDATE_CFG |
-                       MGMT_SUBSCR_NOTIFY_CFG,
-   #endif
-               },
-       },
-   };
-
-   #if HAVE_STATICD
-   static struct mgmt_be_client_xpath staticd_xpaths[] = {
-       {
-           .xpath = "/frr-vrf:lib/*",
-           .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG,
-       },
-       ...
-       {
-           .xpath =
-               "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd/*",
-           .subscribed = MGMT_SUBSCR_VALIDATE_CFG | MGMT_SUBSCR_NOTIFY_CFG,
-       },
-   };
-   #endif
-
-   static struct mgmt_be_client_xpath_map
-       mgmt_client_xpaths[MGMTD_BE_CLIENT_ID_MAX] = {
-   #ifdef HAVE_STATICD
-           [MGMTD_BE_CLIENT_ID_STATICD] = {staticd_xpaths,
-                           array_size(staticd_xpaths)},
-   #endif
-   };
+    #if HAVE_STATICD
+    static const char *const staticd_xpaths[] = {
+            "/frr-vrf:lib",
+            "/frr-interface:lib",
+            "/frr-routing:routing/control-plane-protocols/control-plane-protocol/frr-staticd:staticd",
+            NULL,
+    };
+    #endif
+
+    static const char *const *be_client_xpaths[MGMTD_BE_CLIENT_ID_MAX] = {
+    #ifdef HAVE_STATICD
+            [MGMTD_BE_CLIENT_ID_STATICD] = staticd_xpaths,
+    #endif
+    };
 
+Below are the strings added for zebra operational state support (note zebra is
+not conditionalized b/c it should always be present):
+
+.. code-block:: c
+
+    static const char *const zebra_oper_xpaths[] = {
+            "/frr-interface:lib/interface",
+            "/frr-vrf:lib/vrf/frr-zebra:zebra",
+            "/frr-zebra:zebra",
+            NULL,
+    };
+
+    static const char *const *be_client_oper_xpaths[MGMTD_BE_CLIENT_ID_MAX] = {
+            [MGMTD_BE_CLIENT_ID_ZEBRA] = zebra_oper_xpaths,
+    };
 
 MGMTD Internals
-^^^^^^^^^^^^^^^
+---------------
 
 This section will describe the internal functioning of ``mgmtd``, for now a
 couple diagrams are included to aide in source code perusal.
 
 
-The client side of a CLI change
+The client side of a CLI configuration change
 
 .. figure:: ../figures/cli-change-client.svg
    :align: center
 
 
-The server (mgmtd) side of a CLI change
+The server (mgmtd) side of a CLI configuration change
 
 .. figure:: ../figures/cli-change-mgmtd.svg
    :align: center
+
+
+The client and server sides of oper-state query
+
+.. figure:: ../figures/cli-oper-state.svg
+   :align: center