Adding upstream version 10.1.upstream/10.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

16 files changed, 155 insertions, 101 deletions

diff --git a/doc/developer/building-libyang.rst b/doc/developer/building-libyang.rst
index a46c793..8d9876c 100644
--- a/doc/developer/building-libyang.rst
+++ b/doc/developer/building-libyang.rst
@@ -41,7 +41,7 @@ DEB packages are available as CI artifacts `here
    cd libyang
    git checkout v2.1.128
    mkdir build; cd build
-   cmake -D CMAKE_INSTALL_PREFIX:PATH=/usr \
+   cmake --install-prefix /usr \
          -D CMAKE_BUILD_TYPE:String="Release" ..
    make
    sudo make install
diff --git a/doc/developer/checkpatch.rst b/doc/developer/checkpatch.rst
index d8fe007..4ef261b 100644
--- a/doc/developer/checkpatch.rst
+++ b/doc/developer/checkpatch.rst
@@ -761,15 +761,6 @@ Indentation and Line Breaks
 Macros, Attributes and Symbols
 ------------------------------
 
-  **ARRAY_SIZE**
-    The ARRAY_SIZE(foo) macro should be preferred over
-    sizeof(foo)/sizeof(foo[0]) for finding number of elements in an
-    array.
-
-    The macro is defined in include/linux/kernel.h::
-
-      #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
-
   **AVOID_EXTERNS**
     Function prototypes don't need to be declared extern in .h
     files.  It's assumed by the compiler and is unnecessary.
diff --git a/doc/developer/cross-compiling.rst b/doc/developer/cross-compiling.rst
index af99262..c503487 100644
--- a/doc/developer/cross-compiling.rst
+++ b/doc/developer/cross-compiling.rst
@@ -148,7 +148,7 @@ be built and installed generally like:
    CC=${HOST_ARCH}-gcc \
    CXX=${HOST_ARCH}-g++ \
    cmake \
-       -DCMAKE_INSTALL_PREFIX=/usr/${HOST_ARCH} \
+       --install-prefix /usr/${HOST_ARCH} \
        ..
    make
    make install
diff --git a/doc/developer/logging.rst b/doc/developer/logging.rst
index 52653d3..82cc8b2 100644
--- a/doc/developer/logging.rst
+++ b/doc/developer/logging.rst
@@ -77,7 +77,14 @@ are available:
 
 .. note::
 
-   ``printfrr()`` does not support the ``%n`` format.
+   ``printfrr()`` does not support the ``%n`` format.  It does support ISO C23
+   ``%b``, ``%w99d`` and ``%wf99d`` additions, but the latter two are not
+   supported by the ``frr-format`` plugin yet, and all 3 aren't supported by
+   the older compilers still in use on some supported platforms.
+
+   ``%b`` can be used with ``FMT_NSTD``, but ``%w99d`` and ``%wf99d`` require
+   work in the ``frr-format`` plugin before they are really usable.
+
 
 AS-Safety
 ^^^^^^^^^
@@ -376,7 +383,8 @@ bgpd
 
 .. frrfmt:: %pBD (struct bgp_dest *)
 
-   Print prefix for a BGP destination.
+   Print prefix for a BGP destination.  When using ``--enable-dev-build`` include
+   the pointer value for the bgp_dest.
 
    :frrfmtout:`fe80::1234/64`
 
@@ -557,8 +565,9 @@ Integer formats
    cause compiler warnings when used without the plugin.  Use with
    :c:macro:`FMT_NSTD` if necessary.
 
-   It is possible ISO C23 may introduce another format for these, possibly
-   ``%w64d`` discussed in `JTC 1/SC 22/WG 14/N2680 <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2680.pdf>`_.
+   As anticipated, ISO C23 has introduced new modifiers for this, specifically
+   ``%w64d`` (= ``%Ld``) and ``%w64u`` (= ``%Lu``).  Unfortunately, these new
+   modifiers are not supported by ``frr-format`` yet.
 
 .. frrfmt:: %Lu (uint64_t)
 
diff --git a/doc/developer/mgmtd-dev.rst b/doc/developer/mgmtd-dev.rst
index 2404ffe..b979af0 100644
--- a/doc/developer/mgmtd-dev.rst
+++ b/doc/developer/mgmtd-dev.rst
@@ -317,12 +317,25 @@ Likewise the client should be cleaned up in the daemon cleanup routine.
 Back-End XPATH mappings
 ^^^^^^^^^^^^^^^^^^^^^^^
 
-In order for ``mgmtd`` to direct configuration to your daemon you need to add
+In order for ``mgmtd`` to direct YANG modeled data to your daemon you should add
 some XPATH mappings to ``mgmtd/mgmt_be_adapter.c``. These XPATHs determine which
-configuration changes get sent over the *back-end* interface to your daemon.
-There are 2 arrays to update the first for config support and the second for
-operational state.
-
+YANG modeled data (e.g., config changes) get sent over the *back-end* interface
+to your daemon. There are 4 arrays to possibly update: configuration,
+operational, notification, and RPC. You only need to add entries to the array
+that you require mapping for.
+
+Additionally the back-end client can specify these XPATH mappings when it
+first connects to mgmtd using it's initial ``SUBSCRIBE`` message.
+
+NOTE: the notif array (``be_client_notif_xpaths``), is a slightly different from
+the other 3 types (config, oper and rpc) in that it maps xpaths the back-end
+client wishes to *receive* notifications for, not the ones it may generate.
+Normally a back-end client is generating notifications; however, mgmtd supports
+back-end clients also "subscribing" to receive these notifications as well from
+other back-end clients through notif_xpath maps.
+
+Config Map Example
+""""""""""""""""""
 Below are the strings added for staticd config support:
 
 .. code-block:: c
@@ -342,6 +355,9 @@ Below are the strings added for staticd config support:
     #endif
     };
 
+
+Operational Map Example
+"""""""""""""""""""""""
 Below are the strings added for zebra operational state support (note zebra is
 not conditionalized b/c it should always be present):
 
@@ -358,6 +374,33 @@ not conditionalized b/c it should always be present):
             [MGMTD_BE_CLIENT_ID_ZEBRA] = zebra_oper_xpaths,
     };
 
+
+RPC Map Example
+"""""""""""""""
+Below is the string added for ripd RPC support:
+
+.. code-block:: c
+
+    static const char *const ripd_rpc_xpaths[] = {
+            "/frr-ripd",
+            NULL,
+    };
+
+    static const char *const *be_client_rpc_xpaths[MGMTD_BE_CLIENT_ID_MAX] = {
+    #ifdef HAVE_RIPD
+            [MGMTD_BE_CLIENT_ID_RIPD] = ripd_rpc_xpaths,
+    #endif
+    };
+
+
+Notification Map Example
+""""""""""""""""""""""""
+There are no current back-end daemons that wish to receive other back-end
+notifications so the array is empty. This may change in the future, and of
+course any back-end daemon can utilize the connect (``BeSubscribeReq``) messages
+as well.
+
+
 MGMTD Internals
 ---------------
 
diff --git a/doc/developer/northbound/advanced-topics.rst b/doc/developer/northbound/advanced-topics.rst
index edfc10b..eb75602 100644
--- a/doc/developer/northbound/advanced-topics.rst
+++ b/doc/developer/northbound/advanced-topics.rst
@@ -12,8 +12,7 @@ Auto-generated CLI commands
 In order to have less code to maintain, it should be possible to write a
 tool that auto-generates CLI commands based on the FRR YANG models. As a
 matter of fact, there are already a number of NETCONF-based CLIs that do
-exactly that (e.g. `Clixon <https://github.com/clicon/clixon>`__,
-ConfD’s CLI).
+exactly that (e.g. `Clixon <https://github.com/clicon/clixon>`__).
 
 The problem however is that there isn’t an exact one-to-one mapping
 between the existing CLI commands and the corresponding YANG nodes from
@@ -27,11 +26,6 @@ command for each YANG leaf, (leaf-)list and presence-container. The
 ripd’s ``timers basic`` command, for instance, would become three
 different commands, which would be undesirable.
 
-   This Tail-f’s®
-   `document <http://info.tail-f.com/hubfs/Whitepapers/Tail-f_ConfD-CLI__Cfg_Mode_App_Note_Rev%20C.pdf>`__
-   shows how to customize ConfD auto-generated CLI commands using YANG
-   annotations.
-
 The good news is that *libyang* allows users to create plugins to
 implement their own YANG extensions, which can be used to implement CLI
 annotations. If done properly, a CLI generator can save FRR developers
@@ -76,8 +70,8 @@ Example of how this feature could be provided in the CLI:
 ``commit confirmed [minutes <1-60>]``. The ability to do confirmed
 commits should also be exposed in the northbound API so that the
 northbound plugins can also take advantage of it (in the case of the
-Sysrepo and ConfD plugins, confirmed commits are implemented externally
-in the *netopeer2-server* and *confd* daemons, respectively).
+Sysrepo plugin, confirmed commit is implemented externally in the
+*netopeer2-server* daemon).
 
 Proposed feature: enable/disable configuration commands/sections
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/developer/northbound/architecture.rst b/doc/developer/northbound/architecture.rst
index 5fd89c3..4e84f1d 100644
--- a/doc/developer/northbound/architecture.rst
+++ b/doc/developer/northbound/architecture.rst
@@ -116,7 +116,7 @@ multitude of different management APIs, all of them connected to the
 northbound layer of the FRR daemons. By default, only the CLI interface
 is compiled built-in in the FRR daemons. The other management interfaces
 are provided as optional plugins and need to be loaded during the daemon
-initialization (e.g. *zebra -M confd*). This design makes it possible to
+initialization (e.g. *zebra -M grpc*). This design makes it possible to
 integrate FRR with different NETCONF solutions without introducing
 vendor lock-in. The [[Plugins - Writing Your Own]] page explains how to
 write custom northbound plugins that can be tailored to all needs
diff --git a/doc/developer/northbound/demos.rst b/doc/developer/northbound/demos.rst
index 8a0f6ad..7c5ae0c 100644
--- a/doc/developer/northbound/demos.rst
+++ b/doc/developer/northbound/demos.rst
@@ -8,23 +8,3 @@ This short demo shows some of the capabilities of the new transactional
 CLI:
 
 |asciicast1|
-
-ConfD + NETCONF + Cisco YDK
----------------------------
-
-This is a very simple demo of *ripd* being configured by a python
-script. The script uses NETCONF to communicate with *ripd*, which has
-the ConfD plugin loaded. The most interesting part, however, is the fact
-that the python script is not using handcrafted XML payloads to
-configure *ripd*. Instead, the script is using python bindings generated
-using Cisco’s YANG Development Kit (YDK).
-
--  Script used in the demo:
-   https://gist.github.com/rwestphal/defa9bd1ccf216ab082d4711ae402f95
-
-|asciicast2|
-
-.. |asciicast1| image:: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1.png
-   :target: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1
-.. |asciicast2| image:: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv.png
-   :target: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv
diff --git a/doc/developer/northbound/links.rst b/doc/developer/northbound/links.rst
index 6cec176..e8fb327 100644
--- a/doc/developer/northbound/links.rst
+++ b/doc/developer/northbound/links.rst
@@ -195,14 +195,6 @@ pyangbind
 -  GitHub page: https://github.com/robshakir/pyangbind
 -  Documentation: http://pynms.io/pyangbind/
 
-ConfD
-^^^^^
-
--  Official webpage (for ConfD Basic):
-   http://www.tail-f.com/confd-basic/
--  Training Videos: http://www.tail-f.com/confd-training-videos/
--  Forum: http://discuss.tail-f.com/
-
 Sysrepo
 ^^^^^^^
 
diff --git a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst
index 5cb70ca..07f92c2 100644
--- a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst
+++ b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst
@@ -10,10 +10,10 @@ Operational data
 ~~~~~~~~~~~~~~~~
 
 Writing API-agnostic code for YANG-modeled operational data is
-challenging. ConfD and Sysrepo, for instance, have completely different
-APIs to fetch operational data. So how can we write API-agnostic
-callbacks that can be used by both the ConfD and Sysrepo plugins, and
-any other northbound client that might be written in the future?
+challenging. Sysrepo, for instance, has completely different API to
+fetch operational data. So how can we write API-agnostic callbacks
+that can be used by both the Sysrepo plugin, and any other northbound
+client that might be written in the future?
 
 As an additional requirement, the callbacks must be designed in a way
 that makes in-place XPath filtering possible. As an example, a
@@ -94,27 +94,18 @@ in the northbound architecture:
             */
            void *(*lookup_entry)(struct yang_list_keys *keys);
 
-These callbacks were designed to provide maximum flexibility, and borrow
-a lot of ideas from the ConfD API. Each callback does one and only one
-task, they are indivisible primitives that can be combined in several
-different ways to iterate over operational data. The extra flexibility
-certainly has a performance cost, but it’s the price to pay if we want
-to expose FRR operational data using several different management
-interfaces (e.g. NETCONF via either ConfD or Sysrepo+Netopeer2). In the
+These callbacks were designed to provide maximum flexibility. Each
+callback does one and only one task, they are indivisible primitives
+that can be combined in several different ways to iterate over operational
+data. The extra flexibility certainly has a performance cost, but it’s the
+price to pay if we want to expose FRR operational data using several
+different management interfaces (e.g. Sysrepo+Netopeer2). In the
 future it might be possible to introduce optional callbacks that do
 things like returning multiple objects at once. They would provide
 enhanced performance when iterating over large lists, but their use
 would be limited by the northbound plugins that can be integrated with
 them.
 
-   NOTE: using the northbound callbacks as a base, the ConfD plugin can
-   provide up to 100 objects between each round trip between FRR and the
-   *confd* daemon. Preliminary tests showed FRR taking ~7 seconds
-   (asynchronously, without blocking the main pthread) to return a RIP
-   table containing 100k routes to a NETCONF client connected to *confd*
-   (JSON was used as the encoding format). Work needs to be done to find
-   the bottlenecks and optimize this operation.
-
 The [[Plugins - Writing Your Own]] page explains how the northbound
 plugins can fetch operational data using the aforementioned northbound
 callbacks, and how in-place XPath filtering can be implemented.
@@ -351,10 +342,10 @@ are being iterated over. If that is not done, the list entry returned by
 this callback can become a dangling pointer when used in another
 callback.
 
-Currently the ConfD and Sysrepo plugins run only in the main pthread.
-The plan in the short-term is to introduce a separate pthread only for
-handling operational data, and use the main pthread only for handling
-configuration changes, RPCs and notifications.
+Currently the Sysrepo plugin runs only in the main pthread. The plan in the
+short-term is to introduce a separate pthread only for handling operational
+data, and use the main pthread only for handling configuration changes,
+RPCs and notifications.
 
 RPCs and Actions
 ~~~~~~~~~~~~~~~~
@@ -396,8 +387,8 @@ some EXEC-level commands using YANG so that their functionality is
 exposed to other management interfaces other than the CLI. As an
 example, if the ``clear bgp`` command is modeled using a YANG RPC, and a
 corresponding ``rpc`` callback is written, then it should be possible to
-clear BGP neighbors using NETCONF and RESTCONF with that RPC (the ConfD
-and Sysrepo plugins have full support for YANG RPCs and actions).
+clear BGP neighbors using NETCONF and RESTCONF with that RPC (the Sysrepo
+plugin has full support for YANG RPCs and actions).
 
 Here’s an example of a very simple RPC modeled using YANG:
 
@@ -568,8 +559,7 @@ Now sending the *authentication-failure* YANG notification should be as
 simple as calling the above function and provide the appropriate
 interface name. The notification will be processed by all northbound
 plugins that subscribed a callback to the ``nb_notification_send`` hook.
-The ConfD and Sysrepo plugins, for instance, use this hook to relay the
-notifications to the *confd*/*sysrepod* daemons, which can generate
-NETCONF notifications to subscribed clients. When no northbound plugin
-is loaded, ``nb_notification_send()`` doesn’t do anything and the
-notifications are ignored.
+The Sysrepo plugin, for instance, uses this hook to relay the notifications
+to the *sysrepod* daemon, which can generate NETCONF notifications to subscribed
+clients. When no northbound plugin is loaded, ``nb_notification_send()`` doesn’t
+do anything and the notifications are ignored.
diff --git a/doc/developer/northbound/plugins-sysrepo.rst b/doc/developer/northbound/plugins-sysrepo.rst
index 0cfdb82..f4df68c 100644
--- a/doc/developer/northbound/plugins-sysrepo.rst
+++ b/doc/developer/northbound/plugins-sysrepo.rst
@@ -32,7 +32,7 @@ libyang
    cd libyang
    git checkout v2.1.148
    mkdir build; cd build
-   cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \
+   cmake --install-prefix /usr \
          -DCMAKE_BUILD_TYPE:String="Release" ..
    make
    sudo make install
@@ -51,7 +51,7 @@ Sysrepo
    cd sysrepo/
    git checkout v2.2.150
    mkdir build; cd build
-   cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr \
+   cmake --install-prefix /usr \
          -DCMAKE_BUILD_TYPE:String="Release" ..
    make
    sudo make install
diff --git a/doc/developer/static-linking.rst b/doc/developer/static-linking.rst
index 5342fbf..e9bb928 100644
--- a/doc/developer/static-linking.rst
+++ b/doc/developer/static-linking.rst
@@ -44,7 +44,7 @@ when building libyang statically.
 The resultant cmake command is::
 
    cmake -DENABLE_STATIC=ON -DENABLE_LYD_PRIV=ON \
-         -DCMAKE_INSTALL_PREFIX:PATH=/usr \
+         --install-prefix /usr \
          -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE \
          -DCMAKE_BUILD_TYPE:String="Release" ..
 
diff --git a/doc/developer/topotests-markers.rst b/doc/developer/topotests-markers.rst
index 9f92412..670bf0d 100644
--- a/doc/developer/topotests-markers.rst
+++ b/doc/developer/topotests-markers.rst
@@ -12,6 +12,7 @@ systems, all tests must be marked with at least one of the following markers:
 * eigrpd
 * isisd
 * ldpd
+* mgmtd
 * nhrpd
 * ospf6d
 * ospfd
@@ -64,12 +65,12 @@ Adding a single marker:
 
     import pytest
     ...
-    
+
     # add after imports, before defining classes or functions:
     pytestmark = pytest.mark.bfdd
-    
+
     ...
-    
+
     def test_using_bfdd():
 
 
@@ -79,16 +80,16 @@ Adding multiple markers:
 
     import pytest
     ...
-    
+
     # add after imports, before defining classes or functions:
     pytestmark = [
         pytest.mark.bgpd,
         pytest.mark.ospfd,
         pytest.mark.ospf6d
     ]
-    
+
     ...
-    
+
     def test_using_bgpd_ospfd_ospf6d():
 
 
diff --git a/doc/developer/topotests.rst b/doc/developer/topotests.rst
index 9b9058b..e1702c4 100644
--- a/doc/developer/topotests.rst
+++ b/doc/developer/topotests.rst
@@ -46,6 +46,7 @@ The version of protobuf package that is installed on your system will determine
 which versions of the python protobuf packages you need to install.
 
 .. code:: shell
+
    # - Either - For protobuf version <= 3.12
    python3 -m pip install 'protobuf<4'
 
@@ -712,6 +713,44 @@ Here's an example of collecting ``rr`` execution state from ``mgmtd`` on router
 To specify additional arguments for ``rr record``, one can use the
 ``--rr-options``.
 
+.. _code_coverage:
+
+Code coverage
+"""""""""""""
+Code coverage reporting requires installation of the ``gcov`` and ``lcov``
+packages.
+
+Code coverage can automatically be gathered for any topotest run. To support
+this FRR must first be compiled with the ``--enable-gcov`` configure option.
+This will cause *.gnco files to be created during the build. When topotests are
+run the statistics are generated and stored in *.gcda files. Topotest
+infrastructure will gather these files, capture the information into a
+``coverage.info`` ``lcov`` file and also report the coverage summary.
+
+To enable code coverage support pass the ``--cov-topotest`` argument to pytest.
+If you build your FRR in a directory outside of the FRR source directory you
+will also need to pass the ``--cov-frr-build-dir`` argument specifying the build
+directory location.
+
+During the topotest run the *.gcda files are generated into a ``gcda``
+sub-directory of the top-level run directory (i.e., normally
+``/tmp/topotests/gcda``). These files will then be copied at the end of the
+topotest run into the FRR build directory where the ``gcov`` and ``lcov``
+utilities expect to find them. This is done to deal with the various different
+file ownership and permissions.
+
+At the end of the run ``lcov`` will be run to capture all of the coverage data
+into a ``coverage.info`` file. This file will be located in the top-level run
+directory (i.e., normally ``/tmp/topotests/coverage.info``).
+
+The ``coverage.info`` file can then be used to generate coverage reports or file
+markup (e.g., using the ``genhtml`` utility) or enable markup within your
+IDE/editor if supported (e.g., the emacs ``cov-mode`` package)
+
+NOTE: the *.gcda files in ``/tmp/topotests/gcda`` are cumulative so if you do
+not remove them they will aggregate data across multiple topotest runs.
+
+
 .. _topotests_docker:
 
 Running Tests with Docker
diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst
index 68834ed..f720f62 100644
--- a/doc/developer/workflow.rst
+++ b/doc/developer/workflow.rst
@@ -167,13 +167,13 @@ as early as possible, i.e. the first 2-week window.
 For reference, the expected release schedule according to the above is:
 
 +---------+------------+------------+------------+
-| Release | 2023-11-07 | 2024-03-05 | 2024-07-02 |
+| Release | 2024-03-12 | 2024-07-02 | 2024-11-05 |
 +---------+------------+------------+------------+
-| RC      | 2023-10-24 | 2024-02-20 | 2024-06-18 |
+| RC      | 2024-02-27 | 2024-06-18 | 2024-10-22 |
 +---------+------------+------------+------------+
-| dev/X.Y | 2023-10-10 | 2024-02-06 | 2024-06-04 |
+| dev/X.Y | 2024-02-13 | 2024-06-04 | 2024-10-08 |
 +---------+------------+------------+------------+
-| freeze  | 2023-09-26 | 2024-01-23 | 2024-05-21 |
+| freeze  | 2024-01-30 | 2024-05-21 | 2024-09-24 |
 +---------+------------+------------+------------+
 
 Here is the hint on how to get the dates easily:
diff --git a/doc/developer/zebra.rst b/doc/developer/zebra.rst
index be2952e..482df96 100644
--- a/doc/developer/zebra.rst
+++ b/doc/developer/zebra.rst
@@ -161,6 +161,21 @@ Zebra Protocol Commands
 
 The definitions of zebra protocol commands can be found at ``lib/zclient.h``.
 
+
+Zebra Dataplane
+===============
+
+The zebra dataplane subsystem provides a framework for FIB
+programming. Zebra uses the dataplane to program the local kernel as
+it makes changes to objects such as IP routes, MPLS LSPs, and
+interface IP addresses. The dataplane runs in its own pthread, in
+order to off-load work from the main zebra pthread.
+
+The zebra dataplane API is versioned; the version number must be
+updated along with API changes. Plugins can test the current version
+number and confirm that they are compatible with the current version.
+
+
 Dataplane batching
 ==================