diff options
Diffstat (limited to 'src/lib/yang/yang.dox')
| -rw-r--r-- | src/lib/yang/yang.dox | 276 |
1 files changed, 276 insertions, 0 deletions
diff --git a/src/lib/yang/yang.dox b/src/lib/yang/yang.dox new file mode 100644 index 0000000..e0fcab0 --- /dev/null +++ b/src/lib/yang/yang.dox @@ -0,0 +1,276 @@ +// Copyright (C) 2018-2022 Internet Systems Consortium, Inc. ("ISC") +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at http://mozilla.org/MPL/2.0/. + +/** + @page libyang libkea-yang - Kea YANG Utilities Library + +The libkea-yang library was developed to handle base YANG operations, +such as retrieving YANG schema and configuration and translating +data between YANG and JSON that is understandable by Kea. + +@section yangTranslator Translator between YANG and JSON + +An essential concept is the idea of translator. It is a primitive that is able +to convert certain data structure between YANG and JSON. It is envisaged +that more complex translators will use other translators to handle more +complex data structures. For details, see @ref isc::yang::Translator. +It is also envisioned that translators could do the translation automatically by +relying on capabilities of iterating through data, retrieving data type +information and value information from nodes through the libyang and sysrepo +APIs. + +Note that although the initial focus is on translation from YANG to JSON (so +Kea can retrieve its configuration), the opposite translation direction - +from JSON to YANG - is also very useful, for at least three reasons. First, +in many cases we can use it in tests to check that conversion back and forth +doesn't lose anything: yang = toYang(toJson(yang)). Secondly, YANG modules +cover two major types of data: configuration and run-time state. While +we're initially focusing on getting the configuration, the run-time state +is something that Kea is expected to provide. Kea uses JSON internally in many +places and that data will have to be exported in YANG format. Thirdly, the +Event::Update (SR_EV_UPDATE prior to sysrepo v2) callback allows mid-flight +configuration changes before data is +committed to the sysrepo datastore. If it ever will be used in the future, +changes applied during this step will most likely come from Kea's current JSON +configuration. As such, JSON to YANG translation will be necessary. One +application for this is reverting stuff that is necessary for Kea - Sysrepo +communication like the unix socket. + +All translators take a Session pointer (a structure provided by Sysrepo that +is responsible for maintaining a connection) in constructors and derive from +the basic / base class and recursively from translators for embedded parts. + +@c isc::yang::Translator provides several public methods: + + - @c isc::yang::Translator::checkAndGet() is able to retrieve a YANG node that + can be retrieved through complex logic which is abstracted through a lambda. + + - @c isc::yang::Translator::checkAndGetLeaf() is a convenience wrapper + over @c isc::yang::Translator::getItem(). + + - @c isc::yang::Translator::checkAndGetAndJsonifyLeaf() retrieves elements + that are strings in the YANG schema, but that require passing through + @c isc::data::Element::fromJSON() when translating to ElementPtr. + + - @c isc::yang::Translator::checkAndStringifyAndSetLeaf() does the opposite of + @c isc::yang::Translator::checkAndGetAndJsonifyLeaf(). It takes an arbitrary + Element and sets it as a YANG string in sysrepo. + + - @c isc::yang::Translator::checkAndSetLeaf() is a convenience wrapper + over @c isc::yang::Translator::setItem(). + + - @c isc::yang::Translator::checkAndSetLeafList() is able to push multiple + leaf list nodes to the same xpath to form a leaf list. + + - @c isc::yang::Translator::checkAndSetUserContext() is specifically tailored + for setting user context in sysrepo. It's use is frequent enough to have + earned its own function. + + - @c isc::yang::Translator::deleteItem() deletes the data node found at + given xpath. + + - @c isc::yang::Translator::findXPath() retrieves any data node found at any + xpath. It is computationally intensive. Use sparingly. + + - @c isc::yang::Translator::forAll() iterates over the node found at + given xpath and all its descendants and calls the given function. + + - @c isc::yang::Translator::getData() retrieves any data node found at any + xpath. The difference from @c isc::yang::Translator::findXPath() is that it + does not throw if the data node is not found and instead returns nullopt. + + - @c isc::yang::Translator::getItem() retrieves and translates a leaf + from YANG to Element. + + - @c isc::yang::Translator::getList() retrieves a list from sysrepo and + translates it from YANG to Element. + + - @c isc::yang::Translator::getMandatoryLeaf() fetches a leaf that is expected + to be present in the YANG data node, most of the time a YANG key. + + - @c isc::yang::Translator::schemaNodeExists() checks if an xpath is valid + from the YANG schema point of view. Not used anywhere, but it's here to + substitute logic that had been previously removed. + + - @c isc::yang::Translator::setItem() translates a leaf from Element to + YANG and sets it in sysrepo. + + - @c isc::yang::Translator::setMandatoryLeaf() sets a leaf that is expected + to be present in the Element node, most of the time a YANG key. + + - @c isc::yang::Translator::translateFromYang(optional<DataNode>, string) + translates a YANG leaf to an Element node based on YANG type. + All YANG types are explicitly handled. + + - @c isc::yang::Translator::translateToYang(ConstElementPtr, LeafBaseType) + translates an Element leaf to a string based on the YANG type. + All YANG types are explicitly handled. + +Some of these methods have a counterpart that have "Diverging" in their name. +They are exceptionally used in the case where YANG xpath and Element map key are +different. This facilitates identifying these diverging nodes. + +@section yangTranslatorPool Pool translator + +@c isc::yang::TranslatorPool is the standard example of a translator +for a structured value. Its constructor takes a module name: the code +implements some variants to accommodate the module with shared code +moved into a common private routine. When called with an unsupported +module, generic methods of all structure translators throw +@c isc::NotImplemented. + +Note pools show two shortcomings in IETF modules: + - option sets make to track changes nearly impossible: the only easy + code is to translate the whole configuration. + - prefix and start - end forms of pool ranges are both mandatory. + (reported to authors' so should be fixed in the next version). + +All structure translators depend on @c isc::yang::Translator and +some of them depend on other structures, for instance +@c isc::yang::TranslatorPool depends on +@c isc::yang::TranslatorOptionDataList which itself, as all list translators, +depends on the corresponding list item translator +@c isc::yang::TranslatorOptionData. This multiple inheritance forms +a graph with the basic and the configuration translators at the two ends. +Multiple inheritance and its diamond issue are handled by C++ with +the virtual inheritance: depending classes must be virtually inherited +and explicitly constructed. + +@section yangTranslatorSubnet Subnet translator + +The new thing here is the use of adaptors to move timers from subnets +to pools and back. + +@section yangAdaptor Adapting JSON configuration + +Adaptors are tools which adapt JSON complete or partial configuration +before translation to YANG to ease this translation or after translation +from YANG to follow the Kea syntax, for instance by adding static +components which are not in the module. + +Methods provided by adaptors are class methods (i.e. declared static). +Specific adaptors can be derived from the isc::yang::Adaptor base class. + +There are a few basic adaptors and per structure adaptors. The second +category of adaptors are divided into: + - from JSON to YANG adaptors or pre-processing which adapt a JSON + configuration to make it acceptable by a from JSON to YANG (setXXX) + translators. For a Kea module this kind of adaptors fill some required + but missing fields, or only transform a configuration into a canonical + form. Note for a Kea module and a configuration taken from config-get + or config-write it likely does nearly nothing but the code must + handle any hand written configuration so these adaptors are always + applied. + + - from YANG to JSON adaptors or post-processing which adapt translated + YANG configuration (by getXXX) to make it acceptable by a Kea server. + By definition, they are not defined for Kea modules. + +@section unitTestsSysrepo Running unit tests with Sysrepo + +To run YANG/NETCONF/Sysrepo tests you need to compile Kea with Sysrepo support: + +@verbatim +./configure --with-libyang --with-sysrepo --with-libyang-cpp --with-sysrepo-cpp +@endverbatim + +For details, see Section "YANG/NETCONF support" in the Kea Administrator +Reference Manual: https://kea.readthedocs.io/en/latest/arm/integrations.html#yang-netconf. + +You also need to install YANG modules, so the unit tests are able to +retrieve, add, update and generally interact with the sysrepo information. +There are several Kea modules (*.yang in src/share/yang/modules/), mostly usable +in production, but one called keatest-module is only used in unit tests. To be +able to run unit tests as a non-root user, which is the recommended way, make +sure the sysrepo repository and /dev/shm/sr* are owned by said user. One way to +prevent sporadic chown-ing is to install sysrepo and the Kea modules as +non-root. + +To install all the modules, run the following script: + +@verbatim +./src/share/yang/modules/utils/reinstall.sh +@endverbatim + +If the YANG modules are already installed, they will be upgraded. +To avoid any upgrade hastle, the -u flag may be passed to uninstall the modules +before installing again. + +@verbatim +./src/share/yang/modules/utils/reinstall.sh -u +@endverbatim + +Alternatively to install each module, issue the following command: + +@verbatim +sysrepoctl -i "src/share/yang/modules/${module}.yang" +@endverbatim + +To verify that you have the schemas installed, do this: +@verbatim +sysrepoctl -l +@endverbatim + +Make sure that keatest-module and other necessary modules are on the list. + +As DHCP modules are still being developed, if the revision has been bumped, +reinstalling it will update the module automatically. Otherwise, it can be +useful to uninstall them before reinstalling a more recent version: + +@verbatim +sysrepoctl -u <module-name> +@endverbatim + +Tests use these modules which you can find in src/share/yang/modules in addition +to keatest-module: + - ietf-dhcpv6-server + - kea-ctrl-agent + - kea-dhcp-ddns + - kea-dhcp4-server + - kea-dhcp6-server + +Those modules depend on the following modules: + - ietf-inet-types + - ietf-yang-types + - ietf-interfaces + - kea-types + - kea-dhcp-types + +The following modules are extracted from the IETF DHCPv6 YANG draft: + - ietf-dhcpv6-client + - ietf-dhcpv6-options + - ietf-dhcpv6-relay + - ietf-dhcpv6-types + +All are available in the src/share/yang/modules directory using the +<module-name>[@<revision>].yang syntax for file names. +src/share/yang/modules/utils provides a few utilities for developers: + - check-revisions.sh which verifies if the revision in the file name + and in the file content matches + - check-hashes.sh which detects updates in the file content without + a revision change using the SHA-256 hash of the to YIN translation. + Updates hashes automatically if -a is passed to the script. + - gen-revisions.sh which produces the module / revision table of + the yang_revisions.h header file. + - reinstall.sh which installs all the modules. + +You can run this tool: + +@verbatim +src/lib/yang/pretests/sysrepo_setup_tests +@endverbatim + +to verify that your environment is ready. If there is anything +wrong, it will enumerate the problems and will suggest how to solve +them. + +@section yangMTConsiderations Multi-Threading Consideration for YANG Utilities + +The YANG utilities are not thread-safe. Note as they are used only in a +configuration context it is not a problem, and the libyang and sysrepo libraries +are multi-threaded so their APIs are thread-safe. + +*/ |