Adding upstream version 2.4.1.upstream/2.4.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 381 insertions, 0 deletions

diff --git a/src/lib/hooks/hooks_maintenance.dox b/src/lib/hooks/hooks_maintenance.dox
new file mode 100644
index 0000000..75e1227
--- /dev/null
+++ b/src/lib/hooks/hooks_maintenance.dox
@@ -0,0 +1,381 @@
+// Copyright (C) 2013-2020 Internet Systems Consortium, Inc. ("ISC")
+//
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+// Note: the prefix "hooksmg" to all labels is an abbreviation for "Hooks
+// Maintenance Guide" and is used to prevent a clash with symbols in any
+// other Doxygen file.
+
+/**
+ @page hooksmgMaintenanceGuide Hooks Maintenance Guide
+
+ @section hooksmgIntroduction Introduction
+
+ This document is aimed at Kea maintainers responsible for the hooks
+ system.  It provides an overview of the classes that make up the hooks
+ framework and notes important aspects of processing.  More detailed
+ information can be found in the source code.
+
+ It is assumed that the reader is familiar with the contents of the @ref
+ hooksdgDevelopersGuide and the @ref hooksComponentDeveloperGuide.
+
+ @section hooksmgObjects Hooks Framework Objects
+
+ The relationships between the various objects in the hooks framework
+ is shown below:
+
+ @image html HooksUml.png "High-Level Class Diagram of the Hooks Framework"
+
+ (To avoid clutter, the @ref hooksmgServerHooks object, used to pass
+ information about registered hooks to the components, is not shown on
+ the diagram.)
+
+ The hooks framework objects can be split into user-side objects and
+ server-side objects.  The former are those objects used or referenced
+ by user-written hooks libraries.  The latter are those objects used in
+ the hooks framework.
+
+ @subsection hooksmgUserObjects User-Side Objects
+
+ The user-side code is able to access two objects in the framework,
+ the @ref hooksmgCalloutHandle and the @ref hooksmgLibraryHandle.
+ The @ref hooksmgCalloutHandle is used to pass data between the Kea
+ component and the loaded library; the @ref hooksmgLibraryHandle is used
+ for registering callouts.
+
+ @subsubsection hooksmgCalloutHandle Callout Handle
+
+ The @ref isc::hooks::CalloutHandle has two functions: passing arguments
+ between the Kea component and the user-written library, and storing
+ per-request context between library calls.  In both cases the data is
+ stored in a @c std::map structure, keyed by argument (or context item) name.
+ The actual data is stored in a @c boost::any object, which allows any
+ data type to be stored, although a penalty for this flexibility is
+ the restriction (mentioned in the @ref hooksdgDevelopersGuide) that
+ the type of data retrieved must be identical (and not just compatible)
+ with that stored.
+
+ The storage of context data is slightly complex because there is
+ separate context for each user library.  For this reason, the @ref
+ hooksmgCalloutHandle has multiple maps, one for each library loaded.
+ The maps are stored in another map, the appropriate map being identified
+ by the "current library index" (this index is explained further below).
+ The reason for the second map (rather than a structure such as a vector)
+ is to avoid creating individual context maps unless needed; given the
+ key to the map (in this case the current library index) accessing an
+ element in a map using the operator[] method returns the element in
+ question if it exists, or creates a new one (and stores it in the map)
+ if its doesn't.
+
+ @subsubsection hooksmgLibraryHandle Library Handle
+
+ Little more than a restricted interface to the @ref
+ hooksmgCalloutManager, the @ref isc::hooks::LibraryHandle allows a
+ callout to register and deregister callouts.  However, there are some
+ quirks to callout registration which, although the processing involved
+ is in the @ref hooksmgCalloutManager, are best described here.
+
+ Firstly, a callout can be deregistered by a function within a user
+ library only if it was registered by a function within that library. That
+ is to say, if library A registers the callout A_func() on hook "alpha"
+ and library B registers B_func(), functions within library A are only
+ able to remove A_func() (and functions in library B remove B_func()).
+ The restriction - here to prevent one library interfering with the
+ callouts of another - is enforced by means of the current library index.
+ As described below, each entry in the vector of callouts associated with
+ a hook is a pair object, comprising a pointer to the callout and
+ the index of the library with which it is associated. A callout
+ can only modify entries in that vector where the current library index
+ matches the index element of the pair.
+
+ A second quirk is that when dynamically modifying the list of callouts,
+ the change only takes effect when the current call out from the server
+ completes. To clarify this, suppose that functions A_func(), B_func()
+ and C_func() are registered on a hook, and the server executes a callout
+ on the hook. Suppose also during this call, A_func() removes the callout
+ C_func() and that B_func() adds D_func(). As changes only take effect
+ when the current call out completes, the user callouts executed will be
+ A_func(), B_func() then C_func(). When the server calls the hook callouts
+ again, the functions executed will be A_func(), B_func() and D_func().
+
+ This restriction is down to implementation.  When a set of callouts on a hook
+ is being called, the @ref hooksmgCalloutManager iterates through a
+ vector (the "callout vector") of (index, callout pointer) pairs.  Since
+ registration or deregistration of a callout on that hook would change the
+ vector (and so potentially invalidate the iterators used to access the it),
+ a copy of the vector is taken before the iteration starts.  The @ref
+ hooksmgCalloutManager iterates over this copy while any changes made
+ by the callout registration functions affect the relevant callout vector.
+ Such approach was chosen because of performance considerations.
+
+ @subsection hooksmgServerObjects Server-Side Objects
+
+ Those objects are not accessible by user libraries. Please do not
+ attempt to use them if you are developing user callouts.
+
+ @subsubsection hooksmgServerHooks Server Hooks
+
+ The singleton @ref isc::hooks::ServerHooks object is used to register
+ hooks. It is little more than a wrapper around a map of (hook index,
+ hook name), generating a unique number (the hook index) for each
+ hook registered.  It also handles the registration of the pre-defined
+ context_create and context_destroy hooks.
+
+ In operation, the @ref hooksmgHooksManager provides a thin wrapper
+ around it, so that the Kea component developer does not have to
+ worry about another object.
+
+ @subsubsection hooksmgLibraryManager Library Manager
+
+ An @ref isc::hooks::LibraryManager is created by the @ref
+ hooksmgHooksManager object for each shared library loaded. It
+ controls the loading and unloading of the library and in essence
+ represents the library in the hooks framework. It also handles the
+ registration of the standard callouts (functions in the library with
+ the same name as the hook name).
+
+ Of particular importance is the "library's index", a number associated
+ with the library.  This is passed to the LibraryManager at creation
+ time and is used to tag the callout pointers.  It is discussed
+ further below.
+
+ As the @c LibraryManager provides all the methods needed to manage the
+ shared library, it is the natural home for the static @c validateLibrary()
+ method. The function called the parsing of the Kea configuration, when
+ the "hooks-libraries" element is processed. It checks that shared library
+ exists, that it can be opened, that it contains the @c version() function
+ and that that function returns a valid value. It then closes the shared
+ library and returns an appropriate indication as to the library status.
+
+ @subsubsection hooksmgLibraryManagerCollection Library Manager Collection
+
+ The hooks framework can handle multiple libraries and as
+ a result will create a @ref hooksmgLibraryManager for each
+ of them. The collection of LibraryManagers is managed by the
+ @ref isc::hooks::LibraryManagerCollection object which, in most
+ cases has a method corresponding to a @ref hooksmgLibraryManager
+ method, e.g. it has a @c loadLibraries() that corresponds to the @ref
+ hooksmgLibraryManager's loadLibrary() call. As would be expected, methods
+ on the @c LibraryManagerCollection iterate through all specified libraries,
+ calling the corresponding LibraryManager method for each library.
+
+ One point of note is that @c LibraryManagerCollection operates on an "all
+ or none" principle. When @c loadLibraries() is called, on exit either all
+ libraries have been successfully opened or none of them have. There
+ is no use-case in Kea where, after a user has specified the shared
+ libraries they want to load, the system will operate with only some of
+ them loaded.
+
+ The @c LibraryManagerCollection is the place where each library's index is set.
+ Each library is assigned a number ranging from 1 through to the number
+ of libraries being loaded.  As mentioned in the previous section, this
+ index is used to tag callout pointers, something that is discussed
+ in the next section.
+
+ (Whilst on the subject of library index numbers, two additional
+ numbers - 0 and @c INT_MAX - are also valid as "current library index".
+ For flexibility, the Kea component is able to register its own
+ functions as hook callouts.  It does this by obtaining a suitable @ref
+ hooksmgLibraryHandle from the @ref hooksmgHooksManager.  A choice
+ of two is available: one @ref hooksmgLibraryHandle (with an index
+ of 0) can be used to register a callout on a hook to execute before
+ any user-supplied callouts.  The second (with an index of @c INT_MAX)
+ is used to register a callout to run after user-specified callouts.
+ Apart from the index number, the hooks framework does not treat these
+ callouts any differently from user-supplied ones.)
+
+ @subsubsection hooksmgCalloutManager Callout Manager
+
+ The @ref isc::hooks::CalloutManager is the core of the framework insofar
+ as the registration and calling of callouts is concerned.
+
+ It maintains a "hook vector" - a vector with one element for
+ each registered hook. Each element in this vector is itself a
+ vector (the callout vector), each element of which is a pair of
+ (library index, callback pointer). When a callout is registered, the
+ CalloutManager's current library index is used to supply the "library
+ index" part of the pair. The library index is set explicitly by the
+ @ref hooksmgLibraryManager prior to calling the user library's load()
+ function (and prior to registering the standard callbacks).
+
+ The situation is slightly more complex when a callout is executing. In
+ order to execute a callout, the CalloutManager's @c callCallouts()
+ method must be called. This iterates through the callout vector for
+ a hook and for each element in the vector, uses the "library index"
+ part of the pair to set the "current library index" before calling the
+ callout function recorded in the second part of the pair. In most cases,
+ the setting of the library index has no effect on the callout. However,
+ if the callout wishes to dynamically register or deregister a callout,
+ the @ref hooksmgLibraryHandle (see above) calls a method on the
+ @ref hooksmgCalloutManager which in turn uses that information.
+
+ @subsubsection hooksmgHooksManager Hooks Manager
+
+ The @ref isc::hooks::HooksManager is the main object insofar as the
+ server is concerned. It controls the creation of the library-related
+ objects and provides the framework in which they interact. It also
+ provides a shell around objects such as @ref hooksmgServerHooks so that all
+ interaction with the hooks framework by the server is through the
+ HooksManager object.  Apart from this, it supplies no functionality to
+ the hooks framework.
+
+ @section hooksmgOtherIssues Other Issues
+
+ @subsection hooksmgMemoryAllocation Memory Allocation
+
+ Unloading a shared library works by unmapping the part of the process's
+ virtual address space in which the library lies. This may lead to
+ problems if there are still references to that address space elsewhere
+ in the process.
+
+ In many operating systems, heap storage allowed by a shared library
+ will lie in the virtual address allocated to the library. This has
+ implications in the hooks framework because:
+
+ - Argument information stored in a @ref hooksmgCalloutHandle by a
+ callout in a library may lie in the library's address space.
+
+ - Data modified in objects passed as arguments may lie in the address
+ space. For example, it is common for a DHCP callout to add "options"
+ to a packet: the memory allocated for those options will most likely
+ lie in library address space.
+
+ The problem really arises because of the extensive use by Kea of
+ boost smart pointers. When the pointer is destroyed, the pointed-to
+ memory is deallocated. If the pointer points to address space that is
+ unmapped because a library has been unloaded, the deletion causes a
+ segmentation fault.
+
+ The hooks framework addresses the issue for the @ref hooksmgCalloutHandle
+ by keeping in that object a shared pointer to the object controlling
+ library unloading (the @ref hooksmgLibraryManagerCollection). Although
+ the libraries can be unloaded at any time, it is only when every
+ @ref hooksmgCalloutHandle that could possibly reference address space in the
+ library have been deleted that the library will actually be unloaded
+ and the address space unmapped.
+
+ The hooks framework cannot solve the second issue as the objects in
+ question are under control of the Kea server incorporating the
+ hooks. It is up to the server developer to ensure that all such objects
+ have been destroyed before libraries are reloaded. In extreme cases
+ this may mean the server suspending all processing of incoming requests
+ until all currently executing requests have completed and data object
+ destroyed, reloading the libraries, then resuming processing.
+
+ Since Kea 1.7.10 the unload() entry point is called as the first phase
+ of unloading. This gives more chance to hooks writer to perform
+ necessary cleanup actions so the second phase, memory unmapping
+ can safely happen. The @c isc::hooks::unloadLibraries() function
+ was updated too to return false when at least one active callout
+ handle remained.
+
+ @subsection hooksmgStaticLinking Hooks and Statically-Linked Kea
+
+ Kea has the configuration option to allow static linking.  What this
+ means is that it links against the static Kea libraries and not
+ the sharable ones - although it links against the sharable system
+ libraries like "libc" and "libstdc++" and well as the sharable libraries
+ for third-party packages such as log4cplus and MySql.
+
+ Static linking poses a problem for dynamically-loaded hooks libraries
+ as some of the code in them - in particular the hooks framework and
+ the logging code - depend on global objects created within the Kea
+ libraries.  In the normal course of events (Kea linked against
+ shared libraries), when Kea is run and the operating system loads
+ a Kea shared library containing a global object, address space
+ is assigned for it.  When the hooks framework loads a user-library
+ linked against the same Kea shared library, the operating system
+ recognizes that the library is already loaded (and initialized) and
+ uses its definition of the global object.  Thus both the code in the
+ Kea image and the code in the user-written shared library
+ reference the same object.
+
+ If Kea is statically linked, the linker allocates address space
+ in the Kea image for the global object and does not include any
+ reference to the shared library containing it.  When Kea now loads
+ the user-written shared library - and so loads the Kea library code
+ containing the global object - the operating system does not know that
+ the object already exists.  Instead, it allocates new address space.
+ The version of Kea in memory therefore has two copies of the object:
+ one referenced by code in the Kea image, and one referenced by code
+ in the user-written hooks library.  This causes problems - information
+ put in one copy is not available to the other.
+
+ Particular problems were encountered with global objects the hooks library
+ and in the logging library, so some code to alleviate the problem has been
+ included.
+
+ The issue in the hooks library is the singleton @ref
+ isc::hooks::ServerHooks object, used by the user-written hooks library
+ if it attempts to register or deregister callouts.  The contents of the
+ singleton - the names of the hook points and their index - are set by
+ the relevant Kea server; this information is not available in the
+ singleton created in the user's hooks library.
+
+ Within the code users by the user's hooks library, the @c ServerHooks
+ object is used by @ref isc::hooks::CalloutHandle and @ref
+ isc::hooks::CalloutManager objects.  Both these objects are passed to the
+ hooks library code when a callout is called: the former directly through
+ the callout argument list, the latter indirectly as a pointer to it is
+ stored in the CalloutHandle.  This allows a solution to the problem:
+ instead of accessing the singleton via @c ServerHooks::getServerHooks(),
+ the constructors of these objects store a reference to the singleton
+ @c ServerHooks when they are created and use that reference to access
+ @c ServerHooks data.  Since both @c CalloutHandle and @c CalloutManager are
+ created in the statically-linked Kea server, use of the reference
+ means that it is the singleton within the server - and not the one
+ within the user's hooks library - that is referenced.
+
+ The solution of the logging problem is not so straightforward.  Within
+ Kea, there are two logging components, the Kea logging framework
+ and the log4cplus libraries.  Owing to static linking, there are two
+ instances of the former; but as static linking uses shared libraries of
+ third-party products, there is one instance of the latter.  What further
+ complicates matters is that initialization of the logging framework is
+ in two parts: static initialization and run-time initialization.
+
+ The logging initialization comprises the following:
+
+ -# Static initialization of the log4cplus global variables.
+ -# Static initialization of messages in the various Kea libraries.
+ -# Static initialization of logging framework.
+ -# Run-time initialization of the logging framework.
+ -# Run-time initialization of log4cplus
+
+ As both the Kea server and the user-written hooks libraries use the
+ log4cplus shared library, item 1 - the static initialization of the log4cplus
+ global variables is performed once.
+
+ The next two tasks - static initialization of the messages in the Kea
+ libraries and the static initialization of the logging framework -
+ are performed twice, once in the context of the Kea server and
+ once in the context of the hooks library.  For this reason, run-time
+ initialization of the logging framework needs to be performed twice,
+ once in the context of the Kea server and once in the context of the
+ user-written hooks library.  However, the standard logging framework
+ initialization code also performs the last task, initialization of
+ log4cplus, something that causes problems if executed more than once.
+
+ To get round this, the function @ref isc::hooks::hooksStaticLinkInit()
+ has been written.  It executes the only part of the logging framework
+ run-time initialization that actually pertains to the logging framework
+ and not log4cplus, namely loading the message dictionary with the
+ statically-initialized messages in the Kea libraries.
+ This should be executed by any hooks library linking against a statically
+ initialized Kea.  (In fact, running it against a dynamically-linked
+ Kea should have no effect, as the load operation discards any duplicate
+ message entries.)  The hooks library tests do this, the code being
+ conditionally compiled within a test of the @c USE_STATIC_LINK macro, set
+ by the configure script.
+
+ @note Not everything is completely rosy with logging and static linking.
+ In particular, there appears to be an issue with the scenario where a
+ user-written hooks library is run by a statically-linked Kea and then
+ unloaded.  As far as can be determined, on unload the system attempts to
+ delete the same logger twice.  This is alleviated by explicitly clearing
+ the loggerptr_ variable in the @ref isc::log::Logger destructor, but there
+ is a suspicion that some memory might be lost in these circumstances.
+ This is still under investigation.
+*/