diff options
Diffstat (limited to 'servers/slapd/overlays/slapover.txt')
| -rw-r--r-- | servers/slapd/overlays/slapover.txt | 158 |
1 files changed, 158 insertions, 0 deletions
diff --git a/servers/slapd/overlays/slapover.txt b/servers/slapd/overlays/slapover.txt new file mode 100644 index 0000000..2015d8d --- /dev/null +++ b/servers/slapd/overlays/slapover.txt @@ -0,0 +1,158 @@ +slapd internal APIs + +Introduction + +Frontend, backend, database, callback, overlay - what does it all mean? + +The "frontend" refers to all the code that deals with the actual interaction +with an LDAP client. This includes the code to read requests from the network +and parse them into C data structures, all of the session management, and the +formatting of responses for transmission onto the network. It also includes the +access control engine and other features that are generic to LDAP processing, +features which are not dependent on a particular database implementation. +Because the frontend serves as the framework that ties everything together, +it should not change much over time. + +The terms "backend" and "database" have historically been used interchangeably +and/or in combination as if they are the same thing, but the code has a clear +distinction between the two. A "backend" is a type of module, and a "database" +is an instance of a backend type. Together they work with the frontend to +manage the actual data that are operated on by LDAP requests. Originally the +backend interface was relatively compact, with individual functions +corresponding to each LDAP operation type, plus functions for init, config, and +shutdown. The number of entry points has grown to allow greater flexibility, +but the concept is much the same as before. + +The language here can get a bit confusing. A backend in slapd is embodied in a +BackendInfo data structure, and a database is held in a BackendDB structure. +Originally it was all just a single Backend data structure, but things have +grown and the concept was split into these two parts. The idea behind the +distinct BackendInfo is to allow for any initialization and configuration that +may be needed by every instance of a type of database, as opposed to items that +are specific to just one instance. For example, you might have a database +library that requires an initialization routine to be called exactly once at +program startup. Then there may be a "open" function that must be called once +for each database instance. The BackendInfo.bi_open function provides the +one-time startup, while the BackendInfo.bi_db_open function provides the +per-database startup. The main feature of the BackendInfo structure is its +table of entry points for all of the database functions that it implements. +There's also a bi_private pointer that can be used to carry any configuration +state needed by the backend. (Note that this is state that applies to the +backend type, and thus to all database instances of the backend as well.) The +BackendDB structure carries all of the per-instance state for a backend +database. This includes the database suffix, ACLs, flags, various DNs, etc. It +also has a pointer to its BackendInfo, and a be_private pointer for use by the +particular backend instance. In practice, the per-type features are seldom +used, and all of the work is done in the per-instance data structures. + +Ordinarily an LDAP request is received by the slapd frontend, parsed into a +request structure, and then passed to the backend for processing. The backend +may call various utility functions in the frontend to assist in processing, and +then it eventually calls some send_ldap_result function in the frontend to send +results back to the client. The processing flow is pretty rigidly defined; even +though slapd is capable of dynamically loading new code modules, it was +difficult to add extensions that changed the basic protocol operations. If you +wanted to extend the server with special behaviors you would need to modify the +frontend or the backend or both, and generally you would need to write an +entire new backend to get some set of special features working. With OpenLDAP +2.1 we added the notion of a callback, which can intercept the results sent +from a backend before they are sent to a client. Using callbacks makes it +possible to modify the results if desired, or to simply discard the results +instead of sending them to any client. This callback feature is used +extensively in the SASL support to perform internal searches of slapd databases +when mapping authentication IDs into regular DNs. The callback mechanism is +also the basis of backglue, which allows separate databases to be searched as +if they were a single naming context. + +Very often, one needs to add just a tiny feature onto an otherwise "normal" +database. The usual way to achieve this was to use a programmable backend (like +back-perl) to preprocess various requests and then forward them back into slapd +to be handled by the real database. While this technique works, it is fairly +inefficient because it involves many transitions from network to slapd and back +again. The overlay concept introduced in OpenLDAP 2.2 allows code to be +inserted between the slapd frontend and any backend, so that incoming requests +can be intercepted before reaching the backend database. (There is also a SLAPI +plugin framework in OpenLDAP 2.2; it offers a lot of flexibility as well but is +not discussed here.) The overlay framework also uses the callback mechanism, so +outgoing results can also be intercepted by external code. All of this could +get unwieldy if a lot of overlays were being used, but there was also another +significant API change in OpenLDAP 2.2 to streamline internal processing. (See +the document "Refactoring the slapd ABI"...) + +OK, enough generalities... You should probably have a copy of slap.h in front +of you to continue here. + +What is an overlay? The structure defining it includes a BackendInfo structure +plus a few additional fields. It gets inserted into the usual frontend->backend +call chain by replacing the BackendDB's BackendInfo pointer with its own. The +framework to accomplish this is in backover.c. For a given backend, the +BackendInfo will point to a slap_overinfo structure. The slap_overinfo has a +BackendInfo that points to all of the overlay framework's entry points. It also +holds a copy of the original BackendInfo pointer, and a linked list of +slap_overinst structures. There is one slap_overinst per configured overlay, +and the set of overlays configured on a backend are treated like a stack; i.e., +the last one configured is at the top of the stack, and it executes first. + +Continuing with the stack notion - a request enters the frontend, is directed +to a backend by select_backend, and then intercepted by the top of the overlay +stack. This first overlay may do something with the request, and then return +SLAP_CB_CONTINUE, which will then cause processing to fall into the next +overlay, and so on down the stack until finally the request is handed to the +actual backend database. Likewise, when the database finishes processing and +sends a result, the overlay callback intercepts this and the topmost overlay +gets to process the result. If it returns SLAP_CB_CONTINUE then processing will +continue in the next overlay, and then any other callbacks, then finally the +result reaches the frontend for sending back to the client. At any step along +the way, a module may choose to fully process the request or result and not +allow it to propagate any further down the stack. Whenever a module returns +anything other than SLAP_CB_CONTINUE the processing stops. + +An overlay can call most frontend functions without any special consideration. +However, if a call is going to result in any backend code being invoked, then +the backend environment must be correct. During a normal backend invocation, +op->o_bd points to the BackendDB structure for the backend, and +op->o_bd->bd_info points to the BackendInfo for the backend. All of the +information a specific backend instance needs is in op->o_bd->be_private and +all of its entry points are in the BackendInfo structure. When overlays are in +use on a backend, op->o_bd->bd_info points to the BackendInfo (actually a +slap_overinfo) that contains the overlay framework. When a particular overlay +instance is executing, op->o_bd points to a copy of the original op->o_bd, and +op->o_bd->bd_info points to a slap_overinst which carries the information about +the current overlay. The slap_overinst contains an on_private pointer which can +be used to carry any configuration or state information the overlay needs. The +normal way to invoke a backend function is through the op->o_bd->bd_info table +of entry points, but obviously this must be set to the backend's original +BackendInfo in order to get to the right function. + +There are two approaches here. The slap_overinst also contains a on_info field +that points to the top slap_overinfo that wraps the current backend. The +simplest thing is for the overlay to set op->o_bd->bd_info to this on_info +value before invoking a backend function. This will cause processing of that +particular operation to begin at the top of the overlay stack, so all the other +overlays on the backend will also get a chance to handle this internal request. +The other possibility is to invoke the underlying backend directly, bypassing +the rest of the overlays, by calling through on_info->oi_orig. You should be +careful in choosing this approach, since it precludes other overlays from doing +their jobs. + +One of the more interesting uses for an overlay is to attach two (or more) +different database backends into a single execution stack. Assuming that the +basic frontend-managed information (suffix, rootdn, ACLs, etc.) will be the +same for all of the backends, the only thing the overlay needs to maintain is a +be_private and bd_info pointer for the added backends. The chain and proxycache +overlays are two complementary examples of this usage. The chain overlay +attaches a back-ldap backend to a local database backend, and allows referrals +to remote servers generated by the database to be processed by slapd instead of +being returned to the client. The proxycache overlay attaches a local database +to a back-ldap (or back-meta) backend and allows search results from remote +servers to be cached locally. In both cases the overlays must provide a bit of +glue to swap in the appropriate be_private and bd_info pointers before invoking +the attached backend, which can then be invoked as usual. + +Note on overlay initialization/destruction: you should allocate storage for +config info in the _db_init handler, and free this storage in the _db_destroy +handler. You must not free it in the _db_close handler because a module may +be opened/closed multiple times in a running slapd when using dynamic +configuration and the config info must remain intact. + +--- |