summaryrefslogtreecommitdiffstats
path: root/modules/README.rst
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--modules/README.rst251
1 files changed, 251 insertions, 0 deletions
diff --git a/modules/README.rst b/modules/README.rst
new file mode 100644
index 0000000..1096c37
--- /dev/null
+++ b/modules/README.rst
@@ -0,0 +1,251 @@
+.. SPDX-License-Identifier: GPL-3.0-or-later
+
+.. _modules-api:
+
+*********************
+Modules API reference
+*********************
+
+.. contents::
+ :depth: 1
+ :local:
+
+Supported languages
+===================
+
+Currently modules written in C and Lua(JIT) are supported.
+
+The anatomy of an extension
+===========================
+
+A module is a shared object or script defining specific functions/fields; here's an overview.
+
+.. csv-table::
+ :header: "C", "Lua", "Params", "Comment"
+
+ "``X_api()`` [#]_", "", "", "API version"
+ "``X_init()``", "``X.init()``", "``module``", "Constructor"
+ "``X_deinit()``", "``X.deinit()``", "``module``", "Destructor"
+ "``X_config()``", "``X.config()``", "``module, str``", "Configuration"
+ "``X_layer``", "``X.layer``", "", ":ref:`Module layer <lib-layers>`"
+ "``X_props``", "", "", "List of properties"
+
+.. [#] Mandatory symbol; defined by using :c:func:`KR_MODULE_EXPORT`.
+
+The ``X`` corresponds to the module name; if the module name is ``hints``, the prefix for constructor would be ``hints_init()``.
+More details are in docs for the :c:type:`kr_module` and :c:type:`kr_layer_api` structures.
+
+.. note::
+ The modules get ordered -- by default in the same as the order in which they were loaded. The loading command can specify where in the order the module should be positioned.
+
+
+Writing a module in Lua
+=======================
+
+The probably most convenient way of writing modules is Lua since you can use already installed modules
+from system and have first-class access to the scripting engine. You can also tap to all the events, that
+the C API has access to, but keep in mind that transitioning from the C to Lua function is slower than
+the other way round, especially when JIT-compilation is taken into account.
+
+.. note:: The Lua functions retrieve an additional first parameter compared to the C counterparts - a "state".
+ Most useful C functions and structures have lua FFI wrappers, sometimes with extra sugar.
+
+The modules follow the `Lua way <http://lua-users.org/wiki/ModuleDefinition>`_, where the module interface is returned in a named table.
+
+.. code-block:: lua
+
+ --- @module Count incoming queries
+ local counter = {}
+
+ function counter.init(module)
+ counter.total = 0
+ counter.last = 0
+ counter.failed = 0
+ end
+
+ function counter.deinit(module)
+ print('counted', counter.total, 'queries')
+ end
+
+ -- @function Run the q/s counter with given interval.
+ function counter.config(conf)
+ -- We can use the scripting facilities here
+ if counter.ev then event.cancel(counter.ev)
+ event.recurrent(conf.interval, function ()
+ print(counter.total - counter.last, 'q/s')
+ counter.last = counter.total
+ end)
+ end
+
+ return counter
+
+.. vv Hmm, we do not use these coroutine returns anywhere, so it's unclear whether they still work OK. Splitting work over time is now typically done via the ``event`` timers.
+
+.. The API functions may return an integer value just like in other languages, but they may also return a coroutine that will be continued asynchronously. A good use case for this approach is is a deferred initialization, e.g. loading a chunks of data or waiting for I/O.
+
+.. .. code-block:: lua
+
+ function counter.init(module)
+ counter.total = 0
+ counter.last = 0
+ counter.failed = 0
+ return coroutine.create(function ()
+ for line in io.lines('/etc/hosts') do
+ load(module, line)
+ coroutine.yield()
+ end
+ end)
+ end
+
+The created module can be then loaded just like any other module, except it isn't very useful since it
+doesn't provide any layer to capture events. The Lua module can however provide a processing layer, just
+:ref:`like its C counterpart <lib-layers>`.
+
+.. code-block:: lua
+
+ -- Notice it isn't a function, but a table of functions
+ counter.layer = {
+ begin = function (state, data)
+ counter.total = counter.total + 1
+ return state
+ end,
+ finish = function (state, req, answer)
+ if state == kres.FAIL then
+ counter.failed = counter.failed + 1
+ end
+ return state
+ end
+ }
+
+There is currently an additional "feature" in comparison to C layer functions:
+some functions do not get called at all if ``state == kres.FAIL``;
+see docs for details: :c:type:`kr_layer_api`.
+
+Since the modules are like any other Lua modules, you can interact with them through the CLI and and any interface.
+
+.. tip:: Module discovery: ``kres_modules.`` is prepended to the module name and lua search path is used on that.
+
+
+Writing a module in C
+=====================
+
+As almost all the functions are optional, the minimal module looks like this:
+
+.. code-block:: c
+
+ #include "lib/module.h"
+ /* Convenience macro to declare module ABI. */
+ KR_MODULE_EXPORT(mymodule)
+
+.. TODO it's probably not a good idea to start C module tutorial by pthread_create()
+
+Let's define an observer thread for the module as well. It's going to be stub for the sake of brevity,
+but you can for example create a condition, and notify the thread from query processing by declaring
+module layer (see the :ref:`Writing layers <lib-layers>`).
+
+.. code-block:: c
+
+ static void* observe(void *arg)
+ {
+ /* ... do some observing ... */
+ }
+
+ int mymodule_init(struct kr_module *module)
+ {
+ /* Create a thread and start it in the background. */
+ pthread_t thr_id;
+ int ret = pthread_create(&thr_id, NULL, &observe, NULL);
+ if (ret != 0) {
+ return kr_error(errno);
+ }
+
+ /* Keep it in the thread */
+ module->data = thr_id;
+ return kr_ok();
+ }
+
+ int mymodule_deinit(struct kr_module *module)
+ {
+ /* ... signalize cancellation ... */
+ void *res = NULL;
+ pthread_t thr_id = (pthread_t) module->data;
+ int ret = pthread_join(thr_id, res);
+ if (ret != 0) {
+ return kr_error(errno);
+ }
+
+ return kr_ok();
+ }
+
+This example shows how a module can run in the background, this enables you to, for example, observe
+and publish data about query resolution.
+
+Configuring modules
+===================
+
+There is a callback ``X_config()`` that you can implement, see hints module.
+
+.. _mod-properties:
+
+Exposing C module properties
+============================
+
+A module can offer NULL-terminated list of *properties*, each property is essentially a callable with free-form JSON input/output.
+JSON was chosen as an interchangeable format that doesn't require any schema beforehand, so you can do two things - query the module properties
+from external applications or between modules (e.g. `statistics` module can query `cache` module for memory usage).
+JSON was chosen not because it's the most efficient protocol, but because it's easy to read and write and interface to outside world.
+
+.. note:: The ``void *env`` is a generic module interface. Since we're implementing daemon modules, the pointer can be cast to ``struct engine*``.
+ This is guaranteed by the implemented API version (see `Writing a module in C`_).
+
+Here's an example how a module can expose its property:
+
+.. code-block:: c
+
+ char* get_size(void *env, struct kr_module *m,
+ const char *args)
+ {
+ /* Get cache from engine. */
+ struct engine *engine = env;
+ struct kr_cache *cache = &engine->resolver.cache;
+ /* Read item count */
+ int count = (cache->api)->count(cache->db);
+ char *result = NULL;
+ asprintf(&result, "{ \"result\": %d }", count);
+
+ return result;
+ }
+
+ struct kr_prop *cache_props(void)
+ {
+ static struct kr_prop prop_list[] = {
+ /* Callback, Name, Description */
+ {&get_size, "get_size", "Return number of records."},
+ {NULL, NULL, NULL}
+ };
+ return prop_list;
+ }
+
+ KR_MODULE_EXPORT(cache)
+
+Once you load the module, you can call the module property from the interactive console.
+*Note:* the JSON output will be transparently converted to Lua tables.
+
+.. code-block:: bash
+
+ $ kresd
+ ...
+ [system] started in interactive mode, type 'help()'
+ > modules.load('cached')
+ > cached.get_size()
+ [size] => 53
+
+.. No idea what this talks about, but kept for now:
+.. *Note:* this relies on function pointers, so the same ``static inline`` trick as for the ``Layer()`` is required for C.
+
+Special properties
+------------------
+
+If the module declares properties ``get`` or ``set``, they can be used in the Lua interpreter as
+regular tables.
+