diff options
Diffstat (limited to 'docs/README')
| -rw-r--r-- | docs/README | 273 |
1 files changed, 273 insertions, 0 deletions
diff --git a/docs/README b/docs/README new file mode 100644 index 0000000..86d46a3 --- /dev/null +++ b/docs/README @@ -0,0 +1,273 @@ +tdb - a trivial database system +tridge@linuxcare.com December 1999 +================================== + +This is a simple database API. It was inspired by the realisation that +in Samba we have several ad-hoc bits of code that essentially +implement small databases for sharing structures between parts of +Samba. As I was about to add another I realised that a generic +database module was called for to replace all the ad-hoc bits. + +I based the interface on gdbm. I couldn't use gdbm as we need to be +able to have multiple writers to the databases at one time. + +Compilation +----------- + +add HAVE_MMAP=1 to use mmap instead of read/write +add NOLOCK=1 to disable locking code + +Testing +------- + +Compile tdbtest.c and link with gdbm for testing. tdbtest will perform +identical operations via tdb and gdbm then make sure the result is the +same + +Also included is tdbtool, which allows simple database manipulation +on the commandline. + +tdbtest and tdbtool are not built as part of Samba, but are included +for completeness. + +Interface +--------- + +The interface is very similar to gdbm except for the following: + +- different open interface. The tdb_open call is more similar to a + traditional open() +- no tdbm_reorganise() function +- no tdbm_sync() function. No operations are cached in the library anyway +- added a tdb_traverse() function for traversing the whole database +- added transactions support + +A general rule for using tdb is that the caller frees any returned +TDB_DATA structures. Just call free(p.dptr) to free a TDB_DATA +return value called p. This is the same as gdbm. + +here is a full list of tdb functions with brief descriptions. + + +---------------------------------------------------------------------- +TDB_CONTEXT *tdb_open(char *name, int hash_size, int tdb_flags, + int open_flags, mode_t mode) + + open the database, creating it if necessary + + The open_flags and mode are passed straight to the open call on the database + file. A flags value of O_WRONLY is invalid + + The hash size is advisory, use zero for a default value. + + return is NULL on error + + possible tdb_flags are: + TDB_CLEAR_IF_FIRST - clear database if we are the only one with it open + TDB_INTERNAL - don't use a file, instead store the data in + memory. The filename is ignored in this case. + TDB_NOLOCK - don't do any locking + TDB_NOMMAP - don't use mmap + TDB_NOSYNC - don't synchronise transactions to disk + TDB_SEQNUM - maintain a sequence number + TDB_VOLATILE - activate the per-hashchain freelist, default 5 + TDB_ALLOW_NESTING - allow transactions to nest + TDB_DISALLOW_NESTING - disallow transactions to nest + +---------------------------------------------------------------------- +TDB_CONTEXT *tdb_open_ex(char *name, int hash_size, int tdb_flags, + int open_flags, mode_t mode, + const struct tdb_logging_context *log_ctx, + tdb_hash_func hash_fn) + +This is like tdb_open(), but allows you to pass an initial logging and +hash function. Be careful when passing a hash function - all users of +the database must use the same hash function or you will get data +corruption. + + +---------------------------------------------------------------------- +char *tdb_error(TDB_CONTEXT *tdb); + + return a error string for the last tdb error + +---------------------------------------------------------------------- +int tdb_close(TDB_CONTEXT *tdb); + + close a database + +---------------------------------------------------------------------- +TDB_DATA tdb_fetch(TDB_CONTEXT *tdb, TDB_DATA key); + + fetch an entry in the database given a key + if the return value has a null dptr then a error occurred + + caller must free the resulting data + +---------------------------------------------------------------------- +int tdb_parse_record(struct tdb_context *tdb, TDB_DATA key, + int (*parser)(TDB_DATA key, TDB_DATA data, + void *private_data), + void *private_data); + + Hand a record to a parser function without allocating it. + + This function is meant as a fast tdb_fetch alternative for large records + that are frequently read. The "key" and "data" arguments point directly + into the tdb shared memory, they are not aligned at any boundary. + + WARNING: The parser is called while tdb holds a lock on the record. DO NOT + call other tdb routines from within the parser. Also, for good performance + you should make the parser fast to allow parallel operations. + + tdb_parse_record returns -1 if the record was not found. If the record was + found, the return value of "parser" is passed up to the caller. + +---------------------------------------------------------------------- +int tdb_exists(TDB_CONTEXT *tdb, TDB_DATA key); + + check if an entry in the database exists + + note that 1 is returned if the key is found and 0 is returned if not found + this doesn't match the conventions in the rest of this module, but is + compatible with gdbm + +---------------------------------------------------------------------- +int tdb_traverse(TDB_CONTEXT *tdb, int (*fn)(TDB_CONTEXT *tdb, + TDB_DATA key, TDB_DATA dbuf, void *state), void *state); + + traverse the entire database - calling fn(tdb, key, data, state) on each + element. + + return -1 on error or the record count traversed + + if fn is NULL then it is not called + + a non-zero return value from fn() indicates that the traversal + should stop. Traversal callbacks may not start transactions. + + WARNING: The data buffer given to the callback fn does NOT meet the + alignment restrictions malloc gives you. + +---------------------------------------------------------------------- +int tdb_traverse_read(TDB_CONTEXT *tdb, int (*fn)(TDB_CONTEXT *tdb, + TDB_DATA key, TDB_DATA dbuf, void *state), void *state); + + traverse the entire database - calling fn(tdb, key, data, state) on + each element, but marking the database read only during the + traversal, so any write operations will fail. This allows tdb to + use read locks, which increases the parallelism possible during the + traversal. + + return -1 on error or the record count traversed + + if fn is NULL then it is not called + + a non-zero return value from fn() indicates that the traversal + should stop. Traversal callbacks may not start transactions. + +---------------------------------------------------------------------- +TDB_DATA tdb_firstkey(TDB_CONTEXT *tdb); + + find the first entry in the database and return its key + + the caller must free the returned data + +---------------------------------------------------------------------- +TDB_DATA tdb_nextkey(TDB_CONTEXT *tdb, TDB_DATA key); + + find the next entry in the database, returning its key + + the caller must free the returned data + +---------------------------------------------------------------------- +int tdb_delete(TDB_CONTEXT *tdb, TDB_DATA key); + + delete an entry in the database given a key + +---------------------------------------------------------------------- +int tdb_store(TDB_CONTEXT *tdb, TDB_DATA key, TDB_DATA dbuf, int flag); + + store an element in the database, replacing any existing element + with the same key + + If flag==TDB_INSERT then don't overwrite an existing entry + If flag==TDB_MODIFY then don't create a new entry + + return 0 on success, -1 on failure + +---------------------------------------------------------------------- +int tdb_writelock(TDB_CONTEXT *tdb); + + lock the database. If we already have it locked then don't do anything + +---------------------------------------------------------------------- +int tdb_writeunlock(TDB_CONTEXT *tdb); + unlock the database + +---------------------------------------------------------------------- +int tdb_chainlock(TDB_CONTEXT *tdb, TDB_DATA key); + + lock one hash chain. This is meant to be used to reduce locking + contention - it cannot guarantee how many records will be locked + +---------------------------------------------------------------------- +int tdb_chainunlock(TDB_CONTEXT *tdb, TDB_DATA key); + + unlock one hash chain + +---------------------------------------------------------------------- +int tdb_transaction_start(TDB_CONTEXT *tdb) + + start a transaction. All operations after the transaction start can + either be committed with tdb_transaction_commit() or cancelled with + tdb_transaction_cancel(). + + If you call tdb_transaction_start() again on the same tdb context + while a transaction is in progress, then the same transaction + buffer is re-used. The number of tdb_transaction_{commit,cancel} + operations must match the number of successful + tdb_transaction_start() calls. + + Note that transactions are by default disk synchronous, and use a + recover area in the database to automatically recover the database + on the next open if the system crashes during a transaction. You + can disable the synchronous transaction recovery setup using the + TDB_NOSYNC flag, which will greatly speed up operations at the risk + of corrupting your database if the system crashes. + + Operations made within a transaction are not visible to other users + of the database until a successful commit. + +---------------------------------------------------------------------- +int tdb_transaction_cancel(TDB_CONTEXT *tdb) + + cancel a current transaction, discarding all write and lock + operations that have been made since the transaction started. + + +---------------------------------------------------------------------- +int tdb_transaction_commit(TDB_CONTEXT *tdb) + + commit a current transaction, updating the database and releasing + the transaction locks. + +---------------------------------------------------------------------- +int tdb_transaction_prepare_commit(TDB_CONTEXT *tdb) + + prepare to commit a current transaction, for two-phase commits. + Once prepared for commit, the only allowed calls are + tdb_transaction_commit() or tdb_transaction_cancel(). Preparing + allocates disk space for the pending updates, so a subsequent + commit should succeed (barring any hardware failures). + +---------------------------------------------------------------------- +int tdb_check(TDB_CONTEXT *tdb, + int (*check)(TDB_DATA key, TDB_DATA data, void *private_data), + void *private_data);) + + check the consistency of the database, calling back the check function + (if non-NULL) with each record. If some consistency check fails, or + the supplied check function returns -1, tdb_check returns -1, otherwise + 0. Note that logging function (if set) will be called with additional + information on the corruption found. |