From 18657a960e125336f704ea058e25c27bd3900dcb Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 5 May 2024 19:28:19 +0200 Subject: Adding upstream version 3.40.1. Signed-off-by: Daniel Baumann --- www/34to35.html | 945 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 945 insertions(+) create mode 100644 www/34to35.html (limited to 'www/34to35.html') diff --git a/www/34to35.html b/www/34to35.html new file mode 100644 index 0000000..e7b3596 --- /dev/null +++ b/www/34to35.html @@ -0,0 +1,945 @@ + + + + + +SQLite Changes From Version 3.4.2 To 3.5.0 + + + +

+ +

+Small. Fast. Reliable.
Choose any three. +

+ + +

+ +

+ + + +

Moving From SQLite 3.4.2 to 3.5.0

+ SQLite version 3.5.0 (2007-09-04) introduces a new OS interface layer that + is incompatible with all prior versions of SQLite. In addition, + a few existing interfaces have been generalized to work across all + database connections within a process rather than just all + connections within a thread. The purpose of this article + is to describe the changes to 3.5.0 in detail so that users + of prior versions of SQLite can judge what, if any, effort will + be required to upgrade to newer versions. +

1.0 Overview Of Changes

+ A quick enumeration of the changes in SQLite version 3.5.0 + is provided here. Subsequent sections will describe these + changes in more detail. +

The OS interface layer has been completely reworked: +
1. The undocumented sqlite3_os_switch() interface has + been removed.
2. The SQLITE_ENABLE_REDEF_IO compile-time flag no longer functions. + I/O procedures are now always redefinable.
3. Three new objects are defined for specifying I/O procedures: + sqlite3_vfs, sqlite3_file, and sqlite3_io_methods.
4. Three new interfaces are used to create alternative OS interfaces: + sqlite3_vfs_register(), sqlite3_vfs_unregister(), and + sqlite3_vfs_find().
5. A new interface has been added to provided additional control over + the creation of new database connections: sqlite3_open_v2(). + The legacy interfaces of sqlite3_open() and + sqlite3_open16() continue to be fully supported.
The optional shared cache and memory management features that + were introduced in version 3.3.0 can now be used across multiple + threads within the same process. Formerly, these extensions only + applied to database connections operating within a single thread. +
1. The sqlite3_enable_shared_cache() interface now applies to all + threads within a process, not to just the one thread in which it + was run.
2. The sqlite3_soft_heap_limit() interface now applies to all threads + within a process, not to just the one thread in which it was run.
3. The sqlite3_release_memory() interface will now attempt to reduce + the memory usages across all database connections in all threads, not + just connections in the thread where the interface is called.
4. The sqlite3_thread_cleanup() interface has become a no-op.
Restrictions on the use of the same database connection by multiple + threads have been dropped. It is now safe for + multiple threads to use the same database connection at the same + time.
There is now a compile-time option that allows an application to + define alternative malloc()/free() implementations without having + to modify any core SQLite code.
There is now a compile-time option that allows an application to + define alternative mutex implementations without having + to modify any core SQLite code.

+ Of these changes, only 1a and 2a through 2c are incompatibilities + in any formal sense. + But users who have previously made custom modifications to the + SQLite source (for example to add a custom OS layer for embedded + hardware) might find that these changes have a larger impact. + On the other hand, an important goal of these changes is to make + it much easier to customize SQLite for use on different operating + systems. +

2.0 The OS Interface Layer

+ If your system defines a custom OS interface for SQLite or if you + were using the undocumented sqlite3_os_switch() + interface, then you will need to make modifications in order to + upgrade to SQLite version 3.5.0. This may seem painful at first + glance. But as you look more closely, you will probably discover + that your changes are made smaller and easier to understand and manage + by the new SQLite interface. It is likely that your changes will + now also work seamlessly with the SQLite amalgamation. You will + no longer need to make any changes to the code SQLite source code. + All of your changes can be effected by application code and you can + link against a standard, unmodified version of the SQLite amalgamation. + Furthermore, the OS interface layer, which was formerly undocumented, + is now an officially support interface for SQLite. So you have + some assurance that this will be a one-time change and that your + new backend will continue to work in future versions of SQLite. +

2.1 The Virtual File System Object

+ The new OS interface for SQLite is built around an object named + sqlite3_vfs. The "vfs" stands for "Virtual File System". + The sqlite3_vfs object is basically a structure containing pointers + to functions that implement the primitive disk I/O operations that + SQLite needs to perform in order to read and write databases. + In this article, we will often refer to an sqlite3_vfs objects as a "VFS". +

+ SQLite is able to use multiple VFSes at the same time. Each + individual database connection is associated with just one VFS. + But if you have multiple database connections, each connection + can be associated with a different VFS. +

+ There is always a default VFS. + The legacy interfaces sqlite3_open() and sqlite3_open16() always + use the default VFS. + The new interface for creating database connections, + sqlite3_open_v2(), allows you to specify which VFS you want to + use by name. +

2.1.1 Registering New VFS Objects

+ Standard builds of SQLite for Unix or Windows come with a single + VFS named "unix" or "win32", as appropriate. This one VFS is also + the default. So if you are using the legacy open functions, everything + will continue to operate as it has before. The change is that an application + now has the flexibility of adding new VFS modules to implement a + customized OS layer. The sqlite3_vfs_register() API can be used + to tell SQLite about one or more application-defined VFS modules: +

+int sqlite3_vfs_register(sqlite3_vfs*, int makeDflt);
+

+ Applications can call sqlite3_vfs_register() at any time, though of course + a VFS needs to be registered before it can be used. The first argument + is a pointer to a customized VFS object that the application has prepared. + The second argument is true to make the new VFS the default VFS so that + it will be used by the legacy sqlite3_open() and sqlite3_open16() APIs. + If the new VFS is not the default, then you will probably have to use + the new sqlite3_open_v2() API to use it. Note, however, that if + a new VFS is the only VFS known to SQLite (if SQLite was compiled without + its usual default VFS or if the precompiled default VFS was removed + using sqlite3_vfs_unregister()) then the new VFS automatically becomes the + default VFS regardless of the makeDflt argument to sqlite3_vfs_register(). +

+ Standard builds include the default "unix" or "win32" VFSes. + But if you use the -DOS_OTHER=1 compile-time option, then SQLite is + built without a default VFS. In that case, the application must + register at least one VFS prior to calling sqlite3_open(). + This is the approach that embedded applications should use. + Rather than modifying the SQLite source to insert an alternative + OS layer as was done in prior releases of SQLite, instead compile + an unmodified SQLite source file (preferably the amalgamation) + with the -DOS_OTHER=1 option, then invoke sqlite3_vfs_register() + to define the interface to the underlying filesystem prior to + creating any database connections. +

2.1.2 Additional Control Over VFS Objects

+ The sqlite3_vfs_unregister() API is used to remove an existing + VFS from the system. +

+int sqlite3_vfs_unregister(sqlite3_vfs*);
+

+ The sqlite3_vfs_find() API is used to locate a particular VFS + by name. Its prototype is as follows: +

+sqlite3_vfs *sqlite3_vfs_find(const char *zVfsName);
+

+ The argument is the symbolic name for the desired VFS. If the + argument is a NULL pointer, then the default VFS is returned. + The function returns a pointer to the sqlite3_vfs object that + implements the VFS. Or it returns a NULL pointer if no object + could be found that matched the search criteria. +

2.1.3 Modifications Of Existing VFSes

+ Once a VFS has been registered, it should never be modified. If + a change in behavior is required, a new VFS should be registered. + The application could, perhaps, use sqlite3_vfs_find() to locate + the old VFS, make a copy of the old VFS into a new sqlite3_vfs + object, make the desired modifications to the new VFS, unregister + the old VFS, then register the new VFS in its place. Existing + database connections would continue to use the old VFS even after + it is unregistered, but new database connections would use the + new VFS. +

2.1.4 The VFS Object

+ A VFS object is an instance of the following structure: +

+typedef struct sqlite3_vfs sqlite3_vfs;
+struct sqlite3_vfs {
+  int iVersion;            /* Structure version number */
+  int szOsFile;            /* Size of subclassed sqlite3_file */
+  int mxPathname;          /* Maximum file pathname length */
+  sqlite3_vfs *pNext;      /* Next registered VFS */
+  const char *zName;       /* Name of this virtual file system */
+  void *pAppData;          /* Pointer to application-specific data */
+  int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
+               int flags, int *pOutFlags);
+  int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
+  int (*xAccess)(sqlite3_vfs*, const char *zName, int flags);
+  int (*xGetTempName)(sqlite3_vfs*, char *zOut);
+  int (*xFullPathname)(sqlite3_vfs*, const char *zName, char *zOut);
+  void *(*xDlOpen)(sqlite3_vfs*, const char *zFilename);
+  void (*xDlError)(sqlite3_vfs*, int nByte, char *zErrMsg);
+  void *(*xDlSym)(sqlite3_vfs*,void*, const char *zSymbol);
+  void (*xDlClose)(sqlite3_vfs*, void*);
+  int (*xRandomness)(sqlite3_vfs*, int nByte, char *zOut);
+  int (*xSleep)(sqlite3_vfs*, int microseconds);
+  int (*xCurrentTime)(sqlite3_vfs*, double*);
+  /* New fields may be appended in figure versions.  The iVersion
+  ** value will increment whenever this happens. */
+};
+

+ To create a new VFS, an application fills in an instance of this + structure with appropriate values and then calls sqlite3_vfs_register(). +

+ The iVersion field of sqlite3_vfs should be 1 for SQLite version 3.5.0. + This number may increase in future versions of SQLite if we have to + modify the VFS object in some way. We hope that this never happens, + but the provision is made in case it does. +

+ The szOsFile field is the size in bytes of the structure that defines + an open file: the sqlite3_file object. This object will be described + more fully below. The point here is that each VFS implementation can + define its own sqlite3_file object containing whatever information + the VFS implementation needs to store about an open file. SQLite needs + to know how big this object is, however, in order to preallocate enough + space to hold it. +

+ The mxPathname field is the maximum length of a file pathname that + this VFS can use. SQLite sometimes has to preallocate buffers of + this size, so it should be as small as reasonably possible. Some + filesystems permit huge pathnames, but in practice pathnames rarely + extend beyond 100 bytes or so. You do not have to put the longest + pathname that the underlying filesystem can handle here. You only + have to put the longest pathname that you want SQLite to be able to + handle. A few hundred is a good value in most cases. +

+ The pNext field is used internally by SQLite. Specifically, SQLite + uses this field to form a linked list of registered VFSes. +

+ The zName field is the symbolic name of the VFS. This is the name + that the sqlite3_vfs_find() compares against when it is looking for + a VFS. +

+ The pAppData pointer is unused by the SQLite core. The pointer is + available to store auxiliary information that a VFS information might + want to carry around. +

+ The remaining fields of the sqlite3_vfs object all store pointers + to functions that implement primitive operations. We call these + "methods". The first method, xOpen, is used to open files on + the underlying storage media. The result is an sqlite3_file + object. There are additional methods, defined by the sqlite3_file + object itself that are used to read and write and close the file. + The additional methods are detailed below. The filename is in UTF-8. + SQLite will guarantee that the zFilename string passed to + xOpen() is a full pathname as generated by xFullPathname() and + that the string will be valid and unchanged until xClose() is + called. So the sqlite3_file can store a pointer to the + filename if it needs to remember the filename for some reason. + The flags argument to xOpen() is a copy of the flags argument + to sqlite3_open_v2(). If sqlite3_open() or sqlite3_open16() + is used, then flags is SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE. + If xOpen() opens a file read-only then it sets *pOutFlags to + include SQLITE_OPEN_READONLY. Other bits in *pOutFlags may be + set. + SQLite will also add one of the following flags to the xOpen() + call, depending on the object being opened: +

+ The file I/O implementation can use the object type flags to + changes the way it deals with files. For example, an application + that does not care about crash recovery or rollback, might make + the open of a journal file a no-op. Writes to this journal are + also a no-op. Any attempt to read the journal returns SQLITE_IOERR. + Or the implementation might recognize the a database file will + be doing page-aligned sector reads and writes in a random order + and set up its I/O subsystem accordingly. + SQLite might also add one of the following flags to the xOpen + method: +

+ The SQLITE_OPEN_DELETEONCLOSE flag means the file should be + deleted when it is closed. This will always be set for TEMP + databases and journals and for subjournals. The + SQLITE_OPEN_EXCLUSIVE flag means the file should be opened + for exclusive access. This flag is set for all files except + for the main database file. + The sqlite3_file structure passed as the third argument to + xOpen is allocated by the caller. xOpen just fills it in. The + caller allocates a minimum of szOsFile bytes for the sqlite3_file + structure. +

+ The differences between an SQLITE_OPEN_TEMP_DB database and an + SQLITE_OPEN_TRANSIENT_DB database is this: The SQLITE_OPEN_TEMP_DB + is used for explicitly declared and named TEMP tables (using the + CREATE TEMP TABLE syntax) or for named tables in a temporary database + that is created by opening a database with a filename that is an empty + string. An SQLITE_OPEN_TRANSIENT_DB holds a database table that + SQLite creates automatically in order to evaluate a subquery or + ORDER BY or GROUP BY clause. Both TEMP_DB and TRANSIENT_DB databases + are private and are deleted automatically. TEMP_DB databases last + for the duration of the database connection. TRANSIENT_DB databases + last only for the duration of a single SQL statement. +

+ The xDelete method is used to delete a file. The name of the file is + given in the second parameter. The filename will be in UTF-8. + The VFS must convert the filename into whatever character representation + the underlying operating system expects. If the syncDir parameter is + true, then the xDelete method should not return until the change + to the directory contents for the directory containing the + deleted file have been synced to disk in order to ensure that the + file does not "reappear" if a power failure occurs soon after. +

+ The xAccess method is used to check for access permissions on a file. + The filename will be UTF-8 encoded. The flags argument will be + SQLITE_ACCESS_EXISTS to check for the existence of the file, + SQLITE_ACCESS_READWRITE to check to see if the file is both readable + and writable, or SQLITE_ACCESS_READ to check to see if the file is + at least readable. The "file" named by the second parameter might + be a directory or folder name. +

+ The xGetTempName method computes the name of a temporary file that + SQLite can use. The name should be written into the buffer given + by the second parameter. SQLite will size that buffer to hold + at least mxPathname bytes. The generated filename should be in UTF-8. + To avoid security problems, the generated temporary filename should + contain enough randomness to prevent an attacker from guessing the + temporary filename in advance. +

+ The xFullPathname method is used to convert a relative pathname + into a full pathname. The resulting full pathname is written into + the buffer provided by the third parameter. SQLite will size the + output buffer to at least mxPathname bytes. Both the input and + output names should be in UTF-8. +

+ The xDlOpen, xDlError, xDlSym, and xDlClose methods are all used for + accessing shared libraries at run-time. These methods may be omitted + (and their pointers set to zero) if the library is compiled with + SQLITE_OMIT_LOAD_EXTENSION or if the sqlite3_enable_load_extension() + interface is never used to enable dynamic extension loading. The + xDlOpen method opens a shared library or DLL and returns a pointer to + a handle. NULL is returned if the open fails. If the open fails, + the xDlError method can be used to obtain a text error message. + The message is written into the zErrMsg buffer of the third parameter + which is at least nByte bytes in length. The xDlSym returns a pointer + to a symbol in the shared library. The name of the symbol is given + by the second parameter. UTF-8 encoding is assumed. If the symbol + is not found a NULL pointer is returned. The xDlClose routine closes + the shared library. +

+ The xRandomness method is used exactly once to initialize the + pseudo-random number generator (PRNG) inside of SQLite. Only + the xRandomness method on the default VFS is used. The xRandomness + methods on other VFSes are never accessed by SQLite. + The xRandomness routine requests that nByte bytes of randomness + be written into zOut. The routine returns the actual number of + bytes of randomness obtained. The quality of the randomness so obtained + will determine the quality of the randomness generated by built-in + SQLite functions such as random() and randomblob(). SQLite also + uses its PRNG to generate temporary file names. On some platforms + (ex: Windows) SQLite assumes that temporary file names are unique + without actually testing for collisions, so it is important to have + good-quality randomness even if the random() and randomblob() + functions are never used. +

+ The xSleep method is used to suspend the calling thread for at + least the number of microseconds given. This method is used to + implement the sqlite3_sleep() and sqlite3_busy_timeout() APIs. + In the case of sqlite3_sleep() the xSleep method of the default + VFS is always used. If the underlying system does not have a + microsecond resolution sleep capability, then the sleep time should + be rounded up. xSleep returns this rounded-up value. +

+ The xCurrentTime method finds the current time and date and writes + the result as a double-precision floating point value into pointer + provided by the second parameter. The time and date is in + coordinated universal time (UTC) and is a fractional Julian day number. +

2.1.5 The Open File Object

+ The result of opening a file is an instance of an sqlite3_file object. + The sqlite3_file object is an abstract base class defined as follows: +

+typedef struct sqlite3_file sqlite3_file;
+struct sqlite3_file {
+  const struct sqlite3_io_methods *pMethods;
+};
+

+ Each VFS implementation will subclass the sqlite3_file by adding + additional fields at the end to hold whatever information the VFS + needs to know about an open file. It does not matter what information + is stored as long as the total size of the structure does not exceed + the szOsFile value recorded in the sqlite3_vfs object. +

+ The sqlite3_io_methods object is a structure that contains pointers + to methods for reading, writing, and otherwise dealing with files. + This object is defined as follows: +

+typedef struct sqlite3_io_methods sqlite3_io_methods;
+struct sqlite3_io_methods {
+  int iVersion;
+  int (*xClose)(sqlite3_file*);
+  int (*xRead)(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
+  int (*xWrite)(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);
+  int (*xTruncate)(sqlite3_file*, sqlite3_int64 size);
+  int (*xSync)(sqlite3_file*, int flags);
+  int (*xFileSize)(sqlite3_file*, sqlite3_int64 *pSize);
+  int (*xLock)(sqlite3_file*, int);
+  int (*xUnlock)(sqlite3_file*, int);
+  int (*xCheckReservedLock)(sqlite3_file*);
+  int (*xFileControl)(sqlite3_file*, int op, void *pArg);
+  int (*xSectorSize)(sqlite3_file*);
+  int (*xDeviceCharacteristics)(sqlite3_file*);
+  /* Additional methods may be added in future releases */
+};
+

+ The iVersion field of sqlite3_io_methods is provided as insurance + against future enhancements. The iVersion value should always be + 1 for SQLite version 3.5. +

+ The xClose method closes the file. The space for the sqlite3_file + structure is deallocated by the caller. But if the sqlite3_file + contains pointers to other allocated memory or resources, those + allocations should be released by the xClose method. +

+ The xRead method reads iAmt bytes from the file beginning at a byte + offset to iOfst. The data read is stored in the pointer of the + second parameter. xRead returns the SQLITE_OK on success, + SQLITE_IOERR_SHORT_READ if it was not able to read the full number + of bytes because it reached end-of-file, or SQLITE_IOERR_READ for + any other error. +

+ The xWrite method writes iAmt bytes of data from the second parameter + into the file beginning at an offset of iOfst bytes. If the size of + the file is less than iOfst bytes prior to the write, then xWrite should + ensure that the file is extended with zeros up to iOfst bytes prior + to beginning its write. xWrite continues to extends the file as + necessary so that the size of the file is at least iAmt+iOfst bytes + at the conclusion of the xWrite call. The xWrite method returns + SQLITE_OK on success. If the write cannot complete because the + underlying storage medium is full, then SQLITE_FULL is returned. + SQLITE_IOERR_WRITE should be returned for any other error. +

+ The xTruncate method truncates a file to be nByte bytes in length. + If the file is already nByte bytes or less in length then this + method is a no-op. The xTruncate method returns SQLITE_OK on + success and SQLITE_IOERR_TRUNCATE if anything goes wrong. +

+ The xSync method is used to force previously written data out of + operating system cache and into non-volatile memory. The second + parameter is usually SQLITE_SYNC_NORMAL. If the second parameter + is SQLITE_SYNC_FULL then the xSync method should make sure that + data has also been flushed through the disk controllers cache. + The SQLITE_SYNC_FULL parameter is the equivalent of the F_FULLSYNC + ioctl() on Mac OS X. The xSync method returns + SQLITE_OK on success and SQLITE_IOERR_FSYNC if anything goes wrong. +

+ The xFileSize() method determines the current size of the file + in bytes and writes that value into *pSize. It returns SQLITE_OK + on success and SQLITE_IOERR_FSTAT if something goes wrong. +

+ The xLock and xUnlock methods are used to set and clear file locks. + SQLite supports five levels of file locks, in order: +

+ The underlying implementation can support some subset of these locking + levels as long as it meets the other requirements of this paragraph. + The locking level is specified as the second argument to both xLock + and xUnlock. The xLock method increases the locking level to the + specified locking level or higher. The xUnlock method decreases the + locking level to no lower than the level specified. + SQLITE_LOCK_NONE means that the file is unlocked. SQLITE_LOCK_SHARED + gives permission to read the file. Multiple database connections can + hold SQLITE_LOCK_SHARED at the same time. + SQLITE_LOCK_RESERVED is like SQLITE_LOCK_SHARED in that it is permission + to read the file. But only a single connection can hold a reserved lock + at any point in time. The SQLITE_LOCK_PENDING is also permission to + read the file. Other connections can continue to read the file as well, + but no other connection is allowed to escalate a lock from none to shared. + SQLITE_LOCK_EXCLUSIVE is permission to write on the file. Only a single + connection can hold an exclusive lock and no other connection can hold + any lock (other than "none") while one connection holds an exclusive + lock. The xLock returns SQLITE_OK on success, SQLITE_BUSY if it + is unable to obtain the lock, or SQLITE_IOERR_RDLOCK if something else + goes wrong. The xUnlock method returns SQLITE_OK on success and + SQLITE_IOERR_UNLOCK for problems. +

+ The xCheckReservedLock() method checks to see if another connection or + another process is currently holding a reserved, pending, or exclusive + lock on the file. It returns true or false. +

+ The xFileControl() method is a generic interface that allows custom + VFS implementations to directly control an open file using the + (new and experimental) + sqlite3_file_control() interface. The second "op" argument + is an integer opcode. The third + argument is a generic pointer which is intended to be a pointer + to a structure that may contain arguments or space in which to + write return values. Potential uses for xFileControl() might be + functions to enable blocking locks with timeouts, to change the + locking strategy (for example to use dot-file locks), to inquire + about the status of a lock, or to break stale locks. The SQLite + core reserves opcodes less than 100 for its own use. + A list of opcodes less than 100 is available. + Applications that define a custom xFileControl method should use opcodes + greater than 100 to avoid conflicts. +

+ The xSectorSize returns the "sector size" of the underlying + non-volatile media. A "sector" is defined as the smallest unit of + storage that can be written without disturbing adjacent storage. + On a disk drive the "sector size" has until recently been 512 bytes, + though there is a push to increase this value to 4KiB. SQLite needs + to know the sector size so that it can write a full sector at a + time, and thus avoid corrupting adjacent storage space if a power + loss occurs in the middle of a write. +

+ The xDeviceCharacteristics method returns an integer bit vector that + defines any special properties that the underlying storage medium might + have that SQLite can use to increase performance. The allowed return + is the bit-wise OR of the following values: +

+ The SQLITE_IOCAP_ATOMIC bit means that all writes to this device are + atomic in the sense that either the entire write occurs or none of it + occurs. The other + SQLITE_IOCAP_ATOMICnnn values indicate that + writes of aligned blocks of the indicated size are atomic. + SQLITE_IOCAP_SAFE_APPEND means that when extending a file with new + data, the new data is written first and then the file size is updated. + So if a power failure occurs, there is no chance that the file might have + been extended with randomness. The SQLITE_IOCAP_SEQUENTIAL bit means + that all writes occur in the order that they are issued and are not + reordered by the underlying file system. +

2.1.6 Checklist For Constructing A New VFS

+ The preceding paragraphs contain a lot of information. + To ease the task of constructing + a new VFS for SQLite we offer the following implementation checklist: +

Define an appropriate subclass of the sqlite3_file object. +
Implement the methods required by the sqlite3_io_methods object. +
Create a static and + constant sqlite3_io_methods object containing pointers + to the methods from the previous step. +
Implement the xOpen method that opens a file and populates an + sqlite3_file object, including setting pMethods to + point to the sqlite3_io_methods object from the previous step. +
Implement the other methods required by sqlite3_vfs. +
Define a static (but not constant) sqlite3_vfs structure that + contains pointers to the xOpen method and the other methods and + which contains the appropriate values for iVersion, szOsFile, + mxPathname, zName, and pAppData. +
Implement a procedure that calls sqlite3_vfs_register() and + passes it a pointer to the sqlite3_vfs structure from the previous + step. This procedure is probably the only exported symbol in the + source file that implements your VFS. +

+ Within your application, call the procedure implemented in the last + step above as part of your initialization process before any + database connections are opened. +

3.0 The Memory Allocation Subsystem

+ Beginning with version 3.5, SQLite obtains all of the heap memory it + needs using the routines sqlite3_malloc(), sqlite3_free(), and + sqlite3_realloc(). These routines have existed in prior versions + of SQLite, but SQLite has previously bypassed these routines and used + its own memory allocator. This all changes in version 3.5.0. +

+ The SQLite source tree actually contains multiple versions of the + memory allocator. The default high-speed version found in the + "mem1.c" source file is used for most builds. But if the SQLITE_MEMDEBUG + flag is enabled, a separate memory allocator the "mem2.c" source file + is used instead. The mem2.c allocator implements lots of hooks to + do error checking and to simulate memory allocation failures for testing + purposes. Both of these allocators use the malloc()/free() implementation + in the standard C library. +

+ Applications are not required to use either of these standard memory + allocators. If SQLite is compiled with SQLITE_OMIT_MEMORY_ALLOCATION + then no implementation for the sqlite3_malloc(), sqlite3_realloc(), + and sqlite3_free() functions is provided. Instead, the application + that links against SQLite must provide its own implementation of these + functions. The application provided memory allocator is not required + to use the malloc()/free() implementation in the standard C library. + An embedded application might provide an alternative memory allocator + that uses memory for a fixed memory pool set aside for the exclusive + use of SQLite, for example. +

+ Applications that implement their own memory allocator must provide + implementation for the usual three allocation functions + sqlite3_malloc(), sqlite3_realloc(), and sqlite3_free(). + And they must also implement a fourth function: +

+int sqlite3_memory_alarm(
+  void(*xCallback)(void *pArg, sqlite3_int64 used, int N),
+  void *pArg,
+  sqlite3_int64 iThreshold
+);
+

+ The sqlite3_memory_alarm routine is used to register + a callback on memory allocation events. + This routine registers or clears a callback that fires when + the amount of memory allocated exceeds iThreshold. Only + a single callback can be registered at a time. Each call + to sqlite3_memory_alarm() overwrites the previous callback. + The callback is disabled by setting xCallback to a NULL + pointer. +

+ The parameters to the callback are the pArg value, the + amount of memory currently in use, and the size of the + allocation that provoked the callback. The callback will + presumably invoke sqlite3_free() to free up memory space. + The callback may invoke sqlite3_malloc() or sqlite3_realloc() + but if it does, no additional callbacks will be invoked by + the recursive calls. +

+ The sqlite3_soft_heap_limit() interface works by registering + a memory alarm at the soft heap limit and invoking + sqlite3_release_memory() in the alarm callback. Application + programs should not attempt to use the sqlite3_memory_alarm() + interface because doing so will interfere with the + sqlite3_soft_heap_limit() module. This interface is exposed + only so that applications can provide their own + alternative implementation when the SQLite core is + compiled with SQLITE_OMIT_MEMORY_ALLOCATION. +

+ The built-in memory allocators in SQLite also provide the following + additional interfaces: +

+sqlite3_int64 sqlite3_memory_used(void);
+sqlite3_int64 sqlite3_memory_highwater(int resetFlag);
+

+ These interfaces can be used by an application to monitor how + much memory SQLite is using. The sqlite3_memory_used() routine + returns the number of bytes of memory currently in use and the + sqlite3_memory_highwater() returns the maximum instantaneous + memory usage. Neither routine includes the overhead associated + with the memory allocator. These routines are provided for use + by the application. SQLite never invokes them itself. So if + the application is providing its own memory allocation subsystem, + it can omit these interfaces if desired. +

4.0 The Mutex Subsystem

+ SQLite has always been threadsafe in the sense that it is safe to + use different SQLite database connections in different threads at the + same time. The constraint was that the same database connection + could not be used in two separate threads at once. SQLite version 3.5.0 + relaxes this constraint. +

+ In order to allow multiple threads to use the same database connection + at the same time, SQLite must make extensive use of mutexes. And for + this reason a new mutex subsystem as been added. The mutex subsystem + as the following interface: +

+sqlite3_mutex *sqlite3_mutex_alloc(int);
+void sqlite3_mutex_free(sqlite3_mutex*);
+void sqlite3_mutex_enter(sqlite3_mutex*);
+int sqlite3_mutex_try(sqlite3_mutex*);
+void sqlite3_mutex_leave(sqlite3_mutex*);
+

+ Though these routines exist for the use of the SQLite core, + application code is free to use these routines as well, if desired. + A mutex is an sqlite3_mutex object. The sqlite3_mutex_alloc() + routine allocates a new mutex object and returns a pointer to it. + The argument to sqlite3_mutex_alloc() should be + SQLITE_MUTEX_FAST or SQLITE_MUTEX_RECURSIVE for non-recursive + and recursive mutexes, respectively. If the underlying system does + not provide non-recursive mutexes, then a recursive mutex can be + substituted in that case. The argument to sqlite3_mutex_alloc() + can also be a constant designating one of several static mutexes: +

+ These static mutexes are reserved for use internally by SQLite + and should not be used by the application. The static mutexes + are all non-recursive. +

+ The sqlite3_mutex_free() routine should be used to deallocate + a non-static mutex. If a static mutex is passed to this routine + then the behavior is undefined. +

+ The sqlite3_mutex_enter() attempts to enter the mutex and blocks + if another threads is already there. sqlite3_mutex_try() attempts + to enter and returns SQLITE_OK on success or SQLITE_BUSY if another + thread is already there. sqlite3_mutex_leave() exits a mutex. + The mutex is held until the number of exits matches the number of + entrances. If sqlite3_mutex_leave() is called on a mutex that + the thread is not currently holding, then the behavior is undefined. + If any routine is called for a deallocated mutex, then the behavior + is undefined. +

+ The SQLite source code provides multiple implementations of these + APIs, suitable for varying environments. If SQLite is compiled with + the SQLITE_THREADSAFE=0 flag then a no-op mutex implementation that + is fast but does no real mutual exclusion is provided. That + implementation is suitable for use in single-threaded applications + or applications that only use SQLite in a single thread. Other + real mutex implementations are provided based on the underlying + operating system. +

+ Embedded applications may wish to provide their own mutex implementation. + If SQLite is compiled with the -DSQLITE_MUTEX_APPDEF=1 compile-time flag + then the SQLite core provides no mutex subsystem and a mutex subsystem + that matches the interface described above must be provided by the + application that links against SQLite. +

5.0 Other Interface Changes

+ Version 3.5.0 of SQLite changes the behavior of a few APIs in ways + that are technically incompatible. However, these APIs are seldom + used and even when they are used it is difficult to imagine a + scenario where the change might break something. The changes + actually makes these interface much more useful and powerful. +

+ Prior to version 3.5.0, the sqlite3_enable_shared_cache() API + would enable and disable the shared cache feature for all connections + within a single thread - the same thread from which the + sqlite3_enable_shared_cache() routine was called. Database connections + that used the shared cache were restricted to running in the same + thread in which they were opened. Beginning with version 3.5.0, + the sqlite3_enable_shared_cache() applies to all database connections + in all threads within the process. Now database connections running + in separate threads can share a cache. And database connections that + use shared cache can migrate from one thread to another. +

+ Prior to version 3.5.0 the sqlite3_soft_heap_limit() set an upper + bound on heap memory usage for all database connections within a + single thread. Each thread could have its own heap limit. Beginning + in version 3.5.0, there is a single heap limit for the entire process. + This seems more restrictive (one limit as opposed to many) but in + practice it is what most users want. +

+ Prior to version 3.5.0 the sqlite3_release_memory() function would + try to reclaim memory from all database connections in the same thread + as the sqlite3_release_memory() call. Beginning with version 3.5.0, + the sqlite3_release_memory() function will attempt to reclaim memory + from all database connections in all threads. +

6.0 Summary

+ The transition from SQLite version 3.4.2 to 3.5.0 is a major change. + Every source code file in the SQLite core had to be modified, some + extensively. And the change introduced some minor incompatibilities + in the C interface. But we feel that the benefits of the transition + from 3.4.2 to 3.5.0 far outweigh the pain of porting. The new + VFS layer is now well-defined and stable and should simplify future + customizations. The VFS layer, and the separable memory allocator + and mutex subsystems allow a standard SQLite source code amalgamation + to be used in an embedded project without change, greatly simplifying + configuration management. And the resulting system is much more + tolerant of highly threaded designs. +

+ +

This page last modified on 2020-06-18 21:18:56 UTC

+ -- cgit v1.2.3