diff options
Diffstat (limited to '')
| -rw-r--r-- | www/cksumvfs.html | 314 |
1 files changed, 314 insertions, 0 deletions
diff --git a/www/cksumvfs.html b/www/cksumvfs.html new file mode 100644 index 0000000..2d73ce8 --- /dev/null +++ b/www/cksumvfs.html @@ -0,0 +1,314 @@ +<!DOCTYPE html> +<html><head> +<meta name="viewport" content="width=device-width, initial-scale=1.0"> +<meta http-equiv="content-type" content="text/html; charset=UTF-8"> +<link href="sqlite.css" rel="stylesheet"> +<title>The Checksum VFS Shim</title> +<!-- path= --> +</head> +<body> +<div class=nosearch> +<a href="index.html"> +<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite" border="0"> +</a> +<div><!-- IE hack to prevent disappearing logo --></div> +<div class="tagline desktoponly"> +Small. Fast. Reliable.<br>Choose any three. +</div> +<div class="menu mainmenu"> +<ul> +<li><a href="index.html">Home</a> +<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a> +<li class='wideonly'><a href='about.html'>About</a> +<li class='desktoponly'><a href="docs.html">Documentation</a> +<li class='desktoponly'><a href="download.html">Download</a> +<li class='wideonly'><a href='copyright.html'>License</a> +<li class='desktoponly'><a href="support.html">Support</a> +<li class='desktoponly'><a href="prosupport.html">Purchase</a> +<li class='search' id='search_menubutton'> +<a href="javascript:void(0)" onclick='toggle_search()'>Search</a> +</ul> +</div> +<div class="menu submenu" id="submenu"> +<ul> +<li><a href='about.html'>About</a> +<li><a href='docs.html'>Documentation</a> +<li><a href='download.html'>Download</a> +<li><a href='support.html'>Support</a> +<li><a href='prosupport.html'>Purchase</a> +</ul> +</div> +<div class="searchmenu" id="searchmenu"> +<form method="GET" action="search"> +<select name="s" id="searchtype"> +<option value="d">Search Documentation</option> +<option value="c">Search Changelog</option> +</select> +<input type="text" name="q" id="searchbox" value=""> +<input type="submit" value="Go"> +</form> +</div> +</div> +<script> +function toggle_div(nm) { +var w = document.getElementById(nm); +if( w.style.display=="block" ){ +w.style.display = "none"; +}else{ +w.style.display = "block"; +} +} +function toggle_search() { +var w = document.getElementById("searchmenu"); +if( w.style.display=="block" ){ +w.style.display = "none"; +} else { +w.style.display = "block"; +setTimeout(function(){ +document.getElementById("searchbox").focus() +}, 30); +} +} +function div_off(nm){document.getElementById(nm).style.display="none";} +window.onbeforeunload = function(e){div_off("submenu");} +/* Disable the Search feature if we are not operating from CGI, since */ +/* Search is accomplished using CGI and will not work without it. */ +if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){ +document.getElementById("search_menubutton").style.display = "none"; +} +/* Used by the Hide/Show button beside syntax diagrams, to toggle the */ +function hideorshow(btn,obj){ +var x = document.getElementById(obj); +var b = document.getElementById(btn); +if( x.style.display!='none' ){ +x.style.display = 'none'; +b.innerHTML='show'; +}else{ +x.style.display = ''; +b.innerHTML='hide'; +} +return false; +} +var antiRobot = 0; +function antiRobotGo(){ +if( antiRobot!=3 ) return; +antiRobot = 7; +var j = document.getElementById("mtimelink"); +if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href"); +} +function antiRobotDefense(){ +document.body.onmousedown=function(){ +antiRobot |= 2; +antiRobotGo(); +document.body.onmousedown=null; +} +document.body.onmousemove=function(){ +antiRobot |= 2; +antiRobotGo(); +document.body.onmousemove=null; +} +setTimeout(function(){ +antiRobot |= 1; +antiRobotGo(); +}, 100) +antiRobotGo(); +} +antiRobotDefense(); +</script> +<div class=fancy> +<div class=nosearch> +<div class="fancy_title"> +The Checksum VFS Shim +</div> +<div class="fancy_toc"> +<a onclick="toggle_toc()"> +<span class="fancy_toc_mark" id="toc_mk">►</span> +Table Of Contents +</a> +<div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div> +<div class="fancy-toc1"><a href="#compiling">2. Compiling</a></div> +<div class="fancy-toc1"><a href="#loading">3. Loading</a></div> +<div class="fancy-toc1"><a href="#usage">4. Usage</a></div> +<div class="fancy-toc1"><a href="#verification_of_checksums">5. Verification Of Checksums</a></div> +<div class="fancy-toc1"><a href="#controlling_checksum_verification">6. Controlling Checksum Verification</a></div> +</div> +</div> +<script> +function toggle_toc(){ +var sub = document.getElementById("toc_sub") +var mk = document.getElementById("toc_mk") +if( sub.style.display!="block" ){ +sub.style.display = "block"; +mk.innerHTML = "▼"; +} else { +sub.style.display = "none"; +mk.innerHTML = "►"; +} +} +</script> +</div> + + + + + +<h1 id="overview"><span>1. </span>Overview</h1> + +<p>The checksum VFS extension is a <a href="vfs.html#shim">VFS shim</a> that adds an 8-byte +checksum to the end of every page in an SQLite database. The checksum +is added as each page is written and verified as each page is read. +The checksum is intended to help detect database corruption caused by +random bit-flips in the mass storage device. + +</p><p>The checksum VFS extension requires +SQLite version 3.32.0 (2020-05-22) or later. It will not +work with earlier versions of SQLite. + +</p><h1 id="compiling"><span>2. </span>Compiling</h1> + +<p>The checksum VFS module is a <a href="loadext.html">loadable extension</a>. It is not +included in the <a href="amalgamation.html">amalgamation</a>. It must be added to SQLite +either at compile-time or at run-time. The source code to +the checksum VFS module is in the +<a href="https://sqlite.org/src/file/ext/misc/cksumvfs.c">ext/misc/cksumvfs.c</a> +source file in the +<a href="https://sqlite.org/src">SQLite source tree</a>. + +</p><p>To build the checksum VFS module into a run-time loadable +extension, use commands similar to the following: + +</p><ul> +<li>(linux) → <tt>gcc -fPIC -shared cksumvfs.c -o cksumvfs.so</tt> +</li><li>(mac) → <tt>clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib</tt> +</li><li>(windows) → <tt>cl cksumvfs.c -link -dll -out:cksumvfs.dll</tt> +</li></ul> + +<p>You may want to add additional compiler options, of course, +according to the needs of your project. + +</p><p>To statically link this extension with your product, +compile it like any other C-language module but add the +"-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that +it is being statically linked rather than dynamically linked. + +</p><h1 id="loading"><span>3. </span>Loading</h1> + +<p>To load this extension as a shared library, you first have to +bring up a dummy SQLite database connection to use as the argument +to the <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API call. Then you invoke the +<a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API and shutdown the dummy database +connection. All subsequent database connections that are opened +will include this extension. For example: + +</p><div class="codeblock"><pre>sqlite3 *db; +sqlite3_open(":memory:", &db); +sqlite3_load_extension(db, "./cksumvfs"); +sqlite3_close(db); +</pre></div> + +<p>If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and +statically linked against the application, initialize it using +a single API call as follows: + +</p><div class="codeblock"><pre>sqlite3_cksumvfs_init(); +</pre></div> + +<p>Cksumvfs is a <a href="vfs.html#shim">VFS shim</a>. When loaded, "cksmvfs" becomes the new +default VFS, and it uses the prior default VFS as the next VFS +down in the stack. This is normally what you want. However, in +complex situations where multiple VFS shims are being loaded, +it might be important to ensure that cksumvfs is loaded in the +correct order so that it sequences itself into the default VFS +Shim stack in the right order. + +</p><h1 id="usage"><span>4. </span>Usage</h1> + +<p>Open database connections using the <a href="c3ref/open.html">sqlite3_open()</a> or +<a href="c3ref/open.html">sqlite3_open_v2()</a> interfaces, as normal. Ordinary database files +(without a checksum) will operate normally. Databases with +checksums will return an SQLITE_IOERR_DATA error if a page is +encountered that contains an invalid checksum. + +</p><p>Checksumming only works on databases that have a <a href="fileformat2.html#resbyte">reserve bytes</a> +value of exactly 8. The default value for reserve-bytes is 0. +Hence, newly created database files will omit the checksum by +default. To create a database that includes a checksum, change +the reserve-bytes value to 8 by running code similar to this: + +</p><div class="codeblock"><pre>int n = 8; +sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVE_BYTES, &n); +</pre></div> + +<p>If you do this immediately after creating a new database file, +before anything else has been written into the file, then that +might be all that you need to do. Otherwise, the API call +above should be followed by: + +</p><div class="codeblock"><pre>sqlite3_exec(db, "VACUUM", 0, 0, 0); +</pre></div> + +<p>It never hurts to run the VACUUM, even if you don't need it. +If the database is in WAL mode, you should shutdown and +reopen all database connections before continuing. + +</p><p>From the CLI, use the ".filectrl reserve_bytes 8" command, +followed by "VACUUM;". + +</p><p>Note that SQLite allows the number of reserve-bytes to be +increased but not decreased. So if a database file already +has a reserve-bytes value greater than 8, there is no way to +activate checksumming on that database, other than to dump +and restore the database file. Note also that other extensions +might also make use of the reserve-bytes. Checksumming will +be incompatible with those other extensions. + +</p><h1 id="verification_of_checksums"><span>5. </span>Verification Of Checksums</h1> + +<p>If any checksum is incorrect, the "PRAGMA quick_check" command +will find it. To verify that checksums are actually enabled +and running, use SQL like the following: + +</p><div class="codeblock"><pre>SELECT count(*), verify_checksum(data) + FROM sqlite_dbpage + GROUP BY 2; +</pre></div> + +<p>There are three possible outputs from the verify_checksum() +function: 1, 0, and NULL. 1 is returned if the checksum is +correct. 0 is returned if the checksum is incorrect. NULL +is returned if the page is unreadable. If checksumming is +enabled, the read will fail if the checksum is wrong, so the +usual result from verify_checksum() on a bad checksum is NULL. + +</p><p>If everything is OK, the query above should return a single +row where the second column is 1. Any other result indicates +either that there is a checksum error, or checksum validation +is disabled. + +</p><h1 id="controlling_checksum_verification"><span>6. </span>Controlling Checksum Verification</h1> + +<p>The cksumvfs extension implements a new PRAGMA statement that can +be used to disable, re-enable, or query the status of checksum +verification: + +</p><div class="codeblock"><pre>PRAGMA checksum_verification; -- query status +PRAGMA checksum_verification=OFF; -- disable verification +PRAGMA checksum_verification=ON; -- re-enable verification +</pre></div> + +<p>The "checksum_verification" pragma will return "1" (true) or "0" +(false) if checksum verification is enabled or disabled, respectively. +"Verification" in this context means the feature that causes +SQLITE_IOERR_DATA errors if a checksum mismatch is detected while +reading. Checksums are always kept up-to-date as long as the +<a href="fileformat2.html#resbyte">reserve bytes</a> value of the database is 8, regardless of the setting +of this pragma. Checksum verification can be disabled (for example) +to do forensic analysis of a database that has previously reported +a checksum error. + +</p><p>The "checksum_verification" pragma will always respond with "0" if +the database file does not have a <a href="fileformat2.html#resbyte">reserve bytes</a> value of 8. The +pragma will return no rows at all if the cksumvfs extension is +not loaded. +</p><p align="center"><small><i>This page last modified on <a href="https://sqlite.org/docsrc/honeypot" id="mtimelink" data-href="https://sqlite.org/docsrc/finfo/pages/cksumvfs.in?m=c9b2514ff040a5792">2020-12-10 10:53:35</a> UTC </small></i></p> + |