1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
|
<!DOCTYPE html>
<html><head>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<link href="sqlite.css" rel="stylesheet">
<title>The Checksum VFS Shim</title>
<!-- path= -->
</head>
<body>
<div class=nosearch>
<a href="index.html">
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite" border="0">
</a>
<div><!-- IE hack to prevent disappearing logo --></div>
<div class="tagline desktoponly">
Small. Fast. Reliable.<br>Choose any three.
</div>
<div class="menu mainmenu">
<ul>
<li><a href="index.html">Home</a>
<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a>
<li class='wideonly'><a href='about.html'>About</a>
<li class='desktoponly'><a href="docs.html">Documentation</a>
<li class='desktoponly'><a href="download.html">Download</a>
<li class='wideonly'><a href='copyright.html'>License</a>
<li class='desktoponly'><a href="support.html">Support</a>
<li class='desktoponly'><a href="prosupport.html">Purchase</a>
<li class='search' id='search_menubutton'>
<a href="javascript:void(0)" onclick='toggle_search()'>Search</a>
</ul>
</div>
<div class="menu submenu" id="submenu">
<ul>
<li><a href='about.html'>About</a>
<li><a href='docs.html'>Documentation</a>
<li><a href='download.html'>Download</a>
<li><a href='support.html'>Support</a>
<li><a href='prosupport.html'>Purchase</a>
</ul>
</div>
<div class="searchmenu" id="searchmenu">
<form method="GET" action="search">
<select name="s" id="searchtype">
<option value="d">Search Documentation</option>
<option value="c">Search Changelog</option>
</select>
<input type="text" name="q" id="searchbox" value="">
<input type="submit" value="Go">
</form>
</div>
</div>
<script>
function toggle_div(nm) {
var w = document.getElementById(nm);
if( w.style.display=="block" ){
w.style.display = "none";
}else{
w.style.display = "block";
}
}
function toggle_search() {
var w = document.getElementById("searchmenu");
if( w.style.display=="block" ){
w.style.display = "none";
} else {
w.style.display = "block";
setTimeout(function(){
document.getElementById("searchbox").focus()
}, 30);
}
}
function div_off(nm){document.getElementById(nm).style.display="none";}
window.onbeforeunload = function(e){div_off("submenu");}
/* Disable the Search feature if we are not operating from CGI, since */
/* Search is accomplished using CGI and will not work without it. */
if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){
document.getElementById("search_menubutton").style.display = "none";
}
/* Used by the Hide/Show button beside syntax diagrams, to toggle the */
function hideorshow(btn,obj){
var x = document.getElementById(obj);
var b = document.getElementById(btn);
if( x.style.display!='none' ){
x.style.display = 'none';
b.innerHTML='show';
}else{
x.style.display = '';
b.innerHTML='hide';
}
return false;
}
var antiRobot = 0;
function antiRobotGo(){
if( antiRobot!=3 ) return;
antiRobot = 7;
var j = document.getElementById("mtimelink");
if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href");
}
function antiRobotDefense(){
document.body.onmousedown=function(){
antiRobot |= 2;
antiRobotGo();
document.body.onmousedown=null;
}
document.body.onmousemove=function(){
antiRobot |= 2;
antiRobotGo();
document.body.onmousemove=null;
}
setTimeout(function(){
antiRobot |= 1;
antiRobotGo();
}, 100)
antiRobotGo();
}
antiRobotDefense();
</script>
<div class=fancy>
<div class=nosearch>
<div class="fancy_title">
The Checksum VFS Shim
</div>
<div class="fancy_toc">
<a onclick="toggle_toc()">
<span class="fancy_toc_mark" id="toc_mk">►</span>
Table Of Contents
</a>
<div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div>
<div class="fancy-toc1"><a href="#compiling">2. Compiling</a></div>
<div class="fancy-toc1"><a href="#loading">3. Loading</a></div>
<div class="fancy-toc1"><a href="#usage">4. Usage</a></div>
<div class="fancy-toc1"><a href="#verification_of_checksums">5. Verification Of Checksums</a></div>
<div class="fancy-toc1"><a href="#controlling_checksum_verification">6. Controlling Checksum Verification</a></div>
</div>
</div>
<script>
function toggle_toc(){
var sub = document.getElementById("toc_sub")
var mk = document.getElementById("toc_mk")
if( sub.style.display!="block" ){
sub.style.display = "block";
mk.innerHTML = "▼";
} else {
sub.style.display = "none";
mk.innerHTML = "►";
}
}
</script>
</div>
<h1 id="overview"><span>1. </span>Overview</h1>
<p>The checksum VFS extension is a <a href="vfs.html#shim">VFS shim</a> that adds an 8-byte
checksum to the end of every page in an SQLite database. The checksum
is added as each page is written and verified as each page is read.
The checksum is intended to help detect database corruption caused by
random bit-flips in the mass storage device.
</p><p>The checksum VFS extension requires
SQLite version 3.32.0 (2020-05-22) or later. It will not
work with earlier versions of SQLite.
</p><h1 id="compiling"><span>2. </span>Compiling</h1>
<p>The checksum VFS module is a <a href="loadext.html">loadable extension</a>. It is not
included in the <a href="amalgamation.html">amalgamation</a>. It must be added to SQLite
either at compile-time or at run-time. The source code to
the checksum VFS module is in the
<a href="https://sqlite.org/src/file/ext/misc/cksumvfs.c">ext/misc/cksumvfs.c</a>
source file in the
<a href="https://sqlite.org/src">SQLite source tree</a>.
</p><p>To build the checksum VFS module into a run-time loadable
extension, use commands similar to the following:
</p><ul>
<li>(linux) → <tt>gcc -fPIC -shared cksumvfs.c -o cksumvfs.so</tt>
</li><li>(mac) → <tt>clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib</tt>
</li><li>(windows) → <tt>cl cksumvfs.c -link -dll -out:cksumvfs.dll</tt>
</li></ul>
<p>You may want to add additional compiler options, of course,
according to the needs of your project.
</p><p>To statically link this extension with your product,
compile it like any other C-language module but add the
"-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that
it is being statically linked rather than dynamically linked.
</p><h1 id="loading"><span>3. </span>Loading</h1>
<p>To load this extension as a shared library, you first have to
bring up a dummy SQLite database connection to use as the argument
to the <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API call. Then you invoke the
<a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API and shutdown the dummy database
connection. All subsequent database connections that are opened
will include this extension. For example:
</p><div class="codeblock"><pre>sqlite3 *db;
sqlite3_open(":memory:", &db);
sqlite3_load_extension(db, "./cksumvfs");
sqlite3_close(db);
</pre></div>
<p>If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and
statically linked against the application, initialize it using
a single API call as follows:
</p><div class="codeblock"><pre>sqlite3_cksumvfs_init();
</pre></div>
<p>Cksumvfs is a <a href="vfs.html#shim">VFS shim</a>. When loaded, "cksmvfs" becomes the new
default VFS, and it uses the prior default VFS as the next VFS
down in the stack. This is normally what you want. However, in
complex situations where multiple VFS shims are being loaded,
it might be important to ensure that cksumvfs is loaded in the
correct order so that it sequences itself into the default VFS
Shim stack in the right order.
</p><h1 id="usage"><span>4. </span>Usage</h1>
<p>Open database connections using the <a href="c3ref/open.html">sqlite3_open()</a> or
<a href="c3ref/open.html">sqlite3_open_v2()</a> interfaces, as normal. Ordinary database files
(without a checksum) will operate normally. Databases with
checksums will return an SQLITE_IOERR_DATA error if a page is
encountered that contains an invalid checksum.
</p><p>Checksumming only works on databases that have a <a href="fileformat2.html#resbyte">reserve bytes</a>
value of exactly 8. The default value for reserve-bytes is 0.
Hence, newly created database files will omit the checksum by
default. To create a database that includes a checksum, change
the reserve-bytes value to 8 by running code similar to this:
</p><div class="codeblock"><pre>int n = 8;
sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVE_BYTES, &n);
</pre></div>
<p>If you do this immediately after creating a new database file,
before anything else has been written into the file, then that
might be all that you need to do. Otherwise, the API call
above should be followed by:
</p><div class="codeblock"><pre>sqlite3_exec(db, "VACUUM", 0, 0, 0);
</pre></div>
<p>It never hurts to run the VACUUM, even if you don't need it.
If the database is in WAL mode, you should shutdown and
reopen all database connections before continuing.
</p><p>From the CLI, use the ".filectrl reserve_bytes 8" command,
followed by "VACUUM;".
</p><p>Note that SQLite allows the number of reserve-bytes to be
increased but not decreased. So if a database file already
has a reserve-bytes value greater than 8, there is no way to
activate checksumming on that database, other than to dump
and restore the database file. Note also that other extensions
might also make use of the reserve-bytes. Checksumming will
be incompatible with those other extensions.
</p><h1 id="verification_of_checksums"><span>5. </span>Verification Of Checksums</h1>
<p>If any checksum is incorrect, the "PRAGMA quick_check" command
will find it. To verify that checksums are actually enabled
and running, use SQL like the following:
</p><div class="codeblock"><pre>SELECT count(*), verify_checksum(data)
FROM sqlite_dbpage
GROUP BY 2;
</pre></div>
<p>There are three possible outputs from the verify_checksum()
function: 1, 0, and NULL. 1 is returned if the checksum is
correct. 0 is returned if the checksum is incorrect. NULL
is returned if the page is unreadable. If checksumming is
enabled, the read will fail if the checksum is wrong, so the
usual result from verify_checksum() on a bad checksum is NULL.
</p><p>If everything is OK, the query above should return a single
row where the second column is 1. Any other result indicates
either that there is a checksum error, or checksum validation
is disabled.
</p><h1 id="controlling_checksum_verification"><span>6. </span>Controlling Checksum Verification</h1>
<p>The cksumvfs extension implements a new PRAGMA statement that can
be used to disable, re-enable, or query the status of checksum
verification:
</p><div class="codeblock"><pre>PRAGMA checksum_verification; -- query status
PRAGMA checksum_verification=OFF; -- disable verification
PRAGMA checksum_verification=ON; -- re-enable verification
</pre></div>
<p>The "checksum_verification" pragma will return "1" (true) or "0"
(false) if checksum verification is enabled or disabled, respectively.
"Verification" in this context means the feature that causes
SQLITE_IOERR_DATA errors if a checksum mismatch is detected while
reading. Checksums are always kept up-to-date as long as the
<a href="fileformat2.html#resbyte">reserve bytes</a> value of the database is 8, regardless of the setting
of this pragma. Checksum verification can be disabled (for example)
to do forensic analysis of a database that has previously reported
a checksum error.
</p><p>The "checksum_verification" pragma will always respond with "0" if
the database file does not have a <a href="fileformat2.html#resbyte">reserve bytes</a> value of 8. The
pragma will return no rows at all if the cksumvfs extension is
not loaded.
</p><p align="center"><small><i>This page last modified on <a href="https://sqlite.org/docsrc/honeypot" id="mtimelink" data-href="https://sqlite.org/docsrc/finfo/pages/cksumvfs.in?m=c9b2514ff0">2022-01-08 05:02:57</a> UTC </small></i></p>
|
