1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
|
<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>
<doc>
<title>Suggested Books</title>
<!-- ************************************************************************* -->
<body>
<p>
One of the major goals of dlib is to have documentation that enables
someone to easily make use of its various components. Ideally,
you would read a short description of something, understand it immediately,
and begin using it in your application without any difficulty. Obviously, this
depends partly on the background of the user. For example, if you have
never written C++ software before then it probably isn't going to be this easy.
</p>
<p>
This page is meant to complement the main library documentation by providing
references to books, along with my commentary, which explain most of
the background material needed to understand the various parts of the library.
In most cases these are the books I learned from during the process
of creating dlib. As always, if you disagree with anything or think I have left out
an important text then shoot me an <a href="mailto:davis@dlib.net">email</a>.
</p>
<br/><br/>
<h2>General Programming</h2>
<ul>
<h3>C++</h3>
<ul>
<li> <i>Programming: Principles and Practice Using C++</i> by Bjarne Stroustrup
<ul> This is the sort of book you would use in a freshman introduction-to-programming class.
So if you are just beginning to study programming and are interested in C++ then I think
it is probably safe to say this is one of the best books you could read. </ul> <br/>
</li>
<li> <i>Accelerated C++: Practical Programming by Example</i> by Andrew Koenig and Barbara E. Moo
<ul> If you are new to C++ but already know how to program then this is a great book. It's also
about one fourth the size of the Stroustrup book. </ul> <br/>
</li>
<li> <i>Effective C++: 55 Specific Ways to Improve Your Programs and Designs</i> (3rd Edition) by Scott Meyers
<ul> This is a great intermediate level C++ book. Most people have heard the jokes about
how easy it is to shoot yourself in the foot with C++. This book explains many things you
need to know about the language to avoid doing so on a regular basis. So if you are
writing C++ software then this is a must-read. I would even claim that
you are a danger to the C++ software you touch unless you know what is in this book.
I'm not kidding. Finally, the book isn't just about the quirks of C++. It also discusses many general
software engineering ideas which have wide applicability. So in this
respect it is a great book for any software developer to read.
</ul><br/>
</li>
<li> <i>More Effective C++: 35 New Ways to Improve Your Programs and Designs</i> by Scott Meyers
<ul> Consider this an expansion to Effective C++. If you are going to read the above
book then you would almost certainly benefit from reading this one as well.
</ul><br/>
</li>
<li> <i>The C++ Standard Library: A Tutorial and Reference</i> by Nicolai M. Josuttis
<ul> If you are going to buy a reference book on the C++ standard library then this
is the one to get. I think you
will find it is better than any of the available online references. So if you find
yourself frustrated with the online resources, then this is the book for you.
</ul><br/>
</li>
<li> <a href="http://www.cplusplus.com/reference/">Online C++ Standard Library Reference</a>
<ul> What I said aside, this is a good online reference. I often find myself referring to it
when I do not have the Josuttis book on hand.
</ul><br/>
</li>
</ul>
<h3>Multithreading</h3>
<ul>
<li> <i>Programming with POSIX Threads</i> by David R. Butenhof
<ul> When I was an undergrad, this book was my main resource for learning about multithreading.
It was enjoyable to read, as are all the books on this list, and covered everything
in great depth without becoming overbearing. Also, despite what the title may suggest,
this book is useful for understanding multithreading broadly, not just multithreading
on POSIX systems.
</ul><br/>
</li>
</ul>
<h3>Network Programming</h3>
<ul>
<li> <i>Unix Network Programming, Volume 1: The Sockets Networking API</i> (3rd Edition)
by W. Richard Stevens
<ul> A lot of people call this book the network programming Bible and
this praise is well deserved. If you want a deep understanding of how computer networks
function, including the Internet, then this is the book to read. As with
the Butenhof book above, this is an excellent choice even for people who do not
intend to write software for Unix systems.
</ul><br/>
</li>
</ul>
<h3>WIN32 Programming</h3>
It has been a long time since I needed to refer to these two books. However,
they contained information I couldn't find elsewhere no matter
how hard I looked. So I recommend them in case you need to create or understand
some low level win32 code.
<br/>
<br/>
<ul>
<li> <i>Win32 Programming</i> by Brent E. Rector and Joseph M. Newcomer </li>
<li> <i>Programming Windows</i> by Charles Petzold </li>
<li> <a href="http://msdn.microsoft.com/en-us/library/default.aspx">MSDN Library</a>
<ul> This is Microsoft's online reference documentation. It is very large and sometimes
confusing. But at the end of the day you should be able to find the documentation
for just about every function in the entire Windows API.
</ul><br/>
</li>
</ul>
</ul>
<h2>Computer Science: Algorithms and Data Structures</h2>
<ul>
<li> <i>Introduction to Algorithms</i> by Cormen, Leiserson, Rivest and Stein
<ul> You should get this book if you are looking for a good discussion of the classic computer science
algorithms and data structures (e.g. most of the components on the <a href="containers.html">containers</a>
page).
</ul><br/>
</li>
<li> <i>Algorithms in C++, Parts 1-4: Fundamentals, Data Structure, Sorting, Searching</i>
(3rd Edition) by Robert Sedgewick
<ul> This is another good algorithms book. If you are going to get only one book on this
subject then get the one above. However, when I was learning about these topics I
used both these books and on many occasions I found it helpful to read the description
of an algorithm or data structure in both. Where one description was a little vague or
confusing the other generally filled in the gaps.
</ul><br/>
</li>
</ul>
<h2>Lossless Data Compression</h2>
<ul>
<li> <i>Text Compression</i> by Bell, Cleary, and Witten
<ul> When I was studying data compression this was my most useful
resource. If you are looking to understand how lossless data compression
algorithms work then this is the book you want. It is completely self-contained
and an absolute joy to read. Note that contrary to one of the reviews on
amazon.com, the book <i>Managing Gigabytes</i> is not the second edition of this book;
if this topic interests you then be sure you get the 318 page
book published in 1990.
</ul><br/>
</li>
</ul>
<h2>General Math</h2>
<ul>
<li> <i>Linear Algebra Done Right</i> by Sheldon Jay Axler
<ul> If a matrix seems like an arbitrary grid of numbers or you find that
you are confused by vectors, matrices, and the various things
that get done with them then this book will change your whole view of this subject.
It doesn't teach you any algorithms. Instead, it will give you a general
framework in which to think about all this stuff. Once you have that down
everything else will start to make a lot more sense. If all goes well
you will even start to agree with the following: linear algebra is beautiful. :)
</ul><br/>
</li>
<li> <i>Numerical Linear Algebra</i> by Trefethen and Bau
<ul> While <i>Linear Algebra Done Right</i> is fairly abstract, this book by
Trefethen and Bau will
explain some of the actual algorithms that are often used.
This is a great second book if you find that you want to know more about
the SVD, LU decomposition, or various other algorithms involving linear algebra.
</ul><br/>
</li>
<li> <i>Calculus: Single and Multivariable</i> by Hughes-Hallett, Gleason, and McCallum
<ul>
Some of the books below will require and understanding of basic calculus. So
I'm recommending this book. It was the book I used as an undergrad and I
remember it being alright. That isn't exactly a glowing review so if you
are really considering buying a calculus book you may want to check out
other reviews before picking this one.
</ul><br/>
</li>
<li> <i>Introduction to Real Analysis</i> (third edition) by Bartle and Sherbert
<ul> At some level real analysis is like a really rigorous repeat of calculus.
So if you already have an undergraduate education in calculus and
you are reading things that seem reminiscent of calculus but involve
stuff you haven't seen before (e.g. sup, inf, "sets of numbers", sequences of points)
then you may be in need of a real analysis book. This one is quite good and should
be accessible to someone with the usual undergraduate computer science math background.
</ul><br/>
</li>
</ul>
<h2>Optimization</h2>
The subject of linear algebra is fundamental to optimization. So you must be familiar
with the contents of a book like <i>Linear Algebra Done Right</i> if you are going to study
this area. You will also need to know how to find the derivative of a function and
understand what a derivative is all about. So you will need to know a little bit of
calculus. Finally, once in a while you will need to know a little bit about real
analysis. Ultimately, what you need all depends on how deep you want to go.
<ul>
<li> <i>Practical Methods of Optimization</i> (second edition) by R. Fletcher 1987
<ul> I love this book. When I got it I literally spent my weekends sitting around
reading it for hours. It is a fascinating and well written introduction to
the subject of optimization. This has been my most valuable resource for
learning the fundamentals of optimization and I cannot recommend it highly enough.
</ul><br/>
</li>
<li> <i>Numerical Optimization</i> by Jorge Nocedal and Stephen Wright 2006
<ul> This is a more recent text on optimization that is also very good. It
covers many algorithms not covered by the above book.
</ul><br/>
</li>
<li> <i>Introduction to Derivative-Free Optimization</i> by Conn, Scheinberg, and Vicente
<ul> If you want to understand algorithms like <a href="optimization.html#find_min_bobyqa">BOBYQA</a>
then this is a good recent book on the subject. Note that a book like <i>Practical Methods of Optimization</i>
is almost certainly a prerequisite for reading this book. As an aside, BOBYQA is not discussed in this book but
its predecessor, NEWUOA is.
</ul><br/>
</li>
</ul>
<h2>Machine Learning</h2>
<ul>
<li> <i>Artificial Intelligence: A Modern Approach </i> (3rd Edition) by Stuart Russell and Peter Norvig
<ul> This book is about the much broader field of AI but it contains an excellent introduction
to machine learning and it also covers other useful topics like <a href="bayes.html">bayesian networks</a>.
Moreover, it is very well written and self-contained. So you don't need any particular
background to be able to learn from it apart from a typical undergraduate background
in computer science.
</ul><br/>
</li>
<li> <i>Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond </i>
by Bernhard Schlkopf and Alexander J. Smola
<ul> Most of the machine learning tools in dlib are implementations of various kernel methods.
So if you want a book that covers this topic in great depth as well as breadth then this is
probably the book for you. The most important prerequisite for this book is linear
algebra. Virtually everything in this book depends on linear algebra in a fundamental way.
<p>
The second important subject is optimization. Whenever you see the text
mention the KKT conditions, duality, "primal variables", or quadratic programming it
is talking about ideas from optimization. A good book which will explain all this to you
is <i>Practical Methods of Optimization</i>. Note that this book calls the KKT conditions
just the "KT" conditions. It is talking about the same thing. Also, duality
is something that comes up a lot in optimization but in the context of machine learning
usually people are talking about a particular form known as the Wolfe Dual.
</p>
It would also be good (but maybe not critical depending on which parts you want to read) to
be familiar with real analysis.
</ul><br/>
</li>
<li> <i>Kernel Methods for Pattern Analysis </i> by John Shawe-Taylor and Nello Cristianini
<ul> This is another good book about kernel methods. If you have to choose between
this book and <i>Learning with Kernels</i> I would go with <i>Learning with Kernels</i>. However, it is
good to have both since reading different presentations of difficult subjects
usually makes learning them easier.
</ul><br/>
</li>
<li> <i>Structured Prediction and Learning in Computer Vision</i> by Sebastian Nowozin and Christoph H. Lampert 2011
<ul> If you are looking for a book discussing the background material necessary
for understanding things like the <a href="ml.html#structural_svm_problem">Structural SVM</a>
tools in dlib then this is a good book. It is also available online
in <a href="http://www.nowozin.net/sebastian/papers/nowozin2011structured-tutorial.pdf">PDF form</a>.
</ul><br/>
</li>
</ul>
<h2>Image Processing</h2>
<ul>
<li> <i>Digital Image Processing</i> by Rafael C. Gonzalez and Richard E. Woods
<ul> This is a terrific introduction to digital image processing.
By and large this book doesn't require any special prerequisites. Sometimes
calculus shows up, but not too much.
</ul><br/>
</li>
</ul>
</body>
<!-- ************************************************************************* -->
</doc>
|
