1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
|
# $OpenLDAP$
# Copyright 1999-2022 The OpenLDAP Foundation, All Rights Reserved.
# COPYING RESTRICTIONS APPLY, see COPYRIGHT.
H1: Tuning
This is perhaps one of the most important chapters in the guide, because if
you have not tuned {{slapd}}(8) correctly or grasped how to design your
directory and environment, you can expect very poor performance.
Reading, understanding and experimenting using the instructions and information
in the following sections, will enable you to fully understand how to tailor
your directory server to your specific requirements.
It should be noted that the following information has been collected over time
from our community based FAQ. So obviously the benefit of this real world experience
and advice should be of great value to the reader.
H2: Performance Factors
Various factors can play a part in how your directory performs on your chosen
hardware and environment. We will attempt to discuss these here.
H3: Memory
Scale your cache to use available memory and increase system memory if you can.
H3: Disks
Use fast filesystems, and conduct your own testing to see which filesystem
types perform best with your workload. (On our own Linux testing, EXT2 and JFS
tend to provide better write performance than everything else, including
newer filesystems like EXT4, BTRFS, etc.)
Use fast subsystems. Put each database on separate disks.
H3: Network Topology
http://www.openldap.org/faq/data/cache/363.html
Drawing here.
H3: Directory Layout Design
Reference to other sections and good/bad drawing here.
H3: Expected Usage
Discussion.
H2: Indexes
H3: Understanding how a search works
If you're searching on a filter that has been indexed, then the search reads
the index and pulls exactly the entries that are referenced by the index.
If the filter term has not been indexed, then the search must read every single
entry in the target scope and test to see if each entry matches the filter.
Obviously indexing can save a lot of work when it's used correctly.
In back-mdb, indexes can only track a certain number of entries per key (by
default that number is 2^16 = 65536). If more entries' values hash to this
key, some/all of them will have to be represented by a range of candidates,
making the index less useful over time as deletions cannot usually be tracked
accurately.
H3: What to index
As a general rule, to make any use of indexes, you must set up an equality
index on objectClass:
> index objectClass eq
Then you should create indices to match the actual filter terms used in
search queries.
> index cn,sn,givenname,mail eq
Each attribute index can be tuned further by selecting the set of index types to generate. For example, substring and approximate search for organizations (o) may make little sense (and isn't like done very often). And searching for {{userPassword}} likely makes no sense what so ever.
General rule: don't go overboard with indexes. Unused indexes must be maintained and hence can only slow things down.
See {{slapd.conf}}(5) and {{slapdindex}}(8) for more information
H3: Presence indexing
If your client application uses presence filters and if the
target attribute exists on the majority of entries in your target scope, then
all of those entries are going to be read anyway, because they are valid
members of the result set. In a subtree where 100% of the
entries are going to contain the same attributes, the presence index does
absolutely NOTHING to benefit the search, because 100% of the entries match
that presence filter. As an example, setting a presence index on objectClass
provides no benefit since it is present on every entry.
So the resource cost of generating the index is a
complete waste of CPU time, disk, and memory. Don't do it unless you know
that it will be used, and that the attribute in question occurs very
infrequently in the target data.
Almost no applications use presence filters in their search queries. Presence
indexing is pointless when the target attribute exists on the majority of
entries in the database. In most LDAP deployments, presence indexing should
not be done, it's just wasted overhead.
See the {{Logging}} section below on what to watch out for if you have a frequently searched
for attribute that is unindexed.
H3: Equality indexing
Similarly to presence indexes, equality indexes are most useful if the
values searched for are uncommon. Most OpenLDAP indexes work by hashing
the normalised value and using the hash as the key. Hashing behaviour
depends on the matching rule syntax, some matching rules also implement
indexers that help speed up inequality (lower than, ...) queries.
Check the documentation and other parts of this guide if some indexes are
mandatory - e.g. to enable replication, it is expected you index certain
operational attributes, likewise if you rely on filters in ACL processing.
Approximate indexes are usually identical to equality indexes unless
a matching rule explicitly implements it. As of OpenLDAP 2.5, only
directoryStringApproxMatch and IA5StringApproxMatch matchers
and indexers are implemented, currently using soundex or metaphone, with
metaphone being the default.
H3: Substring indexing
Substring indexes work on splitting the value into short chunks and then
indexing those in a similar way to how equality index does. The storage
space needed to store all of this data is analogous to the amount of data
being indexed, which makes the indexes extremely heavy-handed in most
scenarios.
H2: Logging
H3: What log level to use
The default of {{loglevel stats}} (256) is really the best bet. There's a corollary to
this when problems *do* arise, don't try to trace them using syslog.
Use the debug flag instead, and capture slapd's stderr output. syslog is too
slow for debug tracing, and it's inherently lossy - it will throw away messages when it
can't keep up. See {{slapd.conf}}(5) or {{slapd-config}}(5) for more information on
how to configure the loglevel.
Contrary to popular belief, {{loglevel 0}} is not ideal for production as you
won't be able to track when problems first arise.
H3: What to watch out for
The most common message you'll see that you should pay attention to is:
> "<= mdb_equality_candidates: (foo) index_param failed (18)"
That means that some application tried to use an equality filter ({{foo=<somevalue>}})
and attribute {{foo}} does not have an equality index. If you see a lot of these
messages, you should add the index. If you see one every month or so, it may
be acceptable to ignore it.
The default syslog level is stats (256) which logs the basic parameters of each
request; it usually produces 1-3 lines of output. On Solaris and systems that
only provide synchronous syslog, you may want to turn it off completely, but
usually you want to leave it enabled so that you'll be able to see index
messages whenever they arise. On Linux you can configure syslogd to run
asynchronously, in which case the performance hit for moderate syslog traffic
pretty much disappears.
H3: Improving throughput
You can improve logging performance on some systems by configuring syslog not
to sync the file system with every write ({{man syslogd/syslog.conf}}). In Linux,
you can prepend the log file name with a "-" in {{syslog.conf}}. For example,
if you are using the default LOCAL4 logging you could try:
> # LDAP logs
> LOCAL4.* -/var/log/ldap
For syslog-ng, add or modify the following line in {{syslog-ng.conf}}:
> options { sync(n); };
where n is the number of lines which will be buffered before a write.
H2: {{slapd}}(8) Threads
{{slapd}}(8) can process requests via a configurable number of threads, which
in turn affects the in/out rate of connections.
This value should generally be a function of the number of "real" cores on
the system, for example on a server with 2 CPUs with one core each, set this
to 8, or 4 threads per real core. This is a "read" maximized value. The more
threads that are configured per core, the slower {{slapd}}(8) responds for
"read" operations. On the flip side, it appears to handle write operations
faster in a heavy write/low read scenario.
The upper bound for good read performance appears to be 16 threads (which
also happens to be the default setting).
|
