summaryrefslogtreecommitdiffstats
path: root/debian/FAQ
diff options
context:
space:
mode:
Diffstat (limited to 'debian/FAQ')
-rw-r--r--debian/FAQ669
1 files changed, 669 insertions, 0 deletions
diff --git a/debian/FAQ b/debian/FAQ
new file mode 100644
index 0000000..40e0aba
--- /dev/null
+++ b/debian/FAQ
@@ -0,0 +1,669 @@
+Frequently asked questions -- Debian mdadm
+==========================================
+
+Also see /usr/share/doc/mdadm/README.recipes.gz .
+
+The latest version of this FAQ is available here:
+ http://anonscm.debian.org/gitweb/?p=pkg-mdadm/mdadm.git;a=blob_plain;f=debian/FAQ;hb=HEAD
+
+0. What does MD stand for?
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+ MD is an abbreviation for "multiple device" (also often called "multi-
+ disk"). The Linux MD implementation implements various strategies for
+ combining multiple (typically but not necessarily physical) block devices
+ into single logical ones. The most common use case is commonly known as
+ "Software RAID". Linux supports RAID levels 1, 4, 5, 6 and 10 as well
+ as the "pseudo" RAID level 0.
+ In addition, the MD implementation covers linear and multipath
+ configurations.
+
+ Most people refer to MD as RAID. Since the original name of the RAID
+ configuration software is "md"adm, I chose to use MD consistently instead.
+
+1. How do I overwrite ("zero") the superblock?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ mdadm --zero-superblock /dev/sdXY
+
+ Note that this is a destructive operation. It does not actually delete any
+ data, but the device will have lost its "authority". You cannot assemble the
+ array with it anymore and if you add the device to another array, the
+ synchronisation process *will* *overwrite* all data on the device.
+
+ Nevertheless, sometimes it is necessary to zero the superblock:
+
+ - If you want ot re-use a device (e.g. a HDD or SSD) that has been part of an
+ array (with an different superblock version and/or location) in another one.
+ In this case you zero the superblock before you assemble the array or add
+ the device to a new array.
+
+ - If you are trying to prevent a device from being recognised as part of an
+ array. Say for instance you are trying to change an array spanning sd[ab]1
+ to sd[bc]1 (maybe because sda is failing or too slow), then automatic
+ (scan) assembly will still recognise sda1 as a valid device. You can limit
+ the devices to scan with the DEVICE keyword in the configuration file, but
+ this may not be what you want. Instead, zeroing the superblock will
+ (permanently) prevent a device from being considered as part of an array.
+
+ WARNING: Depending on which superblock version you use, it won't work to just
+ overwrite the first few MiBs of the block device with 0x0 (e.g. via
+ dd), since the superblock may be at other locations (especially the
+ end of the device).
+ Therefore always use mdadm --zero-superblock .
+
+2. How do I change the preferred minor of an MD array (RAID)?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ See item 12 in /usr/share/doc/mdadm/README.recipes.gz and read the mdadm(8)
+ manpage (search for 'preferred').
+
+3. How does mdadm determine which /dev/mdX or /dev/md/X to use?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ The logic used by mdadm to determine the device node name in the mdadm
+ --examine output (which is used to generate mdadm.conf) depends on several
+ factors. Here's how mdadm determines it:
+
+ It first checks the superblock version of a given array (or each array in
+ turn when iterating all of them). Run
+
+ mdadm --detail /dev/mdX | sed -ne 's,.*Version : ,,p'
+
+ to determine the superblock version of a running array, or
+
+ mdadm --examine /dev/sdXY | sed -ne 's,.*Version : ,,p'
+
+ to determine the superblock version from a component device of an array.
+
+ Version 0 superblocks (00.90.XX)
+ ''''''''''''''''''''''''''''''''
+ You need to know the preferred minor number stored in the superblock,
+ so run either of
+
+ mdadm --detail /dev/mdX | sed -ne 's,.*Preferred Minor : ,,p'
+ mdadm --examine /dev/sdXY | sed -ne 's,.*Preferred Minor : ,,p'
+
+ Let's call the resulting number MINOR. Also see FAQ 2 further up.
+
+ Given MINOR, mdadm will output /dev/md<MINOR> if the device node
+ /dev/md<MINOR> exists.
+ Otherwise, it outputs /dev/md/<MINOR>
+
+ Version 1 superblocks (01.XX.XX)
+ ''''''''''''''''''''''''''''''''
+ Version 1 superblocks actually seem to ignore preferred minors and instead
+ use the value of the name field in the superblock. Unless specified
+ explicitly during creation (-N|--name) the name is determined from the
+ device name used, using the following regexp: 's,/dev/md/?(.*),$1,', thus:
+
+ /dev/md0 -> 0
+ /dev/md/0 -> 0
+ /dev/md_d0 -> _d0 (d0 in later versions)
+ /dev/md/d0 -> d0
+ /dev/md/name -> name
+ (/dev/name does not seem to work)
+
+ mdadm will append the name to '/dev/md/', so it will always output device
+ names under the /dev/md/ directory. Newer versions can create a symlink
+ from /dev/mdX. See the symlinks option in mdadm.con(5) and mdadm(8).
+
+ If you want to change the name, you can do so during assembly:
+
+ mdadm -A -U name -N newname /dev/mdX /dev/sd[abc]X
+
+ I know this all sounds inconsistent and upstream has some work to do.
+ We're on it.
+
+4. Which RAID level should I use?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Many people seem to prefer RAID4/5/6 because it makes more efficient use of
+ space. For example, if you have devices of size X, then in order to get 2X
+ storage, you need 3 devices for RAID5, but 4 if you use RAID10 or RAID1+ (or
+ RAID6).
+
+ This gain in usable space comes at a price: performance; RAID1/10 can be up
+ to four times faster than RAID4/5/6.
+
+ At the same time, however, RAID4/5/6 provide somewhat better redundancy in
+ the event of two failing devices. In a RAID10 configuration, if one device is
+ already dead, the RAID can only survive if any of the two devices in the other
+ RAID1 array fails, but not if the second device in the degraded RAID1 array
+ fails (see next item, 4b). A RAID6 across four devices can cope with any two
+ devices failing. However, RAID6 is noticeably slower than RAID5. RAID5 and
+ RAID4 do not differ much, but can only handle single-device failures.
+
+ If you can afford the extra devices (storage *is* cheap these days), I suggest
+ RAID1/10 over RAID4/5/6. If you don't care about performance but need as
+ much space as possible, go with RAID4/5/6, but make sure to have backups.
+ Heck, make sure to have backups whatever you do.
+
+ Let it be said, however, that I thoroughly regret putting my primary
+ workstation on RAID5. Anything device-intensive brings the system to its
+ knees; I will have to migrate to RAID10 at one point.
+
+ Please also consult /usr/share/doc/mdadm/RAID5_versus_RAID10.txt.gz,
+ https://en.wikipedia.org/wiki/Standard_RAID_levels and perhaps even
+ https://en.wikipedia.org/wiki/Non-standard_RAID_levels .
+
+4b. Can a 4-device RAID10 survive two device failures?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ I am assuming that you are talking about a setup with two copies of each
+ block, so --layout=near2/far2/offset2:
+
+ In two thirds of the cases, yes[0], and it does not matter which layout you
+ use. When you assemble 4 devices into a RAID10, you essentially stripe a RAID0
+ across two RAID1, so the four devices A,B,C,D become two pairs: A,B and C,D.
+ If A fails, the RAID10 can only survive if the second failing device is either
+ C or D; if B fails, your array is dead.
+
+ Thus, if you see a device failing, replace it as soon as possible!
+
+ If you need to handle two failing devices out of a set of four, you have to
+ use RAID6, or store more than two copies of each block (see the --layout
+ option in the mdadm(8) manpage).
+
+ See also question 18 further down.
+
+ [0] It's actually (n-2)/(n-1), where n is the number of devices. I am not
+ a mathematician, see http://aput.net/~jheiss/raid10/, which gives the
+ chance of *failure* as 1/(n-1), so the chance of success is 1-1/(n-1), or
+ (n-2)/(n-1), or 2/3 in the four device example.
+ (Thanks to Per Olofsson for clarifying this in #493577).
+
+5. How to convert RAID5 to RAID10?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ To convert 3 device RAID5 to RAID10, you need a spare device (either a hot
+ spare, fourth device in the array, or a new one). Then you remove the spare
+ and one of the three devices from the RAID5, create a degraded RAID10 across
+ them, create the filesystem and copy the data (or do a raw copy), then add the
+ other two devices to the new RAID10. However, mdadm cannot assemble a RAID10
+ with 50% missing devices the way you might like it:
+
+ mdadm --create -l 10 -n4 -pn2 /dev/md1 /dev/sd[cd] missing missing
+
+ For reasons that may be answered by question 20 further down, mdadm actually
+ cares about the order of devices you give it. If you intersperse the "missing"
+ keywords with the physical devices, it should work:
+
+ mdadm --create -l 10 -n4 -pn2 /dev/md1 /dev/sdc missing /dev/sdd missing
+
+ or even
+
+ mdadm --create -l 10 -n4 -pn2 /dev/md1 missing /dev/sd[cd] missing
+
+ Also see item (4b) further up, and this thread:
+ http://thread.gmane.org/gmane.linux.raid/13469/focus=13472
+
+6. What is the difference between RAID1+0 and RAID10?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ RAID1+0 is a form of RAID in which a RAID0 is striped across two RAID1
+ arrays. To assemble it, you create two RAID1 arrays and then create a RAID0
+ array with the two md arrays.
+
+ The Linux kernel provides the RAID10 level to do pretty much exactly the
+ same for you, but with greater flexibility (and somewhat improved
+ performance). While RAID1+0 makes sense with 4 devices, RAID10 can be
+ configured to work with only 3 devices. Also, RAID10 has a little less
+ overhead than RAID1+0, which has data pass the md layer twice.
+
+ I prefer RAID10 over RAID1+0.
+
+6b. What's the difference between RAID1+0 and RAID0+1?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ In short: RAID1+0 concatenates two mirrored arrays while RAID0+1 mirrors two
+ concatenated arrays. However, the two are also often switched.
+
+ The linux MD driver supports RAID10, which is equivalent to the above
+ RAID1+0 definition.
+
+ RAID1+0/10 has a greater chance to survive two device failures, its
+ performance suffers less when in degraded state, and it resyncs faster after
+ replacing a failed device.
+
+ See http://aput.net/~jheiss/raid10/ for more details.
+
+7. Which RAID10 layout scheme should I use
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ RAID10 gives you the choice between three ways of laying out chunks on the
+ devices: near, far and offset.
+
+ The examples below explain the chunk distribution for each of these layouts
+ with 2 copies per chunk, using either an even number of devices (fewer than 4)
+ or an odd number (fewer than 5).
+
+ For simplicity we assume that the chunk size matches the block size of the
+ underlying devices and also the RAID10 device exported by the kernel
+ (e.g. /dev/md/name). The chunk numbers map therefore directly to the block
+ addresses in the exported RAID10 device.
+
+ The decimal numbers below (0, 1, 2, …) are the RAID10 chunks. Due to the
+ foregoing assumption they are also the block addresses in the exported RAID10
+ device. Identical numbers refer to copies of a chunk or block, but on different
+ underlying devices. The hexadecimal numbers (0x00, 0x01, 0x02, …) refer to the
+ block addresses in the underlying devices.
+
+ "near" layout with 2 copies per chunk (--layout=n2):
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ The chunk copies are placed "as close to each other as possible".
+
+ With an even number of devices, they lie at the same offset on the each device.
+ It is a classic RAID1+0 setup, i.e. two groups of mirrored devices, with both
+ forming a striped RAID0.
+
+ device1 device2 device3 device4 device1 device2 device3 device4 device5
+ ─────── ─────── ─────── ─────── ─────── ─────── ─────── ─────── ───────
+ 0 0 1 1 0x00 0 0 1 1 2
+ 2 2 3 3 0x01 2 3 3 4 4
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯
+ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯
+ 254 254 255 255 0x80 317 318 318 319 319
+ ╰──────┬──────╯ ╰──────┬──────╯
+ RAID1 RAID1
+ ╰──────────────┬──────────────╯
+ RAID0
+
+ "far" layout with 2 copies per chunk (--layout=f2):
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ The chunk copies are placed "as far from each other as possible".
+
+ Here, a complete sequence of chunks is striped over all devices. Then a second
+ sequence of chunks is placed next to them. More copies are added as the number
+ 2 goes up.
+
+ It is undesirable, however, to place copies of the same chunks on the same
+ devices. That is prevented by a cyclic permutation of each such stripe.
+
+ device1 device2 device3 device4 device1 device2 device3 device4 device5
+ ─────── ─────── ─────── ─────── ─────── ─────── ─────── ─────── ───────
+ 0 1 2 3 0x00 0 1 2 3 4 ╮
+ 4 5 6 7 0x01 5 6 7 8 9 ├ ▒
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ┆
+ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ┆
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ┆
+ 252 253 254 255 0x40 315 316 317 318 319 ╯
+ 3 0 1 2 0x41 4 0 1 2 3 ╮
+ 7 4 5 6 0x42 9 5 6 7 8 ├ ▒ₚ
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ┆
+ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ┆
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ┆
+ 255 252 253 254 0x80 319 315 316 317 318 ╯
+
+ Each ▒ in the diagram represents a complete sequence of chunks. ▒ₚ is a cyclic
+ permutation.
+
+ A major advantage of the "far" layout is that sequential reads can be spread
+ out over different devices, which makes the setup similar to RAID0 in terms of
+ speed. For writes, there is a cost of seeking. They are substantially slower.
+
+ "offset" layout with 2 copies per chunk (--layout=o2):
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Here, a number of consecutive chunks are bundled on each device during the
+ striping operation. The number of consecutive chunks equals the number of
+ devices. Next, a copy of the same chunks is striped in a different pattern.
+ More copies are added as the number 2 goes up.
+
+ A cyclic permutation in the pattern prevents copies of the same chunks
+ landing on the same devices.
+
+ device1 device2 device3 device4 device1 device2 device3 device4 device5
+ ─────── ─────── ─────── ─────── ─────── ─────── ─────── ─────── ───────
+ 0 1 2 3 0x00 0 1 2 3 4 ) AA
+ 3 0 1 2 0x01 4 0 1 2 3 ) AAₚ
+ 4 5 6 7 0x02 5 6 7 8 9 ) AB
+ 7 4 5 6 0x03 9 5 6 7 8 ) ABₚ
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ) ⋯
+ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
+ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ) ⋯
+ 251 252 253 254 0x79 314 315 316 317 318 ) EX
+ 254 251 252 253 0x80 318 314 315 316 317 ) EXₚ
+
+ With AA, AB, …, AZ, BA, … being the sets of consecutive chunks and
+ AAₚ, ABₚ, …, AZₚ, BAₚ, … their cyclic permutations.
+
+ The read characteristics are probably similar to the "far" layout when a
+ suitably large chunk size is chosen, but with less seeking for writes.
+
+ Upstream and the Debian maintainer do not understand all the nuances and
+ implications. The "offset" layout was only added because the Common
+ RAID Data Disk Format (DDF) supports it, and standard compliance is our
+ goal.
+
+ See the md(4) manpage for more details.
+
+8. (One of) my RAID arrays is busy and cannot be stopped. What gives?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ It is perfectly normal for mdadm to report the array with the root
+ filesystem to be busy on shutdown. The reason for this is that the root
+ filesystem must be mounted to be able to stop the array (or otherwise
+ /sbin/mdadm does not exist), but to stop the array, the root filesystem
+ cannot be mounted. Catch 22. The kernel actually stops the array just before
+ halting, so it's all well.
+
+ If mdadm cannot stop other arrays on your system, check that these arrays
+ aren't used anymore. Common causes for busy/locked arrays are:
+
+ * The array contains a mounted filesystem (check the `mount' output)
+ * The array is used as a swap backend (check /proc/swaps)
+ * The array is used by the device-mapper (check with `dmsetup')
+ * LVM
+ * dm-crypt
+ * EVMS
+ * The array contains a swap partition used for suspend-to-ram
+ (check /etc/initramfs-tools/conf.d/resume)
+ * The array is used by a process (check with `lsof')
+
+9. Should I use RAID0 (or linear)?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ No. Unless you know what you're doing and keep backups, or use it for data
+ that can be lost.
+
+9b. Why not?
+~~~~~~~~~~~~
+ RAID0 has zero redundancy. If you stripe a RAID0 across X devices, you
+ increase the likelyhood of complete loss of the filesystem by a factor of X.
+
+ The same applies to LVM by the way (when LVs are placed over X PVs).
+
+ If you want/must used LVM or RAID0, stripe it across RAID1 arrays
+ (RAID10/RAID1+0, or LVM on RAID1), and keep backups!
+
+10. Can I cancel a running array check (checkarray)?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ See the -x option in the `/usr/share/mdadm/checkarray --help` output.
+
+11. mdadm warns about duplicate/similar superblocks; what gives?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ In certain configurations, especially if your last partition extends all the
+ way to the end of the device, mdadm may display a warning like:
+
+ mdadm: WARNING /dev/sdXY and /dev/sdX appear to have very similar
+ superblocks. If they are really different, please --zero the superblock on
+ one. If they are the same or overlap, please remove one from the DEVICE
+ list in mdadm.conf.
+
+ There are two ways to solve this:
+
+ (a) recreate the arrays with version-1 superblocks, which is not always an
+ option -- you cannot yet upgrade version-0 to version-1 superblocks for
+ existing arrays.
+
+ (b) instead of 'DEVICE partitions', list exactly those devices that are
+ components of MD arrays on your system. So istead of:
+
+ DEVICE partitions
+
+ for example:
+
+ DEVICE /dev/sd[ab]* /dev/sdc[123]
+
+12. mdadm -E / mkconf report different arrays with the same device
+ name / minor number. What gives?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ In almost all cases, mdadm updates the super-minor field in an array's
+ superblock when assembling the array. It does *not* do this for RAID0
+ arrays. Thus, you may end up seeing something like this when you run
+ mdadm -E or mkconf:
+
+ ARRAY /dev/md0 level=raid0 num-devices=2 UUID=abcd...
+ ARRAY /dev/md0 level=raid1 num-devices=2 UUID=dcba...
+
+ Note how the two arrays have different UUIDs but both appear as /dev/md0.
+
+ The solution in this case is to explicitly tell mdadm to update the
+ superblock of the RAID0 array. Assuming that the RAID0 array in the above
+ example should really be /dev/md1:
+
+ mdadm --stop /dev/md1
+ mdadm --assemble --update=super-minor --uuid=abcd... /dev/md1
+
+ See question 2 of this FAQ, and also http://bugs.debian.org/386315 and
+ recipe #12 in README.recipes .
+
+13. Can a MD array be partitioned?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Since kernel 2.6.28, MD arrays can be partitioned like any other block
+ device.
+
+ Prior to 2.6.28, for a MD array to be able to hold partitions, it must be
+ created as a "partitionable array", using the configuration auto=part on the
+ command line or in the configuration file, or by using the standard naming
+ scheme (md_d* or md/d*) for partitionable arrays:
+
+ mdadm --create --auto=yes ... /dev/md_d0 ...
+ # see mdadm(8) manpage about the values of the --auto keyword
+
+14. When would I partition an array?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ This answer by Doug Ledford is shamelessly adapted from [0] (with
+ permission):
+
+ First, not all MD types make sense to be split up, e.g. multipath. For
+ those types, when a device fails, the *entire* device is considered to have
+ failed, but with different arrays you won't switch over to the next path
+ until each MD array has attempted to access the bad path. This can have
+ obvious bad consequences for certain array types that do automatic
+ failover from one port to another (you can end up getting the array in
+ a loop of switching ports repeatedly to satisfy the fact that one array
+ failed over during a path down, then the path came back up, and another
+ array stayed on the old path because it didn't send any commands during
+ the path down time period).
+
+ Second, convenience. Assume you have a 6 device RAID5 array. If a device
+ fails and you are using a partitioned MD array, then all the partitions on
+ the device will already be handled without using that device. No need to
+ manually fail any still active array members from other arrays.
+
+ Third, safety. Again with the RAID5 array. If you use multiple arrays on
+ a single device, and that device fails, but it only failed on one array, then
+ you now need to manually fail that device from the other arrays before
+ shutting down or hot swapping the device. Generally speaking, that's not
+ a big deal, but people do occasionally have fat finger syndrome and this
+ is a good opportunity for someone to accidentally fail the wrong device, and
+ when you then go to remove the device you create a two device failure instead
+ of one and now you are in real trouble.
+
+ Forth, to respond to what you wrote about independent of each other --
+ part of the reason why you partition. I would argue that's not true. If
+ your goal is to salvage as much use from a failing device as possible, then
+ OK. But, generally speaking, people that have something of value on their
+ devices don't want to salvage any part of a failing device, they want that
+ device gone and replaced immediately. There simply is little to no value in
+ an already malfunctioning device. They're too cheap and the data stored on
+ them too valuable to risk loosing something in an effort to further
+ utilize broken hardware. This of course is written with the understanding
+ that the latest MD RAID code will do read error rewrites to compensate for
+ minor device issues, so anything that will throw a device out of an array is
+ more than just a minor sector glitch.
+
+ [0] http://thread.gmane.org/gmane.linux.raid/13594/focus=13597
+
+15. How can I start a dirty degraded array?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ A degraded array (e.g. a RAID5 with only two devices) that has not been
+ properly stopped cannot be assembled just like that; mdadm will refuse and
+ complain about a "dirty degraded array", for good reasons.
+
+ The solution might be to force-assemble it, and then to start it. Please see
+ recipes 4 and 4b of /usr/share/doc/mdadm/README.recipes.gz and make sure you
+ know what you're doing.
+
+16. How can I influence the speed with which an array is resynchronised?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ For each array, the MD subsystem exports parameters governing the
+ synchronisation speed via sysfs. The values are in kB/sec.
+
+ /sys/block/mdX/md/sync_speed -- the current speed
+ /sys/block/mdX/md/sync_speed_max -- the maximum speed
+ /sys/block/mdX/md/sync_speed_min -- the guaranteed minimum speed
+
+17. When I create a new array, why does it resynchronise at first?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ See the mdadm(8) manpage:
+ When creating a RAID5 array, mdadm will automatically create a degraded
+ array with an extra spare drive. This is because building the spare into
+ a degraded array is in general faster than resyncing the parity on
+ a non-degraded, but not clean, array. This feature can be over-ridden with
+ the --force option.
+
+ This also applies to RAID levels 4 and 6.
+
+ It does not make much sense for RAID levels 1 and 10 and can thus be
+ overridden with the --force and --assume-clean options, but it is not
+ recommended. Read the manpage.
+
+18. How many failed devics can a RAID10 handle?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ (see also question 4b)
+
+ The following table shows how many devices you can lose and still have an
+ operational array. In some cases, you *can* lose more than the given number
+ of devices, but there is no guarantee that the array survives. Thus, the
+ following is the guaranteed number of failed devices a RAID10 array survives
+ and the maximum number of failed devices the array can (but is not guaranteed
+ to) handle, given the number of devices used and the number of data block
+ copies. Note that 2 copies means original + 1 copy. Thus, if you only have
+ one copy (the original), you cannot handle any failures.
+
+ 1 2 3 4 (# of copies)
+ 1 0/0 0/0 0/0 0/0
+ 2 0/0 1/1 1/1 1/1
+ 3 0/0 1/1 2/2 2/2
+ 4 0/0 1/2 2/2 3/3
+ 5 0/0 1/2 2/2 3/3
+ 6 0/0 1/3 2/3 3/3
+ 7 0/0 1/3 2/3 3/3
+ 8 0/0 1/4 2/3 3/4
+ (# of devices)
+
+ Note: I have not really verified the above information. Please don't count
+ on it. If a device fails, replace it as soon as possible. Corrections welcome.
+
+19. What should I do if a device fails?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Replace it as soon as possible.
+
+ In case of physical devices with no hot-swap capabilities, for example via:
+
+ mdadm --remove /dev/md0 /dev/sda1
+ poweroff
+ <replace device and start the machine>
+ mdadm --add /dev/md0 /dev/sda1
+
+20. So how do I find out which other device(s) can fail without killing the
+ array?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Did you read the previous question and its answer?
+
+ For cases when you have two copies of each block, the question is easily
+ answered by looking at the output of /proc/mdstat. For instance on a 4 device
+ array:
+
+ md3 : active raid10 sdg7[3] sde7[0] sdh7[2] sdf7[1]
+
+ you know that sde7/sdf7 form one pair and sdg7/sgh7 the other.
+
+ If sdh now fails, this will become
+
+ md3 : active raid10 sdg7[3] sde7[0] sdh7[4](F) sdf7[1]
+
+ So now the second pair is degraded; the array could take another failure in
+ the first pair, but if sdg now also fails, you're history.
+
+ Now go and read question 19.
+
+ For cases with more copies per block, it becomes more complicated. Let's
+ think of a 7 device array with three copies:
+
+ md5 : active raid10 sdg7[6] sde7[4] sdb7[5] sdf7[2] sda7[3] sdc7[1] sdd7[0]
+
+ Each mirror now has 7/3 = 2.33 devices to it, so in order to determine groups,
+ you need to round up. Note how the devices are arranged in decreasing order of
+ their indices (the number in brackes in /proc/mdstat):
+
+ device: -sdd7- -sdc7- -sdf7- -sda7- -sde7- -sdb7- -sdg7-
+ group: [ one ][ two ][ three ]
+
+ Basically this means that after two devices failed, you need to make sure that
+ the third failed device doesn't destroy all copies of any given block. And
+ that's not always easy as it depends on the layout chosen: whether the
+ blocks are near (same offset within each group), far (spread apart in a way
+ to maximise the mean distance), or offset (offset by size/n within each
+ block).
+
+ I'll leave it up to you to figure things out. Now go read question 19.
+
+21. Why does the kernel speak of 'resync' when using checkarray?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Please see README.checkarray and http://thread.gmane.org/gmane.linux.raid/11864 .
+
+ In short: it's a bug. checkarray is actually not a resync, but the kernel
+ does not distinguish between them.
+
+22. Can I prioritise the sync process and sync certain arrays before others?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Upon start, md will resynchronise any unclean arrays, starting in somewhat
+ random order. Sometimes it's desirable to sync e.g. /dev/md3 first (because
+ it's the most important), but while /dev/md1 is synchronising, /dev/md3 will
+ be DELAYED (see /proc/mdstat; only if they share the same physical
+ components.
+
+ It is possible to delay the synchronisation via /sys:
+
+ echo idle >/sys/block/md1/md/sync_action
+
+ This will cause md1 to go idle and MD to synchronise md3 (or whatever is
+ queued next; repeat the above for other devices if necessary). MD will also
+ realise that md1 is still not in sync and queue it for resynchronisation,
+ so it will sync automatically when its turn has come.
+
+23. mdadm's init script fails because it cannot find any arrays. What gives?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ This does not happen anymore, if no arrays present in config file, no arrays
+ will be started.
+
+24. What happened to mdrun? How do I replace it?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ mdrun used to be the sledgehammer approach to assembling arrays. It has
+ accumulated several problems over the years (e.g. Debian bug #354705) and
+ thus has been deprecated and removed with the 2.6.7-2 version of this package.
+
+ If you are still using mdrun, please ensure that you have a valid
+ /etc/mdadm/mdadm.conf file (run /usr/share/mdadm/mkconf --generate to get
+ one), and run
+
+ mdadm --assemble --scan --auto=yes
+
+ instead of mdrun.
+
+25. Why are my arrays marked auto-read-only in /proc/mdstat?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Arrays are kept read-only until the first write occurs. This allows md to
+ skip lengthy resynchronisation for arrays that have not been properly shut
+ down, but which also not have changed.
+
+26. Why doesn't mdadm find arrays specified in the config file and causes the
+ boot to fail?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ My boot process dies at an early stage and drops me into the busybox shell.
+ The last relevant output seems to be from mdadm and is something like
+
+ "/dev/md2 does not exist"
+
+ or
+
+ "No devices listed in conf file found"
+
+ Why does mdadm break my system?
+
+ Short answer: It doesn't, the underlying devices aren't yet available yet
+ when mdadm runs during the early boot process.
+
+ Long answer: It doesn't, but the drivers of those devices incorrectly
+ communicate to the kernel that the devices are ready, when in fact they are
+ not. I consider this a bug in those drivers. Please consider reporting it.
+
+ Workaround: there is nothing mdadm can or will do against this. Fortunately
+ though, initramfs provides a method, documented at
+ http://wiki.debian.org/InitramfsDebug. Please append rootdelay=10 (which sets
+ a delay of 10 seconds before trying to mount the root filesystem) to the
+ kernel command line and try if the boot now works.
+
+ -- martin f. krafft <madduck@debian.org> Wed, 13 May 2009 09:59:53 +0200