What's a bitmap:
A mdadm bitmap, also called a "write intent bitmap", is a mechanism to speed up RAID rebuilds after an unclean shutdown or after removing and re-adding a disk.
With a bitmap, writing data to the RAID goes like this:
- Update bitmap: Mark the RAID chunks you are about to write to as dirty.
- Write the data to the RAID.
- Update bitmap: Mark the RAID chunks that were just written as clean.
The advantage of a bitmap is that if the system goes down in the middle of a write, the rebuild needs to check only the chunks marked as dirty, rather than the whole multi-TB RAID. This can speed up the rebuild process from taking several hours to completing in just a few seconds.
The drawback is lower write performance under normal use (outside rebuilds), since mdadm does additional disk access to update the bitmap.
External vs internal:
- external: Stored as a file on a disk outside the RAID. The advantage over an internal bitmap is better write performance during normal use (outside rebuilds).
- internal: Stored as RAID metadata. The advantage over an external bitmap is that you don't need a non-RAID disk and you save a bit on configuration (the path to the bitmap).
According to the mdadm man page:
Note: external bitmaps are only known to work on ext2 and ext3.
Storing bitmap files on other filesystems may result in serious
problems.
According to a post by Neil Brown, the mdadm author, external bitmaps should work on ext4 too:
I haven't looked inside ext4 but I am fairly confident that external bitmaps
will work properly.
HOWTO:
Bitmaps are added and removed using mdadm --grow --bitmap=XXX ...
, where the XXX is one of:
--bitmap=internal
: Create an internal bitmap.
--bitmap=/var/my_bitmap.bin
: Create an external bitmap at the specified path. The path must reside outside the RAID. A bitmap=...
parameter must be added to the ARRAY entry in /etc/mdadm/mdadm.conf
, and the --bitmap=...
parameter must be passed if you are assembling the RAID from the command line.
--bitmap=none
: Remove/disable any bitmaps.
References:
Comments:
IMO, bitmaps are perhaps primarily of interest for RAID levels 5 and 6, since these have the slowest rebuilds.
I switched from RAID 5 to RAID 10 myself; the rebuilds are so much faster that I don't feel the need for a bitmap, and RAID 10 seems to require far fewer rebuilds in the first place.
My RAID 5 setup used to drop a disk something like once a month, causing 12-14 hour rebuilds. The RAID 10 has only dropped a disk once in half a year, rebuilt in less than an hour.
I don't know if the frequent disk drops I experienced was caused by something other than the RAID level, but the RAID 10 has been far more stable and rebuild speed isn't much of a concern anymore.