Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

After a bit of investigating of a problem I've just encountered, it looks like mdadm has completely forgotten about a new device I'd added to an array. I recently added a new device to the array which went fine, but I've now rebooted (I don't think I had rebooted since making this change) and I've been greeted with all sorts of errors, and I'm fairly sure it's down to this.

So my question is: Should it be safe to just re-add that device back into the array? I really don't want to lose any data here! This is what I think I should do but I'd like confirmation from someone with a bit more experience first :) Bear in mind I've already done this once, and have fscked and grown the filesystem too, and have then written plenty more data to it.

sudo umount /dev/md0
sudo mdadm --stop /dev/md0
sudo mdadm --add /dev/md0 /dev/sdd1
sudo mdadm --grow /dev/md0 --raid-devices=4

A bunch of extra information and steps I've already taken: I rebooted and was greeted with a message along the lines of "Unable to read superblock" for /dev/md0, my RAID array. I immediately suspected mdadm since I had recently been fiddling with it, and cat /proc/mdstat reports this:

md0 : inactive sdb1[0](S) sdc1[1](S)[/code]

Which is wrong; I had just added /dev/sdd1 to the array too. sudo mount -a reported bad superblock on /dev/md0. I can't remember the next command I used now but something else reported that I had tried to write past the end of the filesystem, which makes sense since there was more data on it than could fit onto the first two devices alone.

sudo fdisk -l reports that the new device and partition are alive and well. cat /etc/mdadm/mdadm.conf shows this:

DEVICE /dev/sdb1 /dev/sdc1
ARRAY /dev/md0 level=raid5 num-devices=2 UUID=*snip*

Which shows clearly that it's completely forgotten about the new device I added. Was I supposed to 'save' the config after executing the commands above? I assumed they were lasting changes; I can't see any use in them being lost on restarting.

Edit note: I realised that the 'num-devices=2' is inclusive of the 'spares=1', which means that mdadm hasn't changed a disk from 'in use' to 'spare', it's just plain forgotten about it. I'll update the title to account for that.

share|improve this question
Did you check in syslog why the device got kicked out of the array? – JanC Oct 26 '10 at 1:42
(But maybe check out my answer first.) – JanC Oct 26 '10 at 1:53
up vote 2 down vote accepted

The line:

DEVICE /dev/sdb1 /dev/sdc1

means that mdadm will only check those 2 devices to be used as part of a RAID array.

You need to add /dev/sdd1 to that line, or replace it with:

DEVICE partitions

which is the default and will make it scan all partitions for mdadm superblocks.

share|improve this answer
That seems to have done the trick! The array is back exactly as I thought I had left it. I guess the commands issued to mdadm don't survive a reboot is all that has happened here - in future I'll update mdadm.conf instead. – Ben Hymers Oct 26 '10 at 8:19
The commands certainly don't survive a reboot, but they store all necessary information in the "superblock". Of course, if during boot not all devices are searched for superblocks, some parts of your RAID might not get found... I suggest you use the DEVICE partitions option; it will probably slow down your boot with a couple of milliseconds, but you won't have to think about updating that line anymore. ;-) – JanC Oct 26 '10 at 8:58
I think that's what I had a while ago, but it caused some odd problems that I've seen some other people having where it renames devices. Now that I know about this I shan't forget in future :) Thanks again for the advice! – Ben Hymers Oct 26 '10 at 14:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.