Disk Size Issue after raid 10 to raid 5 Migration using mdadm

Question

I have Expanded my Raid 10 to a Raid 5 using the mdadm utility. I have expanded the raid using the "sudo resize2fs /dev/md0" I then checked the file system using "sudo e2fsck -f /dev/md0" I also ran "sudo tune2fs extents /dev/md0" and upgraded the file system to 64 bit to get over the 16TB max limit.

However i cannot get the drive to come up wil the full storage capacity of 18.2TB I run "df -h" i get a capcity of 9.1TB but if i run "lsblk" i get the full capacity of 18.2TB Please see outputs of the raid and list results below:

itadmin@tronserv:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           787M  5.5M  781M   1% /run
/dev/sdd3       196G  3.0G  183G   2% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/loop0       90M   90M     0 100% /snap/core/8268
/dev/sdd1       511M  6.1M  505M   2% /boot/efi
/dev/md0        9.1T  7.8T  801G  91% /data/raid1
/dev/loop2       92M   92M     0 100% /snap/core/8592
tmpfs           787M     0  787M   0% /run/user/1000

itadmin@tronserv:~$ sudo lsblk
[sudo] password for itadmin:
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
loop0     7:0    0  89.1M  1 loop  /snap/core/8268
loop2     7:2    0  91.3M  1 loop  /snap/core/8592
sda       8:0    0   9.1T  0 disk
└─sda1    8:1    0   9.1T  0 part
  └─md0   9:0    0  18.2T  0 raid5 /data/raid1
sdb       8:16   0   9.1T  0 disk
└─sdb1    8:17   0   9.1T  0 part
  └─md0   9:0    0  18.2T  0 raid5 /data/raid1
sdc       8:32   0   9.1T  0 disk
└─sdc1    8:33   0   9.1T  0 part
  └─md0   9:0    0  18.2T  0 raid5 /data/raid1
sdd       8:48   0 238.5G  0 disk
├─sdd1    8:49   0   512M  0 part  /boot/efi
├─sdd2    8:50   0    12G  0 part  [SWAP]
└─sdd3    8:51   0   200G  0 part  /
itadmin@tronserv:~$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md0 : active raid5 sdc1[3] sda1[1] sdb1[2]
      19532605696 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
      bitmap: 0/73 pages [0KB], 65536KB chunk
itadmin@tronserv:~$ sudo mdadm --query --detail /dev/md0
/dev/md0:
           Version : 1.2
    Creation Time : Mon Dec 16 12:59:46 2019
Raid Level : raid5
Array Size : 19532605696 (18627.74 GiB 20001.39 GB)
     Used Dev Size : 9766302848 (9313.87 GiB 10000.69 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Fri Feb 14 16:39:34 2020
             State : clean
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 64K

Consistency Policy : bitmap

              Name : tronserv:0  (local to host tronserv)
              UUID : ce4acffc:c98f3dfe:c208ff1a:002728e7
            Events : 74713

    Number   Major   Minor   RaidDevice State
       2       8       17        0      active sync   /dev/sdb1
       1       8        1        1      active sync   /dev/sda1
       3       8       33        2      active sync   /dev/sdc1

Any Ideas on how to get the whole raid to come up as a 18.2TB Partition?

Answer 1

There are multiple layers of (virtual) device. All of them need to resize:

1. physical (cannot change)
2. mdadm (mdadm --grow /dev/md0 --size=max)
3. file system (resize2fs /dev/md0)

(you don't use lvm, but that would be another layer)

if you only resize the file system, it still sees the old size of the raid volume, so you have to do the mdadm call first. or more general: go through all layers.

---

But there's another thing: I see that your setup has 3 devices, but your previous raid 10 (if properly used) likely had 4 devices. So, where is your 4th device?

• 3 devices of 10T with raid 5 make (3-1)x10T=20T
• 4 devices would be (4-1)x10T=30T