I have a mdadm RAID-6 in my home server of 5x1Tb WD Green HDDs. Read speed is more than enough - 268 Mb/s in dd. But write speed is just 37.1 Mb/s. (Both tested via dd on 48Gb file, RAM size is 1Gb, block size used in testing is 8kb)
Could you please suggest why write speed is so low and is there any ways to improve it? CPU usage during writing is just 25% (i.e. half of 1 core of Opteron 165) No business critical data there & server is UPS-backed.
Personalities : [raid6] [raid5] [raid4] md0 : active raid6 sda1 sdd1 sde1 sdf1 sdb1 2929683456 blocks super 1.2 level 6, 1024k chunk, algorithm 2 [5/5] [UUUUU] bitmap: 0/8 pages [0KB], 65536KB chunk unused devices: <none>
Things like writeback, barrier, nobh didn't helped. DD blocksize=1M, 8M didn't changed anything. It looks like mdadm physically reads sectors to calculate parity even if it does not matter... Is that correct?
Update: Speed degradation after altered stripe cache was actually because 1 HDD probably failed during testing, nice :-D
Resolved: After increasing stripe cache & switching to external bitmap, my speeds are 160 Mb/s writes, 260 Mb/s reads. :-D