I have 3x 14TB toshiba drives MDAdmin (/dev/md0) raid 5'd together that are setup as a bcache back. I have a 256GB fast SSD as the front of the bache.

writeback is enabled.

After a few days, the device (/dev/bcache0) becomes extremely slow. I mean like 1000th of it's normal speed.

My 2 questions are: -For /dev/MD0, what tuning should I do for those toshiba drives? It's 4k chunks 64k blocks. -Is there any bcache tuning I could do?

I'm not even really sure what other info I should put here. But if you ask, I'll update this post. Thanks!

Update 1- my IOSTAT while getting 100mb/sec read, only 3mb/sec write: https://pastebin.com/wKKf4LTq

The computer is an amd 2990wx w/32gb ram. CPU isn't the issue.

My old 3770k from 2010ish would get hard better read and write speeds than this. It's got to be some sort of setting or tuning. Thanks!

Update 2- While the system is running normally, below is the hdparm output. hdparm takes to long to run when it's not running normally.

/dev/md0:
 Timing cached reads:   11148 MB in  2.00 seconds = 5578.33 MB/sec
 Timing buffered disk reads: 1372 MB in  3.00 seconds = 456.84 MB/sec
/dev/bcache0:
 Timing cached reads:   12564 MB in  2.00 seconds = 6286.57 MB/sec
 Timing buffered disk reads: 1226 MB in  3.00 seconds = 408.66 MB/sec

Thanks!

  • I hope you understand, that what you have is a very unusual setup because the cache device is larger than the backing device. What exactly did you try to accomplish by setting up the bcache? Try dropping the bcache altogether and use the SSD directly. You will benefit from 10 times bigger storage AND much better performance. – Adam Ryczkowski Sep 12 at 16:07
  • @AdamRyczkowski oops! I fixed that typo. 14TB! Not GB... – Chemdream Sep 12 at 16:31
  • What exactly are the speeds of each of those three devices (i.e. SSD, HDD and MD0)? Do you mean sequential read or random read? Just write at least the first significant digit. – Adam Ryczkowski Sep 12 at 18:30
  • The SSD is a SAMSUNG 860 Pro. Max Sequential Read Up to 560 MBps Max Sequential Write Up to 530 MBps 4KB Random Read Random (QD1): Up to 11,000 IOPS Random (QD32): Up to 100,000 IOPS 4KB Random Write Random (QD1): Up to 43,000 IOPS Random (QD32): Up to 90,000 IOPS The 14TB's are Toshiba MG07ACA14TE. 248 MiB/s Typ. 242 MiB/s – Chemdream Sep 12 at 19:07
  • Ok, but that is only two devices - SSD i HDD. Could you also put the speed of the md0? I want to get a feeling of the degree of the slowdown you are experiencing. Also it is better to edit the question rather than posting the information in comments. Just to keep all the relevant info in one place. – Adam Ryczkowski Sep 13 at 5:35

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.