On my local file server I have raid-6 on 7x HDD drives.

dd if=/dev/zero of=tempfile bs=1M count=2048 conv=fdatasync

Local speed test gives me 349 MB/s.

Remote writes to Samba from SSD (>2Gb/s read speed) gives me 259 MB/s writes. But remote writes to iSCSI drive (on Win10 iSCSI initiator) gives me mere 151 Mb/s writes.

raid6 config - 128K chunk size, stripe_cache_size = 8191

Array mounted with options: rw,noatime,nobarrier,commit=999,stripe=128,data=writeback

open-iscsi setup: disk is based on a 4Tb file.

Any hints why iSCSI is slower than Samba on writes? Any hints how to improve iSCSI writes speed?

I assume it has something to do with desire of open-iscsi to flush writes to disk after each operation, which increases write amplification on raid6 due to excessive parity rewrites. But I am not sure how to fix it. Speed it more important than safety of currently written data in case of power outage.


It's reckless to mount a storage array as write back without a battery backup. You get what you deserve during a brownout (bouncing power is the worst), including incomplete writes.

Writethrough mode is the right thing to do without a battery backup.

Speed it more important than safety of currently written data in case of power outage.

If your bank did that with your paycheck, trade how much of your deposit you can get back vs how fast would you get it. Is there any excuse in the world that would justify their position that some or all of your hard earned money is gone forever?

That's how you should look at data storage, like a fiduciary responsibly.

You can't have it both ways.

File systems cache and then writeback or you call sync()). Hard storage cannot call a IO operation complete until the data is physically committed to the disk. Setting a disk to writeback mode relaxes that constraint trading safety for speed.

You pay for write amp anyways on RAID 6, it's just whether you pay for it now or a few ms from now. RAID 6 is where SSD's go to die, that's just the way of it. Gaming it for a few extra hours of service isn't worth the trade off of data loss.

The only way to mitigate this is with a maintenance and monitoring program that looks at the smart data (check once a day or week during off hours because it blocks IO) on the disks and sends an alert for replacement when they've reached 80% of their write lifetime.

So if performance is what you really care about then, then a striped mirror i.e. RAID 10 is what you want. Which will also dramatically reduce the write amp associated with a RAID 5/6.


If you want to have a industrial strength storage array that's secure and fast, you should explore adding NVDIMM or Crosspoint to your storage array build via PMEM.


Listing 3: Benchmarking

$ dd if=/dev/zero of=/mnt/test.dat oflag=direct bs=4k count=$((1024*1024)) 1048576+0 records in 1048576+0 records out 4294967296 bytes (4.3 GB, 4.0 GiB) copied, 4.55899 s, 942 MB/s

it's slowly getting into the hard storage part of the kernel.


Hope this helps.

EDIT: No RAID array is a substitute for a backup program which is routinely verified.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.