20

I have hunch that a certain intermittent bug might only manifest itself when there is a slow disk read rate. Troubleshooting is difficult because I can't reliably reproduce it.

Short of simply gobbling IO with a high priority process, is there any way for me to simulate having a slow hard drive?

7
  • I remember seeing a command to tell harddrives to run at certain bus speeds. I'll see if I can dig it out.
    – Jeremy
    Nov 17, 2010 at 1:22
  • man hdparm take a look at the -X option maybe? There are quite a few things in there you could use to slow down your drive, but some of them risk doing nasty things to the data!
    – Jeremy
    Nov 17, 2010 at 1:27
  • Also, try mounting a network share as a folder (google is your friend), maybe even over wifi, if that is plausible.
    – Jeremy
    Nov 17, 2010 at 1:27
  • 1
    This isn't a direct answer, but: if I had an intermittent bug like this, I would probably try running the process under Valgrind (if it was in a compiled language), because that would likely capture IO race conditions.
    – poolie
    Nov 17, 2010 at 21:55
  • 1
    Are you talking about a bug in an application, or the kernel, or a device driver? Or you don't know at all? It might help if you explained more.
    – poolie
    Nov 18, 2010 at 1:29

10 Answers 10

16

Use nbd, the Network Block Device, and then rate limit access to it using say trickle.

sudo apt-get install nbd-client nbd-server trickle
3
  • +1 for a pretty cool solution. however, it's not a real test because you not going to the real hard disk device driver, which where the problem may lie. Nov 18, 2010 at 1:01
  • 1
    I didn't think he was talking about a driver bug, but that was just an assumption. Let's see.
    – poolie
    Nov 18, 2010 at 1:30
  • 20
    Is it possible to add the actual command to this answer? At the moment you only show how to install the tools required :)
    – Rich
    Sep 18, 2015 at 7:33
5
# echo 1 > /proc/sys/vm/drop_caches

That'll slow you down :)

It'll force you to read from disk, instead of taking advantage of the cached page.

If you really wanted to get sophisticated you could do something like fake a read error every nth time using the scsi fault injection framework.

http://scsifaultinjtst.sourceforge.net/

2
  • 1
    Note: This will only drop the caches once. Then it will immediately start caching again. You might wanna wrap that in a loop. Jan 3, 2018 at 12:54
  • @Androbin I like that idea. You could even add a sleep interval to the loop. Thanks!
    – ppetraki
    Jan 4, 2018 at 2:18
4

Have a USB 1.1 hub? Or a slow SD card? They'll get you down to under 10mbps.

1
  • nice...not technical enough for the +1 but nice Nov 17, 2010 at 2:32
3

This is by no means a complete solution, but it may help in conjunction with other measures: There is an I/O scheduler much like a process scheduler, and it can be tweaked.

Most notably, you can actually choose amongst different schedulers:

~# cat /sys/block/sda/queue/scheduler 
noop anticipatory deadline [cfq] 
~# echo "deadline" > /sys/block/sda/queue/scheduler
~# cat /sys/block/sda/queue/scheduler 
noop anticipatory [deadline] cfq 
~# 

deadline may help you get more strongly reproducible results.

noop, as its name implies, is insanely dumb, and will enable you to wreck absolute havoc on I/O performance with little effort.

anticipatory and cfq both try to be smart about it, though cfq is generally the smarter of the two. (As I recall, anticipatory is actually the legacy scheduler from right before the kernel started supporting multiple schedulers.)

3

You can use a Virtual Machine and throttle disk access ... here are some tips about how do it in Virtualbox 5.8. Limiting bandwidth for disk images https://www.virtualbox.org/manual/ch05.html#storage-bandwidth-limit

2

You could try running a copy of a large file, such as an iso of the Ubuntu install cd, and run it twice. That should slow your drive down quite a bit.

1
  • As the question says, "Short of simply gobbling IO"....
    – poolie
    Sep 18, 2015 at 21:10
2

Apart from trying to slow down the hard drive itself, you could try using filesystem benchmarking tools such as bonnie++ which can cause a great deal of disk I/O.

sudo apt-get install bonnie++
1
  • 1
    As the question says, "Short of simply gobbling IO"....
    – poolie
    Sep 18, 2015 at 21:10
1

I have recently figured out a setup where I've

  • moved the directory to my Google Drive
  • mounted it via the super-duper-slow client google-drive-ocamlfuse
  • created a symlink from the original path to the new one

If 16 seconds latency is not slow enough, you can just unplug your router.

For reference, here is the original use case, where I got the idea for this: https://github.com/goavki/apertium-apy/pull/76#issuecomment-355007128

0

how about make -j64? in articles describing that new 200line performance patch, make -j64 was a task eating a lot of computer resources

2
  • 2
    You should explain what that command actually does.
    – papukaija
    Feb 20, 2011 at 22:21
  • As the question says, "Short of simply gobbling IO"....
    – poolie
    Sep 18, 2015 at 21:10
0

Why not run iotop and see if the process that you are trying to debug is causing lots of disk reads/writes?

1
  • 3
    I think this answer is seen as unhelpful because the mere fact that the process is doing lots of IO may be already known, or not a problem in itself. The issue is that there is some kind of timing-related bug in the way it handles those IOs.
    – poolie
    Nov 17, 2010 at 21:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .