9

I needed to run convert with a lot of images at the same time. The command took quite a while but this doesn't bother me.

The issue is that this command rendered my computer unusable while the command was running (for about 15 minutes).

So is it possible to throttle the command by limiting resources (processor and memory) to the command, directly from the command line? This can only work if I add something to the same line before pressing Enter because once I start the process the computer slows so much that it is impossible for example to switch to "System monitor" and reduce priority.

Edit: top and iotop results

I managed to run top and sudo iotop >iotop.txt while doing one of these convert operations. (The iotop.txt file produced is difficult to read)

Results of top:

   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND           
14275 username     20   0 4043m 3.0g 1448 D   7.0 80.4   0:16.45 convert 

Results of iotop:

[?1049h[1;24r(B[m[4l[?7h[?1h=[39;49m[?25l[39;49m(B[m[H[2JTotal DISK READ:    1269.04 K/s | Total DISK WRITE:[59G0.00 B/s
(B[0;7m  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN(B[0;1;7m     IO>(B[0;7m    COMMAND          [3;2H(B[m2516 be/4 username     350.08 K/s    0.00 B/s  0.00 %  0.00 % zeitgeist-datahub
 7394 be/4 username     568.88 K/s    0.00 B/s 77.41 %  0.00 % --rendere~.530483991[5;1H14275 idle username     350.08 K/s    0.00 B/s 37.49 %  0.00 % convert S~f test.pdf[6;2H2048 be/4 root[6;24H0.00 B/s    0.00 B/s  0.00 %  0.00 % [kworker/3:2]
[5G1 be/4 root[7;24H0.00 B/s    0.00 B/s  0.00 %  0.00 % init

Furthermore, even after the process ends, the computer does not return to the previous performance. I found a way around this by running sudo swapoff -a followed by sudo swapon -a

3 Answers 3

12

man nice, and if you need I/O throttling man ionice.

Let's look at some simple examples.

Throttling CPU usage:

you@box:/$ nice -n 10 /usr/bin/convert blah.gif blah.jpg

From your description, however, CPU usage is almost certainly not your real problem here. More likely, you're having severe I/O contention.

Throttling Disk Usage:

you@box:/$ ionice -c2 -n7 /usr/bin/convert blah.gif blah.jpg

-c2 is "best effort", and -n7 is the lowest "best effort" priority. So, this will throttle the job to lower I/O priority than most other things on the system.

you@box:/$ ionice -c3 /usr/bin/convert blah.gif blah.jpg

-c3 (no priority level necessary) means "Idle only". Jobs set to -c3 ONLY take up otherwise idle disk scheduling time, for virtually no impact at all on the system - but potentially a considerably longer execution time for your job, depending on how busy the rest of the system gets.

11
  • How would I integrate nice in a convert command? If I'm not going wrong, nice works for already running processes.
    – To Do
    Nov 24, 2012 at 15:20
  • 1
    See above... Also, srsly, read man page. It's a very easy man page, not chewy at all. :-)
    – Jim Salter
    Nov 24, 2012 at 15:27
  • I tried it. Sort of helped but not enough. I'll try nice -n 20 next time.
    – To Do
    Nov 24, 2012 at 17:55
  • Honestly I'm not too sure that CPU load is your problem here. Have you tried doing iotop while running the process? Usually a single process consuming lots of CPU will have very little effect on the usability of the system, but a process thrashing the hard drive can VERY quickly bring everything to a screeching halt - especially if you are using a conventional drive, not an SSD.
    – Jim Salter
    Nov 24, 2012 at 17:57
  • Since the whole system goes pretty unusable for you, you might try something along the lines of convert blah.gif blah.jpg & iotop so that iotop will fire up instantly as soon as you've started your convert job.
    – Jim Salter
    Nov 24, 2012 at 17:58
5

Oh, boy, I just caught this... quoting OP:

Furthermore, even after the process ends, the computer does not return to the previous performance. I found a way around this by running sudo swapoff -a followed by sudo swapon -a

OK, so that means you were exhausting the available RAM on your system, which means you're just plain trying to run too many convert processes at once. We'd need to look at your actual syntax in spawning convert to advise, but basically, you need to make sure you don't try to open more simultaneous processes than you have the RAM to comfortably handle.

Since you state what's causing this is convert *.tif blah.pdf, what's happening is that the content of every single TIF and its conversion to PDF are getting stuffed into RAM at once. What you need to do is split the job up so that this isn't necessary. One possibility that leaps to mind is instead doing something like find . -iname '*.tif' | xargs -I% convert % %.pdf, then using pdftk or something like it to glue all the individual pdfs together. If you really want to get fancy, and you have a multicore CPU, this also affords you the chance to write a small script to run conversions in batches of n, where n is your number of cores, and get the whole thing done significantly faster. :)

pdftk how-to: http://ubuntuhowtos.com/howtos/merge_pdf_files (basically boils down to sudo apt-get install pdftk; pdftk *.pdf cat output merged.pdf)

5
  • I'm just running ONE convert *.tif bla.pdf process at a time, but the input is a whole bunch of tif images. It's a way to produce one pdf from a bunch of images that works well but stifles my pc.
    – To Do
    Nov 26, 2012 at 19:49
  • ^^ see edited answer please :)
    – Jim Salter
    Nov 26, 2012 at 20:15
  • This worked perfectly with one simple modification. find . -iname '*.tif' | xargs -I% convert % %.pdf instead of find . -iname '*.pdf' | xargs -I% convert % %.pdf. It was quicker too! Thanks.
    – To Do
    Nov 26, 2012 at 21:54
  • Sorry, malfunction between brain and fingers :-)
    – Jim Salter
    Nov 27, 2012 at 5:14
  • One small modification… It is a good thing to know about the -exec and -execdir options for find. In your case, instead of using xargs, I would use: find -iname '*.tif' -execdir convert {} {}.pdf ';' Dec 4, 2012 at 10:48
2

Here's a pretty good article on using a program called cpulimit to limit the cpu usage of any process:

http://maketecheasier.com/limit-cpu-usage-of-any-process-in-linux/2010/09/22

The problem is like you said finding the PID of the convert process after it has started.

After starting convert if you can still type commands in the terminal you can run ps aux | grep convert to find it's PID, then sudo cpulimit -p PID -l CPU% to start throttling.

Or if starting convert also locks up your terminal the article shows you how you can run cpulimit as a background daemon that will automatically monitor and throttle any processes over a certain amount of CPU usage. Good luck!

1
  • This script is a bad idea, and primitive compared to the built-in nice command. cpulimit will limit the process to a certain % CPU usage whether or not anything else needs the CPU time; nice, by contrast, sets priority levels (40 total!) which allow prioritization of one task above (or below) others.
    – Jim Salter
    Nov 24, 2012 at 17:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .