12

Please consider the following command for the imagemagick package:

sudo convert -resize 460x200 /path/to/test1.jpg /path/to/test2.jpg

Now test1.jpg is a fairly large image (12.5MB), but not unheard of sizes, but the server I am working from only has 1G of memory (Amazon EC2 t2.micro if it helps)

No error is being thrown, and yet no test2.jpg is being created. I have tested this with smaller images and it works fine. I would have thought 1G if memory would be plenty to manage a resize, but perhaps not?

What are my options, do I have to get more memory or am I missing something?

Any suggestions are very welcome!

Requested Edits

strace output shows: +++ killed by SIGKILL +++ along with alot of commands that look normal (opening /usr/share/local and reading long strings, which I assume is image data)

free gives me:

         total       used       free     shared    buffers     cached
Mem:       1016292     278348     737944       6412        460      13356
-/+ buffers/cache:     264532     751760
Swap:            0          0          0

So it looks as if swap is not enabled, so I will look into creating a swapfile to see if this sorts my issues.

File gives me:

testprint10Mbv2.jpg: JPEG image data, EXIF standard

I would also paste the full /path/to/img but unfortunately it contains sensitive data (client email addresses etc)

  • 1
    Why are you using sudo in this command? – Charles Green Jan 22 '16 at 17:15
  • 1
    Received "Killed" message when I didn't – Aphire Jan 22 '16 at 17:15
  • 3
    Ypu can try to limit the memory used for the pixel cache with the argument -limit memory 64MiB (32MiB is a bit too small probably. Default is 1.5GiB). And it should give you some descriptive output in case it fails. – dadexix86 Jan 25 '16 at 12:46
  • 2
    Try running the command with strace appended in front of the command, see what system calls are made, errors thrown. Let us know – Sergiy Kolodyazhnyy Jan 25 '16 at 21:04
  • 2
    Done. I'm glad you got it resolved! the free output confirms that swap is off and the file info confirms that it's not a mis-identified file. Nice edit! +1 – Elder Geek Jan 26 '16 at 19:13
7
+50

I'm assuming you are running out of RAM. You can verify this with watch cat /proc/meminfo while your process is running.

You may have insufficient RAM and/or SWAP to accomplish your task.

Due to the low-ram condition, insure that swap is on with swapon -a and if no swap is setup on the system create a partition for swap on unused diskspace and enable it. This will likely solve the problem. There's a good answer by @Takkat on how to do that here If you don't have access to partitioning the server you can also use a swapfile for swap

  • 1
    OP is about a cloud server, creating a swap partition may not be an option. You may add How to add new swap file too. – user.dz Jan 26 '16 at 21:40
  • @Sneetsher Thats exactly what I did, good comment – Aphire Jan 27 '16 at 8:33
  • @Sneetsher "OP is about a cloud server" wasn't clear (at least to me) from the data in the question but you make a good point. editing answer. – Elder Geek Jan 27 '16 at 14:14
  • @ElderGeek , actually it is but not well highlighted Amazon EC2 t2.micro , I edited question to make clear, added tag too to be easy linked to other related questions. – user.dz Jan 27 '16 at 14:27
  • @Sneetsher I see that now. Thank you for your efforts! – Elder Geek Jan 27 '16 at 14:28
5

The issue in the question was resolved by adding appropriate swap. For completeness let me give a summary of additional options we have when running Image Magick convert on huge files on low memory systems, or when there is no swap available, or the swap was too small.

The methods mentioned below are elaborated in detail in the Image Magick manual:

Really Massive Image Handling


Summary:

  1. Limit memory usage with option -limit

    By doing so Image Magick will create a temporary file for image handling as soon as the given memory limits were exceeded. This needs write permission for Image Magick on the temporary file directory. We can give any path where the temporary file will be created in an environment variable MAGICK_TMPDIR. An example command may look similar to this:

        env MAGICK_TMPDIR=/tempdir nice -5 convert -limit memory 32 -limit map 32 largefile.jpg -resize 640x320 smallfile.png
    

    Working on disk rather than in RAM will slow down the processing speed considerably.

  2. Work with "Memory Mapped Disk Files"

    Creating MPC files is resource demanding but it does not need so much resources to convert from MPC files. Therefore is may be put into consideration in case we need to convert the same source with different parameters several times. The workflow may then be similar to this:

    convert huge.jpg huge.mpc
    convert huge.mpc -resize 50% big.png
    convert huge.mpc -resize 20% small.png
    convert huge.mpc -resize 5% thumb.png
    
  3. Work on small sections of an image using stream

    Using stream claims to only process a part of the source image without the need to load the whole image into memory.

    stream -map rgb -storage-type char -extract 600x400+1900+2900 image.png - | convert -depth 8 -size 600x400 rgb:- tile.png
    

    In above example the -extract option takes size and offset values as defined by the Image Magick geometry. We will have to stick the tiles back together for getting the scaled image as a whole. Unfortunately stream does not work for all image formats but is supposed to work fine on JPEG images.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.