Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I use my computer for scientific programming. It has a healthy 8GB of RAM and 12GB of swap space. Often, as my problems have gotten larger, I exceed all of the available RAM. Rather than crashing (which would be preferred), it seems Ubuntu starts loading everything into swap, including Unity and any open terminals. If I don't catch a run-away program in time, there is nothing I can do but wait - it takes 4-5 minutes to switch to a command prompt eg. Ctrl-Alt-F2 where I can kill the offending process.

Since my own stupidity is out of scope of this forum, how can I prevent Ubuntu from crashing via thrashing when I use up all of the available memory from a single offending program?

At-home experiment*!

Open a terminal, launch python and if you have numpy installed try this:

>>> import numpy
>>> [numpy.zeros((10**4, 10**4)) for _ in xrange(50)]

* Warning: may have adverse effects, monitor the process via iotop or top to kill it in time. If not, I'll see you after your reboot.

share|improve this question
add comment

2 Answers

up vote 7 down vote accepted

The shell built-in ulimit allows you to restrict resources. For your case, to limit memory use in the shell (and its children), use ulimit -v.

Demonstration setting a memory limit of 100 MB (100000 KB):

$ ulimit -v
unlimited
$ python -c '[ "x" * 100000000 ]'
$ ulimit -v 100000
$ python -c '[ "x" * 100000000 ]'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError

It's observed using ps uww -C script-name-here that python requires at least 29MB of memory (VSZ column). The RSS limit grows as your python script needs more memory so adapt that column.

share|improve this answer
1  
Too bad we don't know what the actual app looks like. The ulimit as demonstrated impacts all processes who are children of that shell. We don't even know if it runs as a daemon, or if it has lots of children which would further complicate things. See coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/… –  ppetraki Apr 11 '12 at 18:09
    
Is there no user-friendly front-end for ulimit and cgroups? In Windows, when the OS gets close to running out of RAM, a warning pop-up has always enabled me to avoid system freezes. In Ubuntu, do I have to keep an eye on the memory usage all the time? –  Dan Dascalescu Dec 30 '12 at 6:19
add comment

Cgroups should let you limit your memory usage on a per process basis.

https://en.wikipedia.org/wiki/Cgroups

http://www.mjmwired.net/kernel/Documentation/cgroups/memory.txt

Scientific computing is notoriously memory intensive, by sandboxing your app in a cgroup, the rest of the processes should not become victims as memory pressure will be alleviated.

Alternatively, a VM could be used as a sort of hard limit as the app can only use the memory delegated to the virtual machine, at the expense of performance of course. However a VM is much easier to configure for the uninitiated when compared to setting up and maintaining a cgroup.

Decisions decisions :) Good luck!

share|improve this answer
    
Is there no user-friendly front-end for ulimit and cgroups? In Windows, when the OS gets close to running out of RAM, a warning pop-up has always enabled me to avoid system freezes. In Ubuntu, do I have to keep an eye on the memory usage all the time? –  Dan Dascalescu Dec 30 '12 at 6:19
    
Nope. Also your OS shouldn't be freezing either, that's a separate problem. The windows pop up is just a bandaid to begin with. If you're using tons of memory maybe you should investigate why? For example, modern browsers with their sanboxing can easily consume 70-150M PER TAB. Turning that off or tuning it can free substantial memory used on a regular basis by an end user. Schedulers are supposed to swap out seldom used programs to save space but if they wake up more than they should then you need to fix them, or if your primary task is memory intensive (web) then more ram is needed. –  ppetraki Dec 31 '12 at 15:55
2  
I know what consumes the memory - Chrome. But the OS should protect its stability against applications that run amok nevertheless. –  Dan Dascalescu Jan 3 '13 at 11:30
    
I understand where you're coming from as a user. What you're asking for however means to turn a general computing platform into an appliance. As the least complex (error prone) method an operating system engineer must implement to provide those safeguards is to make dramatically opinionated decisions on how the entire computer is run. That usually comes at the expense of end user flexibility. You see examples of this in tablet computing. In desktops however the flexibility is preserved. –  ppetraki Jun 4 '13 at 14:14
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.