21

On 11.04, /proc/[pid] contains oom_score_adj, but man proc describes the old oom_adj. I have googled unsuccessfully for any information on how to use this.

What I need to arrange is that processes started by 'pbs' or 'gridengine' are killed before anything else gets killed. How can I do that?

12

Based on my own Q&A on Unix&Linux on a similar question.

As Stuart pointed out very well in his answer, valid values are integers in the range of -1000 to 1000 for oom_score_adj. The lower the value, the lower the chance that it's going to be killed.

It's very inconvenient to have to change this value over and over again once you restart the application. The information is simply lost after the process has terminated. Upstart (the init daemon in Ubuntu), has a nice option for this to configure for daemons to make sure it's set whenever it has (re)started:

oom score

[...] snip [...]

Example:

# this application is a "resource hog"
oom score 1000

expect daemon
respawn
exec /usr/bin/leaky-app

So, basically, you can edit the /etc/init/myservice.conf configuration file for the services you like to change, to include a line oom score -1000. I do assume that the services 'pbs' or 'gridengine' you're talking about in your question is Upstart-enabled, otherwise you'll need another way to change this permanently.

7

If you can modify the start script spawning the process that you want to prioritize for oom-killer, add the following to the start script:

echo 1000 > /proc/self/oom_score_adj

The value is inherited for child processes.

6

If you set a high(ish) value for either oom_adj or oom_score_adj then they will be killed first. e.g.

echo 15 > /proc/[pid]/oom_adj

oom_adj goes from -16 to 15 and as mentioned above oom_score_adj accepts -1000 to 1000

  • You need to be root to do this. sudo sh -c "echo -1000 > /proc/[PID]/oom_score_adj" – Adil Jul 9 '14 at 8:52
  • 2
    Actually, you can increase the score for your processes w/o root ("yeah, this can be oom-killed"). Decreasing (below 0) needs root. – Piskvor Jan 27 '17 at 11:13
5

Looking into the code is always a good idea. Normally I use the search function of kernel.org, but its currently down. Github does a good job too. I found this:

Instead, a new tunable, /proc/pid/oom_score_adj, is added that ranges from -1000 to +1000. It may be used to polarize the heuristic such that certain tasks are never considered for oom kill while others may always be considered. The value is added directly into the badness() score so a value of -500, for example, means to discount 50% of its memory consumption in comparison to other tasks either on the system, bound to the mempolicy, in the cpuset, or sharing the same memory controller.

From https://github.com/mirrors/linux-2.6/commit/a63d83f427fbce97a6cea0db2e64b0eb8435cd10#include/linux/oom.h

  • THanks for the pointer, but I'm hoping someone will drive by and shoot in a recipe for the whole problem, which would seem to be arranging for this value to be set for the exec daemon of pbs before it forks any jobs. – bmargulies Sep 9 '11 at 13:29
0

We have choom now.

tl;dr: choom --adjust -100 myprogram.

The man page has more information.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.