How do I use oom_score_adj?

Question

On 11.04, /proc/[pid] contains oom_score_adj, but man proc describes the old oom_adj. I have googled unsuccessfully for any information on how to use this.

What I need to arrange is that processes started by 'pbs' or 'gridengine' are killed before anything else gets killed. How can I do that?

Answer 1

^{Based on my own Q&A on Unix&Linux on a similar question.}

As Stuart pointed out very well in his answer, valid values are integers in the range of -1000 to 1000 for oom_score_adj. The lower the value, the lower the chance that it's going to be killed.

It's very inconvenient to have to change this value over and over again once you restart the application. The information is simply lost after the process has terminated. Upstart (the init daemon in Ubuntu), has a nice option for this to configure for daemons to make sure it's set whenever it has (re)started:

oom score

[...] snip [...]

Example:

# this application is a "resource hog"
oom score 1000

expect daemon
respawn
exec /usr/bin/leaky-app

So, basically, you can edit the /etc/init/myservice.conf configuration file for the services you like to change, to include a line oom score -1000. I do assume that the services 'pbs' or 'gridengine' you're talking about in your question is Upstart-enabled, otherwise you'll need another way to change this permanently.

Answer 2

Looking into the code is always a good idea. Normally I use the search function of kernel.org, but its currently down. Github does a good job too. I found this:

Instead, a new tunable, /proc/pid/oom_score_adj, is added that ranges from -1000 to +1000. It may be used to polarize the heuristic such that certain tasks are never considered for oom kill while others may always be considered. The value is added directly into the badness() score so a value of -500, for example, means to discount 50% of its memory consumption in comparison to other tasks either on the system, bound to the mempolicy, in the cpuset, or sharing the same memory controller.

From https://github.com/mirrors/linux-2.6/commit/a63d83f427fbce97a6cea0db2e64b0eb8435cd10#include/linux/oom.h

Answer 3

If you set a high(ish) value for either oom_adj or oom_score_adj then they will be killed first. e.g.

echo 15 > /proc/[pid]/oom_adj

oom_adj goes from -16 to 15 and as mentioned above oom_score_adj accepts -1000 to 1000

Answer 4

If you can modify the start script spawning the process that you want to prioritize for oom-killer, add the following to the start script:

echo 1000 > /proc/self/oom_score_adj

The value is inherited for child processes.