7

Really strange issue here. I intermittently keep getting really unpleasant CPU spikes, where the CPU gets to 80-90% busy across all cores for about 5 minutes. When I look at conky, in htop or system monitor, and sort by % CPU, I can't see any process that accounts for this much CPU usage.

The only things I've changed since this started are:

  • I moved to kernel version 2.6.35 (home compiled, up from 2.6.24-1)
  • I installed the Nvidia driver 256.44 (up from 256.34)

Now, I am willing to downgrade either/both of those to find the problem but I'd prefer to do this as scientifically as possible and find out what is causing the CPU explosion before I downgrade.

Edit: My precise issue looks like a nvidia regression in their latest driver. Other people are getting similar spikes.

  • 1
    htop behaves weird on my system in the sense that I see high cpu usage in the system monitor applet but htop wasn't showing the process that was using that much cpu (i had user and kernel threads enabled). top behaved normally and showed my runaway process. I'd love to find out why htop was behaving differently than top. – Li Lo Aug 7 '10 at 19:57
  • I think conky uses top directly so I don't think this will help me but thanks for mentioning it as it might help somebody else. – Oli Aug 7 '10 at 20:12
  • It doesn't seem to me that the linked nVidia thread correlates with this problem. People there are reporting CPU spikes from X, but yours is system CPU time (not linked to other process). From your description (10.10, 2.6.35, Core i7), your timer interrupt is freezing like described in this report: bugs.launchpad.net/ubuntu/+source/linux/+bug/665796 – Juliano Oct 29 '10 at 22:47
  • @Juliano It may have been that bug but all I know is it's not an issue any longer. And upgrading nvidia drivers was the only thing I did to fix it. – Oli Oct 30 '10 at 9:21
  • @Oli What nVidia driver version are you using now? – Juliano Oct 30 '10 at 13:53
1

It might be a kernel thread, those are hidden by default in most performance monitors. In htop you can hide/show kernel threads with "K" (shift+k).

  • Nope, no luck there. – Oli Aug 7 '10 at 21:56
3

"the CPU gets to 80-90% busy across all cores for about 5 minutes"

That much usage would possibly enable you to pinpoint the culprit by using pidstat available in the sysstat package.

Simply run pidstat -u | sort -nr -k 7,7 | head -10 and the process that used the most CPU should be the top line.

  • Nice tip about pidstat. First time coming across it. – harperville Mar 14 '14 at 15:12
2

I would try to find the cause for the problem with some shell script:

#!/bin/sh
MAXLOAD=100
CURRLOAD=`uptime | sed 's@.*load average: \([^,]*\).*@\1@' | sed 's@0\?.0\?@@'`

if [ $CURRLOAD -gt $MAXLOAD ]; then                                             
  ps -eo tid,pcpu,comm | sort -n -k 2 | tail -n 5 | \
    mail -s "High load" -e your@addre.ss
fi

The script has two variables MAXLOAD and CURRLOAD. The first one should be a high load multiplied by 100. So if you encounter a spike and see the system load going up to 2 or 3, than you should set MAXLOAD to some value around 200. $CURRLOAD takes the output of uptime, looks for the load and removes the dot as well as leading zeros.

If the load at some point is to high it prints out the five processes with the most CPU utilisation and send them to your@addre.ss.

This script should help you to find the reason for a spike and if you know it you maybe can resolve your issue.

1

To get output from top that you can save: top -b -n1

Stick this in a cronjob and you can look at the minutely processlist even after the problem went away. Example crontab entry:

* * * * * top -b -n1 > /tmp/top_output_$(date +%Y-%m-%d_%H:%M:%S)

This will save it in one file per minute in /tmp

1

There are some recently fixed bugs which might correct this issue. If you are running Ubuntu I'd suggest sticking with the Ubuntu kernel to pick up the patches through regular updates. I'd recommend installing Lucid for the support and stability. You can go with Maverick if there are features that you are aware of that are not in Lucid that you need.

0

I think this is a kernel issue. I would revert to an officially tested version.

  • Fairly sure it's nvidia given that other users of their latest drivers are reporting similar issues. See my edit at the bottom of the question. – Oli Aug 8 '10 at 11:26
  • Time to downgrade that then :) – Thorbjørn Ravn Andersen Aug 8 '10 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.