6

Probably 1 out of 100 sites crashes the VPS every few days. Apache connections go to few hundreds and mysql requests to few millions. That causes the CPU to be at 100%+ permanently and making the server completely unusable. Restart fixes the situation for few days. How to find the evil one?

I've god mod_status with extendedstatus ON but still, it just shows the few recent requests. Not sure how to use that to narrow it down among 100 sites.

The rest of the time the server is using 20-30% of the resources so it's not overloaded with sites.

1 Answer 1

2

I don't known lot about your server configuration but I will try to help.

First start iptables and do not allow to attack your server

iptables -A INPUT -p tcp --dport 80 -m limit --limit 25/minute --limit-burst 100 -j ACCEPT

After that i think that logrotate can help us to find site with highest impact.

Install logrotate

sudo apt-get install logrotate

To verify that logrotate was successfully installed, run this in the command prompt.

logrotate

Configurations and default options for the logrotate utility are present in:

/etc/logrotate.conf

Some of the important configuration settings are : rotation-interval, log-file-size, rotation-count and compression.

Application-specific log file information (to override the defaults) are kept at:

/etc/logrotate.d/

My idea is to monitor http access file. Site with more access must have bigger access log. Logrotate and cron will watch for us on this and send you mail about problem ...

Edit logrotate.conf and add some code

sudo nano /etc/logrotate.conf

/path_to_apache_logs/*.log {
    size 1M
    dateext
    postrotate
      /usr/bin/killall -HUP httpd
      ls -ltr /path_to_apache_logs/ | mail -s "$HOSTNAME: Apache restarted and log files rotated" [email protected]
    endscript
}

You must customise

size - size of log file, change based on your needs path_to_apache_logs - change path based on log path in vhost.conf

More about customization of logrotate.conf you can find here

Setup cron to run logrotate on every 5 min

*/5  *  *  *  *   root    /usr/sbin/logrotate /etc/logrotate.conf

When you are on "attack" you access log will growth fast. Cron will run logrotate to check log file size and if file size is XX will rotate log, restart atache and send you mail.

Because every virtual host have own access file you will known witch site is server "killer"

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .