From a remote machine, I ran 100 concurrent requests against my Ubuntu VPS
ab -n 100 -c 100 http://...
And, as I suspected, the server "crashed". I can still ping it, and it responds very quickly (~50ms). But I can't
ssh into it or reach the web-server. I ran 1000 requests with 10 concurrent before, and it responded quickly and reliably, running at about 80 requests per second.
Fast forward five minutes and I can log in, the web server works great, everything is perfect again.
Uptime tells me the load_average is
0.06, 0.04, 0.05, and I have a quarter gigabyte of memory left free (out of 512MB). Running
netstat -ngives me lots and lots of lines like these:
tcp 0 0 127.0.0.1:9311 127.0.0.1:35030 TIME_WAIT tcp 0 0 127.0.0.1:5984 127.0.0.1:54384 TIME_WAIT tcp 0 0 127.0.0.1:9311 127.0.0.1:35024 TIME_WAIT
The server is running nginx as a reverse proxy, with a handful of cherrypy servers behind it. These servers run on ports between 8000 and 9000, and they listen only to
100 concurrent requests doesn't seem much to me, even though this is the absolute minimum spec server offered by my hosting company. How do I go about investigating what happend when the server crashed?
The server didn't reboot after the crash. There were no messages written to my
kern.log, and there's no firewall in front of the server.