Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

Problem: I occasionally have network issues with my Ubuntu VPS. I cannot SSH to the box, I cannot ping the box by IP address. I can access the box via host Serial terminal. When I access the box via serial, I can't ping out anywhere (far as I can tell), even when pinging by IP address. After some amount of time the network comes back, sometimes without my intervention. Sometimes it comes back when I am fiddling around. But it is hard to tell why. (Edit: It is very consistently out for 1 hour)

Questions: How can I proceed in troubleshooting this issue? What things can I do in order to rule out configuration/software problems in my control so that I can feel more comfortable bringing up the issue to my VPS host?

Things I have tried:

  • Bring eth0 down and up
  • Disable firewall temporarily
  • Checked VPS host advisories for network issues - haven't seen any
  • Reboot the server via Web console
  • Note: None of these have worked for me

Details:

  • Ubuntu 10.04.1 LTS
  • Hosted with Xen virtualization
  • Have root access (SSH) to perform my own upgrades, installs, etc.
  • I have the VPS setup as a VPN server so that I can connect to it "Road Warrior" style and forward all my traffic through the VPS first. So that is the junk with 10.8.X.X
  • All traffic including DNS lookups are forwarded through the VPS
  • Use uncomplicated firewall (ufw) with some basic rules
  • Also acts as a server for some services including Mumble and web server
  • I setup a script on the VPS as a cron job to ping some common internet entities by IP address every 5 minutes. If there is failure in the ping, then it logs it to a file. Simple enough. Consistently the network outage lasts for an hour. It does not always happen at the same time of day. On almost all of the occurrences, the network is down for an hour and then it "magically" comes back.
  • Memory usage on my VPS is typically very high. Usually I am maxed out and using some swap. The memory hog is java, if that detail helps.
  • My provider has been very unhelpful. It has ranged from "we are sorry, we had an unfortunate issue" to "there is no problem now". This is frustrating to me because typically I make a ticket when there is a problem, but the problem is gone by the time the ticket is addressed. The most recent communication has been that they suggest reformatting my VPS and starting over, which i am not keen about.
  • Consistently network outages start on the hour (within 5-10 minutes). That is, network outages do not start around XX:30, XX:45, etc.

netstat -rn

    Kernel IP routing table  
    Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface  
    10.8.0.2        0.0.0.0         255.255.255.255 UH        0 0          0 tun0  
    XX.57.166.0     0.0.0.0         255.255.255.128 U         0 0          0 eth0  
    192.168.50.0    10.8.0.2        255.255.255.0   UG        0 0          0 tun0  
    10.8.0.0        10.8.0.2        255.255.255.0   UG        0 0          0 tun0  
    0.0.0.0         XX.57.166.1     0.0.0.0         UG        0 0          0 eth0  

ip route list

    10.8.0.2 dev tun0  proto kernel  scope link  src 10.8.0.1  
    XX.57.166.0/25 dev eth0  proto kernel  scope link  src XX.57.166.59  
    192.168.50.0/24 via 10.8.0.2 dev tun0  
    10.8.0.0/24 via 10.8.0.2 dev tun0  
    default via XX.57.166.1 dev eth0  metric 100

cat /etc/network/interfaces

    auto eth0  
    iface eth0 inet static  
        address XX.57.166.59  
        gateway XX.57.166.1  
        netmask 255.255.255.128  
    auto lo  
    iface lo inet loopback
share|improve this question

migrated from serverfault.com Feb 7 '12 at 2:04

This question came from our site for professional system and network administrators.

1  
If the OS is the one that came on the VPS you're probably not going to get very far without contacting the hosting company and having them diagnose/troubleshoot. –  Chris S Feb 7 '12 at 2:03
    
What @ChrisS said is extremely true. However, if you have access to the VPS as root, you may consider using 'top' or 'htop' to see whether or not your system is having 100% CPU usage and/or 100% RAM usage (non-cache, but actual usage). If your system is using 100% CPU consistently, or is consistently using 100% RAM, you will run into these issues. (I've run into such issues as you describe in both situations I mentioned here) –  Thomas W. Feb 7 '12 at 2:10
    
@ChrisS yes, the OS installed was a "click to install" type. However, I have root access to the VPS to install my own software, do my own upgrades, etc. –  Josh Feb 7 '12 at 2:53
    
@Josh you might consider reading my comment, as full memory use is a common cause of failed SSH and other failed network stuffs. –  Thomas W. Feb 7 '12 at 3:05
    
@ThomasWard Read your comment. If it were a memory issue though, wouldn't rebooting the VPS resolve the issue? I have rebooted multiple times when the network was out (via serial console login at the control panel website) and this did not help. I would think that any (temporary) memory issues would be resolved by a reboot. 100% CPU is not a common occurrence for long periods on the VPS. –  Josh Feb 7 '12 at 11:39

1 Answer 1

up vote 2 down vote accepted
+50

Firstly if you believe this is a vendor issue that they're not addressing, I'd strongly consider migrating away. I gave VPS.net the benefit of the doubt when their SAN kept crashing (taking down all the VPSes in the process) but after a few months of "We've fixed this for good" and it still crashing, I had to vote with my wallet.

It's surprisingly easy to start a VPS company (you really only need a bit of datacenter space and some servers) so they're not all equal in technical ability even before you get to customer service.

But in terms of getting to the bottom of the problem, I'd first look at stopping things ending up in swap. Leave swap on but do whatever you have to do so you're not pushing things that far. Rein in the Java application or add more RAM. And see what happens. If this is very regular, you shouldn't have to wait long (or pay much) to see a result.

Same with CPU. If you have things running at 100% for extended periods, you want to make sure they're not interfering with other applications. The most simple way to work this can be done by setting the nice value of whatever applications are rampant to something positive. A nice value of something like +10 should let the system get full priority of the resources before your applications. Sidebar: Nice values basically mean the're more polite when it comes to CPU scheduling. Something with a low (eg -20) nice value means they'll get prioritised over all other things with higher nice values.

If you can, expand your testing to other local network items. If they provide a DNS resolver (as a lot of server companies do), ping that constantly (well, a few times a minute) and log the results. If you can still access that during periods of downtime, it's less likely that it's your fault.

And as I say, if this isn't your fault, move. If you spend any more time trying to fix this, you're outweighing any conceivable benefit of staying with these people. I personally have a very good and long experience with Linode but there are lots of good companies out there.

share|improve this answer
    
Thanks for answers. I have decided to go the route of moving away. The service/support just is not helpful. They seemed determined to help for the short term fix ("is it working at this second") and not looking to root cause. –  Josh Feb 19 '12 at 13:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.