I'm running a local Ubuntu 14.04 LTS (apache 2.4.7) webserver configured with a static ip in the /etc/network/interfaces file as follows:
auto lo iface lo inet loopback auto em1 iface em1 inet static address 192.168.2.230 netmask 255.255.255.0 dns-nameservers 126.96.36.199 188.8.131.52 gateway 192.168.2.254
This webserver is connected to two other local pc's (the clients) through a network switch. That network switch in turn is connected to the router (192.168.2.254).
This setup works perfectly fine in normal conditions and I can connect to the webserver from the two local computers without a problem. Now, I was testing if the two clients could still reach the webserver if the router was to go offline by unplugging the network cable running from the network switch to the router, (to simulate the router crashing/being reset or internet access to the outside being unavailable).
Here is where the problem arises, every request made from the client computer to the webserver is now delayed by an even 10 seconds (even static html pages).
If I remove the gateway entry from the /etc/network/interfaces file (on the server) and reload the network interface without a gateway everything is fine again. So, it looks like the server is trying to reach the gateway for 10 seconds and then decides it can also reach the client pc directly. The clients and the server are on the same subnet and have the same netmask 255.255.255.0, by the way.
I've also tried bypassing the switch by directly connecting one of the clients with the server, the same problem persists, it works, but with a 10 second delay. Pinging from the client to the server and from the server to the client goes without a hitch (less than 1 ms).
route -n outputs the following with the gateway set:
Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.2.254 0.0.0.0 UG 0 0 0 em1 192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 em1
I've spend a good 12 hours trying to setup static routes, mess with network-manager and changing the metric and what not, but to no avail. At the moment I've even completely removed network-manager to rule out possible conflicting routes and configurations.
All I want is for the clients to be able to reach the webserver with the same speed regardless of the gateway being there or not. Not setting the gateway at all fixes everything locally but then I can obviously not access the server remotely anymore.
Hopefully my problem and setup is clear, I'm REALLY looking forward to getting this fixed and hope it's something obvious.
EDIT: The loopback interface on the server seems unaffected by the way, the website loads just as quick on the server itself with a valid gateway as without one.