I have weird problem with resolving hostnames on Ubuntu 16.04. I'm searching for solution from friday (4 days!), so I think it's totally not duplicate to similar questions.
I have a server in local network with two interfaces: external (to the internet) and internal (to local network). Both are static configured. I'll show config below.
It was working perfectly since installed in March. Several times I've successfully made 'apt update' with no problem. On friday I tried to update it again, and here the main story begins...
apt update
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-93-generic x86_64)
~$ sudo apt update
Err:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Temporary failure resolving 'us.archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
Temporary failure resolving 'us.archive.ubuntu.com'
Err:4 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
Temporary failure resolving 'us.archive.ubuntu.com'
Reading package lists... Done
Building dependency tree
Reading state information... Done
195 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial/InRelease Temporary failure resolving 'us.archive.ubuntu.com'
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease Temporary failure resolving 'us.archive.ubuntu.com'
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial-backports/InRelease Temporary failure resolving 'us.archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
ping/telnet
~$ ping google.com
ping: unknown host google.com
~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=58 time=3.87 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=58 time=3.93 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=58 time=3.88 ms
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 3.870/3.898/3.939/0.077 ms
~$ telnet 8.8.8.8 53
Trying 8.8.8.8...
Connected to 8.8.8.8.
Escape character is '^]'.
Connection closed by foreign host.
On previous updates there was no such problems with resolving hostnames. Server is primarily used in local network by ip address, so i don't know when this problem arise. So here are some commands, that i executed (external ip address replaced with #):
ifconfig
~$ ifconfig
enp29s0 Link encap:Ethernet HWaddr 00:10:18:25:cd:40
inet addr:#.#.#.# Bcast:#.#.#.# Mask:255.255.255.248
inet6 addr: fe80::210:18ff:fe25:cd40/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:145862 errors:0 dropped:0 overruns:0 frame:0
TX packets:119991 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:14777641 (14.7 MB) TX bytes:22823397 (22.8 MB)
enp3s0 Link encap:Ethernet HWaddr 00:1a:64:c9:93:f8
inet addr:10.0.35.115 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::21a:64ff:fec9:93f8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:779951 errors:0 dropped:0 overruns:0 frame:0
TX packets:608340 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:514425482 (514.4 MB) TX bytes:189891768 (189.8 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:2145438 errors:0 dropped:0 overruns:0 frame:0
TX packets:2145438 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:1185976997 (1.1 GB) TX bytes:1185976997 (1.1 GB)
cat /etc/network/interfaces
~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface - Internal
auto enp3s0
iface enp3s0 inet static
address 10.0.35.115
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.255.255.255
# gateway 10.1.10.102
# # dns-* options are implemented by the resolvconf package, if installed
# dns-nameservers 10.1.10.102
metric 20
# The secondary network interface - External
auto enp29s0
iface enp29s0 inet static
address #.#.#.#
netmask 255.255.255.248
# network #.#.#.#
# broadcast #.#.#.#
gateway #.#.#.#
dns-nameservers 8.8.8.8 8.8.4.4
metric 10
#auto enp6s0
iface enp6s0 inet manual
/etc/resolv.conf
~$ ls -la /etc/resolv.conf
lrwxrwxrwx 1 root root 27 Oct 14 01:46 /etc/resolv.conf -> /run/resolvconf/resolv.conf
~$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
But nmcli didn't show any DNS configured: nmcli
~$ nmcli dev show | grep 'DNS'
~$ nmcli dev show | grep 'IP4'
IP4.ADDRESS[1]: #.#.#.#/29
IP4.GATEWAY: #.#.#.#
What i did:
several times restarted server.
several times restarted systemd-resolved, NetworkManager.
comment and uncomment "dns=dnsmasq" in /etc/NetworkManager/NetworkManager.conf (with restart service and server).
found advice about switch off DNSSEC, but as i found it's already switched off.
made /etc/resolv.conf static file (not symbolic link), get back to symbolic link - all with restarts.
Nothing of this helps...
Today 16 Oct
Today I've made 'apt update' and 'apt upgrade' with some trick: changed in /etc/apt/source.list domains names to their ip addresses. So system updated and become 16.04.3. It doesn't help.
/etc/nsswitch.conf
Remove some settings for "hosts" and leave minimal values.
~$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.
passwd: compat
group: compat
shadow: compat
gshadow: files
#hosts: files mdns4_minimal [NOTFOUND=return] dns
hosts: files dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis
nslookup on custom DNS server
This thing finally drives me crazy. I run nslookup on custom DNS server...
~$ nslookup yandex.ru 208.67.222.222
;; connection timed out; no servers could be reached
tcpdump on 53 port
...check tcpdump - it was showing too much records with "bad udp cksum"...
~# sudo tcpdump -vvv -B 4096 -i enp29s0 host 208.67.222.222 and port 53
tcpdump: listening on enp29s0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:01:57.043326 IP (tos 0x0, ttl 64, id 30657, offset 0, flags [none], proto UDP (17), length 55)
#.#.#.#.55778 > 208.67.222.222.domain: [bad udp cksum 0x445e -> 0x6d74!] 47660+ A? yandex.ru. (27)
ethtool
...execute ethtool to avoid such records...
~$ sudo ethtool --offload enp29s0 rx off tx off
Actual changes:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off
tcp-segmentation-offload: off
tx-tcp-segmentation: off [requested on]
tx-tcp-ecn-segmentation: off [requested on]
...and check tcpdump again:
~# cat tcpdump.log
15:26:08.451181 IP (tos 0x0, ttl 64, id 65257, offset 0, flags [none], proto UDP (17), length 55)
#.#.#.#.54228 > 208.67.222.222.domain: [udp sum ok] 23065+ A? yandex.ru. (27)
15:26:08.498299 IP (tos 0x0, ttl 57, id 4778, offset 0, flags [DF], proto UDP (17), length 119)
208.67.222.222.domain > #.#.#.#.54228: [udp sum ok] 23065 q: A? yandex.ru. 4/0/0 yandex.ru. [4m4s] A 77.88.55.80, yandex.ru. [4m4s] A 5.255.255.80, yandex.ru. [4m4s] A 77.88.55.50, yandex.ru. [4m4s] A 5.255.255.60 (91)
15:26:13.451133 IP (tos 0x0, ttl 64, id 537, offset 0, flags [none], proto UDP (17), length 55)
#.#.#.#.54228 > 208.67.222.222.domain: [udp sum ok] 23065+ A? yandex.ru. (27)
15:26:13.498225 IP (tos 0x0, ttl 57, id 5523, offset 0, flags [DF], proto UDP (17), length 119)
208.67.222.222.domain > #.#.#.#.54228: [udp sum ok] 23065 q: A? yandex.ru. 4/0/0 yandex.ru. [3m59s] A 5.255.255.80, yandex.ru. [3m59s] A 77.88.55.50, yandex.ru. [3m59s] A 5.255.255.60, yandex.ru. [3m59s] A 77.88.55.80 (91)
15:26:18.451231 IP (tos 0x0, ttl 64, id 1389, offset 0, flags [none], proto UDP (17), length 55)
#.#.#.#.54228 > 208.67.222.222.domain: [udp sum ok] 23065+ A? yandex.ru. (27)
15:26:18.498305 IP (tos 0x0, ttl 57, id 6088, offset 0, flags [DF], proto UDP (17), length 119)
208.67.222.222.domain > #.#.#.#.54228: [udp sum ok] 23065 q: A? yandex.ru. 4/0/0 yandex.ru. [3m54s] A 77.88.55.50, yandex.ru. [3m54s] A 5.255.255.60, yandex.ru. [3m54s] A 77.88.55.80, yandex.ru. [3m54s] A 5.255.255.80 (91)
So it seems that Ubuntu works well with DNS server, sending request, getting answers. But somehow cannot parse this answer...
For now I have no any more ideas how to get DNS resolving back to work.
Am I did all right, or something was not, and what else I can do to make it work?
Upd 17 Oct:
ip route output:
~$ ip route
default via x.x.x.41 dev enp29s0 metric 10 onlink
10.0.0.0/8 dev enp3s0 proto kernel scope link src 10.0.35.115
x.x.x.40/29 dev enp29s0 proto kernel scope link src x.x.x.42
169.254.0.0/16 dev enp3s0 scope link metric 1000
~$ ip route get 8.8.8.8
8.8.8.8 via x.x.x.41 dev enp29s0 src x.x.x.42
cache
traceroute to Google DNS:
~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 x.x.x.41 (x.x.x.41) 1.413 ms 1.404 ms 1.420 ms
2 89.221.63.15 (89.221.63.15) 3.451 ms 3.473 ms 3.463 ms
3 195.208.208.232 (195.208.208.232) 3.360 ms 2.851 ms 2.843 ms
4 108.170.250.129 (108.170.250.129) 3.582 ms 108.170.250.97 (108.170.250.97) 4.029 ms 108.170.250.33 (108.170.250.33) 3.737 ms
5 108.170.226.91 (108.170.226.91) 3.679 ms 66.249.94.195 (66.249.94.195) 4.103 ms 108.170.227.65 (108.170.227.65) 3.828 ms
6 8.8.8.8 (8.8.8.8) 4.506 ms 3.786 ms 3.979 ms
x.x.x.42 - server address.
x.x.x.40 and x.x.x.41 - "x" is the same as in server address.
8.8.8.8
, the Google DNS server ? What is the response ofip route
?nslookup
using the Google DNS in the command line ? It does not work ? -- have you tried to set the Google DNS in both interfaces at/etc/network/interfaces
?