2

I've got a low-powered dual-core Celeron machine, running Ubuntu 19.04, acting as my router. It's also running an lxd virtual machine with PiHole for DNS.

It's all working well, and has been for some months. Ubuntu 19.10 is now out though, and I discovered on another machine that the configuration I've been using, which has the networking bridge needed for lxd set up in /etc/network/interfaces, no longer works in 19.10. I've successfully moved another machine with a similar bridge configuration over to NetPlan, but I can't seem to figure out how to do so for the router. I hope that someone here might show me where I'm going wrong.

Here is the original and currently-working /etc/network/interfaces file:

# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

# The WAN interface.
auto ethwan
iface ethwan inet dhcp
        dns-nameservers 208.67.222.222 208.67.220.220

# The (old) LAN interface.
iface eth0 inet manual

# The new LAN interface, on the new USB-to-Ethernet connection.
iface ethlan inet manual

# The *new* LAN interface.
auto br0
iface br0 inet static
        address 192.168.1.1
        netmask 255.255.255.0
        dns-nameservers 208.67.222.222 208.67.220.220
        bridge-ifaces ethlan
        bridge-ports ethlan
        up ifconfig ethlan up

Here's what I tried to use for /etc/netplan/01-network-manager-all.yaml, after commenting out all but the two lo lines above:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  #renderer: NetworkManager
  renderer: networkd

  ethernets:
    # eth0 isn't used, as the hardware isn't reliable
    eth0:
      match:
        macaddress: f4:4d:30:67:0a:6c
      dhcp4: yes
      dhcp6: yes
      set-name: eth0

    ethlan:
      match:
        macaddress: 00:e0:4c:6a:03:3f
      set-name: ethlan
      dhcp4: no
      dhcp6: no

    ethwan:
      match:
        macaddress: 78:32:1b:a8:cc:1c
      dhcp4: yes
      dhcp6: yes
      set-name: ethwan

  bridges:
    br0:
      interfaces: [ethlan]
      dhcp4: no
      dhcp6: no
      addresses: [192.168.1.1/24]
      #gateway4: 192.168.1.1
      #nameservers:
      #  addresses:
      #  - 192.168.1.11

The nameserver is supplied via dnsmasq.

With this setup, the router machine can access the Internet with no problems -- but the lxd virtual machine can't access most sites at all (when I run sudo apt update on it, it says that it hits http://security.ubuntu.com/ubuntu, but can't seem to reach http://archive.ubuntu.com/ubuntu), and machines elsewhere on the network have similar problems (I can't get to Google or DuckDuckGo, but my wife can reach Facebook on her iPad). I suspect the ones that it can reach are due to DNS caching somewhere along the line.

I've tried uncommenting both the gateway4 line and the three nameserver lines, together and separately. The gateway line seems to prevent it from working at all (I'm not sure of the internal details, but that seems to make sense); the nameserver lines don't seem to have any effect. I've also tried it with NetworkManager as the renderer, which also seems to cause it not to work at all.

Can anyone see where I've gone wrong, or suggest a change that might fix it? Thanks!

UPDATE:

The ip route data, with the working /etc/network/interfaces setup:

default via 45.74.106.33 dev ethwan
default dev eth0 scope link metric 1002 linkdown
default dev ethlan scope link metric 1003
default dev wlp2s0 scope link metric 1005 linkdown
45.74.106.32/27 dev ethwan proto kernel scope link src 45.74.106.51
169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.5.39 linkdown
169.254.0.0/16 dev wlp2s0 proto kernel scope link src 169.254.8.207 linkdown
169.254.0.0/16 dev ethlan proto kernel scope link src 169.254.4.254
169.254.0.0/16 dev ethwan scope link metric 1000
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.1

The same data, with the netplan setup:

default via 45.74.106.33 dev ethwan
default via 198.2.97.225 dev ethwan proto dhcp src 198.2.97.228 metric 100
default dev ethlan scope link metric 1003
default dev wlp2s0 scope link metric 1005 linkdown
45.74.106.32/27 dev ethwan proto kernel scope link src 45.74.106.51
169.254.0.0/16 dev wlp2s0 proto kernel scope link src 169.254.8.207 linkdown
169.254.0.0/16 dev ethlan proto kernel scope link src 169.254.4.254
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.1
198.2.97.224/27 dev ethwan proto kernel scope link src 198.2.97.228
198.2.97.225 dev ethwan proto dhcp scope link src 198.2.97.228 metric 100

I think the new 198.2.97.225 address is an additional address that my ISP assigned during my attempts. Other than that, and the rearranging of lines, it looks mostly the same to me, but I don't know a lot about the output of that command.

However, while attempting it, I made a very weird discovery. When I use the netplan configuration above, machines on the local network can get to any site except TLS/SSL sites. curl -v http://google.com returns the 301-redirect data, curl -v https://google.com gets to the line that says * TLSv1.3 (OUT), TLS handshake, Client hello (1): and then it just stops. I believe that explains why some sites worked and others didn't under that configuration.

I thought that might be an iptables problem, except that it works fine under the /etc/network/interfaces setup.

Even more confused now. :-?

1

As a first step in debugging this, try capturing the output of ip route when you have configured the system with ifupdown; then disable ifupdown and enable netplan, reboot, and capture the output again, and compare the results.

One difference that is visible between the two configurations is your handling of the eth0 interface, which under ifupdown you have set to 'manual' configuration and under netplan you are configuring with dhcp. This could certainly cause a difference that interferes with your connectivity.

  • Thanks, I'll give that a try. Should have a chance to do so within a few hours. eth0 isn't being used, because when I reboot there's about a 30% chance that it won't come back up on its own (apparently a hardware problem, and a major inconvenience on a headless router that needs to be accessed via SSH). I replaced it with a cheap USB-to-Ethernet adapter, identified as ethlan in the configurations. – Head Geek Oct 29 at 12:33
1

The router forced my hand: something got corrupted on it yesterday (I'm not sure how, I didn't do anything on it yesterday at all), it was locking up after a few minutes. Rather than reinstalling Ubuntu 19.04, I decided to grit my teeth, install the new Ubuntu 19.10, and find an answer to the problems. I don't regret it, but what a freakin' pain in the tail!

After six hours of research and debugging, it turns out there were several problems.

The first was the simplest: I needed to add a nameservers section to the br0 configuration. I'm not sure this was necessary to the final solution, since it uses dnsmasq to handle that, but it seemed to solve some problems.

The second was that /etc/network/iptables wasn't being automatically read anymore. That was previously done by /etc/network/if-pre-up.d/iptables, a file that I think I created while initially setting up the firewall. Solved that by adding a line to the root's crontab: @reboot /etc/network/if-pre-up.d/iptables.

The final problem was the toughest, and I'm not quite sure how it was actually fixed (UPDATE: see below). After fixing the first two, I could access all sites on the Internet except HTTPS sites, as described above. But it wasn't all HTTPS sites, because Facebook still worked, and I was able to get to https://google.com a couple times as well! So it wasn't actually a problem with the HTTPS system. Spent a lot of time poring over my iptables setup, only to eventually determine that there was nothing wrong with it.

I stumbled over a comment about that being a symptom of an incorrect MTU setting, spent an hour or two playing with various things related to it (all of which broke things further) before deciding that that couldn't be the problem.

Finally stumbled over another comment saying that having IPv6 enabled can cause problems that look like that. Changed the dhcp6 lines to no, and added a link-local: [ ] line or two. I'm not sure what actually did the trick, but suddenly it was all working again.

UPDATE, two days later:

This morning I rebooted the router machine for unrelated reasons. All of a sudden the HTTPS problems were back! Logged into the router again and ran ifconfig to see if the IPv6 link-local addresses had reappeared... br0 had one, but ethwan did not, so I figured that couldn't be the problem.

Then I thought to look at the MTU values that ifconfig provided. I know for a fact that MTU on ethwan was 1500 when it was working, because I checked it when everything started working properly. But it was now set to 576! Apparently the HTTPS problems I was having were all due to the MTU setting, and it was getting mis-set sometimes. Never had a problem with it before I switched to NetPlan, not sure why I am now.

Since I knew the proper MTU value, I put it in the NetPlan configuration file and ran sudo netplan apply. Lo and behold, everything suddenly started working again.

Nice to finally understand what was going on there. I wasn't happy with the IPv6 explanation, something seemed off with it because things started working on a reboot after they would have if that were the answer, if that makes any sense.

I've updated the configuration below to include the MTU change.

Anyway, here's the final NetPlan file, /etc/netplan/01-network-manager-all.yaml, that works on my router:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  #renderer: networkd

  ethernets:
    eth0:
      match:
        macaddress: f4:4d:30:67:0a:6c
      dhcp4: yes
      dhcp6: yes
      set-name: eth0

    ethlan:
      match:
        macaddress: 00:e0:4c:6a:03:3f
      set-name: ethlan

    ethwan:
      match:
        macaddress: 78:32:1b:a8:cc:1c
      dhcp4: yes
      dhcp6: no
      link-local: [ ]
      set-name: ethwan
      mtu: 1500

  bridges:
    br0:
      interfaces: [ethlan]
      addresses: [192.168.1.1/24]
      nameservers:
        addresses: [208.67.222.222, 208.67.220.220]
      link-local: [ ]

I'm posting this in the hopes that it will help others who run into this sort of problem. And maybe myself as well, if I have trouble the next time I need to rebuild this router system. ;-)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.