0

In short, my problem is this. I have a server with 4 active adapters. 2 are on one network (10.50.7.x/24), the other two are on different networks (10.4.1.x/24 and 10.224.1.x/24 respectively). The three networks have distinct connections to the internet. I need each adapter to respond to NATd traffic from it's own address so that services can be reached through each of the three external IPs. For testing, I've been performing ping to see if the traffic leaves the local network(s) [I have a remote device where I can see the pings arriving]. I have tried each of the following configurations, all of which pass sudo netplan --debug apply:

All DHCP:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: yes
      dhcp-identifier: mac
      dhcp4-overrides:
        route-metric: 100
    eno2:
      dhcp4: yes
      dhcp-identifier: mac
      dhcp4-overrides:
        route-metric: 100
    enp3s0f0:
      dhcp4: yes
      dhcp-identifier: mac
      dhcp4-overrides:
        route-metric: 100
    enp3s0f1:
      dhcp4: yes
      dhcp-identifier: mac
      dhcp4-overrides:
        route-metric: 100

Results of ip route:

default via 10.224.1.1 dev enp3s0f1 proto dhcp src 10.224.1.30 metric 100 
default via 10.4.1.2 dev enp3s0f0 proto dhcp src 10.4.1.26 metric 100 
default via 10.50.7.1 dev eno2 proto dhcp src 10.50.7.72 metric 100 
default via 10.50.7.1 dev eno1 proto dhcp src 10.50.7.73 metric 100 
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26 
10.4.1.2 dev enp3s0f0 proto dhcp scope link src 10.4.1.26 metric 100 
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72 
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73 
10.50.7.1 dev eno2 proto dhcp scope link src 10.50.7.72 metric 100 
10.50.7.1 dev eno1 proto dhcp scope link src 10.50.7.73 metric 100 
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30 
10.224.1.1 dev enp3s0f1 proto dhcp scope link src 10.224.1.30 metric 100 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.50.7.72 never leave the network

Individual gateways:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: no
      addresses: [10.50.7.73/24]
      gateway4: 10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    eno2:
      dhcp4: no
      addresses: [10.50.7.72/24]
      gateway4: 10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    enp3s0f0:
      dhcp4: no
      addresses: [10.4.1.26/24]
      gateway4: 10.4.1.2
      nameservers:
        addresses: [10.4.1.10]
    enp3s0f1:
      dhcp4: no
      addresses: [10.224.1.30/24]
      gateway4: 10.224.1.1
      nameservers:
        addresses: [10.224.1.1]

Results of ip route:

default via 10.224.1.1 dev enp3s0f1 proto static 
default via 10.4.1.2 dev enp3s0f0 proto static 
default via 10.50.7.1 dev eno2 proto static 
default via 10.50.7.1 dev eno1 proto static 
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26 
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72 
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73 
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network

Individual Routes:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: no
      addresses: [10.50.7.73/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
      routes:
        - to: 0.0.0.0/0
          via: 10.50.7.1
          metric: 100
          dev: en01
    eno2:
      dhcp4: no
      addresses: [10.50.7.72/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
      routes:
        - to: 0.0.0.0/0
          via: 10.50.7.1
          metric: 100
          dev: en02
    enp3s0f0:
      dhcp4: no
      addresses: [10.4.1.26/24]
      #gateway4: #10.4.1.2
      nameservers:
        addresses: [10.4.1.10]
      routes:
        - to: 0.0.0.0/0
          via: 10.4.1.2
          metric: 100
          dev: enp3s0f0
    enp3s0f1:
      dhcp4: no
      addresses: [10.224.1.30/24]
      #gateway4: #10.224.1.1
      nameservers:
        addresses: [10.224.1.1]
      routes:
        - to: 0.0.0.0/0
          via: 10.224.1.1
          metric: 100

Results of ip route:

default via 10.224.1.1 dev enp3s0f1 proto static metric 100 
default via 10.4.1.2 dev enp3s0f0 proto static metric 100 
default via 10.50.7.1 dev eno2 proto static metric 100 
default via 10.50.7.1 dev eno1 proto static metric 100 
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26 
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72 
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73 
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network

Single routes block with last adapter

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: no
      addresses: [10.50.7.73/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    eno2:
      dhcp4: no
      addresses: [10.50.7.72/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    enp3s0f0:
      dhcp4: no
      addresses: [10.4.1.26/24]
      #gateway4: #10.4.1.2
      nameservers:
        addresses: [10.4.1.10]
    enp3s0f1:
      dhcp4: no
      addresses: [10.224.1.30/24]
      #gateway4: #10.224.1.1
      nameservers:
        addresses: [10.224.1.1]
      routes:
        - to: 0.0.0.0/0
          via: 10.50.7.1
          metric: 100
        - to: 0.0.0.0/0
          via: 10.4.1.2
          metric: 100
        - to: 0.0.0.0/0
          via: 10.224.1.1
          metric: 100

Result of ip route:

default via 10.224.1.1 dev enp3s0f1 proto dhcp src 10.224.1.30 metric 100 
default via 10.224.1.1 dev enp3s0f1 proto static metric 100 
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26 
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72 
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73 
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown

Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network

I can reliably connect to NATd ports through 10.224.1.1, but cannot connect through either of the other two internet connections for obvious reasons.

Can anyone point me in a direction for what else to try, or am I going to have to abandon Netplan?

Here is the working config following Danny's leads:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: no
      addresses: [10.50.7.73/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
      routes:
        - to: 10.50.7.0/24
          via: 10.50.7.73
          table: 2
        - to: 0.0.0.0/0
          via: 10.50.7.1
          table: 2
      routing-policy:
        - from: 10.50.7.0/24
          table: 2
        - to: 10.50.7.0/24
          table: 2
    eno2:
      dhcp4: no
      addresses: [10.50.7.72/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
      routes:
        - to: 10.50.7.0/24
          via: 10.50.7.72
          table: 2
        - to: 0.0.0.0/0
          via: 10.50.7.1
          table: 2
      routing-policy:
        - from: 10.50.7.0/24
          table: 2
        - to: 10.50.7.0/24
          table: 2
    enp3s0f0:
      dhcp4: no
      addresses: [10.4.1.26/24]
      #gateway4: #10.4.1.2
      nameservers:
        addresses: [10.4.1.10]
      routes:
        - to: 10.4.1.0/24
          via: 10.4.1.26
          table: 3
        - to: 0.0.0.0/0
          via: 10.4.1.2
          table: 3
      routing-policy:
        - from: 10.4.1.0/24
          table: 3
        - to: 10.4.1.0/24
          table: 3
    enp3s0f1:
      dhcp4: no
      addresses: [10.224.1.30/24]
      gateway4: 10.224.1.1
      nameservers:
        addresses: [10.224.1.1]
      routes:
        - to: 10.224.1.0/24
          via: 10.224.1.30
          table: 1
        - to: 0.0.0.0/0
          via: 10.224.1.1
          table: 1
      routing-policy:
        - from: 10.224.1.0/24
          table: 1
        - to: 10.224.1.0/24
          table: 1

This did change the default gateway (to the 10.4.x.x gateway for some reason) but that's okay, what is far more important to me is that each adapter always responds through its respective gateway. No longer true. Setting the gateway4 directive for the 10.224.x.x adapter re-established that as the default gateway.

I'll also add that creating / using named tables did not work, so I just used the table numbers and used some comments in the yaml file to annotate them.

Improve this question

1 Answer 1

Reset to default
1

I think you want to set up policy based routing on the non-gatewayed interface, something like:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: no
      addresses: [10.50.7.73/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    eno2:
      dhcp4: no
      addresses: [10.50.7.72/24]
      #gateway4: #10.50.7.1
      nameservers:
        addresses: [10.50.7.104]
    enp3s0f0:
      dhcp4: no
      addresses: [10.4.1.26/24]
      #gateway4: #10.4.1.2
      nameservers:
        addresses: [10.4.1.10]
      routes:
       - to: 10.4.1.0/24
         via: 10.4.1.26
         table: 101
       - to: 0.0.0.0
         via: 10.4.1.1
         table: 101
      routing-policy:
       - from: 10.4.1.0/24
         table: 101
       - to: 10.4.1.0/24
         table: 101
Improve this answer
1
  • Danny, thank you so much! I'll post my final config below.
    – jvedman
    Dec 1, 2019 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.