In short, my problem is this. I have a server with 4 active adapters. 2 are on one network (10.50.7.x/24), the other two are on different networks (10.4.1.x/24 and 10.224.1.x/24 respectively). The three networks have distinct connections to the internet. I need each adapter to respond to NATd traffic from it's own address so that services can be reached through each of the three external IPs. For testing, I've been performing ping to see if the traffic leaves the local network(s) [I have a remote device where I can see the pings arriving]. I have tried each of the following configurations, all of which pass sudo netplan --debug apply:
All DHCP:
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: yes
dhcp-identifier: mac
dhcp4-overrides:
route-metric: 100
eno2:
dhcp4: yes
dhcp-identifier: mac
dhcp4-overrides:
route-metric: 100
enp3s0f0:
dhcp4: yes
dhcp-identifier: mac
dhcp4-overrides:
route-metric: 100
enp3s0f1:
dhcp4: yes
dhcp-identifier: mac
dhcp4-overrides:
route-metric: 100
Results of ip route:
default via 10.224.1.1 dev enp3s0f1 proto dhcp src 10.224.1.30 metric 100
default via 10.4.1.2 dev enp3s0f0 proto dhcp src 10.4.1.26 metric 100
default via 10.50.7.1 dev eno2 proto dhcp src 10.50.7.72 metric 100
default via 10.50.7.1 dev eno1 proto dhcp src 10.50.7.73 metric 100
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26
10.4.1.2 dev enp3s0f0 proto dhcp scope link src 10.4.1.26 metric 100
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73
10.50.7.1 dev eno2 proto dhcp scope link src 10.50.7.72 metric 100
10.50.7.1 dev eno1 proto dhcp scope link src 10.50.7.73 metric 100
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30
10.224.1.1 dev enp3s0f1 proto dhcp scope link src 10.224.1.30 metric 100
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.50.7.72 never leave the network
Individual gateways:
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: no
addresses: [10.50.7.73/24]
gateway4: 10.50.7.1
nameservers:
addresses: [10.50.7.104]
eno2:
dhcp4: no
addresses: [10.50.7.72/24]
gateway4: 10.50.7.1
nameservers:
addresses: [10.50.7.104]
enp3s0f0:
dhcp4: no
addresses: [10.4.1.26/24]
gateway4: 10.4.1.2
nameservers:
addresses: [10.4.1.10]
enp3s0f1:
dhcp4: no
addresses: [10.224.1.30/24]
gateway4: 10.224.1.1
nameservers:
addresses: [10.224.1.1]
Results of ip route:
default via 10.224.1.1 dev enp3s0f1 proto static
default via 10.4.1.2 dev enp3s0f0 proto static
default via 10.50.7.1 dev eno2 proto static
default via 10.50.7.1 dev eno1 proto static
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network
Individual Routes:
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: no
addresses: [10.50.7.73/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
routes:
- to: 0.0.0.0/0
via: 10.50.7.1
metric: 100
dev: en01
eno2:
dhcp4: no
addresses: [10.50.7.72/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
routes:
- to: 0.0.0.0/0
via: 10.50.7.1
metric: 100
dev: en02
enp3s0f0:
dhcp4: no
addresses: [10.4.1.26/24]
#gateway4: #10.4.1.2
nameservers:
addresses: [10.4.1.10]
routes:
- to: 0.0.0.0/0
via: 10.4.1.2
metric: 100
dev: enp3s0f0
enp3s0f1:
dhcp4: no
addresses: [10.224.1.30/24]
#gateway4: #10.224.1.1
nameservers:
addresses: [10.224.1.1]
routes:
- to: 0.0.0.0/0
via: 10.224.1.1
metric: 100
Results of ip route:
default via 10.224.1.1 dev enp3s0f1 proto static metric 100
default via 10.4.1.2 dev enp3s0f0 proto static metric 100
default via 10.50.7.1 dev eno2 proto static metric 100
default via 10.50.7.1 dev eno1 proto static metric 100
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network
Single routes block with last adapter
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: no
addresses: [10.50.7.73/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
eno2:
dhcp4: no
addresses: [10.50.7.72/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
enp3s0f0:
dhcp4: no
addresses: [10.4.1.26/24]
#gateway4: #10.4.1.2
nameservers:
addresses: [10.4.1.10]
enp3s0f1:
dhcp4: no
addresses: [10.224.1.30/24]
#gateway4: #10.224.1.1
nameservers:
addresses: [10.224.1.1]
routes:
- to: 0.0.0.0/0
via: 10.50.7.1
metric: 100
- to: 0.0.0.0/0
via: 10.4.1.2
metric: 100
- to: 0.0.0.0/0
via: 10.224.1.1
metric: 100
Result of ip route:
default via 10.224.1.1 dev enp3s0f1 proto dhcp src 10.224.1.30 metric 100
default via 10.224.1.1 dev enp3s0f1 proto static metric 100
10.4.1.0/24 dev enp3s0f0 proto kernel scope link src 10.4.1.26
10.50.7.0/24 dev eno2 proto kernel scope link src 10.50.7.72
10.50.7.0/24 dev eno1 proto kernel scope link src 10.50.7.73
10.224.1.0/24 dev enp3s0f1 proto kernel scope link src 10.224.1.30
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
Pings to external addresses from no specific address go through 10.224.1.1 Pings to external addresses from 10.4.1.72 never leave the network
I can reliably connect to NATd ports through 10.224.1.1, but cannot connect through either of the other two internet connections for obvious reasons.
Can anyone point me in a direction for what else to try, or am I going to have to abandon Netplan?
Here is the working config following Danny's leads:
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: no
addresses: [10.50.7.73/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
routes:
- to: 10.50.7.0/24
via: 10.50.7.73
table: 2
- to: 0.0.0.0/0
via: 10.50.7.1
table: 2
routing-policy:
- from: 10.50.7.0/24
table: 2
- to: 10.50.7.0/24
table: 2
eno2:
dhcp4: no
addresses: [10.50.7.72/24]
#gateway4: #10.50.7.1
nameservers:
addresses: [10.50.7.104]
routes:
- to: 10.50.7.0/24
via: 10.50.7.72
table: 2
- to: 0.0.0.0/0
via: 10.50.7.1
table: 2
routing-policy:
- from: 10.50.7.0/24
table: 2
- to: 10.50.7.0/24
table: 2
enp3s0f0:
dhcp4: no
addresses: [10.4.1.26/24]
#gateway4: #10.4.1.2
nameservers:
addresses: [10.4.1.10]
routes:
- to: 10.4.1.0/24
via: 10.4.1.26
table: 3
- to: 0.0.0.0/0
via: 10.4.1.2
table: 3
routing-policy:
- from: 10.4.1.0/24
table: 3
- to: 10.4.1.0/24
table: 3
enp3s0f1:
dhcp4: no
addresses: [10.224.1.30/24]
gateway4: 10.224.1.1
nameservers:
addresses: [10.224.1.1]
routes:
- to: 10.224.1.0/24
via: 10.224.1.30
table: 1
- to: 0.0.0.0/0
via: 10.224.1.1
table: 1
routing-policy:
- from: 10.224.1.0/24
table: 1
- to: 10.224.1.0/24
table: 1
This did change the default gateway (to the 10.4.x.x gateway for some reason) but that's okay, what is far more important to me is that each adapter always responds through its respective gateway. No longer true. Setting the gateway4 directive for the 10.224.x.x adapter re-established that as the default gateway.
I'll also add that creating / using named tables did not work, so I just used the table numbers and used some comments in the yaml file to annotate them.
