I am creating a small web cluster starting with MAAS 2.0. I have 5 physical machines each utilizing 2 nics.
My MAAS server runs dhcp in 10.14.0.0/20, each physical node in the cluster gets a static assigned as 10.14.0.101, 102, etc, assigned to eno0. My maas server has IP forwarding setup and my nodes route to 10.14.0.1 for their default gateway. This network is on its own switch dedicated for internal traffic and IPMI.
The second nic on each node is plugged into a switch that is part of the public network, 199.16.X.X. Each node is statically assigned a 199.16.X.X address, no default gateway is assigned. If I assign a default gateway it removes the gateway assigned on the 10.14.0.0/20 network. Maybe not remove, but maybe overwrite is a better explanation.
If I don't have a route for 199.16.X.X then any incoming traffic requests are sent back out 10.14.0.0 which obviously fails.
I've solved my issue by adding a secondary route and adding a few commands to my interfaces files.
echo "1 rt2" >> /etc/iproute2/rt_tables
I've then added the following to my eno2 configuration in /etc/network/interfaces
post-up ip route add 199.16.XXX.8/28 dev eno2 src 199.16.XXX.14 table rt2
post-up ip route add default via 199.16.XXX.9 dev eno2 table rt2
post-up ip rule add from 199.16.XXX.14/32 table rt2
post-up ip rule add to 199.16.XXX.14/32 table rt2
Is there an alternative to this method that is more efficient, or just all the way around a better alternative?