1

Short note:

Well I think I might have a solution but need to prove it first.

Regarding the NAT on the Hardwarerouters to each WAN-line seem just to do NAT for its own subnet, so I will simply need to have them do NAT on the other subnets as well ...

Will put an answer together when its working.

Thanks.


I am setting up a Xen-Hypervisor to replace 3 of my old machines with this one.

Base is a Dell PowerEdge 2950 16GB Ram wich contains two physical NICs Ubuntu server Trusty 14.04 all updates done.

My setup is a bit wierd, but I have 4 networks on my intranet side wich is split up into

192.168.0.0/26 - all routing devices, accesspoints and servers
192.168.0.64/26 - all office and top prio devices
192.168.0.128/26 - entertainment stuff and kids
192.168.0.192/26 - low prio and open dhcp area for guests

this was because I need to turn down bandwidth on guests and low prio stuff to have enough reserved to work. All done with tc and iptables. That is still active on the old Internetline wich only has 0,5MBits/s DL and 96kBit/s UL on that we stuck for 13 years now.

Since last week I put a LTE (4G) 50MBit/s DL 10MBit/s UL Line in addition to the old one to speed up daily work. BUT 4G is based on transferpackages of 30GB and can be extended by 30GB for additional costs. Since my old line will stay for at least another 2 years until SIP-Trunking will be avaiable for me I want to use the old unlimited line for all sort of intensive things like updates and big downloads wich do not need to be ready at a set time or ftp backups etc.

so far this is my situation.

Now I want to built up a linux box that will do the following beside some other common stuff like LAMP and Mail etc.

  • keep my 4 networks and keep them managable like they were before to reserve bandwidth for top Prio like before but now for two lines.

  • use a hardware router/gateway for each line that do NAT and Dialup (have a wrt54gl for the slow DSL line and a Huawei B593u-12 for the LTE side) preferable these should be on 192.168.1.0/24 but I will explain that later.

so my setup looks like this:

LTE (B593u-12)
192.168.1.20
                  DELL2950 Dom-0                 Clients
                  (eth1 192.168.1.5)             (192.168.0.XXX/26)
                  (Xenbr0:X 192.168.0.XXX/26)
DSL (WRT54GL)
192.168.1.21

now we need some routing from 192.168.1.0/24 to the different networks 192.168.0.0/26, 192.168.0.64/26, 192.168.0.128/26, 192.168.0.192/26 wich are all sitting on Xenbr0 (192.168.0.5), Xenbr0:1 (192.168.0.105), Xenbr0:2 (192.168.0.175), Xenbr0:3 (192.168.0.205)

I am having a problem to establish static routes to each WAN-Router, that I do not need to NAT on my Linux Box, so right now I put 4 IPs on each WAN-Router to have it accessable in each network.

Kernel ip_forwand is enabled! I added full ACCEPT policy on iptables FORWARD, INPUT etc. tables to make sure it will not drop any for the time I am sorting the routing on the iproute2 side.

iptables -nvL

Chain INPUT (policy ACCEPT 5418K packets, 762M bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:67

Chain FORWARD (policy ACCEPT 1406K packets, 107M bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out vif4.0 --physdev-is-bridged
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in vif4.0 --physdev-is-bridged
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-out vif3.0 --physdev-is-bridged
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            PHYSDEV match --physdev-in vif3.0 --physdev-is-bridged
    0     0 ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24     ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0           
    0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
    0     0 REJECT     all  --  virbr0 *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT 5215K packets, 9072M bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     udp  --  *      virbr0  0.0.0.0/0            0.0.0.0/0            udp dpt:68

Xen did his vif and virbr0 device additions in the tables but thes are not interesting for my problem afaik.

Will enable iptable sets later, and will need a hint on these too if possible. Is the kernel able to route between all bridges without any extra config, or did I miss something here?

ip route show table main

default via 192.168.1.20 dev eth1 
192.168.0.0/26 dev xenbr0  proto kernel  scope link  src 192.168.0.5 
192.168.0.64/26 dev xenbr0  proto kernel  scope link  src 192.168.0.105 
192.168.0.128/26 dev xenbr0  proto kernel  scope link  src 192.168.0.175 
192.168.0.192/26 dev xenbr0  proto kernel  scope link  src 192.168.0.205 
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.5 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
224.0.0.0/4 dev xenbr0  scope link 

and similar on iproute2 for rule based routing:

ip route show table LTE

default via 192.168.1.20 dev eth1 
192.168.0.0/26 dev xenbr0  proto kernel  scope link  src 192.168.0.5 
192.168.0.64/26 dev xenbr0  proto kernel  scope link  src 192.168.0.105 
192.168.0.128/26 dev xenbr0  proto kernel  scope link  src 192.168.0.175 
192.168.0.192/26 dev xenbr0  proto kernel  scope link  src 192.168.0.205 
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.5

ip route show table DSL

default via 192.168.1.21 dev eth1 
192.168.0.0/26 dev xenbr0  proto kernel  scope link  src 192.168.0.5 
192.168.0.64/26 dev xenbr0  proto kernel  scope link  src 192.168.0.105 
192.168.0.128/26 dev xenbr0  proto kernel  scope link  src 192.168.0.175 
192.168.0.192/26 dev xenbr0  proto kernel  scope link  src 192.168.0.205 
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.5

And as far as I know I need to add static routes back from each WAN-Router to the Linuxbox so that it can find the 192.168.0.0/26 and so on...

That is possible on WRT54GL, thanks to DD-WRT Project and on the LTE one i gained root access and configured a startup script that can do similar things to the box like I am able on DD-WRT.

so I would try this on each device:

route add -net 192.168.0.0 mask 255.255.255.192 gw 192.168.1.5
route add -net 192.168.0.64 mask 255.255.255.192 gw 192.168.1.5
route add -net 192.168.0.128 mask 255.255.255.192 gw 192.168.1.5
route add -net 192.168.0.192 mask 255.255.255.192 gw 192.168.1.5

Each client gets the fitting IP of Dom-0 regarding their network to reach Dom-0 as the standart gateway. So this works fine. All SMB and CIFS Shares are accessable in the entire Infrastructure.

now I can do the following: ping the Dom-0 from each WAN-Routers shell (both eth0 and Xenbr0) ping each WAN-Router from any device in the other 0.XXX/26 nets.

so obviously the routing itself seem to work.

BUT somehow I can not figure out why I can not connect to the Internet from the clients?! The Linux Box itself is online in this scenario and can access both lines and the NATted internet of each WAN-router.

So here I am now. Have read a million and one page about roting, static routes and all sort.

in the end I want to be able to use IFB on eth1 to shape incoming traffic down to a set amount on each line for different rules like subnet, l7-mark, src and dst ip etc. to choose wich line the service will use and how much bandwith it will get via tc and iptables.

is my bridge setup ok?

bridge name bridge id       STP enabled interfaces
virbr0      8000.000000000000   yes     
xenbr0      8000.001d0904bbb4   yes     eth0
                                        vif3.0
                                        vif3.0-emu
                                        vif4.0
                                        vif4.0-emu

vif3 and vif4 are my Xen Guests (use them for developing on mysql etc.), they can access the whole network including all Shares and can ping each device inside the network. So roughly I think it should be ok. Not sure about that STP Option ...

I am a bit confused right now, since I can not see the problem ... hope some of you folks know it better ;)

greets from Germany ...

  • Have you considered using a commercial router? They usually are better suited for things like this... – Daniel Jul 21 '15 at 17:17
  • Well yes, but no. The commercial routers will have the same type of software in the background like my ubuntu server. And I do not count miliseconds, I just want ro create a routing that I can change the way I want it. So I will do it on my Linux box cause thats what Linux is for .... – KRAER Jul 22 '15 at 5:14
  • OK, this is pretty advanced stuff, so you might want to post this question on the Stack Exchange for Network Engineering or something like that. – Daniel Jul 22 '15 at 15:28
  • Well thanks Daniel, thought it would be best to put it in the Ubuntu section since I am building this on Ubuntu. Will check the Network Engineering area out. Since Ubuntu is just another Debian when it comes to core stuff I think they might be able to help ... hopefully. I am pretty sure it is just a tiny bit that I am missing and keep reading over and over again. Should I leave this here online and add a Link to the new Thread in the networkers section? Will punch a solution together as soon as I have it working. – KRAER Jul 22 '15 at 15:43
  • I would do that. This question could be beneficial to other users here. Let me know if you find an answer because I love the fact that you're reinventing the wheel just because you can. – Daniel Jul 22 '15 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.