23

My PC has a total of 4 NICs, 2 wired Gigabit Ethernet network cards and also 2 wireless network cards. (One Broadcom with proprietary drivers and Ralink with open source software, which works much better than Broadcom.)

My cellphone can share its connection wirelessly to my PC, but I also have a wired Internet connection. So I have multiple Internet connections to access the Internet. How can I merge 2 or more connections together and balance them to enjoy one unified Internet experience that it is the sum of all Internet connections connected to it.

For example if I have a modem with an Internet connection of 1024kB/s and another one that offers 512kB/s and one small one that offers 128kB/s, after load balancing and merging all connections (Bonding or Teaming), I could download at a speed of 1664kB/s using all 3 Internet connections as one for example.

This question has always intrigued me.

  • 5
    Seems rather beyond the scope of Ask Ubuntu :) This is pretty complex networking stuff. IMO it's rarely worth it, by the time you take into account lost packets due to a link being down (even temporarily), reordering packets due to some links being slower than others, and such things. I don't know of a "black box" solution that'll do this, could be an interesting project. – Caesium Nov 27 '11 at 17:43
  • 9
    Well the question is about if this can be done in ubuntu and if yes how. – Luis Alvarado Nov 27 '11 at 17:47
  • I created an answer in the 2 internet connections on a single PC post. Linking documents for bounding in Ubuntu. – Lucio Feb 25 '13 at 19:58
  • @Lucio I could mark the question as duplicate or you can move your answer here and I can mark it as accepted. You would also need to be ready for when 13.04 comes out since it will include in Network Manager the bonding options (Only wired bonding slaves right now). – Luis Alvarado Feb 25 '13 at 20:50
11

I do something like that at work using Ubuntu 11.04. We run the Shorewall firewall configuration tool, which besides being excellent at its job, provides some rudimentary multiple ISP routing tools which might fit your needs. You can find some docs about it here: http://www.shorewall.net/MultiISP.html

What it comes down to though, is you can't use multiple ISPs for a single connection... things aren't that simple. The best you can do is try to direct new connections evenly between the different providers.

It is a complex problem. You will probably end up beating your head against the wall (I certainly did) before you are done debugging every problem. So, as other posters have suggested, you might be wise to carefully consider how strong your desire is.

8

You could do it by using the package ifenslave that attaches and detaches slave network interfaces to a bonding device.

  1. Install:

    sudo apt-get install ifenslave
    
  2. Load bonding kernel module

    sudo modprobe bondingle
    
  3. Configure your interfaces:

    sudo vi /etc/network/interfaces
    

    Example config, to combine eth0 and eth1 as slaves to your bonding interface:

    #eth0 is manually configured, and slave to the "bond0" bonded NIC
    auto eth0
    iface eth0 inet manual
    bond-master bond0
    
    #eth1 ditto, thus creating a 2-link bond.
    auto eth1
    iface eth1 inet manual
    bond-master bond0
    
    # bond0 is the bonded NIC and can be used like any other normal NIC.
    # bond0 is configured using static network information.
    auto bond0
    iface bond0 inet static
    address 192.168.1.10
    gateway 192.168.1.1
    netmask 255.255.255.0
    # bond0 uses standard IEEE 802.3ad LACP bonding protocol 
    bond-mode 802.3ad
    bond-miimon 100
    bond-lacp-rate 1
    bond-slaves none
    
  4. Restart Network:

    sudo restart networking
    
  5. Bringing up/down bounded interface:

    ifup bond0
    ifdown bond0
    

    There are several bonding modes as an example we use:

    bond-mode active-backup
    

    Description of active-backup bonding mode:

    Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. This mode provides fault tolerance. The primary option affects the behavior of this mode.

    Source and more info at the Ubuntu community help wiki.

Bonding, means combining several network interfaces (NICs) to a single link, providing either high-availability, load-balancing, maximum throughput, or a combination of these. Source

  • +1 because is one of the reasons for my question. Will still wait for a way to create a virtual network master connection that is the sum of all slave (real) connections. Something like bonding that comes in the latest kernel version. – Luis Alvarado Nov 26 '12 at 13:08
  • 2
    -1 - bonding works in LANs as it's operating at Layer 2. The question is about load balancing two independent WANs. – gertvdijk Dec 29 '12 at 12:33
  • @gertvdijk I don't agree check my update answer to find out that bonding does also load balancing. – pl1nk Dec 31 '12 at 14:33
  • 1
    @pl1nk This is Layer 2 load balancing for two NICs in the same L2 network. This is not the same as multi-ISP load balancing! – gertvdijk Dec 31 '12 at 14:50
  • 3
    @pl1nk My final comment. The question is concerning multiple independent broadband connections, clearly. While they may be one ISP, this does not change the disability to handle this at layer 2. – gertvdijk Dec 31 '12 at 15:10
6

It's a little bit old question, but if you still want to know..

There are 2 typical scenarios, what gertvdijk and pl1nk were arguing in one of the answers:

You have a computer with 2 public IPs (2 different ISPs) and you connect to another host (e.g. a server in a datacenter with a fat pipe that's bigger than the aggregate bandwidth of both ISP connections of your computer). So you establish a bonding connection to the host via your 2 connections and then the host (server) serves your traffic via its own internet connection. In this scenario, you can get almost 100% of the combined bandwidth in both directions for a single connection.

This is a particular case of bonding/teaming/ling aggregation where multiple layer 2 (same network) interfaces are joined together. It could be achieved by establishing VPN layer 2 (tap) connections on each ISP interface from the computer to the host and bonding them together (round-robin mode) to have a single interface. The limiting factor in this scenario is how different are the delays (ping) on each ISP connection to the host. The more similar and stable they are, the better. We use it in one of our installations, it works well. If you would like to know the details about how to implement it just let me know.

Then another scenario would be without an intermediate host, i.e. a direct connection from your ISP interfaces to various webservers around the world. In this case the best you can get is to evenly distribute outgoing connections between the interfaces – i.e. one TCP session goes entirely via one ISP, a second session via another and so on. It is so because when you establish a TCP connection, it has origin and destination IP addresses for each packet and when a server receives a packet from another IP for which a TCP handshake was not performed, it considers the packet as erroneous and drops it. As each ISP connection has its own public IP, for the same TCP session you can't send one packet via one connection from one IP and another via another connection with another IP.

You won't achieve as high aggregate bandwidth utilization for a single computer as with the first scenario, but for a small office it could be a good solution. What you can do to extend it a little bit is to implement custom solutions for specific protocols. For example you could have some sort of a proxy on the gateway (which could be the same computer) for http downloads and ask for different parts of a huge file establishing different TCP sessions via different ISP interfaces. In this case the resulting download rate would be near 100% of the combined bandwidth. It's like offloading to the gateway what ReGet, GetRight and similar downloaders do. Google 'HTTP 206 Partial Content'. I don’t know any out-of-the-box open-source solutions for this scenario, but there are hardware appliances that do exactly this: google 'mushroom networks'.

  • Wow! The first part of your answer is exactly what I am looking for. I have a server in the cloud with Ubuntu and a very fast connection. And the only internet connection I can have here is 300kbps with limited 3G. Can you give more information onhow to achieve the first part of your answer, so that I can buy many 3G dongles and plans to improve the speed of my connection? – Huafu Jun 1 '15 at 22:23
  • 2
    echo "bonding" >> /etc/modules echo -e "alias bond* bonding\noptions bonding max_bonds=10 mode=2 xmit_hash_policy=layer3+4 arp_interval=100 arp_ip_target=10.0.0.1" > /etc/modprobe.d/bonding.conf – Anatoli Jun 3 '15 at 4:06
  • 1
    And enslave all the tap ifaces to this bond: echo "+tapX " >> /sys/class/net/bond0/bonding/slaves Check its state with: cat /proc/net/bonding/bond0 At this stage all internet traffic should flow via bond0 to the server in the datacenter. There you should set routing: echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf sysctl -p /etc/sysctl.conf iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE (eth0 is supposed to be the internet iface) – Anatoli Jun 3 '15 at 4:09
  • 1
    This bond configuration (mode=2 xmit_hash_policy=layer3+4) will make each new connection go via different underlying VPN iface based on the hash (calculated from the IPs and ports involved in the connection) mod iface number. More info: kernel.org/doc/Documentation/networking/bonding.txt. This actually is more like the second solution from my answer, it's because you need a very similar (very low jitter and same bandwidth) underlying connections for round-robin bonding to function correctly (otherwise you get a lot of packet reordering) and you say that your connections are 3G. – Anatoli Jun 3 '15 at 4:09
  • 1
    @Huafu, to have the first solution you just have to change the bonding options instead of mode=2 xmit_hash_policy=layer3+4 y specify mode=0 (see the link @ kernel.org), but first check how similar are the links via 3G. If their ping times differ more than 2-3ms or jitter is more than 1ms you'll have a lot of packet reordering effectively reducing the speed of the aggregated link. You'll have to check aggregated link statistics with netstat -s (look for retransmissions) and iperf -s / iperf -c <server_ip> -d. 1st check the performance of each link, then we can continue with a solution – Anatoli Jun 4 '15 at 16:40
0

i faced a similar problem.. and i was very interested in the approach to the solution according to the first scenario by mr. GTH and Anatoli, i very ask you to lay out the basic configs and scripts, if possible, to test the configuration described in the first scenario..

now i configured vpn connections through different ISP providers, using combined tun/tap interfaces (it's not bonding, what is explained in answer #8) with this utility:

Net-ISP-Balance by Lincoln D. Stein

Load-balance your Internet connection across two or more ISPs for improved bandwidth and reliability

Project home: https://lstein.github.io/Net-ISP-Balance/

This package allows you to load-balance a home or small business Internet connection across two or more ISPs. You may use it with a single host attached to two ISPs, or on a router/firewall machine to load balance your entire LAN. Network traffic is balanced across both ISP connections to increase upload and download capacity, and if one ISP fails, the other ISP(s) will take over automatically.

Bandwidth is distributed on a per-connection level. This means that you will not see the aggregated bandwidth on any particular download or speed benchmark, but you will see the benefits when multiple data transfers are occurring simultaneously, for example, when several individuals in your household are streaming movies. In addition, multi-connection file transfer protocols such as BitTorrent, will see the benefits of the load balancing.

this is a Perl based utility for managing routing and iptables in Linux perfect for our purposes, in fact, first creates a routing table for all providers, and then distributes all LAN traffic evenly between providers, to understand how the utility works, suggest to consider a small example (test configuration) for 3 isp + 1 lan

 #cat /etc/network/balance.conf
 ##service    device   role     ping-ip           
 CABLE3       enp0s3   isp      10.0.2.2
 CABLE8       enp0s8   isp      10.0.3.2
 CABLE9       enp0s9   isp      10.0.4.2
 LAN          enp0s10  lan                        

 #cat /etc/network/interfaces
auto enp0s3
allow-hotplug enp0s3
iface enp0s3 inet dhcp

auto enp0s8
allow-hotplug enp0s8
iface enp0s8 inet dhcp

auto enp0s9
allow-hotplug enp0s9
iface enp0s9 inet dhcp

auto enp0s10
allow-hotplug enp0s10
iface enp0s10 inet static
    address 192.168.1.1/24

#Now work Net-ISP-Balance utility:

## Including rules from /etc/network/balance/pre-run/pre-run-script.pl ##
## Finished /etc/network/balance/pre-run/pre-run-script.pl ##
echo 0 > /proc/sys/net/ipv4/ip_forward
ip route flush all
ip rule flush
ip rule add from all lookup main pref 32766
ip rule add from all lookup default pref 32767
ip route flush table  2
ip route flush table  1
ip route flush table  3
ip route add  10.0.2.0/24 dev enp0s3 src 10.0.2.15
ip route add  10.0.3.0/24 dev enp0s8 src 10.0.3.15
ip route add  10.0.4.0/24 dev enp0s9 src 10.0.4.15
ip route add  0.0.0.0/0 dev enp0s10 src 
ip route add default scope global nexthop via 10.0.4.2 dev enp0s9 weight 1 nexthop via 10.0.3.2 dev enp0s8 weight 1 nexthop via 10.0.2.2 dev enp0s3 weight 1
ip route add table 2 default dev enp0s3 via 10.0.2.2
ip route add table 2 10.0.2.0/24 dev enp0s3 src 10.0.2.15
ip route add table 2 10.0.3.0/24 dev enp0s8 src 10.0.3.15
ip route add table 2 10.0.4.0/24 dev enp0s9 src 10.0.4.15
ip route add table 2 0.0.0.0/0 dev enp0s10 src 
ip rule add from 10.0.2.15 table 2
ip rule add fwmark 2 table 2
ip route add table 1 default dev enp0s8 via 10.0.3.2
ip route add table 1 10.0.2.0/24 dev enp0s3 src 10.0.2.15
ip route add table 1 10.0.3.0/24 dev enp0s8 src 10.0.3.15
ip route add table 1 10.0.4.0/24 dev enp0s9 src 10.0.4.15
ip route add table 1 0.0.0.0/0 dev enp0s10 src 
ip rule add from 10.0.3.15 table 1
ip rule add fwmark 1 table 1
ip route add table 3 default dev enp0s9 via 10.0.4.2
ip route add table 3 10.0.2.0/24 dev enp0s3 src 10.0.2.15
ip route add table 3 10.0.3.0/24 dev enp0s8 src 10.0.3.15
ip route add table 3 10.0.4.0/24 dev enp0s9 src 10.0.4.15
ip route add table 3 0.0.0.0/0 dev enp0s10 src 
ip rule add from 10.0.4.15 table 3
ip rule add fwmark 3 table 3
## Including rules from /etc/network/balance/routes/01.local_routes ##
# enter any routing commands you might want to go in
# for example:
# ip route add 192.168.100.1 dev eth0 src 198.162.1.14

## Finished /etc/network/balance/routes/01.local_routes ##
## Including rules from /etc/network/balance/routes/02.local_routes.pl ##
## Finished /etc/network/balance/routes/02.local_routes.pl ##
iptables -F
iptables -t nat    -F
iptables -t mangle -F
iptables -X
iptables -P INPUT    DROP
iptables -P OUTPUT   DROP
iptables -P FORWARD  DROP

iptables -N DROPGEN
iptables -A DROPGEN -j LOG -m limit --limit 1/minute --log-level 4 --log-prefix "GENERAL: "
iptables -A DROPGEN -j DROP

iptables -N DROPINVAL
iptables -A DROPINVAL -j LOG -m limit --limit 1/minute --log-level 4 --log-prefix "INVALID: "
iptables -A DROPINVAL -j DROP

iptables -N DROPPERM
iptables -A DROPPERM -j LOG -m limit --limit 1/minute --log-level 4 --log-prefix "ACCESS-DENIED: "
iptables -A DROPPERM -j DROP

iptables -N DROPSPOOF
iptables -A DROPSPOOF -j LOG -m limit --limit 1/minute --log-level 4 --log-prefix "DROP-SPOOF: "
iptables -A DROPSPOOF -j DROP

iptables -N DROPFLOOD
iptables -A DROPFLOOD -m limit --limit 1/minute  -j LOG --log-level 4 --log-prefix "DROP-FLOOD: "
iptables -A DROPFLOOD -j DROP

iptables -N DEBUG
iptables -A DEBUG  -j LOG --log-level 3 --log-prefix "DEBUG: "
iptables -t mangle -N MARK-CABLE3
iptables -t mangle -A MARK-CABLE3 -j MARK     --set-mark 2
iptables -t mangle -A MARK-CABLE3 -j CONNMARK --save-mark
iptables -t mangle -N MARK-CABLE8
iptables -t mangle -A MARK-CABLE8 -j MARK     --set-mark 1
iptables -t mangle -A MARK-CABLE8 -j CONNMARK --save-mark
iptables -t mangle -N MARK-CABLE9
iptables -t mangle -A MARK-CABLE9 -j MARK     --set-mark 3
iptables -t mangle -A MARK-CABLE9 -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -i enp0s10 -m conntrack --ctstate NEW -m statistic --mode random --probability 1 -j MARK-CABLE9
iptables -t mangle -A PREROUTING -i enp0s10 -m conntrack --ctstate NEW -m statistic --mode random --probability 0.5 -j MARK-CABLE8
iptables -t mangle -A PREROUTING -i enp0s10 -m conntrack --ctstate NEW -m statistic --mode random --probability 0.333333333333333 -j MARK-CABLE3
iptables -t mangle -A PREROUTING -i enp0s10 -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i enp0s3 -m conntrack --ctstate NEW -j MARK-CABLE3
iptables -t mangle -A PREROUTING -i enp0s3 -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i enp0s8 -m conntrack --ctstate NEW -j MARK-CABLE8
iptables -t mangle -A PREROUTING -i enp0s8 -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -i enp0s9 -m conntrack --ctstate NEW -j MARK-CABLE9
iptables -t mangle -A PREROUTING -i enp0s9 -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -A INPUT  -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A INPUT -d 127.0.0.0/8 -j DROPPERM
iptables -A INPUT   -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT   -p tcp --tcp-flags SYN,ACK ACK -j ACCEPT
iptables -A FORWARD -p tcp --tcp-flags SYN,ACK ACK -j ACCEPT
iptables -A FORWARD -p tcp --tcp-flags SYN,ACK,FIN,RST RST -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 1/s -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j DROPFLOOD
iptables -A INPUT   -i enp0s10 -s 0.0.0.0/0 -j ACCEPT
iptables -A OUTPUT  -o enp0s10 -d 0.0.0.0/0  -j ACCEPT
iptables -A OUTPUT  -o enp0s10 -d 255.255.255.255/32  -j ACCEPT
iptables -A OUTPUT  -p udp -s 0.0.0.0/0 -j ACCEPT
iptables -A FORWARD  -i enp0s10 -o enp0s3 -s 0.0.0.0/0 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A OUTPUT   -o enp0s3                 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A FORWARD  -i enp0s10 -o enp0s8 -s 0.0.0.0/0 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A OUTPUT   -o enp0s8                 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A FORWARD  -i enp0s10 -o enp0s9 -s 0.0.0.0/0 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A OUTPUT   -o enp0s9                 ! -d 0.0.0.0/0 -j ACCEPT
iptables -A OUTPUT  -j DROPSPOOF
iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE
iptables -t nat -A POSTROUTING -o enp0s8 -j MASQUERADE
iptables -t nat -A POSTROUTING -o enp0s9 -j MASQUERADE
## Including rules from /etc/network/balance/firewall/01.accept ##
## This file contains iptables statements that add additional firewall rules

# allow incoming domain packets -- needed for DNS resolution
iptables -A INPUT   -p udp --source-port domain -j ACCEPT
# allow incoming NTP packets -- needed for net time protocol
iptables -A INPUT   -p udp --source-port ntp -j ACCEPT
## Finished /etc/network/balance/firewall/01.accept ##
## Including rules from /etc/network/balance/firewall/01.accept.pl ##
iptables -A INPUT -p tcp -s 0.0.0.0/0 --syn --dport ssh -j ACCEPT
## Finished /etc/network/balance/firewall/01.accept.pl ##
## Including rules from /etc/network/balance/firewall/02.forward.pl ##
iptables -A FORWARD -p udp --source-port domain -d 0.0.0.0/0 -j ACCEPT
iptables -A FORWARD -p udp --source-port ntp    -d 0.0.0.0/0 -j ACCEPT
## Finished /etc/network/balance/firewall/02.forward.pl ##
echo 1 > /proc/sys/net/ipv4/ip_forward
## Including rules from /etc/network/balance/post-run/post-run-script.pl ##
## Finished /etc/network/balance/post-run/post-run-script.pl ##
  • Please, consider deleting this answer and adding a comment requesting more explanation to the corresponding answer instead. This is not an answer. Thank you. – Raffa Aug 21 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.