0

I have a running Debian Wheezy setup I copied exactly to an Ubuntu 12.04

While the Debian setup runs flawlessly, the Ubuntu version dies on the prerouting to containers ( or so it seems )

In short:

  • lxc works
  • containers work and run
  • connecting to container from host OK ( including mixed ports & services )
  • connecting to outside world from container is fine

What does not work is connecting from another box to the host on a port that should be NATed to a container.

The setups: /etc/rc.local

CMD_BRCTL=/sbin/brctl
CMD_IFCONFIG=/sbin/ifconfig
CMD_IPTABLES=/sbin/iptables
CMD_ROUTE=/sbin/route
NETWORK_BRIDGE_DEVICE_NAT=lxc-bridge
HOST_NETDEVICE=eth0
PRIVATE_GW_NAT=192.168.42.1
PRIVATE_NETMASK=255.255.255.0
PUBLIC_IP=192.168.13.100
${CMD_BRCTL} addbr ${NETWORK_BRIDGE_DEVICE_NAT}
${CMD_BRCTL} setfd ${NETWORK_BRIDGE_DEVICE_NAT} 0
${CMD_IFCONFIG} ${NETWORK_BRIDGE_DEVICE_NAT} ${PRIVATE_GW_NAT} netmask ${PRIVATE_NETMASK} promisc up

Therefore lxc network is 192.168.42.0/24 and the host eth0 ip is 192.168.13.100; setup via network manager as static address.

iptables:

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT

*filter
:FORWARD ACCEPT [0:0]
:INPUT DROP [0:0]
:OUTPUT ACCEPT [0:0]

# Accept traffic from internal interfaces
-A INPUT -i lo -j ACCEPT
# accept traffic from lxc network
-A INPUT -d 192.168.42.1 -s 192.168.42.0/24 -j ACCEPT
# Accept internal traffic Make sure NEW incoming tcp connections are SYN 
# packets; otherwise we need to drop them:
-A INPUT -p tcp ! --syn -m state --state NEW -j DROP
# Packets with incoming fragments drop them. This attack result into Linux server panic such data loss.
-A INPUT -f -j DROP
# Incoming malformed XMAS packets drop them:
-A INPUT -p tcp --tcp-flags ALL ALL -j DROP
# Incoming malformed NULL packets:
-A INPUT -p tcp --tcp-flags ALL NONE -j DROP
# Accept traffic with the ACK flag set
-A INPUT -p tcp -m tcp --tcp-flags ACK ACK -j ACCEPT
# Allow incoming data that is part of a connection we established
-A INPUT -m state --state ESTABLISHED -j ACCEPT
# Allow data that is related to existing connections
-A INPUT -m state --state RELATED -j ACCEPT
# Accept responses to DNS queries
-A INPUT -p udp -m udp --dport 1024:65535 --sport 53 -j ACCEPT
# Accept responses to our pings
-A INPUT -p icmp -m icmp --icmp-type echo-reply -j ACCEPT
# Accept notifications of unreachable hosts
-A INPUT -p icmp -m icmp --icmp-type destination-unreachable -j ACCEPT
# Accept notifications to reduce sending speed
-A INPUT -p icmp -m icmp --icmp-type source-quench -j ACCEPT
# Accept notifications of lost packets
-A INPUT -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
# Accept notifications of protocol problems
-A INPUT -p icmp -m icmp --icmp-type parameter-problem -j ACCEPT

# Respond to pings, but limit
-A INPUT -m icmp -p icmp --icmp-type echo-request -m state --state NEW -m limit --limit 6/s -j ACCEPT

# Allow connections to SSH server
-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -m limit --limit 12/s -j ACCEPT

COMMIT

*nat
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -m limit --limit 12/s -j DNAT --to-destination 192.168.42.11:22
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 80 -m state --state NEW -m limit --limit 512/s -j DNAT --to-destination 192.168.42.11:80
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 443 -m state --state NEW -m limit --limit 512/s -j DNAT --to-destination 192.168.42.11:443

-A POSTROUTING -d 192.168.42.0/24 -o eth0 -j SNAT --to-source 192.168.13.100
-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

sysctl:

net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.ip_forward = 1

I've set up full iptables log on the container; none of the packets addressed to 192.168.13.100, port 80 is reaching the container.

I've even tried different kernels ( server kernel, raring lts kernel, etc ), modprobe everything iptables & nat related, nothing. Any ideas?

1

OK, that was a nasty "bug" By default, lxc sets up the internal network based on 255.0.0.0 netmask. Therefore the internal lxc network was trying to find a host within the lxc network for 192.168.13.0 as well, never coming back to the host.

The fix: in the container config file, the ip address should be like 192.168.42.10/24 instead of 192.168.42.10.

tcpdump is a handy tool.

0

You have "-m limit --limit x/s" in your *nat table. Unfortunately features like limit & drop do not work in *nat. You would need to set a limit on the iptables INPUT of the 192.168.42.11 lxc.

One method that works though if you have to block it before it hits the DNAT.

-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -m connlimit --connlimit-above 10 --connlimit-mask 32 -j DNAT --to-destination 127.0.0.1
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -j DNAT --to-destination 192.168.42.11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.