0

I have a running Debian Wheezy setup I copied exactly to an Ubuntu 12.04

While the Debian setup runs flawlessly, the Ubuntu version dies on the prerouting to containers ( or so it seems )

In short:

  • lxc works
  • containers work and run
  • connecting to container from host OK ( including mixed ports & services )
  • connecting to outside world from container is fine

What does not work is connecting from another box to the host on a port that should be NATed to a container.

The setups: /etc/rc.local

CMD_BRCTL=/sbin/brctl
CMD_IFCONFIG=/sbin/ifconfig
CMD_IPTABLES=/sbin/iptables
CMD_ROUTE=/sbin/route
NETWORK_BRIDGE_DEVICE_NAT=lxc-bridge
HOST_NETDEVICE=eth0
PRIVATE_GW_NAT=192.168.42.1
PRIVATE_NETMASK=255.255.255.0
PUBLIC_IP=192.168.13.100
${CMD_BRCTL} addbr ${NETWORK_BRIDGE_DEVICE_NAT}
${CMD_BRCTL} setfd ${NETWORK_BRIDGE_DEVICE_NAT} 0
${CMD_IFCONFIG} ${NETWORK_BRIDGE_DEVICE_NAT} ${PRIVATE_GW_NAT} netmask ${PRIVATE_NETMASK} promisc up

Therefore lxc network is 192.168.42.0/24 and the host eth0 ip is 192.168.13.100; setup via network manager as static address.

iptables:

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT

*filter
:FORWARD ACCEPT [0:0]
:INPUT DROP [0:0]
:OUTPUT ACCEPT [0:0]

# Accept traffic from internal interfaces
-A INPUT -i lo -j ACCEPT
# accept traffic from lxc network
-A INPUT -d 192.168.42.1 -s 192.168.42.0/24 -j ACCEPT
# Accept internal traffic Make sure NEW incoming tcp connections are SYN 
# packets; otherwise we need to drop them:
-A INPUT -p tcp ! --syn -m state --state NEW -j DROP
# Packets with incoming fragments drop them. This attack result into Linux server panic such data loss.
-A INPUT -f -j DROP
# Incoming malformed XMAS packets drop them:
-A INPUT -p tcp --tcp-flags ALL ALL -j DROP
# Incoming malformed NULL packets:
-A INPUT -p tcp --tcp-flags ALL NONE -j DROP
# Accept traffic with the ACK flag set
-A INPUT -p tcp -m tcp --tcp-flags ACK ACK -j ACCEPT
# Allow incoming data that is part of a connection we established
-A INPUT -m state --state ESTABLISHED -j ACCEPT
# Allow data that is related to existing connections
-A INPUT -m state --state RELATED -j ACCEPT
# Accept responses to DNS queries
-A INPUT -p udp -m udp --dport 1024:65535 --sport 53 -j ACCEPT
# Accept responses to our pings
-A INPUT -p icmp -m icmp --icmp-type echo-reply -j ACCEPT
# Accept notifications of unreachable hosts
-A INPUT -p icmp -m icmp --icmp-type destination-unreachable -j ACCEPT
# Accept notifications to reduce sending speed
-A INPUT -p icmp -m icmp --icmp-type source-quench -j ACCEPT
# Accept notifications of lost packets
-A INPUT -p icmp -m icmp --icmp-type time-exceeded -j ACCEPT
# Accept notifications of protocol problems
-A INPUT -p icmp -m icmp --icmp-type parameter-problem -j ACCEPT

# Respond to pings, but limit
-A INPUT -m icmp -p icmp --icmp-type echo-request -m state --state NEW -m limit --limit 6/s -j ACCEPT

# Allow connections to SSH server
-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -m limit --limit 12/s -j ACCEPT

COMMIT

*nat
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -m limit --limit 12/s -j DNAT --to-destination 192.168.42.11:22
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 80 -m state --state NEW -m limit --limit 512/s -j DNAT --to-destination 192.168.42.11:80
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 443 -m state --state NEW -m limit --limit 512/s -j DNAT --to-destination 192.168.42.11:443

-A POSTROUTING -d 192.168.42.0/24 -o eth0 -j SNAT --to-source 192.168.13.100
-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

sysctl:

net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.ip_forward = 1

I've set up full iptables log on the container; none of the packets addressed to 192.168.13.100, port 80 is reaching the container.

I've even tried different kernels ( server kernel, raring lts kernel, etc ), modprobe everything iptables & nat related, nothing. Any ideas?

2 Answers 2

1

OK, that was a nasty "bug" By default, lxc sets up the internal network based on 255.0.0.0 netmask. Therefore the internal lxc network was trying to find a host within the lxc network for 192.168.13.0 as well, never coming back to the host.

The fix: in the container config file, the ip address should be like 192.168.42.10/24 instead of 192.168.42.10.

tcpdump is a handy tool.

0

You have "-m limit --limit x/s" in your *nat table. Unfortunately features like limit & drop do not work in *nat. You would need to set a limit on the iptables INPUT of the 192.168.42.11 lxc.

One method that works though if you have to block it before it hits the DNAT.

-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -m connlimit --connlimit-above 10 --connlimit-mask 32 -j DNAT --to-destination 127.0.0.1
-A PREROUTING -d 192.168.13.100 -p tcp -m tcp --dport 2221 -m state --state NEW -j DNAT --to-destination 192.168.42.11:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .