Setting Static IP and NAT trafic to LXC container

Question

This is my first question in here so please have mercy of structured badly compared to the usual suspects. :-)

I would like to be able to receive real connecting client IP in a LXC container over TCP. Right now it is not possible with my current config using an attached network bridge pretty default assigning dynamic IP adresses and trafic is routed using the LXD network type proxy.

The setup

• Ubuntu 18.04 LTS.
• LXD installed as SNAP (migrated to v. 4.1).
• HAProxy installed as native app.
• The container in scope running on same host as HAProxy.

Container

• Ubuntu 18.04
• Apache2 (serving a Nextcloud install).
• HTTPS terminates in Apache2 in container

HAProxy passes SSL trafic through to the container without terminating it however does so by inspecting the connect header using req_ssl_sni. HAProxy supports the Proxy Protocol also for the TCP stream (v2 I beleive) as it can add the client IP to the TCP header so that any downstream proxy or application which also understands this part of the protocol, can extract that from the header before the connect hits the actual webserver config. This works also for Apache2 with the right module enabled and configured but not íf behind LXD proxy. Even if I add the proxy_protocol=true then it only understands the HTTP connects as there it can access and read the HTTP headers (forward for).

I can get this to work by changing the attached network device to be of the type NAT so the LXD network bridge does not act as reverse proxy but merely do "old school" port forwarding. However I have a ahrd time to make this work "permanently" (is overriden and conatiner cannot start after a restart).

Here is the question

How do I set a DHCP scope for the LXD network config, a fixed DHCP assignment based on container alias and then adds the correct network device to the container so all I have to route to in HAProxy is the LXD bridge address (Gateway address)?

Remember that LXD is installed and running as snap so that fancy (elegant) lxc-net trick is pretty hard to accomplish.

Answer 1

It is a common and expected issue to lose the client IP address when the connection goes through a set of proxies. Because an IP packet only keeps the very last source IP address.

It is difficult to say what's going on in your specific case. The important thing is that it can be done. Here is a tutorial using LXD proxy devices, and using nginx as the reverse proxy, https://www.linode.com/docs/applications/containers/beginners-guide-to-lxd-reverse-proxy/ If you try such a tutorial verbatim, it will help you figure out any missteps.

On a hunch, what could be wrong in your case, is that Apache2 is brain-dead and the default LogFormat does not log the proper client IP address in the access.log. You need to change a %h to an %a. See the tutorial above for more details.