External DHCP with LXD 2.3+

Question

I am trying out LXD. It seems the default way to set up networking is to have the host provide DHCP and NAT for all the guests, but I'd like to omit NAT and allow the guests to use the same DHCP server as the host. This seems like it should be a pretty reasonable thing to want, but I can't get it working. No matter what I do, the guest never gets an address assigned by the external DHCP server.

To make things harder, it looks like LXD's network configuration changed pretty majorly with LXD 2.3, and the vast majority of information I find out there is for earlier versions. I did find this one GitHub issue where someone was trying to do the same thing and posted a new-style configuration that worked for them, but it didn't seem to make a difference when I tried it.

So, the question is, How do I set up a bridge in LXD 2.3+ without NAT and using external DHCP?

Here is some additional detail:

Ubuntu 18.04 with LXD 3.0.0
Host uses interface enp10s0f0
Want to bridge guests to interface enp10s0f1 (which is not otherwise used by the host)
Additional interface enp1s2 unused

dominickpastore@ubuntu:~$ lxc network list
+-----------+----------+---------+-------------+---------+
|   NAME    |   TYPE   | MANAGED | DESCRIPTION | USED BY |
+-----------+----------+---------+-------------+---------+
| enp10s0f0 | physical | NO      |             | 0       |
+-----------+----------+---------+-------------+---------+
| enp10s0f1 | physical | NO      |             | 0       |
+-----------+----------+---------+-------------+---------+
| enp1s2    | physical | NO      |             | 0       |
+-----------+----------+---------+-------------+---------+
| lxdbr0    | bridge   | YES     |             | 1       |
+-----------+----------+---------+-------------+---------+

Note that this is just my current configuration attempt for lxdbr0; I've tried many:

dominickpastore@ubuntu:~$ lxc network show lxdbr0
config:
  bridge.external_interfaces: enp10s0f1
  ipv4.address: none
  ipv4.dhcp: "false"
  ipv6.address: none
  ipv6.dhcp: "false"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/first
managed: true
status: Created
locations:
- none

Update:

I have been playing with this more and it's leaving me even more confused. I'm getting seemingly very erratic behavior. Is this still kind of new and flaky, or am I just doing stuff wrong? For one thing, lxc network unset doesn't seem to work:

dominickpastore@ubuntu:~$ lxc network show lxdbr0
config:
  ipv4.address: 10.4.4.1/24
  ipv4.dhcp: "true"
  ipv4.dhcp.ranges: 192.168.4.230-192.168.4.249
  ipv4.nat: "true"
  ipv4.routing: "true"
  ipv6.address: none
  ipv6.dhcp: "false"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/first
managed: true
status: Created
locations:
- none
dominickpastore@ubuntu:~$ lxc network unset lxdbr0 ipv4.dhcp.ranges
dominickpastore@ubuntu:~$ lxc network show lxdbr0
config:
  ipv4.address: 10.4.4.1/24
  ipv4.dhcp: "true"
  ipv4.dhcp.ranges: 192.168.4.230-192.168.4.249
  ipv4.nat: "true"
  ipv4.routing: "true"
  ipv6.address: none
  ipv6.dhcp: "false"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/first
managed: true
status: Created
locations:
- none

I've also created a second bridge to test two configurations at once, and I can set them very similarly to a NAT+DHCP configuration (not exactly the same because unset isn't working) yet only the original interface gets assigned an address. I can't figure out why the new interface can't get an address at all. Notice the similar IPv4 configuration:

dominickpastore@ubuntu:~$ lxc network show lxdbr0
config:
  ipv4.address: 10.4.4.1/24
  ipv4.dhcp: "true"
  ipv4.dhcp.ranges: 192.168.4.230-192.168.4.249
  ipv4.nat: "true"
  ipv4.routing: "true"
  ipv6.address: none
  ipv6.dhcp: "false"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/first
managed: true
status: Created
locations:
- none
dominickpastore@ubuntu:~$ lxc network show lxdbr1
config:
  ipv4.address: 10.251.34.1/24
  ipv4.dhcp: "true"
  ipv4.nat: "true"
  ipv4.routing: "true"
  ipv6.address: none
  ipv6.nat: "false"
description: ""
name: lxdbr1
type: bridge
used_by:
- /1.0/containers/first
managed: true
status: Created
locations:
- none

Yet, in the container, eth0 works (connected to lxdbr0) and eth1 never gets an address (connected to lxdbr1):

root@first:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
41: eth0@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:64:0e:ea brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.4.4.245/24 brd 10.4.4.255 scope global dynamic eth0
       valid_lft 3101sec preferred_lft 3101sec
    inet6 fe80::216:3eff:fe64:eea/64 scope link
       valid_lft forever preferred_lft forever
43: eth1@if44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:fe:6d:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fefe:6d01/64 scope link
       valid_lft forever preferred_lft forever

Simos · Accepted Answer · 2018-05-12 09:21:35Z

3

The default network setting for LXD is to use a private bridge, lxdbr0.

To have some of your containers get an IP address from your LAN, you can either

Use bridged networking, bridging on the host's Ethernet interface. See for example, https://blog.simos.info/how-to-make-your-lxd-containers-get-ip-addresses-from-your-lan-using-a-bridge/
Use macvlan. See, for example, https://blog.simos.info/how-to-make-your-lxd-container-get-ip-addresses-from-your-lan/

Between the two, the macvlan is very easy to set up. You can have some containers on macvlan, while others using private networking. You can use profiles in LXD to specify which is what. You can even switch from one to another by applying/attaching a different profile.

An issue with macvlan is that the containers on mavclan cannot contact the host over the network.

An issue with both macvlan and bridged networking (on host's Ethernet interface) is that neither works on WiFi. Because the default settings on WiFi is for the access point to accept only a single MAC address from an associated network client. You might get around this limitation if you do not set any security features on the access point.

answered May 12, 2018 at 9:21

Simos

9047 silver badges8 bronze badges

#1 does look like what I was trying to accomplish, except it requires you to set up the bridge manually. I was hoping to use the new(ish) lxc network commands to do it. They are supposed to be able to create and manage a bridge for you, and it seems like it's supposed to let you turn off LXD's NAT and DHCP if you want to (based on what I see in the docs). Do you know if that's possible?
– Dominick Pastore
Commented May 12, 2018 at 18:18
As far as I know, LXD cannot create a (public) bridge on your host. That would require LXD to reconfigure the network settings of the host, and it's very difficult to implement such a thing. The host might use NetworkManager or it might use something less primitive.
– Simos
Commented May 13, 2018 at 19:30
I agree it would be hard for LXD to reconfigure network settings, but it seems to be what the new API (well, since 2.3, according to the blog post) does. At the very least, it can create some sort of bridge interface and assign it an IP address. You can see it clear as day with ip link list and ip addr before and after using the lxc network ... commands. Or at least that's what I was seeing on my fresh Ubuntu 18.04 server install (i.e. no NetworkManager). Now, whether LXD can configure that bridge the way I want is a different matter--but it does look like it's intended to be flexible.
– Dominick Pastore
Commented May 14, 2018 at 2:30
@Dominick I would love to sort that out. If you can give me a list of commands to reproduce, it would be great. I'ld be happy to update blog.simos.info/… if you can find a way to get lxc network to create the bridge in the tutorial.
– Simos
Commented May 15, 2018 at 10:20
The problem is I'm not having much success with the commands. They are creating a bridge and even configuring it some if I tell LXD to do so, but most of the time, the guest still won't connect to the network. I'm not convinced it can't work, though. I'm starting to think there's something else wrong, because I followed your macvlan instructions too (which are very nice, by the way) and still couldn't get a connection. I'll have to dig deeper. In the meantime, I'll post some commands that reproduce some sort of bridge creation later today.
– Dominick Pastore
Commented May 15, 2018 at 13:57

| Show 2 more comments

AnrDaemon · Accepted Answer · 2018-09-29 18:01:13Z

0

Setup containers with macvlan bridged mode interfaces bound to host's physical interface.
Add macvlan bridged interface on host on the same interface.

This way, you will be able to talk to host from containers et vice versa.

If you need macvlan ifupdown helper, I have one https://pastebin.com/yaRH8zC9

edited Sep 29, 2018 at 18:01

answered Sep 28, 2018 at 14:19

AnrDaemon

1135 bronze badges

Add a comment |

Stack Exchange Network

External DHCP with LXD 2.3+

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
dhcp
network-bridge
lxd
.

Hot Network Questions

External DHCP with LXD 2.3+

2 Answers 2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged dhcpnetwork-bridgelxd.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
dhcp
network-bridge
lxd
.