I want to use LXD to bootstrap a root filesystem for deployment on an embedded ARM system from my AMD64 host running ubuntu 16.04. Previously, I've done this with a script and the chroot command, but the script I was using was error-prone and had a bad habit of deleting my /dev entries.

I have been able to copy the image locally using lxc image copy images:ubuntu/16.04/arm64 --alias=ubuntu-server-arm64, and have installed qemu-user-static, but am not able to actually launch a container from this image. I get the following error:

$ lxc launch ubuntu-server-arm64 bootstrap
Creating bootstrap
error: Requested architecture isn't supported by this host

Is there a way to force lxd to ignore the architecture mismatch and use qemu-user-static to run the child container?


According to Stephane Graber who is in charge of the LXC/LXD project (in response to an inquiry from me on their discussions site):

qemu-user-static is a binfmt helper which lets you do on the fly conversion between architectures. It effectively lets you run binaries for architectures other than your current one.

qemu-user-static itself should work fine inside a container and will let you run some binaries of foreign architectures inside it.

Trying to run an entire container through qemu-user-static is very impractical due to some big limitations of qemu-user-static, for example, anything that relies on ptrace (init systems and debugging tools), netlink (all the network tools and some init systems) or thread (a lot more software) will usually fail miserably.

The approach I implemented back in LXC was to create a mixed container where most packages were of the foreign architecture but the init system, network tools, … were of the native architecture. This was kinda-working but also not particularly useful, not to mention very slow.

As this is effectively unsupportable as far as we’re concerned, we are not providing any foreign-architecture images for LXD. You could however build your own by assembling a rootfs of a foreign architecture, then including the needed qemu-user-static binaries, replacing any binary that will not work with emulation and generate this as a LXD image (marking it with the architecture it’s intended to run on rather than the architecture it contains).

So, effectively, attempting to run a foreign architecture on LXD is unsupported, and the LXC method was more or less a 'mixed environment' type of hack.

It sounds like it could also be done, but would require you to create a rootfs of a foreign architecture and basically create the 'image' yourself that runs all this, which for various reasons could be a headache and a lot of work to accomplish.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.