2

I have a testing setup I'm trying to migrate to using VMs rather than bare metal as much as possible. The existing setup is structured to run everything within containers (LXC, specifically), and it won't be going away so if at all possible I want to use the existing structure on the VMs. The VM cluster other services are running in is QEMU.

Naively this seems straightforward, but the last person on my team to try it gave up in confusion. So, here's what I want to know:

  • Are there known issues running LXC inside QEMU VMs in particular or virtual machines in general?

  • Are there obstacles I should expect, and if so, what?

  • Is it more likely to be problems about our idiosyncratic setup than general compatibility issues?

1 Answer 1

1

Compatibility

There should be no issues. QEMU doesn't look any different functionally from regular hardware to the guest system, and LXC does nothing that requires special behavior from the hardware that QEMU doesn't emulate.

But...

You probably don't want to do this, not only because it will hurt performance, but it will also significantly complicate management. You should probably re-evaluate why you think you need VM's under the containers (or, better yet, if you are already using containers for part of your infrastructure, why you need to use VM's instead for the rest of it). Unless you really care about live migration or improved isolation of workloads beyond what containers provide, you're probably better off just adapting the rest of your infrastructure to use containers because they use fewer resources. The only exception to this is systems which provide services that are accessible to the outside world, which should never be on a shared host unless they're in a VM (because crashing a container will quite often crash the host), and even then should generally not be on a shared host with internal services.

OK, but I really want to do this anyway.

In that case, work on adapting your software to use VM's instead of containers. Realistically, it shouldn't be too hard to adapt. It's actually pretty easy to set QEMU up to directly boot a Linux kernel (and therefore bypass the need for a bootloader, and with it the need for a partition table), and once you can do that it's easy to extract that container base image into a filesystem image which you can then use directly as a disk (this would still be a bit lower performance than the containers, but nowhere near as bad as running the containers inside the VM's).

1
  • I don't have control over the machines running the VMs, and cannot choose to make them run containers instead. VMs are not an option for the previously existing bare-metal tests, so they must continue using containers. So I do want to do this, because the alternative is having two parallel infrastructures to run the same tests in different contexts. Commented Apr 26, 2018 at 19:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .