How can you have a docker image layer of a different operating system

Question

I'm new-ish to docker and am trying to understand it a little better by working with from-scratch examples.

I wanted to start as primitively as I could think of: copying a file from my context to the root of my docker image, then validating the file's existence in the container.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.8 (stretch)
Release:        9.8
Codename:       stretch

$ ls -R
.:
demo.txt Dockerfile
$
$ cat Dockerfile
FROM scratch
WORKDIR .
COPY demo.txt /foo
$ 
$ docker build -t demo:v1 -f ./Dockerfile .
Sending build context to Docker daemon  28.67kB
Step 1/3 : FROM scratch
 --->
Step 2/3 : WORKDIR .
 ---> Using cache
 ---> 8eb9da711a99
Step 3/3 : COPY demo.txt /foo
 ---> c57e0e9a316b
Successfully built c57e0e9a316b
Successfully tagged demo:v1
$
$ docker run -it demo:v1 ls -l /foo
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"ls\": executable file not found in $PATH": unknown.

I understand the gist of the problem: FROM scratch is just that -- without even ls. So that's fixable by changing FROM scratch to something more meaty:

$ cat Dockerfile
FROM alpine:3.7
WORKDIR .
COPY demo.txt /foo
$ 
$ docker build -t demo:v1 -f ./Dockerfile .
Sending build context to Docker daemon  28.67kB
Step 1/3 : FROM alpine:3.7
 ---> 6d1ef012b567
Step 2/3 : WORKDIR .
 ---> Using cache
 ---> 51c22fe2fd60
Step 3/3 : COPY demo.txt /foo
 ---> Using cache
 ---> e2c241241653
Successfully built e2c241241653
Successfully tagged demo:v1
$ 
$ docker run -it demo:v1 ls -l /foo
-rw-r--r--    1 root     root             6 Sep  8 23:42 /foo

This bring me to my question: I'm confused by this concept of installing/running a different Linux distribution (alpine) than what is on my host (debian). What happens when I run my docker container built from an image that included FROM alpine:3.7 in its Dockerfile? Is a virtual machine that runs alpine Linux started on my physical PC that runs debian?

If I wanted to do something very primitive -- like ls -- in my docker container, how does one pick the most minimal base image? I picked alpine randomly -- not particularly knowing what I was doing -- when would one need to pick a different distro, like ubuntu, etc.?

Answer 1

I think your question is actually “Why do I even need this” and the simple answer is “Because it has to be self-contained”. Keep in mind that when a container is running, whatever is in it cannot access the host filesystem.

This is a problem because many programs require shared libraries. Let’s take a look at echo:

~$ ldd $(which echo)
        linux-vdso.so.1 (0x00007fffeec73000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1a57c80000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1a57e75000)

So even a program as simple as echo is not self-contained. It depends on libc. If I were to create a FROM scratch Docker image with just the echo binary in it, it could not run. Of course you could (re)compile some software to be statically linked, but that requires time.

Sometimes you also depend on other stuff like root certificates and letting an established distro take care of it is easier.

When you start a container, it does not “run the container OS”. Instead, in most simpler containers, only a single process is started (as defined on the command line or in Dockerfile). More complex containers tend to use their own “init” systems, because classic OS init systems are generally not suitable for use in containers.

Picking a suitable base image isn’t something that could be formalized. You need to think of the requirements and you need to research what is available. There’s really no way around that. You also may have personal preferences. For this particular case, maybe the busybox images would be suitable and quite minimal.

---

With a different type of container software (LXC/LXD) you can in fact run a whole second Linux distribution, with regular init system and whatnot, in a container. This is preferable to full virtualization if that much isolation isn’t required. It offers vastly superior performance, much like Docker.

Isolation using Linux namespaces is sufficient to allow many distributions to share the same running kernel. But it is sort-of valid to think of it as a lightweight virtual machine.

Answer 2

Docker images are like filesystems. A docker image would be a disk which contained an install of Alpine, Ubuntu, Arch Linux... even though your host is Debian. (you could also have images for Debian, or different versions of those, but it's easier to explain if they are completely different)

The trick here is that you are running the same kernel in all of them. Docker is a container solution. Your alpine-based machine will run on a Linux kernel distributed by Debian (constrained by docker to not see the outside world). Alpine will work happily with any Linux kernel new enough to run docker, so it simply works as if you had an Alpine (or Ubuntu, or Arch Linux...) install.

The downside is that you could not run BSD or Windows there, as their userland wouldn't be able to run with a Linux kernel. Plus obviously, they need to use compatible architecture.

What happens when I run my docker container built from an image that included FROM alpine:3.7 in its Dockerfile? Is a virtual machine that runs alpine Linux started on my physical PC that runs debian?

You start a new container which boots an image based on an alpine install, which will be very similar to booting an Alpine install.

Note that you could create a docker instance that did something else, such as starting a bash console for you to work with, and not launch the daemons that the distro would usually run at boot.

If I wanted to do something very primitive -- like ls -- in my docker container, how does one pick the most minimal base image? I picked alpine randomly --

If you wanted to be able to run ls, and only that, you could create an image that only contained ls (as it'd require libc, probably better as a copy of busybox-static). that might be overkill, though.

Alpine is used quite often for this, since it is a very small distribution.

not particularly knowing what I was doing -- when would one need to pick a different distro, like ubuntu, etc

If you needed something provided by that distro. Suppose you wanted to run a console program whose dependencies were stated as "needing packge X, Y and Z from Ubuntu M.N". It would thus make sense to run it on a docker instance based of an "Ubuntu M.N" image.

As you are working from a Debian host, you may be more comfortable working with Debian images, and it would make sense to base your docker images on a Debian one.

This is equivalent to the question "I need to install a new GNU/Linux server, which distribution shall I use?", which ultimately will be a personal choice of the person (or team) amongst a bunch of good options.