Getting docker build to show IDs of intermediate containers

Question

I'm trying to learn docker but experience some discrepancies to what I've read in each tutorial:

When using the following command docker build -t my-app:1.0 . my build fails due to some error in my Dockerfile. Following this answer, I wanted to run the last intermediate container ID. Anyhow, in contrast to all tutorials I've seen so far, my console is not showing any intermediate container IDs:

I'm running Docker 19 on Windows 10. How do I get the intermediate container IDs?

Answer 1

Since version 18.09 Docker has added a new backend for building images, buildkit. This offers a bunch of benefits but doesn't expose intermediate containers in the same way as the old backend. Fortunately you can disable buildkit.

You don't need to disable buildkit permanently for this and you shouldn't because it will make all your builds slower. You can just set an environment variable when you docker build, like this:

Powershell

$env:DOCKER_BUILDKIT=0; docker build .

Linux/macOS

DOCKER_BUILDKIT=0 docker build .

Windows cmd

set DOCKER_BUILDKIT=0& docker build .

Answer 2

I had the same problem. Setting "buildkit" to false in ~/.docker/daemon.json (In Windows you should find daemon.json in C:\ProgramData\Docker\config) solved this for me:

    {
      "experimental": true,
      "features": {
        "buildkit": false
      }
    }

Answer 3

I'm not sure of the reasoning exactly but with Buildkit you can no longer inspect at a particular cache layer like before. You just have to comment out where the RUN command is failing, get it to build successfully – then you can inspect it. A step backwards IMO. Even turning on --progress=plain puts out hashes, but you can't do anything with them. ie: trying to run docker run -ti [hash_id] sh fails as it can't find the image. It's that or you turn off buildkit, but the fact that it is both the default for macOS, and Windows suggests that changing the default of the daemon may not be good for future use.

Answer 4

Expanding the Answer of Michael, for anyone that may be a little bit lost. If you are using the docker desktop on windows, you can find this configuration of daemon.json inside the settings window under the "Docker Engine" tab. Change the json and Apply & Restart.

Settings window on the Docker Engine tab

Answer 5

While using "DOCKER_BUILDKIT=0" to debug a failure. I saw a warning that the legacy builder was going to be depreciated. So this method of debugging will also be rendered non-functional. Hopefully there will be another way of doing this in the future. I know I can edit the dockerfile to the point where the error occurs, just less error prone to not have to edit it.

Answer 6

This is not as much an answer, more a a hackish workaround enabling access to the filesystem after a given build step.

Using docker buildx du --verbose you find get information about all stored intermediate images. This might look like this:

ID:             l9p9z4z73glqo8zqeng1dz86j
Parent:         c4jbrdmcszz1rd6exxwnptcei
Created at:     2024-02-29 11:18:08.588296224 +0000 UTC
Mutable:        false
Reclaimable:    true
Shared:         true
Size:           8.439kB
Description:    [image 5/5] COPY ./docker-entrypoint.sh /docker-entrypoint.sh
Usage count:    1
Last used:      11 minutes ago
Type:           regular

The ID field refers to an containerd snapshot.It is possible to mount this snapshot using ctr(8):

# eval $( ctr -n moby snapshot mount /mnt l9p9z4z73glqo8zqeng1dz86j )

Then you can inspect the content of the filesystem right after the COPY command.