I'm having an issue on fully updated Debian 11 (bullseye), and I've exhausted my troubleshooting knowledge on the subject. I'm currently running 3 PCs all with nearly identical configurations managed by Ansible. One, and only one, of these three systems is throwing an error trying to apt install docker.io either through Ansible (as root) or manually (e.g. sudo apt install docker.io):
~ » sudo apt install docker.io
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
linux-image-5.10.0-15-amd64
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
cgroupfs-mount containerd libintl-perl libintl-xs-perl libmodule-find-perl libmodule-scandeps-perl libproc-processtable-perl libsort-naturally-perl libterm-readkey-perl needrestart runc tini
Suggested packages:
containernetworking-plugins docker-doc aufs-tools btrfs-progs debootstrap rinse rootlesskit xfsprogs zfs-fuse | zfsutils-linux needrestart-session | libnotify-bin iucode-tool
Recommended packages:
criu
The following NEW packages will be installed:
cgroupfs-mount containerd docker.io libintl-perl libintl-xs-perl libmodule-find-perl libmodule-scandeps-perl libproc-processtable-perl libsort-naturally-perl libterm-readkey-perl needrestart runc tini
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 40.0 MB/61.4 MB of archives.
After this operation, 270 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Err:1 http://deb.debian.org/debian bullseye/main amd64 runc amd64 1.0.0~rc93+ds1-5+b2
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:2 http://deb.debian.org/debian bullseye/main amd64 docker.io amd64 20.10.5+dfsg1-1+deb11u1
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:3 http://deb.debian.org/debian bullseye/main amd64 libintl-perl all 1.26-3
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:4 http://deb.debian.org/debian bullseye/main amd64 libintl-xs-perl amd64 1.26-3
404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/r/runc/runc_1.0.0%7erc93%2bds1-5%2bb2_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/d/docker.io/docker.io_20.10.5%2bdfsg1-1%2bdeb11u1_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/libi/libintl-perl/libintl-perl_1.26-3_all.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/libi/libintl-perl/libintl-xs-perl_1.26-3_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
What I've tried
I've verified that I'm connected to the Internet, I can connect to http://deb.debian.org/debian in Firefox on the affected machine, I can update Apt's package info via sudo apt update, and I can upgrade packages via sudo apt upgrade.
Edit: sudo apt update returns the following output:
~ » sudo apt update
Hit:1 http://deb.debian.org/debian bullseye InRelease
Hit:2 http://security.debian.org/debian-security bullseye-security InRelease
Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
Hit:4 http://deb.debian.org/debian bullseye-backports InRelease
Hit:5 https://updates.signal.org/desktop/apt xenial InRelease
Hit:6 https://packages.element.io/debian default InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
I've waited at least a week to see if maybe there was an upstream problem that might be fixed by an update. No change.
I've searched for similar questions here and read the top dozen or so; I haven't seen anyone ask and resolve a similar issue.
I've tried digging into available versions via apt. Running apt show docker.io --all-versions on all three machines yields:
On working machine "M": 20.10.5+dfsg1-1+deb11u2
On working machine "Z": 20.10.5+dfsg1-1+deb11u2
On broken machine "T": 20.10.5+dfsg1-1+deb11u1
I've tried manually installing the version that machines M and Z have:
~ » sudo apt install docker.io=20.10.5+dfsg1-1+deb11u2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Version '20.10.5+dfsg1-1+deb11u2' for 'docker.io' was not found
I've tried using --fix-missing. Full output:
~ » sudo apt install docker.io --fix-missing
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
linux-image-5.10.0-15-amd64
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
cgroupfs-mount containerd libintl-perl libintl-xs-perl libmodule-find-perl
libmodule-scandeps-perl libproc-processtable-perl libsort-naturally-perl libterm-readkey-perl
needrestart runc tini
Suggested packages:
containernetworking-plugins docker-doc aufs-tools btrfs-progs debootstrap rinse rootlesskit
xfsprogs zfs-fuse | zfsutils-linux needrestart-session | libnotify-bin iucode-tool
Recommended packages:
criu
The following NEW packages will be installed:
cgroupfs-mount containerd docker.io libintl-perl libintl-xs-perl libmodule-find-perl
libmodule-scandeps-perl libproc-processtable-perl libsort-naturally-perl libterm-readkey-perl
needrestart runc tini
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 40.0 MB/61.4 MB of archives.
After this operation, 270 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Err:1 http://deb.debian.org/debian bullseye/main amd64 runc amd64 1.0.0~rc93+ds1-5+b2
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:2 http://deb.debian.org/debian bullseye/main amd64 docker.io amd64 20.10.5+dfsg1-1+deb11u1
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:3 http://deb.debian.org/debian bullseye/main amd64 libintl-perl all 1.26-3
404 Not Found [IP: 2a04:4e42:a::644 80]
Err:4 http://deb.debian.org/debian bullseye/main amd64 libintl-xs-perl amd64 1.26-3
404 Not Found [IP: 2a04:4e42:a::644 80]
Selecting previously unselected package cgroupfs-mount.
(Reading database ... 199957 files and directories currently installed.)
Preparing to unpack .../0-cgroupfs-mount_1.4_all.deb ...
Unpacking cgroupfs-mount (1.4) ...
Selecting previously unselected package containerd.
Preparing to unpack .../1-containerd_1.4.13~ds1-1~deb11u2_amd64.deb ...
Unpacking containerd (1.4.13~ds1-1~deb11u2) ...
Selecting previously unselected package libmodule-find-perl.
Preparing to unpack .../2-libmodule-find-perl_0.15-1_all.deb ...
Unpacking libmodule-find-perl (0.15-1) ...
Selecting previously unselected package libmodule-scandeps-perl.
Preparing to unpack .../3-libmodule-scandeps-perl_1.30-1_all.deb ...
Unpacking libmodule-scandeps-perl (1.30-1) ...
Selecting previously unselected package libproc-processtable-perl.
Preparing to unpack .../4-libproc-processtable-perl_0.59-2+b1_amd64.deb ...
Unpacking libproc-processtable-perl (0.59-2+b1) ...
Selecting previously unselected package libsort-naturally-perl.
Preparing to unpack .../5-libsort-naturally-perl_1.03-2_all.deb ...
Unpacking libsort-naturally-perl (1.03-2) ...
Selecting previously unselected package libterm-readkey-perl.
Preparing to unpack .../6-libterm-readkey-perl_2.38-1+b2_amd64.deb ...
Unpacking libterm-readkey-perl (2.38-1+b2) ...
Selecting previously unselected package tini.
Preparing to unpack .../7-tini_0.19.0-1_amd64.deb ...
Unpacking tini (0.19.0-1) ...
Setting up libmodule-find-perl (0.15-1) ...
Setting up tini (0.19.0-1) ...
Setting up libproc-processtable-perl (0.59-2+b1) ...
Setting up cgroupfs-mount (1.4) ...
Setting up libterm-readkey-perl (2.38-1+b2) ...
dpkg: dependency problems prevent configuration of containerd:
containerd depends on runc (>= 0.1.0~); however:
Package runc is not installed.
dpkg: error processing package containerd (--configure):
dependency problems - leaving unconfigured
Setting up libsort-naturally-perl (1.03-2) ...
Setting up libmodule-scandeps-perl (1.30-1) ...
Processing triggers for man-db (2.9.4-2) ...
Errors were encountered while processing:
containerd
E: Failed to fetch http://deb.debian.org/debian/pool/main/r/runc/runc_1.0.0%7erc93%2bds1-5%2bb2_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/d/docker.io/docker.io_20.10.5%2bdfsg1-1%2bdeb11u1_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/libi/libintl-perl/libintl-perl_1.26-3_all.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Failed to fetch http://deb.debian.org/debian/pool/main/libi/libintl-perl/libintl-xs-perl_1.26-3_amd64.deb 404 Not Found [IP: 2a04:4e42:a::644 80]
E: Sub-process /usr/bin/dpkg returned an error code (1)
(The system is then left in a broken state, with Apt suggesting I run apt --fix-broken install. That fails, due to the same issues. To restore the system, I have to run apt remove containerd and apt autoremove for good measure.)
I've opened https://deb.debian.org/debian/pool/main/d/docker.io/ in Firefox and observed that version 20.10.5+dfsg1-1+deb11u1 has no .deb files. Still, I don't know why this particular system won't install +deb11u2 instead.
Based on this answer, I've run apt-cache policy:
# Working machine M
~ » apt-cache policy docker.io
docker.io:
Installed: 20.10.5+dfsg1-1+deb11u2
Candidate: 20.10.5+dfsg1-1+deb11u2
Version table:
*** 20.10.5+dfsg1-1+deb11u2 500
500 http://deb.debian.org/debian bullseye/main amd64 Packages
100 /var/lib/dpkg/status
# Working machine Z
~ » apt-cache policy docker.io
docker.io:
Installed: 20.10.5+dfsg1-1+deb11u2
Candidate: 20.10.5+dfsg1-1+deb11u2
Version table:
*** 20.10.5+dfsg1-1+deb11u2 500
500 http://deb.debian.org/debian bullseye/main amd64 Packages
100 /var/lib/dpkg/status
# Broken machine T
~ » apt-cache policy docker.io
docker.io:
Installed: (none)
Candidate: 20.10.5+dfsg1-1+deb11u1
Version table:
20.10.5+dfsg1-1+deb11u1 500
500 http://deb.debian.org/debian bullseye/main amd64 Packages
That doesn't look seem to answer the question, so I've looked at available versions through apt-cache madison:
# Working machine M
~ » apt-cache madison docker.io
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye/main amd64 Packages
docker.io | 20.10.5+dfsg1-1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye-backports/main Sources
# Working machine Z
~ » apt-cache madison docker.io
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye/main amd64 Packages
docker.io | 20.10.5+dfsg1-1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye-backports/main Sources
# Broken machine T
~ » apt-cache madison docker.io
docker.io | 20.10.5+dfsg1-1+deb11u1 | http://deb.debian.org/debian bullseye/main amd64 Packages
docker.io | 20.10.5+dfsg1-1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u1 | http://deb.debian.org/debian bullseye/main Sources
docker.io | 20.10.5+dfsg1-1+deb11u2 | http://deb.debian.org/debian bullseye-backports/main Sources
The differences I'm spotting in the madison output are (a) machine T has +deb11u1 instead of +deb11u2 on the Packages line and (b) machine T lacks a +deb11u2 Sources line. I still don't see why Apt is looking for a missing version or won't offer any of the other versions available in the same repository.
I've run apt-cache showpkg docker.io on machines T and Z and compared the outputs carefully using diff. Broken machine T shows:
~ » apt-cache showpkg docker.io
Package: docker.io
Versions:
20.10.5+dfsg1-1+deb11u1 (/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_main_binary-amd64_Packages)
Description Language:
File: /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_main_binary-amd64_Packages
MD5: 05dc9eba68f3bf418e6a0cf29d555878
Description Language: en
File: /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_main_i18n_Translation-en
MD5: 05dc9eba68f3bf418e6a0cf29d555878
Reverse Depends:
runc,docker.io 1.13.1~ds1-2
docker-compose,docker.io 1.9.0
lava-dispatcher,docker.io
lava,docker.io
containerd,docker.io 1.12
whalebuilder,docker.io
toil,docker.io
subuser,docker.io
docker-compose,docker.io 1.9.0
ruby-docker-api,docker.io
python3-testinfra,docker.io
piuparts,docker.io
packer,docker.io
libnss-docker,docker.io
lava-dispatcher,docker.io
lava,docker.io
due,docker.io
containerd,docker.io 1.12
docker-clean,docker.io
docker-clean,docker.io
debocker,docker.io
datalad-container,docker.io
cwltool,docker.io
amazon-ecr-credential-helper,docker.io
charliecloud-builders,docker.io
amazon-ecr-credential-helper,docker.io
Dependencies:
20.10.5+dfsg1-1+deb11u1 - init-system-helpers (2 1.54~) libc6 (2 2.4) libdevmapper1.02.1 (2 2:1.02.97) libsystemd0 (0 (null)) adduser (0 (null)) containerd (2 1.4~) iptables (0 (null)) lsb-base (0 (null)) runc (2 1.0.0~rc8~) tini (0 (null)) docker-ce (0 (null)) docker-ce-cli (0 (null)) docker-ce-rootless-extras (0 (null)) apparmor (0 (null)) ca-certificates (0 (null)) cgroupfs-mount (0 (null)) git (0 (null)) needrestart (2 3.1~) xz-utils (0 (null)) docker-doc (0 (null)) aufs-tools (0 (null)) btrfs-progs (0 (null)) debootstrap (0 (null)) e2fsprogs (0 (null)) rinse (0 (null)) rootlesskit (0 (null)) xfsprogs (0 (null)) zfs-fuse (16 (null)) zfsutils-linux (0 (null))
Provides:
20.10.5+dfsg1-1+deb11u1 -
Reverse Provides:
I won't post machine Z's output here for brevity's sake, but the only difference is that it shows +deb11u2 instead of +deb11u1 in all 3 places. I don't see any explanations in this output, either, AFAICT.
That's the limit of my troubleshooting knowledge. What's going on, and how do I fix it? Thanks very much!
