Understanding packet flow through iptables rules for docker container bridge network?

Question

Background I want to understand the relationships between docker container networks and iptables, and generally understand how packets flow from the eth0 interface (in a container), through the default bridge docker0 interface, and to the network interface on the host computer. At the moment, I am having trouble understanding the filter iptables rules and chains for default docker container networks. Without running any containers, the command sudo iptables -t filter -L -v -n shows

$ sudo iptables -t filter -L -v -n
Chain INPUT (policy ACCEPT 108K packets, 12M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 183K  304M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 183K  304M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 107K  301M ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
76705 3634K ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 98618 packets, 14M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
76705 3634K DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
 183K  304M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
76705 3634K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 183K  304M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0   

According to the docs, Docker adds two custom iptables chains named DOCKER-USER and DOCKER, and it ensures that incoming packets are always checked by these two chains first. But it never mentions DOCKER-ISOLATION-STAGE-1. I am having trouble understanding the packet flow through these defined chains and rules.

This is my understanding so far.

1. In Chain FORWARD, the first rule states that all traffic from any source and destination is sent to the DOCKER-USER chain.
2. In the DOCKER-USER chain, there is only one rule that all traffic from any source to any destination has a target RETURN, returning to the FORWARD chain.
3. Then, the second rule in Chain FORWARD states that all traffic from any source and destination is sent to the DOCKER-ISOLATION-STAGE-1 chain.
4. The DOCKER-ISOLATION-STAGE-1 chain has two rules.
   1. Rule 1: All traffic from the docker0 interface to any interface except itself (!docker0) is sent to the DOCKER-ISOLATION-STAGE-2 chain.
   2. Rule 2: All other traffic is returned to its originating chain (e.g., FORWARD or INPUT).
5. The DOCKER-ISOLATION-STAGE-2 chain also has two rules.
   1. Rule 1: All traffic from any interface to the docker0 interface is dropped.
   2. Rule 2: All other traffic is returned to its originating chain.

Question I am having trouble understanding the first rule of chain DOCKER-ISOLATION-STAGE-1. What does it mean when we say "All traffic from the docker0 interface to any interface except itself (!docker0) is sent to the DOCKER-ISOLATION-STAGE-2 chain"?

More precisely, I would like to know,

1. What exactly does it mean by "docker0 interface"?
2. What kind of packet flow scenario would it end up dropping a packet due to the first rule in DOCKER-ISOLATION-STAGE-2?

Follow Up Based on my understanding of the iptables the rules of DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2, matches and therefore drops any outgoing traffic from a Docker container on the docker0 network that is not intended for another container on the same network. This includes traffic destined for other interfaces on the Docker host or external networks.

Is my understanding of docker container networks and ip tables correct?

P.S However, after running two docker containers with open ports at 3000, 3001, the DOCKER chain has changed

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:3000
    0     0 ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.3           tcp dpt:3001

Both containers can ping eachother as well as the host.

Edit Perhaps nat iptable will also help

sudo iptables -t nat -L -v -n
Chain PREROUTING (policy ACCEPT 2 packets, 168 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    5   340 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 47 packets, 3607 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 49 packets, 3775 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  110  6860 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:3000
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.3           172.17.0.3           tcp dpt:3001

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    84 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3000 to:172.17.0.2:3000
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3001 to:172.17.0.3:3001

Answer 1

Question I am having trouble understanding the first rule of chain DOCKER-ISOLATION-STAGE-1. What does it mean when we say "All traffic from the docker0 interface to any interface except itself (!docker0) is sent to the DOCKER-ISOLATION-STAGE-2 chain"?

You got it right.

It routes every packet going outside through the DOCKER-ISOLATION-STAGE-2 chain. Most probably for separated specification of rules restricting packets going from container to Internet.

docker0 is default ethernet bridge to which is every container connected to.