0

Context

I have the following network :

  • a router (IP 192.168.2.254) which handles DHCP on 192.168.2.x network
  • a computer1 (IP 192.168.2.1) under Linux (Debian 11) using ufw as firewall
  • a computer2 (IP 192.168.2.2) under Linux (Debian 11)
  • other computers on the LAN computerN (IP 192.168.2.N) under various OSes

Basically, what I want to achieve is to have a virtual machine vm hosted on computer1 and allow network traffic :

  • from vm to computer1
  • from computer1 to vm
  • from vm to computerN
  • from computerN to vm

In other words, I want my vm to work on the network as if it was one of computerN systems.

I used VirtualBox for a long time and it works flawlessly with the VM's network interface in bridged mode, with the extra benefit that VM's IP is handled by LAN DHCP (192.168.2.x). But I have to try an other virtualization solution since I experience compatibility problems with latest kernels.

I give a try to Qemu/KVM with libvirt (using libvirt-manager as GUI).

What I've done

On computer1 :

  • I enabled the bridge network virbr0 given by libvirt, which has a DHCP with 192.168.122.x range

  • I created and set up a Linux (Debian 11) system on a virtual machine vm with virbr0 associated to the network interface

  • vm gets the IP address 192.168.122.184

  • vm can ping computer1 (its host) and computerN (other computer on the LAN)

  • computer1 can ping vm

  • computerN can not ping vm

What I've tried

  • enabling IPv4 forwarding on computer1, so cat /proc/sys/net/ipv4/ip_forward returns 1
  • allowing forwarding on computer1's firewall with sudo ufw route allow in on enp41s0 out on virbr0
  • defining a static route on router and/or computerN (ip route add 192.168.122.0/24 via 192.168.2.1)

Without the route added, a ping from 192.168.2.2 to 192.168.122.184 just timeouts.

With the route defined directly on 192.168.2.2

Ping gives :

PING 192.168.122.184 (192.168.122.184) 56(84) bytes of data.
From 192.168.2.1 icmp_seq=1 Destination Port Unreachable
From 192.168.2.1 icmp_seq=2 Destination Port Unreachable
From 192.168.2.1 icmp_seq=3 Destination Port Unreachable

and traceroute shows :

traceroute to 192.168.122.184 (192.168.122.184), 30 hops max, 60 byte packets
 1  192.168.2.1 (192.168.2.1)  0.981 ms  0.920 ms  0.869 ms
 2  192.168.2.1 (192.168.2.1)  0.824 ms  0.780 ms  0.477 ms

and tcpdump -ni any icmp on computer1 shows :

tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
04:27:31.443991 enp41s0 In  IP 192.168.2.2 > 192.168.122.184: ICMP echo request, id 19733, seq 1, length 64
04:27:31.444022 enp41s0 Out IP 192.168.2.1 > 192.168.2.2: ICMP 192.168.122.184 protocol 1 port 32450 unreachable, length 92
04:27:32.456447 enp41s0 In  IP 192.168.2.2 > 192.168.122.184: ICMP echo request, id 19733, seq 2, length 64
04:27:32.456477 enp41s0 Out IP 192.168.2.1 > 192.168.2.2: ICMP 192.168.122.184 protocol 1 port 52880 unreachable, length 92
04:27:33.480449 enp41s0 In  IP 192.168.2.2 > 192.168.122.184: ICMP echo request, id 19733, seq 3, length 64
04:27:33.480481 enp41s0 Out IP 192.168.2.1 > 192.168.2.2: ICMP 192.168.122.184 protocol 1 port 3122 unreachable, length 92
04:27:34.504454 enp41s0 In  IP 192.168.2.2 > 192.168.122.184: ICMP echo request, id 19733, seq 4, length 64
04:27:34.504484 enp41s0 Out IP 192.168.2.1 > 192.168.2.2: ICMP 192.168.122.184 protocol 1 port 16339 unreachable, length 92

With the route defined on router

Ping gives :

PING 192.168.122.184 (192.168.122.184) 56(84) bytes of data.
From 192.168.2.1 icmp_seq=1 Destination Port Unreachable
From 192.168.2.254: icmp_seq=2 Redirect Host(New nexthop: 192.168.2.1)
From 192.168.2.1 icmp_seq=2 Destination Port Unreachable
From 192.168.2.254: icmp_seq=3 Redirect Host(New nexthop: 192.168.2.1)
From 192.168.2.1 icmp_seq=3 Destination Port Unreachable
From 192.168.2.254: icmp_seq=4 Redirect Host(New nexthop: 192.168.2.1)
From 192.168.2.1 icmp_seq=4 Destination Port Unreachable
From 192.168.2.254: icmp_seq=5 Redirect Host(New nexthop: 192.168.2.1)
From 192.168.2.1 icmp_seq=5 Destination Port Unreachable

and traceroute shows :

traceroute to 192.168.122.184 (192.168.122.184), 30 hops max, 60 byte packets
 1  192.168.2.254 (192.168.2.254)  0.336 ms  0.478 ms  0.657 ms
 2  192.168.2.1 (192.168.2.1)  0.958 ms  1.084 ms  1.234 ms
 3  192.168.2.1 (192.168.2.1)  1.188 ms  1.320 ms  1.284 ms

Disabling firewall

I tried to disable ufw firewall (ufw disable) on computer1 with the same results.

iptables-save output on computer1 before disabling firewall :

# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:22 2022
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Jul 16 11:20:22 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:22 2022
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -j LIBVIRT_OUT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
COMMIT
# Completed on Sat Jul 16 11:20:22 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:22 2022
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
:LIBVIRT_PRT - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -j LIBVIRT_PRT
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Sat Jul 16 11:20:22 2022

nft list ruleset output on computer1 before disabling firewall :

table ip nat {
    chain DOCKER {
        iifname "docker0" counter packets 0 bytes 0 return
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        counter packets 65 bytes 7530 jump LIBVIRT_PRT
        oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
    }

    chain PREROUTING {
        type nat hook prerouting priority dstnat; policy accept;
        fib daddr type local counter packets 108 bytes 6480 jump DOCKER
    }

    chain OUTPUT {
        type nat hook output priority -100; policy accept;
        ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
    }

    chain LIBVIRT_PRT {
        ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 3 bytes 194 return
        ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
        meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 
        meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 
        ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade 
    }
}
table ip filter {
    chain DOCKER {
    }

    chain DOCKER-ISOLATION-STAGE-1 {
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
        counter packets 0 bytes 0 return
    }

    chain DOCKER-ISOLATION-STAGE-2 {
        oifname "docker0" counter packets 0 bytes 0 drop
        counter packets 0 bytes 0 return
    }

    chain FORWARD {
        type filter hook forward priority filter; policy drop;
        counter packets 0 bytes 0 jump LIBVIRT_FWX
        counter packets 0 bytes 0 jump LIBVIRT_FWI
        counter packets 0 bytes 0 jump LIBVIRT_FWO
        counter packets 0 bytes 0 jump DOCKER-USER
        counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
        oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
        oifname "docker0" counter packets 0 bytes 0 jump DOCKER
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
        iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
    }

    chain DOCKER-USER {
        counter packets 0 bytes 0 return
    }

    chain LIBVIRT_INP {
        iifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto udp udp dport 67 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto tcp tcp dport 67 counter packets 0 bytes 0 accept
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 909 bytes 700373 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
        oifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto tcp tcp dport 68 counter packets 0 bytes 0 accept
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 747 bytes 64760 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
        iifname "virbr0" ip saddr 192.168.122.0/24 counter packets 0 bytes 0 accept
        iifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWI {
        oifname "virbr0" ip daddr 192.168.122.0/24 ct state related,established counter packets 0 bytes 0 accept
        oifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWX {
        iifname "virbr0" oifname "virbr0" counter packets 0 bytes 0 accept
    }
}
table ip mangle {
    chain LIBVIRT_PRT {
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 0 bytes 0 # CHECKSUM fill
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 787 bytes 72954 jump LIBVIRT_PRT
    }
}
table ip6 filter {
    chain LIBVIRT_INP {
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 15 bytes 1937 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 27 bytes 2765 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
    }

    chain FORWARD {
        type filter hook forward priority filter; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_FWX
        counter packets 0 bytes 0 jump LIBVIRT_FWI
        counter packets 0 bytes 0 jump LIBVIRT_FWO
    }

    chain LIBVIRT_FWI {
    }

    chain LIBVIRT_FWX {
    }
}
table ip6 nat {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_PRT
    }
}
table ip6 mangle {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 40 bytes 4562 jump LIBVIRT_PRT
    }
}

iptables-save output on computer1 after disabling firewall :

# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:57 2022
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Jul 16 11:20:57 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:57 2022
*filter
:INPUT ACCEPT [30:1796]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [34:1684]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -j LIBVIRT_OUT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
COMMIT
# Completed on Sat Jul 16 11:20:57 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:20:57 2022
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
:LIBVIRT_PRT - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -j LIBVIRT_PRT
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Sat Jul 16 11:20:57 2022

nft list ruleset output on computer1 after disabling firewall :

table ip nat {
    chain DOCKER {
        iifname "docker0" counter packets 0 bytes 0 return
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        counter packets 69 bytes 7825 jump LIBVIRT_PRT
        oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
    }

    chain PREROUTING {
        type nat hook prerouting priority dstnat; policy accept;
        fib daddr type local counter packets 125 bytes 7500 jump DOCKER
    }

    chain OUTPUT {
        type nat hook output priority -100; policy accept;
        ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
    }

    chain LIBVIRT_PRT {
        ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 4 bytes 267 return
        ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
        meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 
        meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535 
        ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade 
    }
}
table ip filter {
    chain DOCKER {
    }

    chain DOCKER-ISOLATION-STAGE-1 {
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
        counter packets 0 bytes 0 return
    }

    chain DOCKER-ISOLATION-STAGE-2 {
        oifname "docker0" counter packets 0 bytes 0 drop
        counter packets 0 bytes 0 return
    }

    chain FORWARD {
        type filter hook forward priority filter; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_FWX
        counter packets 0 bytes 0 jump LIBVIRT_FWI
        counter packets 0 bytes 0 jump LIBVIRT_FWO
        counter packets 0 bytes 0 jump DOCKER-USER
        counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
        oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
        oifname "docker0" counter packets 0 bytes 0 jump DOCKER
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
        iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
    }

    chain DOCKER-USER {
        counter packets 0 bytes 0 return
    }

    chain LIBVIRT_INP {
        iifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto udp udp dport 67 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto tcp tcp dport 67 counter packets 0 bytes 0 accept
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 937 bytes 702052 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
        oifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto tcp tcp dport 68 counter packets 0 bytes 0 accept
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 775 bytes 66099 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
        iifname "virbr0" ip saddr 192.168.122.0/24 counter packets 0 bytes 0 accept
        iifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWI {
        oifname "virbr0" ip daddr 192.168.122.0/24 ct state related,established counter packets 0 bytes 0 accept
        oifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWX {
        iifname "virbr0" oifname "virbr0" counter packets 0 bytes 0 accept
    }
}
table ip mangle {
    chain LIBVIRT_PRT {
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 0 bytes 0 # CHECKSUM fill
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 817 bytes 74439 jump LIBVIRT_PRT
    }
}
table ip6 filter {
    chain LIBVIRT_INP {
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 16 bytes 2030 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 28 bytes 2858 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
    }

    chain FORWARD {
        type filter hook forward priority filter; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_FWX
        counter packets 0 bytes 0 jump LIBVIRT_FWI
        counter packets 0 bytes 0 jump LIBVIRT_FWO
    }

    chain LIBVIRT_FWI {
    }

    chain LIBVIRT_FWX {
    }
}
table ip6 nat {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_PRT
    }
}
table ip6 mangle {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 42 bytes 4748 jump LIBVIRT_PRT
    }
}

iptables-save output on computer1 after disabling firewall and reboot :

# Generated by iptables-save v1.8.7 on Sat Jul 16 11:28:44 2022
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Jul 16 11:28:44 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:28:44 2022
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
COMMIT
# Completed on Sat Jul 16 11:28:44 2022
# Generated by iptables-save v1.8.7 on Sat Jul 16 11:28:44 2022
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
:LIBVIRT_PRT - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -j LIBVIRT_PRT
-A DOCKER -i docker0 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Sat Jul 16 11:28:44 2022

nft list ruleset output on computer1 after disabling firewall and reboot :

table ip nat {
    chain DOCKER {
        iifname "docker0" counter packets 0 bytes 0 return
    }

    chain LIBVIRT_PRT {
        ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 3 bytes 246 return
        ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
        meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 1 bytes 60 masquerade to :1024-65535 
        meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 2 bytes 142 masquerade to :1024-65535 
        ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade 
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
        counter packets 30 bytes 6587 jump LIBVIRT_PRT
    }

    chain PREROUTING {
        type nat hook prerouting priority dstnat; policy accept;
        fib daddr type local counter packets 60 bytes 3600 jump DOCKER
    }

    chain OUTPUT {
        type nat hook output priority -100; policy accept;
        ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
    }
}
table ip filter {
    chain DOCKER {
    }

    chain DOCKER-ISOLATION-STAGE-1 {
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
        counter packets 6452 bytes 6250876 return
    }

    chain DOCKER-ISOLATION-STAGE-2 {
        oifname "docker0" counter packets 0 bytes 0 drop
        counter packets 0 bytes 0 return
    }

    chain FORWARD {
        type filter hook forward priority filter; policy drop;
        counter packets 6452 bytes 6250876 jump DOCKER-USER
        counter packets 6452 bytes 6250876 jump DOCKER-ISOLATION-STAGE-1
        oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
        oifname "docker0" counter packets 0 bytes 0 jump DOCKER
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
        iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
        counter packets 6452 bytes 6250876 jump LIBVIRT_FWX
        counter packets 6452 bytes 6250876 jump LIBVIRT_FWI
        counter packets 1949 bytes 103516 jump LIBVIRT_FWO
    }

    chain LIBVIRT_INP {
        iifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        iifname "virbr0" meta l4proto udp udp dport 67 counter packets 2 bytes 660 accept
        iifname "virbr0" meta l4proto tcp tcp dport 67 counter packets 0 bytes 0 accept
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 189 bytes 24036 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
        oifname "virbr0" meta l4proto udp udp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto tcp tcp dport 53 counter packets 0 bytes 0 accept
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 2 bytes 660 accept
        oifname "virbr0" meta l4proto tcp tcp dport 68 counter packets 0 bytes 0 accept
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 211 bytes 24948 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
        iifname "virbr0" ip saddr 192.168.122.0/24 counter packets 1949 bytes 103516 accept
        iifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWI {
        oifname "virbr0" ip daddr 192.168.122.0/24 ct state related,established counter packets 4503 bytes 6147360 accept
        oifname "virbr0" counter packets 0 bytes 0 reject
    }

    chain LIBVIRT_FWX {
        iifname "virbr0" oifname "virbr0" counter packets 0 bytes 0 accept
    }

    chain DOCKER-USER {
        counter packets 6452 bytes 6250876 return
    }
}
table ip mangle {
    chain LIBVIRT_PRT {
        oifname "virbr0" meta l4proto udp udp dport 68 counter packets 2 bytes 660 # CHECKSUM fill
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 6724 bytes 6289565 jump LIBVIRT_PRT
    }
}
table ip6 filter {
    chain LIBVIRT_INP {
    }

    chain INPUT {
        type filter hook input priority filter; policy accept;
        counter packets 32 bytes 4339 jump LIBVIRT_INP
    }

    chain LIBVIRT_OUT {
    }

    chain OUTPUT {
        type filter hook output priority filter; policy accept;
        counter packets 55 bytes 5939 jump LIBVIRT_OUT
    }

    chain LIBVIRT_FWO {
    }

    chain FORWARD {
        type filter hook forward priority filter; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_FWX
        counter packets 0 bytes 0 jump LIBVIRT_FWI
        counter packets 0 bytes 0 jump LIBVIRT_FWO
    }

    chain LIBVIRT_FWI {
    }

    chain LIBVIRT_FWX {
    }
}
table ip6 nat {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        counter packets 0 bytes 0 jump LIBVIRT_PRT
    }
}
table ip6 mangle {
    chain LIBVIRT_PRT {
    }

    chain POSTROUTING {
        type filter hook postrouting priority mangle; policy accept;
        counter packets 85 bytes 10138 jump LIBVIRT_PRT
    }
}
Improve this question
9
  • Using a(n "internal") bridge is not the same thing as "bridged" networking (which refers to the fact that a physical NIC being enslaved to the bridge used by the VMs, i.e. merging the physical and virtual LAN into one broadcast domain). You should decide / declare whether you want "bridged" networking back or switch to the "internal" bridge (i.e. routed) approach first.
    – Tom Yan
    Commented Jul 16, 2022 at 2:09
  • If I understand your question right, what I have here is an "internal" bridge.
    – berty
    Commented Jul 16, 2022 at 2:19
  • Yes, if you are fine with that, you can try something like tcpdump -ni any icmp on computer 1. To make sure firewall really isn't the issue, check iptables-save and nft list ruleset. Btw is there anything quirky in your setup? All the LAN hosts still use .254 as their default gateway right?
    – Tom Yan
    Commented Jul 16, 2022 at 2:25
  • For sure, I added tcpdump result in my question. And yes, all hosts use .254 as default gateway.
    – berty
    Commented Jul 16, 2022 at 2:30
  • Again, I'd suggest that you check iptables-save and nft list ruleset after disabling ufw. (If possible, also reboot as well to eliminate the noise / leftover.)
    – Tom Yan
    Commented Jul 16, 2022 at 5:06

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .