QEMU/KVM/libvirt macvtap VEPA not working - ARP request not forward

Question

Hi I've been struggling to get guest networking working when using a macvtap in VEPA mode between two VMs on a host. I've spent hours (days) googling with no joy. Does this network configuration actually work?

I created the vtap's using KVM Manager by adding a NIC, selecting Network source "macvtap", Source mode VEPA, Device model: virtio.

The configuration looks like this (mac address):

vm3-62                                  vm2-62
----------                           ------------
eth1: 172.15.62.105            eth1:  172.15.62.205
(52:54:00:08:9d:8b)            (52::54:00:8a:b1:0f)
           +                               +
           |                               |
            \                              /
              \             host          /
            macvtap1                  macvtap0
          (52:54:00:08:9d:8b)      (52:54:00:8a:b1:0f)
                 \                         /
                  \                       /
                    \                    /
                           bond1.62
                     (98:03:9b:2d:91:a2) 
                              |
                           bond1
                             |
                       NIC port 1 and 2 (active/passive config)

Not sure if the above will format ok, If the above diagram did not format here is a jpeg

The host NIC is connected to a cisco nexus 9000, which I have configured for Reflective Relay 802.1Qbg.

On vm2-62 when I try ping 172.15.62.105, I get Destination Host Unreachable.

When I use tcpdump on the host, I can see the ARP request from vm2-62 looking for the mac for 172.15.62.105 (vm3-62). I can see the request on macvtap0, and on bond1.62 and on bond1, but NOT on macvtap1.

If I manually add ARP entries on vm3-62 & vm2-62, the ping works fine so I think the reflective relay on the switch is configured ok.

It just seems that either the switch is not bouncing back the ARP request or I need to do something in Linux to enable bond1.62 to forward the ARP request to macvtap1.

Any thoughts? .

Thanks

Answer 1

I did get to the bottom of this and the issue was with the switch.

So the problem turned out that the switch (a Cisco Nx9000 series) was not bouncing the ARP request back. But there was more to it than that, my systems are connected in a VXLAN network, and I have the L2 VTEPs configured with arp suppression enabled. When that is disabled, then the switch does reflect the ARP back on the same port as it is supposed to, and everything works fine. I worked with Cisco Engineers and they agreed that the VTEP should also reflect the ARP when ARP suppression is on (the VTEP is suppose to respond to ARP requests in this case). But it seems that the VTEP software gets confused when it is asked to send the broadcast back out the interface it arrived on. In any case cisco say they are looking at a fix, and meantime the workaround is to disable arp suppression.