different subnets on the same interface used by virtual machines in brdiged networking

Question

I have one physical server with one network interface. IP forwarding is enabled and firewall configured properly.

My provider has assign me one primary IP address and 4 failover IP addresses form a different subnet. The problem is I don't know how to configure them in a way that I won't be wasting ipv4 resources.

(the IP addresses in this example are fake, and only for informative and example of setup, but they are an exact analogy)

Main server (host): Primary IP: 100.0.98.116, gateway 100.0.98.115, netmask 255.255.255.252.

Failover IP addresses: 50.76.102.200 50.76.102.201 50.76.102.202 50.76.102.203

How can I use all of them in virtual machines using bridged networking (assigned directly to the host interface)? At the moment, I have to add 50.76.102.201 to the main server primary interface as secondary address, with netmask 255.255.255.252. In the virtual machine, I use the IP address 50.76.102.202 with netmask 255.255.255.252 and gateway 50.76.102.201. But this allows me less IPv4 addresses to use. What would be a way to use them all, and use the gateway 100.0.98.115?

Answer 1

Given that your provider gives you ethernet, you can use all your addresses by tweeking the configuration a bit. How can you do it depends on what the operating system on your VMs are. For the sake of this post I'll assume linux.

Back to basics, ethernet is multipoint and you usually can directly reach all your subnet neighbors (L3 address) by discovering their MAC (L2 address) using ARP. That your VMs are virtual and a virtual switch away from the real ethernet should not matter, in fact you can go directly from a VM NIC to your provider DG without going (logically, i.e. L3) through your host.

The problem is that standard logic, i.e., ARP neighbor finding only works for same subnet destinations, and that's why your DG (default gateway) should be in your subnet. That can be changed in a number of ways.

The easiest to configure (but not at all a good option) is by using proxy ARP. Most routers support answering ARP to non local (i.e. extra subnet) queries. You use that by configuring your default route pointing to your ethernet interface. Not nice because you do ARP for every address you converse with, but works.

    ip address add 50.76.102.200/24 dev eth0
    ip route add default dev eth0

A very easy (and good) alternative is to get a DG address from your service provider in that subnet and use that for your VMs. Say 50.76.102.1.

   ip address add 50.76.102.200/24 dev eth0
   ip route add default via 50.76.102.1 dev eth0

But if not, you can (at least in linux using iproute2) use a DG that is not on your same subnet:

    ip address add 50.76.102.200/24 dev eth0
    ip route add default via 100.0.98.115 dev eth0 onlink

I hope you get the idea.

Answer 2

I have not managed to fix this issue in the manner i was planning. The only way I could use Bridged networking for vms was to lift ip address with .201 on the host as eth0:0 alias with netmask 255.255.255.252. And assign to the vm the IP address .202 with netmask 255.255.255.252 and gateway .201 -- this works just fine. However, too many IP addresses are consumed this way, only 2 hosts usuable.

What I did was to add all ip addresses .200-.203 as aliases eth0:0 - eth0:3 each with netmask 255.255.255.255 so they are all usuable. And i created a virtual NIC host-only for the vms, that have a private lan in 192 range and i use SNAT / DNAT to bind each vm behind 192 range to a public IP address.