what happens when you have 2 network interfaces to the same network in a Unix/Linux OS?

Question

What will happen if you have 2 network interfaces to the same network where the default gateway/router is?

e.g

• network interface e1000g0 - 192.168.1.50
• network interface e1000g1 - 192.168.1.51
• Gateway - 192.168.1.1

If I send a packet to 8.8.8.8, which network interface will be used ? I am using a Solaris 10 box. But assume this question is for all unix/linux general implementation

In my routing table, it seems that e1000g1 is chosen as the UG , but why/how is it chosen?

Answer 1

In Linux and Unix, there can be only one gateway per routing table. On Linux you can have multiple routing tables, but each has a single gateway.

In Linux, it is identified as follows:

    > ip route show
      default via 192.168.73.1 dev eth0  proto static 
      192.168.73.0/24 dev eth0  proto kernel  scope link  src 192.168.73.75  metric 1 

The key line is the one beginning with default, it states that (my) gateway is 192.168.73.1. You change that as follows:

     > ip route del default
     > ip route add default via 192.168.73.1 dev eth0

If I have more than one interface connected, my routing table will look like:

     > ip route show
       default via 192.168.73.1 dev eth0  proto static 
       192.168.73.0/24 dev eth0  proto kernel  scope link  src 192.168.73.75  metric 1 
       192.168.73.0/24 dev wlan0  proto kernel  scope link  src 192.168.73.66  metric 9 

As you can see local traffic can pass thru either eth0or wlan0, but, thanks to the smaller metric value (1 vs. 9) will be routed thru eth0.

WAN traffic will surely go thru eth0 because the default gateway is on dev eth0.

These values are automatically maintained by your network manager which favors, by default, cable over wifi connections. You can however override its choice by means of commands similar to the previous ones,

       > ip route delete default
       > ip route add default via 192.168.73.1 dev wlan0

If you have multiple interfaces of the same type (e.g. multiple ethernet connections) then the priority goes to the interface that was connected first.

The default gateway is identified, so to say, by the network itself: it is one of the parameters which are passed to your machine when a DHCP transition is negotiated (the others are netmask, network, broadcast address). You are unaware of this because the process of DHCP is handled for you by the network-manager. Occasionally, you may wish to setup a static IP address, which does not use DHCP; in this case you will have to provide yourself the four parameters mentioned above. There are techniques for learning these parameters even in networks totally new to you, but normally you will setup a static address only in LANs you already know well.

Apart from your LAN and default gateway routes, you can have specific routes thru either interface. In that case, the order of priority of routes is determined on the basis of the principle most specific routes first. Suppose you have a route for 1.1.1.0/24 thru wlan0. Then, when we need to route packets for, say, 1.1.1.1, both this rule (thru wlan0) and the default rule (thru eth0) apply, but the first one is more specific, hence thse packets will pass thru wlan0. In this case, the dafult gateway becomes the route when all else fails, i.e. when no other route applies.

In Linux you can have multiple routing tables, this is called policy or source routing. In this case you will also need a rule for the kernel to discriminate when to apply the multiple routing tables at your disposal. You find a brief introduction to policy routing here. In this case, the above commands remain identical, except they are followed by the name of the routing table you wish to work on, for instance:

       > ip route show table main
       > ip route show table MyOtherRoutingTable

You will find often references to commands like netstat, route, ifconfig. In Linux (but not on Unix, read the top comment below) they are obsolete. The current command, ip of the iproute2 suite, substitutes for all of these, and a few more. By googling linux ip cheat sheet you can find sites comparing the capabilites of ip and previous utilities.

Edit

On Unix, the commands above would be: you query the routing table as

   >  netstat -rn -f inet
      Routing tables

      Internet:
      Destination        Gateway            Flags        Refs      Use   Netif Expire
      default            192.168.11.1       UGSc           10        0     en0
      127                127.0.0.1          UCS             0        0     lo0
      127.0.0.1          127.0.0.1          UH              2   161444     lo0
      169.254            link#4             UCS             1        0     en0
      169.254.10.9       10:c3:7b:9d:c8:78  UHLSW           0        0     en0
      192.168.11         link#4             UCS             4        0     en0
      192.168.11.1       10:6f:3f:25:c6:33  UHLWIir        11     4589     en0   1162
      192.168.11.56      88:53:2e:10:77:5f  UHLWI           0        0     en0   1187
      192.168.11.65      2:f:b5:70:5b:22    UHLWI           0        0     en0   1081
      192.168.11.67      127.0.0.1          UHS             0        0     lo0
      192.168.11.113     6:20:3e:52:16:4d   UHLWIi          2      278     en0   1188

Here the gateway is identified by the default keyword, and by the G flag.

You change default gateway as:

   > route delete default
   > route add default 192.168.0.1
   > route change default -interface enp0s3

Every comment remains the same, except those about policy routing, which is not supported by Unix kernels.

Answer 2

Having two active different physical interfaces on the same OS instance connected to the same subnet like e1000g0 and e1000g1 in your question is unreliable, and explicitly unsupported at least with Solaris:

On a system whose multiple interfaces connect to the same subnet, you must configure the interfaces into an IPMP group first. Otherwise, the system cannot be a multihomed host.

As stated, one way to have this reliably work is to enable ipmp (IP multipathing). You might also disable packet-forwarding and configure static routes like documented here for Solaris 10.

Alternatively, you might also use the lower level link aggregation, a.k.a. interface bonding in Linux terminology.

Answer 3

I think that the message will be sent through the interface configured in the default route entry. Probably the last one that became up.