What do the entries in my routing table actually mean?

Question

I have a VPN connection (implemented via Open VPN) but am trying to route traffic to certain IPs / domains around it, so they just use my naked internet connection. From my research it looks like the best way to do this is with routing tables. None of the examples I've found have worked so I'd like to actually understand what's going on to troubleshoot more effectively.

When I run "route" with the VPN off, it looks pretty sensible:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.1.1     0.0.0.0         UG    0      0        0 eth1
192.168.1.0     *               255.255.255.0   U     1      0        0 eth1

I suspect the first line sets default behaviour - we route via the gateway. If the destination is anywhere on the 192.168.1.* range / my internal network, the second line asserts a gateway of * (I guess this means use the default from the line above - but if I had a network spanning multiple octets, I could use this to channel certain blocks to certain gateways).

My expectation was that when I turn the VPN on,this would stay more or less the same but my gateway for "default" would shift to some wizardly VPN IP.

If this understanding is correct, I just need to add the IP I want to bypass the VPN as the destination, my actual router (192.168.1.1) as the gateway and things will work well (if the syntax for this is simple I'd love to see it).

Once I turn the VPN on, however, things get messy and I start to question my knowledge:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.172.1.5      128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth1
10.172.1.1      10.172.1.5      255.255.255.255 UGH   0      0        0 tun0
10.172.1.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun0
128.0.0.0       10.172.1.5      128.0.0.0       UG    0      0        0 tun0
168.1.6.15      192.168.1.1     255.255.255.255 UGH   0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     1      0        0 eth1

What is going on here? Can someone explain what these additional lines are and why they appear / disappear as I toggle the vpn?

Thanks for any suggestions! I have encountered a few "what is a routing table" articles but I think they're written for people much smarter than me - I'm still very new to Linux and would love some idiot proof advice :)

Answer 1

Nice, and difficult, question.

First, a general principle: in routing tables, if you have several rules which apply to the same destination, the most specific one is used. For instance, looking at your second RT, suppose you want to ping Google DNS 8.8.8.8. Both the first line and the second line apply, but the first line is slightly more specific because it has a more restrictive netmask than the second one: the first line applies to all IP addresses in the range

    0.0.0.0 -> 127.255.255.255

while the second rule applies to all IP addresses full stop, i.e. those in the range

   0.0.0.0 -> 255.255.255.255

Hence the first rule, which is narrower/more restrictive/more specific, is used: pinging 8.8.8.8 will have to go thru dev tun0 and thru IP address 10.172.1.5.

Second point: your 2nd RT contains two unicast routes: they are those indicated by an H in the fourth column. The presence of unicast routes is clearer if you use, instead of the obsolete command route/netstat, the new command:

   $ ip route show 

(which you should always do) because here unicast routes are indicated by the expression scope link.

This is the key to understanding (Open)VPN routing. The manual states:

scope SCOPE_VAL

the scope of the destinations covered by the route prefix. SCOPE_VAL may be a number or a string from the file /etc/iproute2/rt_scopes. If this parameter is omitted, ip assumes scope global for all gatewayed unicast routes, scope link for direct unicast and broadcast routes and scope host for local routes.

What is a unicast route?

A static unicast route is a manually configured mapping of an IP address to a next-hop destination, hence called a destination specific route... Add static routes when you want to route traffic destined for specific network/host via a different next hop instead of the default route.

In other words: if this rule did not exist, then, since pinging 8.8.8.8 goes thru 10.172.1.5 (in your case) but there is no default gateway on tun0, the packet would be transferred to the eth0 interfce, where there is a default gateway, and would go out of this. But this is what happens normally, and you would not have an (Open)VPN. Instead, there is a unicast rule, and it is as restrictive as it can be: it forces you, no matter whom the packet is addressed to, to contact 10.172.1.1 as the next hop.

Fine, what now: how do we contact 10.172.1.1? dev tun0 has no such rule, hence the packet is transferred to eth0, in the hope of better luck. Now, which of the remaining rules can we apply to forward our ping packet? The fact is, eth0 also has a unicast route,

  168.1.6.15      192.168.1.1     255.255.255.255 UGH   0      0        0 eth1

which forces it to send the packet, as its next hop, to 168.1.6.15. And, from then on, it is not our responsibility any longer.

We can summarize this logical process as follows:

Packet to 8.8.8.8 ---> dev tun0 ---> 10.172.1.5 ---> 168.1.6.15

            (Rule #1)     
                      (Rule #3)
                                  (Rule #6)

Loose ends:

One of the remaining rules, Rule #5

128.0.0.0       10.172.1.5      128.0.0.0       UG    0      0        0 tun0

complements Rule #1

0.0.0.0       10.172.1.5      128.0.0.0       UG    0      0        0 tun0   

The two together provide default rules to all IP addresses in the world, but since each of them is slightly more restrictive than Rule #2

 0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth1

they take precedence over it, and are used to route all of your internet traffic thru the OpenVPN. In fact, the rule above is redundant, and it wad deleted in older version of OpenVPN. But it is now left in place because it is basically never invoked, and it makes re-instating the RT#1 easier, when the OpenVPN is torn down.

Rule #7 is the usual local LAN rule. You are missing a rule like

    10.172.1.0/24 dev tun0  proto kernel  scope link  src 10.172.1.5

which is what would be necessary for you to contact pcs in the remote LAN. Most likely, this is due to the fact that you are using as an OpenVPN server a per-pay service, which has no intention of allowing you access to the other local machines. If instead this OpenVPN were the one thru which you connect to your home LAN whe on the road (for instance), then you would definitely wish to have the ability to contact the other pcs on your LAN, and you would have one such rule.

Lastly, the Rule #4

 10.172.1.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun0

is completely mysterious to me: I do not have it on any of my (numerous) OpenVPNs, and, as far as I understand it, it seems to me totally redundant.

Answer 2

You additionally asked:

"Rule #3 has the ...15 address as the Gateway — why would that match? And if it did match, why would we then start looking at ...1 (the Destination address in the rule). It's as though source/destination got swapped?"

This is difficult to understand because the default topology setting of openvpn is "net30". It seems most solution resources assume we have changed the default topology setting.

This answer: https://serverfault.com/questions/742347/openvpn-server-sending-bogus-default-gateway-information-to-client mentions:

"Your OpenVPN works in "net30" topology mode. In this mode OpenVPN process itself is a router." and ... "i.e. 10.8.0.5 is really an address of OpenVPN router, it is "hidden inside a process". "

10.8.0.5 is equivalent to 10.172.1.15 on your set up, I believe.

https://community.openvpn.net/openvpn/wiki/Topology

net30 is effectively "deprecated" but remains the OpenVPN 2.3-series default for fear of breaking backwards-compat configs that rely on this behavior. Each client is allocated a virtual /30 network, and Windows clients must use the 2 IPs in the center, generally allocating the even IP to the client (though this is convention only.)

The only real reason to use the net30 topology is when requiring support for Windows clients before 2.0.9, or when any Windows clients must be supported and non-Windows clients must be supported that cannot set IP+netmask on the tun adapter. These conditions are rare, and the 2.0.9 client is around 7 years out of date as of 2014.

Windows allocates the fake /30 as a local network, while non-Windows will set up a true Point-to-Point config, treating the IP pairing the same as the p2p topology described below.

Routing to/across the VPN range gets harder in net30 since the clients on-link network is just the /30, so an additional "supernet" route must be added to reach the rest of the VPN, including the server at its designated IP. This complicates routing and makes understanding the config harder.

In /etc/openvpn/server/server.conf you'll see:

Network topology

# Should be subnet (addressing via IP)
# unless Windows clients v2.0.9 and lower have to
# be supported (then net30, i.e. a /30 per client)
# Defaults to net30 (not recommended)
;topology subnet

remove the ";" to enable subnet topology.