Setup routing table for two NICs on two different networks at boot

Question

I have a machine running a derivative of Debian (Raspbian) with two NICs, one Ethernet (eth0) and the other wireless (wlan0). eth0 connects the machine to a private LAN and has a static address, while wlan0 connects the machine to the Internet and is configured with wpa_supplicant. Without additional configuration, I can ping the router on the LAN, and cannot ping, say, google.com. I assumed, after doing some research, that the cleanest way to accomplish this was to setup custom routing tables. The routing rules would route all traffic to and from the LAN network over eth0, while all other traffic would go through wlan0. Correct me if if there is a mistake in that logic.

I've already figured out how I need to set up the routing tables to accomplish this; and I can make it work when I manually set the rules with the ip route command. The issue is that I can't seem to set up the routing tables when bringing the interfaces up by using the post-up command in the interfaces file.

To explain my thought process from the beginning...

This is the output of route -n without additional configuration.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
0.0.0.0         10.109.48.1     0.0.0.0         UG    303    0        0 wlan0
10.109.48.0     0.0.0.0         255.255.240.0   U     303    0        0 wlan0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

According to my understanding, there is a conflict for communication with 'any destination', which is resolved by using eth0 because of it's lower metric. This makes all outbound traffic go through eth0. Ideally, only traffic meant for, and coming from, the LAN should go through eth0. So, I thought by removing the default generated rules and making a single rule that specifies the destination and gateway for the network, I could achieve the behavior I wanted. After running a few commands to delete the default routing rules and adding the new one I described, I got the following routing table.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.109.48.1     0.0.0.0         UG    303    0        0 wlan0
10.109.48.0     0.0.0.0         255.255.240.0   U     303    0        0 wlan0
192.168.0.0     192.168.0.1     255.255.255.0   UG    0      0        0 eth0

Afterwards, I can ping both the LAN's gateway and google.com. So it seems my intuition was correct. Correct me if this was just a fluke.

To make the routing rules permanent, many guides and Serverfault answers I've found suggest that using post-up commands in the interfaces file is the best way to go about that. So I edited my interfaces file, as seen below.

# interfaces(5) file used by ifup(8) and ifdown(8)

# Please note that this file is written to be used with dhcpcd
# For static IP, consult /etc/dhcpcd.conf and 'man dhcpcd.conf'

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

allow-hotplug eth0
iface eth0 inet static
    address 192.168.0.10
    network 192.168.0.0
    netmask 255.255.255.0
    gateway 192.168.0.1
    up ip route delete via 192.168.0.1 dev eth0
    up ip route delete 192.168.0.0/24 via 0.0.0.0 dev eth0
    up ip route add 192.168.0.0/24 via 192.168.0.1 dev eth0

allow-hotplug wlan0
iface wlan0 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

After rebooting, the routing table remains the same as the default. boot.log doesn't show ifup eth0 throwing any errors. I'm at a loss as to what might be causing these issues.

Answer 1

So my issue was not one of concept, but the guides I drew my knowledge from were very outdated. This issue is of the class: Systemd strikes again!.

The current Raspbian distribution follows Debian Jessie, which uses systemd. And systemd doesn't follow everything from /etc/network/interfaces, but uses it to generate rules for /etc/dhcpcd.conf. Systemd bases all interface configuration, whether the addressing is static or dynamic, off of /etc/dhcpcd.conf.

To get the behavior I wanted, I first cut all configuration for eth0 out of /etc/network/interfaces besides

allow-hotplug eth0
iface eth0 inet static

Then I edited the interface eth0 section of /etc/dhcpcd.conf to...

interface eth0
static ip_address=192.168.0.10

Note, I didn't set static router=192.168.0.1. That would generate the default via 192.168.0.1 dev eth0 route I didn't want present (because it conflicts with the default route for wlan0).

With that minimal configuration, I can now use both interfaces to communication over their respective networks.