I'm trying to setup an OpenVPN client on ArchLinux with PrivateInternetAccess. By default PIA pushes routes such that all traffic goes through the VPN. I only want some applications to use the VPN.
To do this, I'm using "policy based routing". I've created a new routing table named "vpn", and will selectively send users to that routing table.
After these commands, the "media" user will be routed to the "vpn" routing table:
$ echo 100 vpn >> /etc/iproute2/rt_tables
$ iptables -t mangle -I OUTPUT -m owner --uid-owner media -j MARK --set-mark 0x1
$ ip rule add fwmark 0x1 table vpn
I've additionally modified my OpenVPN client configuration so that my routing tables will get populated correctly:
$ cat /etc/openvpn/client.conf
client
dev tun
proto udp
remote us-newyorkcity.privateinternetaccess.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca /etc/openvpn/ca.crt
tls-client
remote-cert-tls server
auth-user-pass /etc/openvpn/login.conf
comp-lzo
verb 1
reneg-sec 0
crl-verify /etc/openvpn/crl.pem
# This will override PIA so that traffic will route through our normal gateway
route 0.0.0.0 192.0.0.0 net_gateway
route 64.0.0.0 192.0.0.0 net_gateway
route 128.0.0.0 192.0.0.0 net_gateway
route 192.0.0.0 192.0.0.0 net_gateway
# Calling these scripts will add the PIA routes to the vpn table
script-security 2
up /etc/openvpn/up.sh
down /etc/openvpn/down.sh
$ cat /etc/openvpn/up.sh
#!/bin/sh
ip route add table vpn default via $ifconfig_remote
$ cat /etc/openvpn/down.sh
#!/bin/sh
ip route flush table vpn
Here are my routing tables after running openvpn /etc/openvpn/client.conf
:
$ ip route show table main
0.0.0.0/2 via 192.168.1.1 dev eth0
0.0.0.0/1 via 10.197.1.5 dev tun0
default via 192.168.1.1 dev eth0 src 192.168.1.124 metric 202
10.197.1.1 via 10.197.1.5 dev tun0
10.197.1.5 dev tun0 proto kernel scope link src 10.197.1.6
64.0.0.0/2 via 192.168.1.1 dev eth0
128.0.0.0/2 via 192.168.1.1 dev eth0
128.0.0.0/1 via 10.197.1.5 dev tun0
192.0.0.0/2 via 192.168.1.1 dev eth0
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.124 metric 202
209.95.50.86 via 192.168.1.1 dev eth0
$ ip route show table vpn
default via 10.197.1.5 dev tun0
Things run normally as a user that DOES NOT get routed to the "vpn" table:
$ whoami
jordan
$ ping -c 2 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=38.9 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=46 time=39.0 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 38.952/38.999/39.047/0.203 ms
However, things are failing for a user that DOES get routed to the "vpn" table:
$ whoami
media
$ ping -c 2 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
I took a look at tcpdump -i tun0
to see what's going on. It seems that icmp request are going out via tun0, but not coming back?
$ tcpdump -i tun0
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
15:37:30.134399 IP keep > google-public-dns-a.google.com: ICMP echo request, id 10256, seq 1, length 64
15:37:31.143217 IP keep > google-public-dns-a.google.com: ICMP echo request, id 10256, seq 2, length 64
Thoughts? :(
==== EDIT #1 ====
As a sanity check, if we send all traffic through the VPN (by removing the "route x.x.x.x 192.0.0.0 net_gateway" lines in /etc/openvpn/client.conf), we get icmp responses just fine:
$ tcpdump -i tun0
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
16:26:54.401732 IP keep > google-public-dns-a.google.com: ICMP echo request, id 10480, seq 1, length 64
16:26:54.483122 IP google-public-dns-a.google.com > keep: ICMP echo reply, id 10480, seq 1, length 64
16:26:55.403465 IP keep > google-public-dns-a.google.com: ICMP echo request, id 10480, seq 2, length 64
16:26:55.485068 IP google-public-dns-a.google.com > keep: ICMP echo reply, id 10480, seq 2, length 64
==== EDIT #2 ====
Following MariusMatutiae 's advice, I tried using the --route-noexec
flag to manually set the routes myself in /etc/openvpn/up.sh
. We also disable reverse path filtering in /etc/openvpn/up.sh
and re-enable it in /etc/openvpn/down.sh
:
$ cat /etc/openvpn/up.sh
#!/bin/sh
ip route add table vpn 0.0.0.0/1 via $ifconfig_remote
ip route add table vpn 128.0.0.0/1 via $ifconfig_remote
ip route add table vpn $route_network_1 via $ifconfig_remote
ip route add table vpn $trusted_ip via $route_net_gateway
ip route add table vpn $ifconfig_remote dev tun0 proto kernel src $ifconfig_local
ip route add table vpn 192.168.1.0/24 dev eth0 proto kernel src 192.168.1.124 metric 202
ip route del table main $ifconfig_remote
# Disable reverse path filtering
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
echo 0 > $f;
done
$ cat /etc/openvpn/down.sh
#!/bin/sh
ip route flush table vpn
# Re-enable reverse path filtering
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
echo 1 > $f;
done
Afterwards, my routing tables look like this:
$ ip route show table main
default via 192.168.1.1 dev eth0 src 192.168.1.124 metric 202
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.124 metric 202
$ ip route show table vpn
0.0.0.0/1 via 10.173.1.5 dev tun0
10.173.1.1 via 10.173.1.5 dev tun0
10.173.1.5 dev tun0 proto kernel scope link src 10.173.1.6
128.0.0.0/1 via 10.173.1.5 dev tun0
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.124 metric 202
209.95.50.133 via 192.168.1.1 dev eth0
However, user "media" is still unable to ping 8.8.8.8
. tcpdump -i tun0
still reports that no responses are coming back :(
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f; done
to yourup.sh
file, andfor f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 1 > $f; done
to yourdown.sh
file.rp_filter
was being set correctly withfor f in /proc/sys/net/ipv4/conf/*/rp_filter; do cat $f; done
between runs. Despite these changes, user "media" is still unable toping 8.8.8.8
:(rp_filter
is disabled, except I tried it in single user mode. For instance, usingping -I tun0 8.8.8.8
, orssh -b 10.197.1.6 me@remoteserver
. Can you check this on your system.ping -I tun0 8.8.8.8
as user "jordan" with all the configuration from "Edit #2". It works, but it's not passing the traffic totun0
according totcpdump -i tun0
...