Really weird debian issue: Can't connect to any services running on Debian from any public IP, works fine with private IPs

Question

First of all, it's not a port forwarding issue. By running tcpdump, I can see the requests getting to the debian server, and then they stop.

My debian server is running apache as well as PleX. If I connect to the Debian server using 192.168.1.210, it works flawlessly. I can see the web pages, and I can stream from PleX.

If I leave my network, say, I go to a friends house, I can't access either. Using tcpdump, I can see the packets get to the server, but that's it. Same with canyouseeme.org.

I do have some routing & iptables in place. I use this machine for torrenting + a VPN, so I use routing to keep everything working. The routing is supposed to keep PleX away from tun0, the VPN interface, and the iptables is supposed to keep the user debian-transmission from using anything other than tun0.

 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.172.1.5      128.0.0.0       UG    0      0        0 tun0
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
10.172.1.1      10.172.1.5      255.255.255.255 UGH   0      0        0 tun0
10.172.1.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun0
50.18.0.0       192.168.1.1     255.255.0.0     UG    0      0        0 eth0
54.241.0.0      192.168.1.1     255.255.0.0     UG    0      0        0 eth0
128.0.0.0       10.172.1.5      128.0.0.0       UG    0      0        0 tun0
184.72.0.0      192.168.1.1     255.255.192.0   UG    0      0        0 eth0
184.169.128.0   192.168.1.1     255.255.128.0   UG    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
216.144.236.186 192.168.1.1     255.255.255.255 UGH   0      0        0 eth0

iptables:

target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.1.0/24       owner UID match debian-transmission
REJECT     all  --  anywhere             anywhere             owner UID match debian-transmission reject-with icmp-port-unreachable

Answer 1

It's probablly easiest to describe what is happening by an example.

Lets say your NAT routers IP address is 1.1.1.1 and your friends IP address is 2.2.2.2 Lets also assume for the sake of simplicaity that your friend is not behind a nat router (it makes the example slightly more complicated if they are but it dioesn't fundamentaly change the problem. I'll also assume that the port forward is fowarding port 80 on your external IP to port 80 on the Debian box.

Your friend's computer sends a syn packet with a source address of 2.2.2.2, a randomly chosen source port (lets say 12345), a destination address of 1.1.1.1 a destination port of 80

The packet reaches your NAT, it looks up the port forward and changes the destination IP to 192.168.1.210. The source IP remains as 2.2.2.2, the ports remain the same. A mapping is created so that the reverse translation can be performed on returning packets so far so good.

The packet reaches your server. It gets delivered to the TCP stack which creates an ack packet in response. As normal when a packet is replied to source and destination are swapped. So the packet has a source address of 192.168.1.210 , a destination address of 2.2.2.2 a source port of 80 and a destination port of 12345.

The reply leaves the TCP stack and hits iptables. Your rules are performing a UID match so the owner matching works correctly the packet should get past there.

Then it hits the routing table. Which sends it down the VPN. It hits a NAT in the VPN which modifies it's source in some way, gets back to your friend and gets dropped due to having the wrong source address.

Possible fixes: 1: Add your friends IP to the routing table, obviously doesn't scale very well and may cause torrent traffic to leak. 2: If your nat box is a proper linux system it should be possibel to configure it to change the source as well as the destination of incoming connections. So your torrent box then sees the NAT box as the source rather than seeing your friends system on the internet. 3: use the "advanced routing" features in linux to route based on source port. Unfortunately the advanced routing features are very powerful but also difficult to understand. If you want more information on going down this route check out the "linux advanced routing and traffic control howto"