3

I have a VM that is using a tap device (tap2) and my goal is to filter all the traffic in/out of tap2 using a custom program written by me that runs in userspace.

The topology I came up with routes traffic from tap2 to tap1 where my process is listening to frames and any that are allowed to go onwards to their destination are passed through to tap0 which is connected to enp0s25 using a traditional NAT setup.

                                  ┌─────────────────────────┐                                  
                                  │                         │                                  
                                  │                         │                                  
                                  │                         │                                  
                                  │Userspace filter process │                                  
                              ┌──▶│                         │───┐                              
                              │   │                         │   │                              
                              │   │                         │   │                              
┌────────┐        ┌────────┐  │   │                         │   │   ┌────────┐       ┌────────┐
│        │        │  tap1  │  │   └─────────────────────────┘   │   │  tap0  │       │        │
│  tap2  │───────▶│        │──┘                                 └──▶│        │──────▶│  enp0s25  │
│        │        │        │                                        │        │       │        │
└────────┘        └────────┘                                        └────────┘       └────────┘
     ▲                                                                                         
     │                                                                                         
     │                                                                                         
     │                                                                                         
┌────────┐                                                                                     
│        │                                                                                     
│   VM   │                                                                                     
│        │                                                                                     
└────────┘                                                                                     

I am stuck basically at the first portion of this topology. I have a simple python program that connects to tap1 and prints out any frames it sees: https://gist.github.com/shriphani/f98c212e0f4eb8d63ce3059a28509441

Then, I bring up the appropriate devices:

tap1:

sudo ip tuntap add tap1 mode tap user shriphani
sudo ip addr add 172.19.0.1/24 dev tap1
sudo ip link set tap1 up

tap2:

sudo ip tuntap add tap2 mode tap user shriphani
sudo ip addr add 172.20.0.1/24 dev tap2
sudo ip link set tap2 up

Following this, I added a route from tap2 to tap1:

sudo ip route add 172.20.0.0/24 via dev tap1

which results in this error message:

RTNETLINK answers: File exists

and this is the routing table:

default via 192.168.1.1 dev enp0s25 proto dhcp src 192.168.1.16 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.19.0.0/24 dev tap1 proto kernel scope link src 172.19.0.1 linkdown
172.20.0.0/24 dev tap2 proto kernel scope link src 172.20.0.1
192.168.1.0/24 dev enp0s25 proto kernel scope link src 192.168.1.16
192.168.1.1 dev enp0s25 proto dhcp scope link src 192.168.1.16 metric 100

Which is doubly weird since I don't see that exact route I asked for in the routing table.

Anyway, I then added these iptables commands:

sudo iptables -A FORWARD -i tap2 -o tap1 -j ACCEPT
sudo iptables -t nat -A PREROUTING --in-interface tap2 -j DNAT --to-destination 172.19.0.2
sudo iptables -t nat -A POSTROUTING --out-interface tap1 -j MASQUERADE

and my understanding is, I allow traffic from tap2 -> tap1, route all tap2 to tap1 (to that particular IP addr - not sure if this is right), and then all outgoing traffic from tap1 appears to be coming from tap1 (so far I have no outgoing traffic yet).

When I run my python script attached to tap1, and wget google.com from the VM, I see absolutely nothing printed. Pinging the tap IP range from the host causes the script to respond (i.e. I do ping 172.19.0.2 on the host and the script prints something).

So I have 2 questions about this setup:

  1. Is this topology sensible or are there simpler ways of achieving my goals
  2. If this is a good topology, what exact set of rules do I need to force all traffic from the VM (tap2) to first appear at tap1 following which I can send it out.

Thank you very much. Any help would be appreciated.

4
  • The routes confuse me a lot. What exactly is the point of 172.20.0.0/24 via dev tap1? That subnet doesn't belong to tap1, does it? What's the IP address that the actual VM has? Commented Feb 3, 2022 at 5:56
  • 172.20.0.0/24 is the address range of tap2 (where the VM is on). I am using firecracker to manage my VM and that takes up the whole tap device for itself. The VM itself is on 172.20.0.2 Commented Feb 3, 2022 at 7:57
  • Exactly, so it makes no sense to route that /24 via tap1 in your case. Commented Feb 3, 2022 at 8:34
  • Ok, what rules do you think I should have if I want all tap2 traffic to go through tap1? Commented Feb 3, 2022 at 15:39

0

You must log in to answer this question.

Browse other questions tagged .