How to route traffic between 2 interfaces via iptables?

Question

Introduction

The primary topic of the question is feeding DUT (Device Under Test) system via traffic generator (t-rex).

t-rex can be used from docker image - here is a doc page. The docker image already has downloaded t-rex and 2 virtual interfaces that are bounded to each other.

For better understanding here is a scheme of the built environment.

I need to forward all packets from veth0/veth1 to eth0. I configured iptables rules, but without successful result. The last chain where I can track all generated packets is FORWARD.

Environment configuring

Creating docker network

docker network create --driver=bridge --internal --subnet=172.28.0.0/16 docker-local-net
docker network ls # to check network has been created

Deploying t-rex container [t-rex container; terminal-0; 172.28.0.2]

docker pull trexcisco/trex
docker run -d --privileged --memory="4g" --memory-swap="4g" --cpus="2.0" --network=docker-local-net --name=trex-generator -it bash
docker exec -it trex-generator bash

Deploying DUT container [dut container; terminal-1; 172.28.0.3]

docker run -d --privileged --memory="4g" --memory-swap="4g" --cpus="2.0" --network=docker-local-net --name=network-log-server -it bash
docker exec -it network-log-server bash

nload eth0 # to monitor network interface activity

Note.0: DUT container has preinstalled tools and software.

Note.1: t-rex routes traffic between 2 virtual interfaces via the following config (/etc/trex_cfg.yaml):

 - port_limit    : 2
   version       : 2
   low_end       : true                 #1
   interfaces    : ["veth0", "veth1"]   #2
   port_info     :  # set eh mac addr
                 - ip         : 1.1.1.1
                   default_gw : 2.2.2.2
                 - ip         : 2.2.2.2
                   default_gw : 1.1.1.1

Packet routing configuring [t-rex container; terminal-0; 172.28.0.2]

# add default route
ip route add default via 172.28.0.3 dev eth0

# iptables table resetting
iptables -F && iptables -X
iptables -Z

# iptables nat table resetting
iptables -t nat -F && iptables -t nat -X
iptables -t nat -Z


# forward all input packets from veth0/veth1 to eth0
iptables -A FORWARD -i veth0 -o eth0 -j ACCEPT
iptables -A FORWARD -i veth1 -o eth0 -j ACCEPT

# NAT usage to route packet to DUT
iptables -t nat -A PREROUTING -i eth0 -j DNAT --to-destination 172.28.0.3
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Testing

tcpdump launching inside t-rex container to check eth0 activity [t-rex container; terminal-2; 172.28.0.2]

docker exec -it trex-generator bash
tcpdump -i eth0

t-rex launching [t-rex container; terminal-0; 172.28.0.2]

./t-rex-64 -f avl/sfr_delay_10.yaml -c 1 -d 120 -p
# wait few seconds and interrupt the process via ctrl + C

tcpdump stopping [t-rex container; terminal-2; 172.28.0.2]

...
# ctrl + C
11872 packets captured
11872 packets received by filter
0 packets dropped by kernel

Checking DUT container [dut container; terminal-1; 172.28.0.3]

# nload has not detected any packets

Checking iptables rules [t-rex container; terminal-0; 172.28.0.2]

#iptables -L -nv

Chain INPUT (policy ACCEPT 36332 packets, 3012K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
18419 9208K ACCEPT     all  --  veth0  eth0    0.0.0.0/0            0.0.0.0/0           
18704 9456K ACCEPT     all  --  veth1  eth0    0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 36332 packets, 3012K bytes)
 pkts bytes target     prot opt in     out     source               destination         

# iptables -t nat -L -nv

Chain PREROUTING (policy ACCEPT 4476 packets, 328K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    2   400 DNAT       all  --  eth0   *       0.0.0.0/0            0.0.0.0/0            to:172.28.0.3

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 13392 packets, 922K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 13392 packets, 922K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 4476  328K MASQUERADE  all  --  *      eth0    0.0.0.0/0            0.0.0.0/0 

Questions

• Why I see the different packet count in FORWARD chain and in tcpdump report? (FORWARD: 18704 + 18419 = 37123; tcpdump: 11872)

• How to properly forward ALL packets from veth0/veth1 to eth0?

• How to properly forward ALL packets from eth0:172.28.0.2 to 172.28.0.3 via NAT?

• If there are other configuration inaccuracies (not only routing related), please share your thought.

Answer 1

How to route traffic between 2 interfaces via iptables?

You don't. While technically you could, just use the proper routing mechanism instead of abusing iptables.

docker image

For docker containers, you use an bridge network between containers (in your example, dut and t-rex. Which you can set up using the docker tools.

If you try to do something with iptables while using docker, you are in for a lot of "fun", because docker extensively uses iptables rules, and whatever you do is likely to break what docker does either right now or in a future version.

And I am not familiar with t-rex, nor how the dockerization of it is supposed to work, nor could I quickly google information on this, but my guess is that it is configured so it automatically picks up up to two bridge networks, which you can then connect two up to two other containers. If that is not the case, I'd rather modify the t-rex configuration file then try to fiddle with iptables.