Tc: ingress policing and ifb mirroring

Question

I'm trying to setup traffic shaping on a Linux gateway as written here. The script needs to be customized because I have multiple LAN interfaces. So to shape the LAN side I am planning to create a ifb pseudo device like so:

     modprobe ifb
     ip link set dev ifb0 up
    /sbin/tc qdisc add dev $WAN_INTERFACE ingress
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

The script from the gist repo mentioned above has these lines:

 /sbin/tc qdisc add dev $WAN_INTERFACE handle ffff: ingress
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip sport $INTERACTIVE_PORT 0xffff flowid :1
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip dport $INTERACTIVE_PORT 0xffff flowid :1
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 5 0 u32 match ip src 0.0.0.0/0 police rate $MAX_DOWNRATE_INGRESS burst 20k drop flowid :2

This code and the ifb interface creation code don't get on well together. The customized script gets executed, but ifb0 device doesn't show any traffic stats. If I comment out ingress gist repo code (quoted above), then ifb0 device shows the number of packets that are transferred. Also these lines cannot be executed together:

/sbin/tc qdisc add dev $WAN_INTERFACE ingress
/sbin/tc qdisc add dev $WAN_INTERFACE handle ffff: ingress

I get file exists error. So, how can I shape ingress on WAN_INTERFACE and at the same time also shape the traffic that goes to LAN via ifb0 device?

Answer 1

IFB is an alternative to tc filters for handling ingress traffic, by redirecting it to a virtual interface and treat is as egress traffic there.You need one ifb interface per physical interface, to redirect ingress traffic from eth0 to ifb0, eth1 to ifb1 and so on.

When inserting the ifb module, tell it the number of virtual interfaces you need. The default is 2:

modprobe ifb numifbs=1

Now, enable all ifb interfaces:

ip link set dev ifb0 up # repeat for ifb1, ifb2, ...

And redirect ingress traffic from the physical interfaces to corresponding ifb interface. For eth0 -> ifb0:

tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

Again, repeat for eth1 -> ifb1, eth2 -> ifb2 and so on, until all the interfaces you want to shape are covered.

Now, you can apply all the rules you want. Egress rules for eth0 go as usual in eth0. Let's limit bandwidth, for example:

tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 1mbit

Needless to say, repeat for eth1, eth2, ...

Ingress rules for eth0, now go as egress rules on ifb0 (whatever goes into ifb0 must come out, and only eth0 ingress traffic goes into ifb0). Again, a bandwidth limit example:

tc qdisc add dev ifb0 root handle 1: htb default 10
tc class add dev ifb0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 1mbit

The advantage of this approach is that egress rules are much more flexible than ingress filters. Filters only allow you to drop packets, not introduce wait times, for example. By handling ingress traffic as egress you can setup queue disciplines, with traffic classes and, if need be, filters. You get access to the whole tc tree, not only simple filters.

Answer 2

Based on Sérgio Carvalho answer I made small bash script to limit bandwidth:

File name: netspeed

#!/bin/bash 

#USAGE: sudo ./netspeed -l limit_in_kbit -s
usage="sudo $(basename "$0") -l speed_limit -s
  -l speed_limit - speed limit with units (eg. 1mbit, 100kbit, more on \`man tc\`)
  -s - remove all limits
"

# default values
LIMIT=0
STOP=0

# hardcoded constats
IFACE=ifb0 # fake interface name which will be used for shaping the traffic
NETFACE=wlan0 # interface which in connected to the internet

# shift all required and leave only optional

while getopts ':hl:s' option; do
  case "$option" in
   l) LIMIT=$OPTARG
      ;;
   s) STOP=1
      ;;
   h) echo "$usage"
      exit
      ;;
  esac
done

#
# functions used in script
#
function limitExists { # detected by ingress on $NETFACE qdisc
   # -n equals true if non-zero string length
  if [[ -n `tc qdisc show | grep "ingress .* $NETFACE"` ]]
  then
    return 0 # true
  else
    return 1 # false
  fi

}
function ifaceExists {
  # -n equals true if non-zero string length
  if [[ -n `ifconfig -a | sed 's/[ \t].*//;/^\(lo\|\)$/d' | grep $IFACE` ]]
  then
    return 0 # true
  else
    return 1 # false
  fi
}
function ifaceIsUp {
  # -n equals true if non-zero string length
  if [[ -n `ifconfig | sed 's/[ \t].*//;/^\(lo\|\)$/d' | grep $IFACE` ]]
  then
    return 0 # true
  else
    return 1 # false
  fi
}
function createLimit {
  #3. redirect ingress
  tc qdisc add dev $NETFACE handle ffff: ingress
  tc filter add dev $NETFACE parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev $IFACE

  #4. apply egress rules to local inteface (like wlan0)
  tc qdisc add dev $NETFACE root handle 1: htb default 10
  tc class add dev $NETFACE parent 1: classid 1:1 htb rate $LIMIT
  tc class add dev $NETFACE parent 1:1 classid 1:10 htb rate $LIMIT

  #5. and same for our relaying virtual interfaces (to simulate ingress)
  tc qdisc add dev $IFACE root handle 1: htb default 10
  tc class add dev $IFACE parent 1: classid 1:1 htb rate $LIMIT
  tc class add dev $IFACE parent 1:1 classid 1:10 htb rate $LIMIT
}
function updateLimit {
  #3. redirect ingress
  tc filter replace dev $NETFACE parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev $IFACE

  #4. apply egress rules to local inteface (like wlan0)
  tc class replace dev $NETFACE parent 1: classid 1:1 htb rate $LIMIT
  tc class replace dev $NETFACE parent 1:1 classid 1:10 htb rate $LIMIT

  #5. and same for our relaying virtual interfaces (to simulate ingress)
  tc class replace dev $IFACE parent 1: classid 1:1 htb rate $LIMIT
  tc class replace dev $IFACE parent 1:1 classid 1:10 htb rate $LIMIT
}
function removeLimit {
  if limitExists ; then
    tc qdisc del dev $NETFACE ingress
    tc qdisc del dev $NETFACE root
    tc qdisc del dev $IFACE root
  fi
  if ifaceIsUp ; then
    ip link set dev $IFACE down
  fi
}

#
# main script
#
if [[ `whoami` != "root" ]]; then
  echo "WARNING: script must be executed with root privileges!"
  echo $usage
  exit 1
fi
if [ $STOP -eq 1 ]; then
  echo "REMOVING limit"
  removeLimit
  echo "limit REMOVED"
elif [ "$LIMIT" != "0" ]; then
  # prepare interface
  if ! ifaceExists ; then
    echo "CREATING $IFACE by modprobe"
    modprobe ifb numifbs=1
    if ! ifaceExists ; then
      echo "creating $IFACE by modprobe FAILED"
      echo "exit with ERROR code 2"
      exit 2
    fi
  fi
  # set interface up
  if ifaceIsUp ; then
    echo "$IFACE is already up"
  else
    echo "set $IFACE up"
    ip link set dev $IFACE up # ( use ifconfig to see results)
    if ifaceIsUp ; then
      echo "$IFACE is up"
    else
      echo "enabling $IFACE by ip link FAILED"
      echo "exit with ERROR code 3"
      exit 3
    fi
  fi

  # create/update limits
  if limitExists ; then
    echo "update limit"
    updateLimit
  else
    echo "create limit"
    createLimit
  fi

  echo "limit CREATED"
  exit 0
else
  echo $usage
fi

Answer 3

You have a simple script that do it in a linux router box for incomming and outcomming traffic:

https://github.com/rfrail3/misc/blob/master/tc/traffic-control.sh