Options for bandwidth management on a shared internet connection

Question

The premise:

Rural location, not many options for high-bandwidth internet connections. The fastest by far is Satellite internet, but it's expensive (both equipment and monthly cost, and has latency drawbacks), so multiple households would like to share it.

This is straightforward in itself - set up a router with a subnet for each household (VLANs or physical ports), block traffic between them and either configure the modem to route to those subnets or set up double NAT.

The problem:

The way TCP normally works means that each TCP connection will more or less get 1/n of the available bandwidth, where n is the number of connections. So if one household/user creates lots of connections, they'll get a bigger share of the overall connection. This isn't particularly fair - with a saturated link, each household should get an equal share. On the other hand, when nobody else is using the connection, it should be possible to use the full bandwidth.

For example, say there are 4 households sharing a 12Mbit/s down connection. If one of them is downloading/streaming/whatever, they should be able to use the full 12Mbit/s (or near enough). If 2 households are using the connection, they should get 6Mbit/s each, regardless whether one household is downloading 1 file and the other 11. (without any bandwidth management, each file would download at roughly 1Mbit/s in that case) 3 households get 4Mbit/s each and so on.

What I've worked out so far:

The best location to implement a policy like this (for the downstream) would be at the other end of the narrow pipe, i.e. at the ISP. Obviously, that's not possible in this case, so it would be good to be able to approximate it somehow. But how? Are there off-the-shelf routers that support something like this? Can I configure a Linux or BSD box to do it? It doesn't have to be bullet-proof - a misbehaving TCP server or aggressive UDP service could presumably circumvent anything I can do at my end - but it should work for the common case of most of the traffic consisting of lots of RFC-compliant TCP connections.

To be clear, I'm not talking about prioritising specific applications, but rather aggregate traffic to/from particular ethernet devices or IP address ranges. Giving some traffic a higher priority than other traffic seems to be well-supported, but the situation is less clear when trying to equally allocate bandwidth to classes of traffic.

There's a lot of badly written/unthinkingly regurgitated/otherwise uninformed information on traffic shaping on the web. The documentation on router hardware is terribly unspecific, so I seem to be running around in circles.

As I understand it, the way to get TCP to behave this way is to in effect simulate a slightly narrower pipe than is really available and artificially drop packets to get it to back off. So I think it would be fairly straightforward to give everyone exactly 3Mbit/s in the above example by artificially dropping any extra packets. This doesn't really use the connection efficiently as most of the time there's spare capacity.

Is there a way to do what I'm asking? Am I going about it wrong? I am (or rather the households in questions are) willing to spend money on this - be it a moderately priced off-the-shelf router/appliance or a generic box to run a Linux or BSD distro.

Answer 1

If I were to build something to solve this problem, I would setup this :

We are here going to take your example of 4 computers sharing one link. The network would be shaped like this :

{ ISP }=========[ SOHO router] ===LAN1=== [eth0 |Linux Box| eth1] ===LAN2=== Home Desktops/Laptops

Let's say you use static IP addressing in LAN2:

Linux Box 192.168.1.1
Home 1    192.168.1.11
Home 2    192.168.1.12
Home 3    192.168.1.13
Home 4    192.168.1.14

I would first write a small script to detect which clients are up by pinging them and writing result somewhere. "This is left as an exercise for the reader" Calculate the total number of connected hosts (1-4)

Be sure to handle the case of no hosts online.

Divide the total bandwith but the number of connected hosts. (12Mb for 1, 6Mb for 2, 4Mb for 3, 3Mb for 4)

Next use tc with the HTB algorithm to limit the bandwidth of each address on WAN-side device. First, create the root class :

DEV="eth0"
TC="/sbin/tc"
TOT_BW=12
$TC qdisc add dev $DEV root handle 1: htb default 99

Then, add a class for each "client" (Here, for example, 3 clients)

NB_CLT=3
CLT_BW=$(($TOT_BW/$NB_CLT))mbit
for i in seq $NB_CLT
do
    $TC class add def $DEV parent 1: classid 1:$i htb rate $CLT_BW ceil $CLT_BW burst 15k cburst 1500
    $TC filter add dev $DEV protocol ip parent 1:0 prio 1 flowid 1:$i u32 \
     match ip dst 192.168.1.1$i/32 \
     match ip src [Router IP in LAN1]/32
done

(above script is totally untested and NOT typing-error-free)

Now you still have to properly script that to be run every minute and renew buckets/filters every time a new host goes up or down.

I hope that will point you in the right direction, good luck.

Alternate Solution

Install a local QoS software on all hosts to restrict their bandwidth. I Personnaly use NetLimiter (Non-free)

Answer 2

OpenWRT does seem to support it, though I have never used it myself. You can take a look at the Network Traffic Control page on their website, and especially the second example: plain simple bandwidth sharing (aka traffic shaping) with HTB. This involves simple calls to qdisc, so any Linux box could do it.

Answer 3

The setup described in https://superuser.com/a/1210164/257859 does exactly that:

[...] the queues distribute that limited BW evenly between all the LAN clients (LAN IPs to be precise).