1

I have a lossy (1%) 10gig link between geographic locations which supports legacy systems that have non-tuneable TCP stacks. They have horrible throughput across the link because their TCP stacks equate the loss with congestion, and dramatically reduce the window size. Modern systems, set to use BBR congestion control, and with appropriately sized buffers, achieve 7+ Gbps across the same link.

My question is, does anyone know of a TCP-based bridging or tunneling protocol linux supports? I can't use the typical UDP-based options (vxlan, geneve, gre), or IP protocol options (ipsec), because I've yet to find one that can be tuned to have that same level of performance, either due to design, or when faced with this 1% loss issue. I want to configure a bridge or tunnel between the sites, using modern systems that support BBR and tuning, so they hide the loss from the legacy systems talking across them.

Only option I've found so far has been openvpn set to TCP mode and with encryption disabled, but it didn't have the throughput.

Thanks

Improve this question
13
  • 2
    I'm not sure what you are expecting. If the problem is that the TCP stack of your systems cannot deal with the loss, then setting up a tunnel using this TCP stack will not improve anything - because the bad performance of outer TCP connection will propagate into the tunnel. Using a UDP or similar based tunnel and then use the systems TCP stack inside it will also not help - because the tunnel simply propagates the properties of the link, i.e. including the loss. You would need to completely bypass the systems TCP stack instead, i.e. use some user space implementation in your application.
    – Steffen Ullrich
    Commented Sep 27, 2023 at 17:14
  • Having a pair of client/server proxies as tunnel endpoint (on tcp ports) on each side of the link would avoid the poor tcp stacks to notice anything. I'd say layer 4 or layer 7 tunnels. simplest tool: socat. or nginx, haproxy ... for one side at least. PLUS the OS without suboptimal IP stack that goes along.
    – A.B
    Commented Sep 27, 2023 at 17:26
  • @SteffenUllrich the issue is that the devices in each location that want to talk to each other have inaccessible TCP stacks, reliant on out of date parameters and ancient congestion control algorithms. I can't fix them, and I can't fix the lossy circuit. However, if the ~1% lossy network between them is replaced by linux systems using a tcp-based tunnel, the tunnel endpoints will experience, mitigate, and hide the loss. With tuning and BBR as the congestion control algorithm, they could present what appears to be a multi-gig loss-free connection.
    – Colo Host
    Commented Sep 27, 2023 at 21:39
  • Thanks @A.B. I'd glanced at socat but came away thinking it would not work, perhaps because it can't just bind to an interface and forward all TCP, but I'll re-check. I've used haproxy before, but have not visited it for this issue; I'll see if there's a scenario I can come up with that it could help.
    – Colo Host
    Commented Sep 27, 2023 at 21:40
  • 1
    @SteffenUllrich I agree (with all you told btw). That's why I wrote "PLUS the OS without suboptimal IP stack that goes along." That meant a newer OS is needed to run the proxy (and I gave Linux as an example, to get the transparent proxy feature).
    – A.B
    Commented Sep 28, 2023 at 6:38

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .