I have a remote gateway to which I connect using two IKEv2 VTIs, OpenVPN, Wireshark, or a combination of these in /30 tunnels. Then create an ECMP route using OSPF (FRR) and the rest is like magic. I forward straight to the reverse proxy deep below the edge firewall, or the isolated PBX three networks down the edge, it all just works.
OSPF makes a flat transparent network out of a spidering multipathing topology. It's supposed to be overkill for my needs but I don't know any other way of setting up ECMP routes without a routing nightmare.
As far a load balancing goes, I can influence the preference of a tunnel over the other, though I'm not sure if that would make it stop being ECMP and more like a failover construct, so I stay off of that. The fact that it's a 2:1 public address ratio makes it tricky too, because the remote gateway might only choose a single gateway to reach the local endpoint instead of using both, hence I have not attempted to break a working setup just yet.
I've been reading about bonding/multipathing techniques, MLVPN, OpenVPN over MPTCP transport, ZeroTier with private moons, Glorytun, SOCKS over MPTCP, Project V, perhaps even TOR. I've also given a bit of headspace to bonding at the lower layers, for instance, tap mode OpenVPN (or similar) tunnels to bond EoIP, or multiple PPPoE AKA MLPPP, which I'm not sure but might have the least overhead since it might be an issue when double-tunneling. I played with GRE tunnels on top Hurricane Electric links before, and I learned I don't remember what but the usable MTU didn't meet a minimum for...whatever it was.
So I'm wondering if this is at all worth it or I'm just asking for trouble. After all, the purest ("least tunneled") form of inter-site communication I believe would be MPTCP but that would require at least SOCKS to carry UDP traffic, wouldn't it? Also, the tunnel serves as a mean to keep track of the target seen from the remote gateway without DDNS and it serves to avoid a second NAT on the traffic forwarded to on-prem. I'm afraid I might be missing a bigger picture though, because there must be a reason why they keep developing these techniques if dynamic routing protocols already solve most of the goals they seek to solve. Makes sense?
