Implement pmtu discovery on udp connection that serves as vpn tunnel

Question

I've created a socket programmatically which is bound to virtual interface, and receive all outgoing traffic due to default routing table rule setting. Once outgoing packet arrive, it will be encapsulated with some VPN header and sent to the remote tunnel gateway through the physical interface.

I was wondering if I can implement any sort of path mtu discovery algorithm to the tunnel gateway, in order to avoid packet fragmentation and improve the performance.

one solution I was thinking of is to set the DF bit in the IP header for some random outgoing packets, and catch any icmp response with the following header :

Type: 3 (Destination Unreachable)
Code: 4 (Fragmentation Needed and Don't Fragment was Set)

than I keep response length and send icmp packet with reduced size until I reach the maximum packet size with DF=1 that get success response.

I was wondering if there are better ways to do it.. maybe using some built-in mechanism ?

Perhaps the following setting make this automatically (net.inet.tcp.path_mtu_discovery) ? if not, is there a way to read it somehow from the routing table cache ?

Thanks !

Answer 1

I was wondering if there are better ways to do it..

Set DF on all packets, not just on random ones. (Remember that "path MTU" implies that each path may have its own MTU – you might have 1500 to node B but 1280 to node C.)

maybe using some built-in mechanism ?

This is the mechanism that is usually built-in. With TCP, it's built into the TCP implementation of the OS; when you're building on top of UDP, you're basically implementing your own transport protocol, so it becomes your job to handle MTU discovery as well.

When setting the IP_MTU_DISCOVER socket option (the one that you would use to set the DF bit), Linux itself will read ICMP "Too big" responses and update its route cache. As mentioned in the ip(7) manual under IP_MTU_DISCOVER, you can retrieve the MTU using getsockopt(IP_MTU) without needing to parse ICMP yourself.

is there a way to read it somehow from the routing table cache ?

Create a UDP socket, connect() it to the endpoint's address, then retrieve the path MTU using getsockopt(IP_MTU) – also mentioned in the ip(7) manual under IP_MTU_DISCOVER.

---

Finally, keep in mind that not all routers correctly generate ICMP "Too big" and not all firewalls permit such packets to pass through, so it is unfortunately not enough to rely purely on IP-level PMTUD – you'll most likely want to implement a higher-layer mechanism as well. For example, TCP (at least on Linux) implements its own MTU discovery under the net.ipv4.tcp_mtu_probing sysctl; this is used whenever large TCP segments fail to be delivered.