TL;DR: If you’re experiencing slow traffic on your VPN, try lowering the MSS size. This could prevent your router from segmenting packets and lead to a more efficient connection.
Background
I ran into an interesting problem when testing out a Wireguard VPN connection. We were using a /31 network with a connection between the two ends of the tunnel. Traffic was getting passed successfully, however in one direction we were seeing a 65% difference between the theoretical maximum and the speed of the connection. The other direction was about 90-95%. This was all using iperf – and when either of us ran iperf to a public server, both of us would get the right speed. We even opened a port to test our connection without the VPN, and we were getting about 95% of maximum – so all signs pointed to being some issue with the VPN itself.
Our initial thought was decryption on my pfSense device was only happening on one core, and even though the CPU was at 33%, it was taking up the entire bandwidth of one of the four cores (25%) and thus slowing down traffic. We had no real way of testing this, so we looked for other possibilities.
After checking Reddit and some forums, something that was recommended was setting MSS size to 1420. Amazingly, this worked. What happened?
TCP MSS Clamping
MSS means Maximum Segment Size. The image above is a visualization of this, but an easy way to think about it:
MTU – (TCP header + IP header) = MSS
When a TCP traffic goes through any kind of VPN tunnel, additional headers are added to the original packet to keep it secure. Because of these additional headers, the size of the encapsulated packet can go beyond the MTU (Maximum Transmission Unit) of the VPN interface. The packet can get fragmented or dropped based on the DF (“Don’t Fragment”) policy and this leads to inconsistencies and slowdowns across the network.
Layer 4 MSS – segment size – Transport Layer PDU
Layer 3 IP MTU – datagram size – Network layer PDU
Layer2 Interface MTU – frame size – Data Link layer PDU