The Scenario: The 'Ghost' ConnectionEver had a VPN that claims to be 'connected,' yet refuses to do any actual work? It's a frustrating spot to be in. Pings to the server return instantly. Small DNS queries resolve without a hitch. But the moment you try to pull a 5MB PDF or load a heavy site like Salesforce, the browser spinner just loops until the connection times out. This isn't a total outage; it's a bottleneck.
This behavior is the hallmark of an MTU (Maximum Transmission Unit) mismatch. Somewhere between your laptop and the server, a tunnelâlikely IPsec, GRE, or VXLANâhas a smaller capacity than the standard 1500-byte Ethernet frame. When a large packet hits that narrow pipe, the router needs to break it up. However, modern TCP traffic uses the 'Don't Fragment' (DF) bit to stay efficient. If a packet is too big and can't be fragmented, the router simply drops it and sends back an ICMP error.
Spotting the error: Frag needed and DF setIf you're monitoring traffic with Wireshark or checking router logs during a failure, you'll see this red flag:
Frag needed and DF set (mtu = 1450)
The MTU value in the error tells you the exact ceiling of the next hop. Because most modern web traffic treats fragmentation as a performance killer, the packets don't just get smallerâthey disappear. Your computer keeps waiting for data that the router already threw away.
Finding the bottleneck with PingYou can identify the exact breaking point using a manual ping test. We do this by forcing a specific packet size and forbidding fragmentation. We start high and work our way down.
On Linux:
# Testing a 1500 byte packet (1472 payload + 28 headers)
ping -M do -s 1472 8.8.8.8
On Windows:
ping -f -l 1472 8.8.8.8
If you see 'Packet needs to be fragmented but DF set,' drop the size by 10 or 20 bytes (e.g., 1460, 1440, 1400) until you get a reply. Once you find the magic number that works, add 28 to it. That result is your Path MTU.
The Fix: Three ways to bridge the gap### Option 1: MSS Clamping (The Router-Side Fix)MSS Clamping is the most elegant solution if you manage the gateway. It intercepts the initial TCP handshake and 'tricks' both ends into agreeing on a smaller segment size from the start. This prevents large packets from ever being sent.
On a Linux firewall using iptables, use this rule to automatically align the MSS with your path discovery:
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
For Cisco admins, apply this directly to your tunnel interface to cap segments at 1360 bytes (a safe bet for most VPNs):
interface Tunnel0
ip tcp adjust-mss 1360
Option 2: Adjust the Local Interface MTUWhen you don't have control over the network hardware, you must change how your specific machine talks to the wire. Lowering the MTU on your NIC ensures every packet leaving the system fits the pipe.
On Linux:
# Identify your interface and set to 1400
sudo ip link set dev eth0 mtu 1400
On Windows (PowerShell):
# Set the MTU persistently for your 'Ethernet' adapter
netsh interface ipv4 set subinterface "Ethernet" mtu=1400 store=persistent
Option 3: Stop blocking ICMPPath MTU Discovery (PMTUD) relies on ICMP Type 3 Code 4 messages. Many security teams over-harden firewalls by dropping all ICMP traffic. If you block these 'Destination Unreachable' messages, the sender never learns that it needs to shrink its packets. Ensure your edge security policy allows this specific ICMP code so the network can self-heal.
Verifying the fixAfter applying a fix, go back to your ping test. If you lowered your MTU to 1400, a ping with a payload of 1372 (1372 + 28 = 1400) should now fly through. More importantly, try an actual data-heavy task. An scp file transfer or a large database export that previously hung at 0% should now saturate your bandwidth normally.
Lessons LearnedMTU headaches almost always stem from 'encapsulation overhead.' While standard Ethernet gives you 1500 bytes, every tunnel layer steals a piece of that pie:
- IPsec: Consumes ~50-70 bytes for encryption.- VXLAN: Adds a 50-byte overhead.- GRE: Takes 24 bytes.- PPPoE: Uses 8 bytes (common in DSL/Fiber home connections).When I'm architecting a new VPC or site-to-site link, I use tools like the Subnet Calculator to map out the network hierarchy early. Visualizing the CIDR blocks and potential tunnel overlaps helps prevent these 'ghost' connection issues before the first packet is even sent. If small packets work but big ones fail, check your MTU first.