Packet loss over IPSEC L2L tunnel

[Staticfactory]

I have an L2L Tunnel between an ASA5540 and an ASA5505 that has been working correctly for months. No updates have taken place, but I'm now seeing about 50% packet loss between the sites.

The isakmp/ipsec tunnels are active and are not displaying any errors/drops/discards. I don't see any errors in the log.

Traffic to the outside from both sites is unaffected - only traffic between the sites. If it wasn't for this fact I would assume it would have to do with the ISP.

I have cleared the SAs, checked debug output, restarted devices, and I can't see anything that would cause this type of behavior.

I was hoping someone here has experienced a similar issue or can think of different troubleshooting steps that I can try. ANY ideas would be helpful.

 

[usfregale]

Okay, I'm very new to the ASA, but here's a hunch.

If you have QOS in place I would guess that your low latency queue is filling and resulting in tail drop. That might not show on the tunnels themselves since technically the data is still in the low latency queue. Anyway, that's a theory. Check if it holds water by doing a show priority-queue statistics (interface). Check for drop in the LLQ.

Assuming that that's the issue you could resolve it by increasing the size of the LLQ itself (depending on the load on your ASA and the amount of free memory), the portion of your pipe devoted to the LLQ, or increasing the size of the pipe itself.

Of course you might not be running QOS at all.

Richard

 

[Staticfactory]

Thanks for your reply Richard... that would certainly be a good place to look if we were running QoS on either of the ASAs in question. I was also thinking that it could be related to MTU but it's an issue that just started to appear across the entire subnet.

Fortunately for me, the issue seems to have cleared itself up over night (after 18 hours of packet loss no less). While I would really like to know what the deal was, it is impossible to troubleshoot if I can't replicate it. At this point I'll gladly take my uptime and walk away.