Problem with IPSEC over VPN

[sobak]

I need help with this one BAD. I have a VPN setup between a 2611 and a 7120 router. Both connections are running T-1 Frame Relay to the internet. Connection response is about 50-60ms on a ping reply. The issue that I am running into is this... The VPN gets established but after about 20 minutes it starts to drop packets until it finally drops the VPN all together. Shutting down the interface and then restarting has no effect. Debug on the Crypto Engine states....

CRYPTO_ENGINE: packets dropped: State = 0 conn_id=2002, pak=81B116DC

Two ways will bring the VPN back on-line. Rebooting the remote router (2611) or running these two commands...

clear crypto sa
clear crypto isakmp

Once those two commands are issued the VPN begins to respond again. I have three other VPN's setup on this router and they are working fine, this particular connection is the problem. I've had 5 Cisco Engineers look at it and the final diagnosis was that it was a response time from the internet ISP. On their advice I changed the carrier to the same one we are using on the 7120, we went on-line with that connection today. about 20 minutes after the VPN was restored, it failed again. Everything has been done to this router that I can think of, please anyone that has experience with Cisco IPSEC and VPN's please respond!!!!!

 

[wybnormal]

I hate to ask but have you tried a different level of code? From your descripton, I would guess some type of memory leak. I have not had to use this piece of Cisco ( yet). Have you opened an official TAC trouble ticket?

How far apart are the sites? the times you mention are somewhat slow unless you are running crosscountry. Even then, my longest link at 3200 miles is running about 60-80mS

Mike S
"Diplomacy; the art of saying 'nice doggie' till you can find a rock" Wynn Catlin

 

[sobak]

I have had this case opened three times with Cisco with 5 different Cisco engineers looking at it, Today I am going to open another case with them to address the hardware. The connection is a cross-country vpn, the connection is from the Dallas/Fort Worth to Miami-Dade area. On our last provider we were seeing ping times of about 300ms - 400ms, due to these response times it was pretty logical that it could be a timeout issue with the VPN. After fighting with management to move the connection to a different provider they are sending a little heat my direction because the situation has not improved.

Our Enterprise router (7120) is currently working with three additional VPN's out of that hardware, those are up and working fine without any issues, the only one is this particular connection. I didn't have a need for looking into my code since it had been tested on similar hardware though other connections (although since I've only been working with Cisco connections for about a year I can't rule that out). I am sure that it will take yet a forth call to Cisco to fix the issue, whatever I find out I will make sure I post the solution here to assist others. I guess I opened up this call in desperation of not being able to fix it after working on it for so long.

Thanks for your reply, I really appreciate it a lot....

david e

 

[wybnormal]

I guess my next step would be see what the difference is between the two remote sites working the one that is not. All running the same 2611 router? All running the same Telco provider? I have heard of squirely problems running VPNs through certain Telcos.. perhaps thats a thought. I for one would be very interested in the final resolution of this problem as it's only a matter of time before I get to work on the same problems.. whoopie ;-)

Mike S
"Diplomacy; the art of saying 'nice doggie' till you can find a rock" Wynn Catlin

 

[sobak]

I've ran through verifying the configurations between the ones that work with the one that doesn't (I have one connection to New York that goes into a 2611) and they are virtually identical. Telco's have been eliminated, last provider was running on Cable & Wireless, the new provider is running Bell South.

Whatever I find out I will make sure I post an update to it here. Cisco has the call now and I have given them my views concerning the hardware. Good point of this is that I've sure got some very good Cisco troubleshooting experience.

Thanks, will update when I get more information

david e

 

[sobak]

Okay here is my answer to this problem for all that may run into this in the future......

Upon talking to Cisco, they had me upgrade to a new IOS for the 2600, IOS 12.1.5(T), since the new IOS was about 12 megs in size I had to get some more memory for my router. The combination of Memory and the new IOS solved the problem. My VPN is up and running and has been for about 24 hours now.

david e