PIX VPN CLIENTS disconnect after 10+ LAN to LAN tunnels created

[DrGreen26]

It appears that with a PIX 500 series firewall running IOS 6.3(4) or 6.3(5) has a problem.

We use our firewall to allow roughly 10 users to access the network via a vpn client connection. What I have found is that as I create point to point tunnels (ipsec LAN to LAN), once I get to a certain number of tunnels all of a sudden my vpn clients start dropping off for no apparent reason intermittently. I have duplicated this on a PIX 501, 506 and a 515E and for the life of me I cannot figure out why all of a sudden it is happening and why on 3 different firewalls in 3 different environments in 3 totally different locations. I also know that if I start removing the LAN to LAN tunnels that the vpn client connections stabalize and stay connected.

Below is a copy of my pix config (important info omitted)

Any ideas, thoughts or help would be appreciated..

Thanks

clock timezone EST -5
clock summer-time EDT recurring
fixup protocol dns maximum-length 512
fixup protocol ftp 21
fixup protocol h323 h225 1720
fixup protocol h323 ras 1718-1719
fixup protocol http 80
fixup protocol pptp 1723
fixup protocol rsh 514
fixup protocol rtsp 554
fixup protocol sip 5060
fixup protocol sip udp 5060
fixup protocol skinny 2000
fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol tftp 69
names

object-group network NMH-External-Devices
description AS400 located at Northern Michigan Hospital and IT Subnet for testing
network-object 10.15.200.54 255.255.255.255
network-object NMH-IT 255.255.255.0
object-group network PSSI-Internal
description Devices that access vpn tunnels from inside
network-object AS400 255.255.255.255
network-object 192.168.1.51 255.255.255.255
network-object AS400-2 255.255.255.255
network-object PSSI-Outside 255.255.255.255
object-group network SAMC-External
description South Alabama Medical Center remote tunnel access
network-object 10.0.24.113 255.255.255.255
object-group network conemaugh_external
description This is to Conemaugh medical center
network-object 172.17.24.62 255.255.255.255
object-group network PSSI-Outside
description This is the outside nat address for tunnel access to sites that require nat
network-object PSSI-Outside 255.255.255.255
object-group network ahss_external
network-object 204.139.127.114 255.255.255.255
object-group network ST_Francis
network-object 172.18.4.193 255.255.255.255
network-object 172.18.4.154 255.255.255.255
network-object 172.18.4.203 255.255.255.255
network-object 172.18.4.212 255.255.255.255
object-group network NHHN
description New Hanover Health Network AS 400 addresses
network-object 165.64.250.0 255.255.255.0
object-group network PSSI_NHHN
description IP Hosts to communicate to PSSI
network-object AS400 255.255.255.255
network-object AS400-2 255.255.255.255
network-object 192.168.1.51 255.255.255.255
object-group network midmichigan
description This is the remote network for Mid Michigan
network-object 192.168.81.0 255.255.255.248
object-group service as400 tcp
description These are the port restrictions for telnet and ftp only
port-object eq ftp
port-object eq telnet
object-group service as400-udp udp
description ICMP Protocol
object-group network somerset
description Somerset Health Network Remote Subnets
network-object 172.21.1.1 255.255.255.255
network-object 172.21.1.2 255.255.255.255
object-group network vpn-group
description object group used for acl nonat
network-object 192.168.102.0 255.255.255.0
object-group network wesley
description This object group contains the subnets to Wesley Health care
network-object 10.2.3.248 255.255.255.255
object-group network wafoote
description This is to WA Foote Health Care
network-object 10.2.4.10 255.255.255.255
network-object 10.2.4.20 255.255.255.255
object-group network gerber
description This goes to the Gerber Medical Health System
network-object 172.24.2.9 255.255.255.255
access-list inbound permit icmp any any
access-list inbound permit gre any any
access-list inbound permit tcp any any eq pptp
access-list inbound permit esp any any
access-list inbound permit ah any any
access-list inbound permit udp any any eq isakmp
access-list inbound permit udp any any eq 4500
access-list inbound permit ip 192.168.102.0 255.255.255.0 192.168.1.0 255.255.255.0
access-list outbound permit ip 192.168.1.0 255.255.255.0 10.15.200.0 255.255.255.0
access-list outbound permit ip object-group PSSI-Internal object-group NMH-External-Devices
access-list outbound permit ip object-group PSSI-Outside object-group SAMC-External
access-list outbound permit ip object-group PSSI-Outside object-group conemaugh_external
access-list outbound permit ip object-group PSSI-Outside object-group ahss_external
access-list outbound permit ip object-group PSSI-Outside object-group ST_Francis
access-list outbound permit ip object-group PSSI-Outside object-group NHHN
access-list outbound permit ip object-group PSSI-Outside object-group midmichigan
access-list outbound permit tcp object-group PSSI-Outside object-group midmichigan object-group as400
access-list outbound permit ip object-group PSSI-Internal object-group somerset
access-list outbound permit ip 192.168.102.0 255.255.255.0 any
access-list outbound permit ip any any
access-list outbound permit ip object-group vpn-group object-group ST_Francis
access-list outbound permit ip object-group PSSI-Outside object-group wesley
access-list outbound permit ip object-group PSSI-Internal object-group wafoote
access-list outbound permit ip object-group PSSI-Internal object-group gerber
access-list nmh-vpn permit ip object-group PSSI-Internal object-group NMH-External-Devices
access-list nmh-vpn permit ip object-group NMH-External-Devices object-group PSSI-Internal
access-list nonat permit ip object-group PSSI-Internal object-group NMH-External-Devices
access-list nonat permit ip 192.168.1.0 255.255.9.246 192.168.102.0 255.255.255.0
access-list nonat permit ip object-group PSSI-Internal object-group somerset
access-list nonat permit ip 192.168.1.0 255.255.255.0 object-group vpn-group
access-list samc permit ip object-group PSSI-Outside object-group SAMC-External
access-list conemaugh-vpn permit ip object-group PSSI-Outside object-group conemaugh_external
access-list ahss permit ip object-group PSSI-Outside object-group ahss_external
access-list stfrancis permit ip object-group PSSI-Outside object-group ST_Francis
access-list stfrancis permit ip object-group vpn-group object-group ST_Francis
access-list nhhn permit ip object-group PSSI-Outside object-group NHHN
access-list midmich permit ip object-group PSSI-Outside object-group midmichigan
access-list to-somerset permit ip object-group PSSI-Internal object-group somerset
access-list to-somerset permit ip object-group PSSI-Outside object-group somerset
access-list to-wafoote permit ip object-group PSSI-Internal object-group wafoote
access-list wesley permit ip object-group PSSI-Outside object-group wesley
access-list to-gerber permit ip object-group PSSI-Internal object-group gerber
pager lines 24
logging on
logging timestamp
logging standby
logging monitor debugging
logging buffered debugging
logging trap debugging
logging history critical
logging facility 16
logging device-id hostname
no logging message 105008
no logging message 105009
no logging message 106021
no logging message 305012
no logging message 305011
no logging message 305009
no logging message 710005
no logging message 710006
no logging message 302010
no logging message 111009
no logging message 111008
no logging message 302015
no logging message 302014
no logging message 302013
no logging message 304001
no logging message 111001
no logging message 111005
no logging message 111004
no logging message 111007
no logging message 609002
no logging message 609001
no logging message 302016
icmp permit any outside
mtu outside 1500
mtu inside 1500
ip address outside PSSI-Outside 255.255.255.248
ip address inside 000.00.0.0 255.255.255.0
ip verify reverse-path interface outside
ip verify reverse-path interface inside
ip audit name Attack-Policy attack action alarm drop
ip audit name Info-Policy info action alarm
ip audit interface outside Info-Policy
ip audit interface outside Attack-Policy
ip audit interface inside Info-Policy
ip audit interface inside Attack-Policy
ip audit info action alarm
ip audit attack action alarm drop
ip audit signature 2000 disable
ip audit signature 2001 disable
ip audit signature 2004 disable
ip audit signature 2151 disable
ip local pool vpnclient 192.168.102.1-192.168.102.20

arp timeout 14400
global (outside) 1 interface

nat (inside) 0 access-list nonat
nat (inside) 1 192.168.1.0 255.255.255.0 0 0
nat (inside) 1 192.168.102.0 255.255.255.0 0 0
nat (inside) 1 0.0.0.0 0.0.0.0 0 0
static (inside,outside) tcp PSSI-Outside pptp NT4server pptp netmask 255.255.255.255 0 0
access-group inbound in interface outside
access-group outbound in interface inside
route outside 0.0.0.0 0.0.0.0 65.82.220.241 1

timeout xlate 0:20:00
timeout conn 10:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225 1:00:00
timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute

aaa-server TACACS+ protocol tacacs+
aaa-server TACACS+ max-failed-attempts 3
aaa-server TACACS+ deadtime 10
aaa-server RADIUS protocol radius
aaa-server RADIUS max-failed-attempts 3
aaa-server RADIUS deadtime 10
aaa-server LOCAL protocol local
aaa authentication ssh console LOCAL
aaa authentication http console LOCAL
aaa authentication telnet console LOCAL
http server enable
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable

sysopt connection tcpmss 1200
sysopt connection permit-ipsec
crypto ipsec transform-set tunnel3desmd5 esp-3des esp-md5-hmac
crypto ipsec transform-set tunnel3dessha esp-3des esp-sha-hmac
crypto ipsec transform-set tunnel3desaes esp-aes esp-sha-hmac
crypto ipsec transform-set tunnelaes256 esp-aes-256 esp-sha-hmac
crypto ipsec transform-set tunneldessha esp-des esp-sha-hmac
crypto dynamic-map cisco 1 set transform-set tunnel3desmd5
crypto map pssi 1 ipsec-isakmp dynamic cisco
crypto map pssi 5 ipsec-isakmp
crypto map pssi 5 match address nhhn
crypto map pssi 5 set peer nhhn
crypto map pssi 5 set transform-set tunnel3desmd5
crypto map pssi 7 ipsec-isakmp
crypto map pssi 7 match address to-somerset
crypto map pssi 7 set peer somerset
crypto map pssi 7 set transform-set tunnel3desmd5
crypto map pssi 8 ipsec-isakmp
crypto map pssi 8 match address midmich
crypto map pssi 8 set peer midmichigan
crypto map pssi 8 set transform-set tunnel3desmd5
crypto map pssi 9 ipsec-isakmp
crypto map pssi 9 match address wesley
crypto map pssi 9 set peer wesley
crypto map pssi 9 set transform-set tunnelaes256
crypto map pssi 10 ipsec-isakmp
crypto map pssi 10 match address nmh-vpn
crypto map pssi 10 set peer NMH-External
crypto map pssi 10 set transform-set tunnel3desmd5
crypto map pssi 12 ipsec-isakmp
crypto map pssi 12 match address to-gerber
crypto map pssi 12 set peer 65.43.49.254
crypto map pssi 12 set transform-set tunnel3desmd5
crypto map pssi 15 ipsec-isakmp
crypto map pssi 15 match address samc
crypto map pssi 15 set peer samc
crypto map pssi 15 set transform-set tunnel3desmd5
crypto map pssi 18 ipsec-isakmp
crypto map pssi 18 match address to-wafoote
crypto map pssi 18 set peer wafoote
crypto map pssi 18 set transform-set tunnel3desmd5
crypto map pssi 20 ipsec-isakmp
crypto map pssi 20 match address conemaugh-vpn
crypto map pssi 20 set peer conemaugh
crypto map pssi 20 set transform-set tunnel3desmd5
crypto map pssi 25 ipsec-isakmp
crypto map pssi 25 match address stfrancis
crypto map pssi 25 set peer stfrancis
crypto map pssi 25 set transform-set tunnel3desmd5
crypto map pssi 30 ipsec-isakmp
crypto map pssi 30 match address ahss
crypto map pssi 30 set peer AHSS_RemotePeer
crypto map pssi 30 set transform-set tunnel3desmd5
crypto map pssi client authentication LOCAL
crypto map pssi interface outside
isakmp enable outside
isakmp key ******** address NMH-External netmask 255.255.255.255
isakmp key ******** address samc netmask 255.255.255.255
isakmp key ******** address conemaugh netmask 255.255.255.255
isakmp key ******** address AHSS_RemotePeer netmask 255.255.255.255
isakmp key ******** address stfrancis netmask 255.255.255.255
isakmp key ******** address nhhn netmask 255.255.255.255
isakmp key ******** address midmichigan netmask 255.255.255.255
isakmp key ******** address wesley netmask 255.255.255.255
isakmp key ******** address wafoote netmask 255.255.255.255 no-xauth no-config-mode
isakmp key ******** address somerset netmask 255.255.255.255 no-xauth no-config-mode
isakmp key ******** address 65.43.49.254 netmask 255.255.255.255
isakmp identity address
isakmp keepalive 10
isakmp nat-traversal 20
isakmp policy 1 authentication pre-share
isakmp policy 1 encryption 3des
isakmp policy 1 hash md5
isakmp policy 1 group 2
isakmp policy 1 lifetime 86400
isakmp policy 8 authentication pre-share
isakmp policy 8 encryption 3des
isakmp policy 8 hash md5
isakmp policy 8 group 1
isakmp policy 8 lifetime 86400
isakmp policy 10 authentication pre-share
isakmp policy 10 encryption 3des
isakmp policy 10 hash md5
isakmp policy 10 group 2
isakmp policy 10 lifetime 3600
isakmp policy 15 authentication pre-share
isakmp policy 15 encryption des
isakmp policy 15 hash sha
isakmp policy 15 group 1
isakmp policy 15 lifetime 86400
isakmp policy 20 authentication pre-share
isakmp policy 20 encryption 3des
isakmp policy 20 hash sha
isakmp policy 20 group 2
isakmp policy 20 lifetime 86400
isakmp policy 40 authentication pre-share
isakmp policy 40 encryption 3des
isakmp policy 40 hash md5
isakmp policy 40 group 2
isakmp policy 40 lifetime 28800
isakmp policy 60 authentication rsa-sig
isakmp policy 60 encryption des
isakmp policy 60 hash sha
isakmp policy 60 group 1
isakmp policy 60 lifetime 28800
isakmp policy 80 authentication pre-share
isakmp policy 80 encryption aes-256
isakmp policy 80 hash sha
isakmp policy 80 group 1
isakmp policy 80 lifetime 3600
vpngroup pssi address-pool vpnclient
vpngroup pssi default-domain pssiconsulting.com
vpngroup pssi split-tunnel nonat
vpngroup pssi idle-time 7200
vpngroup pssi password ********
telnet 0.0.0.0 0.0.0.0 inside
telnet timeout 30
ssh 0.0.0.0 0.0.0.0 outside
ssh timeout 30
console timeout 30

Mark C. Greenwood, CNE, CCNA, BICSI II


With more than 16 years experience to share.

 

[Supergrrover]

What's the cpu usage, memory usage and traffic level look like?
What vpn clients are you talking about? (I see pptp and ipsec.)

I would move this line down to the last numbered statement.
crypto map pssi 1 ipsec-isakmp dynamic cisco



Brent
Systems Engineer / Consultant
CCNP, CCSP

 

[DrGreen26]

Well all of my clients are using ipsec. This would be the cisco secure vpn client 4.0.8xxxx that is being used by my remote users.

I'll check the cpu usage and look at moving the crypto map pssi 1 back down to 99, i had it there initially but moved it up to the number one spot for troubleshooting..I had the same problem when it was set to 99..

I am not using any pptp connections but will double check that.

Mark

Mark C. Greenwood, CNE, CCNA, BICSI II


With more than 16 years experience to share.

 

[DrGreen26]

Almost forgot we use to have an nt server that everyone connected to via pptp..that serer has since died (as of last week) and I had to configure the pix for vpn clients...set it up using 3des md5...



Mark C. Greenwood, CNE, CCNA, BICSI II


With more than 16 years experience to share.

 

[Rock6D]

Sounds like it could be a possible licensing issue. Can you perform a show version and post?

 

[horus42]

Please do whatever you usually do and when the clients start droping off do "show blocks" and paste that here

Hope that helps

 

[NetworkGhost]

The max number of IPSEC peers on the pix is 10. See the following link:

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_data_sheet09186a0080148714.html

A good work around is to terminate your peers at the access router. (If you have one)

Free Firewall/Network/Systems Support- firewalls.ath.cx:8080

 

[Supergrrover]

He says that he is experiencing with other platforms as well.

" I have duplicated this on a PIX 501, 506 and a 515E"

For the 506 and the 515, there must be something else going on.



Brent
Systems Engineer / Consultant
CCNP, CCSP

 

[DrGreen26]

I actually have cisco tac running this by their engineers. Because they could not figure out why a 515e would do it and the 506 is capable of 25 ipsec lan to lan / client connections and I am no where close to this.

I actually have a typo because the other 501 is actually another 506. So 2 - 506's and a 515e. All in 3 different locations each configured for different lan to lan and vpn tunnels.

The only way to get things working was to remove the crypto maps from all three firewalls and get the number of crypto maps below 10.

Cisco has escalated this to their engineering group to test and the funny thing is they could not find any problems with any of my configurations other than changing the priority on the vpn client from highest to lowest then back to highest again with no change.



Mark C. Greenwood, CNE, CCNA, BICSI II


With more than 16 years experience to share.