SOHO 91 / PIX 506e VPN Problem 1

[rodneyws]

My present employer is in the process of installing a VPN to connect various remotes sites to the hospital where I work. Presently, 3 sites are online and functioning correctly. However, the 4th and 5th sites I've attempted to bring online have not been able to bring up the VPN tunnel necessary to access the hospital network. However, each of these sites has access to the internet through their router and I can communicate with their router from this site. I believe the problem to be on the hospital side.

At the main site is a Cisco PIX 506e and at each of the remote sites is a Cisco SOHO 91 router connected to a DSL internet connection.

10.1.X.X is the network address of the hospital. 10.2.0.X, 10.4.0.X, and 10.6.0.X are the addresses of functioning remote sites connected to the VPN. 10.5.0.X and 10.7.0.X are the network addresses of the two remote sites I am unable to connect to the VPN. Both exhibit the same problem so I will only be posting the config for 10.7.0.X.

Here is the running-config from the PIX 506e:

PIX Version 6.3(3)
interface ethernet0 auto
interface ethernet1 auto
nameif ethernet0 outside security0
nameif ethernet1 inside security100
enable password DELETED encrypted
passwd DELETED
hostname CRHSEXTSVC
domain-name crispregional.org
fixup protocol dns maximum-length 512
fixup protocol ftp 21
fixup protocol h323 h225 1720
fixup protocol h323 ras 1718-1719
fixup protocol http 80
fixup protocol rsh 514
fixup protocol rtsp 554
fixup protocol sip 5060
fixup protocol sip udp 5060
fixup protocol skinny 2000
fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol tftp 69
names
access-list ipsec permit ip 10.1.0.0 255.255.0.0 10.2.0.0 255.255.255.0
access-list nonat permit ip 10.1.0.0 255.255.0.0 10.4.0.0 255.255.255.0
access-list nonat permit ip 10.1.0.0 255.255.0.0 10.2.0.0 255.255.255.0
access-list nonat permit ip 10.1.0.0 255.255.0.0 10.6.0.0 255.255.255.0
access-list nonat permit ip 10.1.0.0 255.255.0.0 10.5.0.0 255.255.255.0
access-list nonat permit ip 10.1.0.0 255.255.0.0 10.7.0.0 255.255.255.0
access-list 101 permit icmp any any
access-list ipsec2 permit ip 10.1.0.0 255.255.0.0 10.4.0.0 255.255.255.0
access-list ipsec3 permit ip 10.1.0.0 255.255.0.0 10.6.0.0 255.255.255.0
access-list ipsec4 permit ip 10.1.0.0 255.255.0.0 10.5.0.0 255.255.255.0
access-list ipsec5 permit ip 10.1.0.0 255.255.0.0 10.7.0.0 255.255.255.0
pager lines 24
logging buffered debugging
mtu outside 1500
mtu inside 1500
ip address outside 65.168.125.109 255.255.255.248
ip address inside 10.1.100.4 255.255.0.0
ip audit info action alarm
ip audit attack action alarm
pdm history enable
arp timeout 14400
global (outside) 1 interface
nat (inside) 0 access-list nonat
nat (inside) 1 0.0.0.0 0.0.0.0 0 0
access-group 101 in interface outside
conduit permit icmp any any
route outside 0.0.0.0 0.0.0.0 65.168.125.105 1
route outside 65.13.194.214 255.255.255.255 65.168.125.105 1
route outside 67.98.252.16 255.255.255.255 65.168.125.105 1
route outside 67.141.189.225 255.255.255.255 65.168.125.105 1
route outside 68.157.160.145 255.255.255.255 65.168.125.105 1
route outside 209.164.236.244 255.255.255.255 65.168.125.105 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225 1:00:00
timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
aaa-server LOCAL protocol local
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable
sysopt connection permit-ipsec
crypto ipsec transform-set network esp-des esp-md5-hmac
crypto ipsec security-association lifetime seconds 3600
crypto map mymap 1 ipsec-isakmp
crypto map mymap 1 match address ipsec2
crypto map mymap 1 set peer 67.141.189.225
crypto map mymap 1 set transform-set network
crypto map mymap 2 ipsec-isakmp
crypto map mymap 2 match address ipsec
crypto map mymap 2 set peer 65.13.194.214
crypto map mymap 2 set transform-set network
crypto map mymap 3 ipsec-isakmp
crypto map mymap 3 match address nonat
crypto map mymap 3 set peer 209.164.236.244
crypto map mymap 3 set transform-set network
crypto map mymap 4 ipsec-isakmp
crypto map mymap 4 match address ipsec4
crypto map mymap 4 set peer 68.157.160.145
crypto map mymap 4 set transform-set network
crypto map mymap 5 ipsec-isakmp
crypto map mymap 5 match address ipsec5
crypto map mymap 5 set peer 67.98.252.16
crypto map mymap 5 set transform-set network
crypto map mymap 5 set transform-set network
isakmp enable outside
isakmp key ******** address 67.141.189.225 netmask 255.255.255.255
isakmp key ******** address 65.13.194.214 netmask 255.255.255.255
isakmp key ******** address 209.164.236.244 netmask 255.255.255.255
isakmp key ******** address 68.157.160.145 netmask 255.255.255.255
isakmp key ******** address 67.98.252.16 netmask 255.255.255.255
isakmp identity address
isakmp nat-traversal 20
isakmp policy 21 authentication pre-share
isakmp policy 21 encryption des
isakmp policy 21 hash md5
isakmp policy 21 group 1
isakmp policy 21 lifetime 86400
telnet timeout 5
ssh 0.0.0.0 0.0.0.0 outside
ssh timeout 60
management-access inside
console timeout 0
username admin password DELETED encrypted privilege 15
username cisco password DELETED encrypted privilege 15
terminal width 80

Here is the running-config from the Cisco SOHO 91 router being used by the 10.7.0.1 network:

Current configuration : 3257 bytes
!
version 12.3
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname Vienna
!
no logging buffered
enable secret 5 DELETED
!
username admin privilege 15 password 7 DELETED
no aaa new-model
ip subnet-zero
!
!
ip inspect name myfw cuseeme timeout 3600
ip inspect name myfw ftp timeout 3600
ip inspect name myfw rcmd timeout 3600
ip inspect name myfw realaudio timeout 3600
ip inspect name myfw smtp timeout 3600
ip inspect name myfw tftp timeout 30
ip inspect name myfw udp timeout 15
ip inspect name myfw tcp timeout 3600
ip inspect name myfw h323 timeout 3600
!
!
!
!
crypto isakmp policy 11
hash md5
authentication pre-share
!
crypto isakmp policy 12
hash md5
authentication pre-share
crypto isakmp key 0 DELETED address 65.168.125.109
!
!
crypto ipsec transform-set sharks esp-des esp-md5-hmac
!
crypto map nolan 1 ipsec-isakmp
set peer 65.168.125.109
set transform-set sharks
match address 120
!
!
!
!
interface Tunnel0
no ip address
!
interface Ethernet0
ip address 10.7.0.1 255.255.255.0
ip nat inside
no ip route-cache
ip tcp adjust-mss 1348
no cdp enable
hold-queue 32 in
!
interface Ethernet1
no ip address
duplex auto
pppoe enable
pppoe-client dial-pool-number 1
no cdp enable
!
interface Dialer1
ip address negotiated
ip nat outside
encapsulation ppp
dialer pool 1
dialer-group 1
ppp authentication chap pap callin
ppp chap hostname crhsvim
ppp chap password 7 DELETED
ppp pap sent-username crhsvim password 7 DELETED
ppp ipcp dns request
ppp ipcp wins request
crypto map nolan
!
ip nat pool branch 67.98.252.16 67.98.252.16 netmask 255.255.255.2
ip nat inside source route-map nonat interface Dialer1 overload
ip classless
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.1.0.0 255.255.0.0 65.168.125.109
ip http server
no ip http secure-server
!
access-list 23 permit 10.7.0.0 0.0.0.255
access-list 111 permit icmp any any administratively-prohibited
access-list 111 permit icmp any any echo
access-list 111 permit icmp any any echo-reply
access-list 111 permit icmp any any packet-too-big
access-list 111 permit icmp any any time-exceeded
access-list 111 permit icmp any any traceroute
access-list 111 permit icmp any any unreachable
access-list 111 permit udp any eq bootps any eq bootpc
access-list 111 permit udp any eq bootps any eq bootps
access-list 111 permit udp any eq domain any
access-list 111 permit esp any any
access-list 111 permit udp any any eq isakmp
access-list 111 permit udp any any eq 10000
access-list 111 permit tcp any any eq 1723
access-list 111 permit tcp any any eq 139
access-list 111 permit udp any any eq netbios-ns
access-list 111 permit udp any any eq netbios-dgm
access-list 111 permit gre any any
access-list 111 deny ip any any
access-list 120 permit ip 10.7.0.0 0.0.0.255 10.1.0.0 0.0.255.255
access-list 130 deny ip 10.7.0.0 0.0.0.255 10.1.0.0 0.0.255.255
access-list 130 permit ip 10.7.0.0 0.0.0.255 any
dialer-list 1 protocol ip permit
route-map nonat permit 10
match ip address 130
!
!
line con 0
exec-timeout 120 0
no modem enable
stopbits 1
line aux 0
line vty 0 4
exec-timeout 120 0
login local
length 0
!
scheduler max-task-time 5000
!
end

Any thoughts or sugguestions on why the VPN tunnel will not come up would be greatly appreciated.

 

[JOAMON]

Can you post working SOHO config for comparison?

 

[rodneyws]

I've pretty much used the working routers as templates for the new router configurations... just changing IPs and a few other changes.

This is the running-config from a SOHO 91 at a functional site:

Building configuration...

Current configuration : 3310 bytes
!
version 12.3
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname Progressive
!
no logging buffered
enable secret 5 DELETED
!
username admin privilege 15 password 7 DELETED
no aaa new-model
ip subnet-zero
!
!
ip inspect name myfw cuseeme timeout 3600
ip inspect name myfw ftp timeout 3600
ip inspect name myfw rcmd timeout 3600
ip inspect name myfw realaudio timeout 3600
ip inspect name myfw smtp timeout 3600
ip inspect name myfw tftp timeout 30
ip inspect name myfw udp timeout 15
ip inspect name myfw tcp timeout 3600
ip inspect name myfw h323 timeout 3600
!
!
!
!
crypto isakmp policy 11
hash md5
authentication pre-share
!
crypto isakmp policy 12
hash md5
authentication pre-share
crypto isakmp key 0 DELETED address 65.168.125.109
!
!
crypto ipsec transform-set sharks esp-des esp-md5-hmac
!
crypto map nolan 1 ipsec-isakmp
set peer 65.168.125.109
set transform-set sharks
match address 120
!
!
!
!
interface Tunnel0
no ip address
!
interface Ethernet0
ip address 10.2.0.1 255.255.255.0
ip nat inside
no ip route-cache
ip tcp adjust-mss 1348
no cdp enable
hold-queue 32 in
!
interface Ethernet1
no ip address
duplex auto
pppoe enable
pppoe-client dial-pool-number 1
no cdp enable
!
interface Dialer1
ip address negotiated
ip nat outside
encapsulation ppp
dialer pool 1
dialer-group 1
ppp authentication chap pap callin
ppp chap hostname rehabprogressive@bellsouth.net
ppp chap password 7 DELETED
ppp pap sent-username rehabprogressive@bellsouth.net password 7 DELETED
ppp ipcp dns request
ppp ipcp wins request
crypto map nolan
!
ip nat pool branch 65.13.194.214 65.13.194.214 netmask 255.255.255.240
ip nat inside source route-map nonat interface Dialer1 overload
ip classless
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.1.0.0 255.255.0.0 65.168.125.109
ip http server
no ip http secure-server
!
access-list 23 permit 10.2.0.0 0.0.0.255
access-list 111 permit icmp any any administratively-prohibited
access-list 111 permit icmp any any echo
access-list 111 permit icmp any any echo-reply
access-list 111 permit icmp any any packet-too-big
access-list 111 permit icmp any any time-exceeded
access-list 111 permit icmp any any traceroute
access-list 111 permit icmp any any unreachable
access-list 111 permit udp any eq bootps any eq bootpc
access-list 111 permit udp any eq bootps any eq bootps
access-list 111 permit udp any eq domain any
access-list 111 permit esp any any
access-list 111 permit udp any any eq isakmp
access-list 111 permit udp any any eq 10000
access-list 111 permit tcp any any eq 1723
access-list 111 permit tcp any any eq 139
access-list 111 permit udp any any eq netbios-ns
access-list 111 permit udp any any eq netbios-dgm
access-list 111 permit gre any any
access-list 111 deny ip any any
access-list 120 permit ip 10.2.0.0 0.0.0.255 10.1.0.0 0.0.255.255
access-list 130 deny ip 10.2.0.0 0.0.0.255 10.1.0.0 0.0.255.255
access-list 130 permit ip 10.2.0.0 0.0.0.255 any
dialer-list 1 protocol ip permit
route-map nonat permit 10
match ip address 130
!
!
line con 0
exec-timeout 120 0
no modem enable
stopbits 1
line aux 0
line vty 0 4
exec-timeout 120 0
login local
length 0
!
scheduler max-task-time 5000
!
end

 

[JOAMON]

Couple of things I noticed but may not be problem.
Take a look at PIX crypto map mymap 5
has 2 entries for tansform-set
now look at SOHO
Working router has:
ip nat pool branch 65.13.194.214 65.13.194.214 netmask 255.255.255.240
non-working router has:
ip nat pool branch 67.98.252.16 67.98.252.16 netmask 255.255.255.2
Was last part of subnet mask cutoff in post?

 

[rodneyws]

Apparently I lack the skills necessary to copy/paste successfully. There was no duplicate entry on the PIX and the subnet was correct (255.255.255.252) on the remote router. Not sure how those got in my post like that.

 

[JOAMON]

Just noticed something else...
In looking at your nat pool...do you have a 16 block at each sight???
If so and
ip nat pool branch 67.98.252.16 67.98.252.16 netmask 255.255.255.2
is supposed to be
ip nat pool branch 67.98.252.16 67.98.252.16 netmask 255.255.255.240
then address 67.98.252.16 is not useable as it is the network address.
67.98.252.16 255.255.255.240 Subnet Address
67.98.252.17 255.255.255.240
67.98.252.18 255.255.255.240
67.98.252.19 255.255.255.240
67.98.252.20 255.255.255.240
67.98.252.21 255.255.255.240
67.98.252.22 255.255.255.240
67.98.252.23 255.255.255.240
67.98.252.24 255.255.255.240
67.98.252.25 255.255.255.240
67.98.252.26 255.255.255.240
67.98.252.27 255.255.255.240
67.98.252.28 255.255.255.240
67.98.252.29 255.255.255.240
67.98.252.30 255.255.255.240
67.98.252.31 255.255.255.240 Broadcast Address

 

[rodneyws]

Each remote site has a different sized block of IPs assigned to it mainly because each site is utilizing a different ISP (standardization wasn't an option in this rural area). In the case of 67.98.252.16, that subnet mask is 255.255.255.252 so wouldn't that make 67.98.252.19 our broadcast address?

 

[rodneyws]

Just a reminder... the remote site has full access out to the internet... and anyone on the internet can telnet to the external IP of the Cisco SOHO 91s at the remote sites. I've doublechecked the crypto keys on both ends so I'm fairly convinced something is not right on the PIX.

 

[JOAMON]

You post only listed subnet as 255.255.255.2
If it is 255.255.255.252 then you ip information is as follows:
IP Mask Notes ...
67.98.252.16 255.255.255.252 Subnet Address
67.98.252.17 255.255.255.252
67.98.252.18 255.255.255.252
67.98.252.19 255.255.255.252 Broadcast Address
.16 would be the subnet with .19 as broadcast.

 

[rodneyws]

The subnet mask on the router is correctly configured as 255.255.255.252 as assigned by our ISP. I'm not sure how that copied/pasted incorrectly.

 

[JOAMON]

Okay...then wouldn't you want your natpool address to be one of the useable addresses like 67.98.252.18 ?

 

[rodneyws]

I'm not sure.

The natpool address for the first three sites is in fact the external IP of the SOHO 91 at each respective site... so why would we want to change that now?

 

[JOAMON]

Well...generally ip blocks work like:
Example:
67.98.252.18/30
This gives you a block of 4 with 2 useable addresses.
The first address also called the subnet address and the 4th address called the broadcast are not useable addresses.
67.98.252.16 255.255.255.252 Subnet Address
67.98.252.17 255.255.255.252 1st useable
67.98.252.18 255.255.255.252 2nd useable
67.98.252.19 255.255.255.252 Broadcast Address

If you were to apply this to one of the other networks with the natpool of 65.13.194.214 with the same mask you will notice that the natpool address is using the 2nd ip routeable address:
65.13.194.212 255.255.255.252 Subnet Address
65.13.194.213 255.255.255.252 1st useable
65.13.194.214 255.255.255.252 2nd useable
65.13.194.215 255.255.255.252 Broadcast Address
I know that this one is setup using 255.255.255.240 mask but is also uses an ip routeable and not the subnet address.
65.13.194.208 255.255.255.240 Subnet Address
65.13.194.209 255.255.255.240
65.13.194.210 255.255.255.240
65.13.194.211 255.255.255.240
65.13.194.212 255.255.255.240
65.13.194.213 255.255.255.240
65.13.194.214 255.255.255.240
65.13.194.215 255.255.255.240
65.13.194.216 255.255.255.240
65.13.194.217 255.255.255.240
65.13.194.218 255.255.255.240
65.13.194.219 255.255.255.240
65.13.194.220 255.255.255.240
65.13.194.221 255.255.255.240
65.13.194.222 255.255.255.240
65.13.194.223 255.255.255.240 Broadcast Address
If I am correct in this then that could be why everything works except the VPN. Double check the outside IP address and you will find that it is probably not .16 and is either .17 or .18

 

[rodneyws]

So you're sugguesting I contact our ISP and confirm the external IP address? If it is in fact incorrect, why am I able to telnet to 67.98.252.16 and access the SOHO91? (I understand that telnet is not encrypted and I certainly do not make it a habit of doing this from the external side)

 

[rodneyws]

I totally see what you're saying now. Sorry!!! My day has been crazy and I just re-read your previous post... I will definitely be looking into that.

 

[JOAMON]

Well I guess it really depends on how they are subnetting. If they are using CIDR then what I said is true. However some ISP will take a large IP block and give out IP addresses with all of the address all part on one large subnet. If this is the case then you are good. It would be interesting to see the results from a show cdp neighbors.
One other thing that is strange is that none of the addresses in ip ranges:
67.98.252.0 thru 254
209.164.236.0 thru 254
respond to pings.

 

[JOAMON]

Also.....did you notice the double entry in crypto map 5 for the transform-set
crypto map mymap 5 ipsec-isakmp
crypto map mymap 5 match address ipsec5
crypto map mymap 5 set peer 67.98.252.16
crypto map mymap 5 set transform-set network
crypto map mymap 5 set transform-set network
Is this just a glitch from the copy and paste?

 

[JOAMON]

As for the other network.....pix crypto map reads
crypto map mymap 3 ipsec-isakmp
crypto map mymap 3 match address nonat
crypto map mymap 3 set peer 209.164.236.244
crypto map mymap 3 set transform-set network

Should it not be
crypto map mymap 3 ipsec-isakmp
crypto map mymap 3 match address ipsec3
crypto map mymap 3 set peer 209.164.236.244
crypto map mymap 3 set transform-set network

match address does not look right...

 

[JOAMON]

Have any success???

 

[rodneyws]

I should have some free time to test it this afternoon. Like most places, it's a crazy work environment here.