pix501 to asa5505 tunnel works but no traffic 1

[jweppie]

Hi guru's

Been puzzling for a few days now, time to call in some support.

I've got the following network layout and want full traffic between internal LANS

192.168.50.0/24-pix501-xxx.xxx.9.85/20(dhcp) <- internet -> xxx.xxx.195.75-asa5505-10.32.10/24

my pix config :
#######################################################
<snip>
access-list inside_outbound_nat0_acl permit ip 192.168.50.0 255.255.255.0 10.32.1.0 255.255.255.0
access-list outside_cryptomap_20 permit ip 192.168.50.0 255.255.255.0 10.32.1.0 255.255.255.0
access-list acl_outbound permit ip 192.168.50.0 255.255.255.0 any
<snip>
ip address outside dhcp setroute
ip address inside 192.168.50.1 255.255.255.0
<snip>
global (outside) 1 interface
global (inside) 10 interface
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto map outside_map 20 ipsec-isakmp
crypto map outside_map 20 match address outside_cryptomap_20
crypto map outside_map 20 set pfs group2
crypto map outside_map 20 set peer xxx.xxx.195.75
crypto map outside_map 20 set transform-set ESP-3DES-SHA
crypto map outside_map interface outside
isakmp enable outside
isakmp key ******** address xxx.xxx.195.75 netmask 255.255.255.255 no-xauth no-config-mode
isakmp identity address
isakmp log 50
isakmp policy 10 authentication pre-share
isakmp policy 10 encryption 3des
isakmp policy 10 hash sha
isakmp policy 10 group 2
isakmp policy 10 lifetime 86400

sh route
outside 0.0.0.0 0.0.0.0 xxx.xxx.8.1 1 DHCP static
outside xxx.xxx.8.0 255.255.248.0 xxx.xxx.9.85 1 CONNECT static
inside 192.168.50.0 255.255.255.0 192.168.50.1 1 CONNECT static

sh nat
nat (inside) 0 access-list inside_outbound_nat0_acl
nat (inside) 1 192.168.50.0 255.255.255.0 0 0
##########################################################

my ASA config:
##########################################################
name 10.32.1.0 live_lan description live lan
name 192.168.50.0 jaap_lan description LAN Jaap thuis
name xxx.xxx.9.85 pix_jaap description pix Jaap thuis
!
interface Vlan2
nameif outside
security-level 0
ip address xxx.xxx.195.75 255.255.255.192
ospf cost 10
!
interface Vlan3
nameif live
security-level 0
ip address 10.32.1.75 255.255.255.0
ospf cost 10

interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
switchport access vlan 3
<snip>
same-security-traffic permit intra-interface
access-list outside_20_cryptomap extended permit ip live_lan 255.255.255.0 jaap_lan 255.255.255.0
access-list live_nat0_outbound extended permit ip live_lan 255.255.255.0 jaap_lan 255.255.255.0
<snip>
global (outside) 1 interface
nat (live) 0 access-list live_nat0_outbound
route outside 0.0.0.0 0.0.0.0 xxx.xxx.195.75 1
group-policy DfltGrpPolicy attributes
banner none
wins-server none
dns-server none
dhcp-network-scope none
vpn-access-hours none
vpn-simultaneous-logins 3
vpn-idle-timeout 30
vpn-session-timeout none
vpn-filter none
vpn-tunnel-protocol IPSec
password-storage disable
ip-comp disable
re-xauth enable
group-lock none
pfs disable
ipsec-udp disable
ipsec-udp-port 10000
split-tunnel-policy tunnelall
split-tunnel-network-list none
default-domain none
split-dns none
intercept-dhcp 255.255.255.255 disable
secure-unit-authentication disable
user-authentication disable
user-authentication-idle-timeout 30
ip-phone-bypass disable
leap-bypass disable
nem disable
backup-servers keep-client-config
msie-proxy server none
msie-proxy method no-modify
msie-proxy except-list none
msie-proxy local-bypass disable
nac disable
nac-sq-period 300
nac-reval-period 36000
nac-default-acl none
address-pools none
client-firewall none
client-access-rule none
webvpn
functions none
html-content-filter none
homepage none
keep-alive-ignore 4
http-comp gzip
filter none
url-list none
customization value DfltCustomization
port-forward none
port-forward-name value Application Access
sso-server none
deny-message value Login was successful, but because certain criteria have not been met or due to some specific group policy, you do not have permission to use any of the VPN features. Contact your IT administrator for more information
svc none
svc keep-installer installed
svc keepalive none
svc rekey time none
svc rekey method none
svc dpd-interval client none
svc dpd-interval gateway none
svc compression deflate
group-policy GroupPolicy1 internal
group-policy GroupPolicy1 attributes
vpn-access-hours none
vpn-simultaneous-logins 3
vpn-idle-timeout 30
vpn-session-timeout none
vpn-filter none
vpn-tunnel-protocol IPSec l2tp-ipsec
ip-comp disable
re-xauth enable
group-lock value xxx.xxx.9.85
pfs enable
split-tunnel-network-list none
secure-unit-authentication disable
user-authentication disable
user-authentication-idle-timeout none
ip-phone-bypass disable
leap-bypass disable
nem enable
address-pools none
client-access-rule none
vpn-nac-exempt none
aaa authentication ssh console LOCAL
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto map outside_map 20 match address outside_20_cryptomap
crypto map outside_map 20 set pfs
crypto map outside_map 20 set peer pix_jaap
crypto map outside_map 20 set transform-set ESP-3DES-SHA
crypto map outside_map 20 set nat-t-disable
crypto map outside_map interface outside
crypto isakmp identity address
crypto isakmp enable outside
crypto isakmp policy 10
authentication pre-share
encryption 3des
hash sha
group 2
lifetime 86400
tunnel-group xxx.xxx.9.85 type ipsec-l2l
tunnel-group xxx.xxx.9.85 ipsec-attributes
pre-shared-key *
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
!
service-policy global_policy global
prompt hostname context
###########################################################
sh route

Gateway of last resort is xxx.xxx.195.75 to network 0.0.0.0

C xxx.xxx.195.64 255.255.255.192 is directly connected, outside
C 127.1.0.0 255.255.0.0 is directly connected, _internal_loopback
C live_lan 255.255.255.0 is directly connected, live
S* 0.0.0.0 0.0.0.0 [1/0] via xxx.xxx.195.75, outside

sh nat
NAT policies on Interface live:
match ip live live_lan 255.255.255.0 outside jaap_lan 255.255.255.0
NAT exempt
translate_hits = 0, untranslate_hits = 6
match ip live live_lan 255.255.255.0 live jaap_lan 255.255.255.0
NAT exempt
translate_hits = 0, untranslate_hits = 0


So the tunnel comes up, but it seems like no traffic returns from the ASA to the pix.

(RX but no TX, decrypt but no encrypt)

This is a test setup so I can change stuff around freely.
help is appreciated

 

[unclerico]

Can you post the output from sh crypto isakmp sa and sh crypto ipsec sa?? Also, have you run any debugs yet?? You should see a route in each routing table for the remote subnet of each peer, but I don't see them listed.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

thanks for your quick response :
I've set up the tunnel with a ping from local lan 192.168.1.x

on the pix :
###############
jaappix501# sh crypto isakmp sa
Total : 0
Embryonic : 0
dst src state pending created
jaappix501# sh crypto isakmp sa
Total : 1
Embryonic : 1
dst src state pending created
xxx.xxx.195.75 xxx.xxx.9.85 MM_NO_STATE 0 0
jaappix501# sh crypto ipsec sa


interface: outside
Crypto map tag: outside_map, local addr. xxx.xxx.9.85

local ident (addr/mask/prot/port): (192.168.50.0/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (10.32.1.0/255.255.255.0/0/0)
current_peer: xxx.xxx.195.75:500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 24, #pkts encrypt: 24, #pkts digest 24
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0, #pkts decompress failed: 0
#send errors 6, #recv errors 0

local crypto endpt.: xxx.xxx.9.85, remote crypto endpt.: xxx.xxx.195.75
path mtu 1500, ipsec overhead 56, media mtu 1500
current outbound spi: 88bfec2e

inbound esp sas:
spi: 0xe50f52aa(3842986666)
transform: esp-3des esp-sha-hmac ,
in use settings ={Tunnel, }
slot: 0, conn id: 2, crypto map: outside_map
sa timing: remaining key lifetime (k/sec): (4608000/28765)
IV size: 8 bytes
replay detection support: Y


inbound ah sas:


inbound pcp sas:


outbound esp sas:
spi: 0x88bfec2e(2294279214)
transform: esp-3des esp-sha-hmac ,
in use settings ={Tunnel, }
slot: 0, conn id: 1, crypto map: outside_map
sa timing: remaining key lifetime (k/sec): (4607999/28764)
IV size: 8 bytes
replay detection support: Y


outbound ah sas:


outbound pcp sas:
#########################################

On the ASA:

#########################################

flw04# sh crypto isakmp sa

Active SA: 1
Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 1

1 IKE Peer: pix_jaap
Type : L2L Role : responder
Rekey : no State : MM_ACTIVE

flw04# sh crypto ipsec sa
interface: outside
Crypto map tag: outside_map, seq num: 20, local addr: xxx.xxx.195.75

access-list outside_20_cryptomap permit ip live_lan 255.255.255.0 jaap_lan 255.255.255.0
local ident (addr/mask/prot/port): (live_lan/255.255.255.0/0/0)
remote ident (addr/mask/prot/port): (jaap_lan/255.255.255.0/0/0)
current_peer: pix_jaap

#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 2, #pkts decrypt: 2, #pkts verify: 2
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#send errors: 0, #recv errors: 0

local crypto endpt.: xxx.xxx.195.75, remote crypto endpt.: pix_jaap

path mtu 1500, ipsec overhead 58, media mtu 1500
current outbound spi: E50F52AA

inbound esp sas:
spi: 0x88BFEC2E (2294279214)
transform: esp-3des esp-sha-hmac none
in use settings ={L2L, Tunnel, PFS Group 2, }
slot: 0, conn_id: 11, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4274999/28555)
IV size: 8 bytes
replay detection support: Y
outbound esp sas:
spi: 0xE50F52AA (3842986666)
transform: esp-3des esp-sha-hmac none
in use settings ={L2L, Tunnel, PFS Group 2, }
slot: 0, conn_id: 11, crypto-map: outside_map
sa timing: remaining key lifetime (kB/sec): (4275000/28555)
IV size: 8 bytes
replay detection support: Y

##################################################

I thought about either routing or NATting that's going wrong, but I'm not sure were to start (and I've tried a lot )

 

[unclerico]

MM_NO_STATE

You have main mode issues. Run debug crypto isakmp on the PIX.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

jaappix501# sh crypto isakmp sa
Total : 1
Embryonic : 0
dst src state pending created
193.173.195.75 94.211.9.85 QM_IDLE 0 1

I think I did the previous sh crypto isakmp sa before the tunnel was fully up, tried it again...

Here's the debugging stuff from isakmp on the pix

ISAKMP (0): beginning Main Mode exchange
crypto_isakmp_process_block:src:xxx.xxx.195.75, dest:xxx.xxx.9.85 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing SA payload. message ID = 0

ISAKMP (0): Checking ISAKMP transform 1 against priority 10 policy
ISAKMP: encryption 3DES-CBC
ISAKMP: hash SHA
ISAKMP: default group 2
ISAKMP: auth pre-share
ISAKMP: life type in seconds
ISAKMP: life duration (VPI) of 0x0 0x1 0x51 0x80
ISAKMP (0): atts are acceptable. Next payload is 0
ISAKMP (0): processing vendor id payload

ISAKMP (0): SA is doing pre-shared key authentication using id type ID_IPV4_ADDR
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:xxx.xxx.195.75, dest:xxx.xxx.9.85 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing KE payload. message ID = 0

ISAKMP (0): processing NONCE payload. message ID = 0

ISAKMP (0): processing vendor id payload

ISAKMP (0): processing vendor id payload

ISAKMP (0): received xauth v6 vendor id

ISAKMP (0): processing vendor id payload

ISAKMP (0): speaking to another IOS box!

ISAKMP (0): processing vendor id payload

ISAKMP (0): speaking to a VPN3000 concentrator

ISAKMP (0): ID payload
next-payload : 8
type : 1
protocol : 17
port : 500
length : 8
ISAKMP (0): Total payload length: 12
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:xxx.xxx.195.75, dest:xxx.xxx.9.85 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing ID payload. message ID = 0
ISAKMP (0): processing HASH payload. message ID = 0
ISAKMP (0): processing vendor id payload

ISAKMP (0): remote peer supports dead peer detection

ISAKMP (0): SA has been authenticated

ISAKMP (0): beginning Quick Mode exchange, M-ID of 8064366:7b0d6e
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:xxx.xxx.195.75, dest:xxx.xxx.9.85 spt:500 dpt:500
OAK_QM exchange
oakley_process_quick_mode:
OAK_QM_IDLE
ISAKMP (0): processing SA payload. message ID = 8064366

ISAKMP : Checking IPSec proposal 1

ISAKMP: transform 1, ESP_3DES
ISAKMP: attributes in transform:
ISAKMP: SA life type in seconds
ISAKMP: SA life duration (basic) of 28800
ISAKMP: SA life type in kilobytes
ISAKMP: SA life duration (VPI) of 0x0 0x46 0x50 0x0
ISAKMP: encaps is 1
ISAKMP: authenticator is HMAC-SHA
ISAKMP: group is 2
ISAKMP (0): atts are acceptable.
ISAKMP (0): processing NONCE payload. message ID = 8064366

ISAKMP (0): processing KE payload. message ID = 8064366

ISAKMP (0): processing ID payload. message ID = 8064366
ISAKMP (0): processing ID payload. message ID = 8064366
ISAKMP (0): Creating IPSec SAs
inbound SA from xxx.xxx.195.75 to xxx.xxx.9.85 (proxy 10.32.1.0 to 192.168.50.0)
has spi 3272574530 and conn_id 2 and flags 25
lifetime of 28800 seconds
lifetime of 4608000 kilobytes
outbound SA from xxx.xxx.9.85 to xxx.xxx.195.75 (proxy 192.168.50.0 to 10.32.1.0)
has spi 1074578607 and conn_id 1 and flags 25
lifetime of 28800 seconds
lifetime of 4608000 kilobytes
VPN Peer: IPSEC: Peer Info not found during IPSEC addition: Peer ip:xxx.xxx.195.75/500

VPN Peer: IPSEC: Peer Info not found during IPSEC addition: Peer ip:xxx.xxx.195.75/500

return status is IKMP_NO_ERROR
VPN Peer: ISAKMP: Added new peer: ip:xxx.xxx.195.75/500 Total VPN Peers:1
#############################
and debug crypto ipsec sa

jaappix501(config)# IPSEC(key_engine): got a queue event...
IPSEC(spi_response): getting spi 0xfb0a8920(4211771680) for SA
from xxx.xxx.195.75 to xxx.xxx.9.85 for prot 3
IPSEC(validate_proposal_request): proposal part #1,
(key eng. msg.) dest= xxx.xxx.195.75, src= xxx.xxx.9.85,
dest_proxy= 10.32.1.0/255.255.255.0/0/0 (type=4),
src_proxy= 192.168.50.0/255.255.255.0/0/0 (type=4),
protocol= ESP, transform= esp-3des esp-sha-hmac ,
lifedur= 0s and 0kb,
spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x24
IPSEC(key_engine): got a queue event...
IPSEC(initialize_sas): ,
(key eng. msg.) dest= xxx.xxx.9.85, src= xxx.xxx.195.75,
dest_proxy= 192.168.50.0/255.255.255.0/0/0 (type=4),
src_proxy= 10.32.1.0/255.255.255.0/0/0 (type=4),
protocol= ESP, transform= esp-3des esp-sha-hmac ,
lifedur= 28800s and 4608000kb,
spi= 0xfb0a8920(4211771680), conn_id= 1, keysize= 0, flags= 0x25
IPSEC(initialize_sas): ,
(key eng. msg.) src= xxx.xxx.9.85, dest= xxx.xxx.195.75,
src_proxy= 192.168.50.0/255.255.255.0/0/0 (type=4),
dest_proxy= 10.32.1.0/255.255.255.0/0/0 (type=4),
protocol= ESP, transform= esp-3des esp-sha-hmac ,
lifedur= 28800s and 4608000kb,
spi= 0xa9c80539(2848458041), conn_id= 2, keysize= 0, flags= 0x25


seems normal ?

 

[unclerico]

yeah, everything looks normal from what I can tell anyway. so now with the tunnel established what do you see in the routing table??

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

no routes have been added after the tunnel is up. (checked on both sides)

That's bad right ;-)

 

[unclerico]

lol, yep that's bad. Try adding this to each side:

asa(config)# crypto map outside_map 20 set reverse-route

that will enable reverse-route injection. if for some reason you still don't see the route in the routing table then we'll need to configure a static route.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

Any ideas on how to fix the routing ?

 

[unclerico]

did you see my previous post?? did the set reverse-route command not make the routes visible??

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

Sorry about last comment, missed yours there...

I added the reverse route on the ASA, and it shows as a static route (still no traffic though):

C xxx.xxx.195.64 255.255.255.192 is directly connected, outside
C 127.1.0.0 255.255.0.0 is directly connected, _internal_loopback
C live_lan 255.255.255.0 is directly connected, live
S jaap_lan 255.255.255.0 [1/0] via xxx.xxx.195.75, outside
S* 0.0.0.0 0.0.0.0 [1/0] via xxx.xxx.195.75, outside


On the pix this command is not recognized :

jaappix501(config)# crypto map outside_map 20 set reverse route
ERROR: unknown subcommand <reverse>
usage: crypto map <map-name> <seqno> set
{pfs|transform-set|session-key|security-association} ...


So I added the route as a static too with :
route outside 10.32.1.0 255.255.255.0 xxx.xxx.9.85

Still can't ping either way

 

[unclerico]

what version OS's are these running??

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

PIX Version 6.3(4)

ASA Version 7.2(2)

 

[jweppie]

Could the fact that the outside IP from the pix is provided by my ISP (via DHCP) have anything to do with my routing problems ?

I'm still thinking about the ASA that's doing something wrong with the packets, after all I can see my pings arriving on the ASA, but it's not returning them.

 

[unclerico]

1) have you tried to reboot the ASA??
2) DHCP could be an issue if your ISP provides you with a different address each time. Some ISP's will require you to get your IP via DHCP even though it is technically static. If your address is truly dynamic then you'll want to change a few things.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

The IP is more or less static. As long as the cablemodem doesn't get a cold restart. I think the lease is very long and I've been using this IP for a long time.

Rebooted both devices.

No success...

Booting the pix will show :
Allocated IP address = xx.xx.9.85 netmask 255.255.248.0 gateway = xxx.xxx.8.1

Outside interface address added to PAT pool
Inside interface address added to PAT pool

 

[unclerico]

what happens when you ping from a host behind the ASA towards the PIX??

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[jweppie]

I hadn't configured any hosts to use the ASA, so i picked a test server and added this route:


192.168.50.0 10.32.1.75 255.255.255.255 UGH 0 0 0 eth0

I can ping the ASA
PING 10.32.1.75 (10.32.1.75) 56(84) bytes of data
64 bytes from 10.32.1.75: icmp_seq=0 ttl=255 time=0.681 m

Now when I ping a host on my side :
PING 192.168.50.4 (192.168.50.4) 56(84) bytes of data.

--- 192.168.50.4 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3013ms


traceroute to 192.168.50.4 (192.168.50.4), 30 hops max, 38 byte packets
1 192.168.50.4 (192.168.50.4) 0.494 ms 0.229 ms 0.201 ms

Traceroute doesn't show me anything but does come back immediately ??

 

[jweppie]

Ah forget abuot that traceroute, I'm connected through a backdoor (cisco VPN client) to the testserver and it probably found me through the default gateway.

 

[jweppie]

A ping from my host (pc) 192.168.50.4 shows this on the asa logs (which means traffic gets through the tunnel unnatted):

6 Jun 09 2009 10:53:17 302021 192.168.50.4 10.32.1.75 Teardown ICMP connection for faddr 192.168.50.4/1024 gaddr 10.32.1.75/0 laddr 10.32.1.75/0


6 Jun 09 2009 10:53:15 302021 xxx.xxx.195.65 xxx.xxx.195.75 Teardown ICMP connection for faddr xxx.xxx.195.65/0 gaddr xxx.173.195.75/0 laddr xxx.173.195.75/0

6 Jun 09 2009 10:53:15 302021 192.168.50.4 xxx.xxx.195.75 Teardown ICMP connection for faddr 192.168.50.4/1024 gaddr xxx.xxx.195.75/0 laddr xxx.xxx.195.75/0

6 Jun 09 2009 10:53:15 302020 192.168.50.4 xxx.xxx.195.75 Built ICMP connection for faddr 192.168.50.4/1024 gaddr xxx.xxx.195.75/0 laddr xxx.xxx.195.75/0

6 Jun 09 2009 10:53:15 302020 xxx.xxx.195.65 xxx.xxx.195.75 Built ICMP connection for faddr xxx.xxx.195.65/0 gaddr xxx.xxx.195.75/0 laddr xxx.xxx.195.75/0

6 Jun 09 2009 10:53:15 302020 192.168.50.4 10.32.1.75 Built ICMP connection for faddr 192.168.50.4/1024 gaddr 10.32.1.75/0 laddr 10.32.1.75/0


For some reason the ASA does not know how to properly return the packets.

Also, I can see the ISP gateway in there