9608 VPN issue - Invalid Configuration

[uglog]

I have configured our 9608 phone using the below configuration
It appears to exchange keys and fails after 15 secs.

The error is VPN Tunnel Failure - Invalid Configuration

Does anyone know what this generic error actually means?

Is there a way I can access the log files of the phone to see the detailed error?

Any help would be appreciated





################################################## #
## VPN Mode
## 0: Disabled, 1: Enabled.
################################################## #

SET NVVPNMODE 1

################################################## #
## Vendor.
## 1: Juniper/Netscreen, 2. Cisco
## 3: CheckPoint/ Nokia 4: Other
## 5: Nortel.
################################################## #
SET NVVPNSVENDOR 1

################################################## #
## GATEWAY
################################################## #
SET NVSGIP **PUBLIC IP OF REMOTE ENDPOINT**

################################################## #
## Encapsulation Type.
## 0: 4500-4500, 1: Disabled
## 2: 2070-500, 3: ?
## 4: RFC (500-500)
################################################## #
SET NVVPNENCAPS 0

################################################## #
## Copy TOS.
## 1: Yes, 2: No
################################################## #
SET NVVPNCOPYTOS 2

################################################## #
## Authentication Type.
##
## [For Cisco/Juniper/Checkpoint/Other]
## 3: PSK, 4: PSK with Xauth
## 5: RSA signatures with Xauth, 6: Hybrid Xauth
## 7: RSA signatures.
##
## [Nortel Authentication Type]
## 1: Local credentials, 2: Radius Credentials.
## 3: Radius SecureID, 4: Radius Axent.
################################################## #
SET NVVPNAUTHTYPE 3

################################################## #
## Preshared KEY PSK
##################################################
SET NVIKEPSK "PSKKEY"

################################################## #
## VPN User Type.
## 1: Any, 2: User
################################################## #
SET NVVPNUSERTYPE 1

################################################## #
## VPN User name.
################################################## #
SET NVVPNUSER mscep1

################################################## #
## Password Type.
## 1: Save in Flash, 2: Erase on reset
## 3: Numeric OTP, 4: Alpha-Numeric OTP
## 5: Erase on VPN termination.
################################################## #
SET NVVPNPSWDTYPE 1

################################################## #
## User Password.
################################################## #
SET NVVPNPSWD mscep1

################################################## #
## IKE ID (Group Name).
################################################## #
SET NVIKEID mscep

################################################## #
## IKE ID Type.
## 1: IPv4_ADDR, 2: FQDN
## 3: USER_FQDN, 9: DER_ASN1_DN
## 11: Key ID
################################################## #
SET NVIKEIDTYPE 1

################################################## #
## IKE Xchg Mode.
## 1: Aggressive, 2: Identity Protect.
################################################## #
SET NVIKEXCHGMODE 2

################################################## #
## IKE DH Group.
################################################## #
SET NVIKEDHGRP 14

################################################## #
## IKE Encryption Algo.
## 1: AES-128, 2: 3DES
## 3: DEs 4: AEs-192
## 5: AES-256 0: Any
################################################## #
SET NVIKEP1ENCALG 5

################################################## #
## IKE Auth algo.
## 0: Any, 1: MD5
## 2: sHA-1
################################################## #
SET NVIKEP1AUTHALG 1

################################################## #
## IKE Config Mode.
## 0: Enabled, 1: Disabled.
################################################## #
SET NVIKECONFIGMODE 0

################################################## #
## IPsec PFS DH group.
################################################## #
#SET NVPFSDHGRP 14

################################################## #
## IPsec Encryption Algo.
## 1: AES-128, 2: 3DES
## 3: DEs 4: AEs-192
## 5: AES-256 6: None
## 0: Any
################################################## #
SET NVIKEP2ENCALG 5

################################################## #
## IPsec Authentication Algo.
## 0: Any, 1: MD5
## 2: sHA-1
################################################## #
SET NVIKEP2AUTHALG 1

################################################## #
## Specifies the IKE SA lifetime in seconds
################################################## #
SET NVIKEP1LIFESEC 86400
SET NVIKEP2LIFESEC 86400

################################################## #
## Protected Network.
################################################## #

#SET NVIPSECSUBNET 0.0.0.0/0, 0.0.0.0/0
################################################## #
## IKE Over TCP.
## 0: Never, 1: Auto
## 2: Always
################################################## #
SET NVIKEOVERTCP 1

################################################## #
## Craft access
## 0: Enabled, 1: only view option is available?
################################################## #

SET PROCSTAT 0
################################################## #
## VPN craft access
## 0: disabled, 1: view only
## 2: View and edit.
################################################## #

SET VPNPROC 2
################################################## #
## Call Server address
################################################## #

SET MCIPADD 192.168.0.4

################################################## #
## craft access code
################################################## #

SET PROCPSWD 27238

################################################## #
## VPN craft access code
################################################## #

# END

 

[derfloh]

What do you expect from us? You have to compare the phone config with the IPSec config of your firewall, read some logs and check where the error is.

 

[uglog]

Is there a way I can access the log files of the phone to see the detailed error?

 

[derfloh]

You have to check the log files of the firewall.

 

[NYSTECH]

Derfloh is correct. look at your firewall logs

 

[uglog]

Firewall is reporting that the PSK is not matching (by the sounds of below).

We have triple check both ends , and Pre-shared key is correct!

We haven't had this issues with the 10+ IPsec active VPNs we established.

Any ideas what could be different with the 9608?

"VPN-dynamic"[3] 195.XX.XX.XX #4824: sending notification PAYLOAD_MALFORMED to 195.XX.XX.XX:2070
"VPN-dynamic"[3] 195.XX.XX.XX #4824: next payload type of ISAKMP Identification Payload has an unknown value: 228
"VPN-dynamic"[3] 195.XX.XX.XX #4824: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet
| payload malformed after IV
"VPN-dynamic"[3] 195.XX.XX.XX #4824: sending notification PAYLOAD_MALFORMED to 195.XX.XX.XX:2070
"VPN-dynamic"[3] 195.XX.XX.XX #4824: ignoring informational payload, type INVALID_COOKIE msgid=00000000
"VPN-dynamic"[3] 195.XX.XX.XX #4824: received and ignored informational message
"VPN-dynamic"[3] 195.XX.XX.XX #4824: max number of retransmissions (2) reached STATE_MAIN_R2
"VPN-dynamic"[3] 195.XX.XX.XX: deleting connection "VPN-dynamic" instance with peer 195.XX.XX.XX {isakmp=#0/ipsec=#0}

 

[gwebster]

There have been issues in the past with the VPN software on the 9608 which has been resolved in later releases. What version are you running?

 

[uglog]

Currently running version 9.1.9

 

[Pepp77]

Are you set to PSK but then using a userlogin, wouldn't this mean you need PSK with XAuth?

| ACSS SME |

 

[uglog]

I found this VPN config template from another tread, and just adapted it for our environment.

Yes, we have configured the firewall for PSK and also the phone, and I just left the Xauth login in the config file, as assumed the phone would ignore as the PSK parameter is invoked "SET NVVPNAUTHTYPE 3"

Going to hash it out and try again...will advise shortly. thanks

 

[uglog]

Done the following ;

Hashed out the Xauth username (### SET NVVPNUSER mscep1) and password (### SET NVVPNPSWD mscep1) parameters from 96xxvpn.txt, clearing the phone, re-adding group (876), connecting phone to LAN to grab updated VPN config file (96xxvpn.txt).

Then connected it back to the router connected to outside line (separate to firewall), and still getting the same errors in the firewall

"VPN-dynamic"[3] 195.XX.XX.XX #4824: sending notification PAYLOAD_MALFORMED to 195.XX.XX.XX:2070
"VPN-dynamic"[3] 195.XX.XX.XX #4824: next payload type of ISAKMP Identification Payload has an unknown value: 228
"VPN-dynamic"[3] 195.XX.XX.XX #4824: probable authentication failure (mismatch of preshared secrets?): malformed payload in packet

 

[uglog]

Upgrading the firmware to 9.1.10 appears to have corrected that issue with Phase 1 negotiations , no getting issues with Phase 2.

Anyone have any ideas?

sending encrypted notification UNSUPPORTED_EXCHANGE_TYPE to 195.XX.XX.XX:4500
received MODECFG message when in state STATE_MAIN_R3, and we aren't xauth client

 

[uglog]

Hi,
Also does anyone know if this phone supports L2TP?

Thanks

 

[janni78]

Which brand and model of firewall are you using?

"Trying is the first step to failure..." - Homer

 

[uglog]

It's a all-in-one firewall , vpn, anti-virus managed proprietary linux appliance. It isn't branded per-se

 

[janni78]

Sure you're using MD5 as Auth encryption? Probably it's using SHA1.

"Trying is the first step to failure..." - Homer

 

[uglog]

Hi,

Yes, I have checked on the actual phone, the Auth encryption is MD5 for both IKE and And IPsec, see below

################################################## #
## IKE Auth algo.
## 0: Any, 1: MD5
## 2: sHA-1
################################################## #
SET NVIKEP1AUTHALG 1

################################################## #
## IPsec Authentication Algo.
## 0: Any, 1: MD5
## 2: sHA-1
################################################## #
SET NVIKEP2AUTHALG 1

This post suggests that the error is related to IP addresses...

https://supportforums.cisco.com/discussion/12468506/rv130w-shrewsoft-vpn-client-not-working

"The problem is that if your remote network is using the same subnet as the local network then the router is not capable of knowing when it is supposed to send a request to the LAN or the Remote end"

The Protected Network subnet is different on the phone to firewall. I have also tried changing the protected network to 0.0.0.0/24 same error, which leads me to believe not ip related ...I have hit a brick wall

 

[janni78]

Yeah, but are you using MD5 on the VPN router, this wouldn't be recommended, you should use at least SHA-1 for security reasons.

"Trying is the first step to failure..." - Homer

 

[uglog]

Hi,

Changing to SHA-1 hasn't made any difference.

 

[janni78]

malformed payload in packet" usually means that the PSK is incorrect.
You need to make sure that all the settings are correct on both ends.

As you have a "no-name" VPN concentrator there is no Application Notes on how to get this working and it's hard to verify that you've done all the steps correctly.

"Trying is the first step to failure..." - Homer