PIX firewall keeps losing internet connection 1

[mgratama]

My company uses three Pix firewalls at three office locations. The three firewalls talk to each other using a PPTP VPN tunnel. This works fine. However, one of the offices loses its connectin to the internet through HTTP, and only a reboot of the PIX brings it back. while this happens, the VPN tunnel between the offices is fine. I think I have been able to rule out DNS issues..and MCI has run several tests on their circuits and can't find anything wrong
any ideas?
Pix is a Cisco PIX 501.

 

[mgratama]

I should add that I am not at that location and that I have rebooting the firewall remotely with the "reload" command. Terminal Services works fine to their servers and of course drops upon the "reload" and then I can reconnect.
One thing I have not been able to try since I am remote is hooking up a laptop to the T-1 before the firewall to determine whether the firewall is the problem. Problem is very intermittent and thus difficult to troubleshoot.

 

[ChrisAC]

You could do to dump your logs to a syslog server and see if it's hitting a connection limit. If you have a VPN up fine when HTTP stops you could do to connect to the firewall and look at 'sh xlate' and 'sh conn' to see how many connections it can see. I presume that you have a limited licence (10 or 50 users).

Chris.

**********************
Chris A.C, CCNA, CCSA
**********************

 

[mgratama]

Thanks for your reply Chris. There are 50 concurrent licenses so I don't think that's the issue (small office, about 15 people).
Would a rebuild of the script or firmware upgrades do anything? I will now check the sh xlate and sh conn and see what I get.

 

[mgratama]

pixchamp# sh xlate
11 in use, 278 most used
Global 65.199.98.123 Local 192.168.16.1
PAT Global 65.199.98.125(1516) Local 192.168.16.210(3529)
PAT Global 65.199.98.125(1586) Local 192.168.16.210(3617)
PAT Global 65.199.98.125(3674) Local 192.168.16.225(3067)
PAT Global 65.199.98.125(3662) Local 192.168.16.211(2766)
PAT Global 65.199.98.125(1033) Local 192.168.16.2(1434)
PAT Global 65.199.98.125(1059) Local 192.168.16.214(2321)
PAT Global 65.199.98.125(1045) Local 192.168.16.214(2306)
PAT Global 65.199.98.125(3675) Local 192.168.16.214(2589)
PAT Global 65.199.98.125(3663) Local 192.168.16.211(2767)
PAT Global 65.199.98.125(3661) Local 192.168.16.211(2765)

pixchamp# sh conn
20 in use, 59 most used
UDP out 169.254.73.166:137 in 192.168.16.1:137 idle 0:01:02 flags -

TCP out 206.190.50.167:443 in 192.168.16.210:3617 idle 0:05:13 Bytes 2664 flags
UIO
TCP out 192.168.1.1:1026 in 192.168.16.2:3735 idle 0:06:05 Bytes 34113 flags UIO

TCP out 192.168.32.1:1988 in 192.168.16.1:1026 idle 0:00:35 Bytes 33633 flags UI
OB
TCP out 216.239.63.184:80 in 192.168.16.211:2767 idle 0:00:02 Bytes 2142 flags U
FRIO
TCP out 216.155.193.168:5050 in 192.168.16.210:3529 idle 0:00:04 Bytes 45554 fla
gs UIO
TCP out 192.168.1.1:4962 in 192.168.16.2:1026 idle 0:06:05 Bytes 31422 flags UIO
B
UDP out 192.168.32.1:137 in 192.168.16.1:137 idle 0:01:05 flags

TCP out 207.46.2.73:1863 in 192.168.16.214:2321 idle 0:00:06 Bytes 6286 flags UI
O
TCP out 192.168.1.1:4957 in 192.168.16.1:1026 idle 0:05:04 Bytes 130017 flags UI
OB
UDP out 192.168.1.66:161 in 192.168.16.228:1027 idle 0:02:00 flags -

TCP out 192.168.1.1:1026 in 192.168.16.1:4889 idle 0:03:41 Bytes 58096 flags UIO

UDP out 192.168.32.1:138 in 192.168.16.1:138 idle 0:01:20 flags

UDP out 192.168.1.66:161 in 192.168.16.237:1047 idle 0:00:12 flags -

TCP out 192.168.1.33:3817 in 192.168.16.1:3389 idle 0:00:00 Bytes 256118 flags U
IOB
TCP out 64.157.228.21:80 in 192.168.16.214:2306 idle 0:00:02 Bytes 725608 flags
UIO
TCP out 192.168.32.1:1025 in 192.168.16.1:4901 idle 0:01:47 Bytes 45287 flags UI
O
UDP out 192.168.32.202:123 in 192.168.16.1:123 idle 0:01:55 flags

TCP out 192.168.1.165:3153 in 192.168.16.1:445 idle 0:01:48 Bytes 7308 flags UIO
B
UDP out 192.168.1.1:1258 in 192.168.16.221:161 idle 0:00:04 flags

 

[mgratama]

Ok so it does not look like there is a user limit issue. Can excessive bandwidth use make HTTP fail? wouldn't the VPN tunnel fail as well? Could port 80 be getting blocked somehow? I guess I could check that with telnet. Looks like there is no service contract or warranty on the firewall, so I'm not sure about being able to obtain firmware upgrades. All three of them are using version 6.2(2)

 

[mgratama]

See troughput setting below.. should that be modified?

Hardware: PIX-501, 16 MB RAM, CPU Am5x86 133 MHz
Flash E28F640J3 @ 0x3000000, 8MB
BIOS Flash E28F640J3 @ 0xfffd8000, 128KB

0: ethernet0: address is 000b.5f49.2b6e, irq 9
1: ethernet1: address is 000b.5f49.2b6f, irq 10
Licensed Features:
Failover: Disabled
VPN-DES: Enabled
VPN-3DES: Enabled
Maximum Interfaces: 2
Cut-through Proxy: Enabled
Guards: Enabled
URL-filtering: Enabled
Inside Hosts: 50
Throughput: Limited
IKE peers: 5

 

[ChrisAC]

I think that it states "Limited" throughput as it's a limited licence. Do the logs show any problems when http doesn't work?

Also, 6.3(2) is very old and has known bugs. 6.3(5) is out now. You could do with a support contract to be able to get the upgrade. That might fix the problem.

Chris.

**********************
Chris A.C, CCNA, CCSA
**********************

 

[mgratama]

I will work on the support contract and do the upgrade although i doubt a firmware upgrade can be done remotely... probably would lose connectivity. Worst case I drive there and do it after hours. I will let you know how it goes. I appreciate the help.

 

[ChrisAC]

A firmware upgrade can be done quite easily remotely if you have ssh or telnet access. When you loose connectivity when you reboot you just hope and pray that it will come back up ;-)

Chris.

**********************
Chris A.C, CCNA, CCSA
**********************

 

[ermora]

Have you tried to "clear xlate" when you've lost connectivity?

 

[mgratama]

Hi Ermora,

Thanks for your post. Yes I have tried "clear xlate" and sometimes that has brought it back. Most of the time, however, clearing the routing table does not do the trick and a reboot does. What is interesting is that even when it fails, I am able to use the ping command from the config# prompt and reach websites like yahoo.com no problem. Yet the firewall is not allowing me to ping from inside. I took a trip there for the last 2 days and cleaned up viruses and spyware on the machine, thinking that perhaps excessive data packets were causing the firewall to hang... not sure that did the trick.. please continue to post any ideas and I will keep you posted.

 

[mgratama]

Symantec Antivirus version 10 corporate edition did catch a bunch of viruses and adware on some of the office PCs. I also ran Microsoft's antispyware beta1 which is a very nice application, and in some ways seems more effective than Adaware and/or spybot.
Still the internet dropped a couple of times, but not nearly as often. So perhaps it was a move in the right direction..

 

[ermora]

mgratama,
The reason I ask about "clear xlate" is because of the experience I had gone through. The experience was identical (internet access being blocked by PIX, however, everything else - even VPN connections - were being allowed).

Initially I was using the "reload" command and later figured out that "clear xlate" would correct the no-internet access, althought temporarily.

In the end, it was a worm. This worm was found on several Windows 2K Servers that were not up to the latest Service Pack. I had to manually remove the worm, clean the registry, and apply the latest Service Packs.

Once I did that, my problem ceased. But life's not so simple, I have a new problem - not related and appears to be an known issue.

Have you checked for worms or viruses? Have you tried using PAT instead of NAT?

 

[mgratama]

Thanks ermora. I will continue to look for worms, viruses and/or adware/spyware on the servers as mentioned in the previous posts.

Using PAT instead of NAT is something I will consider as well.
Thanks for you reply! This issue has been most frustrating. I was about to reconfigure the PIX from scratch just to be able to rule that out, but I really don't think that's the issue and having to setup new logins for those that VPN in from the outside only to find out that did not solve the problem is not appealing. I think it's something on the server, and I do see some DNS errors, DHCP errors in the event logs which I am investigating. When the internet goes down and VPN connections stay up, a reboot of the main server that runs DNS, DHCP and runs Active Directory for that office, does bring the internet back which makes me think it's not the firewall, but rather a problem on the server. Thanks again for the input and please keep em coming.