Help - "No buffer" and "overruns" on PIX-515E

[JQ95]

All,

I could really use your collective help on this.

I have a PIX515E with DMZ port. Server in the DMZ are a MS-VPN server - ACL is opened up to allow GRE and PPTP from the outside. The config is also set up in anticipation of them moving web and email servers back in-house, but they are not online right now. Firewall also does NAT/PAT for users to surf the internet. There is no IPSec or anything being done on the PIX.

Inside network has 30 PCs or so, DMZ has two servers, and there are 2-3 remote users that VPN into the MS server. Pretty low key low utilization situation.

Install was about 6 weeks ago, and all has been quiet and fine since then. This AM we started having intermittent problems accessing the internet. I did a "sh int" on the PIX and got the results below. (IP addresses have been altered to protect the innocent) Configuration is also shown below (IP addresses removed to protect the innocent).

What concerns me is the large number of "no buffer" and "overruns". These counters were only a few minutes old when I took this - we had just rebooted the PIX. I am not sure why I am seeing so many broadcasts on the inside interface in this short amount of time, either.

My research has led me to believe this could be from two things:
1. Excessive load on PIX. I find this hard to believe based on the low usage situation they are in. Solution is to upgrade PIX or memory.
2. Rule sets/ACLs applied on the router are so complex that it is causing CPU/memory burden. Same solution was recommended.

Call me crazy, but this just makes no sense. The 515 should be plenty for this environment, and my ACL is VERY straightforward (I think, anyway). I am going to dive into the broadcast issue, but nothing has changed on the network so I dont know why it would have such an impact right now. Should I be looking elsewhere?

Somewhat immaterial (considering the current problems) I also cannot ping anything on the internet from the inside network.

Thanks for your help, I appreciate it!

-----------------------
Output from "sh int"
-----------------------
Pix# sh int
interface ethernet0 "outside" is up, line protocol is up
Hardware is i82559 ethernet, address is 0009.b7e0.bab1
IP address w.x.y.120, subnet mask 255.255.255.240
MTU 1500 bytes, BW 100000 Kbit full duplex
53554 packets input, 12243327 bytes, 0 no buffer
Received 10 broadcasts, 0 runts, 0 giants
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
1071645 packets output, 83373087 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collisions, 0 deferred
0 lost carrier, 0 no carrier
input queue (curr/max blocks): hardware (128/128) software (0/82)
output queue (curr/max blocks): hardware (0/128) software (0/676)
interface ethernet1 "inside" is up, line protocol is up
Hardware is i82559 ethernet, address is 0009.b7e0.bab2
IP address 10.1.1.2, subnet mask 255.255.0.0
MTU 1500 bytes, BW 100000 Kbit full duplex
936116 packets input, 72338591 bytes, 20949 no buffer
Received 13489 broadcasts, 0 runts, 0 giants
1080 input errors, 0 CRC, 0 frame, 1080 overrun, 0 ignored, 0 abort
12041 packets output, 3024259 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collisions, 0 deferred
0 lost carrier, 0 no carrier
input queue (curr/max blocks): hardware (128/128) software (0/281)
output queue (curr/max blocks): hardware (0/40) software (0/17)
interface ethernet2 "DMZ" is up, line protocol is up
Hardware is i82559 ethernet, address is 0002.b3a4.4f08
IP address 172.16.1.1, subnet mask 255.255.255.0
MTU 1500 bytes, BW 100000 Kbit full duplex
1963542 packets input, 155229140 bytes, 1075 no buffer
Received 69 broadcasts, 0 runts, 0 giants
153 input errors, 0 CRC, 0 frame, 153 overrun, 0 ignored, 0 abort
38188 packets output, 8506295 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collisions, 0 deferred
0 lost carrier, 0 no carrier
input queue (curr/max blocks): hardware (128/128) software (0/260)
output queue (curr/max blocks): hardware (0/78) software (0/6)

------------------------
Configuration
------------------------
PIX Version 6.1(3)
nameif ethernet0 outside security0
nameif ethernet1 inside security100
nameif ethernet2 DMZ security10
enable password (password) encrypted
passwd (password) encrypted
hostname Pix
domain-name domain.com
fixup protocol ftp 21
fixup protocol http 80
fixup protocol h323 1720
fixup protocol rsh 514
fixup protocol rtsp 554
fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol sip 5060
fixup protocol skinny 2000
names
access-list PING_ANY permit ip any any
access-list OUTSIDE_ACL permit tcp any host 172.16.1.115 eq 1723
access-list OUTSIDE_ACL permit tcp any host w.x.y.115 eq 1723
access-list OUTSIDE_ACL permit gre host 172.16.1.115 any
access-list OUTSIDE_ACL permit gre host w.x.y.115 any
access-list OUTSIDE_ACL permit gre any host 172.16.1.115
access-list OUTSIDE_ACL permit gre any host w.x.y.115
pager lines 24
interface ethernet0 auto
interface ethernet1 auto
interface ethernet2 auto
icmp permit any echo outside
icmp permit any echo DMZ
mtu outside 1500
mtu inside 1500
mtu DMZ 1500
ip address outside w.x.y.120 255.255.255.240
ip address inside 10.1.1.2 255.255.0.0
ip address DMZ 172.16.1.1 255.255.255.0
ip audit info action alarm
ip audit attack action alarm
pdm history enable
arp timeout 14400
global (outside) 1 w.x.y.126 netmask 255.255.255.240
nat (inside) 1 10.1.1.0 255.255.255.0 0 0
static (DMZ,outside) w.x.y.115 172.16.1.115 netmask 255.255.255.255 0 0
static (DMZ,outside) w.x.y.118 172.16.1.118 netmask 255.255.255.255 0 0
static (DMZ,outside) w.x.y.119 172.16.1.119 netmask 255.255.255.255 0 0
access-group OUTSIDE_ACL in interface outside
route outside 0.0.0.0 0.0.0.0 w.x.y.113 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h323
0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable
no sysopt route dnat
telnet timeout 5
ssh timeout 5
terminal width 80

 

[yizhar]

HI.

I would focus on the byte counts:

interface ethernet0 "outside" is up, line protocol is up
53554 packets input, 12,243,327 bytes, 0 no buffer
1071645 packets output, 83,373,087 bytes, 0 underruns

interface ethernet1 "inside" is up, line protocol is up
936116 packets input, 72,338,591 bytes, 20949 no buffer
12041 packets output, 3,024,259 bytes, 0 underruns

interface ethernet2 "DMZ" is up, line protocol is up
1963542 packets input, 155,229,140 bytes, 1075 no buffer
38188 packets output, 8,506,295 bytes, 0 underruns

What are the type and speed of your connection to the Internet?

As you mentioined, you have just rebooted the pix so that traffic seems above normal for your organization.

Now you need to check what is generating the traffic.
Some first suspects are:
mail servers (open relay?, mail loop?, virus?)
peer to peer file sharing like Kazza.

Since the DMZ is involved, check the servers first.

You can issue the following command at the pix:
show conn
To monitor active connections.
Check the ip addresses, protocol+port, and byte count for each connection.

You can also use PDM to see realtime graph of interface bit/byte count.

You can also use syslog messages at level 6.
A log of few minutes will probably give you the key for the mystery.

***

About your config, you have some lines that are not needed and should be removed.
The following lines can be removed:
> access-list OUTSIDE_ACL permit tcp any host 172.16.1.115 eq 1723
> access-list OUTSIDE_ACL permit gre host 172.16.1.115 any
> access-list OUTSIDE_ACL permit gre host w.x.y.115 any
> access-list OUTSIDE_ACL permit gre any host 172.16.1.115

The following lines should remain:
> access-list OUTSIDE_ACL permit tcp any host w.x.y.115 eq 1723
> access-list OUTSIDE_ACL permit gre any host w.x.y.115

You can verify that these are unneeded by using:
show access-list
and checking the hit counters for each line.

Bye
Yizhar Hurwitz

http://come.to/yizhar

http://teachers.sivan.co.il/yizhar

 

[baddos]

13489 broadcasts in a minute or so is way too much. Not to even begin w/ the 73mb already being sent through the inside interface. It looks like something is going crazy on your inside network. Check to see if you have a loop in your switches causing a broadcast storm.

-Bad Dos

 

[dopehead]

Also the "show connection Count" command is useful, instead of displaying all the connections (will be alot if you are taking part in a dos attack or something) it will display how many current connections are in table.