Ideas to debug3600 that becomes slower (long pings)?

[shayv]

Hi

I have cisco 3600 that becomes "heavy" from time to time, it appears as long pings.
Reset to the router solve it, but after a while it comes again.

any ideas?

 

[rburke]

My suggestion whenever it gets bogged down is to do a "show proc cpu" and a "show proc mem" command and see if there is high CPU utilization or if it is running out of memory.

Burke

 

[gconnect]

what do you mean by "long pings"?

 

[shayv]

long ping means long timeout, for components that should be 10 to 20 msec, you get 200-300 msec.

 

[gconnect]

do you have enough ram?
are you experiencing any queue drops (dropping any pkts)?



during these times check the interface load.
'show interfaces'

if it is high, turn off fast-switching (if used) on the interfaces, then define an access-list to pinpoint the traffic that is flooding your router.

int e0
no ip route-cache
exit

access-list 123 permit tcp any any log-input
access-list 123 permit udp any any log-input
access-list 123 permit icmp any any log-input


log it to a syslog server, not the console
logging X.X.X.X
logging trap debugging

if a certain host is transferring huge files or flooding your router using any other data means you will see who. if this is not the case chenck your memory and cpu as rburke suggested above.

 

[shayv]

its the CPU that that becomes very busy 99% as you can see:

Router#sh proc cpu
CPU utilization for five seconds: 99%/57%; one minute: 99%; five minutes: 99%

and the IP input takes most of the CPU :

23 58075172 2401931 24178 41.92% 39.17% 39.46% 0 IP Input

How can i identify which interface and from which IP address is it comes from?

 

[routerman]

During the period of high activity look at the interface statistics, the output indicates input and output load in the range 0 to 255 where 255 is 100%.

Also packet and byte counters are there, should indicate which are the busy interfaces.

With the CPU values you have posted the router is processing many IP packets, you may not be using the optimal switching mode. The 57% value is the interrupt load on the CPU

Check to see if the router is capable of IP CEF. If it is then once you have completed your investigation on source and destination addresses enable it. This should drop the CPU load drastically.

 

[gconnect]

use the access-list i presented from the above post

the source and destination addresses will tell you where they are from

 

[shayv]

how does the "ip route-cache" affect on the CPU load?
what is IP CEF? and how does it affect on the CPU load?

I could see that on 2 interfaces i have error input packets, and packets were dropped.
eth 0/0
"input queue 5/75, 25810 drops"
"83700 input errors"
eth 0/1
"264158 input errors"

regarding output, i could see on all the ports errors in about the same amount as this :

218 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
218 lost carrier, 0 no carrier

Thanks ahead

 

[Frenchie2069]

CEF is Cisco Express Forwarding - it is an advanced routing feature that remembers the result of the lookup to the routing table and switches the packet rather that routing it. It reduces load on the CPU, but increases memory load. It is supported by most 3600s.

Have a look at

http://www.cisco.com/en/US/tech/tk827/tk831/technologies_white_paper09186a00800a62d9.shtml#express

for more information on IP Switching

 

[routerman]

These errors you see, did you `clear counters' and then wait until the problem re-appeared before looking at the interface stats?

One point I noticed recently on a 3725 was that the dropped packet counter only increments on half duplex ports. Not sure why this is.

Did you try the access list to see where the traffic load is originating from?

 

[shayv]

all the ports and the components that they connect to defined as 10Mbps full duplex.
most of the time the router works in 30% cpu,
"CPU utilization for five seconds: 38%/24%; one minute: 31%; five minutes: 29%"

in average once a day(not in the same time) it becomes 99% cpu.

when it works in 30% cpu i can steel see the erros , and packets drop, but only in 2 interfaces (0&1).

can this line "ip route-cache policy" affect the CPU load?

I'll try to add CEF.

any other reasons that can cause errors, and packet drop?

 

[routerman]

Sounds like you have more than one problem here.

The errors are probably due to suspect cables, or mis configured speed and/or duplex settings. However this problem wont affect directly the CPU use.

Input drops are not hardware problems, a document at this URL should assist in tracking the problem.

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a0080094791.shtml#topic2

Also you state that the problem occurs after a time period following a reload. Did you check the output from Sh mem?
I'd recommend noting the largest value for I/O memory after the reload, then at intervals until the problem occurs again. This indicates the largest free memory block, if it decreases over time the IOS may be at fault.



The line "ip route-cache policy" could affect the CPU usage, although this depends on the IOS your running.

It enables fast switching of the policy routing, and should lower the CPU load, However if your using IP CEF you dont need this command, as PBR is CEF switched by default in later IOS images.