7206 VXR high interrupts - overloaded?

[rootconsole]

Helping out on a 7206 in which the CPU seems to stay at a steady 97-99% utilization. 96% of that is interrupts. The output of show int is indicating throttles....dropped packets....lots of flushes...and interface resets. IP CEF is enabled as well as fast switching. Does anyone have a recommended plan of action to alleviate this issue?

wa-sa-rtr1#show interfaces
FastEthernet0/0 is up, line protocol is up
Hardware is DEC21140A, address is 0002.7d1d.3c00 (bia 0002.7d1d.3c00)
Description: Primary LAN
Internet address is 11.11.11.11/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 9/255, rxload 3/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 04:34:38
Input queue: 0/75/18/126452 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1303000 bits/sec, 491 packets/sec
5 minute output rate 3540000 bits/sec, 568 packets/sec
8744973 packets input, 2211354152 bytes
Received 18920 broadcasts, 0 runts, 0 giants, 16 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog
0 input packets with dribble condition detected
10963356 packets output, 1511919124 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
Ethernet1/0 is up, line protocol is up
Hardware is AmdP2, address is 0002.7d1d.3c1c (bia 0002.7d1d.3c1c)
Description: Interface for DSL Network
Internet address is 10.10.0.1/30
MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
reliability 255/255, txload 96/255, rxload 16/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 04:34:42
Input queue: 0/75/20/4660 (size/max/drops/flushes); Total output drops: 12037
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 653000 bits/sec, 443 packets/sec
5 minute output rate 3785000 bits/sec, 545 packets/sec
11040635 packets input, 2479431316 bytes, 2 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
13105503 packets output, 2716631678 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

FastEthernet2/0 is up, line protocol is up
Hardware is AmdFE, address is 0002.7d1d.3c38 (bia 0002.7d1d.3c38)
Description: ---------->Occam internet access
Internet address is 10.13.10.1/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:12, output 00:00:01, output hang never
Last clearing of "show interface" counters 04:34:45
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
551 packets input, 33088 bytes
Received 549 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog
0 input packets with dribble condition detected
2193 packets output, 131736 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
FastEthernet2/1 is up, line protocol is up
Hardware is AmdFE, address is 0002.7d1d.3c39 (bia 0002.7d1d.3c39)
Description: ---------->Occam net link to netvanta port 15
Internet address is 10.10.13.1/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:01, output 00:00:02, output hang never
Last clearing of "show interface" counters 04:34:45
Input queue: 0/75/0/457 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1000 bits/sec, 2 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
47357 packets input, 3840041 bytes
Received 23809 broadcasts, 0 runts, 0 giants, 1 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog
0 input packets with dribble condition detected
1714 packets output, 102840 bytes, 0 underruns
0 output errors, 0 collisions, 4294967295 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

Serial3/1:0 is up, line protocol is up
Hardware is Multichannel T1
Description: T1 to Brownwood
Internet address is 22.22.22.22/30
MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
reliability 255/255, txload 13/255, rxload 1/255
Encapsulation HDLC, crc 16, Data non-inverted
Keepalive set (10 sec)
Last input 00:00:03, output 00:00:00, output hang never
Last clearing of "show interface" counters 04:34:47
Input queue: 0/75/1/8404 (size/max/drops/flushes); Total output drops: 261
Queueing strategy: weighted fair
Output queue: 0/1000/64/261 (size/max total/threshold/drops)
Conversations 0/17/16 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1152 kilobits/sec
5 minute input rate 8000 bits/sec, 11 packets/sec
5 minute output rate 80000 bits/sec, 10 packets/sec
379799 packets input, 45900603 bytes, 0 no buffer
Received 1649 broadcasts, 0 runts, 0 giants, 1 throttles
83 input errors, 0 CRC, 0 frame, 0 overrun, 83 ignored, 0 abort
386654 packets output, 268283726 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
no alarm present
Timeslot(s) Used:1-24, subrate: 64Kb/s, transmit delay is 0 flags

Serial4/0 is up, line protocol is up
Hardware is M2T-T3 pa
Description: DS3 to Wcn
Internet address is 33.33.33.33/30
MTU 4470 bytes, BW 44210 Kbit, DLY 200 usec,
reliability 255/255, txload 39/255, rxload 72/255
Encapsulation HDLC, crc 16, loopback not set
Keepalive set (10 sec)
Restart-Delay is 0 secs
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 04:35:11
Input queue: 2/4096/0/495 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/4096 (size/max)
5 minute input rate 12572000 bits/sec, 8839 packets/sec
5 minute output rate 6822000 bits/sec, 8643 packets/sec
150322226 packets input, 1560370043 bytes, 26 no buffer
Received 1651 broadcasts, 0 runts, 0 giants, 0 throttles
0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 427 ignored, 0 abort
145768866 packets output, 1654286985 bytes, 0 underruns
0 output errors, 0 applique, 7 interface resets
0 output buffer failures, 0 output buffers swapped out
7 carrier transitions
rxLOS inactive, rxLOF inactive, rxAIS inactive
txAIS inactive, rxRAI inactive, txRAI inactive

Serial4/1 is administratively down, line protocol is down
Hardware is M2T-T3 pa
MTU 4470 bytes, BW 44210 Kbit, DLY 200 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation HDLC, crc 16, loopback not set
Keepalive set (10 sec)
Restart-Delay is 0 secs
Last input never, output never, output hang never
Last clearing of "show interface" counters 04:35:12
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 packets output, 0 bytes, 0 underruns
0 output errors, 0 applique, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
rxLOS active, rxLOF inactive, rxAIS inactive
txAIS active, rxRAI inactive, txRAI inactive
---------------------------------------------------------

 

[rootconsole]

rtr1#show ver
Cisco Internetwork Operating System Software
IOS (tm) 7200 Software (C7200-IK9S-M), Version 12.2(23), RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 28-Jan-04 15:11 by kellmill
Image text-base: 0x60008940, data-base: 0x6142E000

ROM: System Bootstrap, Version 12.0(19990210:195103) [12.0XE 105], DEVELOPMENT SOFTWARE
BOOTLDR: 7200 Software (C7200-BOOT-M), Version 12.0(10)S8, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)

wa-sa-rtr1 uptime is 2 hours, 40 minutes
System returned to ROM by power-on
System restarted at 07:20:53 CDT Tue Jul 28 2009
System image file is "slot0:c7200-ik9s-mz.122-23.bin"


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco 7206VXR (NPE300) processor (revision D) with 90112K/40960K bytes of memory.
Processor board ID 20403921
R7000 CPU at 262Mhz, Implementation 39, Rev 2.1, 256KB L2, 2048KB L3 Cache
6 slot VXR midplane, Version 2.0

Last reset from power-on
Bridging software.
X.25 software, Version 3.0.0.
Primary Rate ISDN software, Version 1.1.
4 Ethernet/IEEE 802.3 interface(s)
3 FastEthernet/IEEE 802.3 interface(s)
6 Serial network interface(s)
4 Channelized T1/PRI port(s)
125K bytes of non-volatile configuration memory.

20480K bytes of Flash PCMCIA card at slot 0 (Sector size 128K).
4096K bytes of Flash internal SIMM (Sector size 256K).
Configuration register is 0x2102

 

[rootconsole]

Building configuration...

Current configuration : 12576 bytes
!
! Last configuration change at 10:35:57 CDT Tue Jul 28 2009
!
version 12.2
service nagle
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname rtr1
!
no logging console
enable secret 5 <removed>
!
clock timezone CST -6
clock summer-time CDT recurring
clock calendar-valid
ip subnet-zero
ip cef
!
!


!
!
call rsvp-sync
!
!
!
!
!
!
controller T1 3/0
framing esf
linecode b8zs
channel-group 0 timeslots 1-24
!
controller T1 3/1
framing esf
linecode b8zs
channel-group 0 timeslots 1-24

!
controller T1 3/2
framing esf
linecode b8zs
channel-group 0 timeslots 1-24
!
controller T1 3/3
framing esf
linecode b8zs
channel-group 0 timeslots 1-24
!
!
!
interface FastEthernet0/0
description Primary LAN
ip address ### 255.255.255.0
ip access-group 179 in
ip access-group 179 out
ip nat inside
ip route-cache same-interface
duplex full
!
interface Ethernet1/0
description Interface for DSL Network
ip address 10.10.0.1 255.255.255.252
ip access-group 179 in
ip access-group 179 out
ip nat inside
duplex full
!
interface Ethernet1/1
ip address 10.20.40.1 255.255.255.252
duplex full
!
interface Ethernet1/2

ip address ### 255.255.255.252
duplex full
!
interface Ethernet1/3
no ip address
shutdown
duplex half
!
interface FastEthernet2/0

ip address 10.13.20.1 255.255.255.0 secondary
ip address 10.13.30.1 255.255.255.0 secondary
ip address 10.13.40.1 255.255.255.0 secondary
ip address 10.13.50.1 255.255.255.0 secondary
ip address 10.13.60.1 255.255.255.0 secondary
ip address 10.13.10.1 255.255.255.0
ip nat inside
duplex full
no cdp enable
!
interface FastEthernet2/1

ip address 10.10.1.129 255.255.255.128 secondary
ip address 10.10.13.1 255.255.255.0
duplex full
no cdp enable
!
interface Serial3/0:0
no ip address
shutdown
!
interface Serial3/1:0

ip address 10.20.30.1 255.255.255.252 secondary
ip address #### 255.255.255.252
ip nat outside
!
interface Serial3/2:0
no ip address
shutdown
no cdp enable
!
interface Serial3/3:0
no ip address
shutdown
no cdp enable
!
interface Serial4/0

ip address ### 255.255.255.252
ip access-group 179 in
ip access-group 179 out
ip nat outside
framing c-bit
cablelength 10
dsu bandwidth 44210
serial restart-delay 0
hold-queue 4096 in
hold-queue 4096 out
!
interface Serial4/1
no ip address
shutdown
framing c-bit
cablelength 10
dsu bandwidth 44210
serial restart-delay 0
!
ip nat pool me1 ## ## netmask 255.255.255.248
ip nat pool dsl_pool_1 ## ## netmask 255.255.255.0
ip nat pool dsl_cust ## ## netmask 255.255.255.0
ip nat pool wireless_cust ## ## netmask 255.255.255.252
ip nat pool Occam_cust ## ## netmask 255.255.255.224
ip nat inside source list 10 pool dsl_pool_1 overload
ip nat inside source list 12 pool dsl_cust overload
ip nat inside source list 13 pool wireless_cust overload
ip nat inside source list 14 pool Occam_cust overload
ip nat inside source static 192.168.1.14 ## ##
ip nat inside source static 192.168.1.13 ## ##
ip nat inside source static 192.168.1.12 ## ##
ip nat inside source static 192.168.1.18 ## ##
ip nat inside source static 192.168.1.10 ## ##
ip nat inside source static 192.168.1.19 ## ##
ip nat inside source static 192.168.1.17 ## ##
ip nat inside source static 192.168.1.16 ## ##
ip nat inside source static 192.168.1.11 ## ##
ip nat inside source static 192.168.1.15 ## ##
ip nat inside source static 192.168.10.2 ## ##
ip nat inside source static 10.10.1.2 ## ##
ip nat inside source static 10.10.1.3 ## ##
ip nat inside source static 10.10.1.254 ## ##
ip nat inside source static 10.10.1.100 ## ##
ip nat inside source static 10.10.1.253 ## ##
ip nat inside source static 10.10.1.252 ## ##
ip nat inside source static 10.10.1.22 ## ##
ip nat inside source static 10.10.3.252 ## ##
ip nat inside source static 10.10.3.253 ## ##
ip nat inside source static 10.10.3.254 ## ##
ip nat inside source static 10.10.4.252 ## ##
ip nat inside source static 10.10.4.253 ## ##
ip nat inside source static 10.10.4.254 ## ##
ip nat inside source static 10.10.5.252 ## ##
ip nat inside source static 10.10.5.253 ## ##
ip nat inside source static 10.10.5.254 ## ##
ip nat inside source static 10.11.10.103 ## ##
ip nat inside source static 172.16.0.254 ## ##
ip classless
ip route 0.0.0.0 0.0.0.0 ## ##
ip route 10.0.0.0 255.255.255.0 ## ##
ip route 10.0.10.0 255.255.255.0 ## ##
ip route 10.1.0.0 255.255.0.0 10.10.0.2
ip route 10.10.0.0 255.255.0.0 10.10.0.2
ip route 10.11.0.0 255.255.0.0 10.10.0.2
ip route 10.12.0.0 255.255.0.0 10.10.0.2
ip route ## ## 255.255.255.128 ## ##
ip route ## ## 255.255.255.252 10.10.0.2
ip route ## ## 255.255.255.252 10.10.0.2
ip route ## ## 255.255.255.252 10.10.0.2
ip route ## ## 255.255.255.252 10.10.0.2
ip route ## ## 255.255.255.248 ## ##
ip route ## ## 255.255.255.252 ## ##
ip route ## ## 255.255.255.252 10.10.0.2
ip route ## ## 255.255.248.0 ## ##
ip route ## ## 255.255.248.0 ## ##
ip route ## ##255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route 172.20.10.0 255.255.255.0 ## ##
ip route 172.20.11.0 255.255.255.0 ## ##
ip route 172.20.12.0 255.255.255.0 ## ##
ip route 172.20.13.0 255.255.255.0 ## ##
ip route 192.168.1.0 255.255.255.0 ## ##
ip route 192.168.2.0 255.255.255.0 ## ##
ip route 192.168.3.0 255.255.255.0 ## ##
ip route 192.168.4.0 255.255.255.0 ## ##
ip route 192.168.5.0 255.255.255.0 ## ##
ip route 192.168.6.0 255.255.255.0 ## ##
ip route 192.168.10.0 255.255.255.0 ## ##
ip route 192.168.100.0 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ## 255.255.255.192 ## ##
ip route ## ## 255.255.255.255 ## ##
ip route ## ##255.255.255.255 ## ##
ip route ## ## 255.255.255.0 ## ##
ip route ## ## 255.255.255.128 10.20.30.2
ip route ## ## 255.255.255.255 Null0
no ip http server
!
logging facility local4
logging source-interface FastEthernet0/0
logging ## ##
access-list compiled
access-list 7 remark /*********************************************************
access-list 7 permit 192.168.1.0 0.0.0.255
access-list 7 permit 192.168.2.0 0.0.0.255
access-list 7 permit 192.168.3.0 0.0.0.255
access-list 7 permit 192.168.4.0 0.0.0.255
access-list 7 permit 192.168.5.0 0.0.0.255
access-list 7 permit 192.168.10.0 0.0.0.255
access-list 7 permit 192.168.100.0 0.0.0.255
access-list 7 remark *********************************************************/
access-list 8 remark /*********************************************************
access-list 8 remark ACL for SNMP Monitor
access-list 8 permit ## ##
access-list 8 permit ## ##
access-list 8 permit ## ##
access-list 8 permit ## ##
access-list 8 permit ## ##
access-list 8 remark *********************************************************/
access-list 10 remark /*********************************************************
access-list 10 remark ACL for DSL NAT
access-list 10 permit 10.1.0.0 0.0.255.255
access-list 10 permit 10.10.0.0 0.0.255.255
access-list 10 deny 10.11.110.0 0.0.0.255
access-list 10 permit 10.11.0.0 0.0.255.255
access-list 10 permit 10.12.0.0 0.0.255.255
access-list 10 permit 10.13.0.0 0.0.0.255
access-list 10 remark *********************************************************/
access-list 12 remark /*********************************************************
access-list 12 remark ACL for DSL NAT
access-list 12 permit 10.11.110.0 0.0.0.255
access-list 12 remark *********************************************************/
access-list 13 remark /*********************************************************
access-list 13 remark ACL for Wireless
access-list 13 permit 172.20.0.0 0.0.255.255
access-list 13 permit 192.168.0.0 0.0.255.255
access-list 13 remark *********************************************************/
access-list 14 remark /*********************************************************
access-list 14 remark ACL for Occam
access-list 14 permit 10.13.10.0 0.0.0.255
access-list 14 permit 10.13.20.0 0.0.0.255
access-list 14 permit 10.13.30.0 0.0.0.255
access-list 14 permit 10.13.40.0 0.0.0.255
access-list 14 permit 10.13.50.0 0.0.0.255
access-list 14 permit 10.13.60.0 0.0.0.255
access-list 14 remark *********************************************************/
access-list 179 remark /*********************************************************
access-list 179 deny tcp any any range 135 139
access-list 179 deny udp any any range 135 netbios-ss
access-list 179 deny tcp any any eq 445 log-input
access-list 179 deny udp any any eq 445
access-list 179 deny tcp any any eq 593
access-list 179 deny tcp any any eq 4444
access-list 179 deny udp any any eq 4444
access-list 179 deny tcp any any eq 5554
access-list 179 deny udp any any eq 5554
access-list 179 deny tcp any any eq 9996
access-list 179 deny udp any any eq 9996
access-list 179 deny udp any any eq echo
access-list 179 deny udp any eq echo any
access-list 179 permit tcp any any established
access-list 179 permit tcp any any
access-list 179 permit ip any any
access-list 179 remark *********************************************************/
no cdp run
snmp-server community <removed> RW 8

!
dial-peer cor custom
!
!
!
!
gatekeeper
shutdown
!
banner motd ^C
******************************************************
* *
* WARNING --- WARNING --- WARNING --- WARNING *
* *
* !Unauthorized Access is Strictly Forbidden! *
* *
* Unauthorized access to networks, systems, hosts *
* or any other resources is strictly forbidden. *
* Violators will be prosecuted to the fullest extent *
* of the law. *
* *
******************************************************

^C
!
line con 0
line aux 0
line vty 0 4
exec-timeout 480 0
password 7 ## ##
login
!
ntp clock-period 17179873
ntp update-calendar
ntp server 132.163.4.102
end

 

[unclerico]

these types of problems are so damn hard to troubleshoot. take a look at this document and run through the steps. hopefully it will shed some light on your issue(s):

http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af0.shtml

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[VinceWhirlwind]

I wouldn't be looking at faulty cables, etc..., as you are getting a large number of flushes (not good) but no other errors.

Perhaps the traffic levels (which are fine here) reach huge peaks at some times of day?
Can you setup SNMP monitoring for all interfaces for at least 24 hours (or 5 days is better) to get an idea of the pattern of traffic levels?

 

[burtsbees]

Post a sh proc cpu, and show region...

/

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

 

[rootconsole]

Here is the show proc cpu and show region commands. I can confirm that this only really happens during peak traffic times. They are stating they get quite a lot of complaints from their DSL users when this happens.



rtr1#show proc cpu sort 5min
CPU utilization for five seconds: 73%/68%; one minute: 90%; five minutes: 93%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
37 49706996 51890049 957 4.91% 6.28% 6.52% 0 IP Input
25 166108 136642 1215 0.16% 0.24% 0.23% 0 Per-Second Jobs
92 592972 7317583 81 0.16% 0.10% 0.10% 0 IP NAT Ager
22 31688 58164 544 0.00% 0.03% 0.03% 0 Net Background
95 77844 87255 892 0.00% 0.04% 0.02% 0 SNMP ENGINE
4 77788 15460 5031 0.00% 0.06% 0.01% 0 Check heaps
3 164 1837 89 0.00% 0.05% 0.01% 2 Virtual Exec
55 82172 2298 35758 0.49% 0.04% 0.00% 0 IP Cache Ager
24 6708 136397 49 0.00% 0.01% 0.00% 0 TTY Background
10 76 15 5066 0.00% 0.00% 0.00% 0 OIR Handler
11 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
12 2388 136402 17 0.00% 0.00% 0.00% 0 IPC Periodic Tim
13 1548 136402 11 0.00% 0.00% 0.00% 0 IPC Deferred Por
14 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager
15 49444 246484 200 0.00% 0.00% 0.00% 0 ARP Input
16 340 6897 49 0.00% 0.00% 0.00% 0 HC Counter Timer
17 0 4 0 0.00% 0.00% 0.00% 0 DDR Timers
18 0 2 0 0.00% 0.00% 0.00% 0 Dialer event
19 4 3 1333 0.00% 0.00% 0.00% 0 Entity MIB API
20 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect
21 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
9 11828 136411 86 0.00% 0.00% 0.00% 0 EnvMon
23 32 383 83 0.00% 0.00% 0.00% 0 Logger
8 236 27360 8 0.00% 0.00% 0.00% 0 ALARM_TRIGGER_SC
7 0 2 0 0.00% 0.00% 0.00% 0 Serial Backgroun
26 0 1 0 0.00% 0.00% 0.00% 0 Inode Table Dest
6 0 2 0 0.00% 0.00% 0.00% 0 Timers
28 0 2 0 0.00% 0.00% 0.00% 0 Hawkeye Backgrou
29 0 1 0 0.00% 0.00% 0.00% 0 SONET alarm time
30 0 2 0 0.00% 0.00% 0.00% 0 VNM DSPRM MAIN
31 0 1 0 0.00% 0.00% 0.00% 0 CES Line Conditi
32 0 2 0 0.00% 0.00% 0.00% 0 Flash MIB Update
33 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM Input
34 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM TIMER
35 1956 15 130400 0.00% 0.00% 0.00% 0 TurboACL
36 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R
5 3484 8461 411 0.00% 0.00% 0.00% 0 Pool Manager
39 0 1 0 0.00% 0.00% 0.00% 0 PPPATM Session d
40 0 2 0 0.00% 0.00% 0.00% 0 PASVC create VA
41 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana
42 0 1 0 0.00% 0.00% 0.00% 0 PPP IP Add Route
43 2552 2778 918 0.00% 0.00% 0.00% 0 IP Background
44 180 2381 75 0.00% 0.00% 0.00% 0 IP RIB Update
45 0 2 0 0.00% 0.00% 0.00% 0 SNMP Timers
2 1164 27558 42 0.00% 0.00% 0.00% 0 Load Meter
47 21460 192948 111 0.00% 0.00% 0.00% 0 CEF process
1 0 16 0 0.00% 0.00% 0.00% 0 Chunk Manager
49 504 15806 31 0.00% 0.00% 0.00% 0 TCP Timer
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
50 36 74 486 0.00% 0.00% 0.00% 0 TCP Protocols
27 0 1 0 0.00% 0.00% 0.00% 0 CSP Timer
52 0 1 0 0.00% 0.00% 0.00% 0 RARP Input
53 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers
54 1160 3220 360 0.00% 0.00% 0.00% 0 DHCPD Receive
46 392 2302 170 0.00% 0.00% 0.00% 0 Adj Manager
56 0 1 0 0.00% 0.00% 0.00% 0 COPS
57 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall
58 0 2 0 0.00% 0.00% 0.00% 0 X.25 Background
59 0 3 0 0.00% 0.00% 0.00% 0 gk process
60 0 1 0 0.00% 0.00% 0.00% 0 Crypto HW Proc
62 1880 136401 13 0.00% 0.00% 0.00% 0 Net Serv Timer
63 0 2 0 0.00% 0.00% 0.00% 0 Crypto Support
64 0 1 0 0.00% 0.00% 0.00% 0 TSP
65 0 1 0 0.00% 0.00% 0.00% 0 QOS_MODULE_MAIN
66 4 1 4000 0.00% 0.00% 0.00% 0 CCVPM_HTSP
67 0 1 0 0.00% 0.00% 0.00% 0 CCVPM_R2
68 0 1 0 0.00% 0.00% 0.00% 0 CCSWVOICE
69 0 2 0 0.00% 0.00% 0.00% 0 Background Loade
70 0 1 0 0.00% 0.00% 0.00% 0 sssapp
71 4 2 2000 0.00% 0.00% 0.00% 0 Proxy Session Ap
72 4 22 181 0.00% 0.00% 0.00% 0 Crypto ACL
73 0 1 0 0.00% 0.00% 0.00% 0 Encrypt Proc
74 0 5 0 0.00% 0.00% 0.00% 0 Key Proc
75 0 3 0 0.00% 0.00% 0.00% 0 Crypto CA
76 176 9268 18 0.00% 0.00% 0.00% 0 Crypto IKMP
77 360 21472 16 0.00% 0.00% 0.00% 0 IPSEC key engine
78 0 1 0 0.00% 0.00% 0.00% 0 IPSEC manual key
79 76 13786 5 0.00% 0.00% 0.00% 0 RMON Recycle Pro
80 0 2 0 0.00% 0.00% 0.00% 0 RMON Deferred Se
81 0 1 0 0.00% 0.00% 0.00% 0 SYSMGT Events
82 952 133645 7 0.00% 0.00% 0.00% 0 cerf_daemon_proc
83 0 1 0 0.00% 0.00% 0.00% 0 RMON Packets
84 1104 133627 8 0.00% 0.00% 0.00% 0 trunk conditioni
85 0 1 0 0.00% 0.00% 0.00% 0 trunk conditioni
86 3076 80036 38 0.00% 0.00% 0.00% 0 Net Input
87 3460 27559 125 0.00% 0.00% 0.00% 0 Compute load avg
88 34084 3076 11080 0.00% 0.00% 0.00% 0 Per-minute Jobs
89 24 1151 20 0.00% 0.00% 0.00% 0 CC-API_VCM
90 24028 1277979 18 0.00% 0.01% 0.00% 0 CCPROXY_CT
91 128 4777 26 0.00% 0.00% 0.00% 0 CEF Scanner
48 0 1 0 0.00% 0.00% 0.00% 0 HTTP Timer
93 28452 173494 163 0.00% 0.00% 0.00% 0 IP SNMP
94 11956 87171 137 0.00% 0.00% 0.00% 0 PDU DISPATCHER
51 0 1 0 0.00% 0.00% 0.00% 0 Probe Input
96 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro
97 0 1 0 0.00% 0.00% 0.00% 0 SNMP Traps
98 2328 138763 16 0.00% 0.00% 0.00% 0 NTP
99 0 1 0 0.00% 0.00% 0.00% 0 xcpa-driver
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
100 16 1150 13 0.00% 0.00% 0.00% 0 DHCPD Timer
101 132 39035 3 0.00% 0.00% 0.00% 0 DHCPD Database
wa-sa-rtr1#SHOW PROC CPU SORTED
CPU utilization for five seconds: 94%/87%; one minute: 89%; five minutes: 91%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
37 49831228 51913058 959 6.96% 6.45% 6.47% 0 IP Input
25 166844 136792 1219 0.16% 0.24% 0.24% 0 Per-Second Jobs
92 594168 7322350 81 0.08% 0.10% 0.10% 0 IP NAT Ager
90 24092 1279032 18 0.08% 0.00% 0.00% 0 CCPROXY_CT
5 3504 8481 413 0.00% 0.00% 0.00% 0 Pool Manager
6 0 2 0 0.00% 0.00% 0.00% 0 Timers
7 0 2 0 0.00% 0.00% 0.00% 0 Serial Backgroun
8 236 27390 8 0.00% 0.00% 0.00% 0 ALARM_TRIGGER_SC
9 11832 136559 86 0.00% 0.00% 0.00% 0 EnvMon
10 76 15 5066 0.00% 0.00% 0.00% 0 OIR Handler
11 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
12 2392 136550 17 0.00% 0.00% 0.00% 0 IPC Periodic Tim
13 1548 136550 11 0.00% 0.00% 0.00% 0 IPC Deferred Por
4 77924 15480 5033 0.00% 0.06% 0.01% 0 Check heaps
15 49564 246725 200 0.00% 0.02% 0.00% 0 ARP Input
16 340 6905 49 0.00% 0.00% 0.00% 0 HC Counter Timer
17 0 4 0 0.00% 0.00% 0.00% 0 DDR Timers
18 0 2 0 0.00% 0.00% 0.00% 0 Dialer event
19 4 3 1333 0.00% 0.00% 0.00% 0 Entity MIB API
20 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect
21 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
22 31832 58197 546 0.00% 0.04% 0.03% 0 Net Background
3 196 1865 105 0.00% 0.00% 0.00% 2 Virtual Exec
14 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager
24 6732 136545 49 0.00% 0.00% 0.00% 0 TTY Background
26 0 1 0 0.00% 0.00% 0.00% 0 Inode Table Dest
27 0 1 0 0.00% 0.00% 0.00% 0 CSP Timer
28 0 2 0 0.00% 0.00% 0.00% 0 Hawkeye Backgrou
29 0 1 0 0.00% 0.00% 0.00% 0 SONET alarm time
30 0 2 0 0.00% 0.00% 0.00% 0 VNM DSPRM MAIN
31 0 1 0 0.00% 0.00% 0.00% 0 CES Line Conditi
32 0 2 0 0.00% 0.00% 0.00% 0 Flash MIB Update
33 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM Input
34 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM TIMER
35 1956 15 130400 0.00% 0.00% 0.00% 0 TurboACL
36 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R
2 1168 27589 42 0.00% 0.00% 0.00% 0 Load Meter
39 0 1 0 0.00% 0.00% 0.00% 0 PPPATM Session d
40 0 2 0 0.00% 0.00% 0.00% 0 PASVC create VA
41 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana
42 0 1 0 0.00% 0.00% 0.00% 0 PPP IP Add Route
43 2564 2781 921 0.00% 0.00% 0.00% 0 IP Background
44 180 2384 75 0.00% 0.00% 0.00% 0 IP RIB Update
23 32 383 83 0.00% 0.00% 0.00% 0 Logger
1 0 16 0 0.00% 0.00% 0.00% 0 Chunk Manager
47 21508 193108 111 0.00% 0.00% 0.00% 0 CEF process
48 0 1 0 0.00% 0.00% 0.00% 0 HTTP Timer
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
49 504 15844 31 0.00% 0.00% 0.00% 0 TCP Timer
50 36 74 486 0.00% 0.00% 0.00% 0 TCP Protocols
45 0 2 0 0.00% 0.00% 0.00% 0 SNMP Timers
52 0 1 0 0.00% 0.00% 0.00% 0 RARP Input
53 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers
54 1168 3223 362 0.00% 0.00% 0.00% 0 DHCPD Receive
55 82260 2300 35765 0.00% 0.02% 0.00% 0 IP Cache Ager
56 0 1 0 0.00% 0.00% 0.00% 0 COPS
57 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall
58 0 2 0 0.00% 0.00% 0.00% 0 X.25 Background
59 0 3 0 0.00% 0.00% 0.00% 0 gk process
60 0 1 0 0.00% 0.00% 0.00% 0 Crypto HW Proc
62 1880 136549 13 0.00% 0.00% 0.00% 0 Net Serv Timer
63 0 2 0 0.00% 0.00% 0.00% 0 Crypto Support
64 0 1 0 0.00% 0.00% 0.00% 0 TSP
65 0 1 0 0.00% 0.00% 0.00% 0 QOS_MODULE_MAIN
66 4 1 4000 0.00% 0.00% 0.00% 0 CCVPM_HTSP
67 0 1 0 0.00% 0.00% 0.00% 0 CCVPM_R2
68 0 1 0 0.00% 0.00% 0.00% 0 CCSWVOICE
69 0 2 0 0.00% 0.00% 0.00% 0 Background Loade
70 0 1 0 0.00% 0.00% 0.00% 0 sssapp
71 4 2 2000 0.00% 0.00% 0.00% 0 Proxy Session Ap
72 4 22 181 0.00% 0.00% 0.00% 0 Crypto ACL
73 0 1 0 0.00% 0.00% 0.00% 0 Encrypt Proc
74 0 5 0 0.00% 0.00% 0.00% 0 Key Proc
75 0 3 0 0.00% 0.00% 0.00% 0 Crypto CA
76 176 9278 18 0.00% 0.00% 0.00% 0 Crypto IKMP
77 360 21497 16 0.00% 0.00% 0.00% 0 IPSEC key engine
78 0 1 0 0.00% 0.00% 0.00% 0 IPSEC manual key
79 80 13802 5 0.00% 0.00% 0.00% 0 RMON Recycle Pro
80 0 2 0 0.00% 0.00% 0.00% 0 RMON Deferred Se
81 0 1 0 0.00% 0.00% 0.00% 0 SYSMGT Events
82 964 133768 7 0.00% 0.00% 0.00% 0 cerf_daemon_proc
83 0 1 0 0.00% 0.00% 0.00% 0 RMON Packets
84 1104 133750 8 0.00% 0.00% 0.00% 0 trunk conditioni
85 0 1 0 0.00% 0.00% 0.00% 0 trunk conditioni
86 3084 80122 38 0.00% 0.00% 0.00% 0 Net Input
87 3480 27590 126 0.00% 0.00% 0.00% 0 Compute load avg
88 34144 3081 11082 0.00% 0.01% 0.00% 0 Per-minute Jobs
89 24 1153 20 0.00% 0.00% 0.00% 0 CC-API_VCM
46 396 2305 171 0.00% 0.00% 0.00% 0 Adj Manager
91 128 4783 26 0.00% 0.00% 0.00% 0 CEF Scanner
51 0 1 0 0.00% 0.00% 0.00% 0 Probe Input
93 28496 173725 164 0.00% 0.00% 0.00% 0 IP SNMP
94 11984 87289 137 0.00% 0.00% 0.00% 0 PDU DISPATCHER
95 78000 87373 892 0.00% 0.04% 0.03% 0 SNMP ENGINE
96 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro
97 0 1 0 0.00% 0.00% 0.00% 0 SNMP Traps
98 2332 138921 16 0.00% 0.00% 0.00% 0 NTP
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
99 0 1 0 0.00% 0.00% 0.00% 0 xcpa-driver
100 16 1152 13 0.00% 0.00% 0.00% 0 DHCPD Timer
101 132 39085 3 0.00% 0.00% 0.00% 0 DHCPD Database
wa-sa-rtr1#

 

[rootconsole]

rtr1#show region
Region Manager:

Start End Size(b) Class Media Name
0x05800000 0x05FFFFFF 8388608 Iomem R/W iomem2
0x20000000 0x21FFFFFF 33554432 Iomem R/W iomem
0x55800000 0x55FFFFFF 8388608 Iomem R/W iomem2iomem2_cwt)
0x60000000 0x657FFFFF 92274688 Local R/W main
0x60008940 0x6142DB4B 21123596 IText R/O main:text
0x6142E000 0x61FCDEFF 12189440 IData R/W main:data
0x61FCDF00 0x62308D7F 3387008 IBss R/W main:bss
0x62308D80 0x657FFFFF 55538304 Local R/W main:heap
0x70000000 0x71FFFFFF 33554432 Iomem R/W iomemiomem_cwt)
0x80000000 0x857FFFFF 92274688 Local R/W mainmain_k0)
0xA0000000 0xA57FFFFF 92274688 Local R/W mainmain_k1)