CUCM 8.5 Memory Creep

[MitelInMyBlood]

We recently had to restart the Pub in response to low virtual memory alarm (only 2% available remaining - dangerously close to the wall). CCM was the biggest user, at the time of the restart ccm was consuming 49.8% of available memory by itself and total VM in use was 98.7% (!!) Immediately following the restart total memory consumption was back down to 34% of which ccm was using only about 7%, then the following morning (less than 24 hours later) total memory consumption was up to 42% and now (36 hours since the restart) total VM memory in use is 52%. This seems to be climbing daily and I think some of it is in response to routine programming changes (not added sets, just changes, ie. shared line apps, etc.) Is this normal behavior?

Original MUG/NAMU Charter Member

 

[MitelInMyBlood]

Here's a current capture from the CLI

login as: admin
admin@10.241.0.100's password:
Command Line Interface is starting up, please wait ...

   Welcome to the Platform Command Line Interface

admin:show
      show account
      show accountlocking
      show cert*
      show cli*
      show csr*
      show ctl
      show date
      show diskusage*
      show environment*
      show hardware
      show ipsec*
      show itl
      show logins
      show media*
      show memory*
      show myself
      show network*
      show open*
      show packages*
      show password*
      show perf*
      show process*
      show registry
      show risdb*
      show smtp
      show stats*
      show status
      show tech*
      show timezone*
      show trace
      show ups*
      show version*
      show web-security
      show workingdir

admin:show memory
      show memory count
      show memory modules
      show memory size

admin:show load

Executed command unsuccessfully
No valid command entered
admin:show process memory load

Executed command unsuccessfully
No valid command entered
admin:show process
      show process list
      show process load
      show process name
      show process open-fd
      show process pid
      show process search
      show process user
      show process using-most

admin:show process load?
Syntax:
show process load [options]
no parameters are required

options are:
cont       - the command will repeat continuously
clear      - the screen will clear before displaying output
noidle     - the command will ignore the idle/zombie processes
num XX     - configure the number of processes to be displayed (using most cpu)
             XX is the number of processes, default is 10 processes
             minimum value is 1, use 'all' for all processes
thread     - show threads
cpu        - sort output by CPU usage
memory     - sort output by memory usage
time       - sort output by time usage
             Note: that only one of cpu, memory or time may be specified
page       - pause output

admin:show process load memory
top - 21:08:37 up 2 days,  1:16,  1 user,  load average: 0.52, 0.60, 0.55
Tasks: 169 total,   2 running, 167 sleeping,   0 stopped,   0 zombie
Cpu(s):  8.7% us,  2.0% sy,  0.0% ni, 87.9% id,  1.3% wa,  0.0% hi,  0.0% si
Mem:   4127984k total,  4105516k used,    22468k free,     5272k buffers
Swap:  2048248k total,   577176k used,  1471072k free,  1464732k cached
  PID USER      PR  NI  RES  SHR S %CPU %MEM   TIME COMMAND
13698 ccmbase   15   0 661m  83m S  0.0 16.4  24:23 ccm
 6310 tomcat    23   0 481m  20m S  0.0 11.9 292:25 tomcat
28016 informix  15   0 246m 245m S  0.0  6.1 112:57 cmoninit
13175 drf       15   0 180m  11m S  0.0  4.5   0:33 CiscoDRFMaster
13464 licmgr    18   0 155m  11m S  0.0  3.9   0:09 CiscoLicenseMgr
28111 root      16   0 122m 120m S  0.0  3.0   0:32 cmoninit
13444 ccmservi  15   0 111m  11m S  0.0  2.8  21:06 amc
19608 ccmservi  16   0 100m  11m S  0.0  2.5   0:43 CCMDirSync
13666 ctftp     15   0  98m  15m S  0.0  2.4   1:16 ctftp
28187 root      16   0  95m  93m S  0.0  2.4   0:01 cmoninit
admin:

Original MUG/NAMU Charter Member

 

[MitelInMyBlood]

Per RTMT, tonight the VM Memory in use is now sitting at 60%, up from 52% yesterday and 42% the day before. The (2) subs are fine, only the Pub is doing this.

Original MUG/NAMU Charter Member