Decode Aura MST trace 2

[john3voltas]

Hello there.
I am well aware that one can decode MST using the trace analyzer feature in the "change mst" form.
But that will only work for 8 hours straight. So my question is, can I grab MST logs and run some command(s) on them and somehow get them decoded just as if it was done by enabling trace analyzer?
TIA
Cheers

 

[kyle555]

https://www.tek-tips.com/viewthread.cfm?qid=1778604

https://www.tek-tips.com/viewthread.cfm?qid=1780365

 

[john3voltas]

Hi Kyle555
Your post is indeed very useful so that people can understand what they need to do if they want CM to decode the MST traces.
But I'm not looking for that, otherwise I would just enable the "trace analyzer" option under change mst and let CM do the hardwork for me.
What I am looking for is a way to enable MST without trace analyzer and afterwards decode the traces on a laptop. This is because the trace analyzer option disables itself after 8 hours and I have MST logs that I need to decode so that I can read them.
Is this even possible?
Thanks in advance.
Cheers

 

[kyle555]

You're looking for Mr AvayaTier3

Wasn't sure if that was going to help you or not. Beats me.

 

[AvayaTier3]

The trace analyzer is built into the CM generic software unique to the software that captures the MST data in the log files. mta was built and compiled for linux on the CM server and there were versions compiled to run on Solaris and laptops for Avaya engineers. The source code for these stand-alone programs is updated with each CM software version and not available outside Avaya.

From server linux CLI you can build decoded traces:

sudo logc -t 20210915:0835-20210915:0850 --view mta > /var/home/ftp/pub/trace0915_0850.m

This command extracts MST messages from the logfiles for a fifteen minute window

A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[john3voltas]

Hi AvayaTier3

Thanks for your input on this matter.
Does this mean that even if I don't enable "trace analyzer" option under "change mst" I can still decode the MST traces and am able to read them as if "trace analyzer" had been enabled in the first place?

Thanks in advance.
Cheers

 

[john3voltas]

Actually, I just tried it myself and it didn't work.

User not authorized to execute mta, contact AVAYA
The reason may be that ACM is not running or
the Trace Analyzer may be disabled in ACM admin.

So, from the last sentence, it seems that for logc to work with --view mta then it seems one needs to enable trace analyzer option first.
Am I right? And that only seems to work for 8 hours. Right?

Thanks in advance.
Cheers

 

[G van Hamburg]

try use the full path, as dadmin!

/usr/bin/sudo/opt/ecs/bin/logc/ -c etc.

Freelance Certified Avaya Aura Engineer

 

[john3voltas]

Hi G van Hamburg.
Thanks for your input.
I just did, I tried using the full path for sudo and the full path for logc and yet I still get the same warning message as before.
Any other tips you can throw me?
TIA
Cheers

 

[AvayaTier3]

It is by design that the trace analyzer times out after 8 hours or if manually disabled.
The command from linux bash that I provided requires the trace analyzer being enabled in CM.

From server linux CLI you can build decoded traces:

sudo logc -t 20210915:0835-20210915:0850 --view mta > /var/home/ftp/pub/trace0915_0850.m

This command extracts MST messages from the logfiles for a fifteen minute window

A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[john3voltas]

@AvayaTier3,

Yes, that command works from an avaya linux server's CLI but only if trace analyzer is enabled in change mst.
So, what are MST logs good for if we haven't enabled trace analyzer in advance? We basically can't read them and apparently there is no external tool that can decode them or analyze them afterwards, and output a human-readable file, right?

TIA
Cheers

 

[AvayaTier3]

CM command line can be used with Command: status trace-analyzer, Command: enable trace-analyzer,
Command: disable trace-analyzer. It does not have to be used with Command: change mst

Years of experience allow one to search through files for certain data or signatures for what's stored.

There are lots of data types stored in the /var/log/ecs logfiles
MST data is embedded and stored in the /var/log/ecs logfiles
The data is unique to the pbx it is captured on regarding CM software version and CM pbx configuration.
The trace software is built for the Definity/CM software version. Bugfixes and enhancements are added in each software load. Angel addresses and UserIDs are decoded as stored on the system it was captured on.
The analyzer takes the MST data converts stored Hex data to readable and meaningful data for the pbx it was captured and decoded for.

You can use linux CLI to see MST data in logfiles but it is not useful in the raw form.

grep MST /var/log/ecs/xxx.log | less

change mst default
Log Mst? y (send MST date to /var/log/ecs/*.log files
Trace Analyzer? y
CALL TRACE? y
UID 8C95


A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[AvayaTier3]

Extracting data from /var/log files

[root@server1 ecs]# logv -?

logv  [OPTIONS]... [LOGS...] [-t TIME]  [[-a]FILTERS]...
logc  [OPTIONS]... [LOGS...] [-t TIME] [[-a]FILTERS]...
logw  [OPTIONS]... [LOG] [[-a]FILTERS]...

    Merges and displays the various log files in the system.
    Logv edits (vi) the results, while logc sends (cats) them to the
    standard output. Logw watches the single specified logfile
    for changes applying the specified filters.

    OPTIONS to modify command operation:

    HELP
    -?      produce this message
    -v      show the version of the command

    OUTPUT OPERATION
    -c      cat the log instead (set if called with name logc)
    -ls     list the names and sizes of log files
    -w      watch the log instead (set if called with name logw)

    OUTPUT FORMAT
    -b      remove blank lines
    -d      augment timestamps with time delta between resulting entries
    --dsep[N]
            separator line will be output for times greater than N (1) seconds
    -r      reverse order (latest entry is shown first)
    -s      strip off the timestamp (and other header) on each line
    -st     strip off other header (leaving timestamp) on each line

    VIEW OPERATIONS
    --views List the predefined log views.  A view specifies a log type as
            well as other selection parameters.
    --view VIEWNAME
            Display the log using the parameters associated with VIEWNAME

    LOG FILE OPERATIONS
    -ld dir look for log files in directory dir (instead of default directory)
    -lf file assume file is a log file (multiple -lfs can be given)
    -clear  clear the log files (only logmanager files)

    LOGS
      all        all possible
      lm         Logmanager debug trace
      lxboot     Operating system boot messages
      lxcron     Linux scheduled task log (CRON)
      lxkern     Linux kernel debug messages
      lxsys      Linux syslog
      lxsec      Linux access security log
      lxwtmp     Linux login/logout/reboot log
      lxxfer     Linux file transfer log
      wd         Watchdog logs
      cmds       Platform command history log
      httperr    HTTP/web server error log
      httpssl    HTTP/web SSL request log
      cmrestart  Communication Manager Restart log
            multiple logs can be given--logs are merged and sorted by time

    SELECTION FILTERS
    -l      only the latest file in a log searched (default is all)
    -t TIME filter for a particular date/time
             yyyy[mm[dd:[HH[MM[SS[mmm]]]]]]
                  time pattern
             yyyy[mm[dd:[HH[MM[SS[mmm]]]]]]-yyyy[mm[dd:[HH[MM[SS[mmm]]]]]]
                  time range
             today
             yesterday
            multiple time patterns/ranges can be given--entries that match
            any time will be output

    app-pat filter events for particular application/process (e.g. capro).
            An arbitrary string which does not match any LOG name can be given.
            Unless -g is specified, app-pat must be found at beginning of a
            :-separated field or preceeded by a [ or space.  Multiple app-pats
            can be given--log entries that match any app-pat will be output
            unless -a is also specified.
    -a      app-pats following a -a are applied to the output of those preceding
            the -a.  App-pats following -a are considered to be grep patterns.
    -g      assume app-pat filters are grep patterns instead of appnames
    -i      assume app-pat filters are grep patterns and ignore case of letters

A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[kyle555]

So what you're saying is that you can "enable trace-analyzer" and then
sudo logc -t 20210915:0835-20210915:0850 --view mta > /var/home/ftp/pub/trace0915_0850.m
and that'll decode it even if the trace analyzer was set to no in the mst form at the time the mst was running.

Did I get that right?

 

[AvayaTier3]

Correct

In the example, there would have to be MST lines in the *.log file for the window of time in order to decode MST data.

The analyzer is a decode tool.
mst is a collection tool.

A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[AvayaTier3]

decode data in individual logfiles

dadmin@Server1> dhelp logmst
logmst [-h] [logfile...|-]
Extract and convert MST messages from the trace log into the .MDF
format required by the MTA tools.

-c put all non-MST records in the output as comment lines
-h print hash mark every several records to give you the feeling that
something is happening

logfile reads from the specified logfile. If missing then uses logc to
read the logfiles.
- read logfile information from standard input
------------------------------------------------------------------------------
logc -view mta extracts and decodes mst data across ranges of logfiles based on the window of time (-t option)

A great teacher, does not provide answers, but methods to teach others "How and where to find the answers"

bsh

45 years Bell, AT&T, Lucent, Avaya
Tier 3 for 35 years and counting
[URL unfurl="true"]http://bshtele.com[/url]

 

[john3voltas]

OMG!
I didn't know this could be done.
So, all I need to do is collect MST logs and I can even leave the trace analyzer option disabled.
As soon as I need to review some logs, I simply run the enable trace-analyzer command and run the "sudo logc -t ..." from the command line, ét voilà, human readable logs in my hands.
Thank you all, especially AvayaTier3 for all the help you've given me.
Cheers

 

[kyle555]

Yup! We both learned something today.

Thanks professor!

 

[Wanebo]

Thank you AvayaTier3 for the wonderful information above.

 

[john3voltas]

Hi guys.
Sorry, I know this is not exactly relevant to the subject but it is somehow related to it.
The above set of instructions works perfectly except in cases where the (MST) logs have already been overwritten.
At least on this particular customer, I can now simply turn on MST collecting for 99 days but just to realize that the system only keeps the logs for less than 24 hours because after that they get overwritten.
So, my question is: is there a way to increase the log retention period in CM? Or maybe is there a way to keep sending the logs to another server for "archiving" purposes?
Thanks in advance.
Cheers