CCM6.0 Real Time Display Issue

[aseens]

Hi Folks,

I have a remote site connected via MPLS. We have a small call center group out there that uses the RTD from CCM 6.0. We use unicast to a desktop PC onsite that powers a 55" display. this has been in place and working well for years. The issue seems to be occuring when we launch the display from any PC at that site.

A few weeks ago the supervisor started complaining of intermittant issues where data is not displaying consistantly. Sometimes he sees all data, other times he get the messge "No Data" on some of his billboard displays where others present correctly. The ones that display and those that do not display are random.

Whenever I launch his displays on the CCMA server or my PC that is on the local network I have no issues. This leads me to believe there is a network issue but I am having a hard time convincing my netowrk folks of this. Nortel documentation i regards to how RTD actually works seems very thin. Does anyone have any ideas or troubleshooting suggestion my netowrk team could perform?

 

[nanningmiss]

Is your MPLS bandwidth big enough?

At RTR Properties, you may like to check "Compress Realtime Data Packets" option. It will reduce bandwidth usage.

We were updated to CC7 now. So the path to RTR Properties could be slight differet. But it should be like this:

Start - Programs - Nortel - Contact Center - Manager Administration - Configuration, then choose RTR configuration.

 

[aseens]

MPLS bandwidth is not an issue. If it were we would have much bigger issues than inconsistant RTD. I have done some more investigation. It seems that the supervisor has done some customization to create these displays. He has multiple billboards created and then uses a Public Billboard collection to display 25 Billboads. 5 of these billboards are "summary" billboards and these are the ones that seem to consistantly show "No Data".

When I create a public billboard collection with only the 5 summary billboards and not the other 20 the stats display fine. When I launch the public billboard collection with all 25 billboards locally I get all information displayed ok. This only seems to happen at the remote site.

 

[steeviedude]

nanningmiss suggested you check RTR registry settings. There is a maximum number of Unicast sessions setting there. Think its normally set to 30 by default. NTP States "A separate unicast data stream is required for each unique unicast display on each client. The number of possible unique displays per client is 12— six for Moving Window statistics and six for Interval-to-date statistics. If more than one identical display for a particular statistic type is required on a given client, then only one unicast stream is sent for both."
You can check the concurrent connections by logging onto CCMA as webadmin
In Internet Explorer, go to

http://<app_srv_name>/supportutil/rtdcache.asp

where <app_srv_name> is the URL of the application server.
Enter the name of the Contact Center Manager Administration server.
The following information populates the window:
- a list of active unicast clients
- details of the data in the agent cache
- details of the agent template cache
- details of the skillset template cache
- details of the application template cache
- details of the IVR template cache
- details of the route template cache
- details of the nodal template cache

"A separate unicast data stream is required for each unique unicast display on each client. The number of possible unique displays per client is 12— six for Moving Window statistics and six for Interval-to-date statistics. If more than one identical display for a particular statistic type is required on a given client, then only one unicast stream is sent for both.

 

[nanningmiss]

If the problem happens only to the remote site, more of less it could be related to the bandwidth or latency.

As I said, Compress Realtime Data Packets could help. Or try to change (elongate) Output Rate or Transform Rate.

 

[aseens]

steviedude - what NTP are you pulling this from I cannot find the statement you are referring to.

Thanks,

 

[aseens]

Our vendor increased the max unicast sessions to 40 a couple weeks ago and that did nothing to resolve this issue. I would like to understand more clearly regarding the statement from the NTP "A separate unicast data stream is required for each unique unicast display on each client. The number of possible unique displays per client is 12— six for Moving Window statistics and six for Interval-to-date statistics. If more than one identical display for a particular statistic type is required on a given client, then only one unicast stream is sent for both."

Does this mean that each billboard is a separate Unicast stream. and if he has 25 billboard in a collection he is using 25 of the possible 40 whenever this diplay is launched? I have 20 billboards running and displaying consistantly at the remote site and 25 running locally on my PC with the same stats just fine. That would totoal 45. And I have the unicast limit set to 40. There are other supervisors in our company that are looking at the RTD using unicast in addition to what I am viewing.

The issue at the remote site only seems to occur when I launch the 5 summary billboards.

 

[aseens]

I when in to look at the the RTD Cache and the High Watermark for Unicast since last resart was only 16.

I doubled the default OAM timeout to 20000 ms and restarted the service but this did not seem to have an impact. I wtill have the same result. Successful on local client but still "No Data" on the 5 suammary displays on the remote client.

 

[steeviedude]

aseens apologies but probably bit of a foohpah on my part
NTP = 297-2183-934_12.11_Planning_and_Engineerin

For example, if two Agent/Moving Window displays are opened by the same client, only one Agent/Moving Window data stream is sent. However, if another client PC opens an Agent/Moving Window data stream, a new unicast stream is sent from the server. Two identical streams are open at this point.

Can your network Guys not mirror the users port and capture/analyse the data packets to see where they are being lost or held up? If so establish connection with just the 5 displays causing issue then launch all the rest to see impact on network.

 

[aseens]

HI steeviedude,

My network guy grabbed some captures of the wire at both end and indicated no packet loss either way and latency less then 25ms. He did mention he is seeing ICMP timeout being sent from the client back to the server. The network tech seems to believe the packets are making it to the client PC in plenty of time but the app is not compiling fast enough and is sending ICMP timeoput back to the server, but due to the UDP nature ofthe packets there is no resend. I tested on some different PCs at the remote location with the same result so it is not localized to one device...nic...etc. Im stumped.

 

[steeviedude]

did you try as nanningmiss advised "Or try to change (elongate) Output Rate or Transform Rate." Consider increasing your TTL on the CCMS server. This is found in RTD Multicast Configuration - even if you use unicast to the desktops, you are still multicast from CCMS to CCMA.

 

[steeviedude]

For it to have been working well in the past and suddenly become an issue I would check back through to see if any network updates/changes taken place recently, or changes to PC's?

 

[aseens]

Hi Folks I am back. Issue still persists. I ran wireshark on both the server and the clients for the same time period while I ran the display. There is definately some data not making it to the far end. My network technician is concerned about the large size of the UDP packets and the excessive fragmentation.....he seems to believe this is abnormal operation. My support company is not sure so I am hoping someone can confirm if the UDP fagmentation is normal.