CMS R16 data collection errors

[osheros]

We have a fairly new CMS R16.2 system with 3 ACDs connected. Two of the ACDs function just fine. The third ACD has intermittent data collection errors. These errors show up in the SPI_err logs as:

* "link or protocol failure"
* "switch time seconds changed"
* "clocks differ by > 24 hours" (often with bogus switch times like "343/23/06 123:08:89")
* "SPI configuration error: Two agents at extensions 55106 in split 74 and 68083 in split 84 are logged in with the same login ID: 68083. This may indicate table corruption on the switch. Contact Services to repair table as soon as possible." - Note on this one, extension 55106 isn't a valid extension.

Sometimes we also get bad report data, such as extensions instead of names or invalid amounts of agent stat times.

We've had Avaya check the translations for corruption and they report the CM side is clean. The ACD having the issue is running CM 3.1.4, CC 11.1. The other ACDs are running CM 3.1.4, CC 3 and CM 6.0.1, CC 6 respectively. The two CM 3.1.4 systems are on the same subnet as the CMS server. I've swapped CLAN boards to verify this wasn't the issue and confirmed that I'm not seeing errors on the network switch. Again, we don't have any errors for the other two ACDs.

We have opened a ticket with Avaya but they have been less than helpful. Has anybody seen anything like this before?

Thanks,
Sam

 

[justinsm2]

Is the CMS in close proximity to the CLAN you are using? If so it may be worth trying to use a crossover cable direct connection. That will allow you to rule out the network switch completely.

 

[osheros]

One of the other ACDs is connect to CMS on the same switch and has no issues so I don't think it's a switch problem.

Avaya came back with a thrown dart suggestion that my issue is because the two ACDs without a problems have EAS enabled and the one with issues does not. For the ACD w/out EAS, it is not enable on either the PBX or within CMS swsetup. Avaya's advice is that I enable EAS (which I'm licensed for) on this ACD (both PBX and CMS side). That seems stupid to me.

Anybody else have thoughts or believe Avaya could be right on this?

 

[osheros]

To loop back around... I determined the cause. Oddly, it was a documented fix in the CM 3.1.4 SP 2 release notes (and the CM 3.1.5 SP 2 release notes). Issue is addressed in PSN 1443, which I've attached.

This issue is specific to ACDs without EAS enabled. When a non-EAS agent is logged into a split and a direct extension caller hangs up (abandons before the call goes to cover), CM will send improperly formatted data to CMS, causing CMS to request a PUMP-UP but occasionally actually recording corrupt data to the tables. The bug was fixed in CM 3.1.4 SP2 and 3.1.5 SP2.

<vent>While either of the Avaya tech support's two suggestions (install SP2 or turn on EAS) would have worked, I wasn't about to do either without an explanation as to why either of these would resolve the data collection errors. They didn't seem to understand that applying SP2 would mean a large outage and enabled EAS would mean retraining a bunch of folks in various tiny call centers. It's disappointing their tech support spent no time actually examining the spi_err logs, like I suggested, which might have keyed them into finding the same note I did. But this seems typical for Avaya support these days and now I'm far more of a CMS expert than I ever wanted to be. So.. thanks Avaya.</vent>