NortelGuy1979
Programmer
Hello all,
I have a real stumper - even our vendor is stumped (who I used to work for), and unfortunately we are running Avaya NES Contact Center 7.0 so our support options are limited with Avaya.
We have a redundant CCMS, and two separate CCMA (non-replicating) that were all working fine. One CCMA was a virtual machine, the other was a regular 1U server. The 1U server suddenly stopped being able to pull up RTD. CCMA webpage --> launchpad --> Real-time Reporting.
At first we thought it was all RTD, but then we discovered that if you clicked on standard IVR display, standard nodal display, standard route display, that all three of those worked. The Application, Agent, and Skillset displays do not however. None of the settings on the CCMA have been changed. We ran iceRtdTrace and saw that the server is receiving nodal, route, adn IVR stats, but not the other three. We ran mRCV on the CCMS and saw it is sending all of them just fine to the proper multicast group. We checked the virtual machine - it's working fine for all. We then took down the 1U server, put a laptop w/ the same hard-coded IP address, on the exact same network port, and ran mRcv - and we can see all of the stats coming in. SO, althugh we didn't think IT had made any updates on the non-working server, we decided to rebuild it. Our server team spun up a new VM of W2k3 R1 SP2, 32-bit, Enterprise, and only approved hotfixes. I threw mRcv on it before going to the trouble of reloading CCMA completely - and we get the exact same thing. Only IVR, nodal, and route stats seem to appear. So we threw Wireshark on the server - I can see data coming in on all of the UDP ports from the multicast group - it's all getting there. If I run multiple instances of mRcv, i can see it getting the data on UDP ports 6030 and 6130 for example, and in Wireshark I can see the data on all of the UDP ports coming in - but it's like it's discarding the data.
Has anyone ever seen ANYTHING like this before?
We know it's not the network - we can verify that the data gets there.
We are only running approved hotfixes.
We uninstalled the A/V application.
We ran a netstat to make sure nothing else was binding to UDP ports in question
We're really scratching our heads. Thanks in advance!
-Matthew
Matthew - Technical Support Engineer Sr.
I have a real stumper - even our vendor is stumped (who I used to work for), and unfortunately we are running Avaya NES Contact Center 7.0 so our support options are limited with Avaya.
We have a redundant CCMS, and two separate CCMA (non-replicating) that were all working fine. One CCMA was a virtual machine, the other was a regular 1U server. The 1U server suddenly stopped being able to pull up RTD. CCMA webpage --> launchpad --> Real-time Reporting.
At first we thought it was all RTD, but then we discovered that if you clicked on standard IVR display, standard nodal display, standard route display, that all three of those worked. The Application, Agent, and Skillset displays do not however. None of the settings on the CCMA have been changed. We ran iceRtdTrace and saw that the server is receiving nodal, route, adn IVR stats, but not the other three. We ran mRCV on the CCMS and saw it is sending all of them just fine to the proper multicast group. We checked the virtual machine - it's working fine for all. We then took down the 1U server, put a laptop w/ the same hard-coded IP address, on the exact same network port, and ran mRcv - and we can see all of the stats coming in. SO, althugh we didn't think IT had made any updates on the non-working server, we decided to rebuild it. Our server team spun up a new VM of W2k3 R1 SP2, 32-bit, Enterprise, and only approved hotfixes. I threw mRcv on it before going to the trouble of reloading CCMA completely - and we get the exact same thing. Only IVR, nodal, and route stats seem to appear. So we threw Wireshark on the server - I can see data coming in on all of the UDP ports from the multicast group - it's all getting there. If I run multiple instances of mRcv, i can see it getting the data on UDP ports 6030 and 6130 for example, and in Wireshark I can see the data on all of the UDP ports coming in - but it's like it's discarding the data.
Has anyone ever seen ANYTHING like this before?
We know it's not the network - we can verify that the data gets there.
We are only running approved hotfixes.
We uninstalled the A/V application.
We ran a netstat to make sure nothing else was binding to UDP ports in question
We're really scratching our heads. Thanks in advance!
-Matthew
Matthew - Technical Support Engineer Sr.