Centillion 1400 w/intermittent echo on T1 voice ckts

[64panman]

We have inherited a 12 site voice/data network with a Centillion 1400 at the hub site and 1200's at the remotes. All remotes are connected via point to point fiber back to the hub 1400. Quad T1 cards in the 1400 break out T1 ckts for voice between the hub site PBX and each remote PBX (located behind a Centillion 1200). Timing for the T1s is provided by a Telco ISDN PRI brought in one T1 port and out another at the hub 1400.

Periodically, but with growing frequency, voice callers will hear their voices echo back to them. The distant party does not hear the echo. This problem exhibits itself both on calls made from one site to another through the hub site as well as calls made from one site through the hub and out to the PSTN.

Sometimes the problem will go away if the hub site Centillion 1400 and PBX are reset simultaneously. However, that does not always fix the problem.

How do the Centillions handle Echo Cancellation?


Any input would be appreciated.
Thanks.

 

[ahng1]

64panman,

I have the same problem, what type of phone switch are you using? Have you found any solutions?

Thank You,
Ahng1

 

[64panman]

The phone systems are eOn (formerly Cortelco, formerly ITT)Millennium ISDN PBXs. They can have a maximum of 32 T1 connections in a 1024 port switch. All T1 connections are configured as PRI ckts. The point to point circuits utilize QSIG.

One of the Remote locations is connected by a point to point T1 provided by the local Telco. We'll call this one Location D

Since the original post, we performed significant testing between the Central site and several remotes (we'll call them B and C). We were able to duplicate the Echo problem repeatedly to a specific PSTN phone number from Loc B and Loc C. We tried 3 different sets of control cards at the Central Site, with 1 previous software release and one newer software release. We removed all cards (including digital and analog phones/trunks)in the Central site system except a T1/PRI card to one Remote and a T1/PRI card to a TELCO PSTN PRI. The programming locations and card slots for these two cards were changed as well.

The Echo remained.

The Control equipment, software and T1 card was changed at the Remote location B. The Echo remained.

We put everything back to the way it was originally at all locations and the Echo remained.

Then we went to Remote location D that was connected via Point to Point T1 through a pair of Adtran 120 CSUs (NOT through the Centillion) and placed a dozen calls. NO ECHO. However, if we placed a call to Remote location B or C (which were connected to the Central site via Centillion ATM network) we occasionally had echo.

Finally, if we placed a call out from the Central site to the same test number we would get No Echo. If we then transferred the call to Remote B or C through the Centillion, the Remote end heard Echo.

We are working through another company to try and bring someone in with direct knowledge of the Centillion 1400 and 1200.

If we make progress, I will let you know.

 

[cornet]

Did you guys ever get a solution to this?

I had a similar problem on a customer site using 2 Centillion 1400's with 2 E1 trunk lines connected at to a PBX at each end. The 1400's were connected together via an OC3 link. There was also a redundant route between the 1400's via a Centillion 5000BH OC3 ATM mesh. The whole network was configured to run PNNI.

Each E1 interface was configured to run as a softPVC to it's equivalent E1 trunk on the other 1400 so that bandwidth on the OC3 link could be reserved for the voice channels and so that PVC would use the redundant link in the event of primary circuit failure. It was tested at the time (about 3 yrs ago) and the primary and redundant links worked acceptably.

Anyway, it all worked fine for ages, and then started to display the same problem as you are having. Intermittent echo on various phones. I pondered the problem for a while, but didn't reach any conclusions, and the problem didn't seem to be occuring too frequently so the customer wasn't complaining too much. Also, there were some other network problems which took precedence.

Some expansion of the network and movement of switches led to the mesh being re-engineered and an upgrade in code of the 5000BHs. Consequently, everything was rebooted and it's all worked fine since (about 8 months so far).

I suspect that the problems of echo were probably due to the fact that the voice traffic had probably ended up going via the redundant link where either:

a. the number of switches being traversed to get to the destination, or:

b. the volume of other data traversing the mesh, or:

c. the fact that there was no bandwidth reserved by the 5000BHs for voice traffic as it moved through the mesh via the redundant link,

or a combination of the 3 was causing the echo problem. I don't know if this will be of any help to you, but if it is or if you've fixed the problem it would be intersting to know as I don't know exactly what caused it to go away at my customer site.

BTW, if you need it a have a good tech note on how to set up softPVC's between Centillion switches.

 

[64panman]

The problem has been temporarily resolved. We can't say permanently, because we haven't been able to recreate the problem after the troubleshooting I am about to describe.

After all the PBX troubleshooting with no change, the focus was placed on the Centillions. There had been a fair amount of construction work in the vicinity of the Centillion 1400 in the previous 6 months. The decision was made to unplug, test the fiber, clean the fiber, retest the fiber for each OC3 at the 1400 end. A test call was made to the phone number that was "reliably" experiencing echo. The problem continued for the 1st 8 OC3 circuits. The 9th circuit tested somewhat lower than the first 8, and when it was plugged back in, the echo problem went away. The echo has not returned since Feb 20.

It turns out that the post by ahng1 in this thread was from one of the Techs working on the exact same systems from the Centillion perspective. Neither one of us was aware of who the other post was from until we commented to each other prior to a meeting at the customer premise the following day (small world).

A Nortel System Engineer was brought in the following day (after cleaning the fibers). He was unable to determine any definitive problem in programming related to the issue of echo. He did point out that there was only 1 source of synchronization for the T1 timing. There can be 3. The software was quite old and he felt there may have been problems with re-synchronization. He recommended upgrading all Centillions. (Current Code for 1400 is 7.0.10. Centillion 1200 is 7.0.2.5.

Secondly, The ATM ARE (ATM Routing Engine) was running at 31% free memory. He recommended upgrading from 24MB to 64MB.

The Centillion was set for default config (0) of "Estimated Number of Hosts." Entries over 500 were handled dynamicly. They had over 2400 IP ARP entries. An increase to 3000 Estimated number of Hosts along with more memory would help.

The customer is going to upgrade to the above mentioned specs. In the mean time, no echo.