Dropped calls on IPO4

[573dawn]

Hi All, our system is dropping calls all over the place, and we cannot seem to discover why. Our BP has finally taken some logs and sent them off to Avaya. We have replaced our main DS unit, replaced the cat5 cable back to the PRI, SBC says the circuit tests clean, and I can't correlate the drops to network events.

One thing I have seen in Monitor (which I can barely read) is CALL DELETED. The DS units don't appear to be rebooting every time, although we do have somewhat of an issue there.

I am at a loss and we're headed into our biggest volume weekend of the year, so we're in trouble if we can't figure this one out.

Thanks, Dawn

 

[573dawn]

Great...They are saying clean, we even went to the backup circuit, they changed repeaters at the CO on my PRI and my PTP to my 21, I verified cables were good and even switched out the 21mm router. No change. I am dropping calls at all locations, it just keeps getting worse. I am ready to rip the whole thing out and go back to POTS. At least they were reliable...

Now time to go home and try to catch up on other work while my whole family, down for the 4th, gets to enjoy me working. Yay.

I don't know what else to do.

 

[aarenot]

Did your vendor do a network assessment prior to the implemenetation? Wait, you are Dawn, ofcourse they did not, what am I thinking. Try to do some testing on your network as well across your PTP's. See how they are performing on the data side.

 

[tlpeter]

a network assesstment has nothing to do with a T1 pri
the T1 comes in and goose to the ipoffice and is not going over the network (at least i hope it is that way)


ACA - Implement IP Office
ACA - Voice Services Management
______________
Women and cats can do as they please and men and dogs should relax and get used to the idea!

 

[573dawn]

aarenot, LOL! Nope...

tlpeter, actually, you are partially right. The PRI comes into our main office, then the calls are routed across T1 PTPs to our other locations. I have been unable to find any direct network issue that could be causing this. We have clean Ts, low errors, and hardly any utilization. I rarely see utilization above 5%. I can see some discards, both tx & rx, but the numbers are generally low. In Monitor I can occasionally see packet loss also, but it is very inconsistent and let's say I have 10 call drops in an hour, there are not corresponding indicators of packet loss on the DS units, nor can I see any correlation to discards on the routers.

However, despite my overall confidence in the network, I will try just about anything at this point, so if anyone has any ideas of what else to look for, I am all ears.

Thanks, Dawn

 

[573dawn]

Say, who is responsible for "clock slips" my BP says ATTT, ATT says our equipment. I can frequently see them in SSA.

 

[aarenot]

tlpeter,
I agree the network does not effect the PRI itself, it could effect the ICR's if they go over the T1's causing disconnects, which is why I was asking. That would create calls coming in on the PRI to be dropped possibly, but hopefully we can see that with some monitor traces.

What is the clock set to on the PRI? Do we have good earth grounds on all units control, and expansion at all sites? Clean power? NO dial plan conflicts, or name conflicts? Do you have a temperature issue where the equipment is located? IPO LAN port plugged directly into what(Same unit as the T1)? Do I sound like I am reaching?

 

[573dawn]

One more thing, we have a "mixed" network, the PRI & one PTP are B8ZS/ESF and 2 other PTPs are AMI/SF. Could this make a difference?

 

[573dawn]

aarenot, how do I tell how the clock is set? I will have to check on the grounds at all locations, I am not even sure where the PRI is actually coming in. ATT said the network interface boxes I can see are grounded are POTS, so I have no idea where the PRI is coming in. No name or dial conflicts that I have found after we corrected the duplicate HG names; no overlapping HGs. Temp is an issue at 2 of 4 locations, but the main location is temp controlled, and the site having the most trouble stays relatively cool. The IPO & PTPs are plugged into the same switches at each location (Avaya P133GT2s at 3 of 4).

And no, you don't sound like you are reaching, not after some of the things I've done

Thanks, Dawn

 

[573dawn]

I might mention that the site with the biggest problems is an old building with several add-ons and a wiring closet that boggles the mind...But we never had any problem with POTS, of course they only had 6 POTS line.

 

[aarenot]

Main site, look at the lines, PRI to see the clock setting on the PRI.

On the ground screws on the back of each IPO module, you may want to check that they are grounded to a verified earth ground. Some cold water pipes have PVC somehwere in the line, electrical counduits can have from my experience lack of continuity to ground especially in older multiphase constructed buildings. Have someone who knows how to use a meter test the grounds if you have any doubts.

Put a fan, and maybe a dehumidifier nearby where you have temp issues if you think the temp is serious. Normally they have serious issues with more than just the IP communications when they get hot.

Happy independence day! Hope we here in the colonies, don't offend anyone on the other side of the pond with that. Thanks to you French for all the naval support! I kinda think we returned the favor with the whole WW1, WW2 thing though. May freedom ring, and not just my ears from all the pyrotechnics!

 

[573dawn]

aarenot, thank you for the well wishes for our Independence Day! I left early, forgot there was any such thing as IPO, and had a very nice time with family and friends. Hope you did as well. On to business!

I am looking at the PRI settings, and it looks like they are all default, so here they are:
Switch Type: NI2
Channel allocation: 23 -> 1
Pefix: blank
Test Number: Blank
Clock Quality: Network
CRC Checking: Checked
CSU Operation: NOT checked
Haul Length: 0-115 ft
Line Subtype: PRI
Provider: Local Telco
Framing: ESF
Zero Suppression: B8ZS
Line signalling: CPE
Incoming Route Digits: 3

I have to wonder about the switch type, as our media errors always mention 5ESS, and I wonder about the clock quality being set to network when we have a mixed network (AMI & B8ZS), also the signalling being CPE. Is that the DS units or some other aspect of our CPE? I wonder about alot of things <g>.

 

[kurthansen]

Dawn,

Your mixed network of ami and b8zs are for your PTP T1s correct? Those just plug into your IPO via the ethernet port so it won't care what the network type is. Are you seeing slips on your PTP routers?

Now your PRI is the only digital trunk on your system. I haven't seen slips on a single trunk, I normaly see them when I have more then 1 and I am taking the clocking off the wrong trunk.

Do you drop calls on your digital trunk at all locations or just one site? If it is just one site then there must be some type of network problem between your main site with the PRI and the site with the problems.

 

[Gibsonic]

you can try setting your clock to "Unsuitable" but it ideally should be set to network in a single PRI configuration.

based on the info you gave about the less than ideal environment and cabling feeding the system, i would look there first at that, in my experience, is definatley a potential point of trouble.

I would suggest if at all feasible that you run a new, preferrably sheilded, cat5 demarc extension to your telco equipment if they are in different rooms/parts of the building/complex. Be sure to use a high quality jumper from the demarc extension to your equipment and be sure everything is very well grounded!

How far is the Dmarc from the IPO?

 

[573dawn]

kurthansen, yes, the mix is only the PTPs. I am not sure what I would look for as far as slips on the PTPs, but the most I generally see are some rx/tx discards. Of course I do get some errors on the PTPs, but I've never seen one that was completely error free, and these are well within expectation.

I wish I could tell you that my drops are only at one location, but they are at all locations, including the main. The highest number of drops is at the 21mm, but they have the highest volume of calls, even more than our main site. ATT was seeing some errors coming from our equip at the main, but since we replaced the DS & the cabling, they aren't seeing errors anymore. I DO see clock slips on my DS unit occasionally.

 

[573dawn]

Gibsonic, what will setting the clock to unsuitable do? Not to sound stupid, but in help it says "lines set to unsuitable will not be used", and I only have one line. We have run a new cable from the demarc, cat 5e, not shielded as far as I can tell. Let me see if I can explain the way it all comes in:

There are 3 NTI boxes outside, I don't know which one has the PRI wiring, I was told they are the POTS interfaces, but the PRI doesn't just magically appear out if thin air, it has to be coming through one of the three. From the NTI boxes the wiring goes through a concrete wall and into a closet where it interfaces to the smartjack. From there it is maybe 30 feet of cat5 to the DS. We have tested the cat5 and it tests clean.

The wiring closet I was referring to is at a remote location, the 21mm site which has the highest drops but that could be due to the high volume of calls they receive. Of our 4 locations, each has it's own pattern of incoming calls, but none are free from drops. By far the vast majority of drops are calls at/to/from the 21mm.

 

[deeplydigital]

I just wanted to add that I have been reading your post and have the same issues with my IPO running 4.07. I had hoped that the latest upgrade would take care of some of our issues. We only have one PRI at the main site and three remote sites with VOIP phones running over VPN's. Qwest says it is a timing issue with our equipment but not sure what to change. I am going to check our inside wiring and grounding but their NIU is right next to our IPO and we are using a CAT5E patch cord from their equipment to our PRI card.

 

[aarenot]

deeplydigital,
If you are having issues with phones using VPN this thread really will not help you, you might want to start one requesting help with IP soft phones/IP hard phones/VPN, and their proper configuration.
If you have multiple IPO's residing on your sites networks, which connect your multiple sites via point to point T1, then you should probably start your own thread as well, so we can be more help specifically to your scenario. This is Dawn's situation , but yours may differ in many ways. First question would be have you had a network assesment?
I would start a new thread/post, and copy and paste the url for that post here so we can find it quickly.

 

[573dawn]

Good Morning Posters! I wanted to tell the thread that I asked ATT this morning what type of switch was at the CO, and it is a 5ESS, so I have changed the switch type from NI2 to 5ESS this morning and am keeping my fingers crossed.

In addition, I asked about the PTP that was cut by the brush-hog (that goes to my most trobled location), and he old me they were supposed to have gone out and replaced the terminal(?) on the end that was cut. He indicated that he had placed the trashbag on it to make it obvious, and that if the trashbag was still there than it was unlikely that the complete repair had been made. He is going there now to look at it.

Meanwhile, deeplydigital, we are using digital phones not VOIP phones, and our network is straight PTP T1, no VPNs, so aarenot had the best idea for you: to start a new thread. I would like to mention though that we appear to have some clock slips going on as well, but thus far have not been able to pin down why.

 

[deeplydigital]

I realize that. I was just explaining our entire system. We have digital phones at the main location going out a PRI. The PRI is what we have had problems with not the VPN's. We have had busy signals, dropped calls, static on the lines, etc. Qwest repeatily said that they had tested the circuit and that the problems were on our end. I replaced the T1 card which didn't seam to help. I finally got Qwest to come out by requesting them to ground the NIU. While they were there I asked them to look at the logs locally on their NIU. There were several event errors on their equipment that stated "bad splice". Qwest guy said "that was odd" but decided to splice everything agian. It has been a week now with no issues. I can't say if it was the grounding of his equipment or resplicing the line but so far so good. I hope this helps. Thanks for your help.

 

[573dawn]

deeplydigital, that might help indeed. We have been told that our ground at the NIU at the trouble site is not up to current standards and are going to be replacing it.

Meanwhile, I found a mis-addresses switch at my trouble site and that took care of some of it, for a couple days. Now however, I am getting a steadily increasing number of drop reports. So back to square one...