IPO stability / reliability

[Steerpike58]

We have an IPO 406 in CA and another in NJ, both with a 16-port digital station expansion module, a PRI card, and a VCM card supporting about 10 users. In sept. of last year, the CA system (3.2(57)) became unstable - locking up (no response to ping, to Manager, etc) and requiring a hard power cycle to recover. These became more frequent (several a week) and eventually - after much pain with Avaya support endlessly requesting more traces - they replaced the system and it has since been stable.

Our identical 3.2 (57) system in NJ locked up during a simple configuration update (adding user extension, or similar- something that required a reboot), carried out over the WAN from CA. It turned out, the system had lost it's 'default gateway' setting as well a it's configuration, but remained at our network-assigned address (10.10.1.x). Avaya support said we should not try running even simple config updates over the WAN, so we then dedicated a local workstation in NJ to running updates to that system.

Just this week, we did an update to 4.1 in both locations. The CA update completed without incident, but the NJ update locked up during the final reboot. We had to attach serial cable and re-set the box to recover. When recovered, the box seemed to be 'at' 4.1, and ... was not at 192.168.42.1, nor at our assigned address of 10.10.1.7, but rather, at 10.10.1.77 (a DHCP assigned address, we believe). We reloaded our saved config and the system was good to go.

We have 24/7 phone requirements. Is the IPO typically this flakey? I'm now scared to touch the NJ box for simple updates. What are others' experiences? Are we just terribly unlucky?

Thanks!

 

[ProviderReborn]

To be fair to Avaya they are not normally unreliable.
They do have issues yes. But IP Office is one of our main brands. Alot of our client base is IP Office and we find it a successful product.

Normally an unreliable system can be down to a number of things;
-Poorly commisioned system (ie, not earthed. Poor cabling)
-Badly programmed.
-Wrong Manager versions used to program sites.
-Programming over a VPN/Remote Desktop session
-Poor customer network.



ACA - IP Office Implement
ACA - IP Telephony
CCENT - Cisco ICND1
CCNA - Working towards.

 

[Westi]

I work mainly on the IP Office and a few years back I would have said it needs a lot of work to become my favourite product, but they have gotten a lot better and more reliable. I have systems that run for two years without any reboots and without any problems, but I also know that playing around and "testing" things out sometimes causes them to do undesired things.
As for you mishap with the upgrade and the Ip address, do not do an upgrade unless you know exactly what you are doing.
If you reset the system via the DTE port and default it, it will grab a DHCP address as stated in the documentation, if you read it. Don't blame the system for your own shortcomings that is an approach that is far to common and just not right.

Joe W.

FHandw.
ACA
ACS

 

[crumthwacket]

I agree with the earlier posts that the system is stable but can become unstable if not programmed correctly. The earlier versions required a lot more patience than the current versions, but I have some here in the US that are going on 6 years old and very reliable.

Your post was a little critical and elicited a not-unexpected snarl from the inhabitants, but be nice and they'll take good care of you.

 

[h382]

Provider -

You mention programming via VPN/Remote Desktop as being a reason for unreliable systems.

What do you use to program remote sites across the country. Just curious.

 

[Westi]

I am not sure what the provider uses but I use VNC after dialing into the sites or use a VPN access and all the programming stays local if possible, that way if the network connection from me to the IPO drops while I upload the config it will not make the unit unusable. I never use remote desktop, especially with changes on the VM pro, there is also a tech tip out (#183 I believe) but Avaya told me that already a few years back not to use it. If there is a centralized voicemail then I will however program from the voicemail the remote sites via the VPN if there is no local machine setup for programming.

Joe W.

FHandw.
ACA
ACS

 

[tlpeter]

For remote changes dailin by isdn is the best way
I know that in the states this is not an potion but here in europe it is a proven way to do it

Reliability depends on how the system is configured
I believe that 95% of all the problems is related to programming !!!

Every system has it's downside's
Also every system has is great things but like always you here almost nobody about it



ACA - Implement IP Office
ACS - Implement IP Office
ACA - Voice Services Management
______________
Women and cats can do as they please and men and dogs should relax and get used to the idea!

 

[ProviderReborn]

Our method is.

Dial into the IP Office its self. On the customers VM Pc do a command line of;
Route add 192.168.99.0 MASK 255.255.255.0 192.168.42.1 /p
where 192.168.42.1 is IP Office.

Then just VNC from my laptop onto customers PC.



ACA - IP Office Implement
ACA - IP Telephony
CCENT - Cisco ICND1
CCNA - Working towards.

 

[Westi]

We don't sell the modem cards any more because you can't get the modem cards so we put a modem into the VM Pro computer and dial so into the network via an analog extension and let the local DHCP assign us an address, then VNC over to the VM computer.

Greetings from Canada

Joe W.

FHandw.
ACA
ACS

 

[tlpeter]

I prefer the method of setting dailin on the dhcp options at the system settings
Then you do not need to make an ip route at all
And when you did set up a user for the vmpro client you do not need vnc or remote desktop at all


ACA - Implement IP Office
ACS - Implement IP Office
ACA - Voice Services Management
______________
Women and cats can do as they please and men and dogs should relax and get used to the idea!

 

[Westi]

I also program via Manager locally so I don't have to send the config through the dialup and use the matching release of Manager (still in me from the beginning), I don't have the reliability of ISDN here it is analog lines and PRI's. No IP routes neccessary here either becasue I am getting an address from the DHCP server on site and it acts local, I just have to put my settings not to use the local gateway otherwise it is really slow and all my Internet connectivity tries to go over that connection.

Joe W.

FHandw.
ACA
ACS

 

[tanderson3733]

As a business partner I usually just follow Avaya's recommended remote access policy and put a modem and PC-Anywhere on the VM Pro system.

However for a reliable connection I use LOgMeIn if they have VM Pro, if it's embedded then I get the customer with Manager installed to join a web meeting.

 

[kwing112000]

We use Log me in for hundreds of customers and love it. it is free and easy to use. we have had very few issues and it works thru almost every firewall.

Kevin Wing
ACA- Implement IP Office
Carousel Industries

 

[Shine52]

What is wrong w/ Remote Desktop I wonder? I have used it succesfully for a while now on a few different systems. I've used Logmein, Real VNC as well, all w/ no issues. I would think those programs would be a little more intrusive(for lack of better word) then Remote Desktop. Just wondering why RD is not a good source. Your essentially local on the PC and any changes you make are from that Manager PC, not sure what issues there would be.

I've also used a Citrix client which has RealVNC loaded on the desktop w/out any issues as well.

Figure it out damn-it!

 

[Westi]

RDP creates a new session in the PC and can cause problems with the VM Pro call flows, I have customers that use it, despite my warnings, successfully but the problem is not the 1000 times it works it is the 1 time it screws up the database.
VNC uses the local desktop and just transmits the display and the button presses in the other direction, ujust like having a second monitor and keyboard and mouse.

this will explain it better

http://support.avaya.com/japple/css...eID=306419&temp.bucketID=162326&PAGE=Document

Joe W.

FHandw.
ACA
ACS

 

[Steerpike58]

I'm the original poster and would like to clarify and ask a follow on question. Thanks for all he replies.

First, regarding RDP (remote desktop, terminal services, etc). The Avaya link above has the following clarification:
============= Begin Quote =============
"It should be noted that using alternative remote access software that uses the same windows session as the local logged in console session will not experience this issue and will allow the Voicemail Pro client screens to be viewed and saved correctly.

With Terminal Services for Windows 2003 Server, the problem with Remote Desktop can be circumvented if the Terminal Service client takes control of the local logged in
console session rather than creating a secondary console session. (Reference Tech Tip 91)."
============= End Quote =============
Tech Tip 91 points to a Microsoft link that is broken, but the very simple way to do this is to run rdp from the start/run... dialog, and using the "/console" switch - eg,
mstc /console /v:10.10.0.1 - this connects to the actual console session of the server in question and according to Avaya thus circumvents this problem. Note - you can also add additional commands such as /w:1024 /h:768, etc.

More generally, I can see how a remote business partner may need to 'dial in' or use 'log me in' to traverse firewalls, etc - but I'm doing updates 'inside' the company network so I have complete access to the machines involved. I can literally sit at the computer in CA that runs Manager and VMPro, and can remote connect using RDP, VNC, or whatever to the desktop in NJ that runs the Manager app there.

I did the latest disastrous upgrade with an Avaya certified engineer at my side. He cautioned me to not use RDP for the VMPro portion, as mentioned above, so we froze to death and stood uncomfortably in the server room throughout the upgrade. For the upgrade of NJ, we did use RDP (with /Console switch), but only for the 'Manager' portion (there is no VMPro in NJ) so we did not violate any Avaya recommendation.

In any event, the recent lockup in NJ occured during the upgrade of the IPO unit, being run from the local computer in NJ via RDP in /console mode. The screen was displaying the 'upgrade progress' dialog, then issued the 'communication with IPO lost' (or similar) message. At which point, the box was un-reachable by any remote means. Since the office was empty at the time (after hours), we had to call in local technicians, and do the serial cable reset game (I notice the IPO 500 box has a reset button - novel idea!).

The computer in NJ we used to do the update was about 10 feet from the IPO, and is dedicated to that function. The Avaya Certified technicial said this was the right way to do the upgrade.

I asked the Avaya guy what we could do next time to avoid problems, he said - have an Avaya Certified Tech in the office during the upgrade, with serial cable/etc on hand in case something goes wrong. That, to me, is unacceptable, especially since I have to do the upgrade after hours and after hours involves higher rates for technicians, and - in that particular location - expensive overtime to have local supervision of outside contractors. What should be a 15 minute upgrade seems to require a massive coordination effort.

Does anyone have any suggestions as to how I can handle this differently? I'm considering buying a spare IPO unit and upgrading that 'offline', then transferring it to 'production' when the upgrade is done - which involves swapping the PRI, VCM, etc cards from the current live unit, of course.

What else could be causing these upgrades to lock up?

Thanks again!

 

[h382]

I use LogMeIn for all of my customers, and I love it.

And with Remote Desktop, you can do a /console to get the actual desktop without it opening a new console for you. Actually, I just noticed above this was pointed out already.

 

[gknight1]

logmein.com all the way!

 

[Westi]

steerpike: it seems that you have done all the steps (hopefully) with the Avaya "engineer" on site, but it still failed. So you upgraded the local machine's admin suite and then did the upgrade from there, it still failed and I have to tell you that can happen. I probably upgraded close to 1000 units (some several times) and it happened to me a few times that it crashed.
You are an IT guy I presume and have you ever done an upgrade on a server or PC that you had to revert back? Or even better did you ever have to reinstall Windows on a machine because it was not able to do something and it was the only way.
This is the same thing only it needs some more intervention to do so because if the firmware is not working the machine gets unresponsive and you need the physical access to the machine to fix it.

Joe W.

FHandw.
ACA
ACS

 

[aarenot]

I can see where paying to have a competent, product authorized, and certified system engineer proffessional to do the work is not understood by companies who have no technical expertise, or technical proffessionals in house themselves. After all, anyone who had any technical skills, and or due diligence to become qualified to do IT would also understand the value of trained qualified technical proffessionals, and not leave it to some unqualified, poser , wanna be, who thinks everyone else is overpaid, LOL.