DS30 RJ45 Expansion Module Boot Failure 5

[IPOTS]

PSA:
I upgraded a IP500v2 with 3 DS30B expansion modules and the control unit upgraded fine.
The expansion however did not. They would briefly appear in SSA and disconnect.
They were caught an endless loop.
Turns out the customer network was blocking TFTP and the expansions attempted to upgrade but would never complete.
Even if directly connected to attempt to recover fails.

This procedure resolved the issue.
Plug in USB to Serial adaptor
Check the Window’s Device Manager and verify USB to Serial cable port number.
Connect using Putty and configure the serial connection port and baud rate. (38400, 8, 1, N)
Boot expansion module and press escape until you get an "OK" response.

AT
OK
AT-X

Note: connect your laptop direct to the IP Office with Manager open and TFTP/BootP enabled under preferences.

After initiating the AT-X command you should see the module take the upgrade and come up.

 

[NYSTECH]

What version did you upgrade from/ TO?

 

[IPOTS]

11.1.1 to 11.1.2.4

 

[Richard10d]

I upgraded a larger 11.0.4.7 system this weekend to 11.1.2.4.

I had this problem. Unfortunately the trick did not work for me. I did open a ticket with Avaya and they have a bin update for the modules to fix the problem. Finding your post was so very helpful because it assured me it was a bug and not just a random hardware failure. It also had me prepared with my serial cable and the necessary commands to do the work before they send the solution.

https://support.avaya.com/ext/index?page=content&id=SOLN375062&viewlocale=en_US

It did NOT effect DS30B2 systems, but it DID effect DS30B systems. Here is what Avaya sent in the email.

How to recover the module:

Upgrade the DS30 unit to the binaries provided. [Customer and Business partner must open new SR to get the binaries]

Using a Serial cable to connect to DTE, followed this Process:
Configure PuTTY, HyperTerminal, or equivalent for serial as follows: 38400 - 8 - N - 1 - None - VT100
1. Put the CP bin files on the System’s SD card “system/primary” folder(the files dsaupnV1.bin, nadcpaV1.bin, nadcpaV2.bin from Jira IPOFFICE-171365 to overwrite the existing files with the same name)
2. Press ESC key on the DTE then power the DS unit (to enter the loader)
3. Hit Enter key to see the loader prompt “->”
4. Now enter command AT-X
5. The unit will install the new firmware (it takes about 10 minutes for the image to install)

How to upgrade if the IPO is on an older release: (You can use this for any downgraded systems or systems you have yet to upgrade).

1. Shutdown the 500V2 and disconnect the DS30B
2. Start up the IPO and upgrade to 11.1.2.4
3. Move the CP bin files to the System SD card, “system/primary” folder(the files dsaupnV1.bin, nadcpaV1.bin, nadcpaV2.bin from Jira IPOFFICE-171365 to overwrite the existing files with the same name)
4. Shutdown the 500V2, connect the DS30B (Start it up) and then start the 500V2.
5. If the module still fails to upgrade to the CP build, then the DTE method above is needed.

 

[IPOTS]

I got bit with this one again.
Upgraded an IP50v2 from 11.1.2.2 to 11.1.2.4 and the 30B expansions upgraded fine. The DS-30RJ45 expansions did not.
This was a remote upgrade so I had no way to recover the modules.

Avaya has the new bin files posted on the link provided above:

https://support.avaya.com/ext/index?page=content&id=SOLN375062&viewlocale=en_US

Do not attempt a remote upgrade if you do not have these new bin files loaded on the SD card. If will fail.
The module just bounces in service and out. The only way to recover it the direct connect using the USB to serial cable.

This is a critical issue. Best to know this prior to starting the upgrade. If you are on site make sue you have the USB to serial adaptor with you.

 

[lleeNC]

Fuck... I wished I'd seen this before I upgraded a customer in another state today. Same shit is happening. Upgraded the customer from 9.1 to 10.1 to 11.1.2.4 remotely and now the DS RJ45B stations are in a boot loop. This is for both x16 and x30 models.

Customer is pissed and can't fix this until tomorrow when the customer will be onsite. Fucking great.

WHY AVAYA WHY

 

[IPOTS]

@lleeNC. Make sure you follow the link to the Avaya bulletin. There you will find the bin files you need to copy to the SD card.
The module will not recover without having those files on the the SD card.

 

[lleeNC]

@IPOTS

Thanks. I've already sent instructions to the customer that is onsite and we will be recovering the DS30 and DS16 expansions today. This is a pretty big issue and they were aware of it for a while it seems. Why they didn't put out a notice... you know what I'm not even going to ask that question lol.

 

[lleeNC]

So I did the procedure today. One of the DS30 modules never came back so we are overnight shipping a new one. It's just bricked, no inputs through the DTE and solid red light. Rebooted it multiple times. This is before pushing the new firmware.

All other DS30s along with a DS16 came back after pushing the new firmware.