Random Phones - Acquiring Service. IPO SE Remote Phones

[dsm600rr]

Hello all,

We have a Avaya IPO SE in a 11:11 Cloud with 5 Remote Locations.

One of the locations is having in issue with a few phones, the rest are registered. A handful are showing "Acquiring Service".

When I cleared the "VoIP Security" the down phones logged in. What could be causing the IPO to block a few random remote phones? They are all coming from the same Public at this particular location.

 

[john3voltas]

EDIT:
Sorry, i misinterpreted your message.
It seems you only have a single SE instance. What you seem to have is 5 different phone locations. If the remote phones on the other 4 locations don't support encryption (16xx) then it might still make sense. But if they don't, then my expired certificate theory doesn't make sense after all...

You consider it random. Well, I don't, otherwise they wouldn't all belong to the same location.
Are these J100 series phones? Like J139/J159 or of the sort?
If they are, *that* and the fact that they could register disabling VoIP security makes me think of certificates.
For instance, recently expired identity certificates on the SE instance that they all register to.
Either that or firewall issues that are preventing access to port 5061. They can crossover your firewall to port 5060 but they can't do the same to port 5061. That's another possibility.
But I'd immediately check the certificates under system security and check on their expiry date.
Regards

 

[dsm600rr]

@john3voltas
"It seems you only have a single SE instance. What you seem to have is 5 different phone locations.": Its a very Similar Setup to Powered by. IPO SE in a 11:11 Cloud, 5 locations.

"If the remote phones on the other 4 locations don't support encryption (16xx) then it might still make sense. But if they don't, then my expired certificate theory doesn't make sense after all...": Phones are J139's / J159's / J179's

You consider it random. Well, I don't, otherwise they wouldn't all belong to the same location: This location has 30 phones, 5 are down. 25 are not having any issues. Its definitely a location specific issue, however only effecting a handful of phones and the location.


Are these J100 series phones? Like J139/J159 or of the sort? J139's / J159's / J179's

If they are, *that* and the fact that they could register disabling VoIP security makes me think of certificates. Same cert has been in place for 2mo. DigiCert.

For instance, recently expired identity certificates on the SE instance that they all register to. Certificate set to expire Aug 18th 2025

Either that or firewall issues that are preventing access to port 5061. They can crossover your firewall to port 5060 but they can't do the same to port 5061. That's another possibility. Could be firewall, they have another company that handles that and of course everything "looks" ok. We are using Port 5061 on TLS.

But I'd immediately check the certificates under system security and check on their expiry date. Certificate set to expire Aug 18th 2025

Regards

 

[nnaarrnn]

First off- when did this begin? How long did they work beforehand?

Second- in SSA, Go to: System->VoIP Security->Blacklisted Addresses and see if the public IP of that location is listed, also check Blacklisted Extensions right above that.

If a user or phone hammers the call server with incorrect login info, the system could blacklist that public IP.


Also:
Make sure whoever is handling the firewall has any Deep Packet inspection (DPI-SSL in the SonicWall world) COMPLETELY disabled for that vLAN. It needs to be disabled for the Voice-vLAN->WAN access rule that allows that vLAN to get to the WAN (assuming no VPN tunnel exists here). I'd also have them show you screenshots proving SIP-ALG Transformations are disabled and that Consistent-NAT is enabled. (my line of thinking is that if these *did* work, but now don't, a firewall update/patch could have changed things unexpectedly)

 

[dsm600rr]

@nnaarrnn​

Thank you for taking the time to respond:

We switched them over to TLS 2.5 months ago and they have been fine for the first two months. They have IPO SE in a 11:11 VM w/5 locations using TLS (DigiCert) - no SBC. We have many other clients running the same everything with no issues.
All of a sudden a few weeks ago, one of the 5 locations lost all of their phones. They were all at "Acquiring Service". To make a long story short, we got IT on the horn, they said they would look into it and call me back. About 2 min later they all came back up, I called him to say whatever he did worked, and he told me he did nothing. We basically left it at that.

Fast forward to this week. First of all we have been have power outage all over (I'm in SC) from the Hurricane - so that isn't helping things.
Anyway, we get a call today, that another location (different from the first that happened a few weeks prior) has about 5 phones (out of 20 I believe) that are also showing "Acquiring Service". We get the firewall admin on the horn again - hes not seeing anything odd. So while they are talking, I go into SSA and just clear out all 3 "VoIP Security" Columns and then I see all of the 5 phones log in. This was earlier today when I made this post.

Now, in SSA I am seeing their public blocked for TLS 5061 under "Blacklisted Addresses". Why would the IPO be blocking this locations public? I'm wondering if that locations public gets blocked but some of the phones stay registered unless they reboot or something. With the below example, all of the phones were at "Acquiring Service" at this specific location. When I cleared this error, 20 or so phones logged back in. This was from my testing just now writing this response.

Also, they have an interesting public - thought I would note that. Starts with a 192 - not sure if things could be getting confused?


I will confirm your recommendations with their Firewall admin tomorrow am.
I went to add their public to the IPO's: VoIP > Access Control Lists > IP Whitelist however I had a few questions first.
- Why would just this locations Public be getting blocked in "Blacklisted Addresses"
- When I went to add this to the IP Whitelist, I wasn't able as it erred: "Maximum 20 IP Whitelist entries are supported" - I do not recall adding any however their are 20 in there apparently.

Thanks!

 

[Menniss]

We run a number of systems in the same configuration including some tenanted. (Cloud no SBC)

We don't often have an issue with blacklisting but when it happens for large site its very annoying especially if the customer cant find the phone causing the issue.

Blacklisted Addresses happens if a phone attempts to register with incorrect credentials, its done this based on the Public IP. at 10 incorrect it will block the IP.

The error in the screen shot looks more like a TLS error with incorrect formatting. We had this once when we switch a system that wasn't using TLS to using TLS. Most phone were fine they rebooted and too the new setting file. Some didn't and had to be defaulted.

Do you use DES?

On the whitelist, the system adds these automatically. Annoying that it's limited to 20, we remove the ones we don't care about and add ones that matter. I would be interested to know more about this function if anyone cares to share?

 

[dsm600rr]

@Menniss

I noticed the LAN2 (192.168.43.1) was in the IP Whitelist which is not being used so I deleted it and added their Public IP.

They could have a phone with the incorrect credentials trying to register from that IP over and over, however with how slow Avaya phones are - I don't really see that being the cause - but anything is possible.

This morning I saw the Public blocked again - yet all the phones were still logged in. I went ahead and Whitelisted it anyway.

The IPO has "TLS" Enabled on the LAN1 > VoIP Tab. All phones were reset to default when we cut them over to TLS.

I am not sure what DES is.

Thanks!

 

[derfloh]

Let a monitor trace run with default filters. You should see information why the IP is blocked there.

 

[Menniss]

I agree with derfloh Monitor is where you are going to find out what is causing the issue.

DES is Avaya's provisioning server we use it for all our cloud deployments make deploying the J100s much simpler.

Log in to Device Enrollment Service

des.avaya.com