SMGR - Outbound calls to 9+911 won't strip the first 9

[learningSkype]

we have a strange issue in our environment. our CM is setup to deny 911 calls, which forces callers to dial 9+911. fine, so when I test 9911, I get a busy signal + an audio message saying "No Routes Found" and a "480 Temporarily not available" message in my SIP trace on Session Manager.

it's bewildering and makes me feel like this never worked. I thought you could strip the 9 in Adaptations but every time I go in there and make a change, the calls still tries to outpulse "9911", which is going to fail every time.

how do you tell an Adaptation to look for 9911 and then strip the first 9 so that the call goes out just 911?

so weird...

 

[bignose21]

Think more info is needed, first where is it originating SIP or SIP AST Phones, or do you have G4xx with TDM phones etc. What is the path the call is taking so you mention CM but also mention Adaptation so CM--SM--SBC? or Phone--SM--SBC? If it includes CM like a SIP AST you would normally just have 9 as the FAC for ARS and 911 in ARS as alert/emergency in that case you would only expect to see 911 going to the Route-pattern. Can you do a list trace on the phone does that give you any idea?

 

[gwebster]

@learningskype - if you are based in the U.S. then you must be compliant with “Kari’s Law” which means that you have to allow a call to just 911 (i.e. no 9 access code) to be routed to emergency services.

 

[learningSkype]

bignose, the call originates on VOIP phones, mainly 96xx series. so the call path is the following:

CM > SM > SBC

Caller dials 9+911, CM sees 9911 coming from a VOIP station, strips the first 9 and based on ARS uses a specific route pattern, sends the call over a SIP trunk to the next hop, which is ASM. at this point the digits passed are just 911 and then SMGR or ASM appends the 9 again and then ASM delivers the call to our SBC, but now the digits passed are 9911.

I've been running CM and ASM traces since Friday and I see the CM use ARS and strips the first 9, then sends the call over to Session Manager and a 9 gets appended, then SMGR routes it to our SBC. but the call never gets past the SBC because of the digits getting past, which are 9911.

very strange. I don't know how this ever worked.

 

[bignose21]

So in the session manager you see it in the SIP trace arriving as 911 and leaving as 9911, if you enable SIP & Call Processing in traceSM in the session manager can you not see where it is being added?

 

[phoneguy55]

our CM is setup to deny 911 calls, which forces callers to dial 9+911

Terrible idea. I'm assuming you're in the US, if so you are not compliant with federal law. Please read about Kari's law.

https://www.fcc.gov/mlts-911-requirements

 

[kyle555]

If it's going CM > SM > SBC, I'm not sure why you'd add a 9 to every string going out to the SBC.

Otherwise, you should have an adaptation on CM or the SBC.
If the adaptation is on CM, the digit adaptation would be on "ingress to SM", or if to the SBC it would be on "egress from SM"

maybe you have a match of x, min 3 digit, max 17 digit, insert a 9
If so, having a match for min3/max3 911 delete 0 add nothing would be a pattern that would be a better match in that adaptation to make it not trigger the match that adds a 9.

Your mileage may vary, some assembly required, batteries not included. Don't touch your 911 routing and blame me

I saw this on reddit this morning:

https://www.reddit.com/r/sysadmin/comments/v1h3nr/tenets_of_it_a_list_of_often_humorous_it_wisdom/

You make me think of this one:
Everyone has a test environment, not everyone is lucky enough to have a separate production environment.

 

[ZeroZeroOne]

I'm only guessing, but I've seen a lot of systems where CM was added "behind" an older system and used tie trunks to the old system for outbound calls before SBCs and SIP trunks were in place. A leading "9" was inserted for all PSTN calls routing to the old PBX so that system would route correctly. But when SIP trunks were added, that leading "9" was never removed, or not fully.

Check your route patterns. Make sure there isn't any "9" being inserted. (Example):

                    Pattern Number: 10     Pattern Name: Emerg.
    SCCAN? n    Secure SIP? n     Used for SIP stations? n

    Grp FRL NPA Pfx Hop Toll No.  Inserted                             DCS/ IXC
    No          Mrk Lmt List Del  Digits                               QSIG
                             Dgts                                      Intw
 1: 10   0  781  1                [highlight #FCE94F]9[/highlight]                                     n   user
 2: 11   0  781  1                [highlight #FCE94F]9[/highlight]                                     n   user
 3:                                                                     n   user
 4:                                                                     n   user
 5:                                                                     n   user
 6:                                                                     n   user

     BCC VALUE  TSC CA-TSC    ITC BCIE Service/Feature PARM Sub  Numbering LAR
    0 1 2 M 4 W     Request                                 Dgts Format
 1: y n n n n n  n            rest                               pub-unk   next
 2: y y y y y n  n            rest                               pub-unk   none
 3: y y y y y n  n            rest                                         none

Check the ARS digit conversion tables and even the UDP tables - for each location. You just may find an erroneous entry.

                       ARS DIGIT CONVERSION TABLE
                                 Location: all                Percent Full: 0

  Matching Pattern     Min   Max   Del   Replacement String   Net  Conv ANI Req
  911                  3     3     0     9                    ars   n       n

                       UNIFORM DIAL PLAN TABLE
                                                              Percent Full: 0

  Matching                       Insert                Node
  Pattern           Len Del      Digits       Net Conv Num
 911                7   0        9            ext n

Use the command "list ars route 911# [location]" to verify that 911 is routing out the correct trunks. "list ars route 11#" will also confirm that "9-11" is correctly routing as "911".

In System Manager

Check for an Adaptation assigned to the Communication Manager SIP Entity. There may be a "9" inserted at that step.
Use the Call Routing Test (Elements / Session Manager / System Tools / Call Routing Test) to confirm "911" is routing correctly to your SBC

 

[learningSkype]

gents, thanks a ton for all the input. Your feedback helped me sort this out a lot better in my head. sucks when you don’t have documentation. Everything is a forensics investigation that takes days to piece together.

ZeroZero, I took your advice above and was able to confirm that the CM is using a route pattern that grabs a CO trunk as first choice, but that trunk is down, so the call is being sent to ASM as second choice in the route pattern and that's why my calls are going to the SBC. basically any calls that are routed to ASM are sent over to the SBC.

so my CO trunk ringing busy is contributing to this issue. I was able to find out what happened to the CO trunk and we’re working to restore.

In terms of the comments about dialing 911 vs. 9+911, I have to correct something. We do allow 911 to go out as well as 9+911.

Thanks again all.

 

[learningSkype]

The one thing I still don't understand is how does SMGR decide on which item takes precedence for calls, the Adaptation or the Dial Pattern?

Today I ran some new tests in which we placed test calls and I specifically told my Dial Pattern to use Routing Policy for SBC1 but the call goes out on SBC2. so the Dial Pattern is right, the SIP trunks are up and connected but the Dial Pattern is not kicking in.

so does SMGR look at the Adaptation first or the Dial Pattern first?
or is it related to string dialed?

example, if I have a Dial Pattern for 91 and I add a dial pattern for 91212555 and place a call to 12125557890, does SMGR route to the Dial Pattern that matches 91 or does it use the dial pattern for 91212555?

 

[ZeroZeroOne]

The answer is "Both". Adaptations and Dial Patterns have different but complementary functions.

The Adaptation sits between Session Manager and other servers and can convert dial strings and domain names either going TO or FROM Session Manager. Perhaps you are using an internal domain of pstn.company.com to use specific trunks in Communication Manager to ASM. You could then have an Adaptation on the SBC to strip the domain and put in a specific IP Address for the far-end router to comply with the SIP provider.

Dial Patterns are internal to Session Manager and are used to determine the destination of the call. In our previous example, the number 17815551234@pstn.company.com is received from CM. There is no Adaptation on CM so nothing changes. SM looks at the dial string and domain and selects the Dial Pattern that best matches, "1" for 11-digits for example, and routes to the SBC. The Adaptation on the SBC converts "17815551234@pstn.company.com" to "17815551234@xxx.xxx.xxx.xxx" and sends that to the Provider.

The more digits in the Dial Pattern the better the match. Let's say you have a dial pattern for "1" and another for "1212", both for 11-digits. A dial string "1212xxxyyy" would use the "1212" Dial Pattern since that matches more digits than "1". That's assuming both Patterns are set for "all" domains. If you have "1" for 11-digits with domain "pstn.company.com" but "1212" is set for "all" then "17815551234@pstn.company.com" would have a better match with the "1" with the specific domain.

We're only scratching the surface here to provide a very basic explanation. There are lots of online resources for reading up on SIP routing to provide greater detail.

I hope that helps.

 

[learningSkype]

it helps, thanks again.

I also want to mention that Avaya is not perfect and sometimes you discover a glitch. I found that last night in SMGR 8.1.2. basically when modifying a Dial Pattern and you select "Add" from the Originating Locations and Routing Policies, if you select "Apply The Selected Routing Policies to All Originating Locations" checkmark, it's supposed to take calls from all locations and then route them to the selected Routing Policy.

this doesn't work. even though all items become grayed out, when placing test calls, it doesn't work as designed. you have to actually select each "Originating Location" individually and then select a Routing Policy.

I opened a ticket and Avaya is investigating further. this added to my confusion on this whole issue. it's been about 3-4 days of forensics for me, trying to understand all this routing and then Avaya having a glitch on their product.

fun times!!

many moons ago I knew a really good Nortel PBX engineer that would find glitches and constantly report them to Nortel brass. but that was before they went belly up and were bought out, and then becoming Avaya Blue.

 

[kyle555]

@learningSkype - "Apply The Selected Routing Policies to All Originating Locations" doesn't work the way you think it does. It's not a bug.

In SM global settings, is the box ticked for "Better Matching Dial Pattern or Range in Location ALL Overrides Match in Originator's Location"?
Ticking the top-most box for location ALL is NOT THE SAME as ticking the box for each location.

Read this:

https://documentation.avaya.com/bundle/AvayaSessionManagerAdministering_r8.0/page/DialPatterns.html

There's a writeup in the admin guide of how SM routing works, but I'll try to paint a picture here...

"Better Matching Dial Pattern or Range in Location ALL Overrides Match in Originator's Location" - that makes SM act more like CM/ARS. You'll use a better match in location ALL even if you have a lesser match in your specific location.

The location is determined based on the IP of the oldest Via header in the invite. If that IP is in a SM location, then the call is from that location. As an example, say a SIP phone goes to SM to CM and out to SM. That last invite leg will have the phone IP as the oldest via header. If that IP isn't specified in a location, then it'll use the location of the CM SIP Entity. And if there's nothing there, it'll use the location of the Session Manager.

p.311 for locations:

https://download.avaya.com/css/public/documents/101079189

Here's some examples:

Say you have dial pattern 12x at domain ALL and 1234x also at domain ALL.
Suppose 12x is for location "NY" and 1234x is for location ALL.

You dial 12345 from NY.

If the better matching box is not ticked, you'll rollow the routing policy assigned to the 12x@all pattern for location NY
If the box is ticked, you'll pick 1234x for location all.

As a practical example, let's say you have little SIP gateways with analog trunks at each location. You want to use them if the core SIP trunks are unavailable as a 2nd choice and you also want to use them for 911 as a first choice, and then the SIP trunks if any particular gateway isn't available.

You'd build 2 routing policies for each location.
RP-Location1-SIP-Then-Local which would have 2 entries - the SBC rank 1 and the gateway at location 1 as rank 2.
RP-Location1-Local-Then-SIP which would have 2 entries - the gateway at location 1 as rank 1 and the SBC rank 2.

1x min/max 11 from location 1 would point to RP-Location1-SIP-Then-Local.
911 from location 1 would point to RP-Location1-Local-Then-SIP.
And then if you had a new location you never added, you can have a match for location ALL that goes to a routing policy that only goes to SIP 911.

Let's consider another example:
1-212-555-1212 is a number on your enterprise's Cisco switch. It's available via PSTN or on-net. You have an Avaya SBC tied to the Cisco CUBE. You could also dial it over the PSTN.

Let's say the dial pattern in SM is for domain ALL and location ALL.

If you have Avaya gateways and you have the number in each ARS location with a route to SM then the site's local trunk...
You'll always go out SM to the SBC to the CUBE if it's available.
If the CUBE is down, CM will send you out the local trunk on the gateway.

But if you have SIP gateways connected only to SM, you wouldn't be able to accomplish that with CM routing.
If "better match..." was true, you'd only follow 1-555-212-1212@ALL for your specific location if that match was as good as the match for location ALL.
That means you'd need to match a routing policy match to each originating location for that number to use the SBC first and then the local SIP gateway to do it.

If better match was false, then you could follow the 1x match specific to your location and go SBC first and then each location's specific gateway.

Remember: Ticking the top-most box for location ALL is NOT THE SAME as ticking the box for each location.

And you haven't even gotten into subdomain routing yet

Without the box ticked...
SM looks for a match in your location for the domain.
Then it lops off the most specific subdomain.

So... with the better match box unticked, 1234@east.us.pstn.customer.com will look for a match in your location for:
@east.us.pstn.customer.com
@us.pstn.customer.com
@pstn.customer.com
@customer.com

If it finds nothing, it'll do it all over again for location ALL.

With the box ticked, it does the same thing but considers location ALL in the first pass going over the subdomains.

Fun, huh?