Automating IVR testing

[shilpad]

Hi all,
I'd like to test some IVR apps automatically. This involves three essential actions
1. Responding via a PC in the form of a DTMF signal
2. Capturing audio response
3. Comparing it with the actual voice result from the database

If have prior experience in this area or know a possible soluation then please let me know. Thanks in advance.

regards,

Shilpa

 

[pansophic]

Shilpa,

I have worked on exactly this kind of application before. Most of it is trivial if you purchase a CTI card like Dialogic (now owned by Intel) or Brooktrout card and use either their SDK or a third party SDK. The hardest part, as far as I am concerned, is accurately matching the audio responses. There are many ways to do this, but without getting into some serious DSP analysis it is difficult to say that you definitely got the response that you were expecting.

I have also been doing some work, trying to make a voice modem do the same thing. I know that it is possible, but it is not trivial.

Some of the comparison techniques that I have seen are response duration and keyword match. If you have some widely varing response durations, that may be enough granularity and it is easy to determine. If not, you might want to look into MS's Speech SDK. They you just find a unique word in each response, or a unique sequence of words, and match that way.

Good luck,

mgs

 

[shilpad]

Pansophic,

Thank you for the quick response. It is good to know somebody who knows his meat.
We are currently working on the IVR system using ITSP (net2phone) service. Maybe at this juncture we do not need a CTI system. I'll ge through the Net2phone APIs to see if any facility to capture and store the streams are provided.
Then I'll go through the MS speech SDK and see how the comparison works.
Another issue that bothers me is the test management process. As we are gong to have hundreds of call flow routes, so will be the number of test cases. maybe then I can use winrunner for automation (eg dial IVR numbers) and test director to manage the process.
Please let me know if you've any other feedback.
I really...really appreciate your reply.
Thanks,

Shilpa

 

[pansophic]

shilpad,

I've never used either of these applications. I always wrote my own dialer to test the IVR. I must assume then that your clients will be using Net2Phone to contact your IVR? I like to simulate the client environment when doing my testing if it is possible. If the typical client will use an analog telephone, I want to use an analog Dialogic card. If the typical client will use a T-1 terminated to the telco, then I try to use that configuration. That way your test is actually end-to-end, not just middle-to-end.

One important note. You need to test for failure conditions as well as successful conditions. It is nearly as important to correctly handle incorrect dialing sequences as it is to handle correct dialing sequences.

pansophic