Looking for Automated Testing Tool for PowerBuilder 1

[Statey603]

Hello,

We have been using an old version of Rational Robot [2003] on a Windows XP box to execute automated test scripts on our PowerBuilder 11 application. We have migrated our application to Windows 7 and now need to dismantle the XP test box. We were unable to get the old Rational Robot application running on Windows 7.

I evaluated a few Automated Test Tool trials: AutoIt and Macro Scheduler 14, but they do not seem to be able to handle screens with PowerBuilder DataWindows which comprise most of our screens. The AutoIt forum indicated that PowerBuilder objects do not respond to standard Windows APIs and therefore PowerBuilder applications are not compatible.

I appreciate any information concerning Automated Testing of PowerBuilder applications.

Regards,
Bill

 

[mbalent]

Check out Zeenyx.

They will be presenting their solutions at the Charlotte PowerBuilder Conference in May. Go to NCPBUG.ORG for more information.

Regards,

Matt

"Nature forges everything on the anvil of time"

http://www.anvil-of-time.com

 

[Statey603]

thanks mbalent.
I got a demo of the AscentialTest product made by Zeenyx just the other day. I have not yet evaluated the trial. It looks to be a very nice product, but might be out of our price range.

I did find a few more leads and have listed with my findings below.

TestComplete [by Smart Bear] [this is the most promising so far]
This product is very similar to the ancient, pre-IBM version of Rational Robot [version 2003.06] that we have been using and need to replace for our migration to Windows 7. Here is a brief review of TestComplete.
It allows you to record steps into what are called Keyword Tests which can be played back. These Keyword Tests can be modified. A recording can also be created in a script mode which produces something similar to our old Rational Robot scripts.The script looks like Visual Basic. I had problems with playback of scenarios when I recorded as Script, but have yet to encounter any issues with scenarios recorded as Keyword tests. I am still evaluating. I am hoping to find a way to replicate existing Keyword Tests and then being able to edit, or better yet, loop and provide input parameters for each iteration. I will try to follow up with more details after my evaluationis done.

Below are some other products that I evaluated.

Ranorex Studio 5.3.0This one claims tobe PowerBuilder compatible, but my findings indicate there is a hitch. In order for Ranorex to identify objects on PowerBuilder DataWindows, they need to have their AccessibleName property set. For us, this was a show stopper since we have never populated this property and it would be too dangerous to try to update all of our DataWindows for this purpose. I tried recording without modifying the DataWindows and recorded some simple scenarios, but the playback had random gaps of missing activty. Therefore, I give this application a thumbs down at least if testing legacy applications. It might be nice if oneknew about the AccessibleName property during inital PowerBuilder application project creation.

Macro Express 3.1Pros: This tool recorded and played back scenarios successfully with our PowerBuilder application. It was fairly easy to use. The recorded macros get stored in text based files. Very affordable.
Cons: The Macro Script consists mainly of Mouse Movement events making it extremely difficult to edit recorded script because it is very difficult to tell where you are in the scenario. Might be good if you want quick tests that never need to be modified.


IBM Rational Functional Tester [not completely tested]
Pro: Claims to be PowerBuilder Compatible.
Con: I omnly tested this briefly on an old XP box. It appears to be memory intensive. Numerous times the application became unresponsive. Also, playback of recorded scripts had gaps of missing activity. I hope to evaluate this on a Windows 7 boxwith more memory. Jury still out onthis one.

AutoIt
Con: Determined that this product is not compatible with PowerBuilder application due to use of non-windows APIs by PowerBuilder.

Macro Scheduler 14Con: Looks like this one has the same problem as AutoIt. Recording looks great, but playback unable to display screens with PowerBuilder datawindows.

Selenium [not compatible]
Tools to automate web browsers across many platforms.
This is not a Windows application testing tool.

Unified Functional Testing [not evaluated yet]
Pros: supposed to be PwoerBuilder compatible.
History: Mercury Interactive was bought out by HP, which then discontinued the WinRunner product in favor of their QuickTest Pro product, which it modified to support the API as well. That product is now bundled together with other tools under the name Unified Functional Testing. Things are a bit better with regard to HP’s QuickTest Pro, as they added support for PowerBuilder 12.5 in May of 2012, approximately 6 months after PowerBuilder 12.5 was released.

 

[Statey603]

Performed more testing this afternoon on Windows 7.

Ranorex Studio 5.3.0 [Thumbs down]

Although Ranorex ran faster [as expected], it still got hung up on the very first refeence to a Datawindow object. At this point, I don't have time to investigate if there are settings or options that might make this work, but since I have a few other contenders, I feel enough time has been spent on Ranorex.

TestComplete 10 [continued] [Still the front-runner]

This one is turning out to be my favorite. It is fairly easy to use and I just started evaluating Data-Driven testing. TestComplete supports data input from many sources, such as Excel, CSV, DB table, DB Query.

I will try to follow up with more evaluations and results next week.

 

[Statey603]

Here is a little more info on my evaluations

TestComplete
I have been having difficulties figuring out how to use the same Test scripts with different environments [SIT: System Integration Test, UAT: User Acceptance Test]. Pretty much the only thing that I am having problems with is that the script does not recognize screens because the Windows screen Caption text specifies the environment. I dhave not been able to figure out how to handle this in the script. I believe the Winodow/Screen properties [like Caption] are captured when recording but do not know if they can be altered. If we cannot easily modify scripts so they can be run in either environment, then this tool is not goig to work for us. Maintaining multiple copies of each scenraio script will not be acceptable since we are continually updating the existing scripts as product features are introduced and screens are added or changed.

IBM Ration Functional Tester [RFT]
I started the evaluation of RFT. I already don't like the script piece of it. I beleive it is straight Java, but not so easy to work with. Similar issue as with TestComplete on how to have somewhat generic scripts that can be used in multiple environments with minimum maintenance. I am still in the initial part of my investigation so I might be jumping the gun on declaring this application difficult to use, but the old Rational Robot 2003 version that we are replacing is much easier to use and maintain.

If anybody has any suggestions, examples on using any of the Test Automation Applications with different Database environments [SIT, UAT, etc] and with PowerBuilder, I am all ears.

I will post more when I get a chance.

 

[sahildev80]

Hi,

I have been looking through the above discussion and trying to follow it.

I'm into Engineering analysis.

For engineering analysis, we have a series of test cases.

In each test case, we do a series of steps, these include: click several menu buttons, hit on a particular location in the window, let the program perform a task by hitting a menu button (this takes some minutes), once the program has completed the task, I again hit some buttons.

But all the above tasks are repetitive, they are carried out in an Engineering software application say "Engineering software A" .

Now, I have been doing all the tasks (in the bold above) by hand.

I want a tool that can completely automate the above.

Is it possible by an external application / tool to do this?

Sorry for a naïve question.

Regards,

Ann.

 

[Statey603]

I ended up going with TestComplete.

Below is my overview.

TestComplete is like a newer, better version of Rational Robot. TestComplete supports PowerBuilder and only required a few Name Mapping configuration changes after the first recording session for it to recognize enough about our PowerBuilder objects to consistently record and play back test scenarios. Recorded scenarios are stored in a proprietary Pseudo Code format called Keyword. The Keyword Test steps can easily be modified and more importantly, can be converted to script format in either VB, Java, Delphi, C++ or C#. The TestComplete product supports Data Driven testing which means various import sources can be used to pass input test data into a project script. When Data Driven capability is enabled, the script essentially loops through each row in the input source and pulls in the data. The Data Driven capabilities support many common data formats: Excel file, CSV file, External database, internal database and SQL Query. I was able to create a Data Driven test using an XLS file to pass in data typically modified in each Robot script during each test cycle; for example: Worker Name and Password. This is a huge improvement over Robot since this data will probably only need to be defined in a single XLS file to be updated during each test cycle instead of in each script. I also experimented with additional input data fields: Client IDs, Age Ranges and a flag to allow a row in the XLS to be skipped. I also experimented with converting Keyword tests to script. I found that the Script mode is easier to modify than the Keyword mode. There were a few minor issues with some of the Keyword steps not getting converted properly, but rather simple to remedy. I also confirmed that the script can be modified to add capability; for example: I inserted script to calculate the year for a client’s Date Of Birth based a Client Age passed in from the XLS file.