Test Automation Blog

...use Window.TextSelect if you can not identify something

This is the final part of my hands-on review of TestPartner. Here I will try to demonstrate the old-style Visual Basic scripting approach, which I find to be more reliable.   I will again use my GMail scenario.

As I mentioned before, the only place I find recording useful is in such cases when you want to get to know how the tool ‘sees’ the UI.  I have no experience with the UI of GMail,  so I tried to record the scenario first.

This is the script that I got:

Sub Main()
 
' Begin record on Monday, 13. September 2009 at
' 11:17:07 by Admin
 
    ' Attach to Gmail Email from Google -
    ' Microsoft Internet Explorer IEWindow
    IEWindow("Gmail Email from Google - IEWindow").Attach
 
    ' Attach to Caption='Gmail: Email from
    ' Google'
    HTMLBrowser("Caption='Gmail: Email from Google'").Attach
        HTMLEditBox("Name=Email").SetText "testautomationblog"
        HTMLEditBox("Name=Passwd").SetPasswordText "***"
        HTMLButton("Name=signIn").Click
 
    ' Attach to Gmail - Inbox (2) -
    ' testautomationblog@gmailcom - Microsoft
    ' Internet Explorer IEWindow
    IEWindow("Gmail - Inbox (2) - IEWindow").Attach
 
    ' Attach to Name=cgudh0fxq7w2n
    HTMLFrame("Name=cgudh0fxq7w2n").Attach
        HTMLSpan("ID=':r3'").Click 43, 12
 
    ' Attach to Gmail - Compose Mail -
    ' testautomationblog@gmailcom - Microsoft
    ' Internet Explorer IEWindow
    IEWindow("Gmail - Compose Mail - IEWindow").Attach
 
    ' Attach to ID=':kl'
    HTMLFrame("ID=':kl'").Attach
        HTMLFrame("ID=':kl'").Click 33, 19
        HTMLFrame.Type "Lorem Ipsum Dolor"
 
    ' Attach to Name=cgudh0fxq7w2n
    HTMLFrame("Name=cgudh0fxq7w2n").Attach
        HTMLImage("ID=':ia'").Click
        HTMLDiv("ID='' Index=589").Click 48, 7
        HTMLAnchor("Caption='Drafts (1)'").Click
        HTMLSpan("ID='' Index=38").MouseDown 42, 7
        HTMLDiv("ID='' Index=702").MouseUp 358, 25
        HTMLDiv("ID='' Index=733").Click 35, 12
        HTMLAnchor("Caption='Sign out'").Click
 
' End record on Monday, 13. September 2009 at
' 11:18:22 by Admin
 
End Sub

As I expected, the recorded script could not be run successfully at all.  So what is wrong with it?

The recording has used two ways of addressing the controls. The first one is used with controls that TestPartner attaches to, those seem to be the controls it sees as more important,  for example see line 12. Those controls are identified using something called object maps. The object maps are artifacts that represent definitions of UI controls. When you access a control in a script via an object map, you provide only the name of the map. The actual properties used to identify the control are stored in the map itself. This is the definition of the object map from line 12:

Basically this says that the window we are interested in is an Internet Explorer window with a caption 'Gmail: Email from Google - Microsoft Internet Explorer'. The idea of object maps is that they try to separate how controls are identified from the test logic. If something changes in some particular control, you will have to modify only the relevant object map, but not the script itself.    This looks like a nice idea, but I found that in practice it does not always work that well.

The second way of addressing controls is called 'Raw attach names'. You can see it for example in rows 12-15.  Here you just provide sets of properties and their values as a string.  Notice how TestPartner identifies controls with expressions like "Name=cgudh0fxq7w2n"  and "ID=':r3'".  Unfortunately as we  saw before, those IDs and Names are dynamically generated and change each time you log in GMail. We need a better way to locate those controls. Unfortunately GMail does not provide easy and reliable way to locate some of them (like persistent names or IDs for example), and TestPartner on the other hand does not know how to identify controls based on their relative location in the page.

Still, with some dirty hacks,  I was able to quickly come up with this version of the script which can be executed reliably:

Sub Main()
 
    sText = "Lorem Ipsum Dolor"
 
    ' -- Login
    IEWindow("Caption='Gmail*'").Attach
    HTMLEditBox("Name=Email").SetText "testautomationblog"
    HTMLEditBox("Name=Passwd").SetPasswordText "********"
    HTMLButton("Name=signIn").Click
 
    ' -- Go to compose mail and type the text
    HTMLSpan("InnerText='Compose Mail'").Click
    HTMLFrame("Name='' Index=1").Click
    IEWindow.Type sText
 
    ' -- Center and save
    HTMLImage("Width=20 Height=20 Index=16").Click
    Window.TextSelect "Save Now"
 
    ' -- Wait for the draft to get saved
    Sleep 2
    Window.TextSelect "Draft saved"
 
    ' -- Go to Drafts
    HTMLAnchor("Caption='Drafts*'").Click
    HTMLSpan("InnerText='*" & sText & "'").Click
 
    ' -- Find the text and discard the draft
    Window.TextSelect sText
    Window.TextSelect "Discard"
 
    ' -- Logout
    HTMLAnchor("Caption='Sign out'").Click
 
End Sub

This is a quick summary:
At line 6 I attach to the IE window. This is the only place I attach in the script as I only work in one window. I used raw attach instead of object map. Note how I used a wilcard for the window caption. Lines 7-12 are self explainotary. What you see in lines 13 and 14 is really a small hack. The editor area in GMail is contained in a HTMLFrame.  TestPartner can not find anything meaningful in it to identify it, so I used HTMLFrame("Name='' Index=1"), which means basically the first HTMLFrame with empty name.  Because the HTMLFrame object does not support the SetText method (like the HTMLEditBox in line 7 for example), I first click on it (line 13) and then just type in the text (line 14). I do not feel very good about this solution, but it was the first quick hack I could think of.
Identifying the center button in the toolbar of GMail was also problematic (line 17). All the buttons in GMail also lack unique IDs or Names, so I was able to identify the button only as HTMLImage("Width=20 Height=20 Index=16") - the 16th HTMLImage with Width=20 and Height=20 in the page.
Clicking the 'Save Now' button (line 18) also proved more difficult than I anticipated. For some reason it appeared as an empty unidentifiable Div when I tried to locate it with the object browser.  I used the Window.TextSelect trick, which is the most universal way to make TestPartner click on some text you can not identify otherwise.  This is actually a helpful tip - use Window.TextSelect if you can not identify something.  After clicking the 'Save Now' button we have to wait for the backend to actually save the page before moving on with the test. A quick and somewhat dirty way to do this is shown on lines and - wait for 2 seconds and then verify that the "Draft saved" text appears on the page.  I think the rest of the script needs no explanation.

To sum it up - TestPartner can be buggy and confusing, but after all, it is powerful enough to let you automate almost anything.  Maybe you'll need some time to find the right workarounds. Probably they will be ugly. Overall there are much better tools, especially if you want to do web automation. There are worse too, but I am happy that I don't have to use TestPartner anymore.

"Testers get indignant when they realize that the quality tools purchased at a serious price are themselves full of bugs. Indeed, test tools are often buggier than comparable (but cheaper) development tools. Plan to test your tools and spend time finding workarounds for bugs."

The quote comes from one of the best books on software quality I have read - Lessons Learned in Software Testingby Cem Kaner, James Bach and Bret Pettichord.  Be sure to read it if you haven't already.
I can not agree more with the authors. I spend a lot of my time in test automation trying to find workarounds for issues with the tool. Some of those issues can be considered 'functional idiosyncrasies', but most are just plain and simple bugs. During my work in the field of software quality I have noticed that users are normally quite tolerant of bugs, unless the bugs block them from doing their jobs. So for me there are two kinds of bugs in the test tools - those you can find a workaround for and those you can not.
If the tool is flexible enough and you know how to use it, most of the time you can make it do what you want - like for example using keyboard shortcuts and the clipboard to read the text from an unsupported GUI control. This is one advantage of tools that offer the flexibility of a real programming language for scripting.
There are also such things, for which you can not do much except waiting for the vendor to provide you a fix, and this can take long time. A few years ago I tried to upgrade to the latest TestPartner version (it was 5.2.3 back then). After the upgrade I found that all my automated scripts were causing random crashes in Internet Explorer. I just stayed with the old version until 5.6 came out, where this problem was fixed.

Recording does not work, even with IE

 I realized that I was not fair in my previous test,  as I used the last version of Firefox, which unfortunately is not officially supported by TestPartner. To give it a second chance, I decided to repeat the test with Internet Explorer.  I re-recorded the same scenario, but this time in IE and tried to replay it.

 I got a better result, but only slightly. The recorded script was able to log-in, but could not click the button to create a new mail:

The playback failed at step 12, where the tool tries to ‘Attach’ to an HTML frame, identified with ‘Name=c36gxk9p2bo3a’. Attaching to a UI control in TestPartner means simply declaring that the subsequent actions in the script will be executed within this control.  c36gxk9p2bo3a is a generated name which changes each time when you enter GMail.  This was the reason the script could not find the frame, when trying to replay.

 If I had to code this script by hand, I would omit this step altogether. Usually it is sufficient to only attach to the browser window and not to specific frames within it. Also, I would not locate the other controls by ID and screen coordinates as the recording did in steps 13 and 14. This will likely fail too, even if the problematic step 12 is fixed.

Yes, it is about capture/replay again.

I decided to start my automation tool reviews with TestPartner 6  for two reasons - the first is that it is the UI automation tool that I am currently using on my project.  The second reason is because the 'Visual Tests'  (also called 'Visual Navigator') feature, introduced with the last major release of TestPartner, supposedly "allows application users—who are intimately familiar with the application under test, but often lack the technical expertise to use automated testing tools—to comfortably and confidently capture business transactions, verify the proper functionality is tested and assist in updating the test when inaccuracies are found or the application changes". The quoted text comes from the product review on the site of Compuware. With TestPartner 6 they are basically trying to catch up with the competition, because HP/Mercury QuickTest Professional (QTP) had this feature first - you record your UI actions and what is created is not code, but descriptive and supposedly less frightening to non-technical people list of entries in a table.

I do not actually use this feature and if you have read my previous post you know why, but I came up with this simple scenario to test how it works overal:

1. Open Gmail

2. Log-in

3. Go to 'Compose Mail'
   

4. Enter some text in the mail body

5. Center the text, using the toolbar button

6. Click ‘Save Now’ to save the email as draft

7. Go to the 'Drafts' folder

8. Click on the draft email you just created

9. Use the features of the test tool to check that the contents
   of the text in the email body are the same as entered

10. Discard the draft message

11. Log-out

I decided to use Gmail as a test application, because I wanted to challenge the tool a little by using some AJAX instead of static HTML and also because almost everyone is "intimately familiar" with Gmail.  The scenario is very short and I think that it is reasonable to expect that automation tools should be able to handle well this level of complexity.

I recorded the sequence with TestPartner.  In the screenshot below you will see how the recorded script looks like:

(click image to enlarge)

It basically borrows from QTP a  lot – for each step of the recording you see a ‘Screen Preview’. The steps are shown as a sequence in the so called ‘Storyboard’ and the currently selected step is shown enlarged in the preview. This is not just a screenshot, but contains the metadata for the UI objects in the application at this particular moment.  It is a little like having the real application in front of you at each step and can be nice in case you want to change something little in the middle of a long test.

To the right of the preview you will see the table with the recorded test steps. The descriptions are verbose like Click the text ‘Sign In’.
Using the ‘Logic Toolbox’  to the right you can insert control flow constructs  like loops and If statements, using a series of wizards. You can also insert checks to verify things (like in step 9 in my sequence) and add additional UI interactions.

I am a little skeptical about the whole idea, because to me it seems that if you can handle this king of wizard-assisted programming, then you can also learn to write code and in the long run you will be happier in terms of your efficiency if you do so.

I wanted to replay the script without any changes in the test environment first.  I expected this to work, but afterwards wanted to see if the recorded script would manage to execute successfully if I have other draft emails in the folder (with other subjects and body texts). My intention was to check how well the tool chooses which properties of the UI elements to use for their identification.