Test Automation Blog

Right after the recent release of Windows 7 Microsoft also released a platform update for Windows Vista and Windows Server 2008. The update brings in some of the new technologies from Windows 7 to those operating systems. Besides 'Windows Graphics, Imaging, and XPS Library' (which  actually gives you  new DirectX), 'Windows Portable Devices Platform' and 'Windows Ribbon and Animation Manager Library' (I hate the ribbon interface), the update also includes something called 'Windows Automation API', which "enables accessibility tools and test automations to access the Windows user interface in a consistent way across operating system versions"
Since Active Accessibility Microsoft has been providing programmatic interfaces for UI automation in Windows.  So if you are starting a Windows automation project, think about using this approach.  You can script with the Automation API in any of Microsoft's programming languages (Visual Basic, C# and even C++)  even with the free Express Editions of Visual Studio. This will not only be a really cost efficient approach, compared with the prices of automation tools like QTP or RFT, but probably will produce results which are more stable too.  After all this is the technology Microsoft uses to create UI tests for their own applications.

...use Window.TextSelect if you can not identify something

This is the final part of my hands-on review of TestPartner. Here I will try to demonstrate the old-style Visual Basic scripting approach, which I find to be more reliable.   I will again use my GMail scenario.

As I mentioned before, the only place I find recording useful is in such cases when you want to get to know how the tool ‘sees’ the UI.  I have no experience with the UI of GMail,  so I tried to record the scenario first.

This is the script that I got:

Sub Main()
 
' Begin record on Monday, 13. September 2009 at
' 11:17:07 by Admin
 
    ' Attach to Gmail Email from Google -
    ' Microsoft Internet Explorer IEWindow
    IEWindow("Gmail Email from Google - IEWindow").Attach
 
    ' Attach to Caption='Gmail: Email from
    ' Google'
    HTMLBrowser("Caption='Gmail: Email from Google'").Attach
        HTMLEditBox("Name=Email").SetText "testautomationblog"
        HTMLEditBox("Name=Passwd").SetPasswordText "***"
        HTMLButton("Name=signIn").Click
 
    ' Attach to Gmail - Inbox (2) -
    ' testautomationblog@gmailcom - Microsoft
    ' Internet Explorer IEWindow
    IEWindow("Gmail - Inbox (2) - IEWindow").Attach
 
    ' Attach to Name=cgudh0fxq7w2n
    HTMLFrame("Name=cgudh0fxq7w2n").Attach
        HTMLSpan("ID=':r3'").Click 43, 12
 
    ' Attach to Gmail - Compose Mail -
    ' testautomationblog@gmailcom - Microsoft
    ' Internet Explorer IEWindow
    IEWindow("Gmail - Compose Mail - IEWindow").Attach
 
    ' Attach to ID=':kl'
    HTMLFrame("ID=':kl'").Attach
        HTMLFrame("ID=':kl'").Click 33, 19
        HTMLFrame.Type "Lorem Ipsum Dolor"
 
    ' Attach to Name=cgudh0fxq7w2n
    HTMLFrame("Name=cgudh0fxq7w2n").Attach
        HTMLImage("ID=':ia'").Click
        HTMLDiv("ID='' Index=589").Click 48, 7
        HTMLAnchor("Caption='Drafts (1)'").Click
        HTMLSpan("ID='' Index=38").MouseDown 42, 7
        HTMLDiv("ID='' Index=702").MouseUp 358, 25
        HTMLDiv("ID='' Index=733").Click 35, 12
        HTMLAnchor("Caption='Sign out'").Click
 
' End record on Monday, 13. September 2009 at
' 11:18:22 by Admin
 
End Sub

As I expected, the recorded script could not be run successfully at all.  So what is wrong with it?

The recording has used two ways of addressing the controls. The first one is used with controls that TestPartner attaches to, those seem to be the controls it sees as more important,  for example see line 12. Those controls are identified using something called object maps. The object maps are artifacts that represent definitions of UI controls. When you access a control in a script via an object map, you provide only the name of the map. The actual properties used to identify the control are stored in the map itself. This is the definition of the object map from line 12:

Basically this says that the window we are interested in is an Internet Explorer window with a caption 'Gmail: Email from Google - Microsoft Internet Explorer'. The idea of object maps is that they try to separate how controls are identified from the test logic. If something changes in some particular control, you will have to modify only the relevant object map, but not the script itself.    This looks like a nice idea, but I found that in practice it does not always work that well.

The second way of addressing controls is called 'Raw attach names'. You can see it for example in rows 12-15.  Here you just provide sets of properties and their values as a string.  Notice how TestPartner identifies controls with expressions like "Name=cgudh0fxq7w2n"  and "ID=':r3'".  Unfortunately as we  saw before, those IDs and Names are dynamically generated and change each time you log in GMail. We need a better way to locate those controls. Unfortunately GMail does not provide easy and reliable way to locate some of them (like persistent names or IDs for example), and TestPartner on the other hand does not know how to identify controls based on their relative location in the page.

Still, with some dirty hacks,  I was able to quickly come up with this version of the script which can be executed reliably:

Sub Main()
 
    sText = "Lorem Ipsum Dolor"
 
    ' -- Login
    IEWindow("Caption='Gmail*'").Attach
    HTMLEditBox("Name=Email").SetText "testautomationblog"
    HTMLEditBox("Name=Passwd").SetPasswordText "********"
    HTMLButton("Name=signIn").Click
 
    ' -- Go to compose mail and type the text
    HTMLSpan("InnerText='Compose Mail'").Click
    HTMLFrame("Name='' Index=1").Click
    IEWindow.Type sText
 
    ' -- Center and save
    HTMLImage("Width=20 Height=20 Index=16").Click
    Window.TextSelect "Save Now"
 
    ' -- Wait for the draft to get saved
    Sleep 2
    Window.TextSelect "Draft saved"
 
    ' -- Go to Drafts
    HTMLAnchor("Caption='Drafts*'").Click
    HTMLSpan("InnerText='*" & sText & "'").Click
 
    ' -- Find the text and discard the draft
    Window.TextSelect sText
    Window.TextSelect "Discard"
 
    ' -- Logout
    HTMLAnchor("Caption='Sign out'").Click
 
End Sub

This is a quick summary:
At line 6 I attach to the IE window. This is the only place I attach in the script as I only work in one window. I used raw attach instead of object map. Note how I used a wilcard for the window caption. Lines 7-12 are self explainotary. What you see in lines 13 and 14 is really a small hack. The editor area in GMail is contained in a HTMLFrame.  TestPartner can not find anything meaningful in it to identify it, so I used HTMLFrame("Name='' Index=1"), which means basically the first HTMLFrame with empty name.  Because the HTMLFrame object does not support the SetText method (like the HTMLEditBox in line 7 for example), I first click on it (line 13) and then just type in the text (line 14). I do not feel very good about this solution, but it was the first quick hack I could think of.
Identifying the center button in the toolbar of GMail was also problematic (line 17). All the buttons in GMail also lack unique IDs or Names, so I was able to identify the button only as HTMLImage("Width=20 Height=20 Index=16") - the 16th HTMLImage with Width=20 and Height=20 in the page.
Clicking the 'Save Now' button (line 18) also proved more difficult than I anticipated. For some reason it appeared as an empty unidentifiable Div when I tried to locate it with the object browser.  I used the Window.TextSelect trick, which is the most universal way to make TestPartner click on some text you can not identify otherwise.  This is actually a helpful tip - use Window.TextSelect if you can not identify something.  After clicking the 'Save Now' button we have to wait for the backend to actually save the page before moving on with the test. A quick and somewhat dirty way to do this is shown on lines and - wait for 2 seconds and then verify that the "Draft saved" text appears on the page.  I think the rest of the script needs no explanation.

To sum it up - TestPartner can be buggy and confusing, but after all, it is powerful enough to let you automate almost anything.  Maybe you'll need some time to find the right workarounds. Probably they will be ugly. Overall there are much better tools, especially if you want to do web automation. There are worse too, but I am happy that I don't have to use TestPartner anymore.

Yes, it is about capture/replay again.

I decided to start my automation tool reviews with TestPartner 6  for two reasons - the first is that it is the UI automation tool that I am currently using on my project.  The second reason is because the 'Visual Tests'  (also called 'Visual Navigator') feature, introduced with the last major release of TestPartner, supposedly "allows application users—who are intimately familiar with the application under test, but often lack the technical expertise to use automated testing tools—to comfortably and confidently capture business transactions, verify the proper functionality is tested and assist in updating the test when inaccuracies are found or the application changes". The quoted text comes from the product review on the site of Compuware. With TestPartner 6 they are basically trying to catch up with the competition, because HP/Mercury QuickTest Professional (QTP) had this feature first - you record your UI actions and what is created is not code, but descriptive and supposedly less frightening to non-technical people list of entries in a table.

I do not actually use this feature and if you have read my previous post you know why, but I came up with this simple scenario to test how it works overal:

1. Open Gmail

2. Log-in

3. Go to 'Compose Mail'
   

4. Enter some text in the mail body

5. Center the text, using the toolbar button

6. Click ‘Save Now’ to save the email as draft

7. Go to the 'Drafts' folder

8. Click on the draft email you just created

9. Use the features of the test tool to check that the contents
   of the text in the email body are the same as entered

10. Discard the draft message

11. Log-out

I decided to use Gmail as a test application, because I wanted to challenge the tool a little by using some AJAX instead of static HTML and also because almost everyone is "intimately familiar" with Gmail.  The scenario is very short and I think that it is reasonable to expect that automation tools should be able to handle well this level of complexity.

I recorded the sequence with TestPartner.  In the screenshot below you will see how the recorded script looks like:

(click image to enlarge)

It basically borrows from QTP a  lot – for each step of the recording you see a ‘Screen Preview’. The steps are shown as a sequence in the so called ‘Storyboard’ and the currently selected step is shown enlarged in the preview. This is not just a screenshot, but contains the metadata for the UI objects in the application at this particular moment.  It is a little like having the real application in front of you at each step and can be nice in case you want to change something little in the middle of a long test.

To the right of the preview you will see the table with the recorded test steps. The descriptions are verbose like Click the text ‘Sign In’.
Using the ‘Logic Toolbox’  to the right you can insert control flow constructs  like loops and If statements, using a series of wizards. You can also insert checks to verify things (like in step 9 in my sequence) and add additional UI interactions.

I am a little skeptical about the whole idea, because to me it seems that if you can handle this king of wizard-assisted programming, then you can also learn to write code and in the long run you will be happier in terms of your efficiency if you do so.

I wanted to replay the script without any changes in the test environment first.  I expected this to work, but afterwards wanted to see if the recorded script would manage to execute successfully if I have other draft emails in the folder (with other subjects and body texts). My intention was to check how well the tool chooses which properties of the UI elements to use for their identification.