Configuration of the test data required to be generated can be done by clicking on “Configure” button. After clicking “Configure” button, “Configure Data” window will be displayed. In this window, configuration of test data to be generated can be specified. (Ex: Range of numbers, Length of String, number of data etc). Configuration specified will be displayed in the Description column.

Test Data Generation can also be used as Stand Alone Application. Option of deleting and inserting the parameters will be available only when Test Data Generation is used as Stand Alone Application.Window name will not appear when used as stand alone application.
Any of the parameters can be deleted by clicking on “Delete”,additional parameters can be inserted by clicking on “Insert Parameter” button, this adds another row to the table.

Once all the parameters are configured test data can be generated by selecting the available generation methodology in dropdown list and click on “Generate” button.After clicking “Generate” Button provide the file name and save it.
Generated data will be saved in standard format containing Parameter Name as header for respective generated data.

The generated file will be as following :
The window name will appear as table Title and Parameter Name will appears in first row.

W1
Parameter1,Parameter2
Value111,Value112
Value121,Value122
Value131,Value132

W2
Parameter1,Parameter2
Value211,Value212
Value212,Value222
Value213,Value232

def test():
	java_recorded_version = '1.5.0_24'

	if window('w1'):
		select('First Name', 'ABCDEF')
		select('Password', 'adsf')
		method1('ABC', 'DEF', 'GHI')
	close()

The parameters which are to be converted to data driven can be selected by selecting the checkbox available in the first column. Clicking on the “Continue” button continues execution of test script and breaks at the next function call (usually in the next line of the script). The “Step-In” button will be enabled when there is a call to the function which is defined in a module or another test script. Clicking on “Step-In” button will enter into the module function and converts the module function to data driven. Care should be taken while converting a module function to data driven as the module function might be used in other scripts where it is not expected to be data driven.

When the playback of the script is finished, the “Save Conversion” window is displayed. All the parameters which are selected to be data driven are displayed in tables with respective window names as table-titles.

When the “Save” button is clicked, the converted data driven script and a corresponding spreadsheet containing the entered values for the respective parameters are saved. Once the data driven script is saved the user can choose to generate the data by clicking on “Generate Data” button. On clicking “Generate Data”, another window appears where user is required to configure the type, format and other attributes for each parameter for which the data is to be generated. (You will get to see another post explaining about “Data Generation” very soon.)

After configuring the parameters, when the “Generate” button is clicked, data for the required parameters is generated and saved into the spreadsheet which was created earlier while saving the data driven script.

Checklist for test case:

Checklist will be a list of predefined conditions that should be satisfied by the test application. A Checklist can be inserted into the script while recording and can be verified for these conditions when the script is played. If any Checklist conditions are not satisfied, then the Status of the particular checklist item in the Checklist can be changed and a comment can be added if needed. A screenshot of the AUT can also be added. Test reports are generated in HTML and XML formats in the set location (<Marathon_Project_Directory>/Test Reports).

Recording using “New Test Case”:

Once the recording is started, Marathon main window will be minimized and the “Marathon Control Center” window will be displayed. AUT also starts and the UI of it appears.

While recording, a checklist can be inserted by clicking on the “Insert Checklist” button in the “Marathon Control Center” window. Once the Insert Checklist button is clicked, “Select a Checklist” window will be displayed as shown below.

A list of available Check Lists will be displayed in the “Select a Checklist” window. Any of these Check Lists can be inserted to the script by selecting the Check List and clicking on the “Insert” button.

Creating new Check List:

A new Checklist can be created by clicking “New” button in the “Select a Checklist” window. On clicking New button, “Create a New Checklist” window will be displayed prompting to enter “Name” and “Description”.

“Create New Checklist” window has three tabs. First tab is Header clicking on which, a window “New Header” will be displayed to enter the header.

Second Tab is “Checklist” which will create a Checklist Item to Check List. Clicking on “Checklist” tab will open a window “New Checklist item” in which the Label for the checklist item has to be entered.

Third Tab is “Textbox” which will create Textbox to Check List. Clicking on Textbox will open a similar window as above where Textbox name can be entered.

Once all these fields have been filled click on “Save” button on “Create a New Checklist” window. Filename for Checklist needs to be entered for saving it. This creates a new Checklist and the created Checklist will be visible in “Check Lists” column of “Select a checklist” window as shown in the below image .

Playing Scripts:

To Enable Checklists while playing, click on “Enable Checklists” item in “Marathon” in the Marathon main Window.

While playing the recorded script, when the inserted checklist is reached, the CheckList window will be displayed as below.

In this window the Status of checklist Item can be changed and comments can be added. Screenshot of AUT can also be added by clicking “Screen Capture” button. After clicking the “Screen Capture” Button screenshot of AUT will be generated. In screen shot a required part can be highlighted and comments can be added as “Annotations” for the highlighted part.

Click on “Save” button to save the added annotations and/or comments in the respective windows, Playback of the script continues after saving. The annotations and comments added will be inserted in the reports generated.

Checklist for “Exploratory Test Case”:

While recording an Exploratory test case a checklist can be inserted in the similar way said above (by clicking on the “Insert Checklist” button in Marathon Control Center). The status of the Checklist item can be changed and comments and/or screenshot can be added if needed.

While Playing, execution of script will halt once it reaches the inserted check list and “Check List” window will be displayed. This will show the Checklist Item State, Screenshot if added while recording can be viewed by clicking on “Screen Capture” button.

Change in project settings:

Update the Python Home under Python path in Marathon Project Settings to point the installation directory of Jython version 2.5.

Change in test case:

A simple example test case recorded for SampleApp looks as follows:

def test():
    java_recorded_version = '1.5.0_24'

    if window('Simple Widgets'):
        select('First Name', 'TestName')
        select('Password', 'P@ssword')
        select('Email', 'TestName@Email.com')
        select('Address', 'Address1, Line1')
        click('save')

        if window('Simple Widgets(1)'):
            click('OK')
        close()
    close()

Assuming the data to be fetched is in “SampleAppTestData.csv”, modify the test script to fetch the data from the csv as below.

import csv

def test():
    java_recorded_version = '1.5.0_24'

    reader=csv.DictReader(file("<PATH_TO_CSV>/SampleAppTestData.csv"))

    if window('Simple Widgets'):
        for record in reader:
            select('First Name', record["Name"])
            select('Password', record["Password"])
            select('Email', record["EmailID"])
            select('Address', record["Location"])
            click('save')

            if window('Simple Widgets(1)'):
                click('OK')
            close()
    close()

Note that Name, Password, EmailID, Location in record["<keyword>"] are the headers in the csv.
CSV file used in example looks as following:

Name,Password,EmailID,Location
TestName1,P@ssword1,TestName1@Email.com,"Address1, Line1"
TestName2,P@ssword2,TestName2@Email.com,"Address2, Line2"
TestName3,P@ssword3,TestName3@Email.com,"Address3, Line3"
TestName4,P@ssword4,TestName4@Email.com,"Address4, Line4"
TestName5,P@ssword5,TestName5@Email.com,"Address5, Line5"
TestName6,P@ssword6,TestName6@Email.com,"Address6, Line6"
TestName7,P@ssword7,TestName7@Email.com,"Address7, Line7"
TestName8,P@ssword8,TestName8@Email.com,"Address8, Line8"
TestName9,P@ssword9,TestName9@Email.com,"Address9, Line9"
TestName10,P@ssword10,TestName10@Email.com,"Address10, Line10"
TestName11,P@ssword11,TestName11@Email.com,"Address11, Line11"
TestName12,P@ssword12,TestName12@Email.com,"Address12, Line12"
TestName13,P@ssword13,TestName13@Email.com,"Address13, Line13"
TestName14,P@ssword14,TestName14@Email.com,"Address14, Line14"
TestName15,P@ssword15,TestName15@Email.com,"Address15, Line15"
TestName16,P@ssword16,TestName16@Email.com,"Address16, Line16"
TestName17,P@ssword17,TestName17@Email.com,"Address17, Line17"
TestName18,P@ssword18,TestName18@Email.com,"Address18, Line18"

We are planning to post screencasts that demonstrate various aspects of Marathon. As part of the series, the first screencast is up on YouTube.

Creating A Project

This screencast walks you through creating a Marathon project.

We hope you enjoy it. You can discuss the screencast on marathon-testing group.

Now Marathon supports some additional features:

1. You can create semi automatic test cases, by inserting checklists into the test scripts.
2. There is an exploratory test mode. This mode is similar to recording, except that you can insert and fill the checklists at the same time.
3. New assertion window. When the mouse/key trigger is pressed, the component is highlighted. While the window is open, you can click on different components and insert assertions.
4. While we are at the assertion window, you can also insert a wait. The wait_p works just like assert_p, but waits for the value of the property to change to the expected value. This should make usage of sleep pretty rare indeed.

But the major changes are actually internal.

1. Using dispatch thread to invoke the operations on components.
2. Dependence on jaccess has been reduced. In a later version, I hope to remove the total dependency.
3. Marathon doesn’t override the EventQueue any more. We use AWTEventListener to listen to mouse/keyboard triggers.

The bundled versions of JRuby and Jython are 1.5 and 2.2.1 respectively. If you want to use a different version, you can set the home directory in the project settings.

The command line invocation format changed. I hope the new one is simpler and straight forward. Use marathon -help to look at the options.

Thanks Santhosh for patiently testing this version over the last two weeks. He has been using jgnash as the test application.

Lastly, the users guide has been updated, though not as extensively as I hoped. Still, tomorrow is always there for the updates.

Post your feedback to marathontesting google group.

Suitably impressed (and also inspired) by Using FEST-Swing to discover (via testing) what’s under the hood of JavaFX - I decided to give JavaFX a try with Marathon. This is the way I went about it:

1. Since Marathon can’t (yet) support any JavaFX controls, I searched around and found some examples that purely use swing components from here. They are pretty simple, since I don’t know JavaFX - I can’t make any thing more complicated work.
2. Next is to find the java command line used by javafx when an application is launched. Atleast on OSX it is a shell script. I just modified it and added an ‘echo’ to print out the command line.
3. From the command line, I get a ton of jars.
4. Next is to create a Marathon project. That is simple enough. I set all these jars into the class path and give the main class and save the project.
5. Record a test case - and it works!!!

What does this mean? If you use only Swing components and develop a JavaFX application, Marathon can be used to test it. But, whatever I have seen on the net - it looks like there is going to be proliferation of JavaFX controls. At a quick glance, it looks like Marathon should at least handle SGControl’s for it to be more useful. I will have a look into it soon.

Meanwhile, if anyone can point me to some good JavaFX/Swing application - I would like to give it a try.

Checklists

Checklists are everywhere. The GTD crowd lives by them. A checklist is a tool in testers arsenal when used well makes mundane tasks easy to handle, complex tasks more structured and overall provides systematic way of handling testing.

Marathon Checklists

Marathon brings the goodness of checklists to the automation - making it possible to create semi automatic test scripts.
Marathon checklist is a form containing either checkbox items (along with an explanatory text area) or plain text areas. The forms are stored as plain XML files. These can be created through an (rudimentary) UI while recording or using you favorite text editor.
While recording a test script, you can insert a checklist action. You will be prompted with the existing forms and you can either select a predefined form or create a new one.
While playing a test script with a checklist action, the user will be presented with the form. The form will have options to succeed or fail the test script. In both the cases, the results are written into a XML file. At the end of test run, you can create a checklist report that formats the results and display either in text or html format.
The checklist actions by default are ignored during batch run (from commandline or JUnit view). You can enable the checklist actions by passing a command line parameter (for batch run) or a menu option (for JUnit run).

Exploratory Testing With Marathon

Having the checklists is good and we can make it better by adding an exploratory test mode. The exploratory test mode is similar to recording mode (may be the same option) - where the application is started and the script is recorded. But the recorded script is not used for playback. It is used to provide the step by step operations for reproducing a bug. During this mode, you have an option of adding a test failure. When using this option, you can either provide free form text or use an existing checklist. Marathon will store the results including the recorded part of the script in a XML file. A report option provides the Text or HTML view of the results.
Another feature of exploratory testing mode - for a later release - is the ability to add a window screen shot and making annotation markings on the screenshot image. This screenshot will also be added to the results.

How does these features sound? Do you think these will be useful? Please leave feedback in the comments.