Execution Runner - User Interface

The execution runner is used for navigating through the tests of an execution and setting their status and the status of their test steps.

The execution runner always displays one test at a time. For each test the following information is displayed:

Name of the test
Test description
A list of the test steps of the test.
For each test step name, description and expected result are displayed.

Execution Runner - Operations


Navigate through contained tests	Use the icons located in the top right corner to navigate through the tests of the execution. The tests are presented in the sequence specified in the plan tree of the corresponding test plan. Although you can navigate through the tests, they still have to be executed in the given sequence.
Set the status of test steps	Use the buttons next to a test step to set its status. The test steps are presented in the sequence specified in the definition of the corresponding test and have to be executed in this sequence. You can choose from one of the following status: (Step passed) The test step was executed successfully and the next test step can be executed. (Step failed) The execution of the test step failed. The test step and the corresponding test case is marked as failed. The execution is completed and marked as failed. All remaining test cases and test steps are marked as skipped. (Step failed and continue with next test) The execution of the test step failed. The test step and the corresponding test case is marked as failed. Remaining test steps in the test case are marked as skipped and the execution moves on to the next test case. Although the execution continues its status will be set to failed. (Step failed and continue with next step) The execution of the test step failed and the execution moves on to the next test step. Although the execution continues its status will be set to failed.
Add a test step result	Use the button next to a test step to enter a description of the test step result. Adding a test step result is optional if the test step was marked as passed since the expected result should be already specified. When a test step is marked as "failed" or "failed continue" a test step result is required.
Set the status of tests	Use the buttons in the bottom left corner to set the status of a complete test. You can choose from one of the following status: (Test passed) The test was executed successfully and the next test can be executed. The test and all of its test steps are marked passed. (Test failed) The execution of the test failed. The test is marked as failed and its remaining test steps are marked as skipped. The execution is completed and marked as failed. All remaining tests and test steps are marked as skipped. (Test failed and continue) The execution of the test failed. The test is marked as failed and its remaining test steps are marked as skipped. The execution continues with the next test. Although the execution continues its status will be set to failed.
Add a test result	Use the Test Result input field to specify a test result. Adding a test result is optional if the test was marked as passed. When a test is marked as "failed" or "failed continue" a test result is required.
Add an attachment	Add the bottom of the dialog is a drop area for attachments. To add an attachment, drag an attachment to this area or click on the area to open Windows File Explorer to choose attachments.
Stop the execution	Click Cancel to interrupt the current execution and close the execution runner. The state of the execution will be saved and the execution can be resumed later.
Show execution statistics	If the execution was completed click on the Show execution statistics button to display the execution results. An execution is completed as soon as all tests and test steps are marked as passed or a single test or test step is marked as failed.

Execution Statistics - User Interface

The execution statistics form summarizes the current state of an execution.
It is separated into an Execution Overview section and an Execution details sections.

The Execution Overview displays the amount of passed, skipped and failed tests and test steps for the selected execution and the corresponding percentages.

The Execution Details section displays all tests of the execution and their test steps and the corresponding status.
For each test and test step the following information is displayed:

Issue key of the test (only for contained tests)
Issue summary of the test or name of the test step
Description of the test / test step
Execution result
Status
User who executed the test / test step
Completion date

A test / test step can have one of the following status:

Not Run
Passed
Failed
Skipped

Execution Statistics - Operations

Add issue link

Switch to the Execution Details section and click on the next to a failed test or test step to open the add issue input.

This allows you to create and link a new issue to cover the failed test.
Select a link type and click Create issue to open the JIRA create issue dialog.
Once the new issue is created it will be linked to the failed test on the left hand column.

Export

Switch to the Execution Details section and click on the Export button to open the export menu.
Select an export format from the list you want to export the execution statistics to.
The executions can be exported as XML or Excel file.

The following table describes the export content for each format.

Export format

Content

XML

The resulting file contains detailed information about the execution and the executed tests and their test steps.
The XML export also provides information such as item ids that can be used to integrate another application with T4J via its REST API.
An example XML export can be found below this table.

Excel

The resulting file contains the following information about the execution and the executed tests:

Information about the execution
- Execution status
- Name of the execution
- Creation date, start date and execution date
- Description of the execution
Information about executed tests / test steps
- Issue key of the test (only for executed tests)
- Issue summary of the test or name of the test step
- Execution result
- Status of the test / test step
- The user who executed the test / test step
- The completion date

Example XML export

<?xml version="1.0" encoding="UTF-8"?>
    <result>
        <id>52</id>
        <name>Example execution</name>
        <description>An example execution.</description>
        <deleted>false</deleted>
        <assignee>Admin</assignee>
        <passed>false</passed>
        <failed>true</failed>
        <running>false</running>
        <notrun>false</notrun>
        <created>2016-10-28T12:37:26.163+02:00</created>
        <started>2016-10-28T12:37:45.497+02:00</started>
        <finished>2016-10-28T12:38:16.727+02:00</finished>
        <tests>
            <name>Example Test</name>
            <description>An example test</description>
            <descriptionRendered>&lt;p&gt;An example test&lt;/p&gt;</descriptionRendered>
            <failureMessage/>
            <folders>Subfolder</folders>
            <failed>true</failed>
            <passed>false</passed>
            <notrun>false</notrun>
            <skipped>false</skipped>
            <steps>
                <name>Test Step 1</name>
                <description/>
                <failureMessage>Test step failed.</failureMessage>
                <failed>true</failed>
                <passed>false</passed>
                <notrun>false</notrun>
                <skipped>false</skipped>
                <expectedResult/>
                <run>2016-10-28T12:37:45.497+02:00</run>
                <id>547</id>
                <executedBy>Admin</executedBy>
                <executedByUrl>http://host:port/jira/secure/ViewProfile.jspa?name=admin</executedByUrl>
            </steps>
            <steps>
                <name>Test Step 2</name>
                <description/>
                <failureMessage/>
                <failed>false</failed>
                <passed>false</passed>
                <notrun>false</notrun>
                <skipped>true</skipped>
                <expectedResult/>
                <run>2016-10-28T12:37:45.497+02:00</run>
                <id>548</id>
            </steps>
            <finished>2016-10-28T12:37:45.497+02:00</finished>
            <id>207</id>
            <executedBy>Admin</executedBy>
            <executedByUrl>http://host:port/jira/secure/ViewProfile.jspa?name=admin</executedByUrl>
            <issueUrl>http://host:port/jira/browse/PRJ-19</issueUrl>
            <issueKey>PRJ-19</issueKey>
            <issueStatus>To Do</issueStatus>
            <issueType>New Feature</issueType>
        </tests>
        <tests>
            <name>Example Test 2</name>
            <description>A test without test steps.</description>
            <descriptionRendered>&lt;p&gt;A test without test steps.&lt;/p&gt;</descriptionRendered>
            <failureMessage>Test failed</failureMessage>
            <folders/>
            <failed>true</failed>
            <passed>false</passed>
            <notrun>false</notrun>
            <skipped>false</skipped>
            <finished>2016-10-28T12:38:16.727+02:00</finished>
            <id>209</id>
            <executedBy>Admin</executedBy>
            <executedByUrl>http://host:port/jira/secure/ViewProfile.jspa?name=admin</executedByUrl>
            <issueUrl>http://host:port/jira/browse/PRJ-22</issueUrl>
            <issueKey>PRJ-22</issueKey>
            <issueStatus>To Do</issueStatus>
            <issueType>New Feature</issueType>
        </tests>
    </result>

Execution Runner & Statistics - Reference