This topic explains how to customize a test suite’s properties (such as its name and how individual test cases are run). In this section:

Accessing the Test Suite Configuration Panel

To customize test suite properties, double-click its Test Case Explorer node and use the controls that open in the configuration panel (on the right side of the GUI).

Associating Comments, Requirements, Tasks, Defects, and Feature Requests with Tests

The Requirements and Notes tab of the test suite configuration panel allows you to associate requirements, defects, tasks, feature requests, and comments with a particular test in the test suite.

The HTML report will indicate the artifacts that are associated with each referenced test. For example, here is a report that references a test with an associated comment:

The requirements you define will appear in Structure reports (and also in Report Center for Parasoft Concerto users), allowing managers and reviewers to determine whether the specified test requirements were accomplished. For more information on Structure Reports, see Creating a Report of the Test Suite Structure.

Adding Associations and Comments

To configure associated artifacts for tracking:

  1. Select the appropriate test node from the test suite tree within the Requirements and Notes tab.
  2. To indicate defects, feature requests, requirements, or tasks that are associated with the selected test
    1. Click the Add button.
    2. In the Type box, select a requirement type. Parasoft Concerto will use this information to associate the test suite’s test cases to the specified element type. For instance, if it is associated with a specific bug, information about the test case’s status will be considered for Report Center’s bugs graphs. Custom tags can be added as described in Indicating Code and Test Correlations. Default tags are:
      • @pr: for defects.
      • @fr: for feature requests.
      • @req: for requirements.
      • @task: for tasks.

        You can configure defect/issue tracking tags to match the language that your organization uses to refer to defects or to feature requests. For details, see Indicating Code and Test Correlations.
    3. Enter an ID and a URL for the requirement and click OK.

      If you enable the Preferences> Reports> Report contents option for Requirement/defect details, associations specified here will be shown in the HTML report. If a URL is specified, the HTML report will include hyperlinks.
  3. To add a comment (e.g.,  "this test partially tests the requirement" or "this test fully tests the requirement"), enter it into the Comment field. Comments specified here will be visible in HTML reports.

  4. To add more detailed notes for the test suite, enter them into the Notes field.

Specifying Execution Options (Test Flow Logic, Regression Options, etc.)

Configurable execution options allow you to control factors, such as whether:

These options are configured in the Execution Options tab, which has three sub-tabs: Test Execution, Test Flow Logic, and Regression Options.

Test Execution

You can customize the following options in the Test Execution sub-tab of the Execution Options:

Test Flow Logic

SOAtest allows you to create tests that are dependent on the success or failure of previous tests, setup tests, or tear-down tests. This helps you create an efficient workflow within the test suite. In addition, you can also influence test suite logic by creating while loops and if/else statements that depend on the value of a variable.

Options can be set at the test suite level (options that apply to all tests in the test suite), or for specific tests.

Test Suite Logic Options

In many cases, you may want to have SOAtest repeatedly perform a certain action until a certain condition is met. Test suite flow logic allows you to configure this.

Understanding the Options

To help you automate testing for such scenarios, SOAtest allows you to choose between two main test flow types:

For example:

Setting the Options

To configure test flow logic options that apply across the test suite:

  1. Open the Execution Options> Test Flow Logic tab, then select the top-level node. .

  2. Select the desired flow type.
  3. (Optional) Customize the Maximum number of loops setting, which determines the maximum number of loops to run if the specified condition is never met.
  4. If you chose while/pass fail flow specify the loop conditions by going to Loop until, choosing either Every test or One test, then choosing succeeds or fails—depending on which outcome you want to occur before the test suite proceeds.

  5. If you chose while variable flow, set the while and do conditions as follows.

For a step-by-step demonstration of how to apply while pass/fail test flow logic, see Looping Until a Test Succeeds or Fails - Using Test Flow Logic.

Test-Specific Logic Options

The following options are available for specific tests:

 If any set-up or tear-down tests are available, they display in the left GUI panel and you will be able to configure test logic as follows:

  • The execution of a test can be dependent on a set-up test.
  • A setup test can now be dependent on a previous set-up test.
  • A tear-down test can be dependent on a regular test, set-up test, or previous tear-down test.

This functionality allows you to stop a test (or run a test) if a setup test fails.

Regression Options

The Regression Options controls options allow you to customize how data sources are used in regression tests and which test suite have regression controls. Note that this tab is not applicable for web scenarios tests. Available options are:

If a new row is inserted into or deleted from the data source, all multiple regression controls associated with that data source must be updated.

Defining Variables

The Variables tab allows you to configure variables that can be used to simplify test definition and create flexible, reusable test suites. After a variable is added, tests can parameterize against that variable.

Understanding Variables

You can set a variable to specific value, then use that variable throughout the current test suite to reference that value. This way, you don’t need to enter the same value multiple times—and if you want to modify the value, you only need to change it in one place.

As an alternative to manually setting a variable to a specific value, you can have a data bank tool (e.g., XML Data Bank) or Extension tool set the value of that variable "on-the-fly."

Moreover, if you have a referenced test suite (a test suite that is referenced by a parent test suite—see Using Test Suite References for details), variables can be used to access data sources from the parent test suite.

Variables are available as parameterized values within tool configuration panels (using the ${env_name} convention), as well as through scripting with com.parasoft.api.ExtensionToolContext.getValue(String) and setValue(String, String).  Values are of type string, integer, boolean.

During test execution, variables that are defined in a test suite are visible to all tests within that test suite and any tests that are executed after that test suite—regardless of whether the tests are in the same test suite or a child suite or parent suite. For example, assume that you have the following test suite structure and are executing tests in order:

Any test variables that are defined in test suite A will be visible to any tests within A, B, or C; variables defined in B will be visible only to B and C.

Adding New Variables

  1. Click the Add button.
  2. Enter a new variable name in the Name field.
  3. Select either Integer, Boolean, String, or Data Source from the Type box.
  4. Specify whether you want to use a local value or use a value from a parent test suite.
  5. (For data source type only) Specify the name of the data source and column where the appropriate variables are stored.  The data source should be in the parent test suite (the test suite that references the current test suite).
  6. Enter the variable value in the Value field. If you chose Use local value, the variable will always be set to the specified value (unless it is reset from a data bank tool or Extension tool). If you chose Use value from parent test suite, the value specified here will be used only if a corresponding value is not found in the parent test suite.

  7. Click OK.

Using Variables

Once added, variables can be:

Setting Variables and Logic Through Scripting

Very often, test suite logic and variables will depend on responses from the service itself. Using an Extension tool, you can set a variable in order to influence test flow execution. For example, if Test 1 returns a variable of x=3, then Test 2 will run.

Via the TestSuiteVariable API, you can access a variable and either set it to a value, or get a value from it.  Using this value, you can configure test flow logic.  See com.parasoft.api Interface TestSuiteVariable in the Extensibility API Javadocs for more information. The Javadocs can be accessed by choosing Parasoft> Help, then opening the Parasoft SOAtest Extensibility API book. 

For example, you can enter the following into an Extension tool to set a variable:

from com.parasoft.api import Application

def setVar(input, context):
	context.setValue("x", input.toString())

To get a value from a TestSuiteVariable object x:

varValue = context.getValue("x")

Where varValue will be returned as a string.

For instance, you can add an XML Transformer tool to a test and extract a certain value from that test. Then, you can add an Extension output to the XML Transformer and enter a script to get the value from the Transformer. Finally, you can set up a second test to run only if the correct value is returned from the first test.

Monitoring Variable Usage

To configure SOAtest to show what variables are actually used at runtime, set Console preferences (Parasoft> Preferences> Parasoft> Console) to use normal or high verbosity levels.

After each test, the Console view (Show View> Parasoft> Console) will then display variables used at runtime. For example:

Scenario: ICalculator
    Test 1: first add - success
        get x=0
        set x=10.0
        set Test 1: type=xsd:float
    Test 2: second add - success
        get x=10
        set x=20.0
    Test 3: third add - success
        get x=20      
        set x=30.0
    Test 1: first add - success
        get x=30
        set x=50.0
        set Test 1: type=xsd:float
    Test 2: second add - success
        get x=50
        set x=70.0
    Test 3: third add - success
        get x=70      
        set x=90.0

Viewing such variables is useful for diagnosing the cause of any issues that occur.


For a step-by-step demonstration of how to use variables, see Creating Reusable (Modular) Test Suites.

Specifying SOAP Client Options

You can customize the following options in the SOAP Client Options tab of the test suite configuration panel:

Specifying Browser Playback Options

The Browser Playback Options tab is divided into several sections:

The above options do not apply to tests run from the command line interface.

In command line mode (using soatestcli), SOAtest runs web scenarios in headless mode by default. If you do NOT want to run in headless mode from the cli, use the  -browserTestsVisible command with soatestcli (described in Testing from the Command Line Interface - soatestcli).

Authentication Settings

Basic, NTLM, Digest, and Kerberos authentication schemes are supported, and can be specified in this panel. You can enter a username and password for Basic, NTLM, and Digest authentication, and a service principal for Kerberos authentication.

There are three main authentication options:

To specify when the authentication specified for this test suite should be applied, indicate the path or realm that expects authentication. If only one server requires authentication in a scenario that accesses many servers, you can specify a path or realm used by the server. Then, authentication will be applied only to requests that match that path or realm. For example, if you enter the path, this will cause the supplied authentication to be used only for any request that starts with "".

If you want the authentication apply to every request, leave the path and realm empty.

You can enter multiple authentication credentials for scenarios that access many servers which  authenticate in different ways. SOAtest will use the path and realm to determine when the authentication scheme should be applied to a request.

For this scenario, the Digest authentication will be applied to any requests that start with The Kerberos authentication will be applied to any requests that start with

Note that SOAtest will attempt to use the authentication information specified in the SOAtest Security Preferences when it first records a scenario. If authentication is used and succeeds during recording, the authentication information will be recorded in the Browser Playback Options.