In this section:

Overview

The test impact analysis (TIA) functionality analyzes the coverage data for the application under test (AUT) and determines which tests have been affected by changes since the previous build. The following overview describes the test impact analysis process: 

1. Configure the coverage agent (agent.jar) shipped with SOAtest and attach it to your AUT.
2. Configure your settings to enable SOAtest to communicate with the agent for creating the baseline coverage report.
3. Run your full test suite with the new configuration to collect information about what test cases cover and generate the baseline coverage report.
4. Run the tests affected by change. Run a job that executes SOAtest and passes the baseline coverage report through the -impactedTests argument to run the tests affected by change. This is typically done as part of a CI/CD pipeline for quick validation of code changes.

Configuration

TIA is intended to be implemented as part of an automated process. Perform the following steps to enable TIA.

Deploying Your Application

Deploy your application under test. An application packaged in any zip-based file format, including zip, war, jar, and ear, is supported, as is a hierarchical directory structure that contains standalone class files or class files embedded within a zip-based file.  

Attaching the Coverage Agent to the AUT

SOAtest includes a Java agent that generates the coverage information necessary for SOAtest to determine which tests are affected by changes. 

The agent is shipped in the <INSTALL>/coverage directory. It takes configuration settings from the agent.properties file in the same directory. You should copy the coverage directory that contains the agent.jar and agent.properties files to the machine where the AUT is running.

Configuring the Coverage Agent

See Application Coverage for details.

Create a Baseline Coverage Report

Configure the appropriate application coverage settings and run your full test suite to collect the data and generate the baseline coverage report:

soatestcli.exe -data <Your Workspace> -resource <Your Tests> -settings <Properties File with SOAtest Settings> -config <Your Team's Test Configuration> 

If you already have an automated test run, you can point to its properties file in the -settings argument.

See Testing from the Command Line Interface - soatestcli for details about building test execution commands with SOAtest. You can also manually run tests from the SOAtest GUI.

Executing Tests Impacted by Change

When a new version of the application under test is available, you can run a test impact analysis. To execute only the tests reported by TIA, use the -impactedTests argument, which limit the tests that are run to only those impacted by changes in the AUT:

soatestcli.exe -data <Your Workspace> -impactedTests <Path to Baseline Coverage Report> -settings <Properties File with SOAtest Settings> -config <Your Team's Test Configuration> 

Optional filters:

• -include: Specifies tests that should always be run regardless of whether or not they are part of -impactedTests.
• -exclude: Specifies tests that should not be run regardless of whether or not they are part of -impactedTests.

When using the -impactedTests argument, be sure to pay attention to the settings below. See application coverage settings for more information about these settings. In the example above, these would be configured in the SOAtest settings file specified by the -settings argument.

• application.coverage.binaries: Set to the binaries for the modified version of the application under test.
• application.coverage.binaries.include: Optional, but recommended. If neither this nor application.coverage.binaries.exclude is configured, analysis of binaries will take longer and analyze more of the application than needed.
• application.coverage.binaries.exclude: Optional, but recommended. If neither this nor application.coverage.binaries.include is configured, analysis of binaries will take longer and analyze more of the application than needed.

See Testing from the Command Line Interface - soatestcli for details about building test execution commands with SOAtest.