The test impact analysis (TIA) functionality analyzes the coverage data for your application under test and outputs a coverage report that identifies which tests cover which code in the application. You can process the coverage report using the Selenic plug-in for Maven and run the subset of your Selenium tests that have been affected by changes to the application. In this section:

Overview

Test impact analysis is intended to be implemented as part of an automated process. The following overview describes the test impact analysis process: 

1. Configuration: Configure the coverage agent (agent.jar) shipped with Selenic and attach it to your AUT. Integrate the Selenic plug-in for Maven into your project.  
2. Collect information about what test cases cover: Run your Selenium tests so that the agent can collect data about the tests and the code they cover.
3. Generate a baseline coverage report: Run the TestImpactAnalysis script to process the data collected by the agent to create a baseline coverage report.  
4. Run the tests affected by change: When a new version of the application is available, deploy it to the server and run your Selenium tests using the Selenic plug-in for Maven. Specify the baseline coverage file during execution and only the tests affected by change will run. 

Configuration

Perform the following steps to enable TIA.

Package your Application

Package your application under test into a deployable .war file.  

Configuring the Selenic Plug-in for Maven

Add the plug-in to your project pom.xml file. The following example shows how to add the dependency for Selenic version 2020.1, but you should use the current version in your build file.

<build>
	<plugins>
		<plugin>
			<groupId>com.parasoft.selenic</groupId>
			<artifactId>selenic-maven-plugin</artifactId>
			<version>2020.1.0-SNAPSHOT</version>
		</plugin>
	</plugins>
</build>

Add the selenic-maven-plugin profile and repository to your Maven settings file. The settings may be configured globally or locally. By default, the global settings.xml file is in the <MAVEN_INSTALL>/conf/ directory. The local settings.xml file is in your <USER_HOME>/.m2 directory. If the local file does not exist, copy the settings.xml file from the global directory to the local directory and add the following <profiles> element to the <settings> element:

<profiles>
	<profile>
		<id>selenic-maven-plugin</id>
		<activation>
			  <activeByDefault>true</activeByDefault>
		</activation>
		<pluginRepositories>
			<pluginRepository>
				 <id>selenic-local</id>
				 <url>file:/path/to/selenic/test_impact_analysis/integration/maven</url>
			</pluginRepository>
		</pluginRepositories>
	</profile>
</profiles>

Configuring the TIA_OPTS Environment Variable

The Selenic plug-in for Maven includes an environment variable for passing parameters to the Java VM when executing the tests identified by test impact analysis (see Executing Tests Affected by Change). Add the TIA_OPTS variable to the machine running Maven and specify any JVM parameters you want to include. For example, you could increase the available memory when running TIA to 2GB:

TIA_OPTS=-Xmx2g 

Attaching the Coverage Agent to the AUT

Selenic includes a Java agent that generates the coverage information necessary to determine which tests are affected by changes. 
The agent is shipped in the <INSTALL>/test_impact_analysis/integration/coverage directory. It takes configuration settings from the agent.properties file in the same directory. You should copy the coverage directory that contains the agent.jar and agent.properties files to the machine where the AUT is running.

Configuring the Coverage Agent

Application servers usually contain more than one application. Additionally, common server classes or application libraries do not need to be instrumented. The agent only needs to collect coverage for application source code. Instrumenting all classes would be too time-consuming. For this reason, properly setting the scope of the coverage agent is very important.

You can configure the coverage agent by modifying the properties in the agent.properties and passing the properties to the -javaagent argument. The agent supports several parameters (see Coverage Agent Parameters), but configuring the following settings is suitable for most cases:

jtest.agent.serverEnabled=true
jtest.agent.includes=com/myapp/data,com/myapp/common/**
jtest.agent.excludes=com/myapp/transport/,com/myapp/autogen/*
jtest.agent.associateTestsWithCoverage=true
jtest.agent.collectTestCoverage=true
jtest.agent.autostart=false

Enabling Multi-session Mode

You can enable multi-session mode if multiple users or test processes access the application while the tests are running. This is so that code is executed by the automated tests and code executed by other users or test processes are distinguishable. In multi-session mode,  a user ID is associated with the coverage data associated with distinct users or processes. The user ID is specified with the coverageAgentUserID argument during execution (see coverageAgentUserID for details).

Add the following option to the agent.properties file to enable multi-session mode:

jtest.agent.enableMultiuserCoverage=true

See Collecting Test and Coverage Data in Multi-session Mode for additional details.

Coverage Agent Parameters

The following table describes all properties that can be set for the agent:

When the properties are configured, add a -javaagent  argument when starting your application server to attach the agent and include the agent configuration file:  

-javaagent:'/path/to/agent.jar'=settings='/path/to/agent.properties',runtimeData='/path/to/runtime_coverage'

For your convenience, the coverage directory includes a script that will generate the -javaagent arguments. Run either the agent.sh or agent.bat script and copy the output to your server startup script.

$ ./agent.sh 
Add this Java VM args to your process:
---------------------------------------------------
-javaagent:"/home/selenic/test_impact_analysis/integration/coverage/agent.jar"=settings="/home/selenic/test_impact_analysis/integration/coverage/agent.properties",runtimeData="/home/selenic/test_impact_analysis/integration/coverage/runtime_coverage"
---------------------------------------------------
Press any key to continue . . .  

In the following example, the agent is attached to a Tomcat server with a JAVA_OPTS variable at the beginning of the catalina.sh (Linux) or catalina.bat (Windows) scripts:

if [ "$1" = "start" -o "$1" = "run" ]; then 
JAVA_OPTS='-javaagent:"/home/selenic/test_impact_analysis/integration/coverage/agent.jar"=settings="/home/selenic/test_impact_analysis/integration/coverage/agent.properties",runtimeData="/home/selenic/coverage_storage"' 
fi

if "%1"=="stop" goto skip_instrumentation 
set JAVA_OPTS=-javaagent:"C:\selenic\test_impact_analysis\integration\coverage\agent.jar"=settings="C:\selenic\test_impact_analysis\integration\coverage\agent.properties",runtimeData="C:\selenic\coverage_storage" 
:skip_instrumentation

Verifying that the Agent is Configured

Start the application and verify that the agent is ready by opening <host>:8050/status  in your browser. You should see a JSON object that contains test, session, and testCase properties, e.g.:

{"test":null,"session":"runtime_coverage_20191008_1537_0","testCase":null}

You can also check the directory you specified with the runtimeData property (/home/TIA/coverage_storage in the example above). The directory should contain a set of coverage data files. The files are generated when the agent is started.

Collecting Test and Coverage Data

Run your Selenium tests using the Selenic agent. The Selenic agent should be configured to collect coverage and to connect to the host and port where the AUT and coverage agent are running. You should also specify a local directory for downloading the coverage information from the server:

mvn test -DargLine=-javaagent:/path/to/selenic_agent.jar=collectCoverage=true,coverageAgentHost=<host_name>,coverageAgentPort=<port_number>,coverageDir=path/to/save

You can also configure the agent to save coverage data after test execution. The following table describes all parameters related to test impact analysis usage:

Refer to the Command Line chapter for additional information about Selenic agent command line arguments.   

Collecting Test and Coverage Data in Multi-session Mode

Enable multi-session mode if multiple users or test processes are concurrently accessing the application during Selenium test execution. Multi-session mode enables you to identify the that comes from the Selenium tests being executed. Perform the following actions to run in multi-session mode:

• Enable multi-session mode in the agent.properties file (see Enabling Multi-session Mode).
• Start the SOAtest Web Proxy Server before running the agent to collect coverage during full test runs. The SOAtest Web Proxy Server is deployed as part of the Parasoft Recorder installation. See Installing Parasoft Recorder for additional information.
• Specify the user ID using the coverageAgentUserID argument during execution. Example:

mvn test -DargLine=-javaagent:/path/to/selenic_agent.jar=collectCoverage=true,coverageAgentHost=<host_name>,coverageAgentPort=<port_number>,coverageAgentUserID=<userID>,proxyPort=40090,coverageDir=path/to/save"

Generating Baseline Coverage

Selenic includes a script for creating the baseline coverage report. The script analyzes the coverage data collected by the Selenium test run and the WAR file for the AUT to generate the report. Depending on your operating system, you can run either the TestImpactAnalysis.bat (Windows) or TestImpactAnalysis.sh (Linux/macOS) script located in the <INSTALL>/test_impact_analysis directory.  

#!/bin/bash 
TMP=/tmp

After the test run finishes, run the TestImpactAnalysis.sh or TestImpactAnalysis.bat script using the following syntax to generate the coverage report.

./TestImpactAnalysis.sh -app <PATH TO AUT WAR FILE> -runtimeCoverage <PATH_TO_RUNTIME_COVERAGE> -outputReport <PATH_TO_OUTPUT_REPORT_DIR> -include <SPACE_SEPARATED_PATTERNS> -exclude <SPACE_SEPARATED_PATTERNS>

• The -app   flag specifies the deployable .war. You should specify the same artifact deployed as the AUT for the automated test run (see Package your Application).
• The -runtimeCoverage  flag specifies the directory specified with the coverageDir parameter you configured in the -javaagent arguments when executing your tests (see Collecting Test and Coverage Data).
• The -outputReport  flag is optional and specifies where to output the coverage report.
• The -include  flag is optional and specifies a space-separated list of patterns to include in the analysis. By default, all test classes are included.  
• The -exclude  flag is optional and specifies a space-separated list of patterns to exclude in the analysis. Classes that match patterns specified with the -include flag override excluded patterns.
• Add the -keepRuntimeCoverage  option if you want to keep the runtime coverage data files used to generate the baseline coverage XML report. This parameter does not take a value. If it is not included in the command, the runtime coverage data files will be removed at the end of the execution.  

Example commands:

TestImpactAnalysis.bat ^
-app C:\tomcat\webapps\application.war ^
-runtimeCoverage C:\selenic\runtime_coverage ^
-outputReport C:\selenic\reports ^
-include com\myapp\data\**,com\myapp\common\** ^
-exclude com\myapp\transport\**,com\myapp\autogen\** 

./TestImpactAnalysis.sh \
-app /home/tomcat/webapps/application.war \
-runtimeCoverage /home/selenic/runtime_coverage \
-outputReport /home/selenic/reports \
-include com/myapp/data/**,com/myapp/common/** \
-exclude com/myapp/transport/**,com/myapp/autogen/** 

When the script finishes, the coverage.xml report will be saved to the location specified with the -outputReport  flag. If the flag is not included, the file will be saved to the <USER_HOME>/parasoft/test_impact_analysis/reports/ directory by default.

Executing Tests Affected by Change

When a new version of the application is available, you can execute your tests with the Selenic plug-in for Maven and specify the coverage file, deployed WAR file, and path to the Selenic home directory to execute only the tests reported by TIA:

mvn selenic:affected-tests test -Dapp=/path/to/new/parabank.war -DcoverageReport=/path/to/coverage.xml -DselenicHome=/path/to/selenic

Any JVM arguments you configured with the TIA_OPTS will also be used (see Configuring the TIA_OPT Environment Variable).