In this section:

Introduction

You can monitor and collect coverage data during manual or automated functional tests performed on a running application. You can also send coverage data and test results to DTP, which merges and correlates the data. The application coverage information can be displayed in the DTP Coverage Explorer (see the "Coverage Explorer" chapter in the DTP user manual), which provides insights about how well the application is tested, as well as the quality of your tests.

Prerequisites

Runtime coverage can be collected for Java 1.8 - 17.

Java JDK 8 line break calculations in bytecode are inconsistent with previous Java versions. This may lead to inconsistencies in coverage results.

If you want your coverage report to also include information about the application source code, the following components are required:

Apache Maven, Gradle, or Ant build system

If you use a source control system, ensure that your source control settings are properly configured; see Source Control Settings.

The jtestcov tool requires Java 11 or higher to be installed on your machine. Both Oracle Java SE and OpenJDK are supported. Other free and open source implementations may cause issues with the proper functioning of the coverage tool. It is recommended to set JAVA_HOME and add it to the system PATH. For information about jtestcov usage, see Creating a Coverage Report and Uploading the Data to DTP and Command Line Options for jtestcov.

Process Overview

Jtest ships with a component called the coverage agent. The coverage agent is attached to the application under test (AUT) and monitors the code being executed as the AUT runs. When the coverage agent is attached to the AUT, a REST API is exposed that enables you to mark the beginning and end of each test and test session.

Metadata about the lines of code that can be covered (static coverage data) is collected either by running a dedicated test configuration as part of the application build process or by scanning the application binaries. During test execution, interactions with the coverage agent are written to a dynamic coverage map, which contains markers that specify which lines of code were touched.

Jtest processes the dynamic coverage map and static coverage data. A coverage.xml file, which contains the coverage information, is produced and sent to DTP. When DTP receives the coverage data, it is loaded into a coverage image, which is a special tag that enables you to aggregate coverage data from runs with the same build ID. The coverage image enables you to associate coverage information with specific tests.

Jtest can also collect application coverage which includes only runtime coverage and general information about the classes that have been executed without any details about the class structure other than the lines that have been covered. A simplified report is then sent and appended to existing coverage data on DTP. The coverage data can be viewed if information about class structure (static coverage data) is already available on DTP.

Test results are also sent to DTP from the tool executing the tests (i.e., SOAtest, manual tests, etc.) in a report.xml file. If the build IDs for the coverage data file and the report match, DTP is able to correlate the data and display the coverage information.

You can use either jtestcov or jtestcli to create a coverage report and upload the data to DTP. jtesctcov is a dedicated tool for coverage and offers more options in terms of collecting, merging and creating coverage reports, see Creating a Coverage Report and Uploading the Data to DTP. The jtestcov.jar file is located in the <INSTALL_DIR>/integration directory. jtestcov is a standalone tool that can be copied out from Jtest installation directory and used independently. Therefore, you can easily integrate it within your testing flow. For details about jtestcov command line options, see Command Line Options for jtestcov.

Configuring the Application Under Test for Coverage

The following steps are required to prepare the application under test (AUT):

Optional: Generate the monitor package containing the preset coverage agent configuration and static coverage file, which includes metadata about user classes, methods, and lines; see Generating the Monitor Package.
Configure the coverage agent to collect runtime coverage; see Configuring the Coverage Agent.
Attach the coverage agent to the AUT; see Attaching the Coverage Agent to the AUT.

Generating the Monitor Package

The monitor package is created during the build process by the Jtest Plugin for Maven, Gradle, or Ant. To ensure that coverage is correctly matched with the AUT's source code, generate the package using the same build system and Java version that was used to build the AUT.

The package contains the following:

static_coverage.xml - the file that contains static coverage information,
agent.jar - the Jtest Java coverage agent jar archive. The file is copied from the <INSTALL_DIR>/integration/coverage directory.
agent.properties - the agent settings file that contains scope parameters generated during the build process and other attributes,
agent.sh/agent.bat - the script that generates the Jtest Java agent VM arguments necessary for attaching the agent to the AUT process.
opentelemetry-javaagent.jar and jtest-otel-ext.jar - files used in multiuser coverage mode: the original OpenTelemetry Java agent and the Jtest extension for it. This extension collects data necessary to assign collected coverage to a particular user. The files are copied from the <INSTALL_DIR>/integration/coverage directory.

The package must be generated on the build machine that contains the source code and it can be used until the code changes.

Execute an appropriate command in the AUT's main directory to generate the package:

Maven

mvn package jtest:monitor

Gradle

gradle assemble jtest-monitor -I [INSTALL]/integration/gradle/init.gradle

Ant

ant -lib [INSTALL]/integration/ant/jtest-ant-plugin.jar -listener com.parasoft.Listener jtest-monitor

Ant requires all classes to be compiled before the monitor task is executed. Modify your project prior to the build and configure the task to ensure the correct sequence. The following example shows how the target can be configured:

<target name="jtest-monitor" depends="compile">
             <jtest:monitor/>
</target>

The monitor.zip package will be generated and placed into the build output directory. The path to the location will be printed on the console.

Configuring the Coverage Agent

Application servers usually contain more than one application. Additionally, common server classes or application libraries do not need to be instrumented. Jtest only needs to collect coverage for application source code. Instrumenting all classes would be too time-consuming.

The application on the server is already built, so we cannot gather information about which classes come from the source code. For this reason, properly setting the scope of the coverage agent is very important.

You can configure the coverage agent with:

the agent.properties file
arguments to -javaagent when attaching the coverage agent to the AUT; see Attaching the Coverage Agent to the AUT.

The agent.properties and agent.jar files are included in the <INSTALL_DIR>/integration/coverage directory. The agent.properties file contains default properties that can be modified to properly configure the coverage agent. The following example shows how the coverage agent can be configured with the agent.properties.file:

jtest.agent.autostart=false
jtest.agent.serverEnabled=true
jtest.agent.includes=com/myapp/**
jtest.agent.runtimeData=[...]

If you have generated the monitor.zip package, you can use the agent.properties file included in the package which already contains scope parameters generated during the build process and other attributes (see Generating the Monitor Package).

Adding properties to -javaagent in command line requires modifying their names by removing the "jtest.agent" prefix, for example:

-javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",autostart=true

Properties provided as arguments to -javaagent override properties configured in the agent.properties file. In the above examples, the autostart=true will override jtest.agent.autostart=false from the agent.properties file.

The following table describes all properties that can be used to configure the coverage agent:

Property	Descripton
`jtest.agent.runtimeData`	Specifies where the runtime data will be stored. The following example will create files in the in ‘C:/tmp/myapp/’ directory with `runtime_`’ as the name prefix: `'C:/tmp/myapp/runtime'`
`jtest.agent.includes`	A coma-separated list of patterns that specify classes to be instrumented. The following wildcards are supported: `` matches zero or more characters `` matches multiple directory levels In the following example, all classes from the `com.myapp.data` package and all classes from package and subpackages that start with `com.myapp.common` will be instrumented: `com/myapp/data/,com/myapp/common/**`
`jtest.agent.excludes`	Coma separated list of patterns that specify classes to be excluded from instrumentation. The following wildcards are supported: `` matches zero or more characters `` matches multiple directory levels In the following example, all classes from the `com.myapp.transport` package and all classes from package and subpackages that start with `com.myapp.autogen` will be excluded from instrumentation: `com/myapp/transport/,com/myapp/autogen/**`
`jtest.agent.autostart`	Enables/disables automatic runtime data collection; the default is `true`.
`jtest.agent.serverEnabled`	Enables/disables remote access to coverage sessions.
`jtest.agent.port`	Sets up agent communication port; the default is `8050`.
`jtest.agent.debug`	Enables/disables verbose output to console; the default is `false`.
`jtest.agent.collectTestCoverage`	Enables/disables collecting coverage information for test cases. The default value is `false`.
`jtest.agent.enableMultiuserCoverage`	Enables/disables collecting web application coverage for multiple users; the default value is `false`.
`jtest.agent.autoloadMultiuserLibs`	Enables/disables automatic loading of multiuser libraries (OpenTelemetry javaagent) when `jtest.agent.enableMultiuserCoverage` is set to true. The default value is `true`.
`jtest.agent.testId`	Specifies the initial test ID the coverage will be associated with when the agent is run.
`jtest.agent.testcaseId`	Specifies the initial test case ID the coverage will be associated with when the agent is run.
`jtest.agent.enableJacoco`	Enables/disables collecting line coverage with the Jacoco engine.
`jtest.agent.silentMode`	Enables/disables printing of the agent introduction to the console; the default value is `false`.
`jtest.agent.doNotFailOnVerification`	Enables/disables failing the instrumentation if the verification fails; the default value is `true`.

Attaching the Coverage Agent to the AUT

Optional: If you want to use the monitor.zip package:
1. Extract the contents of the package to the server machine.
2. Run the agent.sh/agent.bat script from the monitor package to generate the Jtest Java agent VM arguments. The scripts output the -javaagent VM argument to the console.
Use the -javaagent flag to connect the agent to the AUT, which will result in the runtime_coverage subdirectory:
Jtest Agent VM argument: -javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to runtime coverage folder]"
See Configuring the Coverage Agent for details about coverage agent configuration using -javaagent.
Add the -javaagent flag to the application startup script to connect the agent to the AUT. Attaching the coverage agent to the AUT exposes a REST API for controlling the agent.
Restart the AUT (see Step 5, 3. in Web Application Coverage Tutorial).
Enter the following URL into the browser bar to verify that the coverage agent is set:
[application_host]:8050/status/
If the application is properly configured, the API will return data:

Connecting with the Coverage Agent via HTTPS

By default, the coverage agent is available via the HTTP protocol. To connect via HTTPS, you need to configure the coverage agent by adding the following options to the Jtest Agent VM argument (see Attaching the Coverage Agent to the AUT):

useSSL=true
certificateStorePath=[path to the keystore file]
certificateStorePassword=[password to the keystore file]
certificateStoreType=[format of the keystore file]
certificateKeyPassword=[password to the SSL certificate]

Your Jtest Agent VM argument added to the application server’s startup script may resemble the following:

-javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to runtime coverage folder]",useSSL=true,certificateStorePath="F:\certificates\keystore.jks",certificateStorePassword=4!ij?ffQMk,certificateStoreType="JKS",certificateKeyPassword=dhW!34

This will enable the coverage agent to recognize the SSL certificate you have configured for your application under test.

There is a number of ways to configure an SSL certificate for an application. If you are using a browser, there may be an option to add your certificate to a list of trusted certificates. If you are using another Java program, you can import the certificate to the Java VM trusted certificates file or add the following Java VM arguments to your Java client command line:

-Djavax.net.ssl.trustStore=[path to the keystore file]
-Djavax.net.ssl.trustStoreType=[format of the keystore file]
-Djavax.net.ssl.trustStorePassword=[password to the keystore file]

Executing Tests to Collect Runtime Coverage

To collect runtime coverage:

You can use SOAtest to run functional tests. When tests are being performed SOAtest will collect runtime coverage, and upload the data to DTP. Refer to the SOAtest user guide for details about performing test sessions.
You can execute tests on an application with the attached coverage agent. Coverage will be collected in the runtime coverage directory. Next, you need to merge the runtime coverage with static coverage and upload the merged data to DTP. For details, see Creating a Coverage Report and Uploading the Data to DTP.

Uploading Test Results to DTP

Follow these steps to upload the test results from executed tests to DTP:

Go to Report Center in the DTP interface.
Click the gear icon and choose Report Center Settings> Additional Settings> Report Center Administration> Tools> Data Collector Upload Form (requires admin permissions).
Click Choose File and browse for the report.xml file.
Click the Upload button to upload the file to DTP.

If you use SOAtest, test results are automatically sent to DTP during test execution. See the Application Coverage section in the SOAtest user guide at https://docs.parasoft.com.

Creating a Coverage Report and Uploading the Data to DTP

You can use either the jtestcov or the jtestcli command to create a coverage report and upload the data to DTP. For details about jtestcov command line options, see Command Line Options for jtestcov.

There are 3 types of coverage report that can be generated:

A static and runtime coverage report: a report containing merged static coverage (including source code information) and runtime coverage. To create this report, run jtestcli with the -runtimeCoverage and -staticCoverage switches or jtestcov with the -runtime and -static switches.
An application coverage report: a static coverage report created by scanning application binaries and runtime coverage. To create this report, run jtestcov with the -app and -runtime switches.
A runtime coverage report: a simplified application coverage report including only runtime coverage and general information about the classes that have been executed without any details about the class structure other than the lines that have been covered. The report can be viewed if there is already existing application or static coverage-based data on DTP. To create this report, run jtestcov with the -runtime switch.

Follow the procedure below to create a coverage report and upload the data to DTP.

Ensure that Jtest is properly configured, including DTP, scope, and authorship settings. See Connecting to DTP, Sending Results and Publishing Source Code to DTP, Configuration.
Configure the following settings in the jtestcov.properties (located in the <INSTALL_DIR>/integration/coverage directory) or jtestcli.properties file in order to properly merge coverage data:
- report.coverage.images - Specifies a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. DTP supports up to three coverage images per report.
- session.tag - Specifies a unique identifier for the test run and is used to distinguish different runs on the same build.
- build.id - Specifies a build identifier used to label results. It may be unique for each build, but it may also label several test sessions executed during a specified build.
If you want to use jtestcli, copy the runtime coverage and static coverage files to the same machine and run jtestcli with the following switches:
- -runtimeCoverage: Specifies the path to runtime coverage data collected using the coverage agent. You can provide a path to an individual .data file with coverage information from one testing session, or a path to a folder that contains many .data files from multiple testing sessions.
- -staticCoverage: Specifies the path to the static coverage file (see Generating the Monitor Package).
- -config: Specifies the URL of the Calculate Application Coverage test configuration that must be run to merge the coverage data.
- -publish: Sends the merged coverage to DTP.
  jtestcli -staticcoverage [path to static_coverage.xml file] -runtimecoverage [path/dir] -config "builtin://Calculate Application Coverage" -publish
  This ensures that Jtest has access to the runtime coverage data generated during test execution, as well as the static coverage data, which is required to fill the coverage.xml file with runtime coverage data.
If you want to use jtestcov, run jtestcov with the following switches:
- -app: Specifies the path to the Application Under Test directory or archive file.
- -runtime: Specifies the path to runtime coverage data collected using the coverage agent. You can provide a path to an individual .data file with coverage information from one testing session, or a path to a folder that contains many .data files from multiple testing sessions.
- -static: Specifies the path to the static coverage file (see Generating the Monitor Package).
- -publish: Sends the merged coverage to DTP.
  Example 1: Generating a report based on runtime data and static coverage from the monitor
  java -jar jtestcov.jar -static [path to static_coverage.xml file] -runtime [path/dir] -publish -settings [path to jtestcov.properties]
  Example 2: Generating a report based on runtime data and coverage from the application binaries
  java -jar jtestcov.jar -app [path/dir] -runtime [path/dir] -publish -settings [path to jtestcov.properties]
  Example 3: Generating a report based on runtime data
  java -jar jtestcov.jar -runtime [path/dir] -publish -settings [path to jtestcov.properties]
If you want to merge previously generated runtime coverage reports, run the jtestcov merge command. To merge two or more reports, run
java -jar jtestcov.jar merge -coverage <REPORT_FILE> -coverage <REPORT_FILE> [-coverage <REPORT_FILE>] [-report <REPORT_DIR>] [-excludeunmatchedruntime]
- -coverage <REPORT_FILE>: Specifies one or more input coverage reports to merge and publish to DTP.
- -report <REPORT_DIR>: (Optional) Generates an xml coverage report to the given folder. By default the report is stored to the .coverage/reports folder.

If you use SOAtest, static and runtime coverage data are merged and sent to DTP during test execution. See the Application Coverage section in the SOAtest user guide at https://docs.parasoft.com.

Reviewing Coverage in DTP

You can use the Coverage Explorer in DTP to review the application coverage achieved during test execution. For details on viewing coverage information, see the DTP user guide at https://docs.parasoft.com.

Known Limitations

If multiple users are simultaneously accessing the same web application, the coverage data they collect may be mixed. To ensure that coverage is properly associated with individual users, the multiuser mode must be enabled (see Configuring the Coverage Agent).
In the multiuser mode, the "default" user (the user who has not specified their ID) may collect extra coverage information from other users who are accessing the same web application.
In the multiuser mode, coverage collected for multithreaded applications may, in some cases, not be assigned to individual users. This is related to OpenTelemetry limitations regarding context sharing between threads.
In the multiuser mode, collecting coverage with the Jacoco engine is not supported.
Coverage data collected for web application initialization is not assigned to a specific user, but to the "default" user.

Command Line Options for jtestcov

This section lists the command line options for jtestcov.

coverage

Default command. Scans application and runtime coverage data in order to generate a coverage report. To generate a report, run:

java -jar jtestcov.jar coverage [-app <APP_LOCATION> | -static <BASE_COVERAGE_REPORT_FILE>] 
[-runtime <RUNTIME_COVERAGE_LOCATION>] [-report <REPORT_DIR>] [-deleteruntime] [-excludeunmatchedruntime]

-app <APP_LOCATION>

Specifies the location of the Application Under Test (AUT) directory or archive file (war, ear, jar, zip).

-static <BASE_COVERAGE_REPORT_FILE>

Specifies the XML coverage report file containing the coverage base of the AUT.

-runtime <RUNTIME_COVERAGE_LOCATION>

Specifies the location of the runtime coverage directory generated during test execution against the AUT.

-report <REPORT_DIR>

Generates an XML coverage report to the given folder. Optional parameter; by default the report is stored to the .coverage/reports folder.

-deleteruntime

Deletes the runtime coverage data files used to generate the coverage XML report.
If this parameter is present, the runtime coverage data files will be deleted after the report is created.

-excludeunmatchedruntime

Excludes from processing runtime coverage data that does not match classes contained in the coverage base.
If this parameter is present, any unmatched runtime coverage data will be discarded from the final coverage report.

impacted

Scans the application and analyzes the baseline coverage report to find tests that need rerunning. In order to generate an LST file for the impacted tests, run:

java -jar jtestcov.jar impacted -app <APP_LOCATION> 
-baseline <BASELINE_COVERAGE_REPORT FILE> [-report <LST_FILE_DIR>]

-app <APP_LOCATION>

Specifies the location of the Application Under Test (AUT) directory or archive file (war, ear, jar, zip).

-baseline <BASELINE_COVERAGE_REPORT_FILE>

Specifies the XML coverage report to use as the baseline.

-report <LST_FILE_DIR>

Generates an LST file in a given location. If an LST file already exists, it will be overwritten. Optional parameter; by default the LST file is stored to the .coverage/lsts folder.

merge

Merges previously generated runtime coverage reports. In order to merge two or more reports, run:

java -jar jtestcov.jar merge -coverage <REPORT_FILE> -coverage <REPORT_FILE> 
[-coverage <REPORT_FILE>] [-report <REPORT_DIR>] [-excludeunmatchedruntime]

-coverage <REPORT_FILE>

Specifies one or more input coverage XML reports to merge and publish to DTP.

-report <REPORT_DIR>

Generates an XML coverage report to the given folder. Optional parameter; by default the report is stored to the .coverage/reports folder.

help

Shows the jtestcov help.

Other arguments:

-include <PATTERN>
-exclude <PATTERN>

Specifies patterns to include/exclude elements during AUT scanning. Optional parameter; by default all elements are accepted.

-publish

Uploads the generated reports to the DTP Server. (Settings pattern: report.dtp.publish=true)

-settings <SETTINGS_FILE>

Reads the custom configuration from a file. <SETTINGS_FILE> should be an existing configuration file with <KEY=VALUE> entries (.properties).

-property

Specifies a custom configuration entry. You can use -property multiple times (if more than one entry includes the same key, the entry specified as first will overwrite all the following entries).

-showdetails

Increases output verbosity.
(Settings pattern: console.verbosity.level=high)

jtestcov Exit Code

The process may exit with the following exit code:

Code	Meaning
1	The process exited with an exception. Check error log.

Introduction

Prerequisites

Process Overview

Configuring the Application Under Test for Coverage

Generating the Monitor Package

Configuring the Coverage Agent

Attaching the Coverage Agent to the AUT

Connecting with the Coverage Agent via HTTPS

Executing Tests to Collect Runtime Coverage

Uploading Test Results to DTP

Creating a Coverage Report and Uploading the Data to DTP

Reviewing Coverage in DTP

Known Limitations

Command Line Options for jtestcov

coverage

-app <APP_LOCATION>

-static <BASE_COVERAGE_REPORT_FILE>

-runtime <RUNTIME_COVERAGE_LOCATION>

-report <REPORT_DIR>

-deleteruntime

-excludeunmatchedruntime

impacted

-app <APP_LOCATION>

-baseline <BASELINE_COVERAGE_REPORT_FILE>

-report <LST_FILE_DIR>

merge

-coverage <REPORT_FILE>

-report <REPORT_DIR>

help

Other arguments:

-include <PATTERN>-exclude <PATTERN>

-publish

-settings <SETTINGS_FILE>

-property

-showdetails

jtestcov Exit Code

-include <PATTERN>
-exclude <PATTERN>