In this section:

Introduction

You can monitor and collect coverage data during manual or automated functional tests performed on a running application. You can also send coverage data and test results to DTP, which merges and correlates the data. The application coverage information can be displayed in the DTP Coverage Explorer (see the "Coverage Explorer" chapter in the DTP user manual), which provides insights about how well the application is tested, as well as the quality of your tests. 

Prerequisites

Runtime coverage can be collected for Java 1.8 - 17.

Java JDK 8 line break calculations in bytecode are inconsistent with previous Java versions. This may lead to inconsistencies in coverage results.

If you want your coverage report to also include information about the application source code, the following components are required:

• Apache Maven, Gradle, or Ant build system

The jtestcov tool requires Java 11 or higher to be installed on your machine. Both Oracle Java SE and OpenJDK are supported. Other free and open source implementations may cause issues with the proper functioning of the coverage tool. It is recommended to set JAVA_HOME and add it to the system PATH. For information about jtestcov usage, see Creating a Coverage Report and Uploading the Data to DTP and Command Line Options for jtestcov.

Process Overview

Jtest ships with a component called the coverage agent. The coverage agent is attached to the application under test (AUT) and monitors the code being executed as the AUT runs. When the coverage agent is attached to the AUT, a REST API is exposed that enables you to mark the beginning and end of each test and test session. 

Metadata about the lines of code that can be covered (static coverage data) is collected either by running a dedicated test configuration as part of the application build process or by scanning the application binaries. During test execution, interactions with the coverage agent are written to a dynamic coverage map, which contains markers that specify which lines of code were touched.  

Jtest processes the dynamic coverage map and static coverage data. A coverage.xml file, which contains the coverage information, is produced and sent to DTP. When DTP receives the coverage data, it is loaded into a coverage image, which is a special tag that enables you to aggregate coverage data from runs with the same build ID. The coverage image enables you to associate coverage information with specific tests. 

Jtest can also collect application coverage which includes only runtime coverage and general information about the classes that have been executed without any details about the class structure other than the lines that have been covered. A simplified report is then sent and appended to existing coverage data on DTP. The coverage data can be viewed if information about source code is already available on DTP. 

Test results are also sent to DTP from the tool executing the tests (i.e., SOAtest, manual tests, etc.) in a report.xml file. If the build IDs for the coverage data file and the report match, DTP is able to correlate the data and display the coverage information. 

You can use either jtestcov or jtestcli to create a coverage report and upload the data to DTP. jtesctcov is a dedicated tool for collecting coverage and offers more options in terms of creating coverage reports, see Creating a Coverage Report and Uploading the Data to DTP. The jtestcov.jar file is located in the <INSTALL_DIR>/integration directory. jtestcov is a standalone tool that can be copied out from Jtest installation directory and used independently. Therefore, you can easily integrate it within your testing flow. For details about jtestcov command line options, see Command Line Options for jtestcov.

Configuring the Application Under Test for Coverage

The following steps are required to prepare the application under test (AUT):

1. Optional: Generate the monitor package containing the preset coverage agent configuration and static coverage file, which includes metadata about user classes, methods, and lines; see Generating the Monitor Package.
2. Configure the coverage agent to collect runtime coverage; see Configuring the Coverage Agent.
3. Attach the coverage agent to the AUT; see Attaching the Coverage Agent to the AUT.

Generating the Monitor Package

The monitor package is created during the build process by the Jtest Plugin for Maven, Gradle, or Ant. To ensure that coverage is correctly matched with the AUT's source code, generate the package using the same build system and Java version that was used to build the AUT. 

The package contains the following:

• static_coverage.xml - the file that contains static coverage information,
• agent.jar - the Jtest Java coverage agent jar archive (copied from the <INSTALL_DIR>/integration/coverage directory) 
• agent.properties - the agent settings file that contains scope parameters generated during the build process and other attributes,
• agent.sh/agent.bat - the script that generates the Jtest Java agent VM arguments necessary for attaching the agent to the AUT process.

The package must be generated on the build machine that contains the source code and it can be used until the code changes.

Execute an appropriate command in the AUT's main directory to generate the package:

Maven

mvn package jtest:monitor

Gradle

gradle assemble jtest-monitor -I [INSTALL]/integration/gradle/init.gradle

Ant

ant -lib [INSTALL]/integration/ant/jtest-ant-plugin.jar -listener com.parasoft.Listener jtest-monitor

<target name="jtest-monitor" depends="compile">
             <jtest:monitor/>
</target>

The monitor.zip package will be generated and placed into the build output directory. The path to the location will be printed on the console. 

Configuring the Coverage Agent 

Application servers usually contain more than one application. Additionally, common server classes or application libraries do not need to be instrumented. Jtest only needs to collect coverage for application source code. Instrumenting all classes would be too time-consuming.

The application on the server is already built, so we cannot gather information about which classes come from the source code. For this reason, properly setting the scope of the coverage agent is very important.

You can configure the coverage agent with:

• the agent.properties file
• arguments to -javaagent when attaching the coverage agent to the AUT; see Attaching the Coverage Agent to the AUT. 

The agent.properties and agent.jar files are included in the <INSTALL_DIR>/integration/coverage directory. The agent.properties file contains default properties that can be modified to properly configure the coverage agent. The following example shows how the coverage agent can be configured with the agent.properties.file:

jtest.agent.autostart=false
jtest.agent.serverEnabled=true
jtest.agent.associateTestsWithCoverage=false
jtest.agent.includes=com/myapp/**
jtest.agent.runtimeData=[...]

If you have generated the monitor.zip package, you can use the agent.properties file included in the package which already contains scope parameters generated during the build process and other attributes (see Generating the Monitor Package). 

Adding properties to -javaagent in command line requires modifying their names by removing the "jtest.agent" prefix, for example:

-javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",autostart=true

The following table describes all properties that can be used to configure the coverage agent:

Attaching the Coverage Agent to the AUT 

1. Optional: If you want to use the monitor.zip package:
   1. Extract the contents of the package to the server machine.

   2. Run the agent.sh/agent.bat script from the monitor package to generate the Jtest Java agent VM arguments. The scripts output the -javaagent VM argument to the console.

2. Use the -javaagent flag to connect the agent to the AUT, which will result in the runtime_coverage subdirectory: 

   Jtest Agent VM argument:
   -javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to monitor dir]\monitor\ runtime_coverage"

   See Configuring the Coverage Agent for details about coverage agent configuration using -javaagent.

3. Add the -javaagent flag to the application startup script to connect the agent to the AUT. Attaching the coverage agent to the AUT exposes a REST API for controlling the agent. 
4. Restart the AUT (see Step 5, 3. in Web Application Coverage Tutorial). 
5. Enter the following URL into the browser bar to verify that the coverage agent is set:
   [application_host]:8050/status/
   If the application is properly configured, the API will return data:
   

Connecting with the Coverage Agent via HTTPS

By default, the coverage agent is available via the HTTP protocol. To connect via HTTPS, you need to configure the coverage agent by adding the following options to the Jtest Agent VM argument (see Attaching the Coverage Agent to the AUT):

• useSSL=true

• certificateStorePath=[path to the keystore file]

• certificateStorePassword=[password to the keystore file]

• certificateStoreType=[format of the keystore file]

• certificateKeyPassword=[password to the SSL certificate]

Your Jtest Agent VM argument added to the application server’s startup script may resemble the following:

-javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to monitor dir]\monitor\ runtime_coverage",useSSL=true,certificateStorePath="F:\certificates\keystore.jks",certificateStorePassword=4!ij?ffQMk,certificateStoreType="JKS",certificateKeyPassword=dhW!34

This will enable the coverage agent to recognize the SSL certificate you have configured for your application under test.

Executing Tests to Collect Runtime Coverage

You can use SOAtest to run functional tests, as well as execute manual tests with Coverage Agent Manager (CAM). Refer to the SOAtest user guide or CAM documentation for details about performing test sessions.

When tests are being performed:

• SOAtest will collect runtime coverage, and upload the data to DTP.
• CAM will collect runtime coverage and test results, which must be manually downloaded. Next, you need to:
  • manually upload test results to DTP; see Uploading Test Results to DTP.
  • merge the runtime coverage with static coverage and upload the merged data to DTP; Creating a Coverage Report and Uploading the Data to DTP.

Uploading Test Results to DTP

If you use Jtest with CAM:

1. Go to Report Center in the DTP interface.
2. Click the gear icon and choose Report Center Settings> Additional Settings> Report Center Administration> Tools> Data Collector Upload Form (requires admin permissions).
3. Click Choose File and browse for the report.xml file you downloaded from CAM.
4. Click the Upload button to upload the file to DTP.

If you use SOAtest, test results are automatically sent to DTP during test execution. See the Application Coverage section in the SOAtest user guide at https://docs.parasoft.com.

Creating a Coverage Report and Uploading the Data to DTP

You can use either the jtestcov or the jtestcli command to create a coverage report and upload the data to DTP. For details about jtestcov command line options, see Command Line Options for jtestcov.

If you use Jtest with CAM, there are 3 types of coverage report that can be generated:

1. A static and runtime coverage report: a report containing merged static coverage (including source code information) and runtime coverage. To create this report, run jtestcli with the -runtimeCoverage and -staticCoverage switches or jtestcov with the -runtime and -static switches.
2. An application coverage report: a static coverage report created by scanning application binaries and runtime coverage. To create this report, run jtestcov with the -app and -runtime switches.
3. A runtime coverage report: a simplified application coverage report including only runtime coverage and general information about the classes that have been executed without any details about the class structure other than the lines that have been covered. The report can be viewed if there is already existing application or static coverage-based data on DTP. To create this report, run jtestcov with the -runtime switch.

Follow the procedure below to create a coverage report and upload the data to DTP.

1. Ensure that Jtest is properly configured, including DTP, scope, and authorship settings. See Connecting to DTP, Sending Results and Publishing Source Code to DTP, Configuration.
2. Configure the following settings in the jtestcov.properties (located in the <INSTALL_DIR>/integration/coverage directory) or jtestcli.properties file in order to properly merge coverage data:
   - report.coverage.images -  Specifies a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. DTP supports up to three coverage images per report.
   - session.tag - Specifies a unique identifier for the test run and is used to distinguish different runs on the same build.
   - build.id - Specifies a build identifier used to label results. It may be unique for each build, but it may also label several test sessions executed during a specified build.
3. If you want to use jtestcli, copy the runtime coverage and static coverage files to the same machine and run jtestcli with the following switches:
   
   • -runtimeCoverage: Specifies the path to runtime coverage that you download with CAM (see Coverage Agent Manager (CAM) section of the DTP documentation for details). You can provide a path to an individual .data file with coverage information from one testing session, or a path to a folder that contains many .data files from multiple testing sessions.
   • -staticCoverage: Specifies the path to the static coverage file (see Generating the Monitor Package).
   • -config: Specifies the URL of the Calculate Application Coverage test configuration that must be run to merge the coverage data.

   • -publish: Sends the merged coverage to DTP.

     jtestcli -staticcoverage [path to static_coverage.xml file] -runtimecoverage [path/dir]
     -config "builtin://Calculate Application Coverage" -publish

     This ensures that Jtest has access to the runtime coverage data generated during test execution, as well as the static coverage data, which is required to fill the coverage.xml file with runtime coverage data.

4. If you want to use jtestcov, run jtestcov with the following switches:
   • -app: Specifies the path to the Application Under Test directory or archive file.
   • -runtime: Specifies the path to runtime coverage that you download with CAM (see Coverage Agent Manager (CAM) section of the DTP documentation for details). You can provide a path to an individual .data file with coverage information from one testing session, or a path to a folder that contains many .data files from multiple testing sessions.
   • -static: Specifies the path to the static coverage file (see Generating the Monitor Package).

   • -publish: Sends the merged coverage to DTP.

     Example 1: Generating a report based on runtime data and static coverage from the monitor

     java -jar jtestcov.jar -static [path to static_coverage.xml file] -runtime [path/dir]
     -publish -settings [path to jtestcov.properties]

     Example 2: Generating a report based on runtime data and coverage from the application binaries

     java -jar jtestcov.jar -app [path/dir] -runtime [path/dir] -publish
     -settings [path to jtestcov.properties]

      Example 3: Generating a report based on runtime data

     java -jar jtestcov.jar -runtime [path/dir] -publish -settings [path to jtestcov.properties]

5. If you want to merge previously generated runtime coverage reports, run the jtestcov merge command. To merge two or more reports, run

   java -jar jtestcov.jar merge -coverage <REPORT_FILE> -coverage <REPORT_FILE>
   [-coverage <REPORT_FILE>] [-report <REPORT_DIR>]

   • -coverage <REPORT_FILE>: Specifies one or more input coverage reports to merge and publish to DTP.
   • -report <REPORT_DIR>: (Optional) Generates an xml coverage report to the given folder. By default the report is stored to the .coverage/reports folder.

If you use SOAtest, static and runtime coverage data are merged and sent to DTP during test execution. See the Application Coverage section in the SOAtest user guide at https://docs.parasoft.com.

Reviewing Coverage in DTP

You can use the Coverage Explorer in DTP to review the application coverage achieved during test execution. For details on viewing coverage information, see the DTP user guide at https://docs.parasoft.com.

Known Limitations

• If multiple users are simultaneously accessing the same web application, the coverage data they collect may be mixed. To ensure that coverage is properly associated with individual users, the multiuser mode must be enabled (see Configuring the Coverage Agent).
• The HTTP or HTTPS protocols are required to enable the multiuser mode, as the user-specific information must be provided with the HTTP header.
• In the multiuser mode, the "default" user (the user who has not specified their ID) may collect extra coverage information from other users who are accessing the same web application.
• In the multiuser mode, assigning coverage collected for multithreaded application to individual users is limited. Coverage data for child threads is not assigned to the user who is actually accessing the application, but to the "default" user.
• In the multiuser mode, collecting coverage with the Jacoco engine is not supported.
• Coverage data collected for web application initialization is not assigned to a specific user, but to the "default" user.

Command Line Options for jtestcov

This section lists the command line options for jtestcov.

coverage

Default command. Scans application and runtime coverage data in order to generate a coverage report. To generate a report, run:

java -jar jtestcov.jar coverage [-app <APP_LOCATION> | -static <BASE_FILES>] 
-runtime <RUNTIME_COVERAGE_LOCATION> [-report <REPORT_DIR>] [-deleteruntime]

-app <APP_LOCATION>

Specifies the location of the Application Under Test (AUT) directory or archive file (war, ear, jar, zip).

-static <BASE_COVERAGE_REPORT_FILE> or <BASE_METADATA_FILE>

Specifies the XML coverage report or metadata file containing the coverage base of the AUT.

-runtime <RUNTIME_COVERAGE_LOCATION>

Specifies the location of the runtime coverage directory generated during test execution against the AUT.

-report <REPORT_DIR>

Generates an XML coverage report to the given folder. Optional parameter; by default the report is stored to the .coverage/reports folder.

-deleteruntime

Deletes the runtime coverage data files used to generate the coverage XML report.
If this parameter is present, the runtime coverage data files will be deleted after the report is created.

impacted

Scans the application and analyzes the baseline coverage report to find tests that need rerunning. In order to generate an LST file for the impacted tests, run: 

java -jar jtestcov.jar impacted -app <APP_LOCATION> 
-baseline <BASELINE_COVERAGE_REPORT FILE> [-report <LST_FILE_DIR>]

-app <APP_LOCATION>

Specifies the location of the Application Under Test (AUT) directory or archive file (war, ear, jar, zip).

-baseline <BASELINE_COVERAGE_REPORT_FILE>

Specifies the XML coverage report to use as the baseline.

-report <LST_FILE_DIR>

Generates an LST file in a given location. If an LST file already exists, it will be overwritten. Optional parameter; by default the LST file is stored to the .coverage/lsts folder.

merge

Merges previously generated runtime coverage reports. In order to merge two or more reports, run:

java -jar jtestcov.jar merge -coverage <REPORT_FILE> -coverage <REPORT_FILE> 
[-coverage <REPORT_FILE>] [-report <REPORT_DIR>]

-coverage <REPORT_FILE>

Specifies one or more input coverage XML reports to merge and publish to DTP.

-report <REPORT_DIR>

Generates an XML coverage report to the given folder. Optional parameter; by default the report is stored to the .coverage/reports folder.

help

Shows the jtestcov help.

Other arguments:

-include <PATTERN>
-exclude <PATTERN>

Specifies patterns to include/exclude elements during AUT scanning. Optional parameter; by default all elements are accepted.

-publish

Uploads the generated reports to the DTP Server. (Settings pattern: report.dtp.publish=true)

-settings <SETTINGS_FILE>

Reads the custom configuration from a file. <SETTINGS_FILE> should be an existing configuration file with <KEY=VALUE> entries (.properties).

-property

Specifies a custom configuration entry. You can use -property multiple times (if more than one entry includes the same key, the entry specified as first will overwrite all the following entries).

-showdetails

Increases output verbosity.
(Settings pattern: console.verbosity.level=high)

jtestcov Exit Code

The process may exit with the following exit code: