In this guide:

Executive Summary

This guide is intended to help you collect coverage information for applications developed with .NET framework or Java technology.

The primary audience for this user guide are people responsible for ensuring compliance with your organization's policy regarding the application coverage level, including QA Engineers, developers, and build masters.

Prerequisites

We assume that you are familiar with Parasoft technologies and have already deployed and licensed the following products:

• dotTEST or Jtest 10.4.0 or higher
• Coverage Agent Manager (CAM) or SOAtest
  
• Parasoft DTP

Application Coverage Workflow

1. Generating the static coverage file.
   The static coverage file (static_coverage.xml) is generated during the build process by Jtest or dotTEST. It is an XML file that contains metadata about user classes, methods, and lines.
2. Attaching the Coverage Agent to the application under test (AUT).
   The Coverage Agent ships with Jtest and dotTEST, and allows you to monitor the code being executed when the AUT is running.
3. Connecting Coverage Agent Manager (CAM) to the Coverage Agent.
   Create the connection before you start interacting with the AUT.
4. Performing test sessions with CAM.
   CAM allows you to mark the beginning and end of test and test sessions while collecting coverage on the running AUT.
5. Downloading test results and the dynamic (runtime) coverage data when a test session is finished and stopped in CAM.
   The dynamic coverage data is saved in the runtime_coverage_[timestamp].data file. The test results are stored in the report.xml file.
6. Uploading the test results to DTP.
   Data Collector Upload Form allows you to upload the report.xml file to DTP.
7. Merging the dynamic and static coverage information and send it to DTP.
   Jtest or dotTEST merges the static coverage data (static_coverage.xml) and dynamic coverage data (runtime_coverage_[timestamp].data) into a coverage.xml file, and sends the coverage.xml file to DTP. If the coverage images, session tags, and build IDs associated with the coverage.xml file and report.xml file match, DTP can properly aggregate, associate, and display the data in a range of reporting mechanisms.

Step 1: Generate the Static Coverage File

The static coverage file must be generated on the build machine that contains the source code. The name of the file is static_coverage.xml.

Generating Static Coverage with dotTEST

Run the following test configuration on the solution:

dottestcli.exe -config "builtin://Collect Static Coverage" -solution SOLUTION_PATH

The location of the static_coverage.xml file will be printed to the console.

Generating Static Coverage with Jtest

The static coverage file is included in a monitor.zip package that is generated during the build process by the Jtest Plugin for Maven, Gradle, or Ant.

1. Add the monitor task or goal to your build command.
   

   Maven

   mvn package jtest:monitor

   Gradle

   gradle assemble jtest-monitor -I [INSTALL]/integration/gradle/init.gradle

   Ant

   ant -lib [INSTALL]/integration/ant/jtest-ant-plugin.jar -listener com.parasoft.Listener jtest-monitor

   Ant requires all classes to be compiled before the monitor task is executed. Modify your project prior to the build and configure the task to ensure the correct sequence. The following example shows how the target can be configured: <target name="jtest-monitor" depends="compile">              <jtest:monitor/> </target>

   Maven

   mvn package jtest:monitor

   Gradle

   gradle assemble jtest-monitor -I [INSTALL]/integration/gradle/init.gradle

   Ant

   ant -lib [INSTALL]/integration/ant/jtest-ant-plugin.jar -listener com.parasoft.Listener jtest-monitor

   Ant requires all classes to be compiled before the monitor task is executed. Modify your project prior to the build and configure the task to ensure the correct sequence. The following example shows how the target can be configured: <target name="jtest-monitor" depends="compile">              <jtest:monitor/> </target>

2. Execute the build command in the AUT's main directory.
   The location of the monitor.zip package that contains the static_coverage.xml file will be printed to the console.
3. Extract the contents of the monitor.zip package to the server machine.

Step 2: Attach the Coverage Agent to the Application Under Test (AUT)

Attaching the Coverage Agent to the AUT allows you enable collecting dynamic (runtime) coverage.

Attaching the Coverage Agent with dotTEST

To attach the Coverage Agent to the AUT, you need to launch the IIS Manager tool shipped with dotTEST on the machine where IIS is installed and the AUT is deployed.

1. Copy the [DOTTEST_INSTALLATION_DIR]\integration\IIS directory to the machine were IIS is installed and the AUT is deployed.
2. Run a console as Administrator.

3. Invoke the following command to launch the IIS Manager tool:

   dottest_iismanager.exe

4. Go to the following address to check the status of the coverage agent: http://host:8050/status. If the Coverage Agent is attached, you should receive the following response:

   {"session":null,"test":null}

Attaching the Coverage Agent with Jtest

To attach the Coverage Agent to the AUT, you need to add Jtest's -javaagent VM argument to the startup script of the server where the AUT is deployed.

1. Extract the contents of the monitor.zip package you generated (see Generating Static Coverage with Jtest) to the server machine. The package contains the agent.sh/agent.bat script.

2. Run the agent.sh/agent.bat script to print the Jtest Java agent VM argument to the console:

   Jtest Agent VM argument: -javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to monitor dir]\monitor\ runtime_coverage"

3. Add the -javaagent argument to the application server’s startup script.
4. Restart the server.

5. Go to the following address to check the status of the coverage agent: http://host:8050/status. If the Coverage Agent is attached, you should receive the following response:

   {"test":null,"session":null,"testCase":null}

Step 3: Connect CAM or SOAtest to the Coverage Agent

You can connect to either CAM or SOAtest to the Coverage Agent. CAM provides an interface for starting and stopping test sessions. Connecting SOAtest enables you to collect application coverage during automated functional test execution.  

Connecting CAM to the Coverage Agent

1. Open CAM in a browser:

   http://[your-Tomcat-host:port]/cam.

2. Click Agents.
   
   
3. Click Add Connection to configure the connection with the Coverage Agent.
   Agent Name: Any user-friendly string that you want to display in CAM (for example, the name of the application).
   Agent Address: The URL of the server where the Coverage Agent is attached to the AUT (see Step 2). The default port number for the Coverage Agent is 8050.
4. Click Test to verify the connection.
   
5. Click Connect after successfully testing the connection.

Connecting SOAtest to the Coverage Agent

You will first need to create a test configuration from the SOAtest UI.

1. Open SOAtest and choose Parasoft> Test Configurations.

Step 4: Perform testing sessions

Perform Testing with CAM

To collect coverage data, you need to start a session. Only one tests session can be in progress at a time.

1. Click the New Session button, enter the following information when prompted:
   
   • Session Name: the name of the session
   • Project: the name of the project in DTP that the results will be associated with
   • Build ID: A build identifier for label results. It may be unique for each build but may also label more than one test sessions that were executed during a specified build.
   This information is critical for the data to be properly merged and correlated in DTP.
   
2. Click the Start Session button to begin a new test session.
3. Click the New Test button.
   
   
4. Enter a name and click Start Test to continue.
5. Perform your manual test on the AUT and enter the data associated with the test.
   Ensure that you select the correct Test result option (Pass/Fail/Incomplete).
   
   
6. Click Stop Test when finished.
7. Continue adding and performing your tests until your scenario is complete.
   Ensure that you stop and start tests in between interacting with the AUT.
8. Click Stop Session when you’ve completed your tests.

Perform Testing with SOAtest

Execute the soatestcli using the application coverage test configuration to run your tests:

./soatestcli -config "/path/to/your/test-configuration/app-cov.properties" -data "/path/to/your/workspace" -resource "/path/to/your/tests/your_tests.tst" -localsettings "/path/to/your/localsettings/file/soatest-app-cov.properties"

Step 5: (CAM only) Downloading Coverage and Test Results from CAM

1. Click Download Coverage to download the runtime_coverage_[timestamp].data file and note the location.
2. Click Download Test Results to download the report.xml file and note the location.
   

Step 6: Uploading the test results to DTP.

Uploading test results to DTP allows you to associate coverage data with individual tests and view the information on DTP.

1. Go to Report Center in the DTP interface.
2. Click the gear icon and choose Report Center Settings> Additional Settings> Data Collector Upload Form (requires admin permissions).
3. Click Choose File and browse for the report.xml file you downloaded from CAM.
4. Click the Upload button to upload the file to DTP.

Step 7: Merge the dynamic and static coverage information and send it to DTP.

dotTEST and Jtest can merge the dynamic and static coverage into one coverage.xml file, and send the merged information to DTP. To ensure that the coverage data is correctly merged and displayed on DTP, configure the following options in the .properties file where you configure your dotTEST or Jtest settings:

• report.coverage.images -  Specifies a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. DTP supports up to three coverage images per report.
• session.tag - Specifies a unique identifier for the test run and is used to distinguish different runs on the same build.
• build.id - Specifies a build identifier used to label results. It may be unique for each build, but it may also label several test sessions executed during a specified build.

To merge the coverage data and send it to DTP, run dotTEST or Jtest with the following arguments:

• -staticcoverage - provides the path to the static_coverage.xml generated by dotTEST or Jtest
• -runtimeCoverage - provides the path to the runtime_coverage_[timestamp].data downloaded via CAM or a path to a folder that contains many .data files from multiple testing sessions.
• -publish - sends the merged coverage data to DTP
• (Jtest only) -config "builtin://Calculate Application Coverage" - specifies the built-in test configuration required to merge the coverage data

Your command line may resemble the following:

dotTEST

dottestcli.exe -runtimeCoverage [path] -staticCoverage [path] -publish

Jtest

jtestcli -staticcoverage [path] -runtimecoverage [path/dir] -config "builtin://Calculate Application Coverage" -publish

The -publish option is required to send the merged coverage data to DTP. Alternatively, you can configure the report.dtp.publish=true option in the .properties file where you configure dotTEST or Jtest.