This topic provides information about how to use SOAtest as part of the Parasoft Application Coverage workflow. For complete information about using the Application Coverage solution, refer to the Collecting Coverage for Web Applications guide. In this section:

Introduction

The Parasoft Application Coverage solution enables you to measure application code coverage while executing tests against your running applications. This enables you to gauge the effectiveness of your existing test suite, determine where additional tests need to be added, and more efficiently execute tests as the application evolves. You can collect application coverage by utilizing the coverage agent shipped with Parasoft Jtest and Parasoft dotTEST, which allows you to monitor the application as tests are being performed.

Application coverage is obtained by merging static and dynamic (runtime) coverage data into one coverage file and uploading the file to DTP for viewing.

• Static coverage data is collected by Parasoft Jtest or Parasoft dotTEST.
• Runtime coverage data is collected by SOAtest.
• Both coverage types are merged and uploaded to DTP by SOAtest.

The following steps describe the Application Coverage workflow with SOAtest:

1. Generating the static coverage file. The static coverage file is generated by Jtest or dotTEST in the .xml or .data format and contains metadata about user classes, methods, and lines.
2. Attaching the coverage agent to the application under test (AUT). The coverage agent ships with Jtest and dotTEST, and allows you to monitor the code being executed when the AUT is running.
3. Configuring SOAtest to connect to the coverage agent and collect runtime coverage, merge it with static coverage, and upload the data to DTP.
4. Executing tests with SOAtest as part of an automated process.
5. Reviewing the application coverage on DTP in the Coverage Explorer.

Anchor
Configuring the Application Under Test for Coverage Configuring the Application Under Test for Coverage

Configuring the Application Under Test for Coverage

Before you start collecting coverage with SOAtest, you need to configure the application under test (AUT) using Parasoft dotTEST or Parasoft Jtest. For details how to perform the following steps, see the Jtest or dotTEST user guide at docs.parasoft.com. Ensure that your SOAtest, dotTest, and JTest versions are in sync to avoid unexpected issues.

1. Generate a static coverage file on a build machine. The file contains metadata about user classes, methods, and lines.
2. (Optional) Customize the coverage agent by configuring the coverage agent options. This step is required if you are collecting coverage information for multiple users that are simultaneously interacting with the AUT. 
3. Attach the coverage agent to the AUT.

Anchor
Configuring SOAtest to Collect and Upload Coverage Configuring SOAtest to Collect and Upload Coverage

Configuring SOAtest to Collect and Upload Coverage 

With the proper configuration, SOAtest will collect application coverage during test execution. To do so, configure a SOAtest settings file (see Configuring Settings) with the following options, as appropriate:

• build.id: Set to the build ID with which to correlate coverage results. If you want to merge SOAtest coverage results with results from other types of testing—such as unit testing and manual testing of the same application—they must all use the same build ID. The build ID set here must match the build ID set for dotTEST or Jtest when collecting the static and dynamic coverage files and uploading the coverage data.
• dtp.project: This value must match the dtp.project value set for dotTEST or Jtest when collecting the static and dynamic coverage files.
• application.coverage.enabled: Set to true. This enables application coverage.
• application.coverage.agent.url: Specify the URL where the application under test and coverage agent are hosted. Include the protocol (HTTP/HTTPS) and port number of the agent. Separate multiple entries with a semi-colon ( ; ). For example: http\://<Agent Host>\:8050;http\://<Agent Host>\:9050.
• application.coverage.user.id: Optional. Specify a user ID so that coverage results can be associated with a specific user. A user ID should only be specified when the coverage agent is configured to run in multi-user mode. See the DTP Engine documentation for details.
• application.coverage.dtp.publish: Set to true to upload coverage reports to DTP or false to not upload coverage reports to DTP. If you have configured multiple coverage agents, their coverage reports will be merged into a single coverage report for DTP. Defaults to the value of report.dtp.publish (see Configuring Settings) or to true if -publish is passed to soatestcli.
• application.coverage.binaries: Specify the URL or local file that contains binaries of the application under test (AUT). You can specify the path to a folder or a .war, .jar, .zip, or .ear file. Optional. Do not use if application.coverage.static.file has been specified.
• application.coverage.binaries.exclude: Specify a comma-separated pattern of binaries to exclude during AUT scanning. Optional. Do not use if application.coverage.static.file has been specified. By default, all binaries are accepted.
• application.coverage.binaries.include: Specify a comma-separated pattern of binaries to include during AUT scanning. Optional. Do not use if application.coverage.static.file has been specified. By default, all binaries are accepted.
• application.coverage.static.file: Specify the path to the static coverage file generated with dotTEST or Jtest in the .xml or .data format. Do not use if application.coverage.binaries has been specified.
• application.coverage.images: Specify a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. Separate multiple entries with a semi-colon ( ; ). For details, see the Parasoft DTP user guide. For example: ${dtp_project};${dtp_project}_Functional Test.

Configuring the Coverage Agent

Application servers usually contain more than one application. Additionally, common server classes or application libraries do not need to be instrumented. The agent only needs to collect coverage for application source code. Instrumenting all classes would be too time consuming. For this reason, properly setting the scope of the coverage agent is very important.

You can configure the coverage agent by modifying the properties in the agent.properties and passing the properties to the -javaagent argument. The agent supports several parameters (see Coverage Agent Parameters), but configuring the default settings is suitable for most cases:

jtest.agent.serverEnabled=true
jtest.agent.includes=com/myapp/data,com/myapp/common/**
jtest.agent.excludes=com/myapp/transport/*,com/myapp/autogen/**
jtest.agent.associateTestsWithCoverage=true
jtest.agent.autostart=false

Coverage Agent Parameters

The following table describes all properties that can be set for the agent:

When the properties are configured, add a -javaagent argument when starting your application server to attach the agent and include the agent configuration file:  

-javaagent:'/path/to/agent.jar'=settings='/path/to/agent.properties',runtimeData='/path/to/runtime_coverage' 

For your convenience, the coverage directory includes a script that will generate the -javaagent arguments. Run either the agent.sh or agent.bat script and copy the output to your server startup script.

$ ./agent.sh 
Add this Java VM args to your process:
---------------------------------------------------
-javaagent:"/home/TIA/test_impact_analysis/integration/coverage/agent.jar"=settings="/home/TIA/test_impact_analysis/integration/coverage/agent.properties",runtimeData="/home/TIA/test_impact_analysis/integration/coverage/runtime_coverage"
---------------------------------------------------
Press any key to continue . . . 

In the following example, the agent is attached to a Tomcat server with a JAVA_OPTS variable at the beginning of the catalina.sh (Linux) or catalina.bat (Windows) scripts:

if [ "$1" = "start" -o "$1" = "run" ]; then
JAVA_OPTS='-javaagent:"/home/TIA/test_impact_analysis/integration/coverage/agent.jar"=settings="/home/TIA/test_impact_analysis/integration/coverage/agent.properties",runtimeData="/home/TIA/coverage_storage"'
fi

if "%1"=="stop" goto skip_instrumentation
set JAVA_OPTS=-javaagent:"C:\TIA\test_impact_analysis\integration\coverage\agent.jar"=settings="C:\TIA\test_impact_analysis\integration\coverage\agent.properties",runtimeData="C:\TIA\coverage_storage"
:skip_instrumentation

Start the application and verify that the agent is ready by opening <host>:8050/status in your browser. You should see a JSON object that contains test, runtime_coverage, and testCase properties, for example:

{"test":null,"session":"runtime_coverage_20191008_1537_0","testCase":null}

You can also check the directory you specified with the runtimeData property (/home/TIA/coverage_storage in the example above). The directory should contain a set of static coverage data files. The files are generated when the agent is started.

Executing Tests to Collect and Upload Coverage

Run your tests in command line mode using the Test Configuration and settings file you have configured (see Configuring SOAtest to Collect and Upload Coverage). This step should be part of an automated process.

Reviewing Coverage in DTP

Go to DTP and open the Coverage Explorer to review the application coverage achieved by your SOAtest tests.  For details on using the Coverage Explorer, see the Parasoft DTP user guide at https://docs.parasoft.com.