...

Introduction

The Parasoft Application Coverage solution enables you to measure test application code coverage while executing tests on against your running applications. This  This enables you to gauge the effectiveness of your existing test suite, determine where additional tests need to be added, and more efficiently execute tests as the application evolves. . You can collect application coverage by utilizing the coverage agent shipped with Parasoft Jtest and Parasoft dotTEST, which allows you to monitor the application as tests are being performed.

Application coverage is obtained by merging static and dynamic (runtime) coverage data into one coverage file and uploading the file to DTP for viewing.

• Static coverage data is collected by Parasoft Jtest or Parasoft dotTEST.
• Runtime coverage data is collected by SOAtest.
• Both coverage types are merged and uploaded to DTP by SOAtest.

The following steps describe the Application Coverage workflow with SOAtest:

1. Generating the static coverage file. The static coverage file (static_coverage.xml) is generated by Jtest or dotTEST . It is an XML file that in the .xml or .data format and contains metadata about user classes, methods, and lines.
2. Attaching the Coverage Agent coverage agent to the application under test (AUT).  The Coverage Agent coverage agent ships with Jtest and dotTEST, and allows you to monitor the code being executed when the AUT is running.
3. Connecting Coverage Agent Manager (CAM) to the Coverage Agent. Create the connection before you start interacting with the AUT.
4. Performing test sessions. You can either execute tests with SOAtest or use Parasoft's Coverage Agent Manager (CAM), which is a functional testing UI that allows you to mark the beginning and end of test and test sessions while collecting coverage on the running AUT. CAM is available from the Parasoft customer portal.
5. Downloading test results and the dynamic (runtime) coverage data when a test session is finished. The dynamic coverage data is saved in the runtime_coverage_<timestamp>.data file. The test results are stored in the report.xml file.
6. Uploading the test results to DTP. Data Collector Upload Form allows you to upload the report.xml file to DTP.
7. Merging the dynamic and static coverage information and send it to DTP. Use Parasoft Jtest or dotTEST to merge the static coverage data (static_coverage.xml) and dynamic coverage data (runtime_coverage_<timestamp>.data) into a coverage.xml file, and send the coverage.xml file to DTP. If the coverage images, session tags, and build IDs associated with the coverage.xml file and report.xml file match, DTP can properly aggregate, associate, and display the data in a range of reporting mechanisms.

Refer to the Collecting Coverage for Web Applications guide for complete details.

Image Removed

...

1. Configuring SOAtest to connect to the coverage agent and collect runtime coverage, merge it with static coverage, and upload the data to DTP.
2. Executing tests with SOAtest as part of an automated process.
3. Reviewing the application coverage on DTP in the Coverage Explorer.

Anchor
Configuring the Application Under Test for Coverage Configuring the Application Under Test for Coverage

Configuring the Application Under Test for Coverage

The first step in the application coverage workflow is Before you start collecting coverage with SOAtest, you need to configure the application under test (AUT) for coverage. This involves generating a static coverage data file that captures information about the source code as well as configuring the coverage agent for the AUT. To do this:

1. On a build machine, generate a static coverage file as described in the Jtest or dotTEST documentation.
2. Configure the application under test with the Parasoft coverage agent as described in the Jtest or dotTEST documentation.

using Parasoft dotTEST or Parasoft Jtest. For details how to perform the following steps, see the Jtest or dotTEST user guide at docs.parasoft.com. Ensure that your SOAtest, dotTest, and JTest versions are in sync to avoid unexpected issues.

1. Generate a static coverage file on a build machine. The file contains metadata about user classes, methods, and lines.
2. (Optional) Customize the coverage agent by configuring the coverage agent options. This step is required if you are collecting coverage information for multiple users that are simultaneously interacting with the AUT. 
3. Attach the coverage agent to the AUT.

Anchor
Configuring SOAtest to Collect and Upload Coverage Configuring SOAtest to Collect and Upload Coverage

Configuring SOAtest to Collect and Upload Coverage 

...

With the proper configuration, SOAtest will collect application coverage during test execution. To configure and execute SOAtest for application coverage data collection:

1. Specify the installation directory of dotTEST or Jtest by doing one of the following:
   • UI: go to Parasoft> Preferences> Reports> Coverage.
   • Command line: set via the settings properties "jtest.install.dir" or "dottest.install.dir" as appropriate (see Configuring Settings).
2. Configure a SOAtest Test Configuration to collect, merge, and upload application coverage data as follows:. You can configure an existing test configuration or create and configure a new one (see Creating a Custom Test Configuration).
   
   1. Open the Test Configuration that you want to configure to collect application coverage data.
   2. Open the Execution> Application Coverage tab.
   3. Enable the Collect application coverage option.
   4. Under Coverage agent host, specify the host name hostname or IP address of the machine where the application under test and coverage agent residesare hosted.
   5. Under Coverage agent port, specify the coverage agent’s port number of the agent. The port number should match the value of the jtest.agent.port setting in the agent.properties file (default is 8050. You can click Test Connection to check if your settings are working properly.). 
   6. Under Coverage agent protocol, specify whether to connect to the coverage agent using http or https protocol. The default is http. 
   7. (Optional) Under Coverage agent user ID, you can specify a user ID so that coverage results can be associated with a specific user. A user ID should only be specified when the coverage agent is configured to run in multi-user mode. See the DTP Engine documentation for details.
   8. If you want test failures reported when the coverage agent connection fails, check Report coverage agent connection failures as test failures. Otherwise connection problems will be reported to the console, but will not cause the test to fail.
   9. Check Retrieve coverage data, then use Coverage data storage directory to indicate where you want the dynamic coverage data stored. You do not need to enable this option if the coverage agent is running on the build machine where static coverage was generated (since, in this case, both the static and runtime data can be published to DTP directly from the build machine).
   10. If you want the coverage data on the AUT deleted as soon as SOAtest retrieves it, also check Delete coverage data on retrieval. We recommend enabling this option in order to prevent the buildup of data files on the machine where the coverage agent is located.
   11. Apply the Test Configuration changes.
   12. You can click Test Connection to verify that SOAtest can communicate with the coverage agent. 
   13. Enable the Upload coverage report to DTP option.
   14. Configure the Static coverage file location option to specify the path to the static coverage file generated with dotTEST or Jtest in the .xml or .data format (see Creating a Custom Test Configuration).
   15. Configure the Coverage image tags option to specify a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. For details, see the Parasoft DTP user guide at https://docs.parasoft.com.
   16. Enable the Report coverage agent connection failures as test failures option for test failures to be reported when the coverage agent connection fails, or when the static coverage file location is misconfigured. Otherwise, the problems will be reported to the console, but will not cause the test to fail.
   17. Click Apply to save your changes.
3. Configure a SOAtest settings Configure a SOAtest localsettings file that sets:
   • The build id ID (build.id) based on how you want to correlate coverage results. If you want to merge these SOAtest coverage results with those results from other types of testing—such as unit testing and manual testing of the same application—they must all use the same build idID. The build id ID set here must match the one that will be set on the DTP engine build ID set for dotTEST or Jtest when collecting the static and dynamic coverage files and uploading the coverage data.
   • The project (dtp.project). This value must match the dtp.project value that will be set on the DTP engine for dotTEST or Jtest when collecting the static and dynamic coverage files and uploading the coverage data.
   Run a test
   • .

Executing Tests to Collect and Upload Coverage

Run your tests in command line mode using the Test Configuration and

...

settings file you have configured (see Configuring SOAtest to Collect and Upload Coverage). This

...

step should be part of an automated process.

Uploading Test Results to DTP

In order for DTP to correlate coverage results with SOAtest test results, it needs details about what tests were executed along with the test execution results. Make sure that the Enable reporting results to DTP option under Parasoft> Preferences> DTP is enabled. A valid license with the Command Line option is required.

Image Removed

Creating a Combined Coverage Report and Uploading it to DTP

The final step in the workflow is to use Jtest or dotTEST to correlate the static and dynamic coverage data. The report produced by this step determines which source code was covered by the executed tests. To perform this processing and upload the results to Parasoft DTP:

1. Use Jtest or dotTEST to generate a coverage.xml file and upload it to DTP as described in the Jtest or dotTEST documentation. SOAtest stores dynamic coverage data in the location specified in the SOAtest Test Configuration’s Coverage data storage field.

Reviewing Coverage in DTP

Reviewing Coverage in DTP

Go to DTP and open the Coverage Explorer to Click on a coverage widget in DTP to open the Coverage Explorer and review the application coverage achieved by your SOAtest tests. The Coverage Explorer shows the merged correlated coverage from SOAtest and other test execution tools when a coverage report is created as described in the DTP Engines documentation. See the DTP Documentation (Report Center> Explorer Views> Coverage Explorer) for   For details on using the Coverage Explorer.

Image Removed

The tests panel shows details about the file and associated coverage. Test names also link to the Test Explorer in DTP so you can view additional test and coverage information.

Image Removed

Test Impact Analysis with DTP

If you use Parasoft DTP, you can install the Parasoft Process Intelligence Pack and deploy the Test Impact Analysis (TIA) flow, which identifies the test cases directly related to your most recent source code modifications. TIA presents test cases affected by changes so that you can run a minimum set of tests rather than retesting the entire system after each modification. For details, see Test Impact Analysis in the DTP documentation. , see the Parasoft DTP user guide at https://docs.parasoft.com.