Page History

...

- CTP exposes a REST API to integrate with your test framework, allowing your test framework to notify CTP when tests are starting and stopping so that the appropriate code coverage can be collected for each individual test case that gets executed.
- CTP exposes a REST API that can be triggered at the end of a testing session so that the runtime coverage data as well as test pass/fail results can be published to DTP.
- (Optional) Set Create baseline build in CTP baseline

Review Phase

Step 5: Review test reports and application coverage in DTP.

...

The jtestcov tool can be found in the downloaded java_agent_coverage zip, under the jtestcov directory. You will need to create a properties file to add DTP Properties for coverage information. See DTP Properties for coverage section for details.

...

Code Block

java -jar <PATH_TO_jtestcov.jar> -ctp -app c:/<PATH_TO_APPLICATION>/parabank.war -include com/parasoft/parabank/parasoft/** -settings c:/<PATH_TO_LICENSE_FILE>/license TO .PROPERTIES>/jtestcov.properties

Info
The `-ctp` flag is used by the coverage tools to pass your CTP license, which is needed when you run the coverage tool independently of CTP as in the example above. When you run soatestcli with application coverage settings, the license is automatically passed when the coverage tool is launched. You can see all available commands and options by running: `java -jar jtestcov.jar help`

Generating static coverage with dottestcov

The dottestcov tool is used similarly to the jtestcov tool detailed above. It can be found in the downloaded dotnet_agent_coverage zip, under the dottestcov directory.

Invoke dottestcov using dottestcov.bat with a command like the example below:

Code Block

dottestcov.bat -ctp -include "C:\Devel\FooSolution\src\QuxProject***.cs" -exclude "C:\Devel\FooSolution\src\QuxProject*\tests**.cs" -app <DIR>\FooSolution.sln -publish -settings <PATH TO .PROPERTIES>\dottestcov.properties

Info
The `-ctp` flag is used by the coverage tools to pass your CTP license, which is needed when you run the coverage tool independently of CTP as in the example above. When you run soatestcli with application coverage settings, the license is automatically passed when the coverage tool is launched.

Code Block

title	Sample License Properties File

# === LICENSE ===

Code Block

title	Sample License Properties File

# === LICENSE ===
 
# === END USER LICENSE AGREEMENT ===
# Set to true to accept the Parasoft End User License Agreement (EULA).
# Please review the EULA.txt file included in the product installation directory.
#parasoft.eula.accepted=false
 
# === END LOCALUSER LICENSE AGREEMENT ===
# Enables local license - be sure to specify password.
#jtest.license.use_network=false
 
# Specifies password for the local license.
#jtest.license.local.password=[LICENSE PASSWORD]
 Set to true to accept the Parasoft End User License Agreement (EULA).
# Please review the EULA.txt file included in the product installation directory.
parasoft.eula.accepted=true

# === NETWORK LICENSE ===
# Enables network license - be sure to configure DTP server settings.
#jtestctp.license.use_network=true
 ctp.license.network.edition=custom_edition
# SpecifiesNote: typecustomize ofthe theCTP networkcoverage license (edition).
# Supported editions: desktop_edition, desktop_compliance_edition, server_edition, server_compliance_edition
#jtest.license.network.edition=server_edition
 
# Enables specific list of license features
#jtest.license.network.edition=custom_edition
#jtest.license.agent tier based on what your CTP license has enabled
ctp.license.custom_edition_features=JtestCTP, Automation, Desktop Command Line,Coverage Tier 5 

# === DTP Publish, Coverage, Test Impact Analysis
 
 
# === DTP SERVER SERVER SETTINGS ===
 
# Specifies URL of the DTP server in the form https://host[:port][/context-path]
#dtp.url=https://localhost:8443
 
# Specifies user name for DTP server authentication.
#dtp.user=admin
 
# Specifies password for DTP server authentication - use jtestcli -encodepass <PASSWORD> to encode the password, if needed.
#dtp.password=admin
 
# Specifies name of the DTP project - this settings is optional.
#dtp.project=[DTP Project Name]
 
# === DTP REPORTING ===
 
# Enables reporting test results to DTP server - be sure to configure DTP server settings.
#report.dtp.publish=true
 
# Specifies a build identifier used to label results. It may be unique for each build
# but may also label more than one test sessions that were executed during a specified build.
#build.id=${dtp_project}-yyyy-MM-dd
 
# Specifies a tag which represents an unique identifier for the run, used to distinguish it from similar runs.
# It could be constructed as minimal combination of following variables that will make it unique or specified manually.
# e.g. ${config_name}-${project_module}-${scontrol_branch}-${exec_env}
#session.tag=[tag]
 
# Specifies a set of tags that will be used to create coverage images in DTP server.
# Coverage images allow you to track different types of coverage, such as coverage for unit, functional, manual tests and others.
# There is a set of predefined tags that will be automatically recognized by DTP, see the examples below.
# You can also specify other tags that will be used to create coverage images.
#report.coverage.images=${dtp_project}
#report.coverage.images=${dtp_project};${dtp_project}_Unit Test
#report.coverage.images=${dtp_project};${dtp_project}_Functional Test
#report.coverage.images=${dtp_project};${dtp_project}_Manual Test
 
# === CONSOLE VERBOSITY LEVEL ===
# Increases console verbosity level to high.
#console.verbosity.level=high

Generating static coverage with dottestcov

The dottestcov tool is used similarly to the jtestcov tool detailed above. It can be found in the downloaded dotnet_agent_coverage zip, under the dottestcov directory.

Invoke dottestcov using either dottestcov.sh or dottestcov.bash, depending on your operating system, with a command like the example below running against Parabank:

Code Block
dottestcov(.sh/.bash) coverage -ctp -app c:/<PATH_TO_APPLICATION>/parabank.war -include com/parasoft/parabank/parasoft/** -settings c:/<PATH_TO_LICENSE_FILE>/license.properties

Info

The -ctp flag is used by the coverage tools to pass your CTP license, which is needed when you run the coverage tool independently of CTP as in the example above. When you run soatestcli with application coverage settings, the license is automatically passed when the coverage tool is launched.

 === CONSOLE VERBOSITY LEVEL ===
# Increases console verbosity level to high.
#console.verbosity.level=high

Anchor

	NoteIncludesExcludes
	NoteIncludesExcludes

Note

title	A Note About Includes and Excludes

Defining includes and excludes is an important part of controlling how much your system processes, which can greatly affect how long it takes to generate your static coverage files. affect how long it takes to generate your static coverage files. In many cases, there is no need to measure code coverage of third-party library code, so it is preferable to limit code coverage to project code using these settings. Includes and excludes should be expressed as comma-separated lists of patterns that specify classes to be instrumented. The following wildcards are supported:

* matches zero or more characters
** matches multiple directory levels

In the following example, all classes from the com.myapp.data package and all classes from package and subpackages that start with com.myapp.common will be instrumented:

-include com/myapp/data/*,com/myapp/common/**

While in this example, all classes from the com.myapp.transport package and all classes from package and subpackages that start with com.myapp.autogen will be excluded from instrumentation:

-exclude com/myapp/transport/*,com/myapp/autogen/**/**

If you are outside of development, working with the application binaries, and not sure what the appropriate include/exclude settings should be, reach out to your colleagues in development to make sure you use the correct patterns. There is usually a standard pattern for packaging company code, like com.myapp.**, that should help you determine the right setting to use.

Verifying static coverage has been uploaded to DTP

...

Attaching the Coverage Agent to a microservice allows you to enable collecting dynamic (runtime) coverage for it. Typically, this is done as part of an automated deployment process coming from a CI/CD pipeline. . As part of this process, you will have the opportunity to review the agent's settings in the agent.properties file; while doing so, ensure that the include and exclude values that are used are the same as those defined in your static coverage process.

Attaching the Coverage Agent for Java

The agent.jar file for the agent as well as the agent.properties file for configuring it can be found in the downloaded java_agent_coverage zip, under the jtest_agent directory.

An argument of the form:

...

Once started, you can check that the agent is running by going to the agent's status endpoint at http://<HOST>:<PORT>/status, where host <HOST> is where the microservice is running and port <PORT> is specified in in agent.properties; by default, it is 8050.

...

You can use the Coverage Wizard tool found in the dottest_agent folder of the downloaded dotnet_agent_coverage zip to set up coverage agents. It can be invoked directly to provide agent configuration via a GUI, or alternatively run using the command line; see the documentation here for more details: Application Coverage for Standalone Applications (dotTEST) or Application Coverage for Web Applications (dotTEST).

...

Once started, you can check that the agent is running by going to the agent's status endpoint at http://<HOST>:<PORT>/status, where host <HOST> is where the microservice is running and port <PORT> is specified in in agent.properties; by default, it is 8050.

...

Note
It is critical to ensure the DTP Project, Build ID, and Coverage Image Tags in the environment match the properties used to generate your static coverage. Otherwise, DTP may fail to associate the runtime and static coverage.

4c:

...

Create baseline build in CTP

...

(Optional)
Anchor
SetCTPBaseline
SetCTPBaseline

This step is only required if you are implementing the Test Impact Analysis workflow outlined in the following section. Test Impact Analysis requires a baseline build with coverage from a full regression test run to use for comparison when calculating impacted tests from a subsequent build. To mark a build as a baseline build to be compared against, send an additional REST request at the end of your test session to the CTP API endpoint at

...

Page tree

Versions Compared

Old Version 5

New Version Current

Key

Generating static coverage with dottestcov

Generating static coverage with dottestcov

Verifying static coverage has been uploaded to DTP

Attaching the Coverage Agent for Java

4c:

Create baseline build in CTP

(Optional)
Anchor
SetCTPBaseline
SetCTPBaseline

Page tree

Page History

Versions Compared

Old Version 5

New Version Current

Key

Generating static coverage with dottestcov

Generating static coverage with dottestcov

Verifying static coverage has been uploaded to DTP

Attaching the Coverage Agent for Java

4c:

Create baseline build in CTP

(Optional) AnchorSetCTPBaselineSetCTPBaseline

(Optional)
Anchor
SetCTPBaseline
SetCTPBaseline