Versions Compared

Old Version 1

changes.mady.by.user Anna Stuchlik

Saved on

compared with

New Version Current

changes.mady.by.user Anna Stuchlik

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

If you use a source control system, ensure that your source control settings are properly configured; see Source Control Settings.

Process Overview

Jtest DTP Engine ships with a component called the coverage agent. The coverage agent is attached to the application under test (AUT) and monitors the code being executed as the AUT runs. When the coverage agent is attached to the AUT, a REST API is exposed that enables you to mark the beginning and end of each test and test session. 

...

Test results are also sent to DTP from the tool executing the tests (i.e., SOAtest, tests executed by the DTP Engine , manual tests, etc.) in a report.xml filexml file. If the build IDs for the coverage data file and the report match, DTP is able to correlate the data and display the coverage information. 

Configuring the Application Under Test for Coverage

The following steps are required to prepare the application under test (AUT):

  1. The static coverage file must be generated. The static coverage file contains metadata about user classes, methods, and lines; see see Generating the Static Coverage File.
  2. The coverage agent must be attached to the AUT; see see Attaching the Coverage Agent to the AUT.
  3. The coverage agent includes default settings for outputting files, determining scope, etc., but you can set properties in a configuration file to meet your application coverage goals; see see Configuring the Coverage Agent.

...

  1. Extract the contents of the monitor.zip package to the server machine.

  2. Run the agent.sh/agent.bat script from the monitor package to generate the Jtest Java agent VM arguments. The scripts output the the -javaagent VM argument to the console. You will need this argument to connect the agent to the AUT, which will result in the runtime_coverage subdirectory: 

    Code Block
    Jtest Agent VM argument:
-javaagent:"[path to agent dir]\agent.jar"=settings="[path to agent properties file]\agent.properties",runtimeData="[path to monitor dir]\monitor\ runtime_coverage"
  3. Add the Add the -javaagent flag to  flag to the application server’s startup script and restart the server (see step 5c in Web Application Coverage Tutorial). Attaching the coverage agent to the AUT exposes a REST API for controlling the agent. 
  4. Enter the following URL into the browser bar to verify that the coverage agent is set:
    [application_host]:8050/status/
    If the application is properly configured, the API will return data:

...

Application servers usually contain more than one application. Additionally, common server classes or application libraries do not need to be instrumented. The Jtest DTP Engine only needs to collect coverage for application source code. Instrumenting all classes would be too time-consuming.

...

  • the agent.properties file
  • arguments to to -javaagent

The agent.properties file is generated in the monitor.zip package ( see see Generating the Static Coverage File). It contains properties that can be modified to properly configure the coverage agent. The following example shows how the coverage agent can be configured with the agent.properties.file:

Code Block
jtest.agent.runtimeData=[path to runtime_coverage directory]
jtest.agent.includes=com/myapp/data,com/myapp/common/**
jtest.agent.excludes=com/myapp/transport/*,com/myapp/autogen/**
jtest.agent.autostart=false

Adding properties to -javaagent requires  requires modifying their names by removing the "jtest.agent" prefix, for example:

...

PropertyDescripton
jtest.agent.runtimeData

Specifies where the runtime data will be stored. The following example will create files in the in ‘C:/tmp/myapp/’ directory with runtime_’ as the name prefix:

'C:/tmp/myapp/runtime'

jtest.agent.includes 

A comaAcoma-separated list separatedlist of patterns that specify classes to be instrumented. The following wildcards are supported:

* matches zero or more characters
** matches multiple directory levels

In the following example, all classes from the com.myapp.data package and all classes from package and subpackages that start with com.myapp.common will be instrumented:

com/myapp/data/*,com/myapp/common/**

jtest.agent.excludes

Coma separated list separatedlist of patterns that specify classes to be excluded from instrumentation. The following wildcards are supported:

* matches zero or more characters
** matches multiple directory levels

In the following example, all classes from the com.myapp.transport package and all classes from package and subpackages that start with com.myapp.autogen will be excluded from instrumentation:

com/myapp/transport/*,com/myapp/autogen/**

jtest.agent.autostartEnables/disables automatic runtime data collection; the default is true.
jtest.agent.port Sets up agent communication port; the default is 8050.
jtest.agent.debug Enables/disables verbose output to console; the default is false.
jtest.agent.collectTestCoverageEnables/disables collecting coverage information for test cases. The default value is false. 
jtest.agent.enableMultiuserCoverage

Enables/disables collecting web application coverage for multiple users; the default value is false. 

Setting this property to true allows you to collect multi-user coverage with Coverage Agent Manager. See the "Coverage Agent Manager (CAM) section of the DTP documentation for details.

jtest.agent.associateTestsWithCoverageEnables/disables associating coverage with particular tests; the default value is true.
jtest.agent.testidSpecifies the initial test ID the coverage will be associated with when the agent is run.

Test Configuration and Execution

You can use SOAtest to run functional tests (refer the Application Coverage chapter of the SOAtest documentation to set up the test configuration), as well as execute manual tests. At the end of the test session, coverage will be saved in saved in runtime_coverage_[timestamp].data files in  files in the directory specified in SOAtest. This information will eventually be merged with the static coverage data to create a a coverage.xml file  file and uploaded to DTP. 

Uploading Test Results to DTP

...

  1. Go to Report Center in the DTP interface.
  2. Click the gear icon and choose Report Center Settings> Additional Settings> Report Center Administration> Tools> Data Collector Upload Form (requires admin permissions).
  3. Click Choose File and browse for the report.xml file xml file you downloaded from CAM.
  4. Click the Upload button to upload the file to DTP.

...

Generating a Dynamic Coverage Data File and Uploading It to DTP

  1. Ensure that Jtest DTP Engine is properly configured, including DTP, scope and authorship scope and authorship settings. See Connecting to DTP 1, Sending Results and Publishing Source Code to DTP, Configuration 1.
  2. Configure the following settings in the jtestclithe jtestcli.properties file in order to properly merge coverage data:
    - report.coverage.images -  Specifies a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. DTP supports up to three coverage images per report.
    - session.tag - Specifies a unique identifier for the test run and is used to distinguish different runs on the same build.
    - build.id - Specifies a build identifier used to label results. It may be unique for each build, but it may also label several test sessions executed during a specified build.
  3. Copy the runtime coverage and static coverage files to the same machine and run jtestcli with the following switches:
    • -runtimeCoverage: Specifies the path to runtime coverage that you download with CAM (see Coverage Agent Manager (CAM) section of the DTP documentation for details). You can provide a path to an individual .data file with coverage information from one testing session, or a path to a folder that contains many .data files from multiple testing sessions.
    • -staticCoverage: Specifies the path to the static coverage file (see Generating the Static Coverage File).
    • -config: Specifies the URL of
    Run
    • the Calculate Application Coverage
    test configuration using the following arguments:
    •  test configuration that must be run to merge the coverage data.
    • -publish: Sends the merged coverage to DTP.
      Code Block
      jtestcli -staticcoverage [path to static_coverage.xml file] -runtimecoverage [path/dir] -config "builtin://Calculate Application Coverage" -publish

      This ensures that Jtest has access to the runtime coverage data generated during test execution, as well as the static coverage data, which is required to fill the coverage.

    xml file

    • xml file with runtime coverage data

    ,

    • .

Reviewing Coverage in DTP

...

  • If multiple users are simultaneously accessing the same web application, the coverage data they collect may be mixed. To ensure that coverage is properly associated with individual users, the multiuser mode must be enabled (see see Configuring the Coverage Agent).
  • The HTTP or HTTPS protocols are required to enable the multiuser mode, as the user-specific information must be provided with the HTTP header.
  • In the multiuser modethe multiuser mode, the "default" user (the user who has not specified their ID) may collect extra coverage information from other users who are accessing the same web application.
  • In the multiuser modethe multiuser mode, assigning coverage collected for multithreaded application to individual users is limited. Coverage data for child threads is not threads is not assigned to the user who is actually accessing the application, but to the "default" user.
  • Coverage data collected for web application initialization is not assigned to a specific user, but to the "default" user.

...