In this section:
Introduction
The Change Based Testing slice calculates change-based testing metrics that can be displayed in multiple forms (pie chart, table report, and Test Explorer). It gathers all the files in a code base and analyzes the test cases associated with each file. If the file has been modified between the baseline and target build and the test cases associated with that file have not been run since the change, then the test case “Action” will be set to “Retest.” Otherwise, the test case “Action” will not be modified.
In order to generate accurate results, the test case information is merged across two builds. Baseline and Target build ID are configured through drop-down menus when the widget is created; the slice expects both builds to have coverage and dynamic analysis data.
When analysis is completed, the results can be viewed in pie chart format for test summaries (total pass, fail, incomplete, retest statistics), the table report for test summaries per file, and the DTP test explorer.
Resubmit data if updating Change Based Testing
If you have upgraded to DTP 5.3.3, coverage data stored in previous versions cannot be used to calculate change-based testing metrics. You will need to send a new test and coverage report to DTP server to and set it as a baseline build to compare with a target build when configuring the widget.
You can resubmit a previous build report to the data collector to populate the data. Once the data is in DTP, make sure to archive the build so that it won't be removed during normal database clean up (see Locking and Archiving Builds).
Comparing builds from different branches is not currently supported. Tests from the development branch, for example, are considered different tests from master branch.
Requirements
- Parasoft Extension Designer 5.3.3 or higher.
- Parasoft DTP 5.3.3 or higher registered in the DTP Server configuration page.
- A filter must be configured in DTP to receive Run Configurations from Coverage and Dynamic Analysis (unit testing, functional testing, manual testing) runs. See Associating Coverage Images with Filters.
- DTP enables you to constrain how much coverage data is processed. Verify that the coverage image associated with the data you want to process is accepted. See Controlling Coverage Data Processing for Enterprise Pack Artifacts.
You can confirm that the filter and build meet these requirements by looking at the Build Administration widget:
Definitions
| DTP Project | The root level at which the analysis tools report data. Comes preconfigured with a “filter” that gathers all the Run Configurations that are reported to the project. |
|---|---|
| Filter | A DTP configuration that organizes how the data from different analysis runs is displayed in dashboard widgets and reports. |
| Run Configuration | A run configuration represents a series of runs (test executions and/or analyses). Runs tagged with the same code analysis and test execution tool, DTP project, test configuration, and session tag are grouped into the same run. Run configurations can narrow the data shown for a given filter. |
| Baseline Build | The build that you want to compare to the target build |
| Target Build | The build you want to analyze; typically, this is the latest build |
See DTP Concepts for additional information.
Installation
See Downloading and Installing Artifacts for instructions on installing DTP Enterprise Pack extensions.
Caching the Data
Because you can run the Change Based Testing slice over extended periods of time, the slice includes a caching mechanism to speed up multiple requests for the same data. When data is requested, the slice first determines if the data is already computed and cached. If the cache exists, the data is returned directly and the lengthy computation is skipped. The cache is cleared and recomputed on the fly, however, if no data is cached, if the cached data is associated with a different build combination, or if additional analysis data has been reported to the build combination. There is one cache per filter and combination of baseline and target build.
Clearing the Cache
Because the slice does not automatically remove cached data, the cache can grow as more filters are introduced. To help you clear out old cache data, the slice provides a way to delete all cached calculations from the PIE database. This flow cleans all cached calculations. The cache also clears at 00:00 every day. You can configure the auto cache clearing setting by editing the Clean Cache inject node.
Widget Configuration
The Change Based Testing slice ships with the Change Based Testing - Pie Chart widget, which allows you to view the change-based testing summary totals, such as the total number of passed, failed, incomplete, and retest test cases. See Adding Widgets for details on adding widgets.
Check Build Administration to Get the Correct Build
By default, Baseline Build is set to Previous Build and Target Build is set to Latest Build. The slice will automatically select the two most recent builds, but these builds may not contain test and coverage details. You should check the Build Administration page in DTP and use an appropriate baseline and target build when configuring the widget as described in the Requirements section. Also see Build Administration.
The widget shows the data in a pie chart.
Troubleshooting
You will receive an error instead of a rendered widget if the coverage information is inaccessible (see Requirements).
The following error is returned when the filter has a coverage tag that does not contain coverage information:
Verify that the filter has a coverage image associated with it (see Associating Coverage Images with Filters) and that Data Collector is configured to accept the coverage image (see Controlling Coverage Data Processing). New reports must be sent to DTP after properly configuration.
The following error is returned when Data Collector is configured to accept your coverage data, but the specific coverage image does not contain data.
Verify that the filter has a coverage image associated with it (see Associating Coverage Images with Filters) and that the correct coverage image was selected when you added the widget (see Widget Configuration).
Exploring the Results
Click on a region in the Change Based Testing - Pie Chart widget to open a custom drill-down report.
You can perform the following actions in this report:
- Click on a link in the File Name, Pass, Fail, or Incomplete column opens the Test Explorer view with the latest build ID.
- Click on a link in the Retest column opens the Test Explorer with the baseline build ID.
