In this section:
Introduction
Test Impact Analysis calculates testing metrics that are affected by code changes and reports the data in DTP interfaces. Test Impact Analysis collects all files in a code base and analyzes the test cases associated with each file. If a file has been modified within the specified date range and the test cases associated with it have not run since the change, then the test case "Action" field will be set to "Retest" in the Test Explorer. Otherwise, the test case "Action" will not be modified.
In order to generate accurate results, the test case information is merged across two builds. Baseline and Target build ID are configured through drop-down menus when the widget is created. Both builds require coverage and dynamic analysis data for the widget to render properly.
Resubmit data if updating Test Impact Analysis
Coverage data stored in previous versions of DTP (5.3.2 and older) cannot be used to calculate testing metrics used in this extension. You will need to send a new test and coverage report to DTP server and set the build containing the new reports as the baseline when configuring the widget.
You can resubmit a previous build report to the data collector to populate the data. Once the data is in DTP, make sure to archive the build so that it won't be removed during normal database clean up (see Locking and Archiving Builds).
Comparing builds from different branches is not currently supported. Tests from the development branch, for example, are considered different tests from master branch.
Requirements
- Parasoft DTP and Extension Designer 2020.1.
- Test and coverage data from the following Parasoft tools are supported:
- C/C++test Standard, dotTESt, and Jtest 10.4.3 or later.
- C/C++test Professional 10.4.3 or later.
- SOAtest 9.10.7 or later.
- The DTP filter must be configured to receive Run Configurations that contain test and coverage data. See Associating Coverage Images with Filters. You can confirm that the filter and build meet these requirements by looking at the Build Administration widget:
Installation
The assets that enable Test Impact Analysis are installed as part of the the Process Intelligence Pack installation. Refer to the Process Intelligence Pack installation instructions for details. After installing the flow, deploy the report to your DTP environment.
- If you have not already done so, install the Process Intelligence Pack.
- Open Extension Designer and click the Services tab.
- Expand the Process Intelligence Engine service category. You can deploy assets under any service category you wish, but we recommend using the Process Intelligence Engine category to match how Parasoft categorizes the assets. You can also click Add Category to create your own service category (see Working with Services for additional information).
- You can deploy the artifact to an existing service or add a new service. The number of artifacts deployed to a service affects the overall performance. See Extension Designer Best Practices for additional information. Choose an existing service and continue to step 6 or click Add Service.
- Specify a name for the service and click Confirm.
- The tabbed interface helps you keep artifacts organized within the service. Organizing your artifacts across one or more tabs does not affect the performance of the system. Click on a tab (or click the + button to add a new tab) and click the vertical ellipses menu.
- Choose Import> Library> Workflows> Process Intelligence> Test Impact Analysis and click anywhere in the open area to add the the artifact to the service.
- Click Deploy to finish deploying the artifact to your DTP environment.
- Return to DTP and refresh your dashboard. You will now be able to add the related widgets.
Caching the Data
Because you can run the Test Impact Analysis slice over extended periods of time, the slice includes a caching mechanism to speed up multiple requests for the same data. When data is requested, the slice first determines if the data is already computed and cached. If the cache exists, the data is returned directly and the lengthy computation is skipped. The cache is cleared and recomputed at runtime, however, under the following conditions:
- no data is cached
- the cached data is associated with a different build combination
- additional analysis data has been reported to the build combination
There is one cache per filter and combination of baseline and target build.
Clearing the Cache
Because the slice does not automatically remove cached data, the cache can grow as more filters are introduced. To help you clear out old cache data, the slice provides a way to delete all cached calculations from the Enterprise Pack database. This flow cleans all cached calculations. The cache also clears at 00:00 every day. You can configure the auto cache clearing setting by editing the Clean Cache inject node.
Widget Configuration
The Test Impact Analysis slice ships with the Tests Impacted by Change - Pie Chart widget. This widget shows the number of tests that passed, failed, are incomplete, and require rerunning. See Adding Widgets for details on adding widgets to a dashboard in DTP.
| Title | You can change the default title of the widget. |
|---|---|
| Filter | Choose a filter from the drop-down menu. A filter is a collection of test execution and code analysis metadata configurations that enables DTP to aggregate, subset, and analyze specific swathes of data. |
| Period | Choose a range of time or number of builds to set as the scope. |
| Baseline Build | The build that you want to compare to the target build |
| Target Build | The build you want to analyze; typically, this is the latest build |
| Coverage Image | Coverage images are identifiers for the coverage data associated with a test run. The filter must be set to the correct coverage image to present information. |
Check Build Administration to Get the Correct Build
By default, the Baseline Build and Target Build are configured to use the dashboard settings. The slice will automatically select the two most recent builds, but these builds may not contain test and coverage details. You should check the Build Administration page in DTP and use an appropriate baseline and target build when configuring the widget as described in the Requirements section. Also see Build Administration.
The widget shows the data in a pie chart.
Troubleshooting
If there are any issues with the data, the widget will print an error describing the problem. The following error messages are examples may appear:
To address the errors:
- Verify that the coverage information is accessible (see Requirements).
- Verify that the filter has a coverage image associated with it (see Associating Coverage Images with Filters).
- Verify that the correct coverage image was selected when you added the widget (see Widget Configuration).
Exploring the Results
Click on a region in the Tests Impacted by Change widget to open the Files Impacted by Change drill-down report.
You can perform the following actions in this report:
- Click on a link in the File Name, Pass, Fail, or Incomplete column opens the Test Explorer view with the latest build ID.
- Click on a link in the Retest column opens the Test Explorer with the baseline build ID.
