In environments where only a subset of tests is executed per build, like incremental testing, resource-constrained pipelines, or Test Impact Analysis (TIA) workflows, individual builds do not provide a complete view of test coverage. In these cases, you can get a complete picture of your test coverage by merging partial test and coverage data from multiple incoming builds into one cumulative build. Doing this enables Test Impact Analysis plus tracking of coverage and test results.
Merging Partial Test and Coverage Data
To merge partial test and coverage data from incoming builds into a cumulative build:
- Choose Report Center Settings from the Settings (gear icon) menu.
- Click Projects in the Administration tab and choose the desired project.
- Scroll down to the Test and Coverage Settings section and click Configure Settings.
- In the dialog that opens, enable Aggregate data across builds and enter the name of the cumulative build into which you want to merge data. If the build does not exist, it will be created for you.
Choose a naming convention for your cumulative builds, like "project-name-cumulative-build", to make designating these builds more intuitive.
- Click Save.
- Repeat as necessary for additional projects.
Test Data Merging Behavior
When aggregating test data across multiple runs, DTP determines final test results based on the project’s configuration.
When Aggregate Data Across Builds is Enabled If the project is configured to aggregate data across builds, DTP merges test results by taking the latest result for each individual test within a Run Configuration. Example:
Final Results for Build X:
Summary: DTP retains the most recent result for each test within the same Run Configuration. | When Aggregate Data Across Builds is Disabled If the project is not configured for aggregation, DTP determines results by taking the latest run for each Run Configuration, without merging individual test results across runs. Example:
Final Results for Build X:
Summary: DTP reflects only the results from the most recent run, and tests not executed in that run are not included. |
Coverage Data Merging Behavior
To ensure correct merging of coverage data in a cumulative build, follow these guidelines:
- Don't mix source-based and binary-based static coverage.
- Submit static coverage for a build before sending runtime coverage for that same build.
