Test failure classification streamlines the debugging process by grouping failures with similar root causes. When you label failed tests, a machine learning model learns from those patterns to automatically classify future failures. This reduces repetitive triage work, speeds up investigations, and helps teams focus on fixing the most impactful issues.
Note: For the purposes of this discussion, "failed tests" refers to both failed and incomplete tests.
Labeling Failed Tests
Labeling a failed test helps capture the underlying reason for the failure so it’s easier to spot patterns across runs. Over time, these labels train the machine learning model to automatically classify similar failures.
Add labels to failed tests in the Prioritization tab of the Test Explorer. After you create a label, it will be saved and available as an option for future use.
DTP tracks the association between labels and the underlying reason for failures, which is visible in the visible in the Traces tab.
Tracking Progress of Labeling
...
You can track how labeling is going for your project by using the Test Failure Classification Diagnostics widget.
Viewing Test Failure Classification Results
DTP provides two widgets and a drilldown report to for viewing the test failure classification results. Both widgets drill down to the same report.
- Test Failure Classification widget - shows Shows predicted root cause of all failed tests. It provides a quick overview of why tests may have failed.
- Test Failure Classification Statistics widget - shows Displays the number of files containing failed tests and , total failed files, how many of the failing tests are labeledtests, labeled tests, and the confidence factor. The count of the predictionsfailed files indicates how concentrated the failures are within specific files. Comparing the total failed tests to labeled tests helps you identify how many failures are new, assuming previously failing tests have already been labeled.
- Test Failure Classification Report - shows Shows the health of the model as well as details about the test failure predictions made with the model.
