In this section:
Azure DevOps is Microsoft's Software as a service (SaaS) platform that provides an end-to-end DevOps programming tools for developing and deploying software. Connecting DTP to Azure DevOps Server Express and Azure DevOps Server provides the following functionality:
The configuration is performed by the Parasoft administrator and only needs to be set up once.
Sign in to your organization in Azure DevOps (https://dev.azure.com/{yourorganization})
From your home page, open your user settings, and then select Personal access tokens.
Select the scopes for this token to authorize for your specific tasks.
Once your Personal access token is created, you can use it anywhere your user credentials are required for authentication in Azure DevOps.
You can configure DTP to generate widgets and reports that help you demonstrate traceability between the requirements stored in Azure DevOps and the test, static analysis, and build review data sent to DTP from Parasoft tools (C/C++test, dotTEST, Jtest).
If you want the Traceability Report to include code review and static analysis information, you must associate your source code files with requirements stored in Azure DevOps. See Associating Requirements with Files for instructions on enabling this optional feature.
DTP interfaces that display and track traceability are enabled by deploying the External System Traceability Report artifact shipped with the Traceability Pack. The Traceability Pack also includes the Sending Test Data to External System flow, which automates part of the requirements traceability workflow.
Use DTP Extension Designer to deploy the External System Traceability Report and the Sending Test Data to External System flow to your environment. Verify that DTP is connected Azure DevOps as described in the Connecting DTP to Azrue DevOps section before deploying the artifacts.
The first step is to install the Traceability Pack artifact. The artifact is a collection of configuration files and assets that enable traceability.
The following requirements are only applicable if you are going to send test results to Azure DevOps.
After configuring the integration with Azure DevOps, developers, testers, and other users can leverage the functionality enabled by the integration.
The Test Explorer and Violations Explorer views enable you to create issues and defects for any test and violation, respectively, regardless of status. Refer to the following sections for details on creating Azure DevOps assets in explorer views:
The following diagram shows how you could implement an automated infrastructure for integrating Parasoft DTP and Parasoft test execution tools into your Azure DevOps environment:
Azure user configuration requirement: user access level must include "Test Plans" in Azure DevOps in order for Parasoft DTP to be able to send test results to Azure DevOps. This is required because Parasoft DTP creates test plans when it sends the results to Azure DevOps.
To set up a Test Plan in Azure DevOps for the local version you need to:
To set up a Test Plan in Azure DevOps for the cloud you need to:
|
To send test data to Azure DevOps:
In your test file, map the Azure DevOps system requirement IDs to the tests using the following annotation format:
For users who have not created their own test case definition to see traceability results in Azure DevOps, use @req:
|
Use @req in Parasoft and trigger sending results from Parasoft DTP to
For users who have created their own test case definition in Azure DevOps, use @test:
* The following annotation maps the test to a requirement:* @test <<Azure DevOps ID> > @TEST... |
Associate each test case definition with your tested requirement in Azure DevOps.
Use the @test <Azure DevOps ID> annotation to associate tests with test definitions in Azure DevOps. When using @test make sure that you have no test case steps defined in the tests whose run statuses you plan to send from Parasoft DTP to Azure DevOps. Parasoft DTP is not able to update test case runs for test cases that have steps defined.
@req <Azure DevOps ID> annotation to associate tests with stories in Azure DevOps. When using @req make sure the following relation type is defined in the Azure DevOps project you plan to send results to: "verified by". Such relation is needed by Parasoft DTP when it creates a test case in Azure DevOps and relates it to a requirement in Azure DevOps.
If you deployed the Sending Test Data to External System flow (see Deploying the Sending Test Data to External System Flow), then unit and functional testing results will automatically be sent to Azure DevOps when Data Collector receives the data from the Parasoft tool. By default, the flow forwards unit and functional test results that were received by Data Collector for any project, but you can configure the flow to only send data for a specific project (see Sending Results from a Specific DTP Project).
You can also manually send a POST request to the DTP REST API endpoint to send results from the DTP database to Azure DevOps. Pass the IDs for the DTP filter and build, as well as the ID for a set of Azure DevOps test cases, as URL parameters in the API call:
curl -X POST -u <username>:<password> "http://<host>:<port>/grs/api/v1.7/linkedApps/configurations/1/syncTestCases?filterId=<filterID>&buildId=<buildID>&testCaseSetId=<Azure DevOps_TEST_CASE_SET_ID>" |
The filter and build IDs are available in the Test Explorer URL:
The following table describes the endpoint parameters.
| Parameter | Value | Description | Required |
|---|---|---|---|
filterId | integer | Specifies the filter ID containing the test data. The filter ID is an integer value and should not be confused with the filter name. | Required |
buildId | string | Specifies the build ID containing the test data. | Required |
| boolean | Setting to Setting to Default is | Optional |
testCaseSetId | ID of set of test cases | Specifies the ID of a test case set in Azure DevOps where Parasoft DTP should create new test case definitions, if not found in Azure DevOps. When a test case definition is found in Azure DevOps, whether in this The value for the testCaseSetID parameter should be an ID for a "Set of Test Cases" item type in Azure DevOps (see Requirements). You must use the ID from Azure DevOps URL and not the ID generated in the UI. You can get the ID from the URL after clicking on the test cases node in the project tree. | Optional |
The set of test cases is the item where DTP can create new test cases as necessary. DTP will create new test cases when it sends results to Azure DevOps if equivalent test cases do not exist, as described in the next step.
filterId and buildId parameters and send the data to the Azure DevOps system requirements.An external-app-sync.log file will also be written to the the <DTP_INSTALL>/logs directory. This log file contains progress information about sending test results from DTP to Azure DevOps.
After DTP processes the report and sends results to Azure DevOps, you should expect a response similar to the following:
{
"createdTestSession": "DTPP-521",
"created": [
"DTPP-519, testName = testBagSumAdd"
],
"updated": [
"DTPP-519, testName = testBagSumAdd",
"DTPP-518, testName = testBagSimpleAdd"
],
"ignored": [
"MAGD-567, testName = testBagNegate",
"QAP-512, testName = testTryThis3",
"QAP-512, testName = testTryThis4",
"MAGD-567, testName = testBagMultiply"
]
} |
Sending Results from a Specific DTP Project
If you are using the Sending Test Data to External System flow to forward unit and functional test results, data will be sent to Azure DevOps for all DTP projects by default. As a result, work items will be updated to include the tests collected for any DTP project that contain annotations matching Azure DevOps IDs. You can configure the flow, however, to only send data for a specific project.
Double-click the node and specify the following string in the Property field:
event.message.resultsSession.project |
When the flow executes, only test results for the specified DTP project will be sent to Azure DevOps.
You will be able to view results in Azure DevOps after sending the test data. The following image shows a System Requirement Specifications requirement in Azure DevOps. The requirement contains several test cases.
You can drill down into a test case to view details, such as test runs for the test case.
If the External System Traceability Report has been deployed to your system (see Enabling the Requirements Traceability Report), you can add widgets to your dashboard to monitor traceability from Azure DevOps requirements to Parasoft DTP tests, static analysis, code reviews for your project. The widgets also drill down to a report that includes additional details.
The widgets will appear in a separate Traceability category when adding widgets to your DTP dashboard. See Adding Widgets for general instructions on adding widgets.
You can configure the following settings:
| Title | You can enter a new title to replace the default title that appears on the dashboard. |
|---|---|
| Filter | Choose Dashboard Settings to use the dashboard filter or choose a filter from the drop-down menu. See Creating and Managing Filters for additional information about filters. |
| Target Build | This should be set to the build ID you executed the tests and code analysis under. You can use the build specified in the dashboard settings, the latest build, or a specific build from the drop-down menu. Also see Configuring Dashboard Settings. |
| Azure DevOps Project | Choose an Azure DevOps project from the drop-down menu. |
This widget shows the number of requirements from the specified Azure DevOps project.
Click on the widget to open the Requirement Traceability report.
This widget shows the percentage of requirements covered by tests against all requirements in the project.
Click the center of the widget to open the main Requirement Traceability report.
The colored-in segment represents the requirements covered by tests. Click on the segment to open the Requirement Traceability report filtered to the With Tests category.
Unit testing, functional testing, static analysis, and peer reviews are common activities for verifying that requirements have been properly and thoroughly implemented. This widget shows the overall status of the project requirements in the context of those software quality activities. You can add a widget for each type of quality activity (tests, static analysis violations, reviews) to monitor the progress of requirements implementation for the project.
Mouse over a section of the chart to view details about quality activity type status. Click on the widget to open the Requirement Traceability report filtered by the selected type.
The report lists the Azure DevOps requirements and Parasoft DTP tests associated with them.
You can perform the following actions:
Clicking on a section of the Azure DevOps Requirements - Pie widget opens a version of the report that includes only the quality activity type selected in the widget. You can use the drop-down menus to switch type and status. You can also disable or enable the Show files/reviews option if you want to hide the Files and Reviews columns in the report. The Files and Reviews columns will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). Disabling the Files and Reviews columns on this screen hides the related tabs in the Requirement Details report.
The Azure DevOps Requirement Details report provides additional information about the files, static analysis findings, and tests associated with a specific Azure DevOps requirement. You can open this report by clicking on a requirement in the Azure DevOps Requirement Traceability report.
The first tab shows the results of the tests that were executed to verify the specific work item.
You can click on the View results in Test Explorer link to view all of the tests associated with the work item in the Test Explorer.
You can also click on individual test names in the table to view each test in the Test Explorer.
The second tab shows the files associated with the specific requirement, as well as the static analysis violation detected in the files. You can click the link the Violations column to view the violations in the Violations Explorer, which provides additional details about the violations.
This tab will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.
If the files include any change reviews or review findings, they will be shown in the third tab with links to view them in the Change Explorer.
This tab will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.