In this section:

Introduction

Azure DevOps is Microsoft's Software as a service (SaaS) platform that provides an end-to-end DevOps programming tools for developing and deploying software. Connecting DTP to Azure DevOps Server Express and Azure DevOps Server provides the following functionality:

• Ability to manually create defects in Azure DevOps from the Violations Explorer and Test Explorer views.
• Ability to send, view, and update Parasoft test results in Azure DevOps system requirements (Sending Test Data to Azure DevOps).
  • For users who plan to track traceability using traceability reports in Azure DevOps. In order to do this, you need to first send test results from Parasoft DTP to Azure DevOps.
• Traceability from Azure DevOps requirements to tests, static analysis results, and code reviews (see Viewing Results in Azure DevOps).
  • For users who plan to track traceability using traceability reports in Parasoft DTP. In order to do this, use @req to associate your automated tests in Parasoft to Azure DevOps requirements.

Configuration

The configuration is performed by the Parasoft administrator and only needs to be set up once. 

Connecting DTP to Azure DevOps

1. Choose Report Center Settings from the settings (gear icon) drop-down menu.
2. Choose External System and choose Azure DevOps from the System type drop-down menu.
3. Enable the Enabled option.
4. Enter the a name for the server in the Name field. The name is required but does not affect the connection settings or render in any other interfaces.
5. Enter the URL of your Azure DevOps system in the Application URL field. Must include your Azure DevOps collection name.
6. The Display URL field defines the URL which is displayed in Parasoft DTP pages when links to your Azure DevOps system are presented in a web browser. Typically, this should be the same as the above Application URL field. However, it might be different, for example, when you work in a reverse proxy environment and links to Azure DevOps from the user's local web browser with Parasoft DTP are different than from the Parasoft DTP server. (e.g., https://dev.azure.com/{yourorganization})
7. Enter your username and Azure DevOps API token in the appropriate fields. (See: Generate Azure DevOps Token below.) The login must have sufficient privileges to create Azure DevOps issues in the projects specified in the Project Associations section. 
8. Click Test Connection to verify your settings and click Save.
   

Generate Azure DevOps Token

1. Sign in to your organization in Azure DevOps (https://dev.azure.com/{yourorganization})

2. From your home page, open your user settings, and then select Personal access tokens.
   

3. Select + New Token.
4. Name your token, select the organization where you want to use the token, and then choose a lifespan for your token.

5. Select the scopes for this token to authorize for your specific tasks.

   1. Under Scopes, select Custom defined.
   2. Click on the Show all scopes.
   3. Select Tokens and check Read & manage.
   4. Select Work items and check Read, write, & manage.
   5. Click Save.
6. When done, make sure to copy the token. For your security, it won't be shown again. Use this token as your password.

Once your Personal access token is created, you can use it anywhere your user credentials are required for authentication in Azure DevOps.

Enabling the Requirements Traceability Report

You can configure DTP to generate widgets and reports that help you demonstrate traceability between the requirements stored in Azure DevOps and the test, static analysis, and build review data sent to DTP from Parasoft tools (C/C++test, dotTEST, Jtest).

If you want the Traceability Report to include code review and static analysis information, you must associate your source code files with requirements stored in Azure DevOps. See Associating Requirements with Files for instructions on enabling this optional feature.

DTP interfaces that display and track traceability are enabled by deploying the External System Traceability Report artifact shipped with the Traceability Pack. The Traceability Pack also includes the Sending Test Data to External System flow, which automates part of the requirements traceability workflow. 

Use DTP Extension Designer to deploy the External System Traceability Report and the Sending Test Data to External System flow to your environment. Verify that DTP is connected Azure DevOps as described in the Connecting DTP to Azrue DevOps section before deploying the artifacts.

Installing the Traceability Pack

The first step is to install the Traceability Pack artifact. The artifact is a collection of configuration files and assets that enable traceability.  

1. Choose Extension Designer from the settings menu (gear icon).
   

   Enabling the Requirements Traceability Report

2. Click the Configuration tab to open Artifact Manager.
3. Click Upload Artifact and browse for the traceability-pack-<version>.zip archive (also see Downloading and Installing Artifacts).
4. Click Install and a collection of assets and configuration files for enabling traceability will be installed.

Requirements

The following requirements are only applicable if you are going to send test results to Azure DevOps.

• Tests executed by the following Parasoft tools are supported:
  • C/C++test Professional, dotTEST, or Jtest 10.4.3 +
  • Selenic 2020.1 +
  • SOAtest 9.10.8 +
• You should have already created requirements in Azure DevOps. 

Usage

After configuring the integration with Azure DevOps, developers, testers, and other users can leverage the functionality enabled by the integration.

Manually Creating Work Items in Azure DevOps

The Test Explorer and Violations Explorer views enable you to create issues and defects for any test and violation, respectively, regardless of status. Refer to the following sections for details on creating Azure DevOps assets in explorer views:

• See Creating an Issue in a Third-party System for instructions on how to manually create work items in Azure DevOps from the Violations Explorer view.
• See Creating an Issue in a Third-party System for instructions on how to manually create work items in Azure DevOps from the Test Explorer view.

Sending Test Data to Azure DevOps

The following diagram shows how you could implement an automated infrastructure for integrating Parasoft DTP and Parasoft test execution tools into your Azure DevOps environment:

To send test data to Azure DevOps:

1. Create requirements in Azure DevOps that you can associate with tests executed by Parasoft tools. 

2. In your test file, map the Azure DevOps system requirement IDs to the tests using the following annotation format:
   For users who have not created their own test case definition to see traceability results in Azure DevOps, use @req:

   * The following annotation maps the test to a requirement: * @req <<Azure DevOps ID> >  @TEST ...

   Use @req in Parasoft and trigger sending results from Parasoft DTP to

   For users who have created their own test case definition in Azure DevOps, use @test:

   * The following annotation maps the test to a requirement: * @test <<Azure DevOps ID> >  @TEST ...

   Associate each test case definition with your tested requirement in Azure DevOps.
   
   

   • Use the @test <Azure DevOps ID> annotation to associate tests with test definitions in Azure DevOps. When using @test make sure that you have no test case steps defined in the tests whose run statuses you plan to send from Parasoft DTP to Azure DevOps. Parasoft DTP is not able to update test case runs for test cases that have steps defined.

   • Use the @req <Azure DevOps ID> annotation to associate tests with stories in Azure DevOps. When using @req make sure the following relation type is defined in the Azure DevOps project you plan to send results to: "verified by". Such relation is needed by Parasoft DTP when it creates a test case in Azure DevOps and relates it to a requirement in Azure DevOps.
     
     Annotating tests links them to entities in Azure DevOps so that they can be visualized in DTP reports. The annotation is added to the unit test file for Parasoft language tools (i.e., C/C++test, dotTEST, Jtest). For functional tests executed with SOAtest, the annotation is added to the .tst file. Refer to your Parasoft tool documentation for details on adding annotations. You must use the ID from Azure DevOps URL and not the ID generated in the UI:
     
     
     
     

   
   

3. Execute your tests as part of the CI process. You can also manually execute the tests from the IDE.
4. As part of the test execution, Parasoft test execution tools will tag the results with the filter and build IDs and send the data to DTP. You can verify the results in DTP by adding Test Widgets to your DTP dashboard and setting the filter and build ID. Developers can download the test execution data from DTP into their IDEs so that they can address any failed tests.
   

5. If you deployed the Sending Test Data to External System flow (see Deploying the Sending Test Data to External System Flow), then unit and functional testing results will automatically be sent to Azure DevOps when Data Collector receives the data from the Parasoft tool. By default, the flow forwards unit and functional test results that were received by Data Collector for any project, but you can configure the flow to only send data for a specific project (see Sending Results from a Specific DTP Project). 
   You can also manually send a POST request to the DTP REST API endpoint to send results from the DTP database to Azure DevOps. Pass the IDs for the DTP filter and build, as well as the ID for a set of Azure DevOps test cases, as URL parameters in the API call:
   

   curl -X POST -u <username>:<password> "http://<host>:<port>/grs/api/v1.7/linkedApps/configurations/1/syncTestCases?filterId=<filterID>&buildId=<buildID>&testCaseSetId=<Azure DevOps_TEST_CASE_SET_ID>"

   The filter and build IDs are available in the Test Explorer URL:
   
   The following table describes the endpoint parameters.
   

   Parameter	Value	Description	Required
   filterId	integer	Specifies the filter ID containing the test data. The filter ID is an integer value and should not be confused with the filter name.	Required
   buildId	string	Specifies the build ID containing the test data.	Required
   groupResultsBySOAtestTST	boolean	Setting to true groups SOAtest results by .tst file. As a result, one .tst file will be associated with one Azure DevOps test. Setting to false associates each test step within a SOAtest .tst with a Azure DevOps test. Default is false	Optional
   testCaseSetId	ID of set of test cases	Specifies the ID of a test case set in Azure DevOps where Parasoft DTP should create new test case definitions, if not found in Azure DevOps. When a test case definition is found in Azure DevOps, whether in this testCaseSetId or another, it is just updated and not moved to this specific test case set.  The value for the testCaseSetID parameter should be an ID for a "Set of Test Cases" item type in Azure DevOps (see Requirements). You must use the ID from Azure DevOps URL and not the ID generated in the UI. You can get the ID from the URL after clicking on the test cases node in the project tree.	Optional

   The set of test cases is the item where DTP can create new test cases as necessary. DTP will create new test cases when it sends results to Azure DevOps if equivalent test cases do not exist, as described in the next step. 
   
   

6. DTP will locate the test results that match the filterId and buildId parameters and send the data to the Azure DevOps system requirements.
   • When DTP locates results with an @req <ID>, it will search for a matching item in Azure DevOps. If a match is found, test results will be added to the test cases associated with the item. If there are no test cases for the requirement ID, then test cases will be created and test runs will be added to them.

   • An external-app-sync.log file will also be written to the the <DTP_INSTALL>/logs directory. This log file contains progress information about sending test results from DTP to Azure DevOps. 

After DTP processes the report and sends results to Azure DevOps, you should expect a response similar to the following:

{
   "createdTestSession": "DTPP-521",
    "created": [
        "DTPP-519, testName = testBagSumAdd"
    ],
    "updated": [
        "DTPP-519, testName = testBagSumAdd",
        "DTPP-518, testName = testBagSimpleAdd"
    ],
    "ignored": [
        "MAGD-567, testName = testBagNegate",
        "QAP-512, testName = testTryThis3",
        "QAP-512, testName = testTryThis4",
        "MAGD-567, testName = testBagMultiply"
    ]
}

Sending Results from a Specific DTP Project

If you are using the Sending Test Data to External System flow to forward unit and functional test results, data will be sent to Azure DevOps for all DTP projects by default. As a result, work items will be updated to include the tests collected for any DTP project that contain annotations matching Azure DevOps IDs. You can configure the flow, however, to only send data for a specific project. 

1. Open Extension Designer and open the service where the Sending Test Data is deployed.
2. Drag a new switch node to the workspace.
   
3. Select and delete the connector line between the "DataCollectorProcessedEvent" node and the "Is dynamic analysis" node.
4. Drag a new connector from the "DataCollectorProcessedEvent" node to the switch node and from the switch node to the "Is dynamic analysis" node.
   
   
   

5. Double-click the node and specify the following string in the Property field:

    event.message.resultsSession.project

6. Specify the name of the DTP project in the string field.
7. (Optional) Provide a more descriptive name for the node.
   
8. Click Done to finish configuring the node and click Deploy.

When the flow executes, only test results for the specified DTP project will be sent to Azure DevOps. 

Viewing Results in Azure DevOps

You will be able to view results in Azure DevOps after sending the test data. The following image shows a System Requirement Specifications requirement in Azure DevOps. The requirement contains several test cases.

You can drill down into a test case to view details, such as test runs for the test case.

Viewing the Traceability Report

If the External System Traceability Report has been deployed to your system (see Enabling the Requirements Traceability Report), you can add widgets to your dashboard to monitor traceability from Azure DevOps requirements to Parasoft DTP tests, static analysis, code reviews for your project. The widgets also drill down to a report that includes additional details.

Adding and Configuring the Widgets

The widgets will appear in a separate Traceability category when adding widgets to your DTP dashboard. See Adding Widgets for general instructions on adding widgets.

You can configure the following settings:

Requirements Widget

This widget shows the number of requirements from the specified Azure DevOps project.

Click on the widget to open the Requirement Traceability report.

Test Coverage Widget

This widget shows the percentage of requirements covered by tests against all requirements in the project.

Click the center of the widget to open the main Requirement Traceability report.

The colored-in segment represents the requirements covered by tests. Click on the segment to open the Requirement Traceability report filtered to the With Tests category.

Pie Widget

Unit testing, functional testing, static analysis, and peer reviews are common activities for verifying that requirements have been properly and thoroughly implemented. This widget shows the overall status of the project requirements in the context of those software quality activities. You can add a widget for each type of quality activity (tests, static analysis violations, reviews) to monitor the progress of requirements implementation for the project.

Mouse over a section of the chart to view details about quality activity type status. Click on the widget to open the Requirement Traceability report filtered by the selected type.

Requirements Implementation Status by Tests

Requirements Implementation Status by Violations

Requirements Implementation by Reviews

Understanding the Requirement Traceability Report

The report lists the Azure DevOps requirements and Parasoft DTP tests associated with them.

You can perform the following actions:

• Disable or enable the Show files/reviews option if you want to hide the Files and Reviews columns in the report. The Files and Reviews columns will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). Disabling the Files and Reviews columns on this screen hides the related tabs in the Requirement Details report.
• Click on a link in the Key column to view the requirement in Azure DevOps.
• Click on a link in the Summary column or one of the Test columns to view the test-related information associated with the requirement in the Azure DevOps Requirement Details Report.
• Click on a link in one of the Files columns to view the static analysis-related information associated with the requirement in the Azure DevOps Requirement Details Report.
• Click on a link in one of the Reviews columns to view the change review-related information associated with the requirement in the Azure DevOps Requirement Details Report.

Requirement Traceability Report by Type

Clicking on a section of the Azure DevOps Requirements - Pie widget opens a version of the report that includes only the quality activity type selected in the widget. You can use the drop-down menus to switch type and status. You can also disable or enable the Show files/reviews option if you want to hide the Files and Reviews columns in the report. The Files and Reviews columns will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). Disabling the Files and Reviews columns on this screen hides the related tabs in the Requirement Details report.  

Understanding the Requirement Details Report

The Azure DevOps Requirement Details report provides additional information about the files, static analysis findings, and tests associated with a specific Azure DevOps requirement. You can open this report by clicking on a requirement in the Azure DevOps Requirement Traceability report.

The first tab shows the results of the tests that were executed to verify the specific work item.

You can click on the View results in Test Explorer link to view all of the tests associated with the work item in the Test Explorer. 

You can also click on individual test names in the table to view each test in the Test Explorer.

The second tab shows the files associated with the specific requirement, as well as the static analysis violation detected in the files. You can click the link the Violations column to view the violations in the Violations Explorer, which provides additional details about the violations.

This tab will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.

If the files include any change reviews or review findings, they will be shown in the third tab with links to view them in the Change Explorer. 

This tab will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.