In this section:

Introduction

Azure DevOps is Microsoft's Software as a service (SaaS) platform that provides an end-to-end DevOps programming tools for developing and deploying software. Connecting DTP to Azure DevOps Server Express and Azure DevOps Server provides the following functionality:

• Ability to manually create defects in Azure DevOps from the Violations Explorer and Test Explorer views.
• Ability to send, view, and update Parasoft test results in Azure DevOps system requirements (Sending Test Data to Azure DevOps).
  • For users who plan to track traceability using traceability reports in Azure DevOps. In order to do this, you need to first send test results from Parasoft DTP to Azure DevOps.
• Traceability from Azure DevOps requirements to tests, static analysis results, and code reviews (see Viewing Results in Azure DevOps).
  • For users who plan to track traceability using traceability reports in Parasoft DTP. In order to do this, use @req to associate your automated tests in Parasoft to Azure DevOps requirements.

Configuration

The configuration is performed by the Parasoft administrator and only needs to be set up once. 

Connecting DTP to Azure DevOps

1. Choose Report Center Settings from the settings (gear icon) drop-down menu.
2. Choose External System, click Edit Settings, and choose Azure DevOps from the System type drop-down menu.
3. Enable the Enabled option.
4. Enter a name for the server in the Name field. The name is required but does not affect the connection settings or render in any other interfaces.
5. Enter the URL of your Azure DevOps system in the Application URL field. Must include your Azure DevOps collection name.
6. The Display URL field defines the URL which is displayed in Parasoft DTP pages when links to your Azure DevOps system are presented in a web browser. Typically, this should be the same as the above Application URL field. However, it might be different, for example, when you work in a reverse proxy environment and links to Azure DevOps from the user's local web browser with Parasoft DTP are different than from the Parasoft DTP server. (e.g., https://dev.azure.com/{yourorganization})
7. Enter your username and Azure DevOps API token in the appropriate fields. (See: Generate Azure DevOps Token below.) The login must have sufficient privileges to create Azure DevOps issues in the projects specified in the Project Associations section. 
8. Click Test Connection to verify your settings and click Confirm.
   

Generate Azure DevOps Token

1. Sign into your organization in Azure DevOps (https://dev.azure.com/{yourorganization})

2. From your home page, open your user settings, and then select Personal access tokens.
   

3. Select + New Token.
4. Name your token, select the organization where you want to use the token, and then choose a lifespan for your token.

5. Select the scopes for this token to authorize for your specific tasks.

   1. Under Scopes, select Custom defined.
   2. Click on the Show all scopes.
   3. Select Test Management and enable Read & write.
   4. Select Tokens and enable Read & manage.
   5. Select Work items and enable Read, write, & manage.
   6. Click Save.
6. When done, make sure to copy the token. For your security, it won't be shown again. Use this token as your password.

Once your Personal access token is created, you can use it anywhere your user credentials are required for authentication in Azure DevOps.

Enabling the Requirements Traceability Report

You can configure DTP to generate widgets and reports that help you demonstrate traceability between the requirements stored in Azure DevOps and the test, static analysis, and build review data sent to DTP from Parasoft tools (C/C++test, dotTEST, Jtest).

If you want the Traceability Report to include code review and static analysis information, you must associate your source code files with requirements stored in Azure DevOps. See Associating Requirements with Files for instructions on enabling this optional feature.

DTP interfaces that display and track traceability are enabled by deploying the External System Traceability Report artifact shipped with the Traceability Pack. The Traceability Pack also includes the Sending Test Data to External System flow, which automates part of the requirements traceability workflow. 

Use DTP Extension Designer to deploy the External System Traceability Report and the Sending Test Data to External System flow to your environment. Verify that DTP is connected Azure DevOps as described in the Connecting DTP to Azrue DevOps section before deploying the artifacts.

Installing the Traceability Pack

The first step is to install the Traceability Pack artifact. The artifact is a collection of configuration files and assets that enable traceability.  

1. Choose Extension Designer from the settings menu (gear icon).
   
2. Click the Configuration tab to open Artifact Manager.
3. Click Upload Artifact and browse for the traceability-pack-<version>.zip archive (also see Downloading and Installing Artifacts).
4. Click Install and a collection of assets and configuration files for enabling traceability will be installed.

Advanced Configuration

You can modify the ExternalSystemSettings.properties configuration file located in the <DTP_DATA_DIR>/conf directory to change the default behavior of the integration. DTP's out-of-the-box integration with Azure DevOps is configured to use default or commonly-used fields and work item types. If you customized your Azure DevOps system, however, then you can configure the following settings to align data in DTP with your custom configuration. 

Requirements

The following requirements are only applicable if you are going to send test results to Azure DevOps.

• Tests executed by the following Parasoft tools are supported:
  • C/C++test Professional, dotTEST, or Jtest 10.4.3 +
  • Selenic 2020.1 +
  • SOAtest 9.10.8 +
• You should have already created requirements in Azure DevOps. 

Usage

After configuring the integration with Azure DevOps, developers, testers, and other users can leverage the functionality enabled by the integration.

Manually Creating Work Items in Azure DevOps

The Test Explorer and Violations Explorer views enable you to create issues and defects for any test and violation, respectively, regardless of status. Refer to the following sections for details on creating Azure DevOps assets in explorer views:

• See "Creating an Issue in a Third-party System" on the Violations Explorer page for instructions on how to manually create work items in Azure DevOps from the Violations Explorer view.
• See "Creating an Issue in a Third-party System" on the Test Explorer page for instructions on how to manually create work items in Azure DevOps from the Test Explorer view.

Sending Test Data to Azure DevOps

The following diagram shows how you could implement an automated infrastructure for integrating Parasoft DTP and Parasoft test execution tools into your Azure DevOps Connect environment:

To send test data to Azure DevOps:

1. Create requirements in Azure DevOps that you can associate with tests executed by Parasoft tools. 

2. In your test file, map the Azure DevOps Connect system requirement IDs to the tests using the following annotation format:
   For users who have not created their own test case definition to see traceability results in Azure DevOps, use @req:

   * The following annotation maps the test to a requirement: * @req <<Azure DevOps ID> >  @TEST ...

   Use @req in Parasoft and trigger sending results from Parasoft DTP to Azure DevOps.

   For users who have created their own test case definition in Azure DevOps, use @test:

   * The following annotation maps the test to a requirement: * @test <<Azure DevOps ID> >  @TEST ...

   Associate each test case definition with your tested requirement in Azure DevOps.
   
   

   • Use the @test <Azure DevOps ID> annotation to associate tests with test definitions in Azure DevOps. When using @test make sure that you have no test case steps defined in the tests whose run statuses you plan to send from Parasoft DTP to Azure DevOps. Parasoft DTP is not able to update test case runs for test cases that have steps defined.

   • Use the @req <Azure DevOps ID> annotation to associate tests with stories in Azure DevOps. When using @req make sure the following relation type is defined in the Azure DevOps project you plan to send results to: "verified by". Such relation is needed by Parasoft DTP when it creates a test case in Azure DevOps and relates it to a requirement in Azure DevOps.
     
     Annotating tests links them to entities in Azure DevOps Connect so that they can be visualized in DTP reports. The annotation is added to the unit test file for Parasoft language tools (i.e., C/C++test, dotTEST, Jtest). For functional tests executed with SOAtest, the annotation is added to the .tst file. Refer to your Parasoft tool documentation for details on adding annotations. You must use the ID from Azure DevOps Connect URL and not the ID generated in the UI:
     
     
     
     

   
   

3. Execute your tests as part of the CI process. You can also manually execute the tests from the IDE.
4. As part of the test execution, Parasoft test execution tools will tag the results with the filter and build IDs and send the data to DTP. You can verify the results in DTP by adding Test Widgets to your DTP dashboard and setting the filter and build ID. Developers can download the test execution data from DTP into their IDEs so that they can address any failed tests.
   

5. If you deployed the Sending Test Data to External System flow (see Deploying the Sending Test Data to External System Flow), then unit and functional testing results will automatically be sent to Azure DevOps Connect when Data Collector receives the data from the Parasoft tool. By default, the flow forwards unit and functional test results that were received by Data Collector for any project, but you can configure the flow to only send data for a specific project (see Sending Results from a Specific DTP Project). 
   You can also manually send a POST request to the DTP REST API endpoint to send results from the DTP database to Azure DevOps Connect. Pass the IDs for the DTP filter and build, as well as the ID for a set of Azure DevOps test cases, as URL parameters in the API call:
   

   curl -X POST -u <username>:<password> "http://<host>:<port>/grs/api/v1.7/linkedApps/configurations/1/syncTestCases?filterId=<filterID>&buildId=<buildID>

   The filter and build IDs are available in the Test Explorer URL:
   
   The following table describes the endpoint parameters.
   

   Parameter	Value	Description	Required
   filterId	integer	Specifies the filter ID containing the test data. The filter ID is an integer value and should not be confused with the filter name.	Required
   buildId	string	Specifies the build ID containing the test data.	Required
   groupResultsBySOAtestTST	boolean	Setting to true groups SOAtest results by .tst file. As a result, one .tst file will be associated with one Azure DevOps test. Setting to false associates each test step within a SOAtest .tst with a Azure DevOps test. Default is false	Optional

   Support for Multiple Azure DevOps Implementations

   If you are using the /syncTestCases REST API endpoint to synchronize test cases between DTP and Azure DevOps, you can configure DTP to distribute test data to different Azure DevOps implementations. The functionality is only available using the API call.

   You can configure DTP to recognize as many Azure DevOps implementations as needed by adding them to the end of the ExternalSystemSettings.properties file located in the <DTP_DATA_DIR>/conf directory. Any time the API sends DTP data to Azure DevOps and does not find a match in ExternalSystemSettings.properties, it will send the data to the Azure DevOps configuration you set up in the UI in Connecting DTP to Azure DevOps instead. Add Azure DevOps configurations to the ExternalSystemSettings.properties file in a format that mirrors the example below:

   azuredevops.config1.appUrl=http://azure.test1:8888/organization1
   azuredevops.config1.displayUrl=http://azure.test1:8888/organization1
   azuredevops.config1.username=john
   azuredevops.config1.apiToken=encrypted_token
   azuredevops.config1.project1.dtpProject=bank
   azuredevops.config1.project1.externalProject=Bank
   azuredevops.config1.project2.dtpProject=atm-core
   azuredevops.config1.project2.externalProject=Atm 

   This configuration—"config1"—will cause data from the DTP project "bank" to be sent to the external project called "Bank" on the server http://azure.test1:8888/organization1, which is will access using the API token "abcdef." The DTP project and external project are linked by virtue of both belonging to "project1" and the appropriate server and login credentials are known because all of them belong to "config1."

   Understanding this, you can use the example above as a template to define additional configurations for your other Azure DevOps implementations. Each configuration must have a unique "azuredevops.config#" value. For example, using the configuration shown above as a base, a second configuration might look something like this:

   azuredevops.config2.appUrl=http://azure.test2:8888/organization1
   azuredevops.config2.displayUrl=http://azure.test2:8888/organization1
   azuredevops.config2.username=jane
   azuredevops.config2.apiToken=encrypted_token
   azuredevops.config2.project1.dtpProject=foo
   azuredevops.config2.project1.externalProject=bar

   You can set up as many configurations as you need. Some things to keep in mind:

   • The "project#.dtpProject=" and "project#.externalProject=" pairs in these examples show how you can associate a DTP project to an external project. Multiple associations are allowed, but a "dtpProject" is expected to be unique across all configurations.
   • The current External System configuration as configured in the Report Center settings UI must be configured to "Azure DevOps."
   • All additional Azure DevOps configurations in the ExternalSystemSettings.properties configuration file must be valid.
   • The "apiToken" property for each configuration must be encrypted or else the configuration is deemed invalid. To encrypt this value, use the -encodepass CLI option included with any Parasoft tool (for example, soatestcli.exe -encodepass <API TOKEN>).
   • Traceability Widgets and reports will only show the primary ADO (that is, the one defined in the UI).
   • Requirements from multiple ADOs cannot be imported into SOAtest.

   The API call will determine which configuration to use based on the filterId that is passed to it (see the table above that describes the endpoint parameters). DTP will look up the project associated with that filter and find a configuration that specifies that DTP project. The first match is used. If no matches are found, the default configuration is used.

   
   

6. DTP will locate the test results that match the filterId and buildId parameters and send the data to the Azure DevOps Connect system requirements.
   • When DTP locates results with an @req <ID>, it will search for a matching item in Azure DevOps. If a match is found, test results will be added to the test cases associated with the item. If there are no test cases for the requirement ID, then test cases will be created and test runs will be added to them.

   • An external-app-sync.log file will also be written to the the <DTP_INSTALL>/logs directory. This log file contains progress information about sending test results from DTP to Azure DevOps. 

After DTP processes the report and sends results to Azure DevOps, you should expect a response similar to the following:

{
   "createdTestSession": "DTPP-521",
    "created": [
        "DTPP-519, testName = testBagSumAdd"
    ],
    "updated": [
        "DTPP-519, testName = testBagSumAdd",
        "DTPP-518, testName = testBagSimpleAdd"
    ],
    "ignored": [
        "MAGD-567, testName = testBagNegate",
        "QAP-512, testName = testTryThis3",
        "QAP-512, testName = testTryThis4",
        "MAGD-567, testName = testBagMultiply"
    ]
}

Sending Results from a Specific DTP Project

If you are using the Sending Test Data to External System flow to forward unit and functional test results, data will be sent to Azure DevOps for all DTP projects by default. As a result, work items will be updated to include the tests collected for any DTP project that contain annotations matching Azure DevOps IDs. You can configure the flow, however, to only send data for a specific project. 

1. Open Extension Designer and open the service where the Sending Test Data is deployed.
2. Drag a new switch node to the workspace.
   
3. Select and delete the connector line between the "DataCollectorProcessedEvent" node and the "Is dynamic analysis" node.
4. Drag a new connector from the "DataCollectorProcessedEvent" node to the switch node and from the switch node to the "Is dynamic analysis" node.
   
   
   

5. Double-click the node and specify the following string in the Property field:

    event.message.resultsSession.project

6. Specify the name of the DTP project in the string field.
7. (Optional) Provide a more descriptive name for the node.
   
8. Click Done to finish configuring the node and click Deploy.

When the flow executes, only test results for the specified DTP project will be sent to Azure DevOps. 

Viewing Results in Azure DevOps

You will be able to view results in Azure DevOps after sending the test data. The following image shows a System Requirement Specifications requirement in Azure DevOps. The requirement contains several test cases.

You can drill down into a test case to view details, such as test runs for the test case.

Viewing the Traceability Report

If the External System Traceability Report has been deployed to your system (see Enabling the Requirements Traceability Report), you can add widgets to your dashboard to monitor traceability from Azure DevOps requirements to Parasoft DTP tests, static analysis, code reviews for your project. The widgets also drill down to a report that includes additional details.

Adding and Configuring the Widgets

The widgets will appear in a separate Traceability category when adding widgets to your DTP dashboard. See Adding Widgets for general instructions on adding widgets.

You can configure the following settings:

Requirements Widget

This widget shows the number of requirements from the specified Azure DevOps project.

Click on the widget to open the Requirement Traceability report.

Test Coverage Widget

This widget shows the percentage of requirements covered by tests against all requirements in the project.

Click the center of the widget to open the main Requirement Traceability report.

The colored-in segment represents the requirements covered by tests. Click on the segment to open the Requirement Traceability report filtered to the With Tests category.

Pie Widget

Unit testing, functional testing, static analysis, and peer reviews are common activities for verifying that requirements have been properly and thoroughly implemented. This widget shows the overall status of the project requirements in the context of those software quality activities. You can add a widget for each type of quality activity (tests, static analysis violations, reviews) to monitor the progress of requirements implementation for the project.

Mouse over a section of the chart to view details about quality activity type status. Click on the widget to open the Requirement Traceability report filtered by the selected type.

Requirements Implementation Status by Tests

Requirements Implementation Status by Violations

Requirements Implementation by Reviews

Understanding the Requirement Traceability Report

The report lists the Azure DevOps requirements and Parasoft DTP tests associated with them.

You can perform the following actions:

• Disable or enable the Show files/reviews option if you want to hide the Files and Reviews columns in the report. The Files and Reviews columns will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). Disabling the Files and Reviews columns on this screen hides the related tabs in the Requirement Details report.
• Click on a link in the Key column to view the requirement in Azure DevOps.
• Click on a link in the Summary column or one of the Test columns to view the test-related information associated with the requirement in the Azure DevOps Requirement Details Report.
• Click on a link in one of the Files columns to view the static analysis-related information associated with the requirement in the Azure DevOps Requirement Details Report.
• Click on a link in one of the Reviews columns to view the change review-related information associated with the requirement in the Azure DevOps Requirement Details Report.

Requirement Traceability Report by Type

Clicking on a section of the Azure DevOps Requirements - Pie widget opens a version of the report that includes only the quality activity type selected in the widget. You can use the drop-down menus to switch type and status. You can also disable or enable the Show files/reviews option if you want to hide the Files and Reviews columns in the report. The Files and Reviews columns will only contain data if the requirements have been mapped to source files files (see Enabling the Requirements Traceability Report). Disabling the Files and Reviews columns on this screen hides the related tabs in the Requirement Details report.  

Understanding the Requirement Details Report

The Azure DevOps Requirement Details report provides additional information about the files, static analysis findings, and tests associated with a specific Azure DevOps requirement. You can open this report by clicking on a requirement in the Azure DevOps Requirement Traceability report.

The first tab shows the results of the tests that were executed to verify the specific work item.

You can click on the View results in Test Explorer link to view all of the tests associated with the work item in the Test Explorer. 

You can also click on individual test names in the table to view each test in the Test Explorer.

The second tab shows the files associated with the specific requirement, as well as the static analysis violation detected in the files. You can click the link the Violations column to view the violations in the Violations Explorer, which provides additional details about the violations.

This tab will only contain data if the requirements have been mapped to source files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.

If the files include any change reviews or review findings, they will be shown in the third tab with links to view them in the Change Explorer. 

This tab will only contain data if the requirements have been mapped to source files (see Enabling the Requirements Traceability Report). If you did not map requirements to files, you can hide this tab by disabling the Show files/reviews option on the main traceability report page and reloading the details report.