Sending Test Data to Azure DevOps and Viewing Results

In this section:

Sending Test Data to Azure DevOps

The following diagram shows how you could implement an automated infrastructure for integrating Parasoft DTP and Parasoft test execution tools into your Azure DevOps Connect environment:

Sending test data to Azure DevOps

To send test data to Azure DevOps:

1. Create requirements in Azure DevOps that you can associate with tests executed by Parasoft tools. 

2. In your test file, map the Azure DevOps Connect system requirement IDs to the tests using the following annotation format:
   For users who have not created their own test case definition to see traceability results in Azure DevOps, use @req:

   * The following annotation maps the test to a requirement: * @req <<Azure DevOps ID> >  @TEST ...

   Use @req in Parasoft and trigger sending results from Parasoft DTP to Azure DevOps.

   For users who have created their own test case definition in Azure DevOps, use @test:

   * The following annotation maps the test to a requirement: * @test <<Azure DevOps ID> >  @TEST ...

   Associate each test case definition with your tested requirement in Azure DevOps.

   • Use the @test <Azure DevOps ID> annotation to associate tests with test definitions in Azure DevOps. When using @test make sure that you have no test case steps defined in the tests whose run statuses you plan to send from Parasoft DTP to Azure DevOps. Parasoft DTP is not able to update test case runs for test cases that have steps defined.

   • Use the @req <Azure DevOps ID> annotation to associate tests with stories in Azure DevOps. When using @req make sure the following relation type is defined in the Azure DevOps project you plan to send results to: "verified by". Such relation is needed by Parasoft DTP when it creates a test case in Azure DevOps and relates it to a requirement in Azure DevOps.
     
     Annotating tests links them to entities in Azure DevOps Connect so that they can be visualized in DTP reports. The annotation is added to the unit test file for Parasoft language tools (C/C++test, dotTEST, Jtest). For functional tests executed with SOAtest, the annotation is added to the .tst file. Refer to your Parasoft tool documentation for details on adding annotations. You must use the ID from Azure DevOps Connect URL and not the ID generated in the UI:
     
     

   
   

3. Execute your tests as part of the CI process. You can also manually execute the tests from the IDE.
4. As part of the test execution, Parasoft test execution tools will tag the results with the filter and build IDs and send the data to DTP. You can verify the results in DTP by adding Test Widgets to your DTP dashboard and setting the filter and build ID. Developers can download the test execution data from DTP into their IDEs so that they can address any failed tests.
   

5. If you deployed the Sending Test Data to External System flow (see Deploying the Sending Test Data to External System Flow), then unit and functional testing results will automatically be sent to Azure DevOps Connect when Data Collector receives the data from the Parasoft tool. By default, the flow forwards unit and functional test results that were received by Data Collector for any project, but you can configure the flow to only send data for a specific project (see Sending Results from a Specific DTP Project). 
   You can also manually send a POST request to the DTP REST API endpoint to send results from the DTP database to Azure DevOps Connect. Pass the IDs for the DTP filter and build, as well as the ID for a set of Azure DevOps test cases, as URL parameters in the API call:
   

   curl -X POST -u <username>:<password> "http://<host>:<port>/grs/api/v1.7/linkedApps/configurations/1/syncTestCases?filterId=<filterID>&buildId=<buildID>

   The filter and build IDs are available in the Test Explorer URL:
   
   The following table describes the endpoint parameters.
   
   

   Parameter	Value	Description	Required
   filterId	integer	Specifies the filter ID containing the test data. The filter ID is an integer value and should not be confused with the filter name.	Required
   buildId	string	Specifies the build ID containing the test data.	Required
   groupResultsBySOAtestTST	boolean	Setting to true groups SOAtest results by .tst file. As a result, one .tst file will be associated with one Azure DevOps test. Setting to false associates each test step within a SOAtest .tst with a Azure DevOps test. Default is false	Optional
   testPlanId	integer	Specifies the test plan ID in Azure DevOps to which to send the results. DTP will generate a test suite in the format of "Results from Parasoft DTP - <DTP_PROJECT_NAME>" in the specified test plan. Test cases to be synced will be copied into this test suite and their test points will be run. If the test suite was generated previously, then it is re-used in future syncTestCases runs against the specified test plan. If this is not specified, DTP will generate a new Test Plan and sync the test case results into that Test Plan.	Optional
   updateResultsOnly	boolean	Specifies a syncing mode where test cases in a particular test plan will be updated in-place. If set to true, the testPlanId parameter must also be provided so that DTP can search in the correct test plan for which test cases to update. In this mode, the test case definitions to be updated must exist before running the sync operation. DTP will not create a new test suite, test plan, or test cases in this mode. Default is false	Optional

   Support for Multiple Azure DevOps Implementations

   If you are using the /syncTestCases REST API endpoint to synchronize test cases between DTP and Azure DevOps, you can configure DTP to distribute test data to different Azure DevOps implementations. The functionality is only available using the API call.

   You can configure DTP to recognize as many Azure DevOps implementations as needed by adding them to the end of the ExternalSystemSettings.properties file located in the <DTP_DATA_DIR>/conf directory. Any time the API sends DTP data to Azure DevOps and does not find a match in ExternalSystemSettings.properties, it will send the data to the Azure DevOps configuration you set up in the UI in Connecting DTP to Azure DevOps instead. Add Azure DevOps configurations to the ExternalSystemSettings.properties file in a format that mirrors the example below:

   azuredevops.config1.appUrl=http://azure.test1:8888/organization1
   azuredevops.config1.displayUrl=http://azure.test1:8888/organization1
   azuredevops.config1.username=john
   azuredevops.config1.apiToken=encrypted_token
   azuredevops.config1.project1.dtpProject=bank
   azuredevops.config1.project1.externalProject=Bank
   azuredevops.config1.project2.dtpProject=atm-core
   azuredevops.config1.project2.externalProject=Atm 

   This configuration—"config1"—will cause data from the DTP project "bank" to be sent to the external project called "Bank" on the server http://azure.test1:8888/organization1, which is will access using the API token "abcdef." The DTP project and external project are linked by virtue of both belonging to "project1" and the appropriate server and login credentials are known because all of them belong to "config1."

   Understanding this, you can use the example above as a template to define additional configurations for your other Azure DevOps implementations. Each configuration must have a unique "azuredevops.config#" value. For example, using the configuration shown above as a base, a second configuration might look something like this:

   azuredevops.config2.appUrl=http://azure.test2:8888/organization1
   azuredevops.config2.displayUrl=http://azure.test2:8888/organization1
   azuredevops.config2.username=jane
   azuredevops.config2.apiToken=encrypted_token
   azuredevops.config2.project1.dtpProject=foo
   azuredevops.config2.project1.externalProject=bar

   You can set up as many configurations as you need. Some things to keep in mind:

   • The "project#.dtpProject=" and "project#.externalProject=" pairs in these examples show how you can associate a DTP project to an external project. Multiple associations are allowed, but a "dtpProject" is expected to be unique across all configurations.
   • The current External System configuration as configured in the Report Center settings UI must be configured to "Azure DevOps."
   • All additional Azure DevOps configurations in the ExternalSystemSettings.properties configuration file must be valid.
   • The "apiToken" property for each configuration must be encrypted or else the configuration is deemed invalid. To encrypt this value, use the -encodepass CLI option included with any Parasoft tool (for example, soatestcli.exe -encodepass <API TOKEN>).
   • Traceability Widgets and reports will only show the primary ADO (that is, the one defined in the UI).
   • Requirements from multiple ADOs cannot be imported into SOAtest.

   The API call will determine which configuration to use based on the filterId that is passed to it (see the table above that describes the endpoint parameters). DTP will look up the project associated with that filter and find a configuration that specifies that DTP project. The first match is used. If no matches are found, the default configuration is used.

6. DTP will locate the test results that match the filterId and buildId parameters and send the data to the Azure DevOps Connect system requirements.
   • When DTP locates results with an @req <ID>, it will search for a matching item in Azure DevOps. If a match is found, test results will be added to the test cases associated with the item. If there are no test cases for the requirement ID, then test cases will be created and test runs will be added to them.

   • An external-app.log file will also be written to the the <DTP_INSTALL>/logs directory. This log file contains progress information about sending test results from DTP to Azure DevOps. 

After DTP processes the report and sends results to Azure DevOps, you should expect a response similar to the following:

{
   "createdTestSession": "DTPP-521",
    "created": [
        "DTPP-519, testName = testBagSumAdd"
    ],
    "updated": [
        "DTPP-519, testName = testBagSumAdd",
        "DTPP-518, testName = testBagSimpleAdd"
    ],
    "ignored": [
        "MAGD-567, testName = testBagNegate",
        "QAP-512, testName = testTryThis3",
        "QAP-512, testName = testTryThis4",
        "MAGD-567, testName = testBagMultiply"
    ]
}

Sending Results from a Specific DTP Project

If you are using the Sending Test Data to External System flow to forward unit and functional test results, data will be sent to Azure DevOps for all DTP projects by default. As a result, work items will be updated to include the tests collected for any DTP project that contain annotations matching Azure DevOps IDs. You can configure the flow, however, to only send data for a specific project. 

1. Open Extension Designer and open the service where the Sending Test Data is deployed.
2. Double-click the Toggle Projects Mode node and set the value from the dropdown to true. This toggles on the ability to only send data to specific projects.
   
3. Click Done.

4. Double-click the Configure Projects node. The logic in this node specifies which DTP projects should have their data sent to Azure DevOps during the sync operation. These project configurations are stored in a JSON array.
   There is a sample project configuration already defined as an example:
   

   { project: 'abc' }

   To add more project configurations, add a comma to the end of the last project configuration and add a new entry below it. For example:

   msg.configuredProjects = [
       { project: 'abc' },
       { project: 'foobar' }
   ];

   These configurations also support an additional option for specifying a plan ID to which to sync results. For example:
   

   { project: 'foobar', plan: 350 }

5. Once the desired projects have been configured, click Done.
6. (Optional) If your configured projects all have plan IDs specified, you can add the updateResultsOnly=true parameter to toggle the syncing mode, if desired. Double-click the linkedApps - syncTestCases node and add the parameter to the end of the Endpoint field, then click Done.
   
   
7. Click Deploy to update the flow with these changes.

When the flow executes, only test results for the specified DTP project will be sent to Azure DevOps. 

Viewing Results in Azure DevOps

You will be able to view results in Azure DevOps after sending the test data. The following image shows a System Requirement Specifications requirement in Azure DevOps. The requirement contains several test cases.

You can drill down into a test case to view details, such as test runs for the test case.