In this section:

Introduction

The Parasoft Findings extension is designed to be used in a post-test execution build step that integrates functional and unit test reports as well as static analysis reports and code coverage reports, including support for quality gate checks, generated by Parasoft tools into your Azure DevOps project. The extension can consume the following report types:

• Parasoft SOAtest 2020.1+ XML reports.
• Parasoft Analyzers 2020.2+ XML reports generated by Parasoft C/C++test, Jtest and dotTEST tools.

Requirements

• Agent version 2.209.0 or higher.

Installing the Parasoft Findings Extension

1. Sign into the Visual Studio Marketplace and click the Azure DevOps tab.
2. Find and select the Parasoft Findings Extension.
3. Click Install.

See https://docs.microsoft.com/en-us/azure/devops/marketplace/install-extension?view=vsts in the Azure DevOps documentation for additional information.

Configuring the Extension

Configuring the Publish Parasoft Results Task

1. Edit an existing pipeline or create a new pipeline and click Add Task.
2. Search for "Publish Parasoft Results" using the search field.
3. Select the extension and click Add. 
4. Review the Results Files field. The predefined minimatch pattern is configured to read the test results, static analysis, and code coverage reports generated by Parasoft tools. You can modify or remove parts of the pattern to suit your needs. See Microsoft's documentation on file matching patterns for more information on working with matching patterns.
   • Static Analysis reports require you to install an extension to Azure DevOps to view; see Viewing Static Analysis Reports for more information.
5. (Optional but strongly recommended) Configure the Root path to Parasoft tool/Java installation field. You can enter the absolute location of the root path of a Parasoft tool (Jtest, dotTEST or C/C++test contains a Java installation) or a standalone Java installation to significantly optimize the efficiency of report processing and avoid potential memory issues.
6. (Optional) Configure the Reference pipeline field. If you want to use specific pipeline as a reference, specify the reference pipeline name. To use current pipeline, leave this field empty.

7. (Optional) Configure the Reference build field. If you want to use specific build as a reference, specify the reference build number. To use the last successful build, leave this field empty.
   

   About Reference Pipelines and Builds

   Reference pipeline and Reference build are not supported in a release pipeline.

   If you are not using quality gates, the reference pipeline and build can be used to filter results by Baseline on the Scans tab.

   If you are using quality gates, they can be used as a baseline build when the quality gate needs one.

8. Choose Save & Queue > Save.

Configuring the Parasoft Static Analysis Quality Gates Task

You can set multiple quality gates that evaluate after issues are reported. These gates adjust the build status based on desired product quality, setting builds to 'unstable' or 'failed'. Each gate measures against a specific metric: the number of issues of a certain type and severity. This feature is not supported in a Release pipeline.

1. Edit an existing pipeline or create a new one and click Add Task.
2. Search for "Parasoft Static Analysis Quality Gate" using the search field.
3. Select the extension and click Add.
4. Configure the following properties as needed.
   • Type: Type of issues to be evaluated for the threshold.
   • Severity: Severity of issues to be evaluated for the threshold.
   • Threshold: The threshold defines the minimum number of issues that will fail a build. If the actual number of issues is greater than or equal to this threshold, then a build is considered unstable or failed, respectively. Default value is 0.
   • Build status if quality gate fails: Determines if the build should be set to unstable or failed if the quality gate is not passed.
5. Choose Save & Queue > Save.

Configuring the Parasoft Code Coverage Quality Gate Task

You can set multiple quality gates that evaluate after coverage results are reported. These gates adjust the build status based on desired product quality, setting builds to 'unstable' or 'failed'. Each gate measures against the value of code coverage of a certain type. This feature is not supported in a Release pipeline.

1. Edit an existing pipeline or create a new pipeline and click Add Task.

2. Search for "Parasoft Code Coverage Quality Gate" using the search field.

3. Select the extension and click Add.

4. Configure the following properties as needed:
   • Type: Type of code coverage to be evaluated for the threshold.
   • Threshold: The threshold defines the actual value of code coverage that will fail a build. If the actual value of code coverage is less than this threshold, then a build is considered unstable or failed, respectively. Default value is 0.0.
   • Build status if quality gate fails: Determines if the build should be set to unstable or failed if the quality gate is not passed.
5. Choose Save & Queue > Save.

Configuring the Parasoft Test Results Quality Gate Task

You can set multiple quality gates that evaluate after test results are reported. These gates adjust the build status based on desired product quality, setting builds to 'unstable' or 'failed'. Each gate measures against the number of test results of a certain type. Please note that the "Newly failed tests" type is not supported in a Release pipeline.

1. Edit an existing pipeline or create a new pipeline and click Add Task.
2. Search for "Parasoft Test Results Quality Gate" using the search field.
3. Select the extension and click Add.
4. Configure the following three properties as needed:
   • Type: Type of test results to be evaluated for the threshold. 
   • Threshold: The threshold defines the number of test results that will fail a build. The default value is 0. A build is considered unstable or failed when either of the following conditions are met:
     • The number of total passed/executed tests is less than threshold.
     • The number of total/newly failed tests is greater than threshold.
   • Build status if quality gate fails: Determines if the build should be set to unstable or failed if the quality gate is not passed.
5. Choose Save & Queue > Save.

Running the Build

A typical build configuration would consist of the following steps:

1. Clean the build checkout directory. In this step you will clean the Parasoft XML report files left over from the previous build runs. You can configure the following options:
   1. Set the Clean option to true under the Get sources step.
   2. Configure a Command Line runner to clean the Parasoft XML report files left over from the previous runs. For instance, you could run the following command to remove all XML files from the build Checkout Directory: rm $(System.DefaultWorkingDirectory)/*.xml
2. Run SOAtest, C/C++test, Jtest, or dotTEST. This step will generate report files in XML format. The XML report files must be saved in the Azure DevOps build project Checkout Directory.
3. Run the Parasoft Findings extension. You can have multiple Parasoft tool runs that would generate multiple XML report files prior to invoking this build step. If the build steps prior to the Parasoft Findings extension build step generate multiple XML report files, then all test results from all these files will be added to the Azure DevOps build results.

Viewing Test Results

Test results reports are collected from the Parasoft report XML files and the number of passed/failed tests and result details are shown. Build results can also be displayed as trending data in a graph for a simple visualization.

Viewing Results of the Last Build

Open the Azure DevOps build configuration page and click on the last build results link next to the build name to view the number of passed/failed tests and result details.
The build results overview page tab provides the success and error count:

The build results shown in the Tests tab contains the list of all Parasoft tests that ran in the build.

Clicking the Build step in the build tree displays logging information in case debugging is needed.

Viewing Statistics and Trends

The build results overview page provides a histogram of the number of failing tests for each run. The dashboard also provides widgets for overall test result trends. 
To view historical details of a particular Parasoft test, click on a test and click View History. A page with the detailed test history will open.

C/C++test Unit Test Results

C/C++test reports for unit test results must be generated with the Overview of checked files and executed tests option enabled. You can enable this option on the command line by setting the report.contexts_details=true property in the settings file.

Viewing Static Analysis Results

Static analysis reports are collected from the Parasoft report XML files and display static analysis data and rule documentation. This feature is not supported in a Release pipeline.

Prerequisites

Viewing static analysis reports in Azure DevOps requires the Azure DevOps extension "SARIF SAST Scans Tab" from Microsoft DevLabs. You can download it for free from the Visual Studio Marketplace. Once this extension is installed, you will have a Scans tab on build details pages.

Configuring the Extension for Static Analysis Reports

If you did not configure the extension to include Static Analysis Reports when you set it up the first time, you will need to make a few changes to it:

1. Edit the pipeline with the Publish Parasoft Results task.
2. Make the following changes to the Publish Parasoft Results task, as appropriate:
   • Add minimatch patterns to the Results files field to specify which Parasoft XML report files are included in the Azure DevOps build report.
   • If you want to view rule documentation from your DTP server (note: for DTP 2023.1 or later only):
     1. Create a settings file that contains a dtp.url property equal to your DTP server address. Save this file.
     2. Specify the relative location for the settings file in the Settings field.
3. Click Save.

Viewing Results of the Last Build

After you have completed the prerequisites and configuration steps described above, your builds will have a Scans tab on which static analysis reports can be viewed. In addition to these reports, clicking a linked rule name on the Scans tab will show that rule's documentation. 

C/C++test Professional Report Settings

Reports for static analysis must be generated with the Add absolute file paths to XML data option enabled if you are using C/C++test Professional. You can enable this option on the command line by setting the report.location_details=true property in the settings file, which allows you to navigate to the source code if it is stored in the Azure Repos Git.

Viewing Code Coverage Results

Code coverage reports are collected from the Parasoft report XML files and display summary information and class-specific coverage data. Only data for the last matched coverage report is shown. This feature is not supported in a Release pipeline.

Viewing Results of the Last Build

Open the Azure DevOps build configuration page and click on the last build results link next to the build name and click the Code Coverage tab.

Click a specific class in the Coverage section to view which lines are covered or uncovered.

C/C++test Professional Reports

Code coverage reports for C/C++test Professional are not supported.

Viewing Quality Gate Results

Static Analysis

To configure the extension for static analysis quality gates, see Configuring the Parasoft Static Analysis Quality Gates Task. The summary of static analysis quality gates results will be shown on the Extensions tab.

Code Coverage

To configure the extension for code coverage quality gates, see Configuring the Parasoft Code Coverage Quality Gate Task. The summary of code coverage quality gates results will be shown on the Extensions tab.

Test Results

To configure the extension for quality gates, see Configuring the Parasoft Test Results Quality Gate Task. The summary of quality gates results will be shown on the Extensions tab.

Troubleshooting

"JavaScript heap out of memory" error

This issue can occur when Parasoft Findings processes large reports. Configure the Root path to Parasoft tool/Java installation field to use Java for report processing instead of JavaScript, which is default. You can check the debug logs to see whether Java is being used to process the report or not.

"JAXP00010004" error when using Java to process large reports

This issue can occur when Parasoft Findings processes large reports, even if the Root path to Parasoft tool/Java installation field is set to use Java. It is a limitation of JAXP. Refer to the Using the jaxp.properties File section for informationg regarding changing the value of the jdk.xml.totalEntitySizeLimit property to a large value.

Alternatively, you can add a system environment variable that sets -DtotalEntitySizeLimit and -Djdk.xml.totalEntitySizeLimit to a large number. For example:

JAVA_TOOL_OPTIONS=-DtotalEntitySizeLimit=2147480000 -Djdk.xml.totalEntitySizeLimit=2147480000

Third-party Acknowledgements

The Parasoft Findings Plugin for Azure DevOps uses the following third-party software:

adm-zip

This software is used under an MIT license.

agent-base

This software is used under an MIT license.

asynckit

This software is used under an MIT license.

axios

This software is used under an MIT license.

azure-devops-node-api

This software is used under an MIT license.

azure-pipelines-task-lib

This software is used under an MIT license.

balanced-match

This software is used under an MIT license.

bindings

This software is used under an MIT license.

brace-expansion

This software is used under an MIT license.

call-bind

This software is used under an MIT license.

combined-stream

This software is used under an MIT license.

concat-map

This software is used under an MIT license.

core-util-is

This software is used under an MIT license.

deasync

This software is used under an MIT license.

debug

This software is used under an MIT license.

delayed-stream

This software is used under an MIT license.

dot-properties

This software is used under an MIT license.

file-uri-to-path

This software is used under an MIT license.

follow-redirects

This software is used under an MIT license.

form-data

This software is used under an MIT license.

fs.realpath

This software is used under an ISC license.

function-bind

This software is used under an MIT license.

get-intrinsic

This software is used under an MIT license.

glob

This software is used under an ISC license.

has-symbols

This software is used under an MIT license.

has

This software is used under an MIT license.

has-proto

This software is used under an MIT license.

https-proxy-agent

This software is used under an MIT license.

immediate

This software is used under an MIT license.

inflight

This software is used under an ISC license.

inherits

This software is used under an ISC license.

interpret

This software is used under an MIT license.

is-core-module

This software is used under an MIT license.

isarray

This software is used under an MIT license.

jline(BSD)

This software is used under a BSD 3-Clause license.

jszip

This software is used under an MIT & GPL-3.0 license.

lie

This software is used under an MIT license.

mime-db

This software is used under an MIT license.

mime-types

This software is used under an MIT license.

minimatch

This software is used under an ISC license.

mockery

This software is used under an MIT license.

ms

This software is used under an MIT license.

node-addon-api

This software is used under an MIT license.

node-fetch

This software is used under an MIT license.

nodejs-file-downloader

This software is used under an ISC license.

object-inspect

This software is used under an MIT license.

once

This software is used under an ISC license.

pako

This software is used under an MIT license and a Zlib license.

path-is-absolute

This software is used under an MIT license.

path-parse

This software is used under an MIT license.

process-nextick-args

This software is used under an MIT license.

promise

This software is used under an MIT license.

proxy-from-env

This software is used under an MIT license.

q

This software is used under an MIT license.

qs

This software is used under a BSD 3-Clause license.

readable-stream

This software is used under an MIT license.

rechoir

This software is used under an MIT license.

reportGenerator

This software is used under an Apache 2.0 license.

resolve

This software is used under an MIT license.

safe-buffer

This software is used under an MIT license.

sanitize-filename

This software is used under WFTPL and ISC licences.

sax

This software is used under an ISC license.

saxon-he

This software is used under an MPL 2.0 license.

saxon-js

This software is used under the following license.

semver

This software is used under an ISC license.

setimmediate

This software is used under an MIT license.

shelljs

This software is used under a BSD 3-Clause license.

side-channel

This software is used under an MIT license.

string_decoder

This software is used under an MIT license.

supports-preserve-symlinks-flag

This software is used under an MIT license.

tr46

This software is used under an MIT license.

truncate-utf8-bytes

This software is used under a WTFPL license and an MIT license.

tunnel

This software is used under an MIT license.

typed-rest-client

This software is used under an MIT license.

underscore

This software is used under an MIT license.

utf8-byte-length

This software is used under a WTFPL license and an MIT license.

util-deprecate

This software is used under an MIT license.

uuid

This software is used under an MIT license.

webidl-conversions

This software is used under a BSD 2-Clause license.

whatwg-url

This software is used under an MIT license.

wrappy

This software is used under an ISC license.

xmlresolver

This software is used under an Apache 2.0 license.

xmlresolver-data

This software is used under an W3C license.

xslt3

This software is used under the following license.