In this section:
Introduction
The Parasoft Findings extension is designed to be used in a post-test execution build step that integrates functional and unit test reports as well as static analysis reports and code coverage reports generated by Parasoft tools into your Azure DevOps project. The extension can consume the following report types:
- Parasoft SOAtest 2020.1+ XML reports.
- Parasoft Analyzers 2020.1+ XML reports generated by Parasoft C/C++test, Jtest and dotTEST tools.
Requirements
- Agent version 2.144.0 or higher.
Installing the Parasoft Findings Extension
- Sign into the Visual Studio Marketplace and click the Azure DevOps tab.
- Find and select the Parasoft Findings Extension.
- Click Install.
See https://docs.microsoft.com/en-us/azure/devops/marketplace/install-extension?view=vsts in the Azure DevOps documentation for additional information.
Configuring the Extension
- Edit an existing pipeline or create a new pipeline and click Add Task.
- Search for "Publish Parasoft Results" using the search field.
- Select the extension and click Add.
- Review the Results Files field. The predefined minimatch pattern is configured to read the test results, static analysis, and code coverage reports generated by Parasoft tools. You can modify or remove parts of the pattern to suit your needs. See Microsoft's documentation on file matching patterns for more information on working with matching patterns.
- Static Analysis reports require you to install an extension to Azure DevOps to view; see Viewing Static Analysis Reports for more information.
- (Optional) Configure the Root path to Parasoft tool/Java installation field. You can enter the absolute location of the root path of a Parasoft tool (Parasoft tools contain a Java installation) or a standalone Java installation to significantly optimize the efficiency of report processing. This is particularly effective (and recommended) for large projects.
- Click Save under the Save & Queue menu when finished.
Running the Build
A typical build configuration would consist of the following steps:
- Clean the build checkout directory. In this step you will clean the Parasoft XML report files left over from the previous build runs. You can configure the following options:
- Set the Clean option to true under the Get sources step.
- Configure a Command Line runner to clean the Parasoft XML report files left over from the previous runs. For instance, you could run the following command to remove all XML files from the build Checkout Directory: rm $(System.DefaultWorkingDirectory)/*.xml
- Run SOAtest, C/C++test, Jtest, or dotTEST. This step will generate report files in XML format. The XML report files must be saved in the Azure DevOps build project Checkout Directory.
- Run the Parasoft Findings extension. You can have multiple Parasoft tool runs that would generate multiple XML report files prior to invoking this build step. If the build steps prior to the Parasoft Findings extension build step generate multiple XML report files, then all test results from all these files will be added to the Azure DevOps build results.
Viewing Test Results Reports
Test results reports are collected from the Parasoft report XML files and the number of passed/failed tests and result details are shown. Build results can also be displayed as trending data in a graph for a simple visualization.
Viewing Results of the Last Build
Open the Azure DevOps build configuration page and click on the last build results link next to the build name to view the number of passed/failed tests and result details.
The build results overview page tab provides the success and error count:
The build results shown in the Tests tab contains the list of all Parasoft tests that ran in the build.
Clicking the Build step in the build tree displays logging information in case debugging is needed.
Viewing Statistics and Trends
The build results overview page provides a histogram of the number of failing tests for each run. The dashboard also provides widgets for overall test result trends.
To view historical details of a particular Parasoft test, click on a test and click View History. A page with the detailed test history will open.
C/C++test Unit Test Results
C/C++test reports for unit test results must be generated with the Overview of checked files and executed tests option enabled. You can enable this option on the command line by setting the report.contexts_details=true property in the settings file.
Viewing Static Analysis Reports
Static analysis reports are collected from the Parasoft report XML files and display static analysis data and rule documentation.
Prerequisites
Viewing static analysis reports in Azure DevOps requires the Azure DevOps extension "SARIF SAST Scans Tab" from Microsoft DevLabs. You can download it for free from the Visual Studio Marketplace. Once this extension is installed, you will have a Scans tab on build details pages.
Configuring the Extension for Static Analysis Reports
If you did not configure the extension to include Static Analysis Reports when you set it up the first time, you will need to make a few changes to it:
- Edit the pipeline with the Publish Parasoft Results task.
- Make the following changes to the Publish Parasoft Results task, as appropriate:
- Add minimatch patterns to the Results files field to specify which Parasoft XML report files are included in the Azure DevOps build report.
- If you want to view rule documentation from your DTP server (note: for DTP 2023.1 or later only):
- Create a settings file that contains a
dtp.urlproperty equal to your DTP server address. Save this file. - Specify the relative location for the settings file in the Settings field.
- Create a settings file that contains a
- Click Save.
Viewing Results of the Last Build
After you have completed the prerequisites and configuration steps described above, your builds will have a Scans tab on which static analysis reports can be viewed. In addition to these reports, clicking a linked rule name on the Scans tab will show that rule's documentation.
C/C++test Professional Report Settings
Reports for static analysis must be generated with the Add absolute file paths to XML data option enabled if you are using C/C++test Professional. You can enable this option on the command line by setting the report.location_details=true property in the settings file, which allows you to navigate to the source code if it is stored in the Azure Repos Git.
Viewing Code Coverage Reports
Code coverage reports are collected from the Parasoft report XML files and display summary information and class-specific coverage data. Only data for the last matched coverage report is shown.
Viewing Results of the Last Build
Open the Azure DevOps build configuration page and click on the last build results link next to the build name and click the Code Coverage tab.
Click a specific class in the Coverage section to view which lines are covered or uncovered.
C/C++test Professional Reports
Code coverage reports for C/C++test Professional are not supported.
Third-party Acknowledgements
The Parasoft Findings Plugin for Azure DevOps uses the following third-party software:
asap
This software is used under an MIT license.
asynckit
This software is used under an MIT license.
axios
This software is used under an MIT license.
azure-pipelines-task-lib
This software is used under an MIT license.
balanced-match
This software is used under an MIT license.
brace-expansion
This software is used under an MIT license.
buffer-from
This software is used under an MIT license.
call-bind
This software is used under an MIT license.
caseless
This software is used under an Apache 2.0 license.
combined-stream
This software is used under an MIT license.
concat-map
This software is used under an MIT license.
concat-stream
This software is used under an MIT license.
core-util-is
This software is used under an MIT license.
delayed-stream
This software is used under an MIT license.
dot-properties
This software is used under an MIT license.
follow-redirects
This software is used under an MIT license.
form-data
This software is used under an MIT license.
fs.realpath
This software is used under an ISC license.
function-bind
This software is used under an MIT license.
get-intrinsic
This software is used under an MIT license.
get-port
This software is used under an MIT license.
glob
This software is used under an ISC license.
has-symbols
This software is used under an MIT license.
has
This software is used under an MIT license.
http-basic
This software is used under an MIT license.
http-response-object
This software is used under an MIT license.
inflight
This software is used under an ISC license.
inherits
This software is used under an ISC license.
interpret
This software is used under an MIT license.
is-core-module
This software is used under an MIT license.
isarray
This software is used under an MIT license.
jline(BSD)
This software is used under a BSD 3-Clause license.
mime-db
This software is used under an MIT license.
mime-types
This software is used under an MIT license.
minimatch
This software is used under an ISC license.
mockery
This software is used under an MIT license.
object-inspect
This software is used under an MIT license.
once
This software is used under an ISC license.
parse-cache-control
This software is used under a BSD 3-Clause license.
path-is-absolute
This software is used under an MIT license.
path-parse
This software is used under an MIT license.
process-nextick-args
This software is used under an MIT license.
promise
This software is used under an MIT license.
q
This software is used under an MIT license.
qs
This software is used under a BSD 3-Clause license.
readable-stream
This software is used under an MIT license.
rechoir
This software is used under an MIT license.
reportGenerator
This software is used under an Apache 2.0 license.
resolve
This software is used under an MIT license.
safe-buffer
This software is used under an MIT license.
sax
This software is used under an ISC license.
saxon-he
This software is used under an MPL 2.0 license.
saxon-js
This software is used under the following license.
semver
This software is used under an ISC license.
shelljs
This software is used under a BSD 3-Clause license.
side-channel
This software is used under an MIT license.
string_decoder
This software is used under an MIT license.
supports-preserve-symlinks-flag
This software is used under an MIT license.
sync-request
This software is used under an MIT license.
sync-rpc
This software is used under an MIT license.
then-request
This software is used under an MIT license.
typedarray
This software is used under an MIT license.
util-deprecate
This software is used under an MIT license.
uuid
This software is used under an MIT license.
wrappy
This software is used under an ISC license.
xmlresolver
This software is used under an Apache 2.0 license.
xslt3
This software is used under the following license.
