This topic provides a general introduction to the reports that SOAtest produces for GUI and cli tests. Report details will vary based on report settings, the Test Configuration used, and the errors found. Not all of the report elements described below will be present in all reports.
Sections include:
Two types of reports can be produced from the command line interface:
For example, if a test generated 5 tasks for Tom and 10 tasks for Joe, the comprehensive report would contain all 15 tasks, Tom’s report would contain 5 tasks, and Joe’s report would contain 10 tasks.
Reports may contain the following sections:
In addition to the report name, the header shows the tool used for the analysis.
The Session Summary section includes high-level information about the report and may include the following:
The Summary - Static Analysis summary section appears in the report whenever you perform static analysis on your project. It shows an overview of findings as a donut chart with various colors representing severities. The same information is shown in a table next to the chart and the total number of findings is shown below.
The Summary - Functional Tests section appears in the report whenever a functional test is run. It will show one or more donut charts representing test coverage, API coverage, or impacted assets, based upon the type of tests you are running.
The Static Analysis details section appears in the report whenever you perform static analysis on your project and shows a table that summarizes findings as well as detail reports of those findings by type and author.
The Static Analysis table contains the following information:
Below this table, findings are listed by rule with the number of violations for each shown in brackets. You can sort these findings by category or severity by clicking the Category or Severity link to the right.
Below the All Findings list is a Findings by Author list showing findings by author associated with the analyzed code. Click an author to view their findings details.
The Details - Functional Tests section appears in the report whenever a functional test is run. It shows a table that summarizes test results as well as detail reports of findings by author.
The Test Suite Summary table contains the following information:
Below this table, findings are listed by author. Click an author to view their finding details. In addition, you can click the error message summary for a finding to view the detailed error report and click View Traffic for any finding to see its associated request and response.
The Details - API Coverage section appears in the report whenever you run tests with 'Calculate API coverage' enabled in the test configuration. It shows a table that summarizes how well your tests covered the related resources.
The API Coverage table contains the following information:
The table is hierarchical. Coverage is calculated for a service as a whole, as well as each resource/method and operation and the tree branches can be collapsed and expanded.
If you want to see what .tsts and specific tests covered a certain method, drill down into the tree, then click the Show tests link. For quick expansion and collapsing, you can use the Expand All / Collapse All links at the top left of the report.
The Details - Change Impact section appears in the report whenever you run change impact analysis against your services. It shows a table that summarizes assets affected by recent changes.
The Asset Summary table contains the following information:
Below this table, findings are listed by author with total changes and changes per asset shown. Click an author to view their finding details and click a findings link to view a more detailed report.
The API Security Issues section appears in the report whenever you run penetration tests against your project.
The API Security Issues table gives an overview of issues found, either by CWE or OWASP 2021 top 10 (as determined by your preference setting in Parasoft > Preferences > Reports > API Security), in a matrix sorted by risk level and confidence. Below that, security findings are sorted by CWE number or OWASP 2021 Top 10 number. Click either the CWE or OWASP number or description to jump to that section of the report for more details about those findings.
In addition to a brief description of the issue, you will also find instance URLs with their associated tests and other information, proposed solutions, and links to additional resources. You can also click View Traffic for any instance URL to see its associated request and response.