This topic provides a general introduction to the reports that SOAtest produces for GUI and cli tests. Report details will vary based on report settings, the Test Configuration used, and the errors found.

Sections include:

Report Types

Two types of  reports can be produced from the command line interface:

• Comprehensive reports: Reports that contain all tasks generated for a single test run.
• Individual reports: Reports that contain only tasks assigned to the specified team member.

For example, if a test generated 5 tasks for Tom and 10 tasks for Joe, the comprehensive report would contain all 15 tasks, Tom’s report would contain 5 tasks, and Joe’s report would contain 10 tasks.

Report Contents

Reports may contain the following sections:

Header/Navigation Bar

The top left cell of the header/navigation bar shows the time and date of the test. The remaining cells (Static Analysis, Test Execution) each link to the named report section.

Static Analysis Section

The Static Analysis section includes several items:

• The Static Analysis trends graph tracks how the total number of lines of code in the project, the lines of project code that were checked during static analysis, and the total number of  reported static analysis tasks vary over time. This graph is created only for tests that are run from the command line and that use the -publish command.
  
  
  
  
• The Overview table shows a basic summary of all static analysis tasks for the tested project(s). It reports the total number of static analysis tasks, the number files checked during static anal-ysis, the total number of project files, the number of lines of code checked during static analy-sis, and the total number of lines of code in the project. It also reports the total time spent performing static analysis.
• The All Tasks by Category table shows the total number of tasks reported for each static analysis rule category and rule. Tasks can be sorted by rule category or rule severity; click the appropriate link in the table header to change the table sorting.
• The Tasks per Author table shows the number of static analysis tasks assigned to each team member. To see details about the static analysis tasks assigned to a particular team member, click the related username.
  • If a team member’s name is listed in green, it means that there are no “recommended static analysis tasks" reported for that team member. "Recommended tasks" are the subset of all reported tasks that  Parasoft Test has selected for that team member to review and address today. This is based on the maximum number of tasks per team member that your team has configured Parasoft Test to report, as described in Goals Tab Settings - Defining Error Reporting and Resolution Targets, and task assignment settings, as described in Configuring Task Assignment and Code Authorship Settings.
  • If tests are being run on a regular basis (nightly, weekly, etc.) this table will also report the "delta total" for each author. This tells you the change in the total tasks relative to the previous test run (e.g., +2 or -5).
• The Task Details section provides details about each reported task.
• The Checked Files (Details) section lists all the files that were checked. For each file, it lists the number of rule violations and the number of suppressed violations. If the file has a violation, it also lists  the line number, rule name, and rule ID for that violation.
• The Active Rules section lists the names and IDs of all rules that were enabled for the test.

Test Execution Section

The Test Execution section includes several items:

• The Test Execution Tasks trends graph tracks how the number of functional test failures change over time. This graph is created only for tests that are run from the command line interface and that use the -publish command.
  
  
  
  
• The Test Suite Summary provides a breakdown of failed tests, successful tests, total test, and the success %. Test suites with failed tests are highlighted in pink.
  
  
  
  
• The Tasks per Author table shows the number of testing tasks assigned to each team member. To see details about the testing tasks assigned to a particular team member, click the related username.
  • If a team member’s name is listed in green, it means that there are no “recommended testing tasks" reported for that team member. "Recommended tasks" are the subset of all reported tasks that SOAtest has selected for that team member to review and address today. This is based on the maximum number of tasks per team member that your team has configured SOAtest to report, as described in Goals Tab Settings - Defining Error Reporting and Resolution Targets, and task assignment settings, as described in Configuring Task Assignment and Code Authorship Settings.
  • If tests are being run on a regular basis (nightly, weekly, etc.) this table will also report the "delta total" for each author. This tells you the change in the total tasks relative to the previous test run (e.g., +2 or -5).
    
• The Task Details section provides details about each reported task.
• The Executed Tests (Details) section lists all executed test cases and their outcomes (pass or fail). For each test suite, it lists the total number of test cases and the number of passed test cases. If a task is reported for a test case, additional details (stack trace, outcome, etc.) are presented.

Team Server Report Link

This link allows you to directly browse to this and other report files available on Team Server. In the reports available on Team Server, all links (for instance, links to Category) are active. All links are not active in emailed reports. Thus, if you want to explore an emailed report in more detail, we recommend that you follow this link and access the report on Team Server.