This topic covers how to analyze and correct static analysis violations.
Section include:
Accessing Results
Results can be accessed from a variety of locations in the GUI, as well as from command-line reports.
For static analysis run in the GUI, results are reported in the Quality Tasks view.
Open a tested source file in the editor and source code that violates the static analysis rule(s) is flagged with markers placed next to the line of code responsible for the violation. Mouse over the marker and review the static analysis rule and other information in the popup window. You can also open the Quality Tasks view message by right-clicking the source code responsible for the problem and choosing Parasoft> Show in Tasks.
For static analysis executed from the command line interface, violations are reported in the Static Analysis section of the report. If violations were sent to Team Server or Development Testing Platform, they can be imported into the GUI as described in the Importing Results into the UI. They will then be available in the Quality Tasks view.
Static analysis errors are not a criteria for functional test failures. For example, tests may execute successfully when you can run static analysis on a web scenario that includes a Browser Playback tool or a Scanning tool, but if static errors are found in the referenced pages, the tests will pass and the static analysis errors will be reported in the Quality Tasks view.
Results from Tests Run in the GUI
Test Progress View
The Test Progress view reports test progress and status.
Note that:
- When a test runs, the view label changes from "Test Progress" to "Testing [Test Configuration name].
- Clicking the Review tasks button displays the results in the Quality Tasks view.
- A results summary for each analysis category is available in expandable sections.
- The toolbar buttons in the upper right corner of the Test Progress view allow you to generate reports.
This opens the Report dialog from which you can configure Report Preferences.
Quality Tasks View
Your assigned quality tasks generated during interactive testing or imported from command-line tests are shown in the Quality Tasks view. If this view is not available, choose Parasoft> Show View> Quality Tasks to open it. To see additional details, drill down into the Quality Tasks view tree. To toggle through the items reported in this view, use the arrow buttons in the view’s toolbar.
The results are presented as a task list that helps you determine how to proceed with testing and code improvement. Tasks are organized by author, category, then by severity (if assigned). Severity levels range from 1 to 5— Severity 1 tasks are estimated to have the greatest chance of eliminating or preventing a critical bug, while Severity 5 tasks are estimated to have the least chance of eliminating or preventing a critical bug.
Tasks are organized into different categories, depending on the product capabilities. To learn about the applicable categories, see the related product documentation.
Source Code Markers
For tests that were run on source files, results are also reported at the source code level.
If you open the editor for a tested source file, markers will be placed next to the source code responsible for problems found. For static code analysis violations, markers are placed next to the line of code responsible for the violation. For unit test errors, markers are placed on the first line of the stack trace that matches the tested class. For unit test failures or for errors where the tested class is not known, markers are placed on the first line of the stack trace that matches the unit test class. To learn what problem a particular marker indicates, place your mouse over the marker and review the information in the popup window. Or, to go directly to the related Quality Tasks view message, right-click the source code responsible for the problem, choose Show In> Quality Tasks (for Eclipse) or Parasoft> Show in Quality Tasks
Console View
To see testing details, open the Console view during test execution. Testing details are reported here when a test is in process, and remain there until they are cleared or until another test is run.
Results from Tests Run from the Command Line
For tests run from the command line, results are recorded in the generated report. If results were sent to Team Server, results can be imported into the GUI as described in Importing Results into the UI. You can then review the results as if the test had been performed in the GUI.
Modifying the Results Layout
The default Quality Tasks view layout is designed for displaying functional test results.
To facilitate the review of static analysis results, SOAtest provides additional two layouts:
- SOAtest Static Analysis Layout: This option is recommended if you are running static analysis against source code. (e.g., from the Scanning perspective).
- SOAtest Static Analysis for Functional Tests Layout: This option is recommended if you are running static analysis by executing a test suite (e.g., a test suite that contains a Browser Playback tool or a Scanning tool). With this layout, SOAtest will report the test that uncovered each static analysis error, as well as the name of the file where the static analysis error occurs. If the error occurs in browser-constructed HTML, SOAtest will report the "Browser-Constructed HTML (<Browser name and version>)" instead of a specific file name.
To choose one of the static analysis layouts:
- Open the pull-down menu on the top right of the Quality Tasks view.
- Choose one of the available formats from the Show shortcut menu that opens.
Responding to Results
For each violation reported, we recommend that you and your team review the rule description and the related code, then decide whether:
- The violation is valid and significant (and the violation should be corrected)
- The rule does not apply in that particular context (and the violation should be suppressed)
- The rule is not well-suited to your projects or priorities (and the related rule should be disabled)
Many teams like to review SOAtest’s static analysis violations during code reviews. Developers check their code using the rules selected by the team’s architect and/or manager. If a developer thinks that it makes sense to ignore a particular rule violation, that developer discusses this at the code review. The team then decides whether the violation should be suppressed, the rule should be disabled, or the violation should be corrected.
Learning More About Violated Rules
The SOAtest rule descriptions can help you determine which rules your team wants to follow, understand how reported violations can impact application reliability, security, maintainability, etc., and learn how to correct reported violations.
To view a rule description file, right-click the static analysis violation message in the Quality Tasks view, then choose View Documentation from the shortcut menu. A yellow "Yield" sign marks the node that you should right-click.
Tips
To learn about the static analysis rules that are included with SOAtest, choose Help> Help Contents, then open the SOAtest Static Analysis Rules book, then browse the available rule description files.
To view a list of all static analysis rules that a given Test Configuration is configured to check, as well as descriptions of all enabled rules:
- Open the Test Configurations dialog by choosing Parasoft> Test Configurations or by choosing Test Configurations in the drop-down menu on the Test Using toolbar button.
- Select the Test Configurations category for which you want a rule list.
- Open the Static tab.
- Click Printable Docs.
If you want to print this list of rules and all related rule descriptions, enable your browser’s Print all linked documents printer option before you print the main the list of rules (the index.html page). If you want to generate a PDF, create a PDF from the index.html page; be sure to configure the PDF generator to include linked pages (in Acrobat, by enabling the Get entire site option).
If you would rather check if your code follows a single coding standard, a predefined category of coding standards, or all available coding standards, you can check coding standards "on the fly." To check coding standards in this way:
- In the Navigator for the SOAtest perspective, select the resource that you want to test.
- Do one of the following:
- Choose Test Using> Built-in> Static Analysis> [desired rule or category of rules] from the Test Using pull-down menu.
- Choose Parasoft> Test Using> Built-in> Static Analysis> [desired rule or cate-gory of rules].
Reviewing and Correcting the Responsible Source Code
To view the source code responsible for the rule violation, double-click the node that shows the line number, or right-click that node and choose Go to from the shortcut menu. The editor will then open and highlight the designated line of code.
You can make the necessary modifications, then save the modified file.
Can’t see the source associated with a static analysis task you imported from Team Server?
If you can’t view the source code for an imported static analysis task (for instance, the source for the incorrect page is being shown), it’s probably because your local SOAtest installation does not have access to the login information required to produce that page.
If you rerun that test in your SOAtest GUI, you should be able to see the code related to the reported violation.
