This topic describes the settings available for SOAtest Test Configurations.

Sections include:


The general procedures related to configuring and sharing Test Configurations are standardized across the Parasoft Test family, and are discussed in Configuring Test Configurations and Rules for Policies.

Scope Tab Settings: Defining What Code is Tested

For source code static analysis only.

During a test, SOAtest will perform the specified action(s) on all code in the selected resource that satisfies the scope criteria for the selected Test Configuration. By default, SOAtest operates on all files in the selected project or asset. However, you can use the Scope tab to configure restrictions such as:

Note that some file filters and line filters are only applicable if you are working with projects that are under supported source control systems.

The Scope tab has the following settings:


Tips

  • Perl-style expressions can be used.
  • The following wildcards are supported:
    • *   matches 0 or more characters except  '/'.
    • ?  matches any single character except '/'.
    • ** matches 0 or more characters, including '/'. This allows you to include path elements.
  • The following sample elements are added by default to the Code Review configuration:
    • **/bin/**/*.properties is added to the sample list of rejected wildcards.
    • (.*?/(bin|obj)(/x86|/x64){0,1}/(Debug|Release)/.*?\\.(dll|exe|pdb))$ is added to the sample list of rejected regexps.

  • Regular expressions can be used to identify specific differences. For instance, if you want to flag only source code changes that add, remove, or modify TODO tags, you would use the Differences regular expression
    .*TODO.*

Examples

A basic file mask might be:

  • *.java, *.xml, *.properties
  • *.c, *.cpp, *.h, *.cc, *.hpp, makefile, .project, .classpath
  • *.c, *.cpp, *.h, *.cc, *.hpp, makefile, *.sln, *.prj, *.res
  • *.cs, *.vb, *.sln, *.prj, *.resx

To include every file whose path has a folder named "bank" or "customer", use: 

  • **/bank/**, **/customer/**

To include every file whose path has a folder with a name that either starts with "bank", includes "customer", or ends with "invoice", use: 

  • **/bank*/**, **/*customer*/**, **/*invoice/**

To include every .java file that 1) has name that starts with "Test", and 2) is located in a folder named "security" (which is within the src/test directory of any project), use:

**/src/test/**/security/Test*.java 

To include every .cs file that 1) is in the ATM solution, 2)  is in the ATMLib project, 3) is within the CompanyTests subfolder, 4) has a name that starts with "Test", use:

ATM/ATMLib/CompanyTests/**/Test*.cs 



Code authorship information and last modified date is determined in the manner set in the Scope and Authorship preferences page; for details about available settings, see Configuring Task Assignment and Code Authorship Settings.

Static Tab Settings: Defining How Static Analysis is Performed

During a test, SOAtest will perform static analysis based on the parameters defined in the Test Configuration used for that test. 

The  Static tab has the following settings:


  • The number next to each rule ID indicates the rule’s severity level. The severity level indicates the chance that a violation of the rule will cause a serious construction defect (a coding construct that causes application problems such as slow performance, security vulnerabilities, and so on). Possible severity levels (listed from most severe to least severe) are:
    • Highest - Level 1
    • High  - Level 2
    • Medium  - Level 3
    • Low  - Level 4
    • Lowest  - Level 5
  • To learn about the rules that are included with SOAtest, choose Help> Help Contents, then open the SOAtest Static Analysis Rules book, then browse the available rule description files.
  • To generate a printable list of all rules that a given Test Configuration is configured to check:
    1. Open the Test Configurations panel by choosing Parasoft> Test Configurations.
    2. Select the Test Configurations category that represents the user-defined Test Configuration you want to modify.
    3. Open the Static tab.
    4. Click the PrintableDocs button.


Execution Tab Settings: Defining How Tests are Executed

During a test, SOAtest will execute test cases based on the parameters defined in the selected Test Configuration’s Execution tab. For any level of execution, the top-level Enable Test Execution option must be enabled.

Functional tab

The Execution> Functional tab has the following settings:

Security tab

The Execution> Security tab allows you to configure penetration testing, which is described in Penetration Testing.

Runtime Error Detection tab

The Execution> Runtime Error Detection tab allows you to configure runtime error detection, which is described in Performing Runtime Error Detection.

Change Impact  tab

The Execution> Change Impact tab contains an option (Perform change impact analysis) that controls whether the current Test Configuration performs change impact analysis during test execution. See Updating Messages with Change Advisor for details.

API Coverage tab

The Execution> API Coverage tab contains options for calculating API Coverage during test execution. See API Coverage for details.

Application Coverage tab

The Execution> Application Coverage tab contains options for collecting application coverage data, which provides visibility into the level of code coverage achieved by your SOAtest tests. See Application Coverage for details.

Common Tab Settings: Defining Common Options that Affect Multiple Analysis Types 

The Test Configuration’s Common tab controls test settings for actions that affect multiple analysis types.

The Common tab has the following settings:

Code Review Tab Settings: Defining Code Review Options

This tab contains settings for automating preparation, notification, and tracking the peer review process, which can be used to evaluate critical SDLC artifacts (source files, tests, etc.) in the context of the organization’s defined quality policies.

For details on configuring Code Review (including details on Code Review tab options), see Code Review. 

Goals Tab Settings: Defining Error Reporting and Resolution Targets

The team manager can specify a reporting limit (such as "Do not report more than 25 static analysis tasks per developer per day") and/or a quality goal (such as "All static analysis violations should be fixed in 2 months"). SOAtest will then use the specified criteria to select a subset of testing tasks for each developer to perform each day. These goals are specified in the Goals tab. Progress towards these goals can then be monitored in reports.

Alternatively, you can set global team goals—goals that may span across multiple Test Configurations and even across Parasoft Test products—as described in Configuring Task Goals. This requires Team Server and a Parasoft Test Automation edition license. If goals are set globally, the Goals tab in the Test Configuration panel will be disabled.

The Goals tab has the following settings:

Static tab

Execution tab