Page tree

Skip to end of metadata
Go to start of metadata

In this section:

Overview

Test configurations define how your code is analyzed and tested, including which static analysis rules are enabled, which tests to run, and other analysis parameters. dotTEST ships with built-in test configurations, but users can create and store their own test configurations in the DTP server (see the DTP documentation for details).

User-defined test configurations that are stored in DTP can be downloaded from the DTP server and stored in the [INSTALL_DIR]/configs/user directory as *.properties files. 

Running a Test Configuration

You can specify which configuration will be run in one of the following ways:

  • Run dottestcli with the -config switch and specify a built-in, user-defined or  DTP-hosted  test configuration:

    -config "builtin://Recommended Rules"
    -config "user://Foo Configuration"
    -config "dtp://Foo Team Configuration" 
    -config "dtp://FooTeamConfig.properties"

    (info)Alternatively, you can add the prefix of the Parasoft tool you are using to specify a DTP-hosted test configuration:

    -config "dottest.dtp://Foo Team Configuration" 

    You can also provide a path or URL to the test configuration .properties file:

    -config "C:\Devel\Configs\FooConfig.properties"
    -config "http://foo.bar.com/configs/FoodConfig.properties"


    For example, your command line may resemble the following:

    dottestcli.exe -solution "C:\Devel\MyFooSolution\MySolution.sln" 
    -config "builtin://Demo" -report "C:\Report"

  • In the .properties file, specify the default configuration that will be run when the -config option is not used:

    dottest.configuration=user://Configuration Name

Viewing Available Test Configurations

Use the -listconfigs switch to print the available test configurations. 

Built-in Test Configurations

The following tables include the test configurations shipped in the [INSTALL]/configs/builtin directory.

Static Analysis

This group includes universal static analysis test configurations. See Security Compliance Pack for test configurations that enforce security coding standards.

Built-in Test ConfigurationDescription
Recommended RulesThe default configuration of recommended rules. Covers most Severity 1 and Severity 2 rules. Includes rules in the Flow Analysis Fast configuration. 
Recommended .NET Core RulesIncludes rules that identify high-severity defects in .NET Core projects.
Find Duplicated CodeApplies static code analysis rules that report duplicate code. Duplicate code may indicate poor application design and lead to maintainability issues.
MetricsComputes values for  several code metrics. 
Flow AnalysisDetects complex runtime errors without requiring test cases or application execution. Defects detected include using uninitialized or invalid memory, null pointer dereferencing, array and buffer overflows, division by zero, memory and resource leaks, and dead code. This requires a special Flow Analysis license option.
Flow Analysis AggressiveIncludes rules for deep flow analysis of code. A significant amount of time may be required to run this configuration.
Flow Analysis FastIncludes rules for shallow depth of flow analysis, which limits the number of potentially acceptable defects from being reported.
Critical RulesIncludes most Severity 1 rules, as well as rules in the Flow Analysis Fast configuration.  
DemoIncludes rules for demonstrating various techniques of code analysis. May not be suitable for large code bases.
Find Memory IssuesIncludes rules for finding memory management issues in the code. 
Find Unimplemented ScenariosIncludes rules for finding unimplemented scenarios in the code. 
Find Unused Code Includes rules for identifying unused/dead code.
Check Code Compatibility against .NET [2.0, 3.0, 3.5, 4.0 Client Profile, 4.0 Full, 4.5, 4.5.1, 4.5.2, 4.6.0, 4.6.1, 4.6.2, 4.7]Includes a set of test configurations that validates the code’s compatibility with the specified version of .NET framework. 
IEC 62304 (Template) A template test configuration for applying the IEC 62304 Medical standard. 
Microsoft Managed Recommended RulesApplies the Microsoft Managed Recommended Rules that identify the most critical issues in your managed code

Security Compliance Pack

This compliance pack includes test configurations that help you enforce security coding standards and practices. See Compliance Packs Rule Mapping for information how the standards are mapped to dotTEST's rules.

(info) Security Compliance Pack requires dedicated license features to be activated. Contact Parasoft Support for more details on licensing.

Displaying compliance results on DTP

Some test configurations in this category have a corresponding "Compliance" extension on DTP, which allows you to view your security compliance status, generate compliance reports, and monitor the progress towards your security compliance goals.  See the "Extensions for DTP" section in the DTP documentation for the list of available extensions, requirements, and usage.

Built-in Test ConfigurationDescription
CWE 3.2

Includes rules that find issues identified in the CWE standard v3.2.

(info) This test configuration is part of Parasoft Compliance Pack solution that allows you to monitor compliance with industry standards using the "Compliance" extensions on DTP.

CWE SANS Top 25 2011

Includes rules that find issues classified as Top 25 Most Dangerous Programming Errors of the CWE-SANS standard. 

(info) This test configuration is part of Parasoft Compliance Pack solution that allows you to monitor compliance with industry standards using the "Compliance" extensions on DTP.

CWE SANS Top 25 2011 + On the Cusp

Includes rules that find issues classified as Top 25 Most Dangerous Programming Errors of the CWE-SANS standard or included on the CWE - On the Cusp list.

(info) This test configuration is part of Parasoft Compliance Pack solution that allows you to monitor compliance with industry standards using the "Compliance" extensions on DTP.

OWASP Top 10-2017

Includes rules that find issues identified in OWASP’s Top 10 standard.

(info) This test configuration is part of Parasoft Compliance Pack solution that allows you to monitor compliance with industry standards using the "Compliance" extensions on DTP.

PCI DSS 3.2Includes rules that find issues identified in PCI Data Security Standard  version 3.2.
Security AssessmentGeneral test configuration that finds security issues.
UL 2900Includes rules that find issues identified in the UL-2900 standard.
Microsoft Secure Coding GuidelinesIncludes rules that enforce Microsoft Secure Coding Guidelines.

Unit Testing and Collecting Coverage

This group includes test configurations that allow you to run and collect coverage data for unit tests.

Built-in Test ConfigurationDescription
Run VSTest TestsRuns NUnit, MSTest, and xUnit tests that are found in the scope of analysis.
Run VSTest Tests with Coverage Runs NUnit, MSTest, and xUnit tests that are found in the scope of analysis and monitors coverage.
Run NUnit TestsRuns NUnit tests that are found in the scope of analysis.
Run NUnit Tests with CoverageRuns NUnit tests that are found in the scope of analysis and monitors coverage.
Execute MSTestsExecutes MSTest tests. See Unit Testing.
Execute MSTests with CoverageExecutes MSTest tests and collects coverage. See Unit Testing.
Calculate Application CoverageProcesses the application coverage data to generate a coverage.xml file. See Application Coverage for Web Applications.
Collect Static CoverageGenerates the static coverage data necessary for application coverage. See Application Coverage for Web Applications.

Compliance Packs Rule Mapping

This section includes rule mapping for the CWE standard. The mapping information for other standards is available in the PDF rule mapping files shipped with Compliance Packs.

CWE SANS Top 25 2011 Mapping

CWE IDCWE NameParasoft Rule ID(s)
CWE-89Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')
  • CWE.89.VPPD
  • CWE.89.TDSQL,
  • CWE.89.TDSQLC
CWE-78Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')
  • CWE.78.VPPD
  • CWE.78.TDCMD
  • CWE.78.AUPS
CWE-120Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')
  • CWE.120.AUK
CWE-79Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
  • CWE.79.VPPD
  • CWE.79.TDRESP
  • CWE.79.TDXSS
CWE-306 Missing Authentication for Critical Function
  • CWE.306.ADSVSP
CWE-862Missing Authorization
  • CWE.862.UAA
CWE-798Use of Hard-coded Credentials
  • CWE.798.HARDCONN
  • CWE.259.HPW
CWE-311Missing Encryption of Sensitive Data
  • CWE.316.RSFSS
  • CWE.316.SSFP
CWE-434Unrestricted Upload of File with Dangerous Type
  • CWE.434.TDFNAMES
CWE-807Reliance on Untrusted Inputs in a Security Decision
  • CWE.807.AUTH
  • CWE.350.IIPHEU
CWE-250Execution with Unnecessary Privileges
  • CWE.250.AUEP
CWE-352Cross-Site Request Forgery (CSRF)
  • CWE.352.VPPD
  • CWE.352.TDRESP
  • CWE.352.VAFT
CWE-22Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')
  • CWE.22.TDFNAMES
CWE-494Download of Code Without Integrity Check
  • CWE.494.IREC
CWE-863Incorrect Authorization
  • CWE.863.AAM
  • CWE.863.UAAMC
CWE-829Inclusion of Functionality from Untrusted Control Sphere
  • CWE.829.DMSC
  • CWE.829.ADLL
  • CWE.827.PDTDP
CWE-732Incorrect Permission Assignment for Critical Resource
  • CWE.732.ADSVSP
CWE-676Use of Potentially Dangerous Function
  • CWE.676.APDM
CWE-327Use of a Broken or Risky Cryptographic Algorithm
  • CWE.327.ICA
  • CWE.327.DNCCKS
  • CWE.327.ACCA
CWE-131Incorrect Calculation of Buffer Size
  • CWE.131.AUK
CWE-307Improper Restriction of Excessive Authentication Attempts
  • CWE.307.LUAFLA
CWE-601URL Redirection to Untrusted Site ('Open Redirect')
  • CWE.601.VPPD
  • CWE.601.TDNET
CWE-134Use of Externally-Controlled Format String
  • CWE.134.TDINPUT
CWE-190Integer Overflow or Wraparound
  • CWE.190.AIWIL
  • CWE.190.INTOVERF
CWE-759Use of a One-Way Hash without a Salt
  • CWE.759.SALT

CWE SANS On the Cusp Mapping

CWE IDCWE NameParasoft Rule ID(s)
CWE-770Allocation of Resources Without Limits or Throttling
  • CWE.770.LEAKS
CWE-129Improper Validation of Array Index
  • CWE.129.ARRAY
CWE-754Improper Check for Unusual or Exceptional Conditions
  • CWE.252.RETVAL
  • CWE.252.CHECKRET
CWE-805Buffer Access with Incorrect Length ValueN/A
CWE-838Inappropriate Encoding for Output Context
  • CWE.838.AIHUE
CWE-330Use of Insufficiently Random Values
  • CWE.330.USSCR
CWE-822Untrusted Pointer DereferenceN/A
CWE-362Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition')
  • CWE.362.LOCKSETGET
  • CWE.362.DIFCS
CWE-212Improper Cross-boundary Removal of Sensitive Data
  • CWE.212.CSG
CWE-681Incorrect Conversion between Numeric Types
  • CWE.681.ECLTS
  • CWE.197.ECLSII
CWE-476NULL Pointer Dereference
  • CWE.476.NR
  • CWE.476.DEREF,
  • CWE.476.CNFA
CWE-841Improper Enforcement of Behavioral WorkflowN/A
CWE-772Missing Release of Resource after Effective Lifetime
  • CWE.772.LEAKS
CWE-209Information Exposure Through an Error Message
  • CWE.209.SENS
  • CWE.209.PEO
  • CWE.209.ACPST
CWE-825Expired Pointer DereferenceN/A
CWE-456Missing Initialization of a VariableN/A

CWE 3.2 Mapping

CWE IDCWE NameParasoft Rule ID(s)
CWE-22

Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')

  • CWE.22.TDFNAMES
CWE-77Improper Neutralization of Special Elements used in a Command ('Command Injection')
  • CWE.77.TDCMD
CWE-78Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')
  • CWE.78.VPPD
  • CWE.78.TDCMD,
  • CWE.78.AUPS
CWE-79Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
  • CWE.79.VPPD
  • CWE.79.TDRESP
  • CWE.79.TDXSS
CWE-80Improper Neutralization of Script-Related HTML Tags in a Web Page (Basic XSS)
  • CWE.80.VPPD
  • CWE.80.TDRESP
CWE-88Argument Injection or Modification
  • CWE.88.TDCMD
  • CWE.88.VPPD
CWE-89Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')
  • CWE.89.VPPD
  • CWE.89.TDSQL
  • CWE.89.TDSQLC
CWE-90Improper Neutralization of Special Elements used in an LDAP Query ('LDAP Injection')
  • CWE.90.VPPD
  • CWE.90.TDLDAP
CWE-99Improper Control of Resource Identifiers ('Resource Injection')
  • CWE.99.TDFNAMES,
  • CWE.99.TDNET
CWE-120Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')
  • CWE.120.AUK
CWE-129Improper Validation of Array Index
  • CWE.129.ARRAY
CWE-131Incorrect Calculation of Buffer Size
  • CWE.131.AUK
CWE-134Use of Externally-Controlled Format String
  • CWE.134.TDINPUT
CWE-190Integer Overflow or Wraparound
  • CWE.190.AIWIL
  • CWE.190.INTOVERF
CWE-191Integer Underflow (Wrap or Wraparound)
  • CWE.191.AIWIL
  • CWE.191.INTOVERF
CWE-197Numeric Truncation Error
  • CWE.197.ECLSII
CWE-201Information Exposure Through Sent Data
  • CWE.201.SELSPLAT
CWE-209Information Exposure Through an Error Message
  • CWE.209.SENS
  • CWE.209.PEO
  • CWE.209.ACPST
CWE-212Improper Cross-boundary Removal of Sensitive Data
  • CWE.212.CSG
CWE-250Execution with Unnecessary Privileges
  • CWE.250.AUEP
CWE-252Unchecked Return Value
  • CWE.252.RETVAL
  • CWE.252.CHECKRET
CWE-259Use of Hard-coded Password
  • CWE.259.HPW
CWE-285Improper Authorization
  • CWE.285.TDSQL
CWE-306Missing Authentication for Critical Function
  • CWE.306.ADSVSP
CWE-307Improper Restriction of Excessive Authentication Attempts
  • CWE.307.LUAFLA
CWE-316Cleartext Storage of Sensitive Information in Memory
  • CWE.316.RSFSS
  • CWE.316.SSFP
CWE-327Use of a Broken or Risky Cryptographic Algorithm
  • CWE.327.ICA
  • CWE.327.DNCCKS
  • CWE.327.ACCA
CWE-328Reversible One-Way Hash
  • CWE.328.ICA
CWE-329Not Using a Random IV with CBC Mode
  • CWE.329.ACCA
CWE-330Use of Insufficiently Random Values
  • CWE.330.USSCR
CWE-350Reliance on Reverse DNS Resolution for a Security-Critical Action
  • CWE.350.IIPHEU
CWE-352Cross-Site Request Forgery (CSRF)
  • CWE.352.VPPD
  • CWE.352.TDRESP
  • CWE.352.VAFT
CWE-362Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition')
  • CWE.362.LOCKSETGET
  • CWE.362.DIFCS
CWE-369Divide By Zero
  • CWE.369.ZERO
CWE-391Unchecked Error Condition
  • CWE.391.LGE
CWE-395Use of NullPointerException Catch to Detect NULL Pointer Dereference
  • CWE.395.NCNRE
CWE-396Declaration of Catch for Generic Exception
  • CWE.396.NCSAE
CWE-397Declaration of Throws for Generic Exception
  • CWE.397.NTSAE
CWE-401Improper Release of Memory Before Removing Last Reference
  • CWE.401.DBDTFF
  • CWE.401.DCDSF
  • CWE.401.DCID
  • CWE.401.DDFODB
  • CWE.401.SRIF
  • CWE.401.TICUFDS
  • CWE.401.TIID
  • CWE.401.CBDM
  • CWE.401.IDWF
  • CWE.401.MDPP
  • CWE.401.ASC
CWE-402Transmission of Private Resources into a New Sphere ('Resource Leak')
  • CWE.402.CSG
CWE-412Unrestricted Externally Accessible Lock
  • CWE.412.NLT
CWE-416Use After Free
  • CWE.416.DISP
  • CWE.416.FIN
CWE-434Unrestricted Upload of File with Dangerous Type
  • CWE.434.TDFNAMES
CWE-457Use of Uninitialized Variable
  • CWE.457.NOTEXPLINIT
CWE-470Use of Externally-Controlled Input to Select Classes or Code ('Unsafe Reflection')
  • CWE.470.TDRFL
CWE-476NULL Pointer Dereference
  • CWE.476.NR
  • CWE.476.DEREF
  • CWE.476.CNFA
CWE-480Use of Incorrect Operator
  • CWE.480.PUO
CWE-481Assigning instead of Comparing
  • CWE.481.AWC
CWE-494Download of Code Without Integrity Check
  • CWE.494.IREC
CWE-499Serializable Class Containing Sensitive Data
  • CWE.499.CSG
CWE-502Deserialization of Untrusted Data
  • CWE.502.IIDC
  • CWE.502.UIS
  • CWE.502.IDC
  • CWE.502.MGODWSPA
CWE-546Suspicious Comment
  • CWE.546.TODO
CWE-561Dead Code
  • CWE.561.UC
CWE-563Assignment to Variable without Use
  • CWE.563.POVR
  • CWE.563.VOVR
CWE-570Expression is Always False
  • CWE.570.CC
CWE-571Expression is Always True
  • CWE.571.CC
CWE-595Comparison of Object References Instead of Object Contents
  • CWE.595.REVT
CWE-601URL Redirection to Untrusted Site ('Open Redirect')
  • CWE.601.VPPD
  • CWE.601.TDNET
CWE-611Improper Restriction of XML External Entity Reference ('XXE')
  • CWE.611.PDTDP
  • CWE.611.USXRS
CWE-613Insufficient Session Expiration
  • CWE.613.ISE
CWE-662Improper Synchronization
  • CWE.662.DIFCS
CWE-676Use of Potentially Dangerous Function
  • CWE.676.APDM
CWE-681Incorrect Conversion between Numeric Types
  • CWE.681.ECLTS
CWE-732Incorrect Permission Assignment for Critical Resource
  • CWE.732.ADSVSP
CWE-759Use of a One-Way Hash without a Salt
  • CWE.759.SALT
CWE-760Use of a One-Way Hash with a Predictable Salt
  • CWE.760.SALT
CWE-770Allocation of Resources Without Limits or Throttling
  • CWE.770.LEAKS
CWE-772Missing Release of Resource after Effective Lifetime
  • CWE.772.LEAKS
CWE-778Insufficient Logging
  • CWE.778.ENFL
CWE-780Use of RSA Algorithm without OAEP
  • CWE.780.UOWR
CWE-798Use of Hard-coded Credentials
  • CWE.798.HARDCONN
CWE-807Reliance on Untrusted Inputs in a Security Decision
  • CWE.807.AUTH
CWE-827Improper Control of Document Type Definition
  • CWE.827.PDTDP
CWE-829Inclusion of Functionality from Untrusted Control Sphere
  • CWE.829.DMSC
  • CWE.829.ADLL
CWE-833Deadlock
  • CWE.833.ORDER
CWE-838Inappropriate Encoding for Output Context
  • CWE.838.AIHUE
CWE-862Missing Authorization
  • CWE.862.UAA
CWE-863Incorrect Authorization
  • CWE.863.AAM
  • CWE.863.UAAMC

Creating Custom Rules

Use RuleWizard to create custom rules. See the RuleWizard User Guide for information about RuleWizard capabilities and usage.

To use the rule, it needs to be enabled in a test configuration and the custom rule file must be located in one of the following directories:

  • [INSTALL_DIR]\rules\user\
  • [DOCUMENTS DIR]\Parasoft\[tool_name]\rules where [DOCUMENTS DIR] refers to the "My Documents" directory on Windows.

  • No labels