Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • CERT C
  • CERT C++
  • CWE
  • CWE Top 25 Most Dangerous Software Errors
  • CWE on the Cusp
  • OWASP Top 10
  • OWASP API Security Top 10

Each organization has different security requirements and environments. This guide is intended to help you set up and execute the security solution under the following scenario:

...

We recognize that each organization is at a different point in their security compliance initiative. You may be using following this guide in your organization, for instance, while building as you build the dev/test infrastructure, for instance, while others are much further along in their development journey. The Prerequisites section describes assumptions we make in this guide regarding the state of your dev/test infrastructure deployment.

...

You can configure your CI tool to apply quality gates as part of your build process. Parasoft integrates with several CI tools that enable you to visualize test execution and code analysis results from Parasoft tools in your continuous integration system interface. Refer to the Appendix for  for supported CI tools.

Configuring quality gates at the CI layer differs from system to system. The Parasoft Findings for Jenkins extension, for example, supports quality gate configuration from the UI. Refer to the documentation for your CI tool for details.

...

Some documentation includes video and PDF training on specific violation topics. The resources are provided through our partnership with the Software Assurance Marketplace (SWaMP) and are free for our users.

Many checkers also include OWASP-specific training through our partnership with HackEDU. Contextual training for a specific static analysis violation is free and accessible from the documentation. Teams can also take advantage of the full security training and certification program available from HackEDU at an additional cost.

...

Your compliance policy should be a measurable and achievable plan for releasing software that conforms to a security standard. You can also define a compliance goal that strives for a higher degree of guideline conformance. Your compliance goal (like your compliance policy) is encapsulated in a code analysis configuration. The difference is that you expect to comply your compliance policy prior to release, whereas you expect to run your compliance goal configuration with acceptable outcomes at some time in the future. This topic is discussed in greater detail in the Implement Your Compliance Strategy section and Achieving Your Compliance Goal sections.

How Parasoft Supports Security Compliance Initiatives

...

Test configurations specify how Parasoft tools should execute their core features and functionality. They define the checkers that are enabled during code analysis, as well as configurations for checkers that support parameterization. Parasoft tools also ship with test configurations for executing unit tests, which specify how to instrument code for collecting coverage during test execution.

The following step are intended to help you refine your compliance plan and define the methods used to achieve compliance:

...

Choosing a Test Configuration

The Parasoft Security Compliance Pack includes several test configurations that check compliance with the supported standards. When you install the Security Compliance Pack in DTP, the test configurations are automatically deployed to your DTP infrastructure. This enables you to centrally manage and distribute test configurations to your teams. Test configurations also ship with Parasoft tools for local execution, but using the test configurations stored in DTP enables you to uniformly apply checkers across teams (see Executing the Test Configuration).

Use the default test configurations as starting points and adapt them to meet your specific needs. Some checkers, for example, will need to be parameterized based on your code. Some checkers may not be a good fit for your codebase and should be disabled. If you do not need to handle input validation, for example, you may want to disable a checker that reports a violation when input validation isn't handled in a manner that's consistent with the guideline. 

Refer to the Parasoft tool documentation for additional information about the default security compliance test configurations included with the tools:

You can review the test configuration prior to execution and make some initial modifications, but fully understanding how to adjust the test configuration is an iterative process. Refer to the Updating Your Configuration section for information about modifying the test configuration.

...

You can run the test configuration from the command line (e.g., as part of the automated CI process) or directly from within the IDE where the Parasoft tool is installed.
See Integration with Your Build Tool for command line execution.Refer  Refer to the following documentation for details on running within the IDE:

...

Executing the test configuration on your complete project using the default settings may take a significant amount of time. One of the purposes of this step is to determine a baseline so that you can make decisions on how to balance resources to meet your compliance policy. You may determine that splitting checkers across several test configurations tuned for different milestones or development phases will be necessary.

By default, test configurations limit the code analysis results to 1000 violations per checker. If a rule is violating more than 1000 times, it means one of three things:

...

Refer to the DTP documentation for details on how to change this setting.

Viewing Results

There are several ways to view code analysis results. By default, Parasoft tools output results to the <TOOL_INSTALL>/reports directory, but you can customize the directory, as well as format (HTML, PDF, XML) when you deploy your tool. You can open the report or load the results into the desktop instances of your Parasoft tools.

Viewing Reports in the Desktop

Refer to the documentation for details on viewing reports in the desktop:

If you integrated Parasoft with your CI tool, then, in addition to the HTML output, you can review results in one of the supported systems. Refer to the Parasoft Findings documentation for details.

Viewing Results in DTP

See the Monitor and Audit Compliance with Your Policy section for details about the available DTP interfaces for viewing static analysis violations.

How Many Violations is Too Many?

By default, the number of violations reported is limited to 1000 per checker. If a checker hits this limit or some lower threshold that fits your organization, the checker needs to be assessed. You should either configure/parameterize the checker, disable for old code, or disable and re-evaluated in the future.
In addition:

  • A large number of sev1 or sev2 violations likely require immediate attention.
  • A large number of sev3 violations may point to a mismatch in your organizations coding culture and the type of violations reported by some checkers.
  • A low number of violations (e.g., fewer than 200) may still represent a problem, especially if they are sev1 or sev2 violations.

Updating Your Configuration

Browse the results and update your test configuration based on whether or not you are going to fix the reported violations. Refer to the DTP documentation for details on how to update the test configuration.

Disable checkers that report violations that you are never going to fix. Suppress violations that are not acceptable in the reported instance but that you would otherwise fix (see Applying Suppressions). Refer to your compliance policy (see Define Compliance Policy) for guidance on which checkers to disable and which violations to suppress.

Some Parasoft checkers require parameterization, which is also configured in the test configuration. Parameterizable checkers have default values that you may need to change based on your code.

If the default severity associated with reported violations is too high or low, you can apply an updated rule map to change the severity—or even the category—to more accurately reflect how the coding pattern impacts your application. Refer to the Advanced Configuration and Strategies for information on changing checker severities and categories.

Applying Suppressions

Suppressions are one of the most important tools you can use when implementing a compliance initiative. A suppression is a code-level annotation that instructs Parasoft to continue checking for the pattern, but not to report it as a violation in the specific instance.

Instead of disabling a checker, for example, you may decide to suppress violations reported for modules associated with legacy code. In this way, the checker will stop reporting violations for code you are not going to touch but continue to report violations for new development.

Suppressions need to be documented in the final compliance report. See Documenting Deviations for details.

You can apply violations either in the desktop instance of your code analysis tool or in DTP.

Suppressing Violations in DTP

You can mark violations for suppression from DTP. The suppression is implemented on the next code analysis run. Refer to the DTP documentation for details on suppressing violations. When static analysis violations are processed in DTP, desktop users can download violations, including suppressions, into their IDEs.

Suppressing Violations in Code

After a static analysis run, you can load the results into your IDE and suppress them using the GUI. In the phase described in the Monitor and Audit Compliance with Your Policy section, you will download the results from DTP. Processed results are referred to in the IDE as "findings."
Refer to the tool documentation for details:

Determine Your Remediation Strategy

After disabling checkers, applying suppressions, and making other changes to the Parasoft reporting mechanisms to encapsulate your compliance policy, you will likely still have violations that need to be fixed. See Remediation Workflow for details on how to prioritize and remediate violations.

Implement Your Compliance Strategy

At this point, your baseline infrastructure is set up and you should know how to handle exceptions. The code analysis tools should be configured to execute the test configuration during the CI process according to your workflow (e.g., nightly, hourly, triggered on check-in, etc.). The tools should also be configured to report analysis findings to DTP.

Developers working on the desktop, however, have different requirements as they run the test configuration locally on a day-to-day basis.

...

Reducing False Positives

Locally executing the test configuration on a representative sample of your code will help you determine the appropriate contexts for minimizing false positives. View the results after the initial execution and update the configuration based on the following factors (see Updating Your Configuration):

  • Parameterization: Many checkers can be parameterized and may need to be tuned to your codebase. You should disable these checkers if they do not provide value based on your project. 
  • Value to the project: You should consider disabling checkers if they do not provide value based on your project.
  • Age and criticality: Many projects include older code that should not be touched because knowledge about the code is no longer available or because it is extremely sensitive.  Proper controls should be put in place to suppress violations related to this kind of code. Do not run SAST on any code that you either have no intention of fixing or where your policy prevents fixing without specific circumstances. 

Applying AI/ML

Parasoft DTP has a powerful AI component that your team can train to prioritize and surface the most important results, while avoiding issues you’re likely to suppress. If you have already been using a Parasoft code analysis tool, then you can train the AI based on your history. If you are new to the Parasoft security solution, then you can manually train the AI to select some violations for you to process.

Image Added

An AI is only as good as your ability to teach it, so take care to be thoughtful and accurate when classifying violations. Refer to the DTP documentation for details.

Viewing Results

There are several ways to view code analysis results. By default, Parasoft tools output results to the <TOOL_INSTALL>/reports directory, but you can customize the directory, as well as format (HTML, PDF, XML) when you deploy your tool. You can open the report or load the results into the desktop instances of your Parasoft tools.

Viewing Reports in the Desktop

Refer to the documentation for details on viewing reports in the desktop:

If you integrated Parasoft with your CI tool, then, in addition to the HTML output, you can review results in one of the supported systems. Refer to the Parasoft Findings documentation for details.

Viewing Results in DTP

See the Monitor and Audit Compliance with Your Policy section for details about the available DTP interfaces for viewing static analysis violations.

How Many Violations is Too Many?

By default, the number of violations reported is limited to 1000 per checker. If a checker hits this limit or some lower threshold that fits your organization, the checker needs to be assessed. You should either configure/parameterize the checker, disable for old code, or disable and re-evaluated in the future.
In addition:

  • A large number of sev1 or sev2 violations likely require immediate attention.
  • A large number of sev3 violations may point to a mismatch in your organizations coding culture and the type of violations reported by some checkers.
  • A low number of violations (e.g., fewer than 200) may still represent a problem, especially if they are sev1 or sev2 violations.

Updating Your Configuration

Browse the results and update your test configuration based on whether or not you are going to fix the reported violations. Refer to the DTP documentation for details on how to update the test configuration.

Disable checkers that report violations that you are never going to fix. Suppress violations that are not acceptable in the reported instance but that you would otherwise fix (see Applying Suppressions). Refer to your compliance policy (see Define Compliance Policy) for guidance on which checkers to disable and which violations to suppress.

Some Parasoft checkers require parameterization, which is also configured in the test configuration. Parameterizable checkers have default values that you may need to change based on your code.

If the default severity associated with reported violations is too high or low, you can apply an updated rule map to change the severity—or even the category—to more accurately reflect how the coding pattern impacts your application. Refer to the Advanced Configuration and Strategies for information on changing checker severities and categories.

Applying Suppressions

Suppressions are one of the most important tools you can use when implementing a compliance initiative. A suppression is a code-level annotation that instructs Parasoft to continue checking for the pattern, but not to report it as a violation in the specific instance.

Instead of disabling a checker, for example, you may decide to suppress violations reported for modules associated with legacy code. In this way, the checker will stop reporting violations for code you are not going to touch but continue to report violations for new development.

Suppressions need to be documented in the final compliance report. See Documenting Deviations for details.

You can apply violations either in the desktop instance of your code analysis tool or in DTP.

Suppressing Violations in DTP

You can mark violations for suppression from DTP. The suppression is implemented on the next code analysis run. Refer to the DTP documentation for details on suppressing violations. When static analysis violations are processed in DTP, desktop users can download violations, including suppressions, into their IDEs.

Suppressing Violations in Code

After a static analysis run, you can load the results into your IDE and suppress them using the GUI. In the phase described in the Monitor and Audit Compliance with Your Policy section, you will download the results from DTP. Processed results are referred to in the IDE as "findings."
Refer to the tool documentation for details:

Determine Your Remediation Strategy

After disabling checkers, applying suppressions, and making other changes to the Parasoft reporting mechanisms to encapsulate your compliance policy, you will likely still have violations that need to be fixed. See Remediation Workflow for details on how to prioritize and remediate violations.

Implement Your Compliance Strategy

At this point, your baseline infrastructure is set up and you should know how to handle exceptions. The code analysis tools should be configured to execute the test configuration during the CI process according to your workflow (e.g., nightly, hourly, triggered on check-in, etc.). The tools should also be configured to report analysis findings to DTP.

Developers working on the desktop, however, have different requirements as they run the test configuration locally on a day-to-day basis.

Regularly running code analysis on the desktop prevents defects from being injected into the code. Checked-out code, however, may refer to functions that are not within the local scope, resulting in additional static analysis violation noise. Additionally, some checkers perform data flow analysis, which simulates application execution to analyze possible paths. These checkers may report additional violations and take longer to execute. Analyzing legacy code, furthermore, can also produce noisy results at a time when developers are trying to remain productive.

...

Advanced Compliance Monitoring Tools

The out-of-the-In addition, Parasoft DTP ships with portfolio-level widgets, such as the Portfolio - Violations Statistics widget, for displaying SAST results across projects in your portfolio.

Advanced Compliance Monitoring Tools

The out-of-the-box configurations are intended to cover as many workflows as possible, but you can leverage additional functionality to further adapt the DTP solution to your needs.

...

Running What-if Scenarios

As you work toward your long-term compliance goal, you may want to incrementally enable additional checkers in your test configuration. Duplicate your primary test configuration and add one or more checkers. Before executing analysis, however, add a filter to your project and configure your tool to send the what-if execution results to the new filter. In this way, you are can keep exploratory analysis results separate from your official compliance data. Refer to the DTP documentation to learn more about filters.

Appendix

The following sections describe the supported components for enabling Parasoft's Security Compliance solution.

Supported Languages

Parasoft can analyze the following languages out of the box:

  • Java
  • .NET-based languages
  • C
  • C++
  • C#
  • VB.NET
  • HTML
  • CSS
  • XML

Parasoft supports several other programming languages via the Parasoft Multi-Language Pack:

  • Android-based applications 
  • Apex
  • Go
  • Groovy
  • JavaScript
  • Kotlin
  • Objective-C
  • PHP
  • Python
  • Ruby
  • Scala
  • Swift
  • Typescript

The Parasoft Multi-Language Pack is available in the marketplace on the Parasoft customer portal.What-if scenarios are mechanisms for understanding the outcomes resulting from a given input. In a SAST context, the results of your current build serve as the input and the states of compliance against your short- and long-term goals are the outcomes. What-if scenarios can help you understand the impact, for example, of enabling a new checker or changing the severity of an active checker.

You can run what-if scenarios without executing additional analysis by creating profiles in DTP Extension Designer, which filter results for the current build in order to preview potential changes to your test configuration. You can also run what-if scenarios by incrementally enabling additional checkers in your test configuration and sending the results to different filters.

Using Profiles to Create What-if Scenarios

The Security Compliance Pack includes a default profile for each set of guidelines. Profiles indicate which checkers were expected in the analysis, which tool ran the checkers, and other details, that are included in dashboard widgets and reports. You can create additional profiles and modify the list of checkers to change the scope of the expected results—add additional checkers to the profile, for example, to understand how the current build would perform if analysis ran with the additional checkers.

Do not modify the default profile. Instead, export a copy of the profile and import it into the model, which is an entity that defines the template for the data contained in the profile. The model defines the expected fields in the profile. The profile defines the expected values in the analysis.

Use the following process to create profile-based what-if scenarios that you can apply to your existing results:

  1. Export a copy of the default profile for you compliance configuration. Profiles are exported as XLSX files, which you can modify before importing or import as-is and modify the profile in the Extension Designer UI.
  2. Import the profile and enable/disable checkers.
  3. 3. In the DTP dashboard, add a compliance widget for each profile and specify the compliance profile the encapsulates each what-if scenario. You can configure a widget to track your long-term goal against the profile with the all checkers you want to eventually use, for example, and configure another widget to track your short-term goal against the profile with a subset of checkers enabled.

You can add a Categories in Compliance widget to your dashboard and click into the report for list of checkers enabled in the compliance profile and number of violations for each checker.

Using Filters to Create What-if Scenarios

You can incrementally enable additional checkers in your test configuration and send the results into different DTP filters. A filter is a mechanism in DTP for reporting results based on run configurations, which are sets of metadata, such as the machine name or IP that ran the execution, environment, build ID, and test configuration. Refer to the DTP documentation to learn more about filters.

Use the following process to create filter-based what-if scenarios to apply to builds incrementally:

  1. Create a duplicate of your primary test configuration.
  2. Enable (or disable) one or more checkers in the duplicate configuration.
  3. Add a filter to your project in DTP and configure your tool to send the what-if execution results to the new filter. In this way, you are can keep exploratory analysis results separate from your official compliance data.

Appendix

The following sections describe the supported components for enabling Parasoft's Security Compliance solution.

Supported Languages

Parasoft can analyze the following languages out of the box:

  • Java
  • .NET-based languages
  • C
  • C++
  • C#
  • VB.NET
  • HTML
  • CSS
  • XML

Parasoft supports several other programming languages via the Parasoft Multi-Language Pack:

  • Android-based applications 
  • Apex
  • Go
  • Groovy
  • JavaScript
  • Kotlin
  • Objective-C
  • PHP
  • Python
  • Ruby
  • Scala
  • Swift
  • Typescript

The Parasoft Multi-Language Pack is available in the marketplace on the Parasoft customer portal.

DTP and Enterprise Pack 2021.1

Standards

  • CERT C
  • CERT C++
  • CWE List Version 4.4
  • CWE Top 25
  • CWE Top 25 + On the Cusp
  • UL 2900
  • OWASP Top 10
  • OWASP API Security Top 10 2019
  • PCI DSS 3.2

Parasoft tools

  • C/C++test 2021.1 (all editions)
  • dotTEST 2021.1
  • Jtest 2021.1

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps 

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

DTP and Enterprise Pack 2020.2

Standards

  • CERT C
  • CERT C++
  • CWE List Version 4.0
  • CWE Top 25
  • CWE Top 25 + On the Cusp
  • UL 2900
  • OWASP Top 10
  • PCI DSS 3.2

Parasoft tools

  • C/C++test 2020.2 (all editions)
  • dotTEST 2020.2
  • Jtest 2020.2

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps 

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

DTP and Enterprise Pack 2020.1

Standards

  • CERT C
  • CERT C++
  • CWE List Version 4.0
  • CWE Top 25
  • CWE Top 25 + On the Cusp
  • UL 2900
  • OWASP Top 10
  • PCI DSS 3.2

Parasoft tools

  • C/C++test 2020.1 (all editions)
  • dotTEST 2020.1
  • Jtest 2020.1

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps 

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

...

Standards

  • CERT C
  • CERT C++
  • CWE Top 25
  • CWE List Version 2.11
  • CWE List Version 3.1
  • CWE List Version 3.2
  • CWE List Version 3.4
  • UL 2900
  • OWASP Top 10
  • PCI DSS 3.2

Parasoft tools

  • C/C++test 10.4.3 (all editions)
  • dotTEST 10.4.3
  • Jtest 10.4.3

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps 

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

...

Standards

  • CERT C
  • CERT C++
  • CWE Top 25
  • CWE List Version 2.11
  • CWE List Version 3.1
  • CWE List Version 3.2
  • OWASP Top 10

Parasoft tools

  • C/C++test 10.4.2 (all editions)
  • dotTEST 10.4.2
  • Jtest 10.4.2

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps (formerly  Visual Studio Team Services)

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

...

Standards

  • CERT C
  • CERT C++
  • CWE Top 25 
  • CWE List Version 2.11
  • CWE List Version 3.1
  • OWASP Top 10

Parasoft tools

  • C/C++test 10.4.1 (all editions)
  • dotTEST 10.4.1
  • Jtest 10.4.1

Continuous integration

Parasoft can report static analysis violations in the following CI systems:

  • Bamboo 5.14 +
  • Jenkins 1.625.1 +
  • TeamCity 2017.1.2 +
  • Microsoft Azure DevOps (formerly  Visual Studio Team Services)

Refer to the Parasoft Findings documentation for details.

You can integrate with other CI systems using the command line interface.

Source control

Refer to the tool documentation for supported SCMs:

...