...
The mapping between Parasoft checkers and coding guidelines may have an impact on how you define compliance in your organization. A checker may report violations for guidelines that are critical to your application, as well as guidelines that are less relevant. This is why the ability to suppress violations (deviate from the standard) is important. Deviations are discussed in greater detail in the Documenting Deviations section.|#_Documenting_Deviations
Parasoft DTP collects the data and outputs guideline-specific visualizations and reports for monitoring and demonstrating compliance. This is discussed in greater detail in the Monitoring and Demonstrating Compliance section.|#_Monitoring_and_Demonstrating
...
Severities
...
Parasoft checkers are identified according to the standard that they check against. For example, a checker that applies the CWE guideline "CWE-22: Improper Limitation of a Pathname to a Restricted Directory" has the ID "CWE.22.TDFNAMES."
Checkers are also categorized by severity. Parasoft tools rank the violations they report from severity 1 ("sev1") to severity 5 ("sev5") with sev1 violations being the most important. The expectation is that sev1, sev2, and sev3 checkers report defects that require remediation, whereas sev5 checkers are used as inspections checkers (sev4 violations are not used in the Parasoft security compliance workflow). Inspection checkers flag code that should be manually reviewed to ensure that the code is compliant with a guideline. See Inspection Rules for additional information.
Critical | Severity 1 |
High | Severity 2 |
Medium | Severity 3 |
Warning | Severity 4 |
Informational | Severity 5 |
...
...
Choose
...
and Adapt a Test Configuration
Test configurations specify how Parasoft tools should execute their core features and functionality. They define the checkers that are enabled during code analysis, as well as configurations for checkers that support parameterization. Parasoft tools also ship with test configurations for executing unit tests, which specify how to instrument code for collecting coverage during test execution.
The following step are intended to help you refine your compliance plan and define the methods used to achieve compliance:
...
The Parasoft Security Compliance Pack includes several test configurations that check compliance with the supported standards. When you install the Security Compliance Pack in DTP, the test configurations are automatically deployed to your DTP infrastructure. This enables you to centrally manage and distribute test configurations to your teams. Test configurations also ship with Parasoft tools for local execution, but using the test configurations stored in DTP enables you to uniformly apply checkers across teams (see Executing the Test Configuration).|#_Execute_and_Refine
Use the default test configurations as starting points and adapt them to meet your specific needs. Some checkers, for example, will need to be parameterized based on your code. Some checkers may not be a good fit for your codebase and should be disabled. If you do not need to handle input validation, for example, you may want to disable a checker that reports a violation when input validation isn't handled in a manner that's consistent with the guideline.
You can review the test configuration prior to execution and make some initial modifications, but fully understanding how to adjust the test configuration is an iterative process. Refer to the Updating Your Configuration section for information about modifying the test configuration.
...
...
Executing the Test Configuration
You can run the test configuration from the command line (e.g., as part of the automated CI process) or directly from within the IDE where the Parasoft tool is installed.
See Integration with Your Build Tool for command line execution.
Refer to the following documentation for details on running within the IDE:
...
To understand the total scope of work, you should disable the limit during the baselining phase and re-enable it after modifying your test configuration. Disabling the 1000-violation limit may result in significantly longer execution time.
Refer to the DTP documentation for details on how to change this setting.
Viewing Results
There are several ways to view code analysis results. By default, Parasoft tools output results to the <TOOL_INSTALL>/reports directory, but you can customize the directory, as well as format (HTML, PDF, XML) when you deploy your tool. You can open the report or load the results into the desktop instances of your Parasoft tools.
...
- A large number of sev1 or sev2 violations likely require immediate attention.
- A large number of sev3 violations may point to a mismatch in your organizations coding culture and the type of violations reported by some checkers.
- A low number of violations (e.g., fewer than 200) may still represent a problem, especially if they are sev1 or sev2 violations.|#_Executing_the_Test
...
Updating Your Configuration
Browse the results and update your test configuration based on whether or not you are going to fix the reported violations. Refer to the DTP documentation for details on how to update the test configuration.
Disable checkers that report violations that you are never going to fix. Suppress violations that are not acceptable in the reported instance but that you would otherwise fix (see Applying Suppressions). Refer to your compliance policy (see Define Compliance Policy) for guidance on which checkers to disable and which violations to suppress. |#_Applying_Suppressions|#_Define_Compliance_Policy
Some Parasoft checkers require
Some Parasoft checkers require parameterization, which is also configured in the test configuration. Parameterizable checkers have default values that you may need to change based on your code.
If the default severity associated with reported violations is too high or low, you can apply an updated rule map to change the severity—or even the category—to more accurately reflect how the coding pattern impacts your application. Refer to the Advanced Configuration and Strategies for information on changing checker severities and categories.Anchor
...
Applying
...
Suppressions
...
Suppressions are one of the most important tools you can use when implementing a compliance initiative. A suppression is a code-level annotation that instructs Parasoft to continue checking for the pattern, but not to report it as a violation in the specific instance.
Instead of disabling a checker, for example, you may decide to suppress violations reported for modules associated with legacy code. In this way, the checker will stop reporting violations for code you are not going to touch but continue to report violations for new development.
Suppressions need to be documented in the final compliance report. See Documenting Deviations for details.
You can apply violations either in the desktop instance of your code analysis tool or in DTP.
Suppressing Violations in DTP
You can mark violations for suppression from DTP. The suppression is implemented on the next code analysis run. Refer to the DTP documentation for details on suppressing violations.
When static analysis violations are processed in DTP, desktop users can download violations, including suppressions, into their IDEs.
Suppressing Violations in Code
After a static analysis run, you can load the results into your IDE and suppress them using the GUI. In the phase described in the Monitoring and Demonstrating Compliance section, you will download the results from DTP. Processed results are referred to in the IDE as "findings."
Refer to the tool documentation for details:
...
After disabling checkers, applying suppressions, and making other changes to the Parasoft reporting mechanisms to encapsulate your compliance policy, you will likely still have violations that need to be fixed. See Remediation Workflow for details on how to prioritize and remediate violations.
...
Implement Your Compliance Strategy
At this point, your baseline infrastructure is set up and you should know how to handle exceptions. The code analysis tools should be configured to execute the test configuration during the CI process according to your workflow (e.g., nightly, hourly, triggered on check-in, etc.). The tools should also be configured to report analysis findings to DTP.
Developers working on the desktop, however, have different requirements as they run the test configuration locally on a day-to-day basis.
Regularly running code analysis on the desktop prevents defects from being injected into the code. Checked-out code, however, may refer to functions that are not within the local scope, resulting in additional static analysis violation noise. Additionally, some checkers perform data flow analysis, which simulates application execution to analyze possible paths. These checkers may report additional violations and take longer to execute. Analyzing legacy code, furthermore, can also produce noisy results at a time when developers are trying to remain productive.
Configure your desktop tools to execute a version of the baseline test configuration with the following adjustments:
...
Clean local code does not necessarily mean that the code will not trigger violations on the server. The automated build executes all checkers in the primary test configuration in the context of the scope of the entire code base. As a result, code that ran clean locally may result in violations when integrated with the rest of the build.
...
Achieving Your Compliance Goal
If you cannot achieve your compliance goal in the current phase of development, then you should split your code analysis checkers across multiple configurations for different phases of the development process.
For example, remediate sev1 violations in the first phase, sev2 in the next, and sev3 in the final phase. This enable your teams remain productive and agile by limiting the potential violations that must be remediated in each build to a manageable amount.
...
The Security Compliance Pack includes several dashboards optimized for viewing high-level information about the code analysis results in DTP. The dashboards include, for example, widgets that group violations according to risk and widgets that show changes in violations over time. Refer to the Security Compliance Pack documentation for details about the widgets and reports included with the compliance pack. Additional details are covered in Monitoring and Demonstrating Compliance. |#_Monitoring_and_Demonstrating
...
Remediation Workflow
The Violations Explorer view in DTP is an interface for reviewing and prioritizing findings, as well as applying remediation workflows. You can flag violations for suppression in subsequent runs (see Documenting Deviations), assign them to a team member, place them into external workflows, etc. Refer to the Violations Explorer view documentation for usage.|#_Documenting_Deviations
Some widgets in the compliance-specific dashboards shipped with the Security Bundle link directly the Violations Explorer, but many widgets link to intermediate reports showing details associated with a specific aspect of the data. Refer to Monitoring and Demonstrating Compliance for details about the dashboards, widgets, and reports.
...
By default, priority is set to "Not defined," which can be changed in the Violations Explorer view. DTP also includes a REST API that enables you to programmatically control the metadata. You can also create an extension with the Extension Designer to automate prioritization when DTP collects the code analysis data. Refer to the API documentation (https://<YOUR_DTP_HOST>:<PORT>/grs/api) or the Extension Designer documentation for details.
There are several rationales for setting metadata:
...
In many applications, certain parts of the codebase have a higher business value or represent greater risk than other parts. You can segment the code into resource groups and assign higher priorities to the findings associated with more valuable resources. Refer to the resource groups documentation for details.
...
for details.
Creating Tickets in External Systems (Optional)
Parasoft ships with out-of-the-box integration points with external ticketing systems (e.g. Jira and VersionOne), enabling you to create issues in the systems based on violations reported to DTP. Refer to the documentation for configuration and usage information.
...
You can also automate ticket creation either by using the REST API or by creating a custom workflow with Extension Designer. If you choose to fully automate ticket creation, make sure that you've fully tested your scripts, including error conditions, to avoid accidentally create thousands of tickets. If you choose to leverage this functionality, creating tickets manually is the recommended workflow.
...
Documenting
...
Deviations
...
When you need to deviate from the security standard, you can apply a suppression in DTP or in the code from the desktop instance of your analysis tool. Suppressing a violation means that the defect pattern is acceptable in this specific instance and that you are going to ignore the violations reported by the tool for the foreseeable future.
A suppression applies to an instance of the violation, not to the checker, itself. The checker will continue reporting violations when the questionable pattern is detected in other parts of the code. If the checker truly does not apply to your application in any instance, disable it in the test configuration. Refer to the DTP documentation for instructions.
Suppressions are also important in terms of the review process associated with inspection rules. Violations reported by inspection rules are designed to trigger a manual review of the code. When the code is verified per the standard, the inspection rule can be suppressed with a rationale that serves as the documentation for the deviation. See Inspection Rules for additional information.
Documenting deviations is critical to any compliance initiative. When you choose to ignore certain violations that are in your stated compliance policy, you must provide justification. You can document deviations in DTP or in the IDE. Additionally, the Security Compliance artifact includes a deviations report that you can review as part of your development process. The report is available in HTML or PDF format.
...
Of course, no developer would suppress a violation simply because he or she didn't want to fix it, but you should implement a workflow for reviewing and approving suppressions, anyway, as part of your compliance policy. The most common way to review and approve suppressions is to leverage your existing code review process. Many organizations use third-party tools or native SCM functionality.
The Security Compliance Pack includes a Deviations Report that you can review in DTP. You should review the report as part of your code review process. You can export the report to PDF and check it into your SCM for auditing purposes. Refer to the documentation for the specific standard you intend to comply with for details:
...
You can set the action to Suppress and assign the violation to a reviewer who will agree and suppress the violation or make the case that the violation should not be suppressed in the comments. In this way, you create an audit trail in DTP around your suppressions. Refer to the Violation Explorer documentation for details.
High Deviation Counts
Coding standards are not perfect, one-size-fits-all sets of rules—which is why deviations are permissible in the first place. A high number of deviations, however, suggests that additional action should be taken.
The checker may simply need to be turned off in the test configuration (see Choose and Adapt a Test Configuration). The severity of the reported violations may need to be lowered (see Updating Rule Maps). The assignee or author may require additional training. The checker may be serving an "inspection rule" role (see Inspection Rules).
It is up to you to determine what is acceptable, but these aspects of your development process should be considered.
...
...
Inspection
...
Rules
...
Checkers that report severity 5 (sev5) violations serve as inspection rules, which flag potential problems that require human oversight (also see Severities). When sev5 violations are reported in DTP, the action field should be set to Review and assigned to a team member for processing. |#_Severities
Inspection rules are noisy by nature. This is because they are intended to serve as a method calling attention to areas of the code that require manual peer review. If your security compliance strategy is to first prioritize compliance coverage, then you should disable sev5 checkers and focus on sev1, sev2, and sev3 violations. Inspection rules will eventually need to be re-enabled, however, because you will not be able to claim full compliance with guidelines that require manual inspection.
Importing Findings into the IDE
...
Developers should treat the remediation of findings assigned to them as a regular part of their day-to-day workflow. They should also strive to check in code that does not trigger new violations so that each iteration brings the project closer to full compliance with the policy.
...
...
Monitor and Audit Compliance with Your Policy
Each standard supported by the Security Compliance Pack includes several components for monitoring and demonstrating compliance. Only violations reported by checkers in the compliance test configuration will appear in the compliance pack dashboards and reports. Refer to the documentation for each compliance artifact for details about adding compliance dashboards, viewing widgets, and viewing reports:
...
The dashboard templates shipped with the compliance pack include compliance specific widgets that show the compliance status for your project from several perspectives. The dashboards also leverage standard DTP widgets and reconfigure instances of the widgets to show data in context of the specific standard.
SCREENSHOT
Compliance Reports
Each compliance artifact includes a primary report that you can export to PDF and use to demonstrate compliance during an audit, as well as several sub-reports that you can review to monitor compliance to your policy.
SCREENSHOT
Compliance Percentage Widgets
The compliance percentage widgets provide at-a-glance status of the project and provide the main link to the compliance report.
SCREENSHOT
Managing a Portfolio
...
DTP allows you to subset findings associated with a project by creating filters. For each project defined in DTP, a default filter with the same name of the project is created. Filters can have different run configurations, teams, and resource groups.
A resource group is a collection of resources (i.e., files and/or folders) defined by a set of one or more Ant file patterns. Resource groups enable more granular views of the data associated with a filter. After associating a resource group with a filter, the data presented in all widgets, reports, and explorer views for static analysis violations and metrics results will be narrowed according to the defined pattern.
Refer to the DTP documentation for details.
...
Advanced Configuration and Strategies
Implementing a security compliance initiative is not a one-size-fits-all process. The following adjustments may be required based on your specific needs. This section describes some of the advanced configurations and strategies that you can implement to help you improve your compliance processes.
...
You can combine quality and security checkers by enabling the quality checkers in your security compliance test configuration.
Be aware the adding quality checkers to your security test configuration may have unintended consequences. If your project is subject to safety-critical regulations, for example, and the quality checkers report violations that you had not planned on fixing, you may still be obligated to remediate those violations.
Be aware that violations associated with quality checkers will be included in the report generated by the tool. Security compliance reports, however, only contain output from the checkers associated with security guidelines.
Custom Checkers
If your code includes logic that is not covered by the checkers shipped with Parasoft tools, you can create custom checkers with Parasoft RuleWizard. No coding or knowledge of the parser is required to write or modify a rule, but expertise in understanding parse trees is advantageous.
You will need to add the <ruleId>.rule file to your tool's rules directory, enable the checker in the test configuration, and define it in the rule map. Refer to the DTP documentation for details.
Changing Severities
If the violation severity reported by a checker does not match your policy, you can modify the rule map in DTP to assign a new severity level, as well as change the checker ID or category. Refer to the DTP documentation for details.
Customizing the Compliance Pack
If you need to add or remove checkers that support a security guideline, you also need to modify the compliance profile. A profile is a set of configurations that perform custom calculations in DTP, such as calculating and reporting compliance statuses in DTP widgets.
A set of security compliance-related profiles are automatically deployed to your DTP environment when you install the Security Compliance Pack. The profiles serve as inputs for the compliance reports that you generate as part of a compliance audit. If you need to make adjustments, we recommend preserving the default profiles shipped with the compliance pack.
You can upload, export, and modify profiles using the DTP Enterprise Pack module. Refer to the Enterprise Pack documentation for general information about profiles.
Running What-if Scenarios
As you work toward your long-term compliance goal, you may want to incrementally enable additional checkers in your test configuration. Duplicate your primary test configuration and add one or more checkers. Before executing analysis, however, add a filter to your project and configure your tool to send the what-if execution results to the new filter. In this way, you are can keep exploratory analysis results separate from your official compliance data. Refer to the DTP documentation to learn more about filters.
...
Appendix
The following sections describe the supported components for enabling Parasoft's Security Compliance solution.
...
Contact your Parasoft representative for Parasoft Multi-Language Pack download and license information.and license information.
DTP and Enterprise Pack 5.4.3
DTP and Enterprise Pack 5.4.2
Standards | CERT C |
|---|---|
Parasoft tools | C/C++test 10.4.2 |
Continuous integration | Parasoft can report static analysis violations in the following CI systems:
|
Source control | Refer to the tool documentation for supported SCMs: |
...
DTP and Enterprise Pack 5.4.1
Standards | CERT C |
|---|---|
Parasoft tools | C/C++test 10.4.1 |
Continuous integration | Parasoft can report static analysis violations in the following CI systems:
|
Source control | Refer to the tool documentation for supported SCMs: |
...
...
Glossary
...
of
...
Terms
...
The following terms are used throughout this guide.
Checker
Also sometimes called 'rules', checkers are pieces of software that check the correctness of code against a specific programming pattern.Anchor
Compliance Goal
A compliance goal is a long-term plan for conforming to a software development standard. In contrast to a Compliance Policy, a compliance goal is not generally achievable within the current software release. Compliance goals are encapsulated in a code analysis configuration that, when executed by a Parasoft tool, reports violations that you plan to remediate in a future release.
...
Compliance Policy
A compliance policy is a measurable and achievable plan for releasing software that conforms to a software development standard. Compliance policies are encapsulated in a code analysis configuration that, when executed by a Parasoft tool, should report no violations.
...
Compliance
The degree to which the code meets the stated goals of a policy or guideline within a standard. When no static analysis violations are reported, the code is in compliance, regardless of how many checkers are enabled. It is possible for the code to be 100% in compliance with a policy, but fail to reach even partial conformance with a standard if, for example, the policy specifies that only one guideline from the standard should be followed.
...
Conformance
Conformance is a measure of how completely the code is in compliance with the guidelines within a standard. To be 100% conformant, the code must be in compliance with all guidelines specified in the standard.
Internationally, the underlying concepts associated with "conformance" and "nonconformance" are referred to as "conformity" and "non-conformity." Adhering to the procedures you've defined is conformity or conformance, whereas not adhering to them is nonconformity or nonconformance. Procedures that meet federal (external) requirements are in compliance, whereas procedures that do not meet requirements are in noncompliance.Anchor
Deviation
See SuppressionAnchor
Findings
See Violations.
...
Guideline
A guideline is a specific rule or principle within a standard that provides guidance on a single programming topic. For example, the OWASP Top 10 standard includes the following guideline:
"A1:2017-Injection: Injection flaws, such as SQL, NoSQL, OS, and LDAP injection, occur when untrusted data is sent to an interpreter as part of a command or query. The attacker's hostile data can trick the interpreter into executing unintended commands or accessing data without proper authorization."
Depending on the standard, guidelines may include example code, risk impact, and other information that help you determine if specific guidelines should be followed for your project.
...
A suppression is a code-level annotation that instructs Parasoft to continue checking for the pattern, but not to report it as a violation in the specific instance. When a violation is marked with a suppression in a compliance context, it is referred to as a deviation from the standard and must be documented with a justification for allowing the violation to remain.Anchor
Violations
Parasoft refers to the raw programming errors reported by code checkers as "violations" of the rules codified by the checkers. Parasoft attaches additional information about the violations, such as code authorship, build information, etc., as part of the reporting process. The processes violations are referred to as "findings." Findings can refer to static analysis violations that appear in the code analysis report or violations that have been processed by DTP and downloaded to the developers' IDEs. Subtle differences exist, but violations and findings (as well as other terminology used throughout the software testing industry, including "errors" and "defects") are often used interchangeably.