This topic explains how to perform penetration testing with Parasoft SOAtest.

Overview

Penetration testing is critical to uncovering security holes in your application. With Parasoft SOAtest, you can take your existing API functional testing scenarios and create security tests, adding penetration testing into your automated CI process.

Creating Penetration Testing Scenarios

SOAtest supports penetration testing of REST and SOAP APIs that are accessible over HTTP or HTTPS.

Penetration testing is supported by starting with a functional test scenario that contains the APIs that need penetration testing and then configuring those scenarios for penetration testing. Existing functional test scenarios can be leveraged and turned into penetration tests, or you can create custom simplified test scenarios meant only for penetration testing.

Repurposing functional tests for use as security tests gives the following benefits:

  1. Since functional tests have already been written, you can reuse work that has already been done, saving time and effort.
  2. To execute certain APIs, you might have to do some setup, like prepping the database or calling other APIs. If you start with functional tests that already work, this setup is already done.

The general workflow for configuring penetration test scenarios is:  

  1. Identify the test scenarios that you want to use for penetration testing and copy them. You can continue executing the original test scenarios for functional testing as normal.
  2. For test clients (for example, SOAP ClientREST ClientEDI Client, or Messaging Client) add the Penetration Testing Tools to the Traffic Object outputs that make the API calls that need penetration testing.

  3. For Browser Playback Tools, add the Penetration Testing Tools to the HTTP Traffic or Browser Contents outputs that need penetration testing. When the tool is chained to an HTTP Traffic output, the tool attacks just the request described by that traffic message. When the tool is chained to Browser Contents, it attacks all requests made by the Browser Playback tool. By default, binary files are ignored unless enabled in Parasoft > Preferences > Browser Settings (see Additional Preference Settings).

  4. As the application changes, update only the functional test scenarios. Whenever you are ready to run the corresponding penetration test scenarios, repeat the above process of copying from the latest set of functional tests and then configuring the copy for penetration testing.

Penetration testing requires a user license with the API Security Testing license feature enabled. Penetration testing tools can be added in CTP as well. See Penetration Testing Tools.

Best Practices for Creating Security Test Scenarios

Inherent risks of penetration testing

By its nature, penetration testing simulates malicious user activity to find security vulnerabilities in the application under test (AUT). Because of this, penetration testing may cause the following effects on the AUT or the system on which the AUT or SOAtest is running:

  • The AUT can be brought down.
  • The AUT data can be modified or corrupted.
  • If an antivirus application is installed on the machines where the AUT or SOAtest are running, it can interpret SOAtest penetration testing activity as suspicious or malicious and generate warnings or take other actions.

Following the best practices will help you mitigate these and other potential issues that may be caused by the execution of penetration tests.

  • You should maintain your functional test scenarios separately from your security test scenarios and run them from separate test jobs. The main reason for this is that adding penetration testing to existing functional tests will likely serve to destabilize the functional tests. You need to select which functional test scenarios should be turned into automated security tests, and then make copies of the functional tests that will be maintained as separate security tests.
  • You need to be selective in which tests you choose since penetration testing is expensive; you need to maximize the attack surface of the API that is covered while minimizing the number of tests. You should consider the following:
    • The Penetration Testing Tool analyzes request/response traffic to understand which parameters in the request are available to be tested. You need to select functional tests that exercise all the parameters in each API, to ensure that every input to the API gets analyzed.
    • Within each scenario, you need to decide which API calls should be penetration tested. The same API may be referenced from multiple scenarios, and you don’t want to duplicate penetration testing on an API that is tested in a different scenario. The Penetration Testing Tool should only get added to the appropriate tests for the API(s) to be penetration tested.
    • The number of scenarios needs to be manageable so that the security test run is short enough to run at least once a day.
  • Your functional test scenarios may have set up or teardown sections for initialization or cleanup. These typically don’t need to be penetration tested.
  • If the functional test has any parameterization, you should remove it. The Penetration Testing Tool doesn't need to see multiple sets of values for the same parameters to know what to test, and sending different sets of values could just lead to making the test runs go longer due to duplicated testing.
  • You should remove all assertions from any functional test scenarios that were converted into penetration test scenarios. API functional tests will usually have assertions that validate the response from the service. When used as security tests, these assertions can fail but will be noisy when reviewing the results, since in this context you only care about the security vulnerabilities that were found.
  • Some API calls add data to the database. When using the Penetration Testing Tool against such APIs, the database can get bloated with information due to the number of attacks that the penetration testing tool directs at the API. In some cases, this can cause unexpected side effects, such as the performance of the application becoming so bad that the automated security test run cannot finish in a reasonable amount of time.  If you see behavior like this, exclude the security tests for that API from your automated run until the development team can fix the problem.
  • You need to consider whether to run your functional and security tests within the same test environment or a different one. Resetting the environment between the functional and security test runs, or using a separate environment, promotes better test stability but is usually not necessary. You can often reuse the same environment, but when you do, you should run the functional tests first and the security tests last, since the security tests can destabilize the environment for the functional tests. When you use different environments, you need to make sure that the original functional test scenarios are configured with variables so that it is easy to point the tests at different endpoints for different environments. SOAtest supports this using environment variables. (See Configuring Testing in Different Environments.)

Executing Penetration Testing Scenarios

To run your penetration testing scenarios, execute them as described in Executing Functional Tests.

  • When a tool with an attached Penetration Testing Tool is executed, the corresponding request and response data is captured and used as a starting point by the Penetration Testing Tool to execute the penetration test.

Because penetration testing can take a long time to run, even for a single API, the Penetration Testing Tool has a built-in timeout that governs how long the tool will run before moving on to the next test. The timeout complies with the timeout configured in the test configuration used to run the test. If the timeout is reached before the penetration testing for the current API is complete, the penetration testing for that API will be interrupted and an error will be reported.

Additionally, you can set global includes and excludes, added as regular expressions, to control what URLs get scanned by the Penetration Testing Tool. Includes are processed before excludes. When no includes are defined, everything is assumed to be included before considering excludes. Includes and excludes get applied regardless of the tool to which the Penetration Testing Tool is chained.

To change the timeout and/or includes and excludes, go to Parasoft > Test Configurations, then select the test configuration and go to Execution > Security.

 

  • The timeout is measured in minutes.
  • To add a regular expression used to determine included or excluded URLs, click Add beside its table and input the regular expression.
  • To remove a previously added URL regular expression, select it and click Remove.

When your penetration test scenarios run, information about what is being scanned is shown in the Console view. It can be valuable to enable high verbosity in the console, which will provide more detail such as what URLs are being scanned and which are being skipped, allowing you to ensure that your include and exclude patterns are performing as expected. To enable high verbosity, go to Parasoft > Preferences > Console and select High.

Reviewing Results

When running via UI, errors are reported to the Quality Tasks view, and details about the error and how to fix it can be seen by double-clicking on each error or right-clicking and choosing View Details.

More detailed results are available by generating an HTML version of the report. The HTML report will organize the errors by CWE or OWASP 2021 Top 10 (as set in Parasoft > Preferences > Reports > API Security; note that it is not necessary to re-execute your tests after changing this setting), Risk, and Confidence.

For more information see Reviewing Results.

Suppressing False Positives

A number of rules used by the Penetration Testing Tool apply heuristics to analyze the AUT HTTP responses and raise alerts. For this reason, some of the reported errors may be false positive. Review each new error to decide whether it is an actionable item or a false positive (you may want to use the Confidence level of the alert to help your assessment).

To prevent the Penetration Testing Tool from reporting a false positive during subsequent runs, you can suppress the error in the in the Quality Tasks view or in the suppression file. See Suppressing Tasks for details.

To suppress an active scan rule for all tests, modify the default active scan policy. See Configuring Scan Policies.

Configuring Scan Policies

SOAtest's penetration testing uses active and passive scan rules to do its analysis. Active scan rules make additional (manipulated) requests to the API to attempt to discover security vulnerabilities. In contrast, passive scan rules make no new requests to the application but instead analyze request/response data captured by the corresponding tool to discover security vulnerabilities. SOAtest leverages OWASP ZAP for penetration testing. 

SOAtest contains three built-in penetration test profiles: one for REST APIs, one for SOAP APIs, and one for non-API requests for web resources. Each profile consists of an active scan policy and a set of passive scan rules. By default, a profile is chosen based on whether the configured test makes a request to a REST API, a SOAP API, or some other web resource (for example, HTML).

If you want to use a custom active scan policy, you can do so by exporting a scan policy from OWASP ZAP and configuring SOAtest to use it. The exported scan policy will configure the set of active scan rules that will be used by the Penetration Testing Tool. When using a custom active scan policy, an appropriate set of passive scan rules will be automatically used based on whether the request is made to a REST API or SOAP API.

To configure SOAtest use a custom ZAP scan policy:

  1. Go to Parasoft > Test Configuration to open the Test Configuration Manager.
  2. Click New to create a new Test Configuration or select an existing one.
  3. Open the Test Configuration’s Execution > Security tab.
  4. Enable Use custom scan policy.
  5. Click Browse and choose your ZAP .policy file.

Creating Custom Active Scan Policies

You can create or modify a custom active scan policy using the SOAtest embedded ZAP located in the SOAtest ZAP installation directory: plugins\com.parasoft.ptest.libs.web_<VERSION>\root\zap (where version is the actual version of the com.parasoft.ptest.libs.web plugin. For example, 10.5.2.202109012000)

If you have a ZAP installation on your machine that is independent of SOAtest, you will likely have a ZAP home directory in one of the following locations:

  • Windows: C:\Users\<username>\OWASP ZAP
  • Linux: ~/.ZAP
  • Mac: ~/Library/Application Support/ZAP

Your independent ZAP installation user settings stored in these directories may come in conflict with the SOAtest embedded ZAP when you launch it. For this reason, you should launch the SOAtest embedded ZAP with a custom ZAP home directory using the -dir command-line argument. Using a command prompt navigate to the SOAtest ZAP installation directory, and launch the SOAtest embedded ZAP with a custom ZAP home directory for example:

  • Windows: zap -dir C:\Users\your_name\ZAP_SOATEST
  • Linux: ./zap.sh -dir ~/.ZAP_SOATEST
  • Mac: ./zap.sh -dir ~/Library/Application Support/ZAP_SOATEST

If you don’t have a ZAP installation on your machine that is independent of SOAtest, you can run the SOAtest embedded ZAP with either zap.bat (Windows) or zap.sh (Linux, Mac) scripts located in the SOAtest ZAP installation directory.

When you launch the SOAtest embedded ZAP, it may try to use a proxy port that is already taken. If so, ZAP will prompt you to select a different port:

To create a new active scan policy or edit an existing one, choose Analyze > Scan Policy Manager from the ZAP top menu. In the Scan Policy Manager dialog, select an existing policy or create a new one. In the Scan Policy dialog, select the active scan rules you would like to apply to your custom policy:

Your custom .policy file will be saved in the policies folder of the ZAP home directory.

Note

If you want to add active scan rules that are not included in the SOAtest embedded ZAP installation, you can add ZAP add-ons that contain these rules to the plugin directory of the SOAtest ZAP installation directory and select those rules in a custom policy as described in this section. However, if you add or modify any ZAP add-ons in the SOAtest ZAP installation directory we can no longer guarantee proper execution of the SOAtest embedded ZAP, so proceed with caution and make necessary backups before proceeding.

Inspecting/Modifying Default Active Scan Policies

You can inspect and/or modify the default active scan policies used by SOAtest. They are located in the following folder of the SOAtest installation: plugins\com.parasoft.ptest.libs.web_version\root\zap\policies (where version is the actual version of the com.parasoft.ptest.libs.web plugin, for example, 10.5.2.202109012000)

This folder contains the following active policy files:

  • Parasoft SOAP.policy – Used for requests to SOAP APIs. This primarily includes Penetration Testing Tools attached to SOAP Clients, but can include other tools that make requests to SOAP APIs.
  • Parasoft REST.policy – Used for requests to REST APIs. This primarily includes Penetration Testing Tools attached to REST Clients, but can include other tools that make requests to REST APIs including other client tools and Browser Playback Tools.
  • Parasoft Web.policy – Used for requests to non-API web resources. This is only used for Penetration Testing Tools attached to Browser Playback Tools for non-API requests.

Once you’ve modified either of these policies, it will be used in the next Penetration Testing Tool invocation.

Penetration Testing Rules Supported

IDRuleCWE IDOWASPRiskTypeProfile
0Directory Browsing 548A01:2021mediumActiveWeb/REST/SOAP
2Private IP Disclosure 200A01:2021lowPassiveWeb/REST/SOAP
3Session ID in URL Rewrite 200A01:2021mediumPassiveWeb/REST/SOAP
6Path Traversal 22A03:2021highActiveWeb/REST/SOAP
7Remote File Inclusion 98A03:2021highActiveWeb/REST
41Source Code Disclosure - Git 541A05:2021highActiveWeb/REST/SOAP
42Source Code Disclosure - SVN 541A05:2021mediumActiveWeb/REST/SOAP
43Source Code Disclosure - File Inclusion 541A05:2021highActiveWeb/REST/SOAP
10003Vulnerable JS Library 829A06:2021mediumPassiveWeb/REST/SOAP
10009In Page Banner Information Leak 200A05:2021lowPassiveWeb/REST/SOAP
10010Cookie No HttpOnly Flag 1004A05:2021lowPassiveWeb/REST/SOAP
10011Cookie Without Secure Flag 614A05:2021lowPassiveWeb/REST/SOAP
10015Incomplete or No Cache-control Header Set 525UnspecifiedlowPassiveWeb/REST
10017Cross-Domain JavaScript Source File Inclusion 829A08:2021lowPassiveWeb/REST/SOAP
10019Content-Type Header Missing 345A05:2021informationalPassiveWeb/REST/SOAP
10020Anti-clickjacking Header 1021UnspecifiedmediumPassiveWeb/REST/SOAP
10021X-Content-Type-Options Header Missing 693A05:2021lowPassiveWeb/REST
10023Information Disclosure - Debug Error Messages 200A01:2021lowPassiveWeb/REST/SOAP
10024Information Disclosure - Sensitive Information in URL 200A01:2021informationalPassiveWeb/REST/SOAP
10025Information Disclosure - Sensitive Information in HTTP Referrer Header 200A01:2021informationalPassiveWeb/REST/SOAP
10026HTTP Parameter Override 20A04:2021mediumPassiveWeb/REST/SOAP
10027Information Disclosure - Suspicious Comments 200A01:2021informationalPassiveWeb/REST/SOAP
10028Open Redirect 601A03:2021highPassiveWeb/REST/SOAP
10029Cookie Poisoning 565A03:2021informationalPassiveWeb/REST/SOAP
10030User Controllable Charset 20A03:2021informationalPassiveWeb/REST/SOAP
10031User Controllable HTML Element Attribute (Potential XSS) 20A03:2021informationalPassiveWeb/REST/SOAP
10032Viewstate 642Unspecifiedhigh, medium, low, informationalPassiveWeb/REST/SOAP
10033Directory Browsing 548A01:2021mediumPassiveWeb/REST/SOAP
10034Heartbleed OpenSSL Vulnerability (Indicative) 119A09:2021highPassiveWeb/REST/SOAP
10035Strict-Transport-Security Header 319A05:2021low, informationalPassiveWeb/REST/SOAP
10036HTTP Server Response Header 200A05:2021low, informationalPassiveWeb/REST/SOAP
10037Server Leaks Information via 'X-Powered-By' HTTP Response Header Field(s) 200A01:2021lowPassiveWeb/REST/SOAP
10038Content Security Policy (CSP) Header Not Set 693A05:2021medium, informationalPassiveWeb/REST/SOAP
10039X-Backend-Server Header Information Leak 200A05:2021lowPassiveWeb/REST/SOAP
10040Secure Pages Include Mixed Content 311A05:2021medium, lowPassiveWeb/REST/SOAP
10041HTTP to HTTPS Insecure Transition in Form Post 319A02:2021mediumPassiveWeb/REST/SOAP
10042HTTPS to HTTP Insecure Transition in Form Post 319A02:2021mediumPassiveWeb/REST/SOAP
10043User Controllable JavaScript Event (XSS) 20A03:2021infoPassiveWeb/REST/SOAP
10044Big Redirect Detected (Potential Sensitive Information Leak) 201A04:2021lowPassiveWeb/REST/SOAP
10045Source Code Disclosure - /WEB-INF folder 541A05:2021highActiveWeb/REST/SOAP
10047HTTPS Content Available via HTTP 311A05:2021lowActiveWeb/REST/SOAP
10048Remote Code Execution - Shell Shock 78A09:2021highActiveWeb/REST/SOAP
10049Content Cacheability 524UnspecifiedinformationalPassiveWeb/REST
10050Retrieved from Cache UnspecifiedUnspecifiedinformationalPassiveWeb/REST/SOAP
10051Relative Path Confusion20A05:2021mediumActiveWeb
10052X-ChromeLogger-Data (XCOLD) Header Information Leak 200A04:2021mediumPassiveWeb/REST/SOAP
10054Cookie without SameSite Attribute 1275A01:2021lowPassiveWeb/REST/SOAP
10055CSP 693A05:2021medium, low, informationalPassiveWeb/REST/SOAP
10056X-Debug-Token Information Leak 200A01:2021lowPassiveWeb/REST/SOAP
10057Username Hash Found 284A01:2021informationalPassiveWeb/REST/SOAP
10061X-AspNet-Version Response Header 933A05:2021lowPassiveWeb/REST/SOAP
10062PII Disclosure 359A04:2021highPassiveWeb/REST/SOAP
10063Permissions Policy Header Not Set 16A01:2021lowPassiveWeb/REST/SOAP
10070Use of SAML UnspecifiedUnspecifiedinformationalPassiveWeb/REST/SOAP
10094Base64 Disclosure 200A04:2021high, informationalPassiveWeb/REST/SOAP
10095Backup File Disclosure 530A04:2021mediumActiveWeb/REST/SOAP
10096Timestamp Disclosure 200A01:2021informationalPassiveWeb/REST/SOAP
10097Hash Disclosure 200A04:2021high, lowPassiveWeb/REST/SOAP
10098Cross-Domain Misconfiguration 264A01:2021mediumPassiveWeb/REST/SOAP
10099Source Code Disclosure 540A05:2021mediumPassiveWeb/REST/SOAP
10103Image Location and Privacy Scanner 200UnspecifiedinformationalPassiveWeb/REST/SOAP
10105Weak Authentication Method 287A01:2021high, mediumPassiveWeb/REST/SOAP
10106HTTP Only Site 311A05:2021mediumActiveWeb/REST/SOAP
10107Httpoxy - Proxy Header Misuse 20A06:2021highActiveWeb/REST/SOAP
10108Reverse Tabnabbing UnspecifiedA04:2021mediumPassiveWeb/REST/SOAP
10109Modern Web Application UnspecifiedUnspecifiedinformationalPassiveWeb/REST/SOAP
10110Dangerous JS Functions 749A04:2021lowPassiveWeb/REST/SOAP
10202Absence of Anti-CSRF Tokens 352A01:2021low, informationalPassiveWeb/REST/SOAP
20012Anti-CSRF Tokens Check352A05:2021highActiveWeb
20015Heartbleed OpenSSL Vulnerability 119A06:2021highActiveWeb/REST/SOAP
20016Cross-Domain Misconfiguration 264A01:2021highActiveWeb/REST/SOAP
20017Source Code Disclosure - CVE-2012-1823 20A06:2021highActiveWeb/REST/SOAP
20018Remote Code Execution - CVE-2012-1823 20A06:2021highActiveWeb/REST/SOAP
20019External Redirect 601A03:2021highActiveWeb/REST
30001Buffer Overflow 120A03:2021mediumActiveWeb/REST/SOAP
30002Format String Error 134A03:2021mediumActiveWeb/REST/SOAP
30003Integer Overflow Error 190A03:2021mediumActiveWeb/REST
40003CRLF Injection 113A03:2021mediumActiveWeb/REST
40008Parameter Tampering 472A04:2021mediumActiveWeb/REST/SOAP
40009Server Side Include 97A03:2021highActiveWeb/REST
40012Cross Site Scripting (Reflected) 79A03:2021highActiveWeb/REST
40013Session Fixation 384A01:2021highActiveWeb/REST/SOAP
40014Cross Site Scripting (Persistent) 79A03:2021highActiveWeb/REST
40015LDAP Injection 90A03:2021highActiveWeb/REST/SOAP
40016Cross Site Scripting (Persistent) - Prime 79UnspecifiedinformationalActiveWeb/REST
40017Cross Site Scripting (Persistent) - Spider 79UnspecifiedinformationalActiveWeb/REST
40018SQL Injection 89A03:2021highActiveWeb/REST/SOAP
40025Proxy Disclosure 200A05:2021mediumActiveWeb/REST/SOAP
40028ELMAH Information Leak 215A05:2021mediumActiveWeb/REST/SOAP
40029Trace.axd Information Leak 215A05:2021mediumActiveWeb/REST/SOAP
40031Out of Band XSS79A03:2021highActiveWeb/REST
40032.htaccess Information Leak 215A05:2021mediumActiveWeb/REST/SOAP
40034.env Information Leak 215A05:2021mediumActiveWeb/REST/SOAP
40035Hidden File Finder 538A05:2021mediumActiveWeb/REST/SOAP
40038Bypassing 403 UnspecifiedA01:2021mediumActiveWeb/REST/SOAP
40039Web Cache Deception UnspecifiedA05:2021mediumActiveWeb/REST/SOAP
40040CORS Header 942A01:2021high, medium, informationalActiveWeb/REST
40042Spring Actuator Information Leak215A01:2021mediumActiveWeb/REST/SOAP
40044Exponential Entity Expansion (Billion Laughs Attack)776A04:2021mediumActiveWeb/REST/SOAP
40045Spring4Shell78A03:2021, A06:2021highActiveWeb/REST/SOAP
90001Insecure JSF ViewState 642A04:2021mediumPassiveWeb/REST/SOAP
90002Java Serialization Object 502A04:2021mediumPassiveWeb/REST/SOAP
90003Sub Resource Integrity Attribute Missing 345A05:2021mediumPassiveWeb/REST/SOAP
90004Insufficient Site Isolation Against Spectre Vulnerability 693A04:2021lowPassiveWeb/REST/SOAP
90005Fetch Metadata Request Headers352UnspecifiedinformationalPassiveWeb/REST
90011Charset Mismatch 436UnspecifiedinformationalPassiveWeb/REST/SOAP
90017XSLT Injection 91A03:2021mediumActiveWeb/REST/SOAP
90019Server Side Code Injection 94A03:2021highActiveWeb/REST/SOAP
90020Remote OS Command Injection 78A03:2021highActiveWeb/REST/SOAP
90021XPath Injection 643A03:2021highActiveWeb/REST/SOAP
90022Application Error Disclosure 200A05:2021mediumPassiveWeb/REST/SOAP
90023XML External Entity Attack 611A03:2021highActiveWeb/REST/SOAP
90024Generic Padding Oracle 209A02:2021highActiveWeb/REST/SOAP
90025Expression Language Injection917A03:2021highActiveWeb
90028Insecure HTTP Method 200A05:2021mediumActiveWeb/REST/SOAP
90030WSDL File Detection UnspecifiedA05:2021informationalPassiveWeb/REST/SOAP
90033Loosely Scoped Cookie 565A08:2021informationalPassiveWeb/REST/SOAP
90034Cloud Metadata Potentially Exposed UnspecifiedA05:2021highActiveWeb/REST/SOAP
90035Server Side Template Injection94UnspecifiedhighActiveWeb/REST
90036Server Side Template Injection (Blind)74UnspecifiedhighActiveWeb/REST
110001Application Error Disclosure via WebSockets 209UnspecifiedmediumPassiveWeb/REST/SOAP
110002Base64 Disclosure in WebSocket message UnspecifiedUnspecifiedinformationalPassiveWeb/REST/SOAP
110003Information Disclosure - Debug Error Messages via WebSocket 200UnspecifiedlowPassiveWeb/REST/SOAP
110004Email address found in WebSocket message 200UnspecifiedinformationalPassiveWeb/REST/SOAP
110005Personally Identifiable Information via WebSocket 359UnspecifiedhighPassiveWeb/REST/SOAP
110006Private IP Disclosure via WebSocket UnspecifiedUnspecifiedlowPassiveWeb/REST/SOAP
110007Username Hash Found in WebSocket message 284UnspecifiedinformationalPassiveWeb/REST/SOAP
110008Information Disclosure - Suspicious Comments in XML via WebSocket 200UnspecifiedinformationalPassiveWeb/REST/SOAP
111001HTTP Verb Tampering (Parasoft proprietary rule)287A07:2021mediumActiveWeb/REST

Integration with Burp Suite

SOAtest uses a preconfigured instance of OWASP ZAP under the hood to perform penetration testing. You also have the option to use the commercial tool Burp Suite for penetration testing by leveraging the Burp Suite Extension.



  • No labels