This section describes how to use test configurations with the SOAtest and Virtualize server.

Introduction

Test configurations are collections of settings that determine how tests are executed and reported. The SOAtest and Virtualize server ships with the following default configurations: 

You can customize as necessary by enabling and disabling properties in the test configuration file. You can also create custom test configurations and add them to the workspace where they can be used to execute tests in your project. 

Built-in Test Configurations

You can run built-in test configurations by specifying builtin:// and the test configuration name when making your call. In the following example, the tests will execute according to the default Run Automated Server Tests configuration:

{
  "general": {
    "config": "builtin://Run Automated Server Tests",

See Test Configuration Properties for information about properties you can customize in the test configuration. See SOAtest Test Configuration Settings for additional information about test configurations.If the folder does not contain a test configuration with the specified name, the test will not run.

User-defined Test Configurations

You can run user-defined test configurations by specifying user:// and the test configuration name when making your call. In the following example, the tests will execute according to the default  Example Configuration configuration:

{
  "general": {
    "config": "user://Example Configuration",

You can add other test configurations to the <WORKSPACE>/TestAssets/configs/user folder and execute them using the user://<TEST_CONFIG_NAME> syntax. You can add test configurations by creating duplicates of the default configuration and modifying them to exercise the tests in different ways. See Test Configuration Properties for information about properties you can customize in the test configuration.

If the folder does not contain a test configuration with the specified name, the test will not run.

Test Configuration Properties

You can modify the following settings for Run Automated Server Tests.properties file, as well as any custom user configurations you may want to create (see User Configurations ).

Functional Tests Properties

com.parasoft.xtest.execution.api.web.execute_functional_tests

This setting enables/disables functional test execution. Set this property to true to enable functional test execution. Default is true.

Example

com.parasoft.xtest.execution.api.web.execute_functional_tests=true

com.parasoft.xtest.execution.api.web.report_successful_traffic

This setting enables/disables the "Test Traffic [All Tests]" section in the API coverage report. The section contains traffic for all test executions, whether or not the tests were successful. Set this property to true to include the Test Traffic [All Tests] section in the report. Default is true.Also see Reviewing API Coverage Results.Example

com.parasoft.xtest.execution.api.web.report_successful_traffic=true

com.parasoft.xtest.execution.api.web.report_traffic_limit

This setting specifies the total amount of traffic data (in KBs) stored during a test execution session, not the amount of data stored per test. The com.parasoft.xtest.execution.api.web.report_successful_traffic setting must be enabled for this property to take effect. Only data from successful tests will count toward the limit if the com.parasoft.xtest.execution.api.web.report_successful_traffic property is enabled. The default is 500.

Example

com.parasoft.xtest.execution.api.web.report_traffic_limit=500

API Coverage Properties

com.parasoft.xtest.execution.api.web.coverage_ref_by_tests

This setting determines if coverable resources from the service definition are included in the API coverage report (see API Coverage - Overview for information about coverable resources). If a constrained SOAP Client, REST Client, or Messaging Client tool touches a coverable resource, coverage information will be associated with the appropriate resource/operation. The following service definitions are supported:

  • OpenAPI/Swagger
  • RAML
  • WADL
  • WSDL

Default is false. 

Example

com.parasoft.xtest.execution.api.web.coverage_ref_by_tests=true

com.parasoft.xtest.execution.api.web.perform_service_coverage_calculation

This setting enables/disables API coverage calculation for a user-defined list of coverable resources. You can specify the resources in the com.parasoft.xtest.execution.api.web.service_coverage_table setting. 

Coverage can be calculated in addition to, or instead of, the automatically calculated coverable resources. Also see API Coverage - Overview.Default is false.

com.parasoft.xtest.execution.api.web.service_coverage_table

This setting specifies a list of resources that should be included as part the API coverage calculation. Specify a semicolon-separated list of definitions using the following format:

<definition_type>@<URI>

The following definition types are supported:

  • OpenAPI/Swagger
  • RAML
  • WADL
  • WSDL

Example

com.parasoft.xtest.execution.api.web.service_coverage_table=WSDL@http://soatest.parasoft.com/calculator.wsdl;OpenAPI/Swagger@file:///C:/openapi3.0/examples/v3.0/petstore.yaml;OpenAPI/Swagger@http://localhost:8080/soavirt/api/v5/def

In this example, the property would include coverage information for the following resources:

  • WSDL@http://soatest.parasoft.com/calculator.wsdl
  • OpenAPI/Swagger@file:///C:/openapi3.0/examples/v3.0/petstore.yaml
  • OpenAPI/Swagger@http://localhost:8080/soavirt/api/v5/def

Application Coverage Properties

com.parasoft.xtest.execution.api.web.collect_application_coverage

This setting enables/disables collecting application coverage data with the coverage agent. The default is false. 

Example

com.parasoft.xtest.execution.api.web.collect_application_coverage=true

com.parasoft.xtest.execution.api.web.application_coverage_host

This setting specifies the host name or IP address of the machine where the coverage agent resides.

Example

com.parasoft.xtest.execution.api.web.application_coverage_host=myserver1.mycompany.com

com.parasoft.xtest.execution.api.web.application_coverage_port

This setting specifies the coverage agent’s port. The default is 8050.

Example

com.parasoft.xtest.execution.api.web.application_coverage_port=8050

com.parasoft.xtest.execution.api.web.application_coverage_user_id

This setting specifies a user ID so that coverage results can be associated with a specific user. A user ID should only be specified when the coverage agent is configured to run in the multi-user mode. For details, see the user guide for your Parasoft code analysis tool (Parasoft Jtest or Parasoft dotTEST) at https://docs.parasoft.com.

Example

com.parasoft.xtest.execution.api.web.application_coverage_user_id=user_qa21

com.parasoft.xtest.execution.api.web.application_coverage_report_as_test_failure

This setting enables/disables reporting test failures when the coverage agent connection fails, or when the location of the static coverage file is misconfigured. Otherwise connection problems will be reported to the console but will not cause the test to fail. The default is false. 

Example

com.parasoft.xtest.execution.api.web.application_coverage_report_as_test_failure=true

com.parasoft.xtest.execution.api.web.application_coverage_upload_to_dtp

This setting enables/disables sending application coverage results to DTP. It requires the com.parasoft.xtest.execution.api.web.collect_application_coverage option to be enabled. The default is true.

In addition, sending results to DTP requires the following settings to be configured:

  • com.parasoft.xtest.execution.api.web.application_coverage_host
  • com.parasoft.xtest.execution.api.web.application_coverage_port
  • com.parasoft.xtest.execution.api.web.application_coverage_static_coverage_file
  • com.parasoft.xtest.execution.api.web.application_coverage_image_tags

Example

com.parasoft.xtest.execution.api.web.application_coverage_upload_to_dtp=true

com.parasoft.xtest.execution.api.web.application_coverage_static_coverage_file

This setting specifies the path to the static coverage file generated by your Parasoft code analysis tool (Parasoft Jtest or Parasoft dotTEST) in the .xml or .data format. For details, see the Parasoft Jtest or Parasoft dotTEST user guide at https://docs.parasoft.com.

Example

com.parasoft.xtest.execution.api.web.application_coverage_static_coverage_file=C:\projects\MyApp\target\jtest\monitor\static_coverage.xml

com.parasoft.xtest.execution.api.web.application_coverage_image_tags

This setting specifies a set of tags that are used to create coverage images in DTP. A coverage image is a unique identifier for aggregating coverage data from runs with the same build ID. For details, see the Parasoft DTP user guide at https://docs.parasoft.com.

Example

com.parasoft.xtest.execution.api.web.application_coverage_image_tags=FT;AppCov;Aggregate

API Security Properties

com.parasoft.xtest.execution.api.web.api_security_policy

This setting specifies whether a default or custom scan policy is used. The value must be either default or custom. The default is default.

Example

com.parasoft.xtest.execution.api.web.api_security_policy=custom

com.parasoft.xtest.execution.api.web.api_security_custom_policy_location

This setting specifies the location of a custom scan policy file. This property value is ignored if com.parasoft.xtest.execution.api.web.api_security_policy is set to default. For more information, see "Configuring Scan Policies" on the Penetration Testing page.Example

com.parasoft.xtest.execution.api.web.api_security_custom_policy_location=C:\projects\soatest\security\custom_security_policy.xml

com.parasoft.xtest.execution.api.web.api_security_timeout

This setting specifies the timeout period for the Penetration Testing Tool in minutes. The default is 15.

Example

com.parasoft.xtest.execution.api.web.api_security_timeout=15

com.parasoft.xtest.execution.api.web.api_security_includes

This setting specifies regular expressions used to determine URLs that should be scanned by the Penetration Testing Tool. Includes are processed before excludes. When no includes are defined, everything is assumed to be included before considering excludes. Use semi-colons ( ; ) to separate multiple regular expressions.

Example

com.parasoft.xtest.execution.api.web.api_security_includes=https?:\/\/localhost:8080\/parabank\/.*;https?\/\/www\.domain\.com\.*

com.parasoft.xtest.execution.api.web.api_security_excludes

This setting specifies regular expressions used to determine URLs that should be excluded from scanning by the Penetration Testing Tool. Excludes are processed after includes. Use semi-colons ( ; ) to separate multiple regular expressions.

Example

com.parasoft.xtest.execution.api.web.api_security_excludes=.*\/(login|logout).*;.*\.gif;http:\/\/localhost:8080\/parabank\/index\.htm