The Execution tab settings define how tests are executed. During a test, SOAtest will execute test cases based on the parameters defined in the selected Test Configuration’s Execution tab. For any level of execution, the top-level Enable Test Execution option must be enabled.
In this section:
The Execution> Functional tab has the following settings:
- Execute functional tests: Determines whether any functional tests are run.
- Enable event logging: Determines whether SOAtest logs the data needed to provide a detailed chronologist sequence of all the events that occurred between the start and end of the test (for instance, all requests sent, responses received, data source rows used, wait times, navigation tasks, and so on). See Exploring Test Event Details for details.
- Execute in load test mode: Determines whether SOAtest executes available tests in load testing mode and alerts you to any outstanding issues that might impact your load testing—for example, incorrectly configured HTTP requests. See Validating Tests for details.
- Auto-configure tests in preparation for load testing: Determines whether SOAtest configures browser-based web scenariosto run in a browser-less load test environment. See Configuring Tests for details.
- Execute only opened Test Suite (.tst) Files (always false in command-line mode): Determines whether SOAtest executes Test Suites that are not currently active (i.e., tests that you are not currently working on).
- Report traffic for all tests: Determines whether reports contain a "Test Traffic [All Tests]" section, which contains traffic for every test execution—whether or not it was successful. If this is enabled, you can also configure the traffic limit: the amount of traffic that will be stored during a test execution session (not per test). The default is 500 KB.
- Launch an application: Allows you to configure a SOAtest test configuration to run an Eclipse launch configuration at the beginning of the execution of the test configuration. For example, assume you want to run a test scenario against a local copy of an application that you start and run within Eclipse. If you want to start your application and run your tests in a single step, you can create a Test Configuration to launch this application as well as execute tests.
- Override default environment during test execution: Configures SOAtest to always use the specified environment for tests run with this Test Configuration—regardless of what environment is active in the Test Case Explorer.
For example, assume you have the following environments:
This is how you set the Test Configuration to always use the "staging server" environment:
- Use playback engine: Allows you to override a test’s playback engine settings at the time of test execution. By default, Test Configurations are set to play web scenarios using the playback engine specified at the test suite level. This allows you to use a single Test Configuration to execute a mixture of tests configured for Selenium and tests configured for the legacy engine. If you select a specific driver here, it will be used regardless of what engine is configured at the test scenario level. See Using Selenium WebDriver for Legacy Browser Recordings and Using the Legacy Native Driver Instead of Selenium for details.
- Use browser: Allows you to override a test’s browser playback settings at the time of test execution. See Configuring Browser Playback Options for details.
- Apply static analysis to: If a Test Configuration runs both static analysis and test execution (e.g., for performing static analysis on a web scenario), this setting determines whether static analysis is performed on the HTTP responses, or the browser contents.
- HTTP Responses refers to the individual HTTP messages that the browser made in order to construct its data model—the content returned by the server as is (before any browser processing).
- Browser-Constructed HTML refers to the real-time data model that the browser constructed from all of the HTML, JS, CSS, and other files it loaded.
The Execution> Security tab allows you to configure penetration testing, which is described in Penetration Testing.
Runtime Error Detection tab
The Execution> Runtime Error Detection tab allows you to configure runtime error detection, which is described in Performing Runtime Error Detection.
Change Impact tab
The Execution> Change Impact tab contains an option (Perform change impact analysis) that controls whether the current Test Configuration performs change impact analysis during test execution. See Updating Messages with Change Advisor for details.
API Coverage tab
The Execution> API Coverage tab contains options for calculating API Coverage during test execution. See API Coverage for details.
Application Coverage tab
The Execution> Application Coverage tab contains options for collecting application coverage data, which provides visibility into the level of code coverage achieved by your SOAtest tests. See Application Coverage for details.