Many SOAtest users create thousands of test cases as they evolve their test suites with Parasoft SOAtest. It is critical to be able to maintain these tests efficiently and effectively as the underlying systems that are being tested change.

This section describes a few capabilities and recommendations that help save time when maintaining and updating large numbers of tests.

Managing Service Definition and Schema Changes

If you are testing web services described with service definition or schemas, we recommend the following best practices:

  1. Use Form Input and constrain your tests to the WSDL/WADL/Schema/OpenAPI/Swagger/RAML definition. SOAP Client and Messaging Client tools include a “constrain” option in the WSDL tab (SOAP Client) and Schema tab (Messaging Client). The REST Client also contains a similar option for WADLs and RAML/OpenAPI/Swagger definitions. When this option is selected, the following happens when the test is executed in a session (for example, when you execute the test from the command line during nightly testing). First, the definition files will be parsed from the given location. Second, the generated request from Form Input/Form JSON view will automatically conform to the recently parsed definition files.  More details on this feature are provided below.
  2. For endpoint URLs, use Environment variables to manage updates globally, or select the “WSDL” option in SOAP Client so the URL gets updated automatically based on the WSDL.
  3. Use Change Advisor to identify tests that require updates, then update the applicable tests by applying change templates.
  4. Use Parasoft search and replace to update your tool values such as XPath values in assertion tools, diff tools, data bank tools.
  5. Create a regression test for the WSDL, WADL, and/or schemas that are associated with your tests. SOAtest's Test Suite wizard includes an option to create WSDL/WADL tests in addition to functional tests. One of these WSDL/WADL tests is a regression test that captures the current version of the WSDL/WADL and serves as a change indicator; whenever the definition file gets modified on the server, the test will fail and therefore alert you to the change.
  6. Use regression control test suite level update capabilities.

Automatic Form Input Updates

When the "constrain" option is selected, Form Input views in SOAP Clients, Message Stubs, and Messaging Clients can be updated automatically. The same applies to REST Clients that are constrained to a WADL, RAML, or OpenAPI/Swagger definition. This operation is performed by "refreshing" the definition files (by clicking on the Refresh button in one of these tools). Parasoft SOAtest attempts to preserve existing Form Input settings so that only minimal manual changes are required. Parasoft SOAtest uses sophisticated heuristics that map existing test data.  

For example, new elements appear and removed elements are removed without impacting existing ones. Also, changes to the type of an element attempt to carry the current values whenever possible. The remapping of test data according to the new definition file may not always be successful. Typically, this process is more likely to succeed for changes in data types as defined by schema files because these are relatively minor changes. However, this process is less likely to succeed for operation changes as defined by the definition file document because these are relatively major changes.

Updating Messages Specified in Form Input with Change Advisor, Change Impact Analysis

With Change Advisor, you can perform a change impact analysis to identify how changes to a service definition (for example, renamed operations, added elements or types, modified namespaces, and so on) impact the messages that your messaging tools send.

The analysis can either: 

The result of this analysis helps you assess the scope and nature of changes that will be required to update your tests. Once a change template defining how the new service maps to the previous one has been defined (by you or another team member, such as an architect), you can apply that change template to automatically update the tests, individually or in bulk. 

For details, see Updating Messages with Change Advisor.

Automatically Updating Diff Tool Regression Controls

Use of an XML Diff Tool is a popular and an effective way to apply validation on the service responses. It provides the convenience of capturing the entire response message to validate it in its entirety and it allows various batch update mechanisms to be executed at the test or test suite level by right-clicking and choosing Update/Create Regression Controls, then following the wizard. If you wish to manage the regression control updates more selectively, then open an editor for the test suite and go to the Regression Options section under the Execution Options tab. There you can select which tests should be updated when updating regression controls at the test suite level.

Updating Other Tool Values Using Search and Replace

The Parasoft search and replace functionality helps you identify and update values that need to be changed as services and environments evolve. For example, if an element or namespace changed, you can use search and replace to update the impacted XPath values in assertion tools, diff tools, data bank tools, and so on.

For details, see Updating Tool Values with Search and Replace.

Workflow for Updating Tests

With the above capabilities a tester would follow these steps:

  1. Notice a WSDL/WADL/Schema Regression test failure.
  2. Inspect some of the response errors.
  3. Update the impacted tools as needed. You can:

Other Recommendations and Practices to Manage Change