You can automatically create API tests for web applications during manual testing using the SOAtest Smart API Test Generator. The out-of-the-box implementation of the Smart API Test Generator is designed to help you start API testing very quickly, but you can also “train” it to create more specific tests for your projects. In this section:

Overview

You can define rules that determine test generation behaviors by creating Smart Test Template (.stt) files. Smart Test Template files consist of one or more Resource Suites, which enable you to determine the paths your rules will be applied to. Resource Suites contain one or more Resource Templates. You can chain tools to Resource Templates and configure them to define specific test generation behaviors. Resource Templates also enable you to configure the specific scope that the chained rules will apply to. 

When the Smart API Test Generator creates a new API test, it will read and apply the rules specified in the .stt file. 

Manually Creating and Configuring Test Templates

You can create and manage .stt files using the Smart Test Templates View. Additionally, you can add multiple .stt files and organize them into folders.

  1. Choose Parasoft> Show View> Smart Test Templates to open the Smart Test Templates view (if not already open).
  2. Click the Add Template button and specify a name and location for the .stt file.
  3. (Optional) You can specify a path to create nested folders to organize your .stt file.
     e
  4. Click Next.

  5. Choose the type of Resource Suite to create. You can create an empty suite or initialize the suite based on a service definition.

    If you are using a definition file, see the following sections for additional information:

    Creating Tests from an OpenAPI/Swagger Definition

    Creating Tests From a RAML Definition

    Creating Tests From a WADL

  6. Click Next if you are creating a suite based on a service definition and specify the location of your definition file. Skip this step to create an empty suite.
     
  7. Click Finish.
  8. Expand the .stt file and double-click the Resource Suite. 
  9. (Optional) You can disable the Use Default Name option and specify a name for the file.
  10. Specify a pattern in the Match field to identify the endpoints you want the rules in this suite to apply to (see Defining Smart API Test Generation Scope). The Resolved field renders a preview of the pattern. Nested Resource Suites inherit the Match settings from the parent Resource Suite. You can determine how to structure template files and configure matching patterns accordingly. See Example Smart Test Template for a common use-case.  
  11. Right-click the Resource Suite and choose Add New> Resource Template.
  12. (Optional) Disable the Use Default Name option and specify a name for the Resource Template in the Name field.

  13. Choose a method from the Method drop-down menu and specify a pattern in the Match field to identify the endpoints you want the rules in the template to apply to (see Defining Smart API Test Generation Scope). The Resolved field renders a preview of the pattern. Resource Templates inherit the Match settings from the parent Resource Suite. You can disable the Exact Match option if you want to allow the Smart API Test Generator to generate tests for all subpaths specified in the Match field. The Exact Match option is enabled by default. 

    When the Smart API Test Generator matches the setting in the Resource Template, configurations from all chained tools are applied. You can determine how to structure template files and configure matching patterns accordingly. See Example Smart Test Template for a common use-case.

    Chaining JSON Assertors

    JSON Assertor tools chained to Resource Templates can be configured to use the values from traffic if present. See Adding JSON Assertions for additional information. 

  14. Right-click the Resource Template and choose Add Output...
  15. Choose a tool to include in your generated test and configure the settings. For example, you can chain and configure an HTTP Authentication tool so that all test suites generated with the template automatically authenticate against the application under test (see Enabling Authentication for additional information).
  16. Add additional Resource Suites and Resource Templates as necessary. The .stt file is processed in order from first to last. Each Resource Suite is processed according to its hierarchy before the next suite is processed. You can add as many suites and templates and chain as many tools as necessary to train the Smart API Test Generator. 
  17. Save your changes.

Generate a test from an existing traffic file or create a new test using the SOAtest Smart API Test Generator browser extension and the configurations defined in the .stt file will be applied. 

Defining Smart API Test Generation Scope

Patterns you define in Resource Suites and Resource Templates will be matched according to the following rules:

  • Curly braces, e.g., {}, indicate wildcards and can contain any text. For example, you can specify a pattern that translates into scheme, host, and port:

    {scheme}://{host}:{port}
     
  • Asterisks (*) also indicate wildcards. A single asterisk and curly braces, e.g., {}, may be used interchangeably:

    {scheme}://{host}:*/parabank/*/accounts/{id}

  • You can use a double-asterisk to match multiple directories.

    {scheme}://{host}:*/parabank/**/{id}

  • Excluding scheme, host, and port will match any scheme, host, and port. For example, the following pattern matches the specific path on any host, port, and method:

    /parabank/services_proxy/bank/accounts/{id}
     
  • Specifying a negative number or a number greater than 65535 for the port treats the port as a wildcard.
  • Resource Templates and nested Resource Suites inherit the Match settings from the parent Resource Suite. 
  • You can disable the Exact Match option if you want to allow the Smart API Test Generator to generate tests for all subpaths specified in the Match field. The Exact Match option is enabled by default in Resource Templates. 

Default Scope

By default, the Smart API Test Generator will create tests for all resources called in the scenario and only apply training tools to resources identified with the Resource Template tools. You can configure the Smart API Test Generator, however, to include or exclude specific resource URLs from test generation by specifying Ant-style patterns in the includeURLPatterns and excludeURLPatterns in the tst_configuration.properties file. See Test Creation Properties for additional information. 

Enabling Authentication

The Smart API Test Generator uses the HTTP Authentication tool for configuring authentication credentials for your tests. The HTTP Authentication tool is a special tool that you can only use in .stt files. When the Smart API Test Generator matches the settings in the Resource Template tool, credentials configured in the HTTP Authentication tool are applied to the generated tests. 

  1. Right-click a Resource Template tool and choose Add Output...
  2. Expand the Request menu and choose Transport Header.
  3. Choose HTTP Authentication from the tools panel and click Finish. 
  4. Configure the authentication settings in the HTTP Authentication tool. The following authentication types are supported:
    • Basic (simple authentication built into the HTTP protocol)
    • NTLM
    • Kerberos
    • Digest

See the following sections for additional information on configuring authentication settings:

SOAtest

Virtualize

Adding Headers

The Smart API Test Generator uses the HTTP Header tool to add header fields to your tests. The HTTP Header tool is a special tool that you can only use in .stt files. When the Smart API Test Generator matches the settings in the Resource Template tool, any headers configured in the HTTP Header tool will automatically be added to the generated tests. You also can use this tool to override header values captured during test creation. 

  1. Right-click a Resource Template tool and choose Add Output...
  2. Expand the Request menu and choose Transport Header.
  3. Choose HTTP Header from the tools panel and click Finish. 
  4. Click Add in the Tool Settings section to define values.
  5. Save your changes.

Adding JSON Assertions

You can configure JSON Assertions chained to Resource Template tools to use values from traffic. Use == or equals as the operator and set the value field [Smart - From Traffic].

JSON Assertor Behaviors by Type

There are several types of assertions you can use in your tests (see JSON Assertor for additional information). The following sections describes how the Smart API Test Generator applies settings for each type of assertion.

Value Assertions

  • The Expected Value field for Value Assertions will be set from the recorded traffic if the field is set to [Smart - From Traffic].
  • The Element value field and the expected value field for Value Occurrence Assertions, Numeric Assertions, and String Comparison Assertions will be set based on the recorded traffic if the field is set to [Smart - From Traffic] and the operation is set to == or equal.
  • The Expected value field for Value Occurrence Assertions will be set to 1 or 0 if the field is set to [Smart - From Traffic] and the operation is set to ==.
  • Fields for Regular Expression Assertions, Expression Assertions, and Custom Assertions will not be set from traffic.

Structure Assertions

  • The expected value field for Occurrence Assertions will be set to 1 or 0 if the field is set to [Smart - From Traffic] and the operation is set to ==.
  • Fields for Has Contents Assertions, Has Children Assertions, and Type assertions will not be set from traffic.

Difference Assertions

  • The Base Value field will be set based on the recorded traffic if the field is set to [Smart - From Traffic] and if the traffic value validates successfully.
  • The Difference Value field for Numeric Assertions will be set to 0 if the field is set to [Smart - From Traffic].
  • Difference configuration fields for Date Difference Assertions and DateTime Difference Assertions will not be set from traffic.

Range Assertions

  • The Lower Bound and Upper Bound fields will be set based on the recorded traffic if the field is set to [Smart - From Traffic] and the value validates successfully. In this case, the same traffic value will be applied to all fields in the assertion.

Additional JSON Assertor Behaviors

By default, the Smart API Test Generator ignores timestamps, but it can be configured to ignore other parameters to fit your needs. See Test Creation Parameters for additional information.  

Training Templates on Assertions from a .tst File

You can apply assertion logic from an existing .tst file to train Smart API Test Generator. See Training the Smart API Test Generator Using a .tst File.

Adding Test Suite References

You can reference other test suites in your .stt file, which enables you to include additional set-up steps that may be necessary for your tests to execute properly. Only one referenced test suite will be added per Resource Template tool. Tests included by reference are added as the first test in the generated suite. See Reusing and Modularizing Test Suites for End-to-end Testing for additional information about references.

  1. Right-click a Resource Template tool and choose Add Output...
  2. Expand the Suite category and choose Beginning.
  3. Choose Test Suite Reference in the Smart category and click Finish.   
  4. Specify the location of the test suite you want to reference in the Test Suite Reference editor.

    You can click File System or Workspace to browse for the .tst file you want to reference. 
  5. Save your changes. 

Training the Smart API Test Generator Using a .tst File

You can use existing REST Clients to teach the Smart API Test Generator authentication settings and JSON assertion logic. You can create training rules from the .tst file, Test Suite, or REST Client node. SOAtest will create an HTTP Authentication or a JSON Assertor rule under the resource template that most closely matches the path specified in the REST Client. If no match is identified, then new .stt files will be created for each rule. The .stt file(s) will contain Resource Suites and Resource Templates with their Match settings configured to match the REST Client's endpoint. 

The match pattern for new .stt files is based on the type of rule you are teaching the Smart API Test Generator. For assertions, the parent suite match is set using the REST client's basePath. Subpaths are configured in a nested suite match and/or a resource template match. The resource template will be configured to match the "Exact" field at the specific method type.

For authentication, the parent suite match is set using the REST client's scheme, host, and port. The resource template contained in the parent suite will be configured to match to the REST client's basePath and all subpaths and method types.

  1. Right-click the .tst file, Test Suite, or REST Client node in the Test Explorer and choose Train Smart Test Template.

  2. Review the rules when prompted. Smart API Test Generator will attempt to match resources in existing .stt files and add new rules accordingly. If a matching .stt is not found, a new file will be created. 
     

    Training Smart API Test Generator with JSON Assertors

    Equality assertions (assertions configured with the == or equals operator) with fixed values will be created based on traffic. If the value is parameterized with a non-equality assertions, the assertor chained to the resource will be a converted to a fixed value set to [Smart - User Input].  For parameterized equality assertions the value will be [Smart - From Traffic].

    New assertions will be added to existing assertors in the .stt file.

  3. Click OK to update or add a new .stt file to the Smart Test Templates view that contains the settings configured in the REST Client. If a new .stt file is created, the file name will be determined based on whether you are teaching the Smart API Test Generator authentication settings or assertion logic. For assertions, the original client's basePath will be used for the name. For authentication, the name will be based on the host name configured in the original client. If the host name and/or basePath are parameterized in the original client, the resource will be named with "Template.stt". You can right-click the file in the Smart Test Templates view and rename it. 
  4. Double-click the Resource Suite and the Resource Template and configure their Match settings to define the test generation scope. See Defining Smart API Test Generation Scope. 
  5. You can manually add additional resources to finish configuring the .stt file. See Manually Creating and Configuring Test Templates.

Example Smart Test Template

The structure of your test templates will depend on how you want to test your application. In this example, an .stt represents an application under test (CTP). 

The resource suite is configured to match any scheme, port, and path at host "emdemo." As a result, any tests generated for emdemo will include tools references in the Test Reference Suite. Additionally, tests will be configured with authentication and header settings configured in the HTTP Authentication and HTTP Header tools, respectively.  

Each Resource Template points to a specific path under emdemo. Any tests generated for the paths specified in each Resource Template will include configurations from the chained tools. For example, "Resource 3" is configured to match the "/em/virtualassets/manage" path. As a result, tests that contain this path will include a JSON Assertor configured to validate asset configurations that are returned in the response. 

  • No labels

Need assistance? Visit our support page