Overview

CTP's "create from traffic" functionality allows rapid creation of parameterized virtual assets from recorded HTTP/S, JMS, or MQ traffic—reducing the time and effort involved in defining complex responses. It also facilitates the editing and extension of data so data captured in one environment (e.g., functional testing) can be easily reused in others (e.g., performance testing, development, security testing, etc.).

For each pass through the wizard, CTP creates one virtual asset (.pva) file with Message Responders that are automatically parameterized with values from a repository that's created on a Parasoft Data Repository server. Data values can then be manipulated independently of the virtual asset. 

CTP's create from traffic functionality is designed to process JSON or XML format messages. To create virtual assets from traffic files using other message formats (e.g., EDI, plain text, etc.), use the corresponding "Generate Parameterized Messages" wizard on Virtualize desktop.

Video Tutorial: Creating a Virtual Asset from Recorded Traffic

Learn how to create virtual assets from recorded traffic in this tutorial.

Prerequisites

Creating virtual assets from traffic requires:

• Virtualize 9.10 or higher connected to CTP (see Integrating Virtualize Server and/or SOAtest Server with CTP for details).
• A supported traffic file stored under the recorded_traffic folder on the Virtualize server where you want to create the virtual assets. Acceptable formats include:
  • Logs produced by Virtualize message proxies or recording proxies (see Configuring Message Proxies for details).
  • Message traces or logs captured at the network level using network sniffing tools.
  • HTTP logs obtained by having your application log its traffic.
• A Parasoft Data Repository Server running and connected to CTP (see Test Data Assistant Setup for details).

Creating Virtual Assets 

To create virtual assets from traffic:

1. In the left pane, select the server or folder where you want the new virtual asset added.
2. Choose Create Virtual Asset from the page-level action menu.
   
3. (Optional) Modify the name of the newly-created virtual asset.
4. Set Create to From Traffic.

5. If you want to apply the settings saved in a template file (stored in the Virtualize Server's traffic_templates folder), check Apply a template file, and then select the template file that you want to apply.  If you apply a template, settings in subsequent wizard pages will be pre-populated from the settings. You can adjust them as needed without impacting the template. You can also capture the modified settings in a new template.  
   

6. Under Repository server, select the Data Repository server where you want the associated data stored.
7. Under Repository name, select or enter the name of the repository where you want the associated data stored, then click Next.
   
   
8. In the next wizard page, review the recommended grouping strategy, adjust it if desired, then click Next. For details about the available grouping strategies, see Understanding the Grouping Strategies. Here is a quick overview of the options:
   • HTTP method: HTTP methods will determine whether or not the message is processed.  
   • URL paths: URL paths will determine whether or not the message is processed. 
   • URL parameters: URL parameters will determine whether or not the message is processed. 
   • Request body: XPaths will be used to determine whether or not the message is processed.
     
     
9. In the following wizard page, review the results of the message grouping.
   
   • The total number of request/response pairs processed, along with the number of invalid pairs, are shown at the top.
   • The types of columns shown depend on the grouping strategy applied. 
   • Each table row represents the criteria for defining a group. One group will be generated for each table row. One responder will be generated for each group. 
     
     

10. If you want to modify the automatically-configured request correlations and/or specify the correlated WSDL/schema, expand the associated table row, then configure the correlations and/or WSDL/schema. See Customizing Data Source Correlations for details on how to configure correlations.

    
    

    Why specify a WSDL/schema?

    Specifying a WSDL or schema ensures that data definitions are tailored to the WSDL/schema rather than inferred by the traffic. Moreover, when you're working with this virtual asset in Virtualize desktop, you also gain richer editing functionality and Change Advisor capabilities.

11. If you want to save the settings you used in this wizard, check Export a reusable template and indicate the desired file name. By default, traffic will be recorded in a file named %n_%d_%t.traffictemplate within the current server's recorded_traffic folder (this will be created if it does not exist). When specifying the file name, you can use variables such as %d (current date) %t (current time), %n (name), and %u (unique time-based id).  

12. Click Create Asset.

The following items will be created and configured:

• A Message Responder with parameterized elements as well as preconfigured responder correlation and data source correlation.
   
• (For new data repositories) A new Data Repository with applicable data sets and record types. One data set will be added per message "group" identified by analyzing the traffic.
   
• (For existing data repositories) New data sets and record types will be added to the existing repository.
   
• A repository data source will be added for each added data set and the Message Responder will be configured to use this data source.

Understanding the Grouping Strategies

When you create virtual assets from traffic, the associated Virtualize server groups request/response pairs by operation/type. The grouping method is influenced by both a) Virtualize’s analysis of the given traffic and b) the grouping criteria that you choose to apply. Recommended grouping strategies are automatically selected; these recommendations can be adjusted and overridden as you see fit. 

Virtualize takes the approach of analyzing the traffic in two stages: first, it recommends what grouping strategies should be enabled, and then it uses the selected strategies to divide the messages into groups. When creating PVAs, one responder is generated for each group.

The recommended grouping strategies are selected based on the following heuristics:

• Grouping based on Request Body is selected in the following cases:
  • SOAP Traffic
  • Non-XML message formats
  • Non-HTTP traffic
• Grouping based on HTTP Method and URL Paths is selected in other cases—except that HTTP method is only selected if there are variations in the methods within the provided traffic.

When grouping, Virtualize:

• Favors specific over general. Virtualize generates as specific of an expression as possible matching all messages within that group only (not matching messages in other groups). This ensures that generated responders correlate correctly for the same traffic.
• Follows the grouping order (HTTP method, URL path, URL parameter and then request body content). For example, if URL paths are enough to differentiate between groups, Virtualize does not create Request Body correlations and so on for each of these grouping criteria steps.

The following grouping strategies can be applied:

• HTTP Methods
• URL Paths
• URL Parameters
• Request Body

HTTP Methods

Split the generated Message Responders based on the HTTP method (POST, GET, PUT, etc.). When this strategy is enabled, each unique HTTP method results in a Message Responder getting generated.

For example, if the traffic contains 10 POST messages and 15 GET messages (regardless of their order), Virtualize will create 2 responders in the generated PVA: one for the POST messages and one for the GET messages (assuming that no other grouping strategies have triggered further splits in the groups).

The resulting groups can be customized so that a group can capture messages of more than one HTTP method. For example, if both POST and PUT are enabled within a group, then messages using the POST method and messages using the PUT method will match the group and get assigned to it. In other words, the responder that is generated based on that group will correlate with both POST and PUT messages (it will accept either one).

URL Paths

Split the generated Message Responders based on the uniqueness of URL paths. Each unique HTTP URL path results in a Message Responder getting generated. The configuration for grouping by HTTP URL Paths essentially defines that uniqueness.

When groups are first generated based on this strategy, the paths are analyzed to produce a set of path expressions that would best characterize the type of messages. That characterization uses a path analysis algorithm that is inherently subjective. It is designed to account for the most common patterns in REST services, but it is possible that the generated path criteria need to be adjusted for optimal results.

For example, if the traffic contains the following HTTP URL paths (omitting HTTP methods, URL parameters and other factors for brevity)

/service/order/v1/1/summary
/service/order/v1/2/summary
/service/order/v1/2/summary
/service/customer/v1/a/contact
/service/customer/v1/b/contact
/Foo
/Foo

Virtualize will create three responders in the generated PVA:

• Responder 1: for the summary paths (3 messages)
• Responder 2: the contact path (2 messages)
• Responder 3: for the Foo paths (2 messages)

These responders will be generated using Ant-style wildcards, where * matches zero or more characters  and ** matches zero or more directories.

URL Parameters

Split the generated Message Responders based on the presence of HTTP URL parameters. Each set of one or more URL parameters results in a Message Responder getting generated.

For example, if the traffic contains the following URL parameters

?oid=1&category=women
?oid=2&category=women
?oid=1&category=babies
?ssn=1234567890&state=CA&category=silver
?ssn=1234567891&state=CA&category=gold

Virtualize will split the responders in the generated PVA based on the presence of oid, ssn and state parameters as follows (each responder to have the following responder correlation criteria)

• Responder 1: the URL parameter oid is present (3 messages)
• Responder 2: the URL parameters ssn and state are present (2 messages)

Request Body

Split the generated responders based on a list of XPath expressions that are evaluated on the request content. Each XPath expression represents a group and XML Message correlation criteria.

When this strategy is enabled, SOAP messages are (by default) grouped based on first element name under Body. For non-SOAP XML content, messages are grouped based on root element name.

Note that when a custom message format (or a built-in non-XML one) is used, the XPath expressions are based on the converted XML format of the requests

Customizing Data Source Correlations

Responder correlations (described above) determine which messages a Message Responder tool accepts and processes. Various messages sent to the virtual asset URL are routed to specific Message Responder tools (each of which handles different operations) based on the settings here. For example, one Message Responder tool might respond to customer registration messages, another might respond to payment messages, and another might function as a default "catch all" that is used when none of the others match. 

After the automatically-configured responder correlations are processed,values in the incoming message are compared (matched literally or with an expression) to values in a data source and the responder uses the matching row data to create a response message. If no data source row is found matching the specified incoming message value, then (with the default failover settings) Virtualize will continue searching the responder suite for a responder with matching responder correlation.

Virtualize automatically populates the following data source correlations (if applicable) and allows you to customize the initial correlations:

• Request Body Correlations
• Request URL Path Correlations
• Request URL Parameter Correlations
  
  

If you did not specify a template, the page’s initial state shows the data source correlations that were automatically-generated from the current traffic file.

If you specified a template, the page’s initial state shows the data source correlations defined in the template. In this case, data source correlations won’t be automatically-generated from the current traffic file.

To customize the automatically-configured data source correlations:

1. Clear the Autoconfig box.
2. Expand the row for the responder whose correlations you want to configure.
   
   
   
3. Use the available controls to configure correlations. 

• • Request body correlations can be configured with the XPath builder, which is described in Specifying XPaths, as well as by entering values in the table
  • URL parameter and URL path correlations can be configured by adding rows to the applicable tables.
  • When you’re configuring correlations, note that you can enter the name of a data source column, or select from the list of columns available in the data source associated with this tool (selected in the Data Source column at the top of the configuration area).
    

Request Body Correlations

Virtualize will generate a "name"-based XPath for each operation/group; this will be used to set the Responder Correlation for that operation. For example, if the element name under the SOAP Body is "SubmitOrder", then the XPath expression set to the responder correlation section would look like local-name(/*/*[local-name(.)="Body"]/*)="SubmitOrder".

Note that when the message is not XML, XPaths and parameter selections are applied to the converted XML version of the request message.

For each group of messages belonging to the same operation, requests are mutually compared to determine the parameters that differ among the requests. Virtualize’s auto-configuration from traffic will automatically analyze the differences in the request elements, then use the results of this analysis to generate XPath expressions for the responses available within that operation/group. The goal is to automatically generate a response for each distinct request element. If the traffic is a SOAP Envelope, then structural differences are allowed as long as the messages share the same operation element (this is the first element under the SOAP Body). If the traffic is generic XML, then structural differences are allowed as long as the messages have the same root node.

If you want to override the initial configuration shown in the wizard page, use the available controls to specify the XPath(s) and column name(s) that you want to use. See Specifying XPaths for details on using the XPath selector.

Request URL Parameter Correlations

For request URL parameters, if there is any difference in the URL parameters in the invocations belonging to a message group, such as a different number of parameters, different parameters (names), or different parameter values, then Virtualize’s auto-configuration from traffic will automatically configure correlations based on those differences.

For example, assume the following parameters in the invocations for a specific message group:

countryCode=US&brandCode=HG
countryCode=Uk&brandCode=HG&channelCode=3
countryCode=US
countryCode=UK
brandCode=HG

Based on the differences in the parameters, Virtualize would automatically configure 3 data source correlation rows for this message group: countryCode, brandCode, and channelCode.

If you want to override the initial configuration shown in the wizard page, use the available controls to specify the parameter(s) and column name(s) that you want to use. For example, if you wanted to configure a data source correlation for the productCode parameter, you might use:

Request URL Path Correlations

For URL path, if there is any difference in the URL paths in the invocations belonging to a group, then Virtualize’s auto-configuration from traffic will configure correlations based on those differences.

For example, assume the following paths in the invocations for a specific message group:

/customer/123/account/1920384
/customer/203/account/4922434
/customer/302/account/7349463

Based on the differences in segments 1 and 3 (using 0-based indexing), Virtualize would add 2 data source correlation rows for this message group: one for path index 1, and one for path index 3.

If you want to override the initial configuration shown in the wizard page, use the available controls to specify which path segment to use for correlations. Path segments can be matched with one or more data source column name, then be parameterized with various data source columns. In the dialog that opens, specify the path segment you want to use (you can click the related path segment or enter the desired path index), then specify a name for the data source column.

For example, if you wanted to use path index 2 and match it to the account data source column, you would configure the correlation as follows: