In this section:
Introduction
The Parasoft Kafka Extensions are custom extensions for SOAtest and Virtualize. The Kafka Extensions package includes a transport extension and a listener extension.
The Kafka Transport Extension adds support for the Apache Kafka transport to applicable messaging client tools in SOAtest. This enables you to take full advantage of SOAtest's rich interface when configuring, sending, and validating messages received over Kafka.
Kafka Transport is often used in conjunction with the Apache Avro extension. For more information about the Avro extension including a basic example for using Kafka with Avro, see Apache Avro extension.
The Kafka Listener Extension adds support for the Apache Kafka so Virtualize assets can read a message from a topic and respond with a message on another topic.
Requirements
- Apache Kafka brokers 0.10.0.0 or later
- Parasoft SOAtest and/or Virtualize 2023.2 or later
Installation
This artifact can be installed from the UI or the command line.
UI Installation
- Go to Parasoft > Preferences and click System Properties.
- Click Add JARs and select the com.parasoft.soavirt.kafka-<VERSION>.jar file.
- Click Apply.
- Restart SOAtest.
Command Line Installation
Add the com.parasoft.soavirt.kafka-<VERSION>.jar file to the system.properties.classpath property in your settings properties file. For example:
system.properties.classpath=<PATH_TO_JAR>/com.parasoft.soavirt.kafka-1.2.0.jar
Kafka Transport
Usage
You can configure message client tools (for example, SOAP Client, EDI Client, and Messaging Client) to use the Kafka Transport by enabling it in the tool's Transport tab. Choose Kafka from the Transport menu and configure the available options.
If your system has more than one custom extension, you will instead need to choose Custom Extension from the Transport menu and Kafka from the Select Implementation menu that appears.
Configuration
The transport can be configured as a producer, consumer, or both. To configure the transport as a producer only, leave the consumer configuration settings empty. To configure the transport as a consumer only, leave the producer configuration settings empty.
Connection Settings
| Brokers | Specify a comma-separated list of servers where Kafka is running, including the ports. |
|---|---|
| Security Protocol | Specifies the security protocol. You can specify following values:
The Transport Layer Security (TLS) settings will be applied when this Security Protocol is set to either SASL_PLAINTEXT and SASL_SSLYou cannot fully configure the
Specify the file in the Additional Settings File Path field under the Producer Settings or Consumer Settings. |
| Log Level | Specify how much information you want logged to the console. The default is |
Producer Settings
| Topic | Specify the name of a topic to which the producer will write messages. |
|---|---|
| Partition | Specify the integer number for the partition in a topic (for example, If empty, the record key will be used to generate a partition number. |
| Record Key Serializer | Specify whether the key should be written/sent as a string, binary, Avro, or as null. Options are The default value is |
| Record Key | Specify the record key. If Record Key Serializer is set to string, record keys will be converted to UTF-8. If Record Key Serializer is set to bytes, record keys starting with 0x will be hex decoded and all others will be converted to UTF-8. If Record Key Serializer is set to avro, record keys will be converted from an Avro message modeled as XML to binary. |
| Idempotent | Specify |
| Request Timeout | Specify the how long to wait in milliseconds to send and receive an acknowledgment before timing out. Default is 30000. |
| Additional Settings File Path | Specify a properties file containing additional settings that may be required to connect to your application. This setting is optional. If this field is empty, SOAtest will use default Kafka settings for any additional settings your application may require that are not available in the UI. |
Consumer Settings
| Topic | Specify the name of a topic or a comma-separated list of topics to which the consumer should subscribe. You can specify a partition within the topic with a colon and the integer number for the partition. For example, |
|---|---|
| Group id | Specify the group to which the consumer belongs. This property is required when partition is not specified with the topic. |
| Offset | Specify the position within a topic/partition to begin reading records. |
| Poll interval | Specify a duration in milliseconds between polling. Default is 100. |
| Max Poll Records | Specify the maximum number of records to poll between intervals. Default is 500. |
| Commit Offsets | Specify a mode for committing offsets. Commits are points in the partition at which the consumer can resume processing records.
Default is |
| Timeout | Specify how many milliseconds the consumer should wait for messages to be published to the subscribed topic/partition before disconnecting. Default is 30000. |
| Additional Settings File Path | Specify a properties file containing additional settings that may be required to connect to your application. This setting is optional. If this field is empty, SOAtest will use default Kafka settings for any additional settings your application may require that are not available in the UI. |
Producer Header Settings
If producer headers are configured, they will be published along with the record message to the Kafka topic. Custom headers are displayed with hexadecimal values in the SOAtest traffic viewer. Up to five headers can be supported.
| Header (1-5) Key | Specify a text string key for the custom header. |
|---|---|
| Header (1-5) Value | Specify a text string for a value as UTF-8 bytes or a hexadecimal string prefixed with 0x for a binary value for the custom header. To specify a "null" value, leave the field empty. To specify an empty string as a value, enter "0x". |
Consumer Header Settings
If consumer headers are configured, SOAtest will wait for a record that matches all of the specified header keys and values. Custom headers are displayed with hexadecimal values in the SOAtest traffic viewer. Up to five headers can be supported.
| Header (1-5) Key | Specify a text string key for the custom header. |
|---|---|
| Header (1-5) Value | Specify a text string for a value as UTF-8 bytes or a hexadecimal string prefixed with 0x for a binary value for the custom header. To specify a "null" value, leave the field empty. To specify an empty string as a value, enter "0x". |
Transport Layer Security Settings
| TLS Protocol | Defines the Transport Layer Protocol to use when establishing a secure connection. Default: |
|---|---|
| Security Provider | The name of the security provider used for TLS connections. Default value is the default security provider of the JVM. |
| Key Store File Location | Defines an absolute or relative path to the asset (.tst, .pva, .pvn) key store file. |
| Key Store Type | Defines the type of Key Store File provided. Different Key Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Key Store Type names. |
| Key Store Password | Defines the password to be used to extract the keys/certificates from the Key Store File. If left blank, an attempt will be made to extract the keys without a password. |
| Key Password | Specifies the password of the private key in the key store file. This setting is optional for clients. |
| Trust Store File Location | Defines an absolute or relative path to the asset (.tst, .pva, .pvn) Trust Store file. Default: |
| Trust Store Type | Defines the type of Trust Store File provided. Different Trust Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Trust Store Type names. |
| Trust Store Password | Defines the password to be used to extract the keys/certificates from the Trust Store File. If left blank, an attempt will be made to extract the keys without a password. |
Connection Management Settings
Keep connection alive | Enable this option to keep the client connection alive and reused for subsequent publishing. |
|---|---|
Close connection after test execution | Enable this option to close the client connection directly after publishing. |
Kafka Listener
Usage
After installation, the Kafka Listener option will appear in the Virtual Asset Deployment Settings. The Kafka implementation can be used for all responders except SQL Responders.
- Open the virtual asset settings and click the Transport tab.
- Click the Custom tab and configure the listener settings (see Configuration). If your system has more than one custom extension, choose Kafka Listener from the Select Implementation menu.
Configuration
The listener must be configured with both a producer and a consumer.
Connection Settings
| Brokers | Specify a comma-separated list of servers where Kafka is running, including the ports. |
|---|---|
| Security Protocol | Specifies the security protocol. You can specify following values:
The Transport Layer Security (TLS) settings will be applied when this Security Protocol is set to either SASL_PLAINTEXT and SASL_SSLYou cannot fully configure the
Specify the configuration file in the Additional Settings File Path field under the Producer Settings or Consumer Settings. |
Consumer Settings
| Topic | Specify the name of a topic or a comma-separated list of topics to which the consumer should subscribe. You can specify a partition within the topic with a colon and the integer number for the partition. For example, |
|---|---|
| Group Id | Specify the group to which the consumer belongs. This property is required when partition is not specified with the topic. |
| Offset | Specify the position within a topic/partition to begin reading records. |
| Max Poll Records | Specify the maximum number of records to poll between intervals. Default is 500. |
| Commit Offsets | Specify a mode for committing offsets. Commits are points in the partition at which the consumer can resume processing records.
Default is |
| Worker Threads | Specify the number of worker threads to run in each instance. A minimum of 1 is required. Default is 50. |
| Additional Settings File Path | Specify any additional Kafka consumer configurations that are not available in the Virtualize UI. The file should conform to a normal Java properties file with https://kafka.apache.org/0100/documentation.html#consumerconfigs |
Producer Settings
| Topic | Specify the name of a topic to which the producer will write messages. |
|---|---|
| Partition | Specify the integer number for the partition in a topic (for example, If empty, the record key will be used to generate a partition number. |
| Record Key Serializer | Specify whether the key should be written/sent as a string, binary, Avro, or as null. Options are The default value is |
| Record Key | Specify the record key. If Record Key Serializer is set to string, record keys will be converted to UTF-8. If Record Key Serializer is set to bytes, record keys starting with 0x will be hex decoded and all others will be converted to UTF-8. If Record Key Serializer is set to avro, record keys will be converted from an Avro message modeled as XML to binary. |
| Idempotent | Specify |
| Request Timeout | Specify the how long to wait in milliseconds to send and receive an acknowledgment before timing out. Default is 30000. |
| Additional Settings File Path | Specify any additional Kafka producer configurations that are not available in the Virtualize UI. The file should conform to a normal Java properties file with https://kafka.apache.org/0100/documentation.html#producerconfigs |
Transport Layer Security
| TLS Protocol | Defines the Transport Layer Protocol to use when establishing a secure connection. Default is |
|---|---|
| Security Provider | The name of the security provider used for TLS connections. Default value is the default security provider of the JVM. |
| Key Store File Location | Defines an absolute or relative path to the asset (.tst, .pva, .pvn) key store file. |
| Key Store Type | Defines the type of Key Store File provided. Different Key Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Key Store Type names. |
| Key Store Password | Defines the password to be used to extract the keys/certificates from the Key Store File. If left blank, an attempt will be made to extract the keys without a password. |
| Key Password | Specifies the password of the private key in the key store file. This setting is optional for clients. |
| Trust Store File Location | Defines an absolute or relative path to the asset (.tst, .pva, .pvn) Trust Store file. Default is |
| Trust Store Type | Defines the type of Trust Store File provided. Different Trust Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Trust Store Type names. |
| Trust Store Password | Defines the password to be used to extract the keys/certificates from the Trust Store File. If left blank, an attempt will be made to extract the keys without a password. |
Logging
Log Level | Specify how much information you want logged to the console. The default is |
|---|
Third-party Content
These extensions include items that have been sourced from third parties as outlined below:
- Apache Kafka clients (Apache License 2.0)
- SLF4J (MIT license)
Additional license details are available in this plugin's licenses folder.
