You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

In this section:

Overview

The Parasoft Kafka Listener Extension adds support for the Apache Kafka message format so that Virtualize assets can read a single message from a topic and respond with a single message on another topic. 

Requirements

  • Apache Kafka brokers 0.10.0.0 or later
  • Parasoft Virtualize 2020.2 or later

Installation

This artifact can be installed from the UI or the command line.

UI Installation

  1. Choose Parasoft > Preferences.
  2. In the System Properties preferences page, click Add JARs.
  3. In the file chooser that opens, select com.parasoft.virtualize.listener.kafka-1.0.jar.
  4. Click Apply and restart Virtualize.

Command Line Installation

Add the kafka.jar file to the system.properties.classpath property in your settings properties file. For example:

system.properties.classpath=<path to jar>/kafka-listener-1.0.jar

Usage

After installation, the Kafka Listener option will appear in the Virtual Asset Deployment Settings. The Kafka implementation can be used for all responders except SQL Responders.

  1. Open the virtual asset settings and click the Transport tab.  
  2. Click the Custom tab and configure the listener settings (see Configuration). If your system has more than one custom extension, choose Kafka Lister from the Select implementation drop-down menu.

Configuration

The listener can be configured as a producer, consumer, or both. To configure the listener as a producer only, leave the consumer configuration settings empty. To configure the listener as a consumer only, leave the producer configuration settings empty.  

Connection Settings

BrokersSpecify a comma-separated list of servers where Kafka is running, including the ports.
Security Protocol

Specifies the security protocol. You can specify following values: 

  • PLAINTEXT - Specifies an unauthenticated, non-encrypted channel.
  • SSL - Specifies for an SSL channel. 
  • SASL_PLAINTEXT - Specifies an SASL-authenticated, non-encrypted channel. This option requires additional configuration (see SASL_PLAINTEXT and SASL_SSL).
  • SASL_SSL - Specifies an SASL-authenticated, SSL channel (see SASL_PLAINTEXT and SASL_SSL).

The Transport Layer Security (TLS) settings will be applied when this Security Protocol is set to either SSL or SASL_SSL.

SASL_PLAINTEXT and SASL_SSL

You cannot fully configure the SASL_PLAINTEXT and SASL_SSL security protocols in the UI fields provided. If you want to use these security protocols, you must create a configuration file and adjust the sasl.jaas.config property to include the com.parasoft.soavirt.kafka.shaded package prefix. The following example demonstrates how to configure the package for the plain login module:

com.parasoft.soavirt.kafka.shaded.apache.kafka.common.security.plain.PlainLoginModule 

Specify the configuration file in the Additional Settings File Path field under the Producer Settings or Consumer Settings.

Consumer Settings

Topic

Specify the name of a topic or a comma-separated list of topics to which the consumer should subscribe.

You can specify a partition within the topic with a colon and the integer number for the partition. For example, topic1:0.

Group IdSpecify the group to which the consumer belongs. This property is required when partition is not specified with the topic.
OffsetSpecify the position within a topic/partition to begin reading records.
Max Poll RecordsSpecify the maximum number of records to poll between intervals. Default is 500.
Commit Offsets

Specify a mode for committing offsets. Commits are points in the partition at which the consumer can resume processing records.

  • autocommit: In this mode, Kafka will determine offset commits.
  • lastProcessedMessage: In this mode, the last message processed is set as the commit offset.
  • off: In this mode, no offsets are committed.

Default is autocommit.

Worker ThreadsSpecify the number of worker threads to run in each instance. A minimum of 1 is required. Default is 50.
Additional Settings File Path

Specify any additional Kafka consumer configurations that are not available in the Virtualize UI. The file should conform to a normal Java properties file with key=value on each line. Refer to Kafka documentation for details about consumer configuration properties:

https://kafka.apache.org/0100/documentation.html#consumerconfigs

Producer Settings

TopicSpecify the name of a topic to which the producer will write messages.
Partition

Specify the integer number for the partition in a topic (e.g., 0, 1, 2).

If empty, the record key will be used to generate a partition number.

Record Key SerializerSpecify whether the key should be written/sent as a string or as null. The default value is string.
Record KeySpecify the record key.
Idempotent

Specify true to insure that only one message and no duplicates will be sent to the broker. Default is false.

Request TimeoutSpecify the how long to wait in milliseconds to send and receive an acknowledgment before timing out. Default is 30000.
Additional Settings File Path

Specify any additional Kafka producer configurations that are not available in the Virtualize UI. The file should conform to a normal Java properties file with key=value on each line. Refer to Kafka documentation for details about producer configuration properties:

https://kafka.apache.org/0100/documentation.html#producerconfigs 

Transport Layer Security

TLS Protocol

Defines the Transport Layer Protocol to use when establishing a secure connection.

Default: TLSv1.2 

Security ProviderThe name of the security provider used for TLS connections. Default value is the default security provider of the JVM.
Key Store File LocationDefines an absolute or relative path to the asset (.tst, .pva, .pvn) key store file.
Key Store Type

Defines the type of Key Store File provided. Different Key Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Key Store Type names.

Key Store Password

Defines the password to be used to extract the keys/certificates from the Key Store File.

If left blank, an attempt will be made to extract the keys without a password.

Key PasswordSpecifies the password of the private key in the key store file. This setting is optional for clients.
Trust Store File Location

Defines an absolute or relative path to the asset (.tst, .pva, .pvn) Trust Store file.

Default: Trust All Certificates 

Trust Store Type

Defines the type of Trust Store File provided. Different Trust Store Types can be handled depending on the security providers that have been loaded into the JVM. If additional security providers have been added to the JVM, check the security provider's documentation to gather the available Trust Store Type names.

Trust Store Password

Defines the password to be used to extract the keys/certificates from the Trust Store File. If left blank, an attempt will be made to extract the keys without a password.

Logging 

Log Level

Specify how much information you want logged to the console. The default is 2  (warn).

Third-party Content

This extension includes items that have been sourced from third parties as outlined below.

Additional license details are available in this plugin's licenses folder.

  • No labels