On-Premises Kafka

You can create an on-premises Kafka connector to connect to an on-premises Kafka deployment. This enables data sources implemented as Kafka clients to send messages to real-time digital twin instances running within an on-premises ScaleOut Digital Twins deployment.

Connecting to Kafka

When creating an on-premises Kafka connector using the ScaleOut Digital Twins UI, specify the connection properties required for a producer and for a consumer to connect to the local Kafka deployment. The single property “bootstrap.servers” and “group.id” is often sufficient for localhost deployments. If the Kafka deployment requires other authentication properties, include each of them on a seperate line. For example, if the Kafka service was running locally on port 9092 alongside an on-premises ScaleOut Digital Twins deployment, you would specify the following connect.properties:

bootstrap.servers=localhost:9092
group.id=<group_name>

Configuring Topics

Specify each model’s topics as comma-separated list of the form:

<model name>,<topic for replies to data sources>,<topic for sending to instances>

Each list is specified on a separate line.

Note

Topics specified in the configuration must already exist. You can create the topics using the Kafka APIs or via Kafka convenience scripts

Note

By default, Kafka consumers use the configuration property “auto.offset.reset=latest”. If the topic for messaging digital twin instances contains messages that the Kafka connection should read, please specify the configuration property “auto.offset.reset=earliest”. Please read more about this configuration property from Apache Kafka’s documentation.