A Brief Introduction into Apache Kafka
...
To setup Kafka-Messaging we have to create back-end configuration part (JSON format) and Eagle Streams (Inbound/Outbound) with MC2KAFKA delivery method.
Make sure that your topic already exists in Kafka instance or ask admin to do it for you.
...
Message Stream Title | the stream name |
---|---|
Instance | mc2 instance with its AppID (=10000) |
Delivery Method | MC2KAFKA |
Stream Direction | Inbound |
Delivery Format | choose from list (e.g.: XML) |
Ruleset File | your working ruleset file to process data being read from topic into :IN_MESSAGE: variable; For EJM workflows use: eagle_ml-2-0_cm/mc2/ejm/workflow/workflow_listener.rsf For simple extracts use: eagle_ml-2-0_cm/out/xml/extract_service.rsf |
Kafka Instance Name | KafkaName reference name from connection.json file (required) |
Topic Name | Your Kafka Topic name (required) |
MC events log | MC log level |
Consumers count | The number of consumers that connect to kafka server (default: 2) |
Auto Offset Reset | What to do when there is no initial offset (uniquely identifies each record) or if an offset is out of range: earliest: automatically reset the offset to the earliest offset latest: automatically reset the offset to the latest offset fail: throw exception to the consumer. (default: earliest) |
Batch records size | Max records to be processed at the same time (default: 1) |
Data Encryption | No (default) / SSL |
URI parameters | leave it empty (advanced configuration) |
SSL TrustStore Password SSL KeyStore Password SSL Key Password | leave it empty (support of SSL (Secure Sockets Layer) and its successor, TLS (Transport Layer Security), which are protocols for establishing authenticated and encrypted connections) |
...
As you already know Kafka stores all messages in abstract items called Topics. JMS as you may know has the same abstraction but they have difference, but before describe it we have to understand some other concepts. Lets quick review Eagle JMS feature. We believe it will help you to make decision what to use in the end for your needs - JMS or Kafka Messaging.
...