This document describes how to enable TLS for communication between Kafka nodes in MC2 Kafka Service cluster and also enabling the TLS communication between MC2 Extract Service, Pyruleservice and MC2 Kafka Service.
MC2 Kafka Service TLS Configuration
This section describes how to enable the TLS Configuration for MC2 Kafka Service. MC2 Kafka Service default configuration expects the keystore and truststore in the following files:
estar/tpe/dynamic/mc2/private/kafkaservice/server.keystore.jks
estar/tpe/dynamic/mc2/private/kafkaservice/server.truststore.jks
Generating the Kafka Service Key and Certificate
This section explains how to generate a self-signed key and certificate for MC2 Kafka Service. If a certificate and key signed by a Certicate Authority are available follow the Java keytool documentation to add the key and certificate to estar/tpe/dynamic/mc2/private/kafkaservice/server.keystore.jks. You can use same key and certificate for both Kafka Service and MC2 Service/Pyruleservice.
Generate a self-signed certificate
cd estar/tpe/dynamic/mc2
mkdir -p private
cd private
mkdir -p kafkaservice
cd kafkaservice
openssl req -x509 -newkey rsa:4096 -keyout kafka-server-key.pem -out kafka-server-cert.pem -days 365 -nodes
Note: if a CA certificate is available you can save them as kafka-server-key.pem and kafka-server-cert.pem and follow the instructions below.
2. Create a PEM file with server key and certificate, which will be used to create the Java JKS keystore:
cat kafka-server-key.pem kafka-server-cert.pem > kafka-server-keycert.pem
3. Create the Kafka Server keystore:
openssl pkcs12 -export -in kafka-server-keycert.pem -out server.keystore.pkcs12 -name kafkasrv -noiter -nomaciter
Save the password you enter when prompted. We’ll refer to it later as kafka_key_password
4. Import the certificate into the truststore:
keytool -import -v -trustcacerts -alias kafkaserver -file kafka-server-cert.pem -keystore server.truststore.jks
When prompted for a password use the password from step 3 - kafka_key_password
Note: If you have a root and intermediate CA certificates import them in the server.truststore.jks
5. Create the JKS keystore:
keytool -v -importkeystore -srckeystore server.keystore.pkcs12 -srcstoretype PKCS12 -destkeystore server.keystore.jks -deststoretype JKS
When prompted for keystore password and source password use the password from step 3 - kafka_key_password
6. Store the kafka key password in an encrypted eagle storage file. This step can be executed only on one of the app servers - the password file will be created in a shared location:
./cmdmgr.bin credentials --type=aes --action=add --name=kafkaservice_key --user=kafkaservice_key --pass=kafka_key_password --file=../estar/tpe/dynamic/mc2/kafkaservice/epasswd
./cmdmgr.bin credentials --type=aes --action=add --name=kafkaservice_keystore --user=kafkaservice_keystore --pass=kafka_key_password --file=../estar/tpe/dynamic/mc2/kafkaservice/epasswd
./cmdmgr.bin credentials --type=aes --action=add --name=kafkaservice_truststore --user=kafkaservice_truststore --pass=kafka_key_password --file=../estar/tpe/dynamic/mc2/kafkaservice/epasswd
7. On each app server node open cfg/db_connection.ini file and add the following sections:
[kafkaservice_key] DBType= DBName=kafkaservice_key EstarName=kafkaservice_key Credfile=estar/tpe/dynamic/mc2/private/kafkaservice/epasswd [kafkaservice_keystore] DBType= DBName=kafkaservice_keystore EstarName=kafkaservice_keystore Credfile=estar/tpe/dynamic/mc2/private/kafkaservice/epasswd [kafkaservice_truststore] DBType= DBName=kafkaservice_truststore EstarName=kafkaservice_truststore Credfile=estar/tpe/dynamic/mc2/private/kafkaservice/epasswd
8. On each app server node edit the estar/tpe/cfg/kafkaservice/kafka-service.yml - create this file if it does not exist. Add to this file the following parameters:
eagle.enable.ssl: true eagle.kafka.instance.kafkaBrokerConfigs: ssl.client.auth: required
9. Restart MC2 Kafka service:
cd eaglemgr
./restart starweb kafkaservice