...
./restart starweb kafkaservice
MC2 Extract Service Kafka TLS Configuration
This section describes how to configure MC2 Extract Service to connect to MC2 Kafka Service with the TLS Connections enabled.
First follow the steps in MC2 Kafka Service TLS Configuration section.
By default MC2 Extract Service uses the keystore and truststore files in:
estar/tpe/dynamic/mc2/private/kafka_ssl_client/client.truststore.jks
estar/tpe/dynamic/mc2/private/kafka_ssl_client/client.keystore.jks
Generate a self-signed certificate
cd estar/tpe/dynamic/mc2
mkdir -p private
cd private
mkdir -p kafka_ssl_client
cd kafka_ssl_client
openssl req -x509 -newkey rsa:4096 -keyout keyfile.pem -out certfile.pem -days 365
Write down the password used when creating the certificate. We will refer to this password as client_cert_password
Note: if a CA certificate is available you can save them as keyfile.pem and certfile.pem and follow the instructions below.
2. Create a PEM file with server key and certificate, which will be used to create the Java JKS keystore:
cat keyfile.pem certfile.pem > keycert.pem
3. Create the Kafka client keystore:
openssl pkcs12 -export -in keycert.pem -out client.keystore.pkcs12 -name kafkacli -noiter -nomaciter
Use the same client_cert_passwordwhen prompted.
4. Import the certificate into the truststore:
keytool -import -v -trustcacerts -alias kafkaclient -file certfile.pem -keystore client.truststore.jks
When prompted for a password use the password from step 1 - client_cert_password
Note: If you have a root and intermediate CA certificates import them in the client.truststore.jks
5. Create the JKS keystore:
keytool -v -importkeystore -srckeystore client.keystore.pkcs12 -srcstoretype PKCS12 -destkeystore client.keystore.jks -deststoretype JKS
When prompted for keystore password and source password use the password from step 1 - client_cert_password
6. Import the Kafka Service CA, intermediate certificates and root certificate, if available, to client.trustore.jsk. If using the Kafka Service self-signed certificates the command will be:
keytool -import -v -trustcacerts -alias kafkaservice -file ../kafkaservice/kafka-server-cert.pem -keystore client.truststore.jks
When prompted for the keystore password enter the client_cert_password
8. Add the MC2 Kafka Service certificate, CA certificate and intermediate certificates, if available, to a cafile.pem file. If using the Kafka Service self signed certificate run:
cp ../kafkaservice/kafka-server-cert.pem ./cafile.pem
7. Import the client certificate to the Kafka Service truststore. If using the selfsigned certificate execute the following commands:
cd ../kafkaservice
keytool -import -v -trustcacerts -alias kafkaservice -file ../kafka_ssl_client/certfile.pem -keystore server.truststore.jks
When prompted for the keystore password enter the kafka_key_password
8. Store the client kafka key password in an encrypted eagle storage file. This step can be executed only on one of the app servers - the password file will be created in a shared location:
./cmdmgr.bin credentials --type=aes --action=add --name=mc2ejmkey --user=mc2ejmkey --pass=client_cert_password --file=../estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
./cmdmgr.bin credentials --type=aes --action=add --name=mc2ejmkeystore --user=mc2ejmkeystore --pass=client_cert_password --file=../estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
./cmdmgr.bin credentials --type=aes --action=add --name=mc2ejmtruststore --user=mc2ejmtruststore --pass=client_cert_password --file=../estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
./cmdmgr.bin credentials --type=aes --action=add --name=mc2pykafka --user=mc2pykafka --pass=client_cert_password --file=../estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
Make sure you replace client_cert_password with the correct password in the above commands.
9. On each app server node open cfg/db_connection.ini file and add the following sections:
Code Block | ||
---|---|---|
| ||
[mc2ejmkey]
DBType=
DBName=mc2ejmkey
EstarName=mc2ejmkey
Credfile=estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
[mc2ejmkeystore]
DBType=
DBName=mc2ejmkeystore
EstarName=mc2ejmkeystore
Credfile=estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
[mc2ejmtruststore]
DBType=
DBName=mc2ejmtruststore
EstarName=mc2ejmtruststore
Credfile=estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
[mc2pykafka]
DBType=
DBName=mc2pykafka
EstarName=mc2pykafka
Credfile=estar/tpe/dynamic/mc2/private/kafka_ssl_client/epasswd
|
10. On one of the app servers edit the estar/tpe/dynamic/mc2/cfg/extractservice.yml
file - this file is on a shared location and will affect all MC2 Extract Service instances. Create this file if it does not exist. The file should have the following parameters set at the beginning of the file:
Code Block | ||
---|---|---|
| ||
eagle.kafka.camel.defaultCfg.connectionParameters.securityProtocol: SSL |
In the extractservice-lb section the following parameter should be set:
Code Block | ||
---|---|---|
| ||
---
spring:
profiles: extractservice-lb
application:
name: extractservice-lb
eagle.pyservice.environment:
SECURE_KAFKASERVICE: 1 |
11. Restart extract service on all nodes:
cd eaglemgr
./restart starweb extractservicelb extractserviceworker