...
Create a COPS ticket to create the new topics for tenant:
dna.kafka.client.tenant.prod (80 partitions, 1-week retention)
dna.kafka.client.tenant.prod.schema (1 partition, 1-year retention)
dna.kafka.client.tenant.prod.processed (1 partition, 1-week retention)
...
Request to add CONFLUENT_API_KEY, CONFLUENT_API_SECRET, CONFLUENT_API_KEY_WEST, CONFLUENT_API_SECRET_WEST to vault secrets secrets-tenant-tenantin both eastus2 and westus2 Azure zones for service sd-svc-dna-kafka-client. The prod part of the topic name designates the tenant environment. Can also be dev or stage.
The producer’s secrets should be shared with the producer.
Request the consumer group (client id) dna-kafka-consumer-tenant (replace tenantwith the tenant id) to have access to write and read from all tenant topics.
You can use https://eagleinvsys.atlassian.net/browse/COPS-11087 as a template.
2. Onboard the tenant to Data Rules. See Data Rules Implementation Process
3. If data needs to be loaded to existing tables, please follow https://eagleinvsys.atlassian.net/wiki/spaces/SOLUTIONSDEL/pages/3620963123/Load+to+existing+SnowFlake+DB+tables
34. If the producer will use producer side schema validation, publish the schema to dna.kafka.client.tenant.prod.schema
45. Create a COPS ticket to use the DNA Kafka Client swagger UI to create the Kafka Consumer configuration as:
...
The schemadrift parameter can be added to envConfig section if needed. Please check https://eagleinvsys.atlassian.net/wiki/spaces/SOLUTIONSDEL/pages/3495919683/Schema+drift Schema Drift