Confluent Cloud
An overview of Confluent Cloud integration with Secoda
Last updated
An overview of Confluent Cloud integration with Secoda
Last updated
To integrate Confluent Cloud with Secoda, we need the following
Generate API Key and API Secret for Secoda to connect to Confluent Cloud and extract metadata
Schema Registry to extract schema metadata for Topics. This is an optional step.
Make a note of the bootstrap server, cluster and environment IDs
Connect Confluent Cloud to Secoda using the API key, API secret, cluster and environment IDs and optionally connect the Schema Registry
Login to Confluent Cloud
Click on the cluster that needs to be connected to Secoda and click on the API Keys section under Cluster Overview on the left
Click on Add Key button to generate a new API key and secret. API key will be the username and API secret will be the password for connecting to Secoda
This is an optional step. If provided, Secoda can extract metadata for the topic schemas.
Login to Confluent Cloud
Select the environment from Environments on the left. This should be the environment that contains the cluster for which API key and secret has been generated here
On the right side, find the section Stream Governance API and copy the content under Endpoint. This is the Schema Registry URL.
Generate Schema Registry API key and make a note of both the API Key and secret. API Key will be the username for Schema Registry and API secret will be the password.
Login to Confluent Cloud
Select the environment from Environments on the left. This should be the environment that contains the cluster for which API key and secret has been generated here
Environment ID can be found on the right side. Make a note of it.
Select the cluster and click on Cluster Settings under Cluster Overview on the left
Make a note of the Cluster ID in the section Identification and Bootstrap server in Endpoints
In Secoda, head to the Integrations page and click New Integration
Find Confluent Cloud under Data Pipeline
Fill in the connection form with all the necessary details
Click Test Connection to make sure everything is correct
Head to the Sync History tab on the side bar and click Run sync