Databricks
An overview of the Databricks integration with Secoda
Last updated
Was this helpful?
An overview of the Databricks integration with Secoda
Last updated
Was this helpful?
There are three steps to get started using Databricks with Secoda:
Create an access token
Connect Databricks to Secoda
Whitelist Secoda IP Address
Note: You can run Databricks on Secoda using AWS or GCP.
In your Databricks console go to the User Settings and generate a new access token. Save the value to be used to connect Databricks to Secoda in the second step.
To have query history and popularity you must provide admin privileges to the token.
Go to https://app.secoda.co/integrations/new and select the Databricks integration.
Enter in the following credentials:
Host: This is the URL of your Databricks workspace, i.e, dbc-dc31b5a2-597d.cloud.databricks.com
Databricks Workspace Id: The numerical id of your workspace, located in the url of your Databricks instance, after the "/?o=". https://<instance_id>.cloud.databricks.com/?o=<workspace_id>.\
Access Token: The access token you generated in the first step
Warehouse ID (Recommended) or Cluster ID: This is the resource what SQL queries will run on. For the optimal experience, use a Databricks serverless SQL warehouse.
To ingest table and column level lineage using Databricks Unity Catalog, a Warehouse ID must be specified.
After entering in the information into Secoda, click "Test Connection". After the connection is successful your can Submit and run the initial extraction.
If your Databricks instance is behind a firewall, you'll have to whitelist Secoda's IP address to allow for metadata extractions.
