dbt

An overview of the dbt integrations with Secoda

Secoda integrates seamlessly with both dbt Cloud and dbt Core, enhancing your ability to manage and visualize data transformations and dependencies within your workspace. This guide details the integration process, the display of dbt metadata in Secoda, and how to utilize dbt features effectively.

You can learn more about the integration setup by clicking into the linked documents.

dbt is a secondary integration that adds additional metadata on to your data warehouse or relational database tables. Before connecting dbt make sure to connect a data warehouse or relational database first. These include Snowflake, BigQuery, Postgres, Redshift, etc.

How dbt metadata appears in Secoda

Once the integration is established:

• If connected, Jobs will appear in the Catalog which you can click into to see Test results and additional metadata for those Tests.

• Data warehouse or relational database tables associated with dbt will display a dbt icon next to their titles.

• A 'Tests' tab will appear for resources where dbt Tests have been run.

• You can view dbt metadata overlaid on the lineage graphs to understand dependencies and transformations better.

• Within the lineage tab, lineage nodes will feature a checkmark icon. Clicking on these icons reveals which dbt Tests have been run and their statuses.

• See the video below of what a correctly functioning integration should look like:

This integration empowers teams to track and verify data transformations directly within the Secoda environment, ensuring transparency and accuracy in data operations. Whether you use dbt Core or dbt Cloud, Secoda facilitates a comprehensive view of your data landscape.

Syncing metadata back to dbt

You can seamlessly sync metadata updates from Secoda directly back to your dbt models. This streamlines workflows and enhances data governance by ensuring that your dbt models stay synced with the latest metadata in Secoda.

Here's how you can set it up:

1. Set up a GitHub Integration: Connect your GitHub account to enable syncing between Secoda and your code repository.

2. Initiate a Metadata Push: Trigger a metadata push via the GitHub sync history tab within Secoda. This process is straightforward and integrates directly with your workflow.

3. Automatic Pull Request Generation: Once a metadata push is initiated, a Pull Request is automatically created in GitHub. This PR includes updates for column and table descriptions, owner details, and tags, ensuring that your dbt models are always up-to-date.

Benefits:

• Keeps your dbt models consistently updated with the latest metadata from Secoda.

• Enhances collaboration by ensuring all team members work with the most current data definitions.

• Reduces manual errors by automating the sync process.