Secoda Fields Explained
This page explains all the fields that are used to add metadata to the resources in Secoda.
The fields below are referenced in the JSONL file, the SDK Documentation, and in the API docs.
Base Resource Fields
The following fields apply to all the Resources in Secoda.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
entity_type | Enum
| The type of resource. | "table" | |
databuilder_id | String | A unique and unchanging ID created for the resource. | "secoda.snowflake.customers" | |
parent_databuilder_id | String | A unique and unchanging ID referencing the parent for the resource. | "secoda.snowflake.orders" | |
title | String | The title or name for the resource. | "customer" | |
description | String | The description for the resource. | "This table reflects all the customers in the workspace" | |
definition | String | The mark down documentation for the resource. | "<h2>This is my documentation</h2>" | |
external_updated_at | String (datetime format) | The last time the resource was updated in the source. | "2000-10-31T01:30:00.000-05:00" | |
native_type | String | The type of the resource as it's referred to in the source. | "view" |
For specific resource types, additional fields are available or required, as listed below.
Column Specific Fields
Field Name | Data Type | Description | Required (for Columns) | Example |
---|---|---|---|---|
sort_order | Integer | The order in which the columns should show up. | 4 | |
type | String | The datatype of the column. | "integer" | |
is_pk | Boolean | Defines whether the column is a primary key. | "true" | |
parent_databuild_id | String | A unique and unchanging ID referencing the parent for the resource. For a column, the parent is either another column or a table. | "snowflake.orders.table" |
Table Specific Fields
Field Name | Data Type | Description | Required (for Tables) | Example |
---|---|---|---|---|
schema | String | The schema from which the table is extracted. | "sales" | |
database | String | The database from which the table is extracted. | "sales_db" |
Dashboard Specific Fields
Field Name | Data Type | Description | Required (for Dashboards) | Example |
---|---|---|---|---|
group | String | The group that the dashboard is extracted from. | "analytics" |
Chart Specific Fields
Field Name | Data Type | Description | Required (for Charts) | Example |
---|---|---|---|---|
product | String | The name of the source. | "tableau" | |
parent_databuilder_id | String | A unique and unchanging ID referencing the parent for the resource. For a chart, the parent is a dashboard. | "tableau.analytics.dashboard" |
Lineage Fields
The lineage fields can be used to build a connection between any two resources in Secoda.
Each Lineage object should have a to_identifier
to represent a target resource, and a from_identifier
to represent a source resource. One or both of these resources must be an internal resource (a part of the custom integration that is being built).
The to_identifier
and from_identifier
accept a LineageID object, which is made up of a type, as well as several fields unique to each type. The type and relevant fields are broken down below.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
from_identifier | Object | A LineageID object identifying the type of lineage, and source resource. | "from_identifier": { "type": "external_column", "cluster": "data_warehouse_cluster", "database": "marketing_db", "schema": "public", "table": "campaign_data", "column": "click_through_rate" } | |
to_identifier | Object | A LineageID object identifying the type of lineage, and target resource. This field does not accept | "to_identifier": { "type": "internal_resource", "databuilder_id": "internal.dashboard.654" } |
LineageID Fields
The LineageID object will always consist of a type
field, along with several other fields depending on the type.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
type | Enum
- | The type of resource in the context of lineage. | "internal_resource" |
Internal Resource Fields
If the source or target resource is internal to the integration, the type of the LineageID object should be internal_resource
, and the following field is required in addition to type.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
databuilder_id | String | A unique and unchanging ID referencing the internal resource in Secoda. | "internal.dashboard.123" |
External Table Fields
If the source or target resource is a table external to the integration, the type of the LineageID object should be external_table
, and the following fields are required in addition to type.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
cluster | String | The name of the cluster that the external table belongs to in Secoda. | "sales" | |
database | String | The name of the database that the external table belongs to in Secoda. | "sales_db" | |
schema | String | The name of the schema that the external table belongs to in Secoda. | "staging" | |
table | String | The name of the external table. | "customers" |
External Column Fields
If the source or target resource is a column external to the integration, the type of the LineageID object should be external_column
. In addition to type, all of the fields from the External Table type are required, along with a field for column.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
column | String | The name of the external column. | "customer_id" |
Tables from Query Fields
For the source resource (the LineageID object in the from_identifier
field), the type can also be tables_from_query
. This type requires an SQL query that will be processed to determine what the source resources are.
Field Name | Data Type | Description | Required | Example |
---|---|---|---|---|
sql | String | An SQL query used to determine lineage. The SQL query must return resources that exist in Secoda. | "SELECT * FROM public.users INNER JOIN public.orders ON users.id = orders.user_id" |
Last updated