Lineage | Secoda Docs

Secoda Docs

Get Started

Get Lineage

get

Get a lineage object by its ID.

Authorizations

Path parameters

lineage_idstringRequired

The ID of the lineage entry to retrieve.

Example: 34d983b3-bfe0-4b49-abea-0692c8be6693

Responses

200

application/json

404

Not Found

500

Internal Server Error

get

GET /lineage/manual/{lineage_id} HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Accept: */*

{
  "id": "d935a76c-2125-4c30-9389-ef42fa7e23bb",
  "created_at": "2023-08-16T23:12:53.245493Z",
  "updated_at": "2023-08-16T23:12:53.245507Z",
  "from_entity": "7879886b-9e0e-4d02-8b37-c7d3c496a587",
  "to_entity": "b500ef23-c9b4-4259-a87d-d77b8b588f06",
  "direction": "DOWNSTREAM",
  "is_manual": true
}

Delete Lineage

delete

Delete a lineage object by its ID.

Authorizations

Path parameters

lineage_idstringRequired

The ID of the lineage entry to delete.

Example: 34d983b3-bfe0-4b49-abea-0692c8be6693

Responses

204

No Content

404

Not Found

500

Internal Server Error

delete

DELETE /lineage/manual/{lineage_id} HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Accept: */*

No content

Preview Lineage from Query

get

Preview Lineage from a Query.

Authorizations

Query parameters

sqlstringRequired

The SQL query to analyze for lineage relationships

Example: SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id

dialectstringRequired

SQL dialect of the query (e.g. postgresql, bigquery)

Example: postgresql

table_idstringOptional

ID of target table (optional)

Example: 550e8400-e29b-41d4-a716-446655440000

default_databasestringOptional

Default database name for resolving table references (optional)

Example: production

default_schemastringOptional

Default schema name for resolving table references (optional)

Example: public

Responses

200

application/json

400

Bad Request

404

Provided table ID does not exist

500

Internal Server Error

get

GET /lineage/lineage_from_query HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Accept: */*

{
  "sql": "SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id",
  "source_tables": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "created_at": "2023-10-12T14:59:58.781935Z",
      "updated_at": "2023-10-12T15:06:19.251573Z",
      "title": "customers",
      "schema_name": "public",
      "database_name": "production",
      "native_type": "table",
      "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
      "integration_title": "Production Snowflake",
      "integration_type": "snowflake"
    }
  ],
  "column_lineages": [
    {
      "from_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      },
      "to_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      }
    }
  ],
  "target_table": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "created_at": "2023-10-12T14:59:58.781935Z",
    "updated_at": "2023-10-12T15:06:19.251573Z",
    "title": "customers",
    "schema_name": "public",
    "database_name": "production",
    "native_type": "table",
    "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
    "integration_title": "Production Snowflake",
    "integration_type": "snowflake"
  }
}

List Lineage

get

Get lineage objects with optional filtering by source and destination entity.

Authorizations

Query parameters

from_entity__instringOptional

The source resource.

Example: 7879886b-9e0e-4d02-8b37-c7d3c496a587

to_entity__instringOptional

The destination resource.

Example: b500ef23-c9b4-4259-a87d-d77b8b588f06

Responses

200

List of lineage entries.

application/json

Responseall of

400

Bad Request

500

Internal Server Error

get

GET /lineage/manual HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Accept: */*

{
  "links": {
    "next": "https://api.secoda.co/resource/all/?page=2",
    "previous": "None"
  },
  "meta": {
    "page": 1,
    "previous_page": "None",
    "next_page": 2
  },
  "count": 315,
  "total_pages": 7,
  "results": [
    {
      "id": "d935a76c-2125-4c30-9389-ef42fa7e23bb",
      "created_at": "2023-08-16T23:12:53.245493Z",
      "updated_at": "2023-08-16T23:12:53.245507Z",
      "from_entity": "7879886b-9e0e-4d02-8b37-c7d3c496a587",
      "to_entity": "b500ef23-c9b4-4259-a87d-d77b8b588f06",
      "direction": "DOWNSTREAM",
      "is_manual": true
    }
  ]
}

Create new Lineage

post

Create a new lineage object manually. Only one direction of the relationship needs to be created. The opposite direction of the lineage is automatically generated.

Authorizations

Body

from_entitystringOptional

Indicates the source of the resource data.

Example: 7879886b-9e0e-4d02-8b37-c7d3c496a587

to_entitystringOptional

Points to the final destination of the resouce data through various transformations and entities.

Example: b500ef23-c9b4-4259-a87d-d77b8b588f06

directionstringOptional

Represents the flow of the resource data. Values can be - Upstream Downstream

Example: DOWNSTREAM

Responses

201

Created

application/json

400

Bad Request

500

Internal Server Error

post

POST /lineage/manual HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Content-Type: application/json
Accept: */*
Content-Length: 130

{
  "from_entity": "7879886b-9e0e-4d02-8b37-c7d3c496a587",
  "to_entity": "b500ef23-c9b4-4259-a87d-d77b8b588f06",
  "direction": "DOWNSTREAM"
}

{
  "id": "d935a76c-2125-4c30-9389-ef42fa7e23bb",
  "created_at": "2023-08-16T23:12:53.245493Z",
  "updated_at": "2023-08-16T23:12:53.245507Z",
  "from_entity": "7879886b-9e0e-4d02-8b37-c7d3c496a587",
  "to_entity": "b500ef23-c9b4-4259-a87d-d77b8b588f06",
  "direction": "DOWNSTREAM",
  "is_manual": true
}

Create Lineage from Query

post

Create Lineage from a Query.

Authorizations

Body

sqlstringRequired

The SQL query to analyze for lineage relationships

Example: SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id

dialectstringRequired

SQL dialect of the query (e.g. postgresql, bigquery)

Example: postgresql

table_idstringOptional

ID of target table (optional)

Example: 550e8400-e29b-41d4-a716-446655440000

default_databasestringOptional

Default database name for resolving table references (optional)

Example: production

default_schemastringOptional

Default schema name for resolving table references (optional)

Example: public

Responses

200

application/json

400

Bad Request

404

Provided table ID does not exist

500

Internal Server Error

post

POST /lineage/lineage_from_query HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Content-Type: application/json
Accept: */*
Content-Length: 237

{
  "sql": "SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id",
  "dialect": "postgresql",
  "table_id": "550e8400-e29b-41d4-a716-446655440000",
  "default_database": "production",
  "default_schema": "public"
}

{
  "sql": "SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id",
  "source_tables": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "created_at": "2023-10-12T14:59:58.781935Z",
      "updated_at": "2023-10-12T15:06:19.251573Z",
      "title": "customers",
      "schema_name": "public",
      "database_name": "production",
      "native_type": "table",
      "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
      "integration_title": "Production Snowflake",
      "integration_type": "snowflake"
    }
  ],
  "column_lineages": [
    {
      "from_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      },
      "to_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      }
    }
  ],
  "target_table": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "created_at": "2023-10-12T14:59:58.781935Z",
    "updated_at": "2023-10-12T15:06:19.251573Z",
    "title": "customers",
    "schema_name": "public",
    "database_name": "production",
    "native_type": "table",
    "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
    "integration_title": "Production Snowflake",
    "integration_type": "snowflake"
  }
}

Create Lineage from Query and Removes Existing Upstream Non-Manual Lineage from Target Table

patch

Create Lineage from Query and Removes Existing Upstream Non-Manual Lineage from Target Table.

Authorizations

Body

sqlstringRequired

The SQL query to analyze for lineage relationships

Example: SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id

dialectstringRequired

SQL dialect of the query (e.g. postgresql, bigquery)

Example: postgresql

table_idstringOptional

ID of target table (optional)

Example: 550e8400-e29b-41d4-a716-446655440000

default_databasestringOptional

Default database name for resolving table references (optional)

Example: production

default_schemastringOptional

Default schema name for resolving table references (optional)

Example: public

Responses

200

application/json

400

Bad Request

404

Provided table ID does not exist

500

Internal Server Error

patch

PATCH /lineage/lineage_from_query HTTP/1.1
Host: api.secoda.co
Authorization: Bearer string
Content-Type: application/json
Accept: */*
Content-Length: 237

{
  "sql": "SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id",
  "dialect": "postgresql",
  "table_id": "550e8400-e29b-41d4-a716-446655440000",
  "default_database": "production",
  "default_schema": "public"
}

{
  "sql": "SELECT a.customer_id, b.order_id FROM customers a JOIN orders b ON a.customer_id = b.customer_id",
  "source_tables": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "created_at": "2023-10-12T14:59:58.781935Z",
      "updated_at": "2023-10-12T15:06:19.251573Z",
      "title": "customers",
      "schema_name": "public",
      "database_name": "production",
      "native_type": "table",
      "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
      "integration_title": "Production Snowflake",
      "integration_type": "snowflake"
    }
  ],
  "column_lineages": [
    {
      "from_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      },
      "to_column": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "created_at": "2023-10-12T14:59:58.781935Z",
        "updated_at": "2023-10-12T15:06:19.251573Z",
        "title": "customer_id",
        "entity_type": "column",
        "type": "varchar",
        "schema_name": "public",
        "database_name": "production",
        "native_type": "varchar(255)",
        "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
        "integration_title": "Production Snowflake",
        "integration_type": "snowflake"
      }
    }
  ],
  "target_table": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "created_at": "2023-10-12T14:59:58.781935Z",
    "updated_at": "2023-10-12T15:06:19.251573Z",
    "title": "customers",
    "schema_name": "public",
    "database_name": "production",
    "native_type": "table",
    "integration_id": "337c5803-b484-4128-a24d-dcc55dee2ea2",
    "integration_title": "Production Snowflake",
    "integration_type": "snowflake"
  }
}