# Getting started with Secoda Welcome to Secoda! Your data organization's single source of truth for data. Not using Secoda to manage your data documentation yet? [Sign up for free](http://app.secoda.co/) ## How Secoda works Secoda stands for **Se**archable **Co**mpany **Da**ta. Your Secoda workspace is the single source of truth for your organization’s data. Secoda is a data enablement tool that automates the process of tracking and documenting data lineage, enhances data documentation, and improves data discovery. By connecting your data sources to Secoda and setting up data lineage and documentation, you can improve transparency and understandability of your data and make it easier for others to use and trust. The data discovery features offered by Secoda, such as the search, data glossary, and data lineage, can help you and your team find and access the data you need more easily. Inviting your team to join Secoda allows you to collaborate and share data assets more efficiently, improving collaboration and data management within your organization. Overall, using Secoda can help streamline data management, enhance data governance, and improve productivity within your organization. ## Get started by role ### [Admin](/readme/secoda-as-an-admin) As a Secoda workspace admin, you're responsible for setting up your data integrations and inviting new team members. There are other tasks that we recommend [as an admin](/readme/secoda-as-an-admin). ### [Editor](#an-editor) As a Secoda Editor, you are able to view and add context to all of the data resources within your workspace. Check out some resources on getting started [as an editor](/readme/secoda-as-an-editor). ### [Viewer](#a-viewer) As a Viewer in Secoda, you get to view and search all of the published data resources. Check out some docs to help you get started [as a viewer](/readme/secoda-as-a-viewer). ## See Secoda in action Read up on some [customer case studies](https://www.secoda.co/customers) about how our power customers have integrated the product into their workflows. {% hint style="info" %} Good to know: We understand that some integrations and data structures are more complex than others, so if you're having any issues, reach out directly on our [Slack community](https://join.slack.com/t/secodacommunity/shared_invite/zt-mhnu278g-FktKZmZ51SDQtlu3NRAxqg) or {% endhint %} # Secoda as an admin Here are some important Admin tasks to get your workplace up and running. As an Admin in Secoda, your role is crucial in shaping the workspace landscape and ensuring smooth operations across your organization’s data platform. Here’s your guide to leveraging Admin privileges effectively: **Getting Started:** * [ ] [**Introduction to Secoda**](/): Familiarize yourself with the platform’s core functionalities. **Configuration and Integration:** * [ ] **Single Sign-On Setup**: Explore single [sign-on options](/readme/secoda-as-an-admin/sign-in-options) and set them up for your organization. * [ ] **Data Source Integration**: Decide which data sources to connect with Secoda and follow[ the integration guides](/readme/secoda-as-an-admin/connect-your-data) provided. * [ ] **Automations**: Learn how to use [Automations](/features/automations) to streamline data management tasks, and check out how[ other Secoda users are leveraging the features](/features/automations/automations-use-cases). * [ ] **Workspace Customization**: * [ ] Customize the [workspace homepage](/features/homepage) to reflect your organizational needs. * [ ] [Adjust settings and create Teams](/best-practices/best-practices-for-setting-up-your-workspace) to tailor the user experience and workspace functionality. * [ ] Set Resource Visibility: Choose to automatically [publish all resources](/features/publishing) or keep them as drafts until ready for broader viewing. **Optimize Platform Setup and Usage:** * [ ] **Best Practices**: Read our [best practices guides](/best-practices) to ensure you are maximizing the platform’s potential. **User Management**: * [ ] **Invite Teammates**: Encourage collaboration by [inviting teammates](/readme/secoda-as-an-admin/invite-teammates) to the workspace. * [ ] **Manage Roles and Permissions**: Control [access and responsibilities](/user-management) effectively to ensure proper data governance. **Enhancement and Engagement:** * [ ] **Metadata Management**: Start [editing and enriching](/resource-and-metadata-management/add-documentation) metadata to improve [data discoverability](/features/search) and [Secoda AI](/features/ai-assistant)’s accuracy. * [ ] **Documentation and Glossary Terms**: Populate the platform with essential [documents](/features/documents) and [terms](/features/glossary) that provide context to your data resources. * [ ] **Data Quality:** Check out your [Data Quality Score](/features/data-quality-score) and suggestions from Secoda on how you might improve it. * [ ] **User Engagement Strategies**: Explore strategies to boost [user adoption](/readme/secoda-as-an-admin/user-engagement-and-adoption) and engagement across your organization. **Communication and Collaboration:** * [ ] **Slack Integration**: Consider [connecting Secoda to Slack](/extensions/slack-connection/slack-user-guide) for seamless communication and data inquiries. * [ ] **FAQs**: Populate the [Questions](/features/ask-questions-in-secoda) section with FAQs to help users understand common data queries. **Monitoring and Maintenance:** * [ ] **Notification Settings**: Configure [notifications](/features/notifications) to keep updated on important changes and updates. * [ ] **Activity Logs**: Monitor [user activities](/features/activity-log) and data usage to maintain governance and compliance. **Explore Further:** * [ ] **Advanced Features**: Dive deeper into the [advanced features](/features) available for Admins in Secoda. Explore more through our detailed guides and community discussions. # Deployment options Check out the deployment options for setting up Secoda. Secoda offers three main deployment options depending on your organization's needs. ## **Multi-Tenant Cloud** You can select from the following regions for hosting our multi-tenant Cloud version of the app: 1. US (us-east-1) 2. EU (eu-central-1) 3. APAC (ap-southeast-1) With our multi-tenant setup (and most software SaaS), organization's application data is stored on a shared database. All application data is stored within a private VPC and is separated at the software-level using row-level security, global checks, and automated tests to ensure data is securely separated. ## **Single-Tenant Cloud** A single-tenant Cloud set up is an isolated version of the app in it's own VPC in the region of your choice. The following holds true for this option: * All of your organization's application data exists in it's own database that is not shared with any other Secoda customers. * The Secoda team manages all application upgrades and infrastructure changes. * You can select your own custom URL, i.e, . This provides stronger security guarantees, dedicated resources, and can offer lower latency by configuration in a nearby AWS region. ## [**Self-Hosted**](/enterprise/self-hosted-secoda) (On-Premise) There are three options for self-hosting Secoda: 1. [AWS ECS](https://github.com/secoda/terraform-aws-secoda) (recommended) 2. [Docker Compose](https://github.com/secoda/docker-compose) 3. [Kubernetes](https://github.com/secoda/helm) {% hint style="info" %} The self-hosted options require DevOps support from the customer and Secoda which can result in higher costs. {% endhint %} # Sign in options Learn about Secoda's options for logging into the app through single sign on and email There are a variety of ways to sign into Secoda, depending on your existing systems and levels of security. When first accessing the app at app.secoda.co, you'll be shown this screen below. You have the options to sign in with email, SAML, a Microsoft account or a Google account.
## Microsoft SSO, Google SSO and email Continuing with email and password, Microsoft and Google are included in every contract. You don't need to configure anything in order to access Secoda in the following ways: * **Continue with Microsoft:** Choose this option if your organization uses Microsoft for email, as this will allow you to sign in with SSO. The Azure Admin will likely have to authorize Secoda as an allowed application to enable SSO with Microsoft. * **Continue with Google:** Choose this option if your organization uses Google for email, as this will allow you to sign in with SSO * **Continue with email:** Choose this option if you don't have a Microsoft of Google email address ## SAML options If you'd like to fine tune permissions and security, your organization can implement SAML SSO. For SAML, we have created guides for the following identity providers (IdPs): * [Okta SAML SSO](/enterprise/saml/okta-saml) * [Azure Active Directory SAML SSO](/enterprise/saml/microsoft-azure-ad-saml) * [Google Workspace SAML SSO](/enterprise/saml/google-saml) * [OneLogin SAML SSO](/enterprise/saml/onelogin-saml) Enterprise plan subscribers can opt-in on the settings page to use: * [SCIM](/enterprise/saml/scim) * [SAML Attributes](/enterprise/saml/attributes) ## Enforce SSO type Workspace admins have the option to enforce SSO with a specific identity provider (IdP). In Settings > Workspace Settings > Enforce SSO, a workspace admin can enforce Google OAuth, Microsoft OAuth, or SAML. Once chosen, users will not be able to login through email or another IdP. If they attempt to, they will be redirected to the type that you've specified.
# Settings Learn how to access and edit Account and Workspace settings. The Settings page in Secoda allows Admins to configure Account and Workspace-level settings. Editors and Viewers can manage their Profile and Notification settings here, but they won't have access to most Workspace settings. ## Accessing Settings To access Settings, click on the workspace name in the top left of the UI, and select "Settings."

Secoda Settings

## Account Settings * **Profile**: Users can update their profile photo and name. They also have the option to leave the workspace if necessary. * **Notifications**: Choose which notifications to receive and where to receive them. Read more: [Notifications](/features/notifications). ## Workspace Settings The following settings are configurable only by Admins: * **General**: Add a workspace logo, edit the workspace name, view the current version of your workspace, and delete the workspace if needed. * **Catalog:** Set a default view for the catalogs across your workspace. Choose which properties to show, and in what order. * **Security**: Add additional email domains allowed to sign up for the workspace and set up SSO. Read more: [Sign in options](/readme/secoda-as-an-admin/sign-in-options). * **Appearance**: Configure resource pages to be full-width and hide the '?' help and feedback button from viewers and guests. * **Members**: Manage workspace members and groups. Read more: [Invite your teammates](/readme/secoda-as-an-admin/invite-teammates). * **Data**: View potential PII data identified by Secoda and apply the PII tag in bulk. Manage the criteria for PII identification. Read more: [PII identifier](/resource-and-metadata-management/tags/auto-pii-tagging). * **AI**: Manage Secoda AI preferences, including AI governance, tools, and custom instructions. Read more: [Secoda AI](/features/ai-assistant). * **Import & Export**: Bulk import and export metadata. Read more: [Import and export resources](/resource-and-metadata-management/import-and-export-data). * **Publishing**: Configure your publishing settings. Read more [Publishing](/features/publishing). * **Billing**: View billing settings and plan details. * **API**: Access API documentation and view the history of API keys used. Read more: [https://github.com/secoda/gitbook/blob/master/readme/secoda-as-an-admin/broken-reference/README.md](https://github.com/secoda/gitbook/blob/master/readme/secoda-as-an-admin/broken-reference/README.md "mention"). * **Tags**: Create and manage tags to categorize your data resources. Read more: [Tags](/resource-and-metadata-management/tags). * **Audit Log**: View all historical activity in the workspace. Read more: [Publishing](/features/publishing#audit-log). # Connect your data Start documenting and making your data discoverable for your team by connecting your data resources to Secoda. ## Benefits to connecting your data to Secoda 1. Improved data lineage: By connecting your data to Secoda, you can automate the process of tracking and documenting data lineage. This can help you understand where your data comes from, how it has been transformed, and how it is used within your organization. 2. Enhanced data documentation: Connecting your data to Secoda allows you to automatically generate documentation for your data assets, including descriptions, definitions, and metadata. This can help improve the transparency and understandability of your data, making it easier for others to use and trust. 3. Improved data discovery: By connecting your data to Secoda, you can make it easier for others to discover and access your data assets. This can help improve collaboration and ensure that team members have the information they need to make informed decisions. 4. Enhanced data governance: Connecting your data to Secoda allows you to establish clear roles and responsibilities for data management and ensure that all team members follow established data governance policies and procedures. ## How does Secoda integrate into your data? Secoda integrates with your databases, data warehouses and BI tools to fetch metadata, query history and activity from data sources to make it easier to share and document data knowledge with stakeholders. As soon as the metadata and query history have been added to Secoda, they are compiled into a unified metadata store by way of a query parser. By parsing queries, we look at what data is there, where it is, and how it is used. If queries are available, the tables, databases and dashboards will display a data column and table level lineage model, to easily see from one place what the data dependencies are, where the data came from, and where it's being used. ## How to add integrations Adding integrations is simple. 1. Navigate over to the Integrations tab on the side panel and click New Integration. Note: only Workspace Admins have access to this tab. 2. Choose which type of integration you'd like to connect from our long list of products that we support 3. See the documentation on the right panel (or here [Integrations](/integrations)) for specific instructions on setting up. 4. Under "Associated Teams", click which Teams you'd like to have access to all of the metadata in that integration. 5. See the Metadata seamlessly flow into your [Catalog](/features/catalog).
## Current integrations See all of our integrations below. {% content-ref url="/pages/EzDXmZ4uxzeMHOBrM66R" %} [Integrations](/integrations) {% endcontent-ref %} {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](https://app.secoda.co/) 👈 {% endhint %} # Define service accounts To make your popularity calculations more accurate, you'll want to mark emails as Service Accounts. ## Overview of service accounts Service accounts are emails associated with the integration that **do not** exist as Members in the Secoda Workspace. These emails can fall under two categories: actual humans (your coworkers) who are not yet members in Secoda yet; or bot accounts associated with the integration. {% hint style="info" %} Ideally, you'd want to have the human-related emails checked off so that they are calculated towards the popularity of a resource. However, for bot accounts you may not want them included because it would alter the accuracy of the popularity metric. {% endhint %} To define which emails should be counted, please navigate to the Popularity tab of the integration settings. You'll see a list of emails captured from the integration that do not exist as Members in the Secoda Workspace. You'll need to select all the emails that you'd like to count towards Popularity, and uncheck those that should not be counted. For example, if `bob@company.com` runs a query against your Redshift instance, but is not a member of the Secoda workspace, `bob@company.com` will be considered a Service Account in Secoda and will show up in the Popularity tab of the integrations settings as seen below.
If an email associated with a Service Account signs into Secoda, it will be converted from a Service Account to a Secoda user and show up in the Members tab of the workspace settings. To learn more about how Popularity is calculated, please see the documentation [here](https://docs.secoda.co/faq#how-is-the-popularity-calculated). {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](https://app.secoda.co/) 👈 {% endhint %} # Choose which schemas to extract You might not need all of your schemas within Secoda. This page will walk you through how to hide these from viewers and editors on your team. ## How to hide schemas from Secoda After connecting your data resources to Secoda, you can select which schemas, or groups, you'd like to see in Secoda. To do this, start by going to the **Integrations** page on the side bar. From here, select the Integration and click on the **Schema** or **Groups** tab. Note: Schema is for databases and warehouses, while Groups is for data visualization tools. On the **Schema/Group** page, you'll find all of the schema or groups that Secoda has pulled from your integration. Select the ones that you would like to be accessible on Secoda. The unchecked ones will not be extracted.
After you make any changes to your selections, you must click **Submit** to have the changes apply in your next extraction. If you have new schemas or groups in the source that are not reflected in Secoda, you must click **Refresh** to have them brought in. Refresh will not impact your current selections, but will bring in any new schemas or groups as selected by default. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](https://app.secoda.co/) 👈 {% endhint %} # Customize the workspace As an admin, you can change the appearance of your Secoda workspace to suit your team. ## **How to customize Secoda's appearance** There are various personalization options for Secoda users. These personalization options are displayed to viewers when they first come into Secoda. {% hint style="info" %} As an Admin, you're able to "lock" some of these personalizations. If you don't lock these, then Editors and Viewers can personalize their workspace as they'd like. {% endhint %} ### **Edit workspace logo and name** * Go to **Settings** from the sidebar and click into Workspace Settings * Change the Logo to your organization's logo and change the name, if applicable
### Allowed Domains If using multiple domains for your organization, you can add these domains to the "Allowed domains" section in the Security settings. This will allow members with email IDs ending in those domains to sign up to the workspace directly.

Security Settings

### Edit the Homepage You can change your personal homepage background, and Admins can edit the Team homepages as well. Simply click into **Home** and click Edit in the top right. You can also pin resources to the homepage to point users in the right direction when they first join your workspace. To learn more about the home page, see the Custom Homepage link below! {% content-ref url="/pages/8Lf64SudXpg75vLiSM8W" %} [Homepage](/features/homepage) {% endcontent-ref %} {% hint style="info" %} Not using Secoda to manage your data knowledge yet? Sign up for free [here](https://app.secoda.co) 👈 {% endhint %} # Populate Questions with FAQs Use our Questions feature to add frequently asked questions that might help your members navigating the workspace. Before rolling out the tool to a larger group, we recommend populating the Questions section with some FAQs for your users. This will help users see the value in how Questions can save them time. Read up on Secoda's Questions feature here: {% content-ref url="/pages/HULyNz3XAdIdtpsA2VKE" %} [Questions](/features/ask-questions-in-secoda) {% endcontent-ref %} Some example questions that we have seen from customers: * Where can I find this data? * How is our team's data organized? * What is the number of new customers from last month? * What is the difference between how we use these two datasets? # Invite your teammates Secoda is a tool built for collaboration. This section will go over how to add members of your team, how to control their permissions and roles, and how to group them. ## **Benefits** to inviting teammates to Secoda * **Enhanced Collaboration:** Streamlines data access and collaboration, reducing confusion and improving team efficiency. * **Centralized Data Management:** Ensures all team members access up-to-date data and documentation, minimizing errors. * **Improved Security:** Allows precise control over data access, safeguarding sensitive information. * **Robust Data Governance:** Facilitates the implementation of data governance policies by defining clear roles and responsibilities. ## **How to Invite Teammates** * **Initial Setup:** As a workplace Admin, you're prompted to invite colleagues when you first access Secoda. * **Inviting Users:** Navigate to the [Settings](/readme/secoda-as-an-admin/settings), then select '**Members**' to access the invitation controls. * **Sending Invites:** Use the 'Invite members' button for bulk invitations or the Invite link for individual invites. * **Assigning Roles:** Manage user roles directly from the Members page, adjusting access as necessary. For more on what permissions certain roles have, check out [**Roles.**](/user-management/roles) {% hint style="warning" %} Only users with a business domain name can be invited to the workspace. If using multiple business domains, you can add them to the [allowed domains list](/readme/secoda-as-an-admin/customize-the-workspace#allowed-domains) in the Security settings. {% endhint %}
## **Creating Groups and Teams** Establish groups and teams to manage user roles and access effectively. Learn more about [User management](/user-management). ## Deactivating Members To Deactivate a member from your workspace, click **Deactivate** beside their name. The user's name will still appear on resources that they had previously owned, but will have a strike through their name. This can be helpful context to know who had the historical knowledge of a dataset. ![](https://secoda-public-media-assets.s3.amazonaws.com/06102896-2f76-4c6c-b45b-454c39ffb8f1.png) ## Video resource {% embed url="" %} {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](https://app.secoda.co/) 👈 {% endhint %} # Joining and navigating between multiple workspaces You are able to join more than one Secoda workspace with the same email. Users can be part of multiple Secoda workspaces, using the same email. This might be relevant if your team is hoping to have different production and staging workspaces, or if you have multiple clients to extend the product to. Follow the instructions on [how to Invite Teammates to Secoda](/readme/secoda-as-an-admin/invite-teammates) to invite users to the desired workspace. If you are a user of more than one workspaces, you will be able to navigate between them by clicking on the workspace logo in the top left corner of your screen as demonstrated below: {% embed url="" %} # Onboard new users Learn some best practices on rolling out Secoda to new users. ## **Setting Up for Success** Once your data is connected and initial documentation is in place, think about the primary users of Secoda. This guide helps you determine the initial rollout group and outlines steps for effective onboarding. ## **Organizing the Workspace** Before adding new users, it's crucial to organize your workspace by building out Teams. This customization ensures a tailored user experience. For guidance, explore our documentation on [Setting up your workspace](/best-practices/best-practices-for-setting-up-your-workspace). ## **Choosing Your Rollout Group** The approach varies by organization. While some may want universal access, others prefer starting with a smaller, more controlled group. Consider starting with a focus group of knowledgeable individuals who can provide valuable feedback and help refine documentation. #### Questions to Identify Target Users: * Who is the most knowledgable about the different data across your organization? * Whose workflows could be improved and automated with a tool like Secoda? These insights can help form a core group of champion editors within Secoda. ## Best Practices for Onboarding There are many different ways that our users go about onboarding their teammates into Secoda. Here are some steps that we suggest as best practices. * [ ] **Outline Internal Policies:** Before onboarding new editors, outline policies so that users know their responsibilities while working in the tool. Tip: These can be documented within Secoda in an "Onboarding" [Collection](/features/collections-1) so that they can always reference them. Some examples: * What does Ownership mean (i.e. Owners must stay up to date on documenting data; Owners must notify other users of any changes) * Documentation guidelines and expectations (i.e. What information *must* be included in definitions; How do you use certain Tags) * [ ] **Introduction:** Use our [Onboarding email templates](/readme/secoda-as-an-admin/onboarding-new-users/onboarding-email-templates) to introduce new users to Secoda. * [ ] **Training Sessions:** Organize training tailored to different user roles—Viewers and Editors. * [ ] Check out our [Training session guide](/readme/secoda-as-an-admin/onboarding-new-users/training-session-guide) for tips on what to cover. * [ ] Share relevant pre-reads ahead of the session to prepare users for an engaging discussion:[Secoda as a viewer](/readme/secoda-as-a-viewer) [Secoda as an editor](/readme/secoda-as-an-editor). * [ ] **Workspace Invitation:** Add users to designated [Groups](/user-management/groups) and [Teams](/user-management/teams). If necessary, create new ones. * [ ] **Follow-up:** Send a recap email with next steps and useful links, such as the [Introduction to Secoda guide](https://secoda.notion.site/Secoda-Intro-Guide-277512fb0c224b8a920fa0b099a26810) and answers to session queries. ### Encourage Engagement * [ ] **Slack** **Channel:** If you have a dedicated Secoda or data-related Slack channel, invite and encourage users to ask questions and share feedback. * [ ] Make sure to share this feedback with our team as we are always looking for ways to improve the product! * [ ] **User Engagement:** Check our guide on [User engagement and adoption](/readme/secoda-as-an-admin/user-engagement-and-adoption) for tips to excite users about Secoda. ## **Workshops to Enhance Documentation** Once you've onboarded your initial rollout group, you might find it helpful to host workshop sessions to get the documentation flowing. Here are some ideas you might consider: * **Teams Workshops:** Regular sessions can help teams decide how to structure their resources, assign ownership, and adopt [Custom tags](/resource-and-metadata-management/tags/custom-tags). * **Dictionary Workshop:** Compile a list of frequently used terms to populate the Dictionary. * **Documentation Days:** Establish dedicated workshop times for users to update and refine existing documentation. By following these steps, you can ensure a smooth and effective rollout of Secoda, enhancing data governance and collaboration across your organization. # Onboarding email templates Below you’ll find Secoda’s suggested email templates for various stakeholders. Feel free to use the following templates as you begin onboarding users onto Secoda. We've separated these by Role in Secoda, and have left space for you to insert your workspace-specific details (in red). ### Viewers in Secoda Typically end / business users, data *consumers* > Hi name, > > We’ve recently implemented a new data management tool into our data stack, called Secoda. Secoda stands for **Se**archable **Co**mpany **Da**ta and will enable you to discover the assets that you need in order to make data-driven decisions. With this platform, you’ll be able to:\\ > > * Search in plain language to find information about the data in our organization > * Ask Questions to the Data team > * Explore the end-to-end Lineage of our data stack > * See a list of verified Dictionary terms and Metrics that drive our company KPIs > * ...and much more! > > We’ve set you up with a Viewer account, which you can access at this link (insert link to your workspace). Add relevant context here re: Team they will be in, data assets they should be aware of, which integrations you’ve set up that they can explore etc. > > You can read more about the product and its capabilities [here](/readme/secoda-as-a-viewer/introduction-guide). ### Editors in Secoda Typically Data Analysts, Project/Product Managers, Engineers, data *producers* > Hi name, > > We’ve recently implemented a new data management tool into our data stack, called Secoda. Secoda stands for **Se**archable **Co**mpany **Da**ta and will be the one-stop-shop for all data needs including: searching, cataloging, viewing lineage, data quality monitoring, and managing data governance. With this platform, you’ll be able to: > > * Find verified resources to do your own discovery and analysis > * Automatically document data assets that you own > * Explore the end-to-end Lineage of our data stack and identify breaking changes > * Configure essential Monitors to be alerted about changes to your data > > We’ve set you up with an Editor account, which you can access at this link (insert link to your workspace). Add relevant context here re: Team they will be in, data assets they will own and should add documentation to, any documentation standards, which integrations you’ve set up for them etc.\\ > > You can read more about the product and its capabilities [at this searchable site](https://docs.secoda.co/) and see an editor's checklist [here](/readme/secoda-as-an-editor).\\ # Onboarding Homepage template Create a Document explaining to new users what Secoda is and how they can use it Use this guide to create an introduction Document within Secoda for new users of the tool. Depending on how you have set up your workspace with Teams, this might look different. ## Steps to create an Onboarding Homepage If you've created a default Public team that all users have access to, like the **General** default team, or another Team that is public to all users: 1. Create a Document in that Team and title it something along the lines of "Introduction to Secoda" or "What is Secoda?" 2. Add content that you'd want new users of Secoda to know when first accessing the tool. We've shared some ideas [below](#template). 3. Pin that Document to the General Team's Homepage by going to General Team > Home > Edit > Add Notepad > Write up a blurb similar to what's below in green and use the @ symbol to link the Document you've just created.
If you're only using **Private** Teams to organize your workspace, you can follow the same steps above but make sure to add the Intro Document to all of the relevant private Teams. You can do this by clicking into the Document and editing the sidebar Teams metadata:
## Template Welcome to Secoda! Secoda stands for **Se**archable **Co**mpany **Da**ta and will enable you to discover the assets that you need in order to make data-driven decisions. Secoda is an innovative data discovery and cataloging tool designed to help us manage and understand our data landscape. It helps us streamline the way we access and govern data, providing a single source of truth for our data assets. Secoda allows us to easily document, search, and collaborate on data knowledge within our company, enabling us to make more informed decisions and maintain compliance with data governance practices. Secoda integrates with all of our data sources and tools, including **\*insert your integrations here\*.** Secoda will be the one-stop-shop for all data needs including searching, cataloging, viewing lineage, data quality monitoring, and managing data governance. With this platform, you'll be able to: * Find verified resources to do your own discovery and analysis * Search in plain language to find information about the data in our organization * Ask Questions to the Data team * Explore the end-to-end Lineage of our data stack and identify breaking changes * See a list of verified Dictionary terms and Metrics that drive our company KPIs * Automatically document data assets that you own * Configure essential Monitors to be alerted about changes to your data Check out additional documentation of all of Secoda's features at [docs.secoda.co](https://docs.secoda.co/). **Feel free to add additional context about how you've set up Teams and which they should join, any documentation standards, if you've defined a Ownership, Verification or Tagging process etc.** # Training session guide A guide of what you might want to cover in your onboarding sessions ### Introduction * Introduce yourself and your roles with the project * Introduce Secoda high-level - AI-powered data search, cataloging, lineage and documentation platform for data teams * Explain your main use cases for the product, what issues and pain points it will help address * Explain which data is currently integrated and documented * Encourage users to browse and search our docs.secoda.co site for documentation ### **Demo** Share your screen showing your Secoda workspace. #### Homepage * Homepage is the first thing you'll see in the product. This is your own personal customizable homepage where you can: * Edit the cover image * Add widgets to pin certain resources that you come back to often * Add a private notepad for your personal notes * Filter the widgets for most popular to see what your teammates are searching for * While you're here, you can also explain the different roles (Viewers, Editors, Admins) and what access the group should expect based on their roles {% embed url="" %} #### Teams * Click into Teams and explain how you've designed them; point out which Team the group you're presenting to will be added to * Demo how to join and leave a Team and how it appears in the left sidebar once you join
* Click into a Team to explain the different sections within a Team * The Team Homepage functions similarly to the Main homepage, but it is configured by the Admins and is specific to that Team - pin resources that the whole Team will benefit from having easy access to * The Catalog hosts all of your data sources - show the different integrations * Metrics is where you can define key metrics your team uses, and visualize them by writing the SQL query used to calculate them * The Dictionary is a space for glossary terms and metrics - show what you've built out so far and how you plan to use that feature * Talk about your current use cases for Documents - what type of documentation have you built out * Questions is where FAQs live, and also can be used for data requests - explain if/how you plan to incorporate these into workflows * Collections are essentially folders that allow you to group a subset of each of these resources - you might consider having separate Collections for different projects within a team * Note that Admins can remove any of these sections to simplify the user's experience if they don't need access to certain features #### Search * Choose a resource that is particularly enriched (maybe one that is important/relevant to the group that you're presenting to) and Search for it * Click into the resource and show off all of the metadata that you've added like descriptions, owners, tags, verification etc. * Explain how you are defining ownership and verification at the moment, if applicable * Navigate to the Lineage tab to show that feature, if applicable * Click into Search and show off all of the search filters that will help users narrow down their search, more details here [#search](#search "mention")
#### AI Assistant * If users don't know exactly what they want to search for, show them that they can use the AI Assistant to search in plain language * Show a few examples of asking the AI some questions about your data: examples [https://github.com/secoda/gitbook/blob/master/readme/secoda-as-an-admin/onboarding-new-users/broken-reference/README.md](https://github.com/secoda/gitbook/blob/master/readme/secoda-as-an-admin/onboarding-new-users/broken-reference/README.md "mention") * If your team is interested in documenting PII, you could ask it to find you all of the tables with potential PII data {% embed url="" %} #### Slack Integration * Open up Slack to demo how to use the Slack integration to ask the AI questions about your data directly in Slack [Slack user guide](/extensions/slack-connection/slack-user-guide) {% embed url="" %} #### Additional Features Demo You may want to highlight additional features depending on who you are providing training for. Consider the persona that you are working with, and which features might be useful for improving their current workflows. For example, a data analyst might benefit from seeing how the [Queries](/features/queries) feature works in Documents. A data owner would benefit from hearing about the different ways to [Setting up your workspace](/best-practices/best-practices-for-setting-up-your-workspace#automate-documentation), like using Automations & the Propagate metadata feature. A data owner might also want to know about [Monitors](/features/monitoring) and how they can set up data quality alerting for their core tables. ### Next steps * Invite the users to Secoda and add them to their respective Teams * Ask users to review the resources that they/their team owns - update descriptions and other metadata if necessary * Ask users to test out the AI Assistant and provide feedback * Share any relevant documentation with users, including[Secoda as a viewer](/readme/secoda-as-a-viewer) [Secoda as an editor](/readme/secoda-as-an-editor) # User engagement and adoption This page will outline some outreach strategies to increase engagement with the tool. After you've onboarded a group of users, it's important to check back in within a few weeks to ensure that they are still engaging with the platform. If you are seeing less engagement than you had originally expected, do not get discouraged. As the workspace Admins, there are some strategies that you could implement to help change user behaviors and remind them to check Secoda for their data questions. ## **Establish workflows that Secoda can improve** Let's say that most of the questions you hear about data are coming through Slack or in another communication channel. As Secoda admins, staying alert about these questions is important. Consider connecting our [Slack](/extensions/slack-connection) integration so that users can continue asking questions there, but still get answers and value from Secoda without having to go to the app. {% hint style="info" %} Make sure to update your documentation with new content that arises from these questions so that Secoda remains the source of truth, and content doesn't get lost in Slack! Check out our feature to [Slack](/extensions/slack-connection#push-slack-thread-into-secoda-questions). {% endhint %} Or let's say that your analytics team would prefer to stay in their BI tool, but still get value out of Secoda. Add a to-do item to their Secoda onboarding checklist to download the [Chrome](/extensions/chrome-extension) so that they can view and edit Secoda metadata directly in their workflows. Check out other ways for [Integrating Secoda into existing workflows](/best-practices/integrating-secoda-into-existing-workflows). ## **Send out recurring communications** As a Secoda Workspace Admin, it's important to stay visible within the data community at your organization. There are quick and easy communications you can send out to keep Secoda top of mind for your users - one example being **Weekly** [Tips and tricks to share with new users](/readme/secoda-as-an-admin/user-engagement-and-adoption/tips-and-tricks-to-share-with-new-users)**.** Use our [Announcements](/features/announcements) to share news about new verified data sources, a new integration you've connected, or fields that are going to be deprecated, as examples. ## **Assign ownership to resources** Assigning ownership at either the Group or individual level encourages users to get into the tool and document their resources within Secoda. Owners also receive notifications about changes to their resources, which encourages them to stay on top of documentation. Consider outlining both Ownership and Documentation standards or expectations so that users understand their roles and responsibilities. ## **Highlight user successes** Everyone appreciates a little recognition and kudos for their efforts. Consider the following ideas to give accolades to your user's hard work within the product. * Celebrate wins in Town Hall / All Hands meetings or in Slack * Highlight a data community member of the month in a monthly newsletter * Allow users to vote on well-documented resources that helped them complete a project Use the [Analytics](/features/analytics-dashboard) to identify some of these metrics. ## **Make documentation fun!** Friendly competition, anyone? Here are some gamification strategies to get your users excited about documenting their data. * Create a data owner leaderboard and recognize users who: * Own the most number of resources * Have added the most terms to the Dictionary * Have added the most enrichment to a single resource * Own the most utilized resource (dashboard, document etc.) * Plan a data scavenger hunt to get users acquainted with their data landscape, and find areas where data documentation could be improved ## **Create a data community** People love being apart of a community and Secoda should be a place for people with similar questions and ideas to come together to collaborate. Some ideas to foster a data community could include: * Host "Expert Lunch & Learns" to show Secoda in practice * Data owner in your organization presents on and discusses a specific dataset, how it is currently used for analysis, and open the conversation up to other data owners * Secoda user highlights how the product has improved a data value stream of theirs, reducing the amount of time spent on a particular task * Host "Metric Brainstorm Sessions" * Gather a group to discuss an organizational metric that needs defining * Have everyone come up with their own understanding of the metric, share definitions, and workshop until you all come to an agreement on a strong definition * Career development opportunities * Secoda is a tool for everyone across the organization and everyone should have the opportunity to grow their data skillset * Consider hosting a data onboarding program, SQL trainings, or other additional trainings to promote a data culture through Learning & Development groups ## Adoption & defining KPIs When we think about user adoption of a new tool, we like to bake in a certain amount of days buffer of expectation (this number will be different for each organization). Using Secoda is a new "habit" for users for them to unlearn old behaviors (i.e. searching in Confluence/Snowflake) and learn to go to Secoda first. Use the [Analytics](/features/analytics-dashboard) to track usage of the tool and define your own team's KPIs in order to define and measure success. # Tips and tricks to share with new users Want to boost engagement and keep new users excited about Secoda? Consider sharing weekly "tips & tricks" with new Secoda users to show them what can be done in the product. We've gathered a list of some ideas that we encourage you to share via email, Slack, or whatever communication channel your organization uses. Consider posting these one at a time, as quick reminders to your end users about all the functionality within the product. **Suggested format:** > :wave:**Secoda tip of the week** > > **Did you know that you can \*insert tip\*?** > > **\*Insert screenshot, screen recording or link to the relevant documentation\*** Below are some examples separated by theme. ## Search and discoverability tips Did you know that you can ... 1. ask a question in the search bar in plain language? Simply click into Search from the Homepage or in the top left of any page, and start typing.
2. filter to see the most popular resources in your workspace? In this clip, I show filtering for the most popular dashboards, for example.
3. filter to find resources that aren't documented yet? Let's say you want to find resources that have no description yet so that you can ensure everything is documented. Set the Description filter to Blank, and voila!
4. create and save search filters that you often go back to? Let's say you and your teammates often searching for the columns that are tagged as PII. To save these filter, simply create a[ Search view](/features/search#search-views) which can be shared with specific Teams.
5. type / from any screen in the app to bring up the search bar? 6. bookmark resources that you often go back to and pin them to your personal Home page?
## AI tips Did you know that you can ... 1. ask the AI Assistant to write queries for your analysis? New to SQL and need some quick help? The AI Assistant can be a helpful resource for this.
2. ask the AI Assistant to translate what queries mean? Here I've copied a query that I found in the Query history of one of my popular tables. I wasn't sure what it was calculating, but Secoda AI read the SQL and explained it to me.
3. use the @ symbol to tell the AI Assistant which resource you'd like to reference, improving its accuracy?
4. ask the AI Assistant about how two resources are related through lineage? Tag the resources in your question and let the AI do its magic.
## Editing tips Did you know that you can ... 1. generate descriptions for resources with one click? Use Secoda's [AI Description editor](/resource-and-metadata-management/add-documentation/ai-description-editor) to save you time (hint: you can also do this in bulk!)
2. bulk edit resources with similar names to have the same metadata? Here I demonstrate how you can apply Verification and PII tags in bulk to similarly-named resources to save time. ([Propagation](/resource-and-metadata-management/add-documentation/propagating-metadata))
3. bulk edit resources similarly to how you would in Excel or G-Sheets, with a simple drag down? If you add metadata to one row, you can drag that down to apply this in bulk to the other resources below it.
4. see a list of flagged resources that may contain PII, and then update them with a PII tag with one click? ([PII identifier](/resource-and-metadata-management/tags/auto-pii-tagging))
5. be notified when there are changes to the source data that might indicate the documentation needs to be updated? Toggle on [Notifications](/features/notifications#schema-change-notifications) which will alert subscribers and owners about changes to those resources. 6. create templates for Documents & Dictionary terms to ensure a set of documentation standards are met?
7. download a filtered subset of the catalog to share with others? Simply filter the catalog how you'd like, and click Export resources as CSV.
8. upload a list of existing dictionary terms from a CSV into Secoda's Dictionary? Want a quick automated way to upload a list of terms without having to individually create them? Follow our CSV standards then choose the file to upload by following these steps[Import and export resources](/resource-and-metadata-management/import-and-export-data#importing-metadata-into-secoda). ## Slack integration tips Did you know that you can ... 1. ask the AI Assistant questions in Slack privately, without having to post to the channel? 2. ask Questions about Secoda metadata anywhere in Slack? 3. push threads from Slack to Secoda's Questions, ensuring that all context is captured? 4. receive notifications about changes to resources directly in Slack? Simply configure your [Notifications](/features/notifications) accordingly and stay up to date on all updates. # Secoda as an editor Here you can find resources for new Editors within Secoda As an Editor in Secoda, you are pivotal in enriching data resources and fostering a data-driven culture within your organization. Here's how you can make the most out of your role: **Start Here:** * [ ] **High-level** [introduction to Secoda](/readme/secoda-as-a-viewer/introduction-guide) * [ ] **Video Tutorials:** * [ ] Video: [Introduction to Secoda](/#video-resource-introduction-to-secoda) * [ ] Video: [Initial steps for new Editors in Secoda](https://www.loom.com/share/ba344cf7b50d41c8af25b67b2a943712?sid=fcca1054-2431-4d54-b3fe-4ab364e49b0f) **Enhance and Enrich:** * [ ] **Best Practices:** Read up on[ ](https://chat.openai.com/c/eab2984d-7157-4080-873f-f2cc92e2f22b)[best practices within Secoda](/best-practices). * [ ] **Identify Ownership:** Determine which data resources you own and begin [adding valuable context and enrichment](/best-practices/documentation-best-practices). * [ ] **Chrome Extension:** Download and use the [Chrome extension](/extensions/chrome-extension) to manage Secoda metadata directly in tools like Tableau and Snowflake. **Engage and Enhance:** * [ ] **Secoda AI Assistant:** Interact with the [Secoda AI Assistant directly in Slack](/extensions/slack-connection/slack-user-guide) for data inquiries—ensure your Admins have enabled this integration. * [ ] **Documentation and Glossary Terms:** Populate the tool with essential [Documents](/features/documents) and [Glossary terms](/features/glossary) related to your resources. **Customize and Configure:** * [ ] **Notification Settings:** Set your [notification preferences ](/features/notifications)to stay updated on changes and updates. * [ ] **Personal Homepage Customization:** Tailor your [personal homepage](/features/homepage) with links to frequently accessed resources for quicker navigation. **Explore Further:** * [ ] **Learn More:** Dive deeper into the extensive features available to you as an Editor in Secoda. [Explore features](/features). # Secoda as a viewer Viewers within Secoda are mainly business users who need insight into the metadata within their organization. Here are some helpful first steps with documentation links for new Viewers within Secoda. **Get Started:** * [ ] **High-level** [introduction to Secoda](/readme/secoda-as-a-viewer/introduction-guide) **Explore and Learn:** * [ ] **Explore Search Capabilities:** Dive into the product and [Search](/features/search) for data you frequently use. * [ ] **Navigate the Catalog:** Watch a video on [Navigating the Catalog](/features/catalog#navigating-the-catalog-video) * [ ] **Engage with AI Assistant:** Experiment with the AI Assistant. Check out these [tips to enhance your interactions](/features/ai-assistant/best-practices). **Deepen Your Understanding:** * [ ] **Best Practices:** Read up on [best practices within Secoda](/best-practices). * [ ] **Collections:** Learn about Collections curated by your Admins. [Learn](/features/collections-1) [more](/features/collections-1). * [ ] **Glossary and Documentation:** Explore your organization's business terms in the [Glossary](/features/glossary) and access [Documents](/features/documents) you have permissions to view. **Interact and Collaborate:** * [ ] **Questions Feature:** Use the [Questions feature](/features/ask-questions-in-secoda) to ask your data team about your organization's data. Viewers can respond to and interact with questions for Admins and Editors to review and mark them as answers as required. * [ ] **Resource Management:** [Request changes to resources](/readme/secoda-as-a-viewer/requesting-changes-in-secoda) within Secoda, such as adding descriptions to resources where you're an expert or owner. * [ ] **Slack Integration:** Test out searching Secoda directly within Slack. Learn how to use the [Slack integration](/extensions/slack-connection/slack-user-guide). # Introduction guide This Viewer guide provides new users with an overview of the tool Secoda stands for **Se**archable **Co**mpany **Da**ta. Your Secoda workspace is the single source of truth for your organization’s data. Secoda is a useful tool for business users because it streamlines data management and enhances data governance within an organization. By automating data lineage and documentation, Secoda can help business users understand where their data comes from, how it has been transformed, and how it is being used. This can help improve the transparency and trustworthiness of the data, making it easier for business users to make informed decisions. The data enablement features offered by Secoda, such as Secoda AI, the Search function, data dictionaries, and data lineage visualization, can help business users quickly find and access the data they need, improving productivity and efficiency. Additionally, inviting team members to Secoda allows business users to collaborate and share data assets more easily, improving collaboration and data management within the organization. ## Outcomes * **Faster time to value:** With Secoda, teams have been able to reduce onboarding time from 2 months to a few days for new data members. * **Flexibility to scale with growth, quickly:** With Secoda, all customers can find the most used dashboards, the definition of a KPI and what data the marketing team is using. * **Increase accuracy:** With Secoda, teams can trust their data resources, tag sensitive information and ensure that all members have the ability to use trustworthy company data. * **Faster decisions:** With Secoda, all members can find the most used dashboards, the definition of a KPI and what data the marketing team is using. **Two customer case studies:** 1. [**7Shifts Case Study Case Study**](https://www.secoda.co/customers/7shifts) 2. [**PartnerStack Case Study Case Study**](https://www.secoda.co/customers/partnerstack-doubled-data-team-output-2-5x) ## Resources for Getting Started We’ve added some resources below that show you how you can use the different features of Secoda. Check out our introduction to Secoda video [here](/#video-resource-introduction-to-secoda). ### 1. Collaborating within your workspace #### Teams When you're added to Secoda, your workspace Admins should add you to your relevant Team, and you will be able to see all of the resources (Catalog, Glossary, Documents and Questions) tied to that Team. You can check out all of your workspace's Teams and join Public ones by following [**these steps**](/user-management/teams#joining-teams). This diagram shows the structure of Teams:

Structure of Teams

#### **Asking questions** With Secoda, you can find past questions, as well as pose new questions to the data team. Since Secoda is connected to your organization’s data, questions and answers can include links to relevant tables, queries, or even other questions that can provide more context. Read more about that [**here**](/features/ask-questions-in-secoda). #### Notifications Users can change their notification settings by going to **Settings -> Notifications.** You can change how often you receive these notifications, where you'll receive the notifications and which kinds of notifications you'll receive. Read more about notifications [**here**](/features/notifications). ### 2. Searching within Secoda Search is super important to Secoda users and we are constantly improving the functionality to provide the best experience. You can search for any resource in plain language, or by the table name and data source if you already know what you're looking for. #### Secoda AI Assistant The [**Secoda AI**](/features/ai-assistant) greatly improves user experience when searching for resources. Once your workspace Admins have enabled it, you can find the AI Assistant on the lefthand panel under Search. You can also Ask AI right from the main Search and[ **within Slack**](/extensions/slack-connection/slack-user-guide#secoda-ai-slackbot). This feature will be helpful for business users who might not know where to begin when looking at data. Check out some examples of prompts to ask the AI below! {% embed url="" %} Check out our other [**Search tips and tricks**](/features/search) for filtering down your search results. ### 3. **Creating a single source of truth** Every user across the organization should feel confident searching for and utilizing data in order to make business decisions. We consider Secoda to be your organization’s single source of truth where you can access all the relevant resources and metadata about those resources that you need in order to do your job effectively. #### Catalog The [**Catalog**](/features/catalog) is the home to all metadata for any Tables, Columns, Dashboards, Charts, Jobs and Events in Secoda. All catalog resources are accessible via Secoda’s Search, and can be referenced in other resources in your workspace. #### **Glossary** The [Glossary](/features/glossary) is where you can define terms and metrics that are relevant to your organization. All documented terms are accessible via Secoda’s Search, and can be referenced in other resources in your workspace. It’s an especially helpful space for newer employees getting acquainted with the company or their role. Some common use cases for glossary terms are defined metrics and company specific jargon. The overall goal of a glossary is to get everyone in your organization speaking the same language. #### Documents [Documents](/features/documents) in Secoda are robust, and allow you to combine text, queries, photos, and live charts in a notebook interface. Use this tab to find information about your organization’s data that is not directly tied to one specific table, term, or resource. Just like the Glossary, all Documents can be found using Search, and can be directly linked in other resources in the workspace. We see customers using Documents in a similar way you might use other collaborative documentation tools, like Confluence and Notion. #### Collections Using [Collections](/features/collections-1) is a great way to organize your Team's resources. You can think of a Collection as a folder that hosts a group of resources (tables, dashboards, glossary terms, documents, questions) pertaining to one particular subject. These folders can be organized however makes sense to your organization, but some use cases we see are by project or the big nouns in your company. See our doc on the[ **steps to setting up Collections within Secoda**](/features/collections-1). ### 4. Customize your workspace Each user has their own individual Home page! Learn how you can [**customize it here**](/features/homepage#personal-homepage) to add personal notes and flag specific resources you often visit. # Requesting changes in Secoda Learn how Viewers can request changes and make data contribution collaborative on any Resource in Secoda. ## How Viewers can Request Changes If a Viewer runs into information that needs correction, or needs to be updated, they can Request Changes on any Resource in Secoda. 1. Click the `Request change` button found in the three dots (...) at the top right of a Resource.
2. Choose the desired Field to update, and type in the updated Text to the Edit box. When you're finished, click `Submit`
## What Happens After a Change is Requested? The **Owner** of the Resource the change was requested on, will receive a notification in their [Inbox](/features/data-inbox) and will be able to either Approve or Deny the change. If the change is Approved, the Requested Change will be applied to the Resource. ## Video Resource {% embed url="" %} # Best practices Check out our documentation around best practices when working with Secoda. {% content-ref url="/pages/qNTcIkdnvQKqMDR5xwLH" %} [Setting up your workspace](/best-practices/best-practices-for-setting-up-your-workspace) {% endcontent-ref %} {% content-ref url="/pages/fcLm5DfMom2KLEJCSY8t" %} [Integrating Secoda into existing workflows](/best-practices/integrating-secoda-into-existing-workflows) {% endcontent-ref %} {% content-ref url="/pages/HKIGCB2QuAOq9LYzkTsF" %} [User guide](/features/ai-assistant/best-practices) {% endcontent-ref %} {% content-ref url="/pages/NPRLNJYBfgIModdBAOcZ" %} [Documentation best practices](/best-practices/documentation-best-practices) {% endcontent-ref %} {% content-ref url="/pages/PCLyOE2243W9nxmjgb9P" %} [Glossary best practices](/best-practices/dictionary-best-practices) {% endcontent-ref %} {% content-ref url="/pages/amtJ0jJx9gB9PJaMAG7C" %} [Data governance](/best-practices/data-governance) {% endcontent-ref %} {% content-ref url="/pages/CCpuUpzdqDI10dVj4G1R" %} [Data quality](/best-practices/data-quality) {% endcontent-ref %} {% content-ref url="/pages/5eCKWKJldLnNxoEX2uPw" %} [Slack <> Questions workflow](/best-practices/slack-less-than-greater-than-questions-workflow) {% endcontent-ref %} {% content-ref url="/pages/OZv0WCpO70Da8Kc6XpXh" %} [Defining resources workflow](/best-practices/verifying-resources-workflow) {% endcontent-ref %} {% content-ref url="/pages/h0WDy019FG1laB50iiIl" %} [Exposing Secoda to external clients](/best-practices/external-teams) {% endcontent-ref %} # Setting up your workspace Learn about some best practices for setting up that we've seen work well After configuring your integrations and importing metadata, it's essential to consider the layout and structure of your workspace. This guide outlines best practices for effectively organizing your workspace and features to ensure new users are well-prepared upon onboarding. ## **Organizing teams** Teams are the primary way to manage your organization's resources. Learn more about the capabilities and setup process for [Teams](/user-management/teams). ### **General team usage** By default, all metadata from connected integrations is accessible via the General Team Catalog. The General Team is set to Public, but you can change it to Private or rename it in the Team Settings. **Use cases for the public 'General' team:** * Serve as a common space for all workspace users to access. * House a glossary of commonly used terms. * Store internal processes and best practices documentation. * Include onboarding documents for new users to navigate the workspace. ### **Creating specific teams** 1. **Identify your users**: Start by identifying both your current and potential future users. Determine their roles and consider grouping them to streamline access management. For a detailed understanding of how Roles, Teams, and Groups interact within our RBAC framework, refer to our [User management](/user-management) documentation. 2. **Establish teams**: Create Teams reflecting the departments or roles of your users. For example: * If your initial power users are part of the Data Platform team, create a "Data Platform" Team and add relevant users or [Groups](/user-management/groups). * For future rollouts to the Business Intelligence team, prepare a "Business Intelligence" Team for upcoming onboarding. #### **Customizing teams** * **Sidebar customization**: [Adjust the sidebar](/user-management/teams#customizing-your-teams-sidebar) to match the access needs of each Team. For example, less technical users might only need access to specific Collections, so you can choose to hide unnecessary sections like the Catalog, Glossary, Documents, or Questions. * **Assign integrations or resources to teams**: To limit which metadata is added to each Team, at the highest level you can [add whole integrations to Teams in the settings](/readme/secoda-as-an-admin/connect-your-data#how-to-add-integrations). Utilize [these tricks](/features/catalog#limiting-resource-access-in-a-catalog) to add only necessary schemas, tables, and columns to each Team. ## Automating documentation Secoda offers several features to help automate the enrichment and documentation of your data. For more details, check out [Documentation best practices](/best-practices/documentation-best-practices). #### **Initial documentation setup** Creating robust documentation from the start is crucial. Ensure that Documents, Glossary terms, and Catalog resource descriptions are well-documented to showcase Secoda's value immediately to new users. This initial content not only helps in accurate searches but also enhances the AI Assistant's functionality. #### **Prioritizing certain documentation** Begin by documenting the most frequently used resources, especially those relevant to the initial users you're bringing into Secoda. Use the [Verified identifier](/resource-and-metadata-management/tags/verified-tag) tag to signify that these resources are business-approved and provide the most reliable data. ## **Building out FAQs** Leverage Secoda's Questions feature to support new users effectively. Admins should pre-populate the Questions section with FAQs, providing answers and marking correct responses. This helps guide new users and facilitates their engagement with the data team. ## **Integrating with Slack** Integrate Secoda with Slack to enhance internal communications. This integration allows users to query data through Slack and receive workspace notifications without leaving the platform. Learn more about [setting up](/best-practices/slack-less-than-greater-than-questions-workflow) and [using the Slack integration](/extensions/slack-connection/slack-user-guide). ## **Creating search views** Develop [search views](/features/views) to aid Search navigation for new users. These views, once saved, are accessible to all users, helping them locate needed information efficiently. ## **Enhancing the homepage** The [Homepage](/features/homepage#team-homepage) is the first interface users interact with in Secoda. Enhance user experience by pinning critical resources and writing notes in the Notepad feature. Explore how-to videos on optimizing [Homepage](/features/homepage) functionality. # Integrating Secoda into existing workflows Check out best strategies on integrating Secoda into your current day-to-day flows. Let's face it - bringing on a new tool to your already-hectic workflow can seem stressful. With Secoda, users shouldn't have to change current behaviors to engage with the product. We've iterated on and added many features to allow Secoda to seamlessly integrate with your existing workflows. ### [Secoda's AI Slackbot](/extensions/slack-connection/slack-user-guide#secoda-ai-slackbot) * Connect the AI Assistant to your data requests Slack channel so that users don't have to leave their Slack workflow to find answers. Check out [best practices ](/best-practices/slack-less-than-greater-than-questions-workflow)here. * End result: * Improved data literacy * Reduced time spent on repetitive replies to users * More time for analysts to focus on their most important tasks ### [Chrome](/extensions/chrome-extension) * Download the Chrome extension to enable your users to access Secoda metadata directly in the other tools they're currently working in, like BigQuery and Tableau * If the extension is enabled, a user can see and edit important information like ownership and descriptions coming from Secoda without having to move across multiple tabs ### Enable [Notifications](/features/notifications#slack-notifications) * Users can pick and choose notifications and alerts regarding changes in the workspace to be sent over Slack * If you own or subscribe to a resource, you will receive notifications regarding any changes to the schema - indicates it might be time to edit documentation or edit a dashboard * End result: * Improved data quality * Reduced time spent on manual notifications ### [Jira](/extensions/jira) Integration * Connect the Jira integration to seamless manage your data requests coming in from Jira, directly in the Secoda Questions feature ### [Confluence](/extensions/confluence) Integration * Connect the Confluence integration to bring in content from Confluence spaces and pages * Why integrate Confluence? * Users don't have to change their current behavior of documenting there * Documentation from Confluence becomes easily searchable within Secoda, as well as from Slack after you connect the Slack integration * Ask the AI assistant questions about what exists in Confluence to speed up documentation and discovery * Link documents to related resources like tables in the Catalog, or metrics in the Dictionary # Documentation best practices Use cases for using Secoda to automate repetitive documentation tasks, saving time for your data team. By leveraging Secoda, data teams can automate manual and repetitive tasks which would improve these processes in terms of efficiency. This page provides some common solutions for leveraging Secoda's features to save time. {% hint style="info" %} Note: When making edits to descriptions in Secoda, ensure that you have the [integration's permissions set ](/integrations/integration-settings#permissions)so that descriptions made in Secoda persist. {% endhint %} ## Automate enrichment within Secoda [User guide](/features/ai-assistant/best-practices) * Our AI Assistant can be utilized in many ways, and we find it extremely powerful when it comes to documentation. * The AI Assistant is available in the sidebar on each resource in your workspace. Simply ask it to write up documentation on a resource that you're clicked into to generate rich documentation in just seconds. Check out the video resource here: [User guide](/features/ai-assistant/best-practices#how-to-use-the-ai-assistant). * The [AI description editor](/resource-and-metadata-management/add-documentation/ai-description-editor) can similarly be used to generate descriptions of your resources with just one click. This can be done on the Catalog resources, Dictionary metrics and Collections. [Automations](/features/automations) * Define rules once for updating documentation, and have these automatically updated on a set cadence * Automations can make updates to tags, descriptions, owners as well as help organize your workspace by moving resources to specific Teams or Collections, for example [Bulk editing](/resource-and-metadata-management/add-documentation/bulk-editing-resources) * Check out our bulk editing features to make bulk metadata edits to your resources [Import and export resources](/resource-and-metadata-management/import-and-export-data#importing-metadata-into-secoda) * Create a CSV to bulk import metadata for resources already existing in Secoda. For example, if you have descriptions for your Oracle assets living in a G-sheet, you can upload these onto your existing resources in Secoda. [Propagation](/resource-and-metadata-management/add-documentation/propagating-metadata) * You might find that a lot of your data resources are related, and it can become redundant having to manually update resources that should have the same tags and owners, for example. * Our propagate metadata feature allows you to bulk edit these related resources! * End result: * Reduced time spent on documentation * Improved documentation and data literacy [PII identifier](/resource-and-metadata-management/tags/auto-pii-tagging) * The PII Identifier automatically flags potential PII data assets across your workspace by searching for keywords. * You can even customize the list of keywords depending on how your organization defines personal information. [Slack AI Assistant](/extensions/slack-connection/slack-user-guide#secoda-ai-slackbot) * With the Slack integration, you have the ability to ask the AI Assistant questions in Slack, get a response, and then automatically push those answers into your workspace. * This means less redundancy with answering questions and additional documentation in the product. [https://github.com/secoda/gitbook/blob/master/best-practices/broken-reference/README.md](https://github.com/secoda/gitbook/blob/master/best-practices/broken-reference/README.md "mention") * Use Secoda's APIs to bulk edit the name, descriptions, and tags on your resources. * Automatic PII detection through the API can identify resources that need to be more governed for auditing purposes * PII can be tagged, and automatic reporting for audits can be generated ## Identify what needs to be documented If you'd like to identify which resources have not been documented yet, you can do so by following these steps to [Search](/features/search#search-within-the-catalog). If you'd like to be notified of changes that may result in a need for updated documentation, you can subscribe to schema change notifications for those particular resources. Set those up here[Notifications](/features/notifications#h_3a4bfd6458). **Try out our** [**ROI Calculator**](https://www.secoda.co/data-discovery-roi-calculator) **to estimate how much money your team could save by automating tasks with Secoda.** # Glossary best practices Read up on some best practices and strategies around building out a Glossary within Secoda We've put together some ideas around best practices when building out a Glossary within Secoda. There's not one way to go about these as every organization has different requirements, but here are some ideas that have proven to work well :rocket: ## Choosing terms If you have an **existing list of terms** and definitions hosted elsewhere... * Use our [Import feature](/resource-and-metadata-management/import-and-export-data#importing-metadata-into-secoda) to automatically bulk upload these into Secoda via CSV format If you don't have a list yet, and need to start fresh... * It's important to initially focus on high-impact terms that are important to your users within Secoda * Consider hosting a small group workshop with your workspace Admins, or some core editors * Brainstorm a list of highly-used, high-impact terms across your organization or data team * Think of terms that are often misused, undefined, or confusing for your stakeholders * Vote on the priority of each term and make plans to gather the right people in a room to develop definitions for these ## [Templates](/resource-and-metadata-management/add-documentation/templates) * Utilizing the Templates feature within the UI is a great way to ensure that documentation standards are set and followed * Admins of each Team should create glossary templates so that editors know what kind of information is expected to be added to each term ## Organization * Since each Team has it's own section for a glossary, you might consider ways to organize the many terms that your organization needs to define * Consider using the General Team glossary as an onboarding tool * Define more general terms that span across the whole organization * Define company jargon and acronyms that can often get confusing for new hires * Team-specific glossaries * Define terms by Team, based on which team *owns* those terms and therefore owns the documentation of them * Consider [Nesting terms](/features/glossary#nesting-terms) so that they are easier to navigate for your users ## Enrich glossary terms * When defining terms, it's often necessary to involve multiple stakeholders to reach agreement. Secoda's [Defining resources workflow](/best-practices/verifying-resources-workflow) can help facilitate this process. * Add enrichment to glossary terms by adding them to relevant Collections, adding owners and tags, link them to related terms * Don't forget that you can make bulk changes to terms and use some of our other documentation tips listed here [Documentation best practices](/best-practices/documentation-best-practices) * This ensures that they're even more searchable within the UI, and so users can see how terms and resources are related across the workspace ## Verification * Consider using our [Verification tag](/resource-and-metadata-management/tags/verified-tag) in order to indicate that a term is ready to be used, and the definition has been approved by the data team or a list of relevant stakeholders * The goal of this tag is to provide trust in the data for users, so that users feel confident using the term in their workflows # Data governance This page will outline some best practices around data governance Data governance is a hot topic in the data space as it covers the ways to ensure that your company's data is **secure, accurate, reliable, and accessible**. There are many features and capabilities within Secoda that can help you achieve your data governance goals. Here are some best practices to consider to enable data governance across your organization: ## Define roles and responsibility When rolling out a tool like Secoda, it is important to define roles within your organization and who will be taking on which responsibilities. This will ensure that every user knows what needs to be done so that the metadata within the product is **accurate** and up to date. It will also ensure that only the relevant people are able to edit and view certain resources that may be private. Consider these questions: * Who is a part of the initial group to enrich and implement Secoda? How are we delegating these tasks? * Who will be the data champions who will own the data resources? How do we define ownership? * Which users and stakeholders will we onboard? Which Teams do we need to create, and which resources will they need access to? Once you have a grasp on the makeup of your Teams and users, [**roles**](/user-management/roles) can be assigned using our RBAC approach, owners can be set, and [**Teams**](/user-management/teams) can be created. [Set permissions at the Team level](/user-management/teams#editing-member-settings) so that only the right users have access to editing the metadata in that Team. Enforce [ownership](/resource-and-metadata-management/assigning-owners) of critical data so that it is kept up to date. {% hint style="info" %} Consider creating an [Automation](/features/automations) that assigns ownership to owner-less resources, to ensure that resources don't get lost. Use the template we provided in the UI called "Assign ownership for schema tables". {% endhint %} ## Identify critical data elements It's important to start this initiative by identifying the data that most impacts your business. Consider these questions: * What data is most critical to your business? * Which reports and dashboards are relevant and up to date? * What are questions that pop up all of the time? Start here since they are more likely to have a larger impact on more users. Another way to identify critical data elements once you've integrated your data into Secoda is by using our [Lineage feature](/features/data-lineage). Users can look at the overall lineage and see which are some key nodes that touch a lot of parts of a pipeline. They can also make note of important data resources and make a note of anything upstream of that asset, as it shouldn't also be marked as critical since it's dependent on it. [**Popularity**](/features/popularity) metadata could be another important field to look at since usage data can help us understand what's most important to the business. Read more about these ideas here: . ## Enrich, enrich, enrich Your users should feel confident using and **accessing** the right data, but we often see questions and concerns like: * What does this data mean? * Where can I find data on *this subject*? * Who's responsible or the subject matter expert? * Is this the right data? * Is it up to date? * Is this sensitive data? This is why enrichment is so important to Secoda, and if done well, should answer all of the questions above. Add descriptions, ownership, and tagging to make your important resources easier to locate when searching within Secoda. Define **standards** for your editors to follow so they know which types of metadata needs to be included in their documentation. Some of this can be addressed by creating [Templates](/resource-and-metadata-management/add-documentation/templates) for documentation! {% hint style="info" %} Consider using our [**Verified identifier**](/best-practices/verifying-resources-workflow) on resources that have checked the box on each of those questions, indicating that it is ready for use by your Members. Using this system will provide your users confidence and **reliability** in using the data. Check out some tips on implementing a [Defining resources workflow](/best-practices/verifying-resources-workflow) at your organization! To enable **security measures**, use our [**PII Identifier feature**](/resource-and-metadata-management/tags/auto-pii-tagging) to tag sensitive resources to alert Members in your workspace. An [Automations](/features/automations) can be created that automatically tags new resources with certain qualities, as PII. {% endhint %} ## Read about how one team automated data governance with Secoda's help {% embed url="" %} In summary, the team uses the following strategies to work towards their data governance goals: * They organized the workspace so that each team has their own [**Collection**](/features/collections-1) which acts as a single source of truth for all of their documentation * They map every table to a Collection , and ensured that every table has a defined [**owner**](/resource-and-metadata-management/assigning-owners) * Tip: Try out [**Automations**](/features/automations) to push new tables into the right Collection, and set the owners automatically * They rely heavily on [**Lineage**](/features/data-lineage) for understanding downstream impacts of their schema changes * They use [**Announcements**](/features/announcements) as well as [**Slack**](/extensions/slack-connection/slack-user-guide) to notify relevant stakeholders about changes * They have a very defined workflow using our [**APIs**](https://github.com/secoda/gitbook/blob/master/best-practices/broken-reference/README.md) where documentation criteria is required by the developer in order to create new tables and push them to Secoda # Data quality This page will outline some best practices around data quality At Secoda, we understand the critical role that data quality plays in ensuring reliable analytics and decision-making processes. High data quality prevents inconsistencies, errors, and anomalies, thus enhancing the accuracy and completeness of your data. ## Key features supporting data quality * [**Data quality score (DQS)**](/features/data-quality-score)**:** * Secoda provides a Data Quality Score out of 100, reflecting the quality of data connected to Secoda. This score helps you gauge the integrity and reliability of your data at a glance. * [**Monitoring**](/features/monitoring) **Tools:** * Set data quality monitors on your data and receive alerts when thresholds are breached. This feature helps you stay on top of unexpected changes in your data. * [**Column profiling**](/features/column-profiling)**:** * Gain insights into data distribution, count, and uniqueness within columns. This tool helps you understand data at a granular level and ensures that your data remains structured and query-ready. * [**Stale data identification**](/features/removing-stale-data)**:** * Automatically identify and hide stale data within your workspace, ensuring that users interact only with the most current and relevant data sets. * [**Resource documentation**](/features/filters#use-case-filtering-for-undocumented-resources)**:** * Easily identify undocumented resources within the UI. Secoda aids in maintaining rigorous documentation standards, ensuring that all data assets are properly described. * [**Notification**](/features/notifications#h_3a4bfd6458) **and** [**Subscription**](/features/notifications#subscribe-to-resource-changes)**:** * Configure notifications to stay informed about resource changes. This ensures that you are immediately aware of any schema changes that might affect downstream dependencies, allowing for quick adjustments. ## Why prioritize data quality? Improving data quality directly enhances the utility, accessibility, and reliability of your organizational data. By implementing robust data quality measures, Secoda helps organizations foster a culture of data trustworthiness and reliability. Our tools are designed to provide actionable insights, promote transparency, and drive better outcomes by ensuring that your data meets the highest standards of quality. # Clean up your data Clean up your data resources with Secoda's help. Secoda has built-in tools designed to streamline the process of data cleanup, enhancing the overall health of your data environment. This guide will explore how you can leverage these tools to improve data quality, enhance security, reduce storage costs, and boost productivity. {% hint style="info" %} Make sure you check the [lineage graph](/features/data-lineage) in Secoda before deprecating a resource in the source! {% endhint %} ## Key benefits of effective data cleanup * **Efficient Resource Management:** Quickly identify underutilized data, reducing time spent on manual checks. * **Enhanced Data Quality and Security:** Improve the accuracy and protectiveness of your data assets. * **Increased Analyst Productivity:** Ensure analysts have access to relevant and reliable data. * **Cost Reduction:** Decrease expenses associated with storing outdated or unused data. ## Features for cleaning up your data ### Access [Popularity](/features/popularity) metadata * Utilize the Popularity metadata to determine which data resources are least accessed. * Sorting the Catalog by Popularity helps identify candidates for deprecation based on minimal views or queries. ### [Automations](/features/automations) to identify stale assets * Set up Automations to tag resources that haven’t been accessed or updated within a specific timeframe as "Stale" or "Candidates for Deprecation." * Consider adding a property to the Automation to push these to a private Team or specific Collection, if that's helpful for review purposes. * Then, filter the Catalog by these tags to manage these resources efficiently!
### [Use cases](/features/monitoring/monitoring-use-cases) * Implement Cardinality or Unique Percentage Monitors on essential data resources. * Alerts from these monitors can indicate duplicates or other data quality issues, prompting cleanup actions. {% hint style="info" %} [Secoda’s API](https://github.com/secoda/gitbook/blob/master/best-practices/broken-reference/README.md) extends the functionality of the above features, enabling programmatic management of data cleanup tasks. If you require assistance with the API, the [Secoda Community Slack](https://via.intercom.io/c?url=https%3A%2F%2Fjoin.slack.com%2Ft%2Fsecodacommunity%2Fshared_invite%2Fzt-mhnu278g-FktKZmZ51SDQtlu3NRAxqg\&h=13f5aaa171821956434fc25f4c759a803f98a84f-dssmg53d_11:24933\&l=d215b12164c764d92e3bca464c2434cae72f7a22-8270396) is available for support. {% endhint %} ## Cost containment resources * Articles: * [Data Stack Cost Management Best Practices | Secoda](https://www.secoda.co/blog/managing-costs-of-the-modern-data-stack-at-scale) * [How To Increase the ROI of Your Data Team | Secoda](https://www.secoda.co/blog/from-cost-center-to-revenue-generator-unleashing-the-potential-of-data-teams) * [4 ways to improve data quality | Secoda](https://www.secoda.co/blog/4-ways-to-improve-data-quality) * Webinar: [Mastering cost containment for modern data teams](https://www.youtube.com/watch?v=1R8y8LZRJCU) * [Snowflake Costs](/integrations/data-warehouses/snowflake-integration/snowflake-costs) # Tool migrations using Secoda Migrating from one tool to another? Check out the following best practices to ease in the transition. ## Introduction Migrating tools within your technology stack can be challenging, especially when ensuring continuity and data integrity. Secoda provides a robust platform that supports smooth transitions by allowing you to handle migrations without losing critical information or disrupting existing workflows. Here’s a guide on how to leverage Secoda effectively during tool migrations. ## Case Study Reference Before starting your migration, it might be helpful to learn from others who have successfully managed similar transitions. We recommend reviewing our case study on a customer’s experience with tool migration: [Textus Tool Migration Case Study](https://www.secoda.co/customers/textus-case-study). This case study provides valuable insights into managing the transition effectively using Secoda. ## Migration Strategy #### 1. Maintain Both Tools During Transition * **Dual Integration**: Initially, keep both the legacy and the new tool integrated within Secoda. This approach prevents any loss of dependencies and maintains data continuity throughout the migration phase. #### 2. Documentation and Lineage * **Transfer Documentation**: Utilize Secoda’s ability to seamlessly transfer documentation from the legacy tool to the new one. This not only saves time but also helps maintain the consistency and accuracy of data documentation across platforms. Use [Automations](/features/automations) to propagate the existing documentation to the new sources with the same names. * **Visualize Changes**: Take advantage of Secoda’s visualization tools to view [lineage](/features/data-lineage) and compare the old and new setups side by side. This helps in understanding the impact of the transition and assists in validating the migration process. #### 3. Completing the Migration * **Remove Legacy Integration**: Once you are confident that the new tool is fully functional and all necessary data has been migrated successfully, you can safely remove the integration of the legacy tool from Secoda. This final step helps in decluttering the workspace and focusing on the new tool. ## Benefits of Using Secoda for Tool Migrations * **Continuity and Integrity**: Ensures that all dependencies are preserved during the transition, thereby maintaining the integrity and continuity of data. * **Efficiency**: Reduces the manual effort required in transferring documentation and other metadata, making the migration process more efficient. * **Clarity and Oversight**: Provides clear visibility into both old and new setups during the transition, which helps in better decision-making and reduces the risk of errors. ## Conclusion Tool migrations are complex but planning with the right strategies and tools like Secoda can simplify the process. By following these best practices, you can ensure a smooth transition with minimal disruption to your operations. # Slack <> Questions workflow Best practices for using Slack to answer user questions The Slack integration with Secoda allows users to stay in Slack to ask questions about data. We've talked with many Secoda users and put this page together to highlight some best practices on how to implement this workflow at your organization. ### Initial questions to consider 1. Is there a centralized Slack channel that your colleagues go to when asking questions about data? 1. If yes, consider connecting the [Slack integration](/extensions/slack-connection) to this channel. 2. If no, work with the relevant stakeholders to create this channel (or delete other channels so this knowledge is centralized) and socialize it to your organization. 2. Who normally has the most questions about data? 1. Make sure that these users are included in the connected Slack channel. 3. Who normally has the answers about data? 1. Make sure that these users are included in the connected Slack channel. 4. Have questions to the data team been managed elsewhere in the past? Would it make sense to bring historical questions and requests into Secoda so that they are searchable? 1. Check to see if we integrate with your tools for managing this under [Integrations](/integrations), or reach out to Customer Support to add a native integration ### Set up the centralized channel * Consider testing the functionality out in a private channel with a smaller subset of users * Follow the steps to create the connection here [Slack](/extensions/slack-connection) * Enable the AI Assistant within your workspace, which will allow it to generate responses directly in a Slack thread - steps here [Secoda AI](/features/ai-assistant#set-up) ### Communicate what can be done * Enable users to get the most out of the integration by providing them with documentation - Capabilities are outlined in this doc [Slack user guide](/extensions/slack-connection/slack-user-guide) * Share tips in the channel to remind users about what can be done, what questions can be asked etc. [Tips and tricks to share with new users](/readme/secoda-as-an-admin/user-engagement-and-adoption/tips-and-tricks-to-share-with-new-users#slack-integration-tips) ### Set rules and standards These will be different for all organizations, but some could include: * If the AI provides an accurate answer, make sure to check off **Answered** so that the question and answer can be pushed into Secoda so that they're searchable in the future * If an important thread happens in a separate channel, make sure to manually push this into Secoda following[ these steps](/extensions/slack-connection#push-slack-thread-into-secoda-questions) * Tag users/Secoda admins who might have the answer so that they can hop in the thread to Approve or Deny the AI-generated answer, as well as add additional context * Set SLAs so that all questions are responded to and answered correctly, either by the AI or a colleague, in a certain timeline * Designate someone on the data team to be "on-call" who monitors the Slack questions coming in, and ensures that SLAs are followed * If a colleague answers the question in the thread, consider adding what you've learned to the documentation in Secoda * Consider typing out some frequently asked questions and answers into the [Questions](/features/ask-questions-in-secoda) section in your workspace, so that these are easily picked up by the AI within Slack # Defining resources workflow Best practices around collaborating with your team to define resources in your workspace ## Introduction Defining resources accurately and consistently is essential for effective data governance. This workflow is designed to guide teams in properly defining any type of resource, such as metrics, business terms, documents, and tables, ensuring they meet our organizational standards for accuracy and reliability. Secoda users can utilize the Questions feature in the UI to align on defining resources. This document outlines the best practices that we've seen work best when identifying and defining resources, then marking them as [Verified](/resource-and-metadata-management/tags/verified-tag). ## Workflow Overview 1. **Initiate a Question**: Initiate the process by creating a [Question](/features/ask-questions-in-secoda) in the UI about the resource you wish to define. This could be a metric, a business term, a document, or any other data resource. For example: * Metric: "How do we define and calculate this metric?" * Dictionary term: "What does this term mean within our context?" * Document: "What are the usage guidelines for this document?" * Catalog resource: "Which version of this data is most accurate and up-to-date?" 2. **Collaboration and Discussion**: Engage relevant stakeholders by tagging them in the comment section of the question. Facilitate a discussion where all parties can contribute their insights and knowledge. 3. **Consensus**: Reach a consensus based on the discussion and mark the definition as the correct Answer. 4. **Resource Creation**: Formalize the definition by creating or updating the resource with the agreed-upon definition or other metadata. 5. **Linking to Documentation**: Link the discussion from the Questions feature to the resource as a [Related resource](/resource-and-metadata-management/relating-resources) for auditing capabilities. This linkage serves as a historical record of the decision-making process. ## Verification of Defined Resources After a resource is defined, it can be marked as Verified to further assure its reliability: * **Verification Process**: Tag the resource as 'Verified' to indicate that it has undergone a thorough review process and is deemed a reliable source of truth. * **Importance of Verification**: This tag helps end-users understand that the resource is not only defined but also validated by the team, reflecting a significant investment in ensuring its accuracy and relevance. By following this workflow, we ensure that our resources are not only defined with precision but also verified for their trustworthiness, enhancing the overall data governance across the organization. ## Video resource **Check out the video resource showing this workflow!** {% embed url="" %} # Streamline data access: Private and public teams workflow Check out this workflow for managing metadata access in your workspace by using Teams. Hoping to set up your workspace so that only the right users have access to work-in-progress resources? So that business users are only viewing the most up-to-date, Verified, resources? Consider setting up [Private and Public Teams ](/user-management/teams)and only flow resources into a Public team once they are approved and tagged as "Verified". Viewers can still search and discover resources in that Private team, and request access to the metadata. ## How to set this up 1. Set up a [Private team](/user-management/teams#team-settings) that only certain users have access to * A Private team means that users must be invited by an Admin to accessing the resources in this team. * This Private team should be shared with your data team, the Admins/Editors in your workspace, SMEs, or whoever you believe is approved to viewing and editing the "unpolished" data resources. * Add all of your new, "Draft", or work-in-progress resources into this team. 2. Set up a Public team that all users have access to * A Public team means that anyone can join this Team and access the resources added to it. * Add all users, including end-users, to this team. * Add all of your "Published"/Verified resources to this team, so that users know that these are approved by your team and ready to be used. 3. Automations can help automatically move resources to the Public and Private teams, instead of doing this manually. Here are some examples: 1. **Automation 1**: All new resources coming into Secoda through the integrations get added only to the Private team * This ensures that all new resources are getting approved before being exposed to end users * ```
``` * The users in that Private Team can edit the resource documentation and complete a[ verified workflow](/best-practices/verifying-resources-workflow) (optional) to verify the resource. 2. **Automation 2:** All resources that have a description and owner set, should be tagged as Verified, marked as Published, and get added to the Public team for end-user consumption. * Note: You can choose to define Verification differently than "having a description and owner", this is just an example. * This automation ensures that resources are tagged as Verified, Published, and made accessible to business users in the Public team. * ```
``` # Exposing Secoda to external clients Best practices for exposing Secoda to your external customers or clients Several of Secoda's users use Secoda as a tool to organize data for their own company's external customers. In these cases, they would not want their customers to have any awareness of one another for security reasons. There are a few best practices you may want to follow to ensure that users are only seeing the content and usernames specific to their organization. ## Setting up Teams for your clients If set up in the following way, the affected users will: * Only be able to search for or view **resources** within the team * Only be able to search for or view **users** within the team * Not be able to search for or view any other **teams** within the workspace 1. Create a Private Team for each external customer * [*Read up on how to make a Team Private*](/user-management/teams#creating-teams)*.* 2. User Role Configuration * **Assign 'Guest' Role**: In the [Members Settings](https://app.secoda.co/settings/members), set the user's roles to Guest when adding them to Secoda. * *Guests in Secoda will only see usernames of the Team that they are apart of.* * *Note: All other roles will get pulled into the default Team (General), even if it is Private. For enhanced security, ensure these users are Guests.* * **Add Users to Teams**: Add the relevant users to their corresponding Private team(s) in the Teams Settings. * Consider setting up [Groups](/user-management/groups) for your clients. 3. Resource Management * **Add Resources**: Add relevant resources into the respective Private Teams. * **Search Visibility**: Resources in Private Teams are hidden from search results for users outside that Team. * **Automations**: Use *an* [*Automation*](/features/automations) for bulk resource allocation. * *Note: Adding a parent resource (e.g., a database table) automatically adds child resources (e.g., columns in the table) into the Team.* By following these guidelines, you can effectively manage data access for external customers using Secoda, ensuring that each customer's information remains secure and isolated. # Resource management This section will guide you in editing and improving the context for your data resources in your workspace. ## **Getting started with resource management in Secoda** Secoda ingests metadata and resources from your data warehouse, BI tool and ETL processes. After the metadata has been extracted, Secoda gives data teams an interface to document and manage their metadata in the same place that the rest of their data knowledge lives. This section will highlight the different ways that data teams can manage, document and share metadata and resources in Secoda. ## What is a resource? A resource in Secoda encompasses any entity that can be enhanced with metadata and properties. This broad category includes tables, dashboards, charts, columns, queries, documents, dictionary terms, and more. Most resources can be accessed through their own dedicated page, which typically features tabs for Lineage and Documentation, alongside a side panel displaying the resource’s metadata and editable properties. For a breakdown of the information on the side panel, check out the page below. {% content-ref url="/pages/rnGgtdxWtF6SkaJNV8wz" %} [Resource sidesheet](/resource-and-metadata-management/resource-sidebar) {% endcontent-ref %} ### What is resource metadata? Metadata in Secoda refers to details about the resource that are automatically captured and cannot be modified. This includes information such as creation timestamps, resource popularity, and the resource type. Such information is crucial for understanding the lifecycle and usage of each resource within your organization. ### What are resource properties? Properties in Secoda are editable attributes that provide additional context about a resource. These attributes can be manually entered in Secoda or pulled directly from the source. Editable properties include descriptions, documentation, ownership details, and custom properties, which offer a deeper level of customization and control. **Metadata and properties are the lifeblood of Secoda, the more you put in, the more you'll get out of the tool!** To edit properties in bulk, check out our **Editing Properties** section below! {% content-ref url="/pages/4Y2OXBNJD3cMSQqiM2QF" %} [Editing properties](/resource-and-metadata-management/add-documentation) {% endcontent-ref %} If you're looking to add some custom properties or tags, the pages linked with walk you through that process! {% content-ref url="/pages/iQs4FYIStImAZkgvGK4d" %} [Custom properties](/resource-and-metadata-management/adding-custom-properties) {% endcontent-ref %} {% content-ref url="/pages/77pbFgPU5slwO85cdnxC" %} [Custom tags](/resource-and-metadata-management/tags/custom-tags) {% endcontent-ref %} Learn how to assign owners to your resources, or relate resources to one another below! {% content-ref url="/pages/NtSFGovuz6cl8Bbg8pq1" %} [Assigning owners](/resource-and-metadata-management/assigning-owners) {% endcontent-ref %} {% content-ref url="/pages/d5lyHHmT58rSkupZ1aqw" %} [Related resources](/resource-and-metadata-management/relating-resources) {% endcontent-ref %} For changes that require manipulation outside of Secoda, you have the flexibility to export your resources to a CSV file, make offline edits, and then re-import them to apply updates across your Secoda environment. Detailed instructions on this process can be found in the corresponding section below. {% content-ref url="/pages/p0WoIv6Yrfy1m8MuDTFd" %} [Import and export resources](/resource-and-metadata-management/import-and-export-data) {% endcontent-ref %} # Editing properties After you've connected your data to Secoda, you can start adding documentation and metadata to your data to enrich it for your users. ## **Why enrich your data?** * **Clearer insights:** Makes data easier to understand and trust, so everyone can make better decisions. * **Streamlined governance:** Defines clear responsibilities, ensuring data is used correctly. * **Boosted collaboration:** Helps teams easily share and utilize data. * **Stronger security:** Protects sensitive information and controls who can access it. ## How to add context to your data Secoda integrates with your data sources to automatically bring in existing descriptions for tables, dashboards, and columns. For instance, if you connect your dbt YAML files, Secoda will sync the descriptions directly to the corresponding columns and tables. ### Automating documentation Automatically add property documentation by leveraging some best practices within Secoda:[Documentation best practices](/best-practices/documentation-best-practices). ### Manually add enrichment 1. **Locate your data**: Use the Search feature or navigate directly through the Catalog to find the dataset you want to enrich. 2. **Edit descriptions**: On the dataset overview page, click under the dataset's name to add or update its description. 3. **Enhance documentation**: Navigate to the Documentation tab to add detailed documentation, guides, or any supplementary material. #### Further enrichment options: * **Tagging**: Apply [tags](/resource-and-metadata-management/tags) like PII, Verified, or custom tags to categorize and mark the status of data assets. * **Relate resources**: Create [links](/resource-and-metadata-management/relating-resources) between different datasets to establish and visualize relationships. * **Assign ownership**: Define [ownership](/resource-and-metadata-management/assigning-owners) for datasets to clarify responsibilities and direct queries. # AI description editor Use Secoda's AI Description Editor to efficiently generate and edit resource descriptions. ## Overview The AI Description Editor is a powerful tool in Secoda that integrates seamlessly into your Catalog, Glossary, and Collections. It enables one-click bulk edits to the Description field, significantly accelerating the process of enriching metadata with consistent and relevant content. ## How it works The AI Assistant generates descriptions for your resources by analyzing existing metadata in your workspace. It adjusts to match the tone and style prevalent in your current descriptions, ensuring consistency across the board. You have the flexibility to accept the suggested description, request additional iterations, or directly edit the text as needed. {% hint style="info" %} Admins can further tailor the auto-generated descriptions by providing [Secoda AI](/features/ai-assistant#descriptions-custom-instructions) in the AI Settings. {% endhint %} ## How to add AI-generated descriptions To utilize AI-generated descriptions, ensure the AI Assistant is activated in your workspace. An Admin must enable this feature through [the following steps](/features/ai-assistant#enabling-the-secoda-ai). ### In the catalog The AI can generate descriptions for any resource. Follow these steps: **Individual resource descriptions**: 1. Navigate to the Catalog and select a resource. 2. Click "Add Description" button adjacent to the description field. Select the purple AI editor button to generate a description. 3. You can :ballot\_box\_with\_check:keep the AI-generated description, request a new one by clicking :arrows\_clockwise: Try again, or :x: reject it. 4. See the description filled out automatically! {% embed url="" %} **Bulk description edits**: 1. Select multiple resources by ticking the boxes next to each Column Title, or select all checking the top left button. 2. Choose "Apply AI Description" from the Actions bar. 3. A notification will appear in the corner of your screen showing the progress of the bulk metadata generation. Once completed, descriptions will automatically update in your Catalog. 4. Once complete, the descriptions will automatically appear in your Catalog! {% embed url="" %} If you'd like to add more content to the description, you can simply click into the description to edit them. ### In the glossary and collections This feature can also be used to add descriptions to your Glossary terms and Collections. Simply click into those sections and follow the same steps outlined above! {% embed url="" %} # Bulk editing This section will go over how to edit data resources in bulk- descriptions, columns, tables, badges, tags, etc. ## Introduction Adding documentation manually to one resource at a time can be useful for small amounts of unique data. However, if you have large volumes of data requiring documentation, Secoda offers efficient solutions through both manual and automated bulk editing. ## **Bulk editing** 1. **Navigate:** * Go to any section where bulk edits are needed, such as the Catalog, Collections, Metrics, Dictionary, Documents, or Questions. * For this example, we'll use the **Catalog**. 2. **Filter resources:** * Apply relevant [Filters](/features/filters) to narrow down the resources you want to edit. * For instance, filter by Integration=Tableau and Type=Dashboard to filter on Tableau dashboards. 3. **Select resources:** * **Select all:** Click the checkbox in the upper left corner to select all visible resources. * **Select individually:** Check boxes one by one for specific resources. * **Select a range:** Click the first checkbox, hold the Shift key, and click the last checkbox in the range you want to edit. 4. **Execute actions:** * View all available [#actions-options](#actions-options "mention")by clicking the three-dot Actions button at the bottom of the screen. Options like Set PII and Verify might be suggested based on the context. * Choose the appropriate bulk update action from the menu and confirm your selection to apply changes. ### Actions options * [Propagation](/resource-and-metadata-management/add-documentation/propagating-metadata) * Set Collections * Set tags * Set owners * Set published * Set PII * Verify * Apply AI description * Delete (from workspace) * Remove from collection ### **Video resource** To help you visualize the process of making bulk edits in Secoda, we've included a GIF that covers the essential steps: filtering resources, selecting them, choosing an action, and applying the changes. As you can see below, we've filtered for PII resources, selected all, chose the Verify action, and applied the bulk changes.

Bulk Catalog Edits

## **Bulk editing with automations** * **Set up automation:** Learn how to use Automations in Secoda to handle bulk edits. For instance, you can create an Automation that applies the same definition to all columns named "customer\_id". * **Guide:** [Read up on setting up Automations](/features/automations) to bulk edit resources in Secoda. ## **Conclusion** Secoda’s bulk editing features are designed to simplify the process of updating large data sets, making it easy to maintain consistency and accuracy across your resources. Whether through automation or manual selection, these tools help you efficiently manage and document your data. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](http://app.secoda.co/) 👈 {% endhint %} # Propagation This section will go over how to propagate properties between resources. ## Introduction Secoda offers a robust feature that allows you to copy properties from one resource to related resources within your workspace. This can streamline the management of properties across related resources. If you'd like to Automate this process even further, check out [Rule-based automations](/features/automations/automations-use-cases) for some examples. ## Steps to propagate 1. **Select the resource:** * In the Catalog, identify the resource that contains the properties you wish to propagate. Check the box next to the resource to select it. 2. **Initiate propagation:** * Click the three-dot Actions button. * Choose "Propagate" from the menu. 3. **Choose properties:** * Select the specific properties you would like to propagate from the source resource, like Owners and Tags. 4. **Select target resources:** * Specify which related resources should receive the properties. You can filter for these resources by: * Exact name * Similar name * Downstream lineage 5. **Configure propagation settings:** * You can select to propagate to one, multiple, or all related resources. Similarly, choose to apply one or multiple properties as needed. * Decide whether to override existing properties on the target resources: * **'Override existing properties' checked:** The selected properties will replace any existing properties on the target resources. * **'Override existing properties' unchecked:** Properties will only be added on the target resources that are currently empty. #### Additional options and considerations * **Bulk selection:** If you need to propagate properties to several resources, consider using the 'Select All' option for efficiency. * **Verification:** After propagation, verify that the properties have been correctly applied to the target resources. This can help ensure consistency and accuracy across your data ecosystem. By following these steps, you can effectively manage and synchronize documentation across various resources in your Secoda workspace, enhancing data governance and usability. {% embed url="" %} # Templates Admins can create templates for data questions, documents and glossary terms, and more. ### Overview As an Admin, you can create templates to standardize and streamline various tasks. Here's what you can create templates for: * **Questions**: Ensure all relevant information is provided in team requests. * **Documents**: Make sure users capture all required documentation. * **Glossary Terms**: Enforce inclusion of all necessary context in term descriptions. * **Table, Column, and Dashboard Documentation**: Streamline adding additional context to any resource in Secoda. ### Creating a template 1. Navigate to Settings > Templates 2. Click "Create template" 3. Select the type of resource that the template will be associated with
### Editing a template Depending on the resource type, there are different properties and content you can customize on a template.
**Standard template content**\ This conent can be configured for all templates * Title: The name of the template * Description: Short form content using a limited version of the rich text editor * Documentation: Long form content that can contain rich information **Template properties**\ These properties can be configured on document, glossary, collection, and question templates * Owners * Teams * Collections * Tags * Related Resources **Special Fields** * Synonyms: For glossary and dictionary terms * Priority and Assignee: For questions ### Using a template To apply a template for collections, glossary, documents, or questions, navigate to the list page and click the dropdown menu to select a template.
To apply a template for tables, columns, and dashboards navigate to the resource page then to the Documentation tab and click the "Templates" button.
### Default template You can set a template as the default format for editors and viewers: 1. Click on the three dots beside the template you'd like to set as default. 2. From the menu, select "Set as default." In this menu, you can also deletel templates.
{% hint style="info" %} Not using Secoda to manage your data knowledge yet? Sign up for free [here](https://app.secoda.co) 👈 {% endhint %} # Resource sidesheet Explaining the information in the sidesheet. When looking at a resource, you can click on the Information symbol in the top right to see a sidesheet of relevant information for your resource. ### Properties Properties refer to fields that can be edited. {% hint style="info" %} For some integrations, properties such as tags and owners can be imported from the source. To learn more about how to maintain properties in the source or in Secoda, navigate to the [Property Management](/integrations/integration-settings#metadata-management-this-functionality-is-coming-soon) in the Integration Settings. {% endhint %} **Status**: This refers to whether the resource is Published or a Draft. More information about Publishing can be found [here](/features/publishing). **Tags**: A list of custom tags added to the resource. More information about adding custom tags can be found [here](/resource-and-metadata-management/tags/custom-tags). **Collections**: A list of all the collections that resource is part of. Learn more about [Collections](/features/collections-1) here. **Owners**: A list of [Users or User Groups](/user-management) that are owners of the resource. **Governance**: A tag denoting whether the resource is PII or not. More information about setting Governance Tags can be found [here](/best-practices/data-governance). **Verification**: A tag denoting whether the resource is Verified or not. More information about setting Verification can be found [here](/resource-and-metadata-management/tags/verified-tag). **Teams**: A list of [Teams](/user-management/teams) that the resource can be found in. Any custom properties will also show up in this section. Learn more about Custom Properties [here](/resource-and-metadata-management/adding-custom-properties). ### Metadata Metadata refers to fields that are uneditable in Secoda. Use the toggle to see the Metadata generated in Secoda, or in the integration. See below for explanations of metadata: **Row count:** If available, this field indicates the number of rows in that table. **Table size:** If available, this field indicates the digital size of the table. **Popularity:** The [Popularity](/features/popularity) field refers the number of Queries run on the resource (if it's a table) or the number of Views received by the resource (if it's a dashboard) in the last 24 hours, relative to the time of the last sync. For instance, if a Snowflake sync happened 3 days ago, Popularity will show the number of queries run on that table in the period from 4 days ago to 3 days ago (24 hours before the time of the sync). **Views**: The number of views of the resource in Secoda. ### Subscribers This is a list of all the users that are subscribed to get notifications about this resource. Learn more about [Notifications](/features/notifications) and [Subscribers](#subscribers) here. # Assigning owners Assigning owners to resources ## **Benefits to assigning ownership** Assigning ownership to data provides several key benefits: * **Clarity on Impact:** Easily identify who will be affected by changes to a dataset, whether those changes are upstream or downstream. * **Communication Efficiency:** Know exactly who to contact for information about specific data. * **Change Notifications:** Owners receive updates when datasets undergo changes. * **Data Management:** Quickly identify and address datasets that are abandoned and may need maintenance or removal. ## **How to assign owners to data in Secoda** After navigating to a Secoda resource, Editors and Admins can assign Owners by clicking the Owners field and selecting the relevant members from the workspace. The owners field is accessible through the Catalog view, or in the [Resource sidesheet](/resource-and-metadata-management/resource-sidebar). Owners can be assigned to data within Secoda by Editors and Admins: 1. Navigate to a resource within the Secoda Catalog or use the resource side panel. 2. Click on the Owners field. 3. Select relevant member(s) or a Group from the workspace. ### Group owners Ownership can also be assigned at the Group level: * Create [Groups](/user-management/groups) within the settings. * Search for the Group name when adding an owner. Benefits: Group ownership ensures consistent management even when individual members leave the organization.

Owners in Catalog view

Owners in the side panel

## Automating ownership Please see the [Integrations](/integrations)docs to see which integrations automatically bring in ownership metadata. Create an [Automations](/features/automations) to define ownership and have these fields automatically update on a set schedule. Try out the [Propagation](/resource-and-metadata-management/add-documentation/propagating-metadata) & [Bulk editing](/resource-and-metadata-management/add-documentation/bulk-editing-resources) features to manually bulk add owners to your resources. ## Responsibilities of owners Owners play a crucial role in maintaining the integrity of data: * **Documentation:** Keep all related documentation up to date. * **Queries and Communication:** Address questions posted in the Questions tab of a dataset. * **Notifications:** Receive notifications through email, the Secoda app, and, if integrated, through Slack. * **Engagement:** Add comments or FAQs to datasets to enhance the documentation available to all users. By effectively managing ownership, organizations can ensure data remains reliable, up-to-date, and valuable. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](http://app.secoda.co/) 👈 {% endhint %} # Custom properties Custom properties allow you add additional context to resources in Secoda. ## Overview Custom properties allow you to enrich the metadata of your resources within Secoda, providing additional context and enhancing organization. This guide outlines the methods to add custom properties to individual resources and in bulk. ## (New) Adding custom properties to the catalog To add a custom property to all resources by resource type, you can follow these steps. 1. Go to Settings > Properties. 2. Click the "Create property" button. 3. Select the label, type, and resource types that you'd like the property to be applied to. 4. Review your property and click "Confirm". After this is completed, the custom property will appear up in the Catalog table and the pages for the visibility type(s) selected.
## Bulk updating (New) custom properties To efficiently update custom properties to multiple resources you can use the **Import CSV** functionality or the **API**: 1. **Import CSV file:** Ensure the custom properties are visible on the table you want to bulk update. Then click "Export page as CSV". This will be your template. Open the CSV export and modify the columns containing custom properties. "String" custom properties can contain up to 255 characters of text. "Select", "User", or "Resource" custom properties must contain an array of (v4) uuids. Then click "Import custom properties from CSV" and select the file you modified.

Export page as CSV

Sample columns in CSV file

2. **Use the API:** Create a custom script to update the custom properties. {% content-ref url="" %} {% endcontent-ref %} *** ## (Legacy) Adding custom properties to the properties sidebar This feature is considered **legacy**, and we recommend adding custom properties to the catalog (above). 1. **Access the resource:** Navigate to the specific resource (document, table, column, etc.) within Secoda. 2. **Add the property:** Use the "+ Add Property" button located on the right side panel. 3. **Choose property type:** Select from Text, Checkbox, User, Tag, Resource, Date, Number, Select, and Multiselect Property types. 4. **Edit the property:** Click into the value to edit the Property. 5. **Delete the property:** Click into the Property type to Delete it.

Creating a Property

Editing & Deleting a Property

*** By following these instructions, you can tailor the metadata within your Secoda environment to meet your specific organizational needs, enhancing both the utility and accessibility of your data resources. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](http://app.secoda.co/) 👈 {% endhint %} # Tags Create Tags in Secoda to label resources across the workspace with additional metadata. ## What are Tags? Tags are a static way to label Resources with useful context that other users in your workspace need to know. Secoda has two built-in default tags, [PII identifier](/resource-and-metadata-management/tags/auto-pii-tagging) and [Verified identifier](/resource-and-metadata-management/tags/verified-tag). To add additional tags to your workspace, you can create [Custom tags](/resource-and-metadata-management/tags/custom-tags) based on how you'd like to categorize data in your organization. ## Bulk Tagging There are a few ways that you can tag your resources in bulk. Create an Automation to tag similar resources [Automations](/features/automations). Use the [Bulk editing](/resource-and-metadata-management/add-documentation/bulk-editing-resources) feature in the Catalog to drag a Tag down to add it to multiple resources at once (similar to the click and drag feature in Excel and G-Sheets). You can also bulk tag using our [Propagation](/resource-and-metadata-management/add-documentation/propagating-metadata) feature in the Catalog. If there's a tag that you'd like to propagate to other resources with the same or similar name, or in the resource's downstream lineage, you can do so using this feature. ## Filtering for Tags You might want to filter your search for resources with the same Tag. You can do that by clicking `Add filter` within the Search window. {% embed url="" %} # Custom tags Creating and managing custom tags in Secoda to help categorize your resources. ## How to Create Custom Tags 1. Navigate to Workspace settings by clicking on the workspace name in the top left of the UI, and selecting "Settings." 2. Under Workspace settings, select `Tags`. 3. Click the `Create tag` button. 4. Enter Name, an optional Description, and choose a colour to represent the tag.
## Group Tags For organizations that use tags extensively and require further categorization: 1. **Create a Tag Group:** * Click 'Create group'. * Name the group and add relevant tags. 2. **Organize Existing Tags:** * To add an existing tag to a group, use the three-dot menu on the tag and select '-> Move tag to group'.

Group Tags

## Deleting Tags You can delete a Custom Tag by selecting the three dots and clicking **Delete** on the same Settings page. **Note:** This action cannot be undone. A warning will appear to confirm your decision before permanently deleting the tag.

Deleting Tag Warning

## Use cases for Tags Tags are mainly used for categorization purposes, opposed to Collections which function more like folders. Tags can help differentiate between types of data, track resource status, or indicate data sensitivity. **Common Examples of Tags:** * **Project Status:** * "Draft" – Work in progress; details may be incomplete. * "In Review" – Undergoing evaluation by stakeholders. * "Deprecated" – Outdated and should not be used. * **Data Type:** * "Production" - Live data * "Test" – Testing environment * **Importance Levels:** * "Level 1" – Highest business impact. * "Level 2" – Moderate business impact. * "Level 3" – Lower business impact. * **Business Context:** * "SOPs" – Standard operating procedures. * "Critical Data" – Essential business data elements. * "Key Account" – High-value customer accounts. * **Security Classifications:** * "Confidential" – Restricted access required. * "Restricted" – Limited access. * "Public" – Openly accessible. * "HIPAA Compliant" - Indicates that the data meets the standards of the Health Insurance Portability and Accountability Act. * "GDPR Sensitive" - Denotes data subject to the General Data Protection Regulation, requiring specific handling procedures. * **Project Management:** * "Urgent" - Tasks or resources that require immediate attention. * "High Priority" - Important tasks that are critical to project success but less immediate than urgent items. * "Low Priority" - Tasks that are on the project radar but can be addressed later. By effectively using custom tags, teams can enhance their workflow management, data organization, and security protocols, ensuring that all stakeholders have clear visibility into the status and classification of resources within the workspace. # PII identifier Secoda's PII identifier finds PII columns, tables or dashboards in your data in Secoda. PII (Personally Identifiable Information) tags are used to identify data assets that contain personal information that could be used to identify an individual. PII tags can be used to help organizations protect personal data and comply with data privacy regulations such as GDPR and CCPA. ## **How to Tag PII in Secoda** {% hint style="info" %} You can automate this process by creating an [Automations](/features/automations). Check out the template in the UI to get started! {% endhint %} You can easily find and tag PII data in Secoda. To do so, go to [**Settings**](/readme/secoda-as-an-admin/settings) **-> Features -> PII scanning**
Once you're in the PII scanning tab, you can see all the tables and columns that Secoda has identified that may contain PII data. You can select or deselect any of these columns or tables before applying the PII tag. Secoda identifies PII columns based on a set of keywords that match column names. Some examples include first name, last name, address etc. If you'd like to customize these key words, you can do so by clicking on the settings button by the PII columns. Note, keywords are case and space sensitive. ## Governance Metadata After tagging your data with the PII tag, these will populate in the **Governance** metadata column in your Catalog.
## PII and Previews Secoda offers the ability to [Preview](/features/data-previews) the first 50 rows of a table. If permissions are granted for Preview, any column of the table that is marked as PII **will not show** as part of the Preview. We recommend setting up the permissions for each Integration according to the governance policies for your organizational structure. Preview and Queries can be disabled for the workspace, or available to specific User Groups/Users. More information about how to set up permissions for Queries and Previews can be found in their respective documentation. Information about roles within Secoda can be found [here](/user-management/roles). ## Removing PII Tags There are several ways to remove a PII tag: 1. You can click the PII value and select "Not PII". 2. You can select a number of resources (e.g., columns) in a table view with the checkmarks, choose the "Set PII" action, and select "Not PII". 3. You can use Automations with the edit action "Remove PII". ## Benefits of Using PII tags 1. **Enhanced data privacy**: By using PII tags, organizations can identify and protect personal data, helping to ensure that it is only accessed by authorized personnel and used in accordance with data privacy regulations. 2. **Improved data governance**: PII tags can help organizations establish clear policies and procedures for handling personal data and ensure that they are being followed. This can improve data governance and reduce the risk of data breaches or privacy violations. 3. **Enhanced data security**: PII tags can help organizations identify and secure personal data, reducing the risk of unauthorized access or misuse. 4. **Improved compliance**: By using PII tags, organizations can demonstrate compliance with data privacy regulations and reduce the risk of fines or other penalties. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](http://app.secoda.co/) 👈 {% endhint %} # Verified identifier Verified tags in Secoda are another way to categorize your resources by giving them a "stamp of approval". Secoda's Verified tag influences search ranking and is a good indicator for end users that the verified data has been approved by relevant stakeholders. This gives those users confidence is using these resources. Check out our best practices documentation for Admins and Editors who are interested in implementing a [Defining resources workflow](/best-practices/verifying-resources-workflow). ## Benefits of Verifying Resources Verifying data through verified tags in Secoda can provide several benefits to organizations: 1. Improved data quality: By verifying data through verified tags, organizations can ensure that their data assets are of high quality and can be trusted by others. This can help reduce the risk of errors or inconsistencies and improve the overall quality of the data. 2. Enhanced data governance: Verified tags can help organizations establish clear standards for data quality and ensure that data is being used appropriately. This can help improve data governance and ensure that data is being used in a responsible and ethical manner. 3. Improved data discovery: Verified tags can influence search results, causing verified assets to rank higher in search results. This can help users find and access high-quality data more easily, improving productivity and efficiency. 4. Enhanced trust and credibility: By verifying data through verified tags, organizations can demonstrate to others that their data is of high quality and can be trusted. This can help enhance the credibility and trustworthiness of the data and the organization as a whole. Overall, verifying data through verified tags in Secoda can help organizations improve data quality, enhance data governance, improve data discovery, and enhance trust and credibility. {% hint style="info" %} You can use the Verified tag on Catalog resources, Documents, Dictionary terms and Metrics. {% endhint %}
# Import and export resources Edit the properties or bring in new resources through this feature. ## Overview This guide provides a step-by-step process for exporting and importing metadata using CSV files in Secoda. This allows for bulk editing and updating of metadata to ensure consistency and prevent duplication across your data resources. {% hint style="info" %} If your goal is to introduce net new resources from an integration without connecting the integration in the app, you should utilize the [Custom Integrations](/integrations/custom-integrations-and-marketplace/custom-integration) feature. If you are looking to edit existing resources, such as glossary terms, documents, etc., the following instructions will guide you through the process. {% endhint %} ## Video resource {% embed url="" %} ## E**xporting resources from Secoda** To export resources from Secoda: 1. **Navigate to settings:** Go to [Settings](https://app.secoda.co/settings/import) → Import & Export data. 2. **Select resource type:** Choose the type of resource you wish to export. 3. **Download the export:** Once the Artifact is prepared as a Bulk export of your resources, click the "Download Artifact" button.
Below is an example of the results from exporting your data:
## Preparing a CSV for import Before importing new or updated metadata into Secoda: 1. **Edit your CSV:** After exporting, you can edit your descriptions and other properties in CSV format or Google Sheets. 2. **Set up your CSV:** Use the exported data as a base, the provided template in the UI, or create your own CSV adhering to the required structure: **Required columns:** * **id:** Leave this blank for new resources. * **title:** The name of the resource. * **entity\_type:** Possible values include table, column, dashboard, chart, job, glossary, document, collection, question, or event. **Optional columns:** * **description:** A brief description of the resource. * **definition:** Markdown-friendly documentation of the resource. * **pii:** Governance tag (TRUE or FALSE). * **verified:** Verified status (TRUE or FALSE). * **published:** Published status (TRUE or FALSE). * **collections:** List of associated collection names, e.g., \['Marketing', 'Engineering']. * **owners:** List of associated owner emails that should correspond to existing users in Secoda, e.g., \['']. * **tags:** List of associated tag names, e.g., \['production']. * **tables:** List of tables that need to be updated, e.g., \[Team 1; Team2;] ## Importing updated metadata into Secoda 1. **Access import settings:** Log into Secoda, navigate to Settings > Import & Export. 2. **Upload CSV file:** Click "Select File" under the Import section, choose your prepared CSV file, and click "Upload". Monitor the import process through the displayed logs.
## Verifying an import After importing, it's crucial to verify that all properties have been correctly updated: 1. **Check data resource:** Visit a data resource within Secoda to confirm that the properties reflect the recent imports. 2. **Resolve issues:** If there are discrepancies or if you encounter issues, please contact . By following these steps, you can efficiently manage the metadata of your resources in Secoda, ensuring data integrity and streamlined operations across your organization. {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](http://app.secoda.co/) 👈 {% endhint %} # Related resources This section will go over how to relate resources. Secoda is a powerful knowledge management platform that allows users to organize and access their resources in a centralized location. One key feature of Secoda is the ability to link resources to each other using the "Related Resources" function. The Related Resources function in Secoda is a powerful tool for building knowledge networks. By linking resources together and creating relationships between them, you can create a web of interconnected knowledge that allows you to explore different topics and ideas in a more holistic way. ## How are related resources created? #### Automatically * Secoda identifies resources within the same lineage graph and tags them as "Related" * If another asset is referenced by using the @ tagging feature, Secoda automatically establishes a link between the two resources, indicating their relationship as "Related." #### Manually * Users can manually add Related resources by clicking into a resources and editing the metadata
{% hint style="info" %} Note: For resources within the catalog section, the related property is shown as a tab on the resource page {% endhint %} Overall, the Related Resources function in Secoda is a powerful feature that allows you to link resources together and create relationships between them. By using this feature, you can build knowledge networks and explore different topics and ideas in a more comprehensive way. # User management This guide details the setup and management of Roles, Groups, Teams, and Collections in Secoda, ensuring secure and efficient access control across the platform. ## **Introduction** Role-based access control (RBAC) in Secoda is designed to ensure that only authorized users have access to specific resources, protecting sensitive information and enhancing platform functionality. This guide outlines how Secoda utilizes Roles, Groups, Teams, and Collections to manage access and permissions efficiently. Permissions in Secoda are inherited hierarchically. When you grant someone access to a Team, they automatically gain access to all resources within that Team. Similarly, giving someone access to a Collection grants them access to all resources within that Collection. If you'd like to limit resources access at an individual level, check out [Sharing](/features/sharing-resources). ## **Teams** Teams are made up of individual members or groups and serve as the primary way to assign access to resources within Secoda. You can customize Teams to fit specific needs: * **Resource access:** Define what resources a team can access, ranging from comprehensive (like multiple integrations) to selective (such as a single column). * **Feature visibility:** Customize the end-user experience by toggling off specific features like Metrics or Catalog visibility. This helps streamline the interface for team members, making it simpler and more focused on their needs. * **Types of teams:** Teams can be Public, open to any user, or Private, requiring an invitation to join. For more details on teams, see [Teams](/user-management/teams). ## Roles Each member in Secoda is assigned a role that defines what they can do: * **Viewer:** Can view metadata and resources within the Teams they belong to but cannot make changes or add new content. * **Editor:** Can edit metadata, add new content, and modify existing resources within any public Team where they hold editor permissions. * **Admin:** Controls the entire platform, managing roles, settings, and all content across all Teams. {% hint style="info" %} **Role Customization at the Team Level:** In Secoda, you can further customize access by adjusting a member’s role within specific teams. For example, a member might be an Editor at the organization level but restricted to a Viewer role in a sensitive team. This flexibility allows for precise control over who can view and edit resources within each team, enhancing security and compliance. {% endhint %} For more details on roles, see [Roles](/user-management/roles). ## **Groups** Groups are sets of members who share common access needs, simplifying the management of permissions. Groups can also be used to set Ownership of resources at the Group level, instead of at an individual level. For more details on groups, see [Groups](/user-management/groups). ## **Collections** Collections are folders within a Team that hold resources related to a specific function, theme or department. For example, a "Revenue" collection within the Finance Team might include all necessary reports, tables, definitions, and documents about revenue. For more details on collections, see [Collections](/features/collections-1). ## **Practical example: Role, team, group and collections dynamics** Imagine onboarding both the Business Intelligence and Marketing departments into Secoda. You'd like each team to have a dedicated space to develop and manage resources essential to their daily operations. Here's how to set it up efficiently: * **Team creation:** Create the Teams and decide whether or not users should freely be able to join any Team, by setting them as Public or Private. * **Group creation:** Form Groups like 'Marketing Editorial Group' (Editors) and 'Marketing Business Users' (Viewers) based on the Marketing team's needs and assign members to them appropriately. Add these groups to the Marketing team. * **Inter-team collaboration:** Suppose Marketing Editors frequently collaborate with the BI team, requiring access to certain BI dashboards. You could: * Add the Marketing Editors Group to the BI team, but alter their permissions within that Team to be Viewers. * For ease of discovery, the BI team could gather a folder of the key BI dashboards into a Collection in their Team for the Marketing Team's consumption. This setup ensures everyone has the appropriate level of access while maintaining tight control over important data. ## Video resource {% embed url="" %} # Roles ### Introduction[​](https://datahubproject.io/docs/authorization/policies#introduction) Secoda provides the ability to declare fine-grained access controls through Custom Roles. Some examples of roles that can be created are: * Table Owners should be allowed to edit documentation, but not Tags. * Jenny, our Data Steward, should be allowed to edit Tags for any Dashboard, but no other metadata. * John, a Data Analyst, should be allowed to edit the Related Resources for a specific Data Pipeline he is a downstream consumer of. * The Data Platform team should be allowed to manage users and groups, view platform analytics, and manage roles. {% hint style="info" %} **Custom Roles** is available to customers on [**Enterprise** **plan**](https://www.secoda.co/pricing)**.** Interested in upgrade options? Click on the "Upgrade" button on the Members and Permissions page under Settings, or reach out to the Customer team (). {% endhint %} ### What is a Custom Role?[​](https://datahubproject.io/docs/authorization/policies#what-is-a-policy) Custom roles allow workspace administrators to create tailored permission sets that go beyond Secoda's default roles (Admin, Editor, Viewer, and Guest). With custom roles, you can define precise access levels for different teams and use cases within your organization. ### Understanding Custom Roles Custom roles provide granular control over: * Resource access (tables, dashboards, documents, etc.) * Feature permissions (API access, monitoring, automation, etc.) * Administrative capabilities Unlike default roles which have predefined permission sets, custom roles let you: * Choose specific permissions for each feature * Set different access levels for different resource types * Create role-based access control (RBAC) that matches your organization's needs ### Creating a Custom Role To create a custom role: 1. Navigate to Settings > Members and permissions 2. Click on the "Roles" tab 3. Select "Create Role" 4. Provide: * Role name * Description * Select permissions for each feature category #### Permission Categories Custom roles can be configured with permissions across several categories: * **User Management** * Users: Create, update, read, or delete users * Groups: Manage group memberships and settings * Roles: Create and modify roles * **Resource Management** * Read: View resources and their metadata * Write: Edit resources and their properties * Manage: Full control including deletion. This includes management of properties including description, owner, tags, verified, etc. * **Settings** * Workspace: Configure general workspace settings * Security: Manage SAML and security settings * API Keys: Generate and manage API access * Properties: Configure custom properties * Billing: Access billing and subscription settings * Import/Export: Manage data imports and exports * Appearance: Customize workspace appearance * **Features** * AI Assistant: Configure and use Secoda AI * Quality Score: Manage data quality metrics * Questions: Create and manage Q\&A * Automations: Set up automated workflows * Monitors: Configure data monitoring * Views: Create and manage custom views * Analytics: Access usage analytics * Queries: View and manage queries * Lineage: View and edit data lineage * Tags: Create and manage resource tags * Collections: Organize resources in collections ### Best Practices 1. **Principle of Least Privilege**: Grant only the permissions necessary for each role 2. **Document Role Purposes**: Add clear descriptions to explain each role's intended use 3. **Regular Review**: Periodically audit custom roles to ensure they align with current needs ### Permissions Out of the box, Secoda is deployed with a set of default Roles. The set of default roles are Viewers, Editors, and Admins. {% hint style="info" %} Manage is all permissions (Create, Update, Delete, and View) {% endhint %} **User Management** | Name | Admin | Editor | Viewer | | ------ | ------ | ------ | ------ | | Groups | Manage | View | View | | Roles | Manage | View | View | | Teams | Manage | View | View | | Users | Manage | View | View | **Resource Management** | Name | Admin | Editor | Viewer | | --------- | ------ | -------------- | ------ | | Resources | Manage | Create, Update | View | **Settings** | Name | Admin | Editor | Viewer | | ----------------- | ------ | -------------- | ------ | | API keys | Manage | Create, Update | None | | Billing | Manage | None | None | | Import and export | Manage | None | None | | Workspace | Manage | View | None | **Features** | Name | Admin | Editor | Viewer | | ------------------ | ------ | -------------- | ------------ | | Analytics | Manage | View | None | | Announcements | Manage | Manage | View | | Automations | Manage | View | None | | Column profile | Manage | Manage | View | | Data quality score | Manage | Create, Update | View | | Integrations | Manage | View | None | | Lineage | Manage | Create, Update | View | | Monitors | Manage | Create, Update | None | | Policies | Manage | Manage | View | | Preview | View | View | View | | Properties | Manage | Manage | View | | Secoda AI | Manage | View | View | | Questions | Manage | Create, Update | Create, View | | Queries | Manage | Create, Update | View | | Tags | Manage | Create, Update | View | | Views | Manage | Create, Update | View | # Teams Teams is user friendly way to view and access your team's data within Secoda. The Teams feature allows users to organize all of their team-specific resources into one space. When a user in Secoda is placed into a Team, they will see all the resources tied to that Team. "General" is the Default Team that all members will be added to. In the example below, the user has access to the "General" Team and the "Finance" Team, as well as the Catalog resources, Collections, Dictionary terms, Documents and Questions that those Teams have access to.
## Creating Teams Only Workspace Admins are able to create Teams within Secoda. **Steps to create a Team** 1. Click on Teams from the left side bar 2. Click "+New Team" 3. Choose a Team Name & Icon and set the Team Permissions The two Permissions options are **Public** and **Private**. A Public Team means that anybody within the Workspace can see, browse and join the Team. A Private Team is hidden and only viewable by Admins and the existing Team members. {% hint style="info" %} Example use case: You might want to create a Private Team if the Team works with sensitive data that only specific employees should have access to. {% endhint %} ## Manage metadata access with Teams Check out our workflow documentation [Streamline data access: Private and public teams workflow](/best-practices/streamline-data-access-private-and-public-teams-workflow) which outlines a potential solution to managing which users have access to which resources in Secoda. ## Joining Teams Joining Teams is simple - navigate over to Teams and see all the Public Teams that you are able to join and/or leave. Watch as the Team is automatically added to your side bar once you join. If you need to join a Private Team, an Admin will need to add you.
## Team Settings ### Customizing Your Team's Sidebar In the Team Settings, Admins can configure which Resources they would like specific Teams to be able to access. In the example below, the Admin wants to make sure that the Sales team can only see the Collections relevant to the Sales team. This makes for a seamless user experience in which the users won't have to go looking for their Team content.
### Editing Member Settings Admins can invite Members or Groups to Public and Private Teams in the Team Settings, as well as remove them. They can also edit user permissions at the Team level here. You can turn any workspace editor into a viewer for a specific Team, for example, and then all of the resources associated with the Team will inherit these permissions. This is a great way to ensure that only the right people are making changes to the metadata. **Note:** All workspace Admins can view/edit any resource across the workspace, so you can't turn an Admin into a Viewer for a particular Team, for example
## Adding Integrations to Teams Admins can pick and choose which Integrations they'd like certain Teams to have access to. Once an Integration (or multiple Integrations) is toggled on, the data resources will populate in the Team's Catalog. **Steps to Add Integrations to Teams** 1. Click into Integrations 2. Click into which Integration you'd like to edit access to 3. Under Basic Settings type in which Team name you'd like to add to Associated Teams and click Submit
### Select subset of resources to be added to the Team You can use an Automation to automatically add a subset of resources to the Team instead of the entire integration. Check out the video here:[Catalog](/features/catalog#limit-resources-in-a-catalog) ## Roles Capabilities Viewers, Editors and Admins within Secoda all have different capabilities when it comes to Teams. These are outlined in the graph below. Note: contact a Workspace Admin if you need assistance with Teams in your workspace.
CapabilityViewer AccessEditor AccessAdmin Access
Join Public Teamtruetruetrue
Leave Public or Private Teamtruetruetrue
View Resources in a Teamtruetruetrue
Add Resources to a Teamfalsetruetrue
Invite members to a Private Teamfalsefalsetrue
Edit Team Settingsfalsefalsetrue
Join Private Team (without an invite)falsefalsetrue
Create a Teamfalsefalsetrue
Associate a Team with an Integrationfalsefalsetrue
## Video tutorial {% embed url="" %} # Groups Organize your members into Groups by team or job function Groups are a way to organize the members within your workspace. Once you organize your team members into Groups, you can then assign them to the appropriate Team which will give them access to all of their resources within that Team. Read more about Teams here: [Teams](/user-management/teams) **Benefits of Creating Groups** * **Resource Ownership**: Assign resource ownership directly at the Group level. For example, assign ownership of all dashboards to the Business Intelligence Group in Secoda. * **Efficient Team Assignment**: Add an entire Group to a Team in one action, bypassing the need to add members individually. ## How to create Groups To create a new Group, click **Groups** in the Organization settings. Then, click **Add new group**.
To add new members to a Group, click **Select user**. You can also add an emoji to represent this group by clicking on the letter beside the group name.
You can assign members to a Group and Team upon sending them an invitation from the Invite members view.
# Integrations This page walks through the current integrations that Secoda supports and what's on the roadmap. Secoda currently integrates with many different tools, see below for the types of tools we integrate with and instructions around how to set up the integration in your Secoda workspace. {% hint style="info" %} **Good to know:** Secoda ships product updates weekly! If you don't see your data tool or source here, message us on Slack or email us at and we'll add it to the roadmap. {% endhint %} {% content-ref url="/pages/bg2sY4xUkWHoeIQtX8jv" %} [Data warehouses](/integrations/data-warehouses) {% endcontent-ref %} {% content-ref url="/pages/jnCXVntfyMhZ3jrxYe7Q" %} [Databases](/integrations/databases) {% endcontent-ref %} {% content-ref url="/pages/oJncKxWeIacb9F4SDQAb" %} [Data visualization tools](/integrations/data-visualization-tools) {% endcontent-ref %} {% content-ref url="/pages/2KDTjV7s5hh3tmCyBzVC" %} [Data pipeline tools](/integrations/data-pipeline-tools) {% endcontent-ref %} {% content-ref url="/pages/zWGlC8zomUay0mec8b8p" %} [Data transformation tools](/integrations/data-transformation-tools) {% endcontent-ref %} {% content-ref url="/pages/WGe6ah0HwjROO4YoxFkN" %} [Data quality tools](/integrations/data-quality-tools) {% endcontent-ref %} {% hint style="info" %} Not using Secoda to manage your data documentation yet? Sign up for free [here](https://app.secoda.co/) 👈 {% endhint %} # Integration settings Customize and manage your integration after setup. On the **Integrations** page, you can easily view and search through all your integrations. You can see details about your integrations including the last and upcoming runs, as well as the status of the most recent run. Additionally, you can use the command palette to run or delete multiple integrations at once. {% hint style="warning" %} Deleting an integration from the workspace will remove all associated resources. Adding the integration back will not preserve the changes (e.g. metadata updates) that were made when originally connected and rather will act as a new integration. {% endhint %} If you click into an integration, the following options become available. ### Enable Integration After selecting an integration, you'll find the **Enabled** toggle in the top right corner. By default, all integrations are enabled. If toggled off, the integration is paused, meaning it will not run automatically based on the schedule set. ### Run Sync In the top right corner, you'll also see the **Run Sync** button. This action triggers a manual sync. Clicking this button allows you to choose whether the sync should **Pull** or **Push** metadata. {% hint style="info" %} Not all integrations will support both Pull and Push. Learn more about what integrations are supported for Push Metadata [here](/integrations/push-metadata-to-source). {% endhint %} ### Syncs This is the first page you’ll see when you navigate to an integration. It provides a detailed history of past syncs, allowing you to review each sync's stages, the number of resources pulled, and any errors that may have occurred. ### Schedule To automate your syncs, use this page to set the run frequency with a Cron Expression. You can learn more about Cron Expressions [here](https://crontab.guru/). If you don’t specify a schedule, the default is `0 0 * * *`, which runs the sync daily at midnight UTC. ### Schemas & Groups If applicable to the integration, use this page to select which Groups or Schemas you want to sync. Click the **Refresh** button to check for any new Groups or Schemas available for import. By default, all Groups and Schemas will be selected and included in the sync. If you'd like to change this default behaviour, navigate to the [Resource Management](#resource-management) section in the [Preferences.](#preferences) In addition the Teams that a Database, Schema, or Group are associated with can be configured on this tab under the "Team visibility" column. By default, the Teams will be inherited from the Integration settings but you can override the Teams on any Database, Schema, or Group.
### Preferences #### Query permissions This section lets you choose which users in your workspace have permission to query the resource through the preview, query block, and Secoda AI features, if the integration supports querying.
#### Popularity This preference allows you to deselect any accounts that you do not want to contribute to resource Popularity. The Popularity of a data resource in Secoda is determined by the number of times it has been queried or viewed in the last 24 hours, with data pulled directly from the integration. Learn more about Popularity [here](/features/popularity). #### Property Import This preference allows your to determine the import behaviour of properties, such as descriptions and tags, from your data source as properties in Secoda. The following options are available for importing properties: * **Never**: No properties are imported from your data source * **Smart**: Properties are imported from your data source until modified in Secoda. Once you edit a property in Secoda, it won't be overwritten by future imports * **Always**: Properties are always imported from your data source. Overwrites any existing values in Secoda with each sync #### Resource Management This section offers several options. If a toggle is specific to a particular integration, please refer to the integration page in the Secoda documentation for more details about that toggle. The options listed below are shared across all integrations: * **Disable Automatic Staling:** Prevents the automatic removal of resources that are not present in subsequent syncs for this integration. By default, if a resource is not extracted in a subsequent sync (ex. it is removed from the source), it is removed from the workspace. With this setting enabled, those resources will remain accessible in the workspace. * **Disable Extraction of Resources from New Schemas:** Prevents the automatic extraction of resources from newly added groups or schemas. By default, if a new schema or group is added in the source, the subsequent sync will extract all resources from that schema or group. Disabling this option means you'll need to manually select the new schema or group in the [Groups or Schema](#groups-or-schemas) page to extract those resources. #### Filtering In this section, you can use the filters to exclude resources (schemas, tables, columns) from extraction based on their title. Excluding resources based on other properties is not currently supported. ### FAQs
How does property management for descriptions, tags, and owners work? Secoda uses change tracking to determine the source of truth for properties like descriptions, owners, and tags. The source property is used when: * Property is empty in Secoda * Property hasn't been modified by users in Secoda * Source provides new data User modifications are preserved in Secoda when: * Users have manually edited the property in Secoda Additional cases: * First sync: Source data populates all empty properties * Empty source values: Don't overwrite existing Secoda data * Compliance integrations (e.g., Cyera, and Dataplex): Tags are appended rather than replaced The system prioritizes preserving user curation while allowing source systems to populate and update unmodified properties.
# Data warehouses Secoda currently integrates with the following data warehouse tools: {% content-ref url="/pages/MGlxa95tEx15brTLNiDz" %} [Redshift](/integrations/data-warehouses/redshift-integration) {% endcontent-ref %} {% content-ref url="/pages/yKV0EEgUQNvr9Rp1EUIa" %} [Databricks](/integrations/data-warehouses/databricks-integration) {% endcontent-ref %} {% content-ref url="/pages/m2BuUOJV7i2mgIxEdluW" %} [Snowflake](/integrations/data-warehouses/snowflake-integration) {% endcontent-ref %} {% content-ref url="/pages/wJDiq6fFvuTPiFZTpix1" %} [BigQuery](/integrations/data-warehouses/bigquery-integration) {% endcontent-ref %} {% content-ref url="/pages/UJ76v54vfV11WRT1yQwh" %} [Apache Hive](/integrations/data-warehouses/apache-hive) {% endcontent-ref %} {% content-ref url="/pages/rXsC5eOds95ltRwtJ0WJ" %} [MotherDuck](/integrations/data-warehouses/motherduck) {% endcontent-ref %} Secoda also integrates into databases, Bi tools and more integrations, which can be found here: {% content-ref url="/pages/jnCXVntfyMhZ3jrxYe7Q" %} [Databases](/integrations/databases) {% endcontent-ref %} {% content-ref url="/pages/oJncKxWeIacb9F4SDQAb" %} [Data visualization tools](/integrations/data-visualization-tools) {% endcontent-ref %} {% content-ref url="/pages/2KDTjV7s5hh3tmCyBzVC" %} [Data pipeline tools](/integrations/data-pipeline-tools) {% endcontent-ref %} # BigQuery An overview of the BigQuery integration with Secoda {% content-ref url="/pages/VX0tkDbt11AA2U19yhei" %} [BigQuery Metadata Extracted](/integrations/data-warehouses/bigquery-integration/bigquery-metadata) {% endcontent-ref %} ## Getting Started with Big Query There are three steps to connect Big Query with Secoda: 1. Enable the Big Query API 2. Create a service account for Secoda 3. Connect Big Query to Secoda #### Enable Big Query API for GCP Log in to your existing project on GCP and [enable the BigQuery API](https://cloud.google.com/bigquery/docs/quickstarts/quickstart-web-ui). Once you’ve done so, you should see BigQuery in the [“Resources” section](https://cl.ly/0W2i2I2B2R0M) of Cloud Platform. #### Create a service account for Secoda To provide [least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) to Secoda for extracting Big Query metadata, you can create a new service account following the steps below. Refer to [Google Cloud’s documentation about service accounts](https://cloud.google.com/iam/docs/creating-managing-service-accounts) for more information. 1. From the Navigation panel on the left, go to **IAM & admin** > **Service accounts** 2. Click **Create Service Account** along the top 3. Enter a name (for example: “secoda”) and click **Create** 4. When assigning permissions, make sure to grant the following permissions: a) If you're creating the service account via the GCP console add the following roles: ``` BigQuery Metadata Viewer BigQuery Resource Viewer BigQuery Data Viewer BigQuery Job User ``` If you want to enable metadata push feature, add the following role: ``` BigQuery Data Editor ``` If you want to enable syncing policy tags, add the following roles: ``` Data Catalog Admin Data Catalog Tag Editor Tag User ``` b) If you're programatically creating the service account add the following roles: ``` roles/bigquery.metadataViewer roles/bigquery.resourceViewer roles/bigquery.dataViewer roles/bigquery.jobUser ``` If you want to enable metadata push feature, add the following role: ``` roles/bigquery.tables.update ``` If you want to enable syncing policy tags, add the following roles: ``` roles/datacatalog.admin roles/datacatalog.tagEditor roles/datacatalog.tagUser ``` 5\. [Create a JSON key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys). The downloaded file will be used to create your warehouse in the next section. #### Connect Big Query to Secoda * Log into your Secoda profile at * From the Navigation panel on the left go **Integrations** > **Add new integration** * Select **Big Query** * Enter in the project name and paste the JSON key file contents that was downloaded * Click "Connect" # BigQuery Metadata Extracted List of all the metadata that Secoda pulls from/pushes to BigQuery ### Metadata pulled Secoda pulls the following metadata from BigQuery: * Tables/Views * Name * Description * Last Updated Timestamp * Schema * Database * Frequent users * Policy tags * Labels * Fields (Fields are referred to as Columns in Secoda) * Name * Description * Type * Foreign Key * Primary Key * Policy tags * Labels * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Stored Procedures (Stored Procedures are referred to as Creation Queries in Secoda) * Queries (Popularity) * Lineage * BQ Column <-> BQ Column * BQ Table <-> BQ Table * BQ Column <-> BQ View * BQ Table <-> Tables from other sources * BQ Table <-> Dashboards from other sources * BQ Table, Views <-> Runs from other sources * Preview of first 50 rows (Optional) ### Metadata pushed If enabled, Secoda pushes the following metadata to BigQuery: * Tables * Description * Columns * Description ### BigQuery Labels in Secoda Since BigQuery labels are key-value pairs, these are imported into Secoda as custom properties. If the values are not set, these will not show up in Secoda. # Databricks An overview of the Databricks integration with Secoda {% content-ref url="/pages/ZAVVCXGCNo9Wc5cfG0jG" %} [Databricks Metadata Extracted](/integrations/data-warehouses/databricks-integration/metadata-extracted) {% endcontent-ref %} ## **Getting Started with Databricks** There are three steps to get started using Databricks with Secoda: 1. Create an access token 2. Connect Databricks to Secoda 3. Whitelist Secoda IP Address ### Create an access token In your Databricks console go to the **User Settings** and generate a new access token. Save the value to be used to connect Databricks to Secoda in the second step. {% hint style="info" %} To have query history and popularity you must provide admin privileges to the token. {% endhint %} ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(12\)%20\(1\).png) ### Grant Secoda Access For each warehouse you plan to connect to Secoda, the credentials must have `Can monitor` permissions (set via `SQL Warehouses > [My Warehouse] > Permissions`).
{% hint style="info" %} `Can use` can be selected but will not allow for any warehouse-level query history to be accessed. `Can view` does not provide sufficient permissions {% endhint %} For each catalog you want to connect to Secoda, the credentials must have the following permissions: * `USE_CATALOG` * `USE_SCHEMA` * `BROWSE` * `SELECT`
### Connect Databricks to Secoda Go to and select the Databricks integration. Enter in the following credentials: * **Host:** This is the URL of your Databricks workspace, i.e, [dbc-dc31b5a2-597d.cloud.databricks.com](https://dbc-dc31b5a2-597d.cloud.databricks.com/) * **Databricks Workspace Id:** The numerical id of your workspace, located in the url of your Databricks instance, after the "/?o=". `https://.cloud.databricks.com/?o=`.\\ * **Access Token:** The access token you generated in the first step * **Warehouse ID (Recommended) or Cluster ID:** This is the resource what SQL queries will run on. For the optimal experience, use a [Databricks serverless SQL warehouse](https://docs.databricks.com/en/admin/sql/serverless.html). {% hint style="info" %} To ingest table and column level lineage using Databricks Unity Catalog, a Warehouse ID must be specified. {% endhint %} After entering in the information into Secoda, click "Test Connection". After the connection is successful your can Submit and run the initial extraction. ### Whitelist Secoda IP Address If your Databricks instance is behind a firewall, you'll have to whitelist [Secoda's IP address](/faq#what-are-the-ip-addresses-for-secoda) to allow for metadata extractions. ### FAQs
What cloud providers are supported? Databricks on the major cloud providers including AWS, GCP, and Azure are supported.
# Databricks Metadata Extracted List of all the metadata that Secoda pulls from Databricks Secoda pulls the following metadata from Databricks: * Jobs * Title * URL * Job Runs * Status * Started at timestamp * Finished at timestamp * Error * Dashboard * Title * Description * Last updated at timestamp * External usage (Number of Views) * Owners * URL * Table * Title * Description * Columns * Title * Description * Type * Lineage via Unity Catalog * Databricks Table <-> Databricks Table * Databricks Column <-> Databricks Column * Databricks Table <-> Tables from other sources * Preview of first 50 rows (Optional) # Redshift An overview of the Redshift integration with Secoda {% content-ref url="/pages/Ro6CXveCZzhec02WPDvB" %} [Redshift Metadata Extracted](/integrations/data-warehouses/redshift-integration/redshift-metadata) {% endcontent-ref %} ## Getting Started with Redshift There are three steps to connect Redshift with Secoda: 1. Create a database user 2. Connect Redshift to Secoda 3. Whitelist Secoda IP Address #### **Create a Database User** The username and password you’ve already created for your cluster is your admin password, which you should keep for your own usage. For Secoda, and any other 3rd-parties, it is best to create distinct users. This will allow you to isolate queries from one another using [WLM](http://docs.aws.amazon.com/redshift/latest/dg/c_workload_mngmt_classification.html) and perform audits easier. To create a [new user](http://docs.aws.amazon.com/redshift/latest/dg/r_Users.html), you’ll need to log into the Redshift database directly and run the following SQL commands: Secoda only uses the system tables for our metadata extraction, the extraction query can be viewed [here](https://www.notion.so/Redshift-4a22df9ed18b4a6cb35eefd418f65727). ``` -- Create a user named "secoda" that Secoda will use when connecting to your Redshift cluster. CREATE USER secoda PASSWORD ''; -- Allows the "secoda" user to query metadata -- Explaination of query here -> https://stackoverflow.com/questions/48567440/granting-permissions-on-redshift-system-tables-to-non-superusers ALTER USER secoda SYSLOG ACCESS UNRESTRICTED; -- Complete this query for any schemas you would like Secoda to extract GRANT REFERENCES ON ALL TABLES IN SCHEMA TO secoda -- Optionally provide SELECT access for the preview, query blocks, and profiling GRANT SELECT ON ALL TABLES IN SCHEMA TO secoda GRANT SELECT ON svv_table_info TO secoda; ``` When connecting to Redshift in Secoda, use the username/password you’ve created here instead of your admin account. #### **Connect Redshift to Secoda** After creating a Redshift warehouse, the next step is to connect Secoda: 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select ‘Redshift’ 3. Enter your Redshift credentials 4. Click 'Connect' #### Whitelist Secoda IP Address VPCs keep servers inaccessible to traffic from the internet. With VPC, you’re able to designate specific web servers access to your servers. In this case, you will be whitelisting the [Secoda IPs](/faq#what-are-the-ip-addresses-for-secoda) to read from your data warehouse. #### **Networking** Redshift clusters can either be in a **EC2 Classic subnet** or **VPC subnet**. If your cluster has a field called `Cluster Security Groups`, proceed to [EC2 Classic](https://docs/connections/storage/catalog/redshift/#ec2-classic) Or if your cluster has a field called `VPC Security Groups`, proceed to [EC2 VPC](https://segment.com/docs/connections/storage/catalog/redshift/#ec2-vpc) #### **EC2-Classic** 1. Navigate to your Redshift Cluster settings: `Redshift Dashboard > Clusters > Select Your Cluster` 2. Click on the Cluster Security Groups 3. Open the Cluster Security Group 4. Click on “Add Connection Type” 5. Choose Connection Type CIDR/IP and authorize the [Secoda IPs](/faq#what-are-the-ip-addresses-for-secoda) to read into your Redshift Port #### **EC2-VPC** 1. Navigate to your `Redshift Dashboard > Clusters > Select Your Cluster` 2. Click on the VPC Security Groups 3. Select the “Inbound” tab and then “Edit” 4. Allow Secoda to read into your Redshift Port using the [Secoda IP addresses](/faq#what-are-the-ip-addresses-for-secoda). You can find more information on that [here](http://docs.aws.amazon.com/redshift/latest/mgmt/managing-clusters-vpc.html). 1. Navigate back to your Redshift Cluster Settings: `Redshift Dashboard > Clusters > Select Your Cluster` 2. Select the “Cluster” button and then “Modify” 3. Make sure the “Publicly Accessible” option is set to “Yes” # Redshift Metadata Extracted List of all the metadata that Secoda pulls from/pushes to Snowflake ### Metadata pulled Secoda pulls the following metadata from Redshift: * Tables * Name * Description * Schema * Database * External Usage (Popularity) * External Updated At * Views * Name * Description * Schema * Database * External Usage (Popularity) * External Updated At * Columns * Name * Description * Type * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Creation Query * Common Queries * Lineage * Redshift Column <-> Redshift Column * Redshift View <-> Redshift View * Redshift Table <-> Redshift Table * Redshift Table <-> Redshift View * Redshift Table <-> Dashboards from other sources * Redshift Table <-> Tables from other sources * Redshift Table, Views <-> Jobs from other sources * Preview of first 50 rows (Optional) ### Metadata pushed If enabled, Secoda pushes the following metadata to Redshift: * Tables * Description * Columns * Description # Snowflake An overview of the Snowflake integration with Secoda {% content-ref url="/pages/keHjUL2vAJ5G4uBut2nO" %} [Snowflake Metadata Extracted](/integrations/data-warehouses/snowflake-integration/snowflake-metadata) {% endcontent-ref %} ## **Getting Started with Snowflake** There are four steps to connect Snowflake with Secoda. 1. Create Role for Secoda 2. Create User for Secoda 3. Whitelist [Secoda IP Address](#h_e7eac6e3f5) 4. Connect Snowflake to Secoda in the Secoda UI You **must** be either an `ACCOUNTADMIN`, or have `MANAGE GRANTS` privileges in order to run the commands necessary to connect. We recommend naming the User, Role, and Warehouse, `SECODA_USER`, `SECODA_ROLE`, `SECODA_WAREHOUSE` respectively. However, naming them this way is not necessary to integrate. ### **Step 1: Create Role for Secoda** Navigate to Worksheets, select a database, and run the following commands in that database. You'll need to run these commands for all of the databases that you'd like Secoda to import metadata from. ``` CREATE ROLE SECODA; GRANT imported privileges on database SNOWFLAKE to ROLE SECODA; GRANT USAGE ON WAREHOUSE "" TO ROLE SECODA; // ====== Existing Tables & Schemas begin; set database_name = ; // Usage on database object GRANT USAGE ON DATABASE identifier($database_name) TO ROLE SECODA; // Usage on existing schemas GRANT USAGE,MONITOR ON ALL SCHEMAS IN DATABASE identifier($database_name) TO ROLE SECODA; // References for INFORMATION_SCHEMA to existing tables GRANT SELECT ON ALL TABLES IN DATABASE identifier($database_name) TO ROLE SECODA; GRANT SELECT ON ALL VIEWS IN DATABASE identifier($database_name) TO ROLE SECODA; // ====== Future Tables & Schemas // Read access to all schemas created in the future (but not current ones) GRANT USAGE,MONITOR ON FUTURE SCHEMAS IN DATABASE identifier($database_name) TO ROLE SECODA; // Reference for INFORMATION_SCHEMA to all tables created in the future (but not current ones) GRANT SELECT ON FUTURE TABLES IN DATABASE identifier($database_name) TO ROLE SECODA; commit; ``` \[Optional] If you are using Snowflake Dynamic Tables, to bring in those tables, you have to grant `MONITOR` permissions to the SECODA role on all of those tables. ``` GRANT MONITOR ON TABLE database.schema.some_dynamic_table TO ROLE SECODA; ``` This step has to be repeated for every single dynamic table that you wish to bring into Secoda. \[Optional] If you are using Snowflake Snowpies, to bring in those pipes, you have to grant `MONITOR USAGE` permissions to the SECODA role and `MONITOR` permissions on all of those pipes. The stages that the pipes reference must also be granted access privileges. ``` GRANT MONITOR USAGE ON ACCOUNT TO ROLE SECODA; GRANT MONITOR ON PIPE database.schema.some_pipe TO ROLE SECODA; GRANT USAGE ON ALL STAGES IN DATABASE identifier($database_name) TO ROLE SECODA; ``` \[Optional] If you are using Snowflake Streamlit, to bring in those apps, you have to grant `MONITOR` permissions to the SECODA role on all of those apps. ``` GRANT USAGE ON ALL STREAMLITS IN DATABASE identifier($database_name) TO ROLE SECODA; ``` ### Step 2: Create User for Secoda ``` CREATE USER SECODA_USER MUST_CHANGE_PASSWORD = FALSE DEFAULT_ROLE = SECODA PASSWORD = "my_strong_password"; *-- Do not use this password * GRANT ROLE SECODA TO USER SECODA_USER; // This sets the default role, required for metadata extractions ALTER USER SECODA_USER SET DEFAULT_ROLE=SECODA ``` {% hint style="info" %} If you would like to enable the [Push to Snowflake](/integrations/push-metadata-to-source) feature, the SECODA\_USER must be the owner of the tables, have INSERT privileges on the table, and MODIFY privileges on the schema and database. {% endhint %} #### Key-Pair Authentication If you would like you use key-pair authentication instead of a password you will need to: [Configure the key-pair in Snowflake](https://docs.snowflake.com/en/user-guide/key-pair-auth). Once the key is created, you can run the following command to connect the key to the `SECODA_USER`. 
ALTER USER SECODA_USER SET RSA_PUBLIC_KEY='my_public_key';
### **Step 3: Whitelist Secoda IP Addresses** If you create a network policy with Snowflake, add the following [Secoda IP addresses](/faq#what-are-the-ip-addresses-for-secoda) to the “Allowed IP Addresses” list. ### **Step 4: Connect Snowflake to Secoda** 1. In the Secoda App, select `Add Integration` on the Integrations page. Search for and select “Snowflake”. 2. Add your credentials as follows: * User - The name of the User created in Step 2. * Password - The Password set in Step 2. * Alternatively you can select the key-pair authentication and enter the private key and passphrase created in step 2. * Account - This is the Account ID of your cluster. * Warehouse - The Warehouse set in Step 1. #### **How do I find my Account ID?** You can find the Account ID in the Snowflake URL. The account ID is usually a substring of the URL, before `snowflakecomputing.com`. **If your Snowflake URL does not contain `snowflakecomputing.com`, see** [**here**](#account-id-is-not-part-of-url) **to determine your Account ID.** The account ID will likely be the business name, as well as the cloud region, if Snowflake is cloud hosted. See below for some examples. 1. URL: `https://secoda.snowflakecomputing.com` ACCOUNT ID: `secoda` 2. URL: `https://secoda.us-east-1.snowflakecomputing.com` ACCOUNT ID: `secoda.us-east-1` 3. URL: `https://secoda.west-europe.azure.snowflakecomputing.com` ACCOUNT ID: `secoda.west-europe.azure` ### Troubleshooting #### Account ID is not part of URL Snowflake has made some recent changes where URLs can be different than the standard format above. In these cases, you can find the correct account id by: 1. Clicking on the account selector in Snowflake 2. Hovering over the specific account you want to connect to\ \ \\ 3. Then clicking the copy account URL button on the 3rd section that shows the accounts details (organization id,\ \ ![](https://secoda-public-media-assets.s3.amazonaws.com/f461ba8b-ea39-402c-849a-e84b96d73a16.png) This should create a URL ending with `snowflakecomputing.com`and you can follow the steps abvoe to determine the account id. #### Account usage not authorized In order to resolve this error, please run the following command: `GRANT imported privileges on database SNOWFLAKE to ROLE SECODA;` #### Could not connect to Snowflake backend after 0 attempt(s) This error could be the result of an incorrect Account ID. Please double check that your Account ID is properly added. **No active warehouse selected in the current session** This error can be due to the warehouse name not being fully uppercase. Updating the warehouse name to all uppercase letters should resolve this issue. #### Missing schemas Secoda will only show schemas for which tables can be accessed. So most often when a schema is missing, its because the `Secoda` role is lacking permissions to read tables in said schema. Its possible that since the role was first created, tables were recreated (often by ETL-like tools) and the permissions were reset. To test this, you can run the following query in your Snowflake instance as the `Secoda` role. ``` USE SECONDARY ROLES NONE; SELECT table_schema, table_name from YOUR_DATABASE.information_schema.tables; ``` This will output a list of all the tables (and corresponding schemas) the Secoda role has access to for a certain database. If some tables are not listed, it means the role is missing the requisite permissions to access them. Some common reasons for table permissions getting reset are: * DBT recreating tables without copying permissions. You can use the [copy grants](https://docs.getdbt.com/reference/resource-configs/snowflake-configs#copying-grants) setting to prevent this. # Snowflake Metadata Extracted List of all the metadata that Secoda pulls from/pushes to Snowflake ### Metadata pulled Secoda pulls the following metadata from Snowflake: * Tables * Name * Description * Last Updated Timestamp * External Usage (Popularity) * Schema * Database * Frequent Users * Tags * Views * Name * Description * Last Updated Timestamp * External Usage * Schema * Database * Frequent Users * Tags * Columns * Name * Description * Type * Foreign Key * Primary Key * Tags * Column Profile * Min * Max * Median * STD Deviation * Value distribution * Statistic Value Count * Percent Filled * Unique * Creation Query * Common Queries * Lineage * Snowflake Column <-> Snowflake Column * Snowflake Table <-> Snowflake Table * Snowflake Table <-> Dashboards from other sources * Snowflake Table <-> Jobs from other sources * Snowflake Column <-> Snowflake View * Snowflake Table <-> Snowflake View * Pipes * Name * Copy statement * Comment * Integration * Created on * Pattern * Pipe usage history * Start time * End time * Bytes inserted * Files inserted * Streamlit * Name * Comment * Url * Preview of first 50 rows (Optional) ### Metadata pushed If enabled, Secoda pushes the following metadata to Snowflake: * Descriptions * Tags It only looks at the tables that have been published and all of their columns. If a table isn't published in Secoda and you run a sync, it will not push back to the source. {% hint style="warning" %} Please ensure ensure the `SECODA` role has `INSERT` table privileges, as well as `MODIFY` schema and database. You can check what privileges the role has by running `SHOW GRANTS TO ROLE SECODA`. The SECODA user must also be an owner of the tables. {% endhint %} # Snowflake Costs Learn about and set monitors on your Snowflake costs in Secoda Everyone is thinking about cost containment these days, and so are we! You can track your Snowflake costs, using our Snowflake Cost widget as part of the [Analytics Dashboard](/features/analytics-dashboard).
## How does the Widget work? We provide cost analysis for *everything* in your Snowflake warehouse. This information is provided through the information schemas of each table, for the entire warehouse. Visualizing your cost trends using the Widget **will not** impact your overall costs, as Secoda is already querying the information schemas to extract the metadata for your workspace. To learn more about what queries Secoda makes to your Snowflake instance, navigate to the [Secoda impact on Snowflake Costs](#secodas-impact-on-snowflake-costs) section below. ## Add Monitors on Snowflake costs You can even use our monitoring features on a Snowflake cost widget to be alerted about changes to daily costs. Simply create the widget, then use the three dot menu to create a monitor.
Add filters on usage type and balance source, and voila! You can catch changes before they become larger issues.
## Secoda's Impact on Snowflake Costs At every sync, Secoda runs queries on your Snowflake instance. Generally, these queries costs anywhere between 0.1 to 1 credit in total. Secoda queries Snowflake in two ways during each sync. The first, is querying the information schema of each database in Snowflake. The second, is querying the `snowflake.account_usage.query_history` table. From these two queries, Secoda is able to extract and calculate [Metadata](/integrations/data-warehouses/snowflake-integration/snowflake-metadata), [Cost](/features/analytics-dashboard), [Lineage](/features/data-lineage), [Query History](/features/queries), [Popularity](/features/popularity) and [profile](/features/column-profiling) the Snowflake Columns. Note, queries outside of a sync, such as those from [Monitors](/features/monitoring), are not included in the cost above, and can vary depending on how the feature is implemented in your workspace. # Snowflake Native App {% hint style="info" %} The Secoda Snowflake Native App is currently in early access. Features and functionality may change before the full release. {% endhint %} The Secoda Native Snowflake App enables you to integrate your Snowflake warehouse with Secoda without providing direct access to your Snowflake account. The native application runs securely within your Snowflake environment. {% hint style="info" %} The Secoda Native App processes your data within your Snowflake environment and only sends metadata and limited processed information to Secoda's servers, allowing you to use Secoda's features without providing direct access to your raw data. {% endhint %} ## Installation Process To get started with the Secoda Native Snowflake App, follow these four steps: 1. Install the native app from Snowflake Marketplace 2. Configure the application's connection credentials 3. Set up network rules (if required) 4. Complete the setup process ### 1. Installing from Marketplace You can install the app directly from the Snowflake Marketplace. Once installed, you will see the application interface:

Secoda Native App Interface

### 2. Configure Connection Credentials Next, you'll need to configure the connection and provide account-level privileges: 1. Open the app and navigate to the Connections tab

Connections Tab

2. Enter your Secoda API key (You can generate an API key from your Workspace settings)

API Key Entry

3. Grant the necessary privileges to the application

Required Privileges

### 3. Network Rule Setup By default, the native app can connect to Secoda's US Cloud service (app.secoda.co). If you use any other URL to access Secoda, you will need to create a network rule. {% hint style="info" %} You can skip this step if you use Secoda at app.secoda.co {% endhint %} To create a network rule: 1. In Snowflake, navigate to Admin → Security → Network Rules 2. The network rule name depends on your app installation name * If you installed the app as `SECODA_APP`, the rule will be named `SECODA_APP_SECODA_API_NETWORK_RULE` 3. Add your custom Secoda endpoint to the network rule

Network Rule Configuration

### 4. Create Integration in Secoda Before proceeding with the application configuration, you need to create a Native Snowflake integration in Secoda:

Creating a Native Snowflake Integration in Secoda

You will not need to configure any authentication for this integration. After creating the integration in Secoda, return to Snowflake and launch the "INSTALL\_SECODA" Streamlit program from the application header. ## Setup Wizard The setup process guides you through three essential steps: ### 1. Connect to Secoda * By default, the installation connects to Secoda's US cloud (app.secoda.co) * Custom endpoint options are available for EU cloud, single-tenant, or on-premise deployments ### 2. Select Integration * Choose which Secoda integration to use with your Snowflake data * The app will list compatible Snowflake integrations from your Secoda account * If no integrations exist, you'll need to create one in Secoda first ### 3. Configure Features * **Catalog**: Enable to discover, explore, and document your Snowflake data assets * **Monitor**: Enable to track data quality and freshness metrics {% hint style="success" %} Once the setup is complete, your Secoda Native Snowflake App is ready to use! {% endhint %} ## Using the Native App You can interact with the Secoda Native Snowflake App in two ways: 1. **Streamlit-powered GUI**: A user-friendly interface for configuring and managing your integration 2. **SQL commands**: Execute SQL statements via a worksheet to perform the same operations Most functionality available through the GUI can also be accessed via SQL commands, offering flexibility based on your team's preferences. ### Required Permissions For each database you want to monitor or catalog, grant the following permissions: ```sql -- Replace placeholders with your actual database, schema, and table names GRANT USAGE ON DATABASE TO APPLICATION secoda_app; GRANT USAGE ON SCHEMA . TO APPLICATION secoda_app; GRANT SELECT ON TABLE .. TO APPLICATION secoda_app; ``` ## Monitoring The monitoring feature allows you to create data quality monitors that run within your Snowflake account using your Snowflake credits. Only the final calculated information is sent to Secoda. Secoda tracks these metrics and can alert stakeholders when values exceed configured thresholds or when anomalies are detected. ### Creating Monitors You can create monitors using SQL commands. Here are some common examples: ```sql -- Basic row count monitor CALL SECODA_APP.MONITORING.CREATE_MONITOR( MONITOR_KEY => 'users_row_count', NAME => 'Users Table Row Count', METRIC_TYPE => 'row_count', TABLE_CATALOG => 'your_database', TABLE_SCHEMA => 'your_schema', TABLE_NAME => 'your_table' ); -- Column-based monitor with manual thresholds CALL SECODA_APP.MONITORING.CREATE_MONITOR( MONITOR_KEY => 'users_max_age', NAME => 'Users Maximum Age', METRIC_TYPE => 'max', TABLE_CATALOG => 'your_database', TABLE_SCHEMA => 'your_schema', TABLE_NAME => 'your_table', COLUMN_NAME => 'age', THRESHOLDS_METHOD => 'manual', THRESHOLDS_MAX => 100, THRESHOLDS_MIN => 20 ); -- Custom SQL monitor CALL SECODA_APP.MONITORING.CREATE_MONITOR( MONITOR_KEY => 'active_users', NAME => 'Count of Active Users', METRIC_TYPE => 'custom_sql', TABLE_CATALOG => 'your_database', TABLE_SCHEMA => 'your_schema', TABLE_NAME => 'your_table', QUERY => 'SELECT COUNT(*) FROM your_database.your_schema.your_table WHERE is_active = TRUE' ); -- Freshness monitor CALL SECODA_APP.MONITORING.CREATE_MONITOR( MONITOR_KEY => 'data_freshness', NAME => 'Data Freshness', METRIC_TYPE => 'freshness', TABLE_CATALOG => 'your_database', TABLE_SCHEMA => 'your_schema', TABLE_NAME => 'your_table', COLUMN_NAME => 'last_updated' ); ``` ### Running and Managing Monitors To run a specific monitor: ```sql CALL SECODA_APP.MONITORING.RUN_MONITOR('monitor_key'); ``` You can also manage your monitors: ```sql -- Pause a monitor CALL SECODA_APP.MONITORING.PAUSE_MONITOR('monitor_key'); -- Resume a monitor CALL SECODA_APP.MONITORING.RESUME_MONITOR('monitor_key'); -- Update a monitor's thresholds CALL SECODA_APP.MONITORING.UPDATE_MONITOR( MONITOR_KEY => 'monitor_key', THRESHOLDS_METHOD => 'manual', THRESHOLDS_MAX => 100 ); -- Clear all monitors CALL SECODA_APP.MONITORING.CLEAR_MONITORS(); ``` ## Catalog The Catalog feature automatically discovers and syncs metadata about all schemas, tables, and columns that the app has been granted access to. The catalog syncs automatically at 12 AM UTC daily. After the sync completes, it may take a few hours for the metadata to be available in Secoda. ### Manual Catalog Sync You can also manually trigger a catalog sync: 1. Open the "MANAGE\_CATALOG" Streamlit application 2. Click "Push Catalog" to start the sync process 3. Once complete, go to Secoda and open the integration page 4. Click "Pull Metadata" to import the metadata into Secoda {% hint style="info" %} The catalog sync process takes approximately 1 hour for 200 tables and runs in a managed XSMALL warehouse. {% endhint %} # Apache Hive An overview of the Hive integration with Secoda {% content-ref url="/pages/UQ3aB9lmwtQMpMxxnBgt" %} [Apache Hive Metadata Extracted](/integrations/data-warehouses/apache-hive/metadata-extracted) {% endcontent-ref %} ## **Getting Started with Apache Hive** There are two main ways of integration Apache Hive to Secoda. 1. Direct connection to Apache Hive 2. Using MySQL metastore ### Direct Connection to Hive **1. Create Role for Secoda** To ensure controlled access, we'll begin by creating a dedicated role for Secoda in Hive: ```sql -- Create a new role for Secoda CREATE ROLE SECODA; -- Grant read permissions on database GRANT SELECT ON DATABASE .* TO ROLE SECODA; ``` **2. Create User for Secoda** Creating a user in Hive will depend on the underlying authentication and user management system in place (like LDAP, Kerberos, etc.). Typically, a Hive admin would create users outside of Hive. Once they exist, you can grant roles to them within Hive: ```sql -- Grant the role to user GRANT ROLE SECODA TO USER SECODA_USER; ``` **3. Connect Hive to Secoda** To integrate Secoda with your Hive setup: * In the Secoda App, select `Add Integration` on the Integrations page. * Search for and select "Hive". * Provide the necessary credentials: * `User`: The username (as created above). * `Password`: The password for the user. * `Host`: Hive Server2 host. * `Port`: This is usually 10000 but might vary based on your specific setup. **4. Security: Whitelisting IPs** The process of whitelisting IPs would be external to Hive. It typically occurs at the level of Hive Server2 or in the underlying infrastructure, such as firewalls or security groups if you're using a cloud platform. Ensure that you add the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda) to the list of allowed IPs. ### MySQL Metastore Connection **1. Connect Metastore to Secoda** To integrate Secoda with your Hive setup: * In the Secoda App, select `Add Integration` on the Integrations page. * Search for and select "Hive". * Provide the necessary credentials: * `User`: The username for your MySQL user * `Password`: The password for the user. * `Host`: MySQL server host. * `Port`: This is usually 3306 but could be different * `Database`: This is the database for your MySQL metastore * Please ensure the user has select access to this database **2. Security: Whitelisting IPs** Ensure that you add the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda) to the list of allowed IPs on your MySQL server. # Apache Hive Metadata Extracted List of all the metadata that Secoda pulls from/pushes to Apache Hive Secoda pulls the following metadata from Apache Hive: * Table * Title * Columns * Title * Description * Type * Comments * Preview of first 50 rows (Optional) # Azure Synapse An overview of the Azure Synapse integration with Secoda {% content-ref url="/pages/iVVb5322EhID5fPv1CR5" %} [Azure Synapse Metadata Extracted](/integrations/data-warehouses/azure-synapse/metadata-extracted) {% endcontent-ref %} ## Getting Started with Azure Synapse 1. **Create a Login** To ensure controlled access within Azure Synapse, start by creating a login for Secoda. This has to be done on the `master` database on your dedicated SQL pool. Then assign the `dbmanager` on `master` database to the `SECODA` user. ```sql CREATE LOGIN SECODA WITH PASSWORD 'YourSecurePassword'; CREATE USER SECODA FROM LOGIN SECODA; EXEC sp_addrolemember 'dbmanager', 'SECODA'; ``` 2. **Create User for Secoda** Switch to the primary database in your dedicated SQL pool and create a Secoda user there as well. Then assign the `db_owner` role to it. ```sql CREATE USER SECODA FROM LOGIN SECODA; EXEC sp_addrolemember 'db_owner', 'SECODA'; ``` 3. **Connect Azure Synapse to Secoda** To integrate Secoda with your Azure Synapse setup: * In the Secoda App, go to the Integrations page and select "Add Integration". * Search for and select "Azure Synapse". * Provide the necessary credentials: * **User**: The username (as created above). * **Password**: The password for the user. * **Host**: Azure Synapse server host. * **Port**: (Specify the default port for Synapse or note that it might vary based on your specific setup.) 4. **Security: Whitelisting IPs** Whitelisting IP addresses in Azure typically involves managing Network Security Group rules or using Azure Firewall. Ensure that you add the[ Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda) to the list of allowed IPs. To do this, go to the Azure Portal, navigate to the appropriate resource (e.g., Azure Synapse), and manage its network settings to add the IP address to the allow list. # Azure Synapse Metadata Extracted List of all the metadata that Secoda pulls from to Azure Synapse Secoda pulls the following metadata from Azure Synapse: * Table * Title * Views * Title * Columns * Title * Type * Creation Query * Common Queries * Lineage * Synapse Column <-> Synapse Column * Synapse Table <-> Synapse Table * Synapse Table <-> Dashboards from other sources * Synapse Table <-> Jobs from other sources * Snowflake Synapse <-> Synapse View * Preview of first 50 rows (Optional) # MotherDuck An overview of the MotherDuck integration with Secoda {% content-ref url="/pages/OWf9YqIJFWjr2mwrRKuK" %} [MotherDuck Metadata Extracted](/integrations/data-warehouses/motherduck/motherduck-metadata-extracted) {% endcontent-ref %} ### Getting started with MotherDuck To integrate MotherDuck with Secoda, follow these two steps: 1. Retrieve your MotherDuck service token 2. Connect MotherDuck to Secoda #### Retrieve your MotherDuck service token To find your service token, follow these steps: 1. Navigate to your MotherDuck instance you want to connect 2. Click on your profile in the top right corner 3. Click on settings from the dropdown 4. Under the General tab, click the 'Copy token' button #### Connect MotherDuck to Secoda After retrieving the service token, the next step is to connect to Secoda: 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select MotherDuck 3. Enter your service token that you have copied 4. Click connect # MotherDuck Metadata Extracted List of all the metadata that Secoda pulls from MotherDuck Secoda pulls the following metadata from MotherDuck: * Databases * Name * Schema * Name * Tables and Views * Name * Columns * Name * Type * Lineage * MotherDuck Table <--> MotherDuck Table * MotherDuck View <--> MotherDuck Table # ClickHouse ClickHouse is an open-source column-oriented database management system that allows generating analytical data reports in real-time. The ClickHouse integration in Secoda allows you to connect your ClickHouse instance to catalog and document your data. ### Getting Started with ClickHouse Create a user with the following permissions * `SELECT` access on system tables (`system.tables`, `system.columns`) * `SELECT` access on the databases and tables you want to catalog * Access to query history information * If using metadata push capabilities, the user needs `ALTER` permissions on tables Gather the additional information: * Host name of your ClickHouse instance, e.g., `wo3evzjkve.us-east-1.aws.clickhouse.cloud` * Port number, e.g., 8443 * Username with access to ClickHouse * Password for authentication ### Setup 1. In Secoda, navigate to the Integrations page 2. Click "Add Integration" 3. Select "ClickHouse" from the list of available integrations 4. Fill in the following credentials: ```yaml Host: your-clickhouse-host Port: 8443 User: your-username Password: your-password ``` 5. Click "Test Connection" 6. Select databases and schemas 7. Click "Continue" to trigger the initial sync See [ClickHouse Metadata Extracted](/integrations/data-warehouses/clickhouse/clickhouse-metadata-extracted)for the full list of information sycned from ClickHouse. ### Monitoring Capabilities The ClickHouse integration supports various monitoring features: * Table row counts * Column-level statistics, e.g., null % and unique %. ### Troubleshooting Common issues and solutions: 1. **Connection Failed** * Verify the host and port are correct * Ensure the user has proper permissions * Check if the ClickHouse server is accessible from your network 2. **Missing Tables** * Verify the user has SELECT permissions on the schemas * Check if tables are in system or information\_schema databases (these are excluded) 3. **Query History Not Showing** * Ensure the user has access to system tables * Verify query history retention settings in ClickHouse ### Limitations * External tables may have limited metadata support * Some system tables are excluded from extraction * Query history retention depends on ClickHouse settings * Metadata push is not supported for external tables ### FAQ **Q: Can I connect to multiple ClickHouse instances?** A: Yes, you can create multiple ClickHouse integrations in Secoda, each pointing to a different instance. **Q: Does the integration support SSL connections?** A: Yes, ClickHouse connections are secured using standard SSL/TLS protocols. **Q: How often does the integration sync?** A: By default, the integration syncs once per week, but this can be configured in the integration settings. **Q: Can I preview data in Secoda?** A: Yes, the integration supports data preview capabilities for tables and query results. # ClickHouse Metadata Extracted List of all the metadata that Secoda pulls from ClickHouse #### Metadata pulled Secoda pulls the following metadata from ClickHouse: * Tables * Name * Description * Schema * Database * External Usage (Popularity) * External Updated At * Byte Size * Rows * Views * Name * Description * Schema * Database * External Usage (Popularity) * External Updated At * Columns * Name * Description * Type * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Creation Query * Query History * SQL * User * Frequency * Lineage * ClickHouse Column <-> ClickHouse Column * ClickHouse View <-> ClickHouse View * ClickHouse Table <-> ClickHouse Table * ClickHouse Table <-> ClickHouse View * ClickHouse Table <-> Dashboards from other sources * ClickHouse Table <-> Tables from other sources * ClickHouse Table, Views <-> Jobs from other sources * Preview of first 50 rows (Optional) # Databases Secoda currently integrates with the following database tools: {% content-ref url="/pages/s74MtfEtPhAvHWdBdXMW" %} [Salesforce](/integrations/databases/salesforce-integration) {% endcontent-ref %} {% content-ref url="/pages/2iaat0tFgv7Uzjvja7Vl" %} [Microsoft SQL Server](/integrations/databases/microsoft-sql-server) {% endcontent-ref %} {% content-ref url="/pages/ne9AtsW5Gm6G7IuIwB8p" %} [MySQL](/integrations/databases/mysql-integration) {% endcontent-ref %} {% content-ref url="/pages/rtO5hFChlFClotFGh7Be" %} [Oracle](/integrations/databases/oracle-integration) {% endcontent-ref %} {% content-ref url="/pages/dQ7MUcwnnLMfwbhO9s5L" %} [Postgres](/integrations/databases/postgres-integration) {% endcontent-ref %} {% content-ref url="/pages/qo9g5WNdE3PJI7mVQ7H0" %} [MongoDB](/integrations/databases/mongodb) {% endcontent-ref %} {% content-ref url="/pages/6MfqPuwwmQ2pLXDWFgzk" %} [SingleStore](/integrations/databases/singlestore) {% endcontent-ref %} {% hint style="info" %} Don't see an integration for a tool you use? Message us on Slack or email us at and we'll add it to the roadmap. {% endhint %} # Druid An overview of the Druid integration with Secoda ## Getting Started with Druid There are 3 steps to get started using Druid with Secoda: 1. Retrieve Druid Host and Port. 2. Retrieve Username and Password 3. Connect Druid to Secoda ### **Retrieve Druid Host and Port** Identify the host domain and **port** used to access your Druid instance. These can be derived from the URL you use to access Druid in your browser: * Format: `http://:/` * Default port: `8888`. ### **Retrieve Username and Password** Druid **requires authentication** to connect. Ensure you have the correct **username** and **password** that you use to log in to your Druid instance. ### **Connect Druid to Secoda** After retrieving all necessary credentials, the next step is to connect to Secoda. 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select Druid 3. Enter your Druid credentials (host domain, port + username, password if applicable) 4. Click 'Connect' # Druid Metadata Extracted List of all the metadata that Secoda pulls from Druid Secoda pulls the following metadata from Druid: * Tables * Name * Columns * Name * Type * Preview on first 50 rows (Optional) Since Druid does not utilize databases or schemas, all tables and columns are assigned to a default database and schema, both named **'druid'**. # MySQL An overview of the MySQL integration with Secoda {% content-ref url="/pages/coBti2V9mOHuzs5CLVKk" %} [MySQL Metadata Extracted](/integrations/databases/mysql-integration/metadata-extracted) {% endcontent-ref %} ## **Getting Started with MySQL** There are three steps to get started using MySQL with Secoda: 1. Create a database user 2. Connect MySQL to Secoda 3. Whitelist Secoda IP Address #### **Create a Database User** The username and password you’ve already created for your cluster is your admin password, which you should keep for your own usage. For Secoda, and any other 3rd-parties, it is best to create distinct users. To create a new user, you’ll need to log into the MySQL database directly and run the following SQL commands: ``` -- Create a user named "secoda" that Secoda will use when connecting to your MySQL database. CREATE USER 'secoda'@'localhost' IDENTIFIED BY ''; -- Complete this query for any databases you would like Secoda to extract from GRANT SELECT ON .* TO 'secoda'@'localhost'; -- Complete this query for any schemas you would like Secoda to extract from GRANT SELECT ON .* TO 'secoda'@'localhost'; ``` When connecting to MySQL in Secoda, use the username/password you’ve created here instead of your admin account. #### **Connect MySQL to Secoda** After creating a MySQL user, the next step is to connect Secoda: 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select "MySQL" 3. Enter your MySQL credentials 4. Click 'Connect' ### **Security** VPCs keep servers inaccessible to traffic from the internet. With VPC, you’re able to designate specific web servers access to your servers. In this case, you will be whitelisting the Secoda IPs to read from your data warehouse. Allow Secoda to read into your MySQL database using the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda). # MySQL Metadata Extracted List of all the metadata that Secoda pulls from MySQL Secoda pulls the following metadata from MySQL: * Tables and Views * Name * Description * Schema * Database * Columns * Name * Description * Type * Sort Order * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Lineage * MySQL Table <-> My SQL Table * MySQL Column <-> MySQL Column * MySQL Column <-> MySQL View * MySQL Table <-> MySQL Table * MySQL Table/View <-> Tables from other sources * MySQL Table/View <-> Dashboards from other sources * MySQL Table/View <-> Jobs from other sources * Preview on first 50 rows (Optional) # Microsoft SQL Server Microsoft SQL Server Integration with Secoda {% content-ref url="/pages/LfnoYN3wTV1KHmrwE27D" %} [Microsoft SQL Server Metadata Extracted](/integrations/databases/microsoft-sql-server/metadata-extracted) {% endcontent-ref %} ### Getting Started with Microsoft SQL Server There are four high-level steps to start using Microsoft SQL Server with Secoda: 1. Set up environment 2. Create a database user 3. Whitelist the Secoda IP address 4. Integrate Microsoft SQL Server in Secoda #### Supported Authentication Methods | Method | How it Authenticates | Typical Use Case | | --------------------------- | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | Direct (SQL Authentication) | SQL Server username & password | Quick setup for any environment. | | Azure AD (AAD Password) | Azure Active Directory account | Cloud or hybrid environments using centralized identity management. | | Azure AD Service Principal | Azure Active Directory service principal | Similar benefits to Azure AD Password. Useful for tenants with enforced MFA. | | Windows AD (NTLMv2) | Domain credentials via NTLMv2 | On-premise or hybrid domains. Strongly recommended to pair with reverse/forward SSH tunnels to provide a layer of end-to-end encryption. | #### Set up Environment {% hint style="info" %} This step is only applicable to Azure SQL DB. {% endhint %} If using Azure, you'll need to complete some setup depending on your authentication method: **Direct** 1. Enable SQL authentication 1. Navigate to your SQL Server in the Azure portal. 2. Under Settings -> Microsoft Entra ID in the left menu, uncheck "Support only Microsoft Entra authentication for this server" and save your changes. **Azure AD Service Principal** 1. Register a service principal application 1. In the Azure portal, open Microsoft Entra ID. 2. In the left menu, navigate to Manage -> App registrations. 3. Press "New registration", create a display name for your service principal, and press Register. 4. In your created application, note down the value of "Application (client) ID". Later, you will paste this into Secoda as the "client ID". 5. Navigate to Manage -> Certificates and secrets. 6. Press "New client secret" and fill out the description and expiry fields, then press "Add". 7. Note down the value of the client secret under "Value". Later, you will paste this into Secoda as the "client secret". #### Create a Database User For each database you wish to connect to Secoda, you will need a SQL user. The username and password you originally set up for your cluster is your admin account. Keep this account for your own use. For Secoda (or any other third-party tool), create a separate, limited-scope user. How to do this depends on your chosen authentication method: **Direct & Windows AD** {% code overflow="wrap" %} ```sql -- Create a user named "secoda" that Secoda will use when connecting to your Microsoft SQL Server database. CREATE USER secoda WITH PASSWORD = ''; -- Grant read-only access on each database you would like Secoda to extract from. GRANT SELECT ON DATABASE TO secoda; ``` {% endcode %} Use these credentials (not your admin account) when configuring the integration in Secoda. **Azure AD Password** {% code overflow="wrap" %} ```sql -- Replace with the Azure AD user CREATE USER FROM EXTERNAL PROVIDER; -- Grant read-only access on each database you would like Secoda to extract from. GRANT SELECT ON DATABASE:: TO ; ``` {% endcode %} **Azure AD Service Principal** {% code overflow="wrap" %} ```sql -- Replace with the display name of your service principal CREATE USER FROM EXTERNAL PROVIDER; -- Grant read-only access on each database you would like Secoda to extract from. GRANT SELECT ON DATABASE:: TO ; ``` {% endcode %} #### Connect Microsoft SQL Server to Secoda 1. In the **Integrations** tab of the Secoda app, click **Add Integration**. 2. Select **Microsoft SQL Server**. 3. Enter your connection details. 4. Choose an authentication method. 5. Click **Connect**. {% hint style="info" %} You can connect to an entire server and integrate its databases through a single integration. This enhancement simplifies integration, which previously required separate integrations for each database. {% endhint %} #### Security If using Azure SQL DB, you will need to allow Secoda through the Azure server-level firewall: 1. In the Azure portal, navigate to your SQL Server. 2. In the left menu, navigate to Security -> Networking. 3. Add a new firewall rule for Secoda. See below for IP addresses to whitelist. If your SQL Server is inside a VPC or behind a firewall, whitelist Secoda’s outbound IP addresses so our workers can reach the host. Alternatively, use a reverse **SSH Tunnel**. Once an SSH tunnel is configured (if you are using one), choose **SSH Tunnel** in the connection form and provide the tunnel details.\ \ See the full list here: [What are the IP addresses for Secoda?](https://docs.secoda.co/faq#what-are-the-ip-addresses-for-secoda) ### Troubleshooting | Issue | Possible Cause | Resolution | | -------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | Timeout or “host unreachable” | The server is on a private network. | Whitelist the Secoda IPs or use a reverse SSH tunnel | | Authentication failures | Wrong auth type selected. | Verify you picked the correct method (SQL Login, Azure AD, or Windows AD) and that the credentials are valid. | | Permission errors | The `secoda` user lacks SELECT rights. | Re-run the `GRANT SELECT` statement for each required database. | | Login is from an untrusted domain | Wrong auth type selected or incorrect username and password | Check that NTLMv2 is enabled on the SQL server and ensure username and password are correct | | Login timeout expired (0) (SQLDriverConnect) | Connection failure while using Azure AD Service Principal | Ensure that the client ID is correct and the client secret is not expired. | # Microsoft SQL Server Metadata Extracted List of all the metadata that Secoda pulls from Microsoft SQL Server Secoda pulls the following metadata from Microsoft SQL Server: * Tables and Views * Name * Description * Schema * Database * Columns * Name * Description * Type * Sort Order * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * MS SQL Stored Procedures * Creates lineage between the tables that are in the stored procedure * Adds the stored procedure under the Queries tab of the associated tables * Lineage * MS SQL Table <-> MS SQL Table * MS SQL Column <-> MS SQL Column * MS SQL Column <-> MS SQL View * MS SQL Table <-> MS SQL Table * MS SQL Table/View <-> Tables from other sources * MS SQL Table/View <-> Dashboards from other sources * MS SQL Table/View <-> Jobs from other sources * Preview on first 50 rows (Optional) # Oracle An overview of the Oracle integration with Secoda {% content-ref url="/pages/NDET0l8AHSkerV5Zm45N" %} [Oracle Metadata Extracted](/integrations/databases/oracle-integration/metadata-extracted) {% endcontent-ref %} ## **Getting Started with Oracle** There are three steps to get started using Oracle with Secoda: 1. Create a database user 2. Whitelist Secoda IP Address 3. Connect Oracle to Secoda #### **Create a Database User** The username and password you’ve already created for your database is your admin password, which you should keep for your own usage. For Secoda, and any other 3rd-parties, it is best to create distinct users. To create a new user, you’ll need to log into the Oracle database directly and run the following SQL commands: ``` -- Create a secoda user CREATE USER secoda IDENTIFIED BY ''; -- Run this query for any schemas you'd like to import into Secoda BEGIN FOR table IN (SELECT owner, table_name FROM all_tables WHERE owner='') LOOP EXECUTE IMMEDIATE 'grant select on '||table.owner||'.'||table.table_name||' to secoda'; END LOOP; END; ``` #### **Whitelist Secoda IP Address** VPCs keep servers inaccessible to traffic from the internet. With VPC, you’re able to designate specific web servers access to your servers. In this case, you will be whitelisting the Secoda IPs to read from your data warehouse. Allow Secoda to read into your Oracle database from the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda). #### **Connect Oracle to Secoda** After creating a Oracle user, the next step is to connect Secoda: 1. In the Secoda App, select "New Integration" from 2. Search for and select "Oracle" 3. Enter your Oracle credentials 4. Click "Test Connection" 5. Click "Submit" 6. Select the schemas you'd like to import and click "Submit" 7. Click "Run Initial Extraction" # Oracle Metadata Extracted List of all the metadata that Secoda pulls from Oracle Secoda pulls the following metadata from Oracle: * Tables * Name * Description * Schema * Database * Columns * Name * Description * Type * Sort Order * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Lineage * Oracle Table <-> Oracle Table * Oracle Column <-> Oracle Column * Oracle Table <-> Tables from other sources * Oracle Table <-> Dashboards from other sources * Oracle Table <-> Jobs from other sources * Preview on first 50 rows (Optional) # Salesforce An overview of the Salesforce integration with Secoda {% content-ref url="/pages/as638oucrUEH6SdEyuFG" %} [Salesforce Metadata Extracted](/integrations/databases/salesforce-integration/metadata-extracted) {% endcontent-ref %} ### Getting Started with Salesforce There are 4 steps to get started using Salesforce with Secoda: 1. Pick a method to connect to Salesforce, there are 2 options: 1. Use your Salesforce account's username and password (tab **Password**) 2. Use Salesforce OAuth (tab **OAuth**) 2. Setup User Permissions, Connected App, and retrieve Consumer Key and Consumer Secret 3. Retrieve your host 4. Connect Salesforce to Secoda {% hint style="info" %} The current authentication flows available for the Salesforce Integration in Secoda are as follows. * **Passwords Flow:** Requires a "Connected App." Salesforce is actually sunsetting these soon, so you won't be able to create new ones in the future. * **OAuth Flow:** Works with "External Client Apps" and is the more future-proof method.
{% endhint %}
Moving forward with the Passwords Flow If you’d still prefer this route, you’ll need to manually enable Connected Apps in Salesforce first. Since new Connected Apps cannot be created, you'll have to use an existing Connected App for this connection method. * In Setup, search for "External Client Apps" and click Settings. * Enable "Allow creation of connected apps" and confirm the popup. * Click the "New Connected App" button (now enabled). * Follow steps 3, 5, and 6 in our [Salesforce Integration Guide](https://docs.secoda.co/integrations/databases/salesforce-integration).
### Setup User Permissions, Connected App, and retrieve Consumer Key and Consumer Secret Make sure the profile associated with your user has API Enabled permission. You can verify this: 1. Go to Setup > Administration > Users > Users and click your user's profile ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(3\)%20\(1\).png) 2. Make sure `API Enabled` is ticked ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(1\)%20\(4\).png) If you haven't already, create a new Salesforce Connected App: 1. Go to Setup > Platform Tools > Apps > App Manager and click New Connected App ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(6\)%20\(3\).png) 2. Follow the instruction below to complete the form to create new Connected App (or modify your existing one) * There are no specific requirements for what you name the app 3. If you're using Username & Password flow (tab **Password**): * Tick `Enable OAuth Settings` * For `Oauth Scopes`, we need at least `Manage user data via APIs (api)` * For Callback URL, you can use `http://localhost` ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(5\).png) 4. If you're using Salesforce OAuth flow (tab **OAuth**): * For Callback URL, enter `https://app.secoda.co/api/v1/oauth/from_oauth/` (or `https://.secoda.co/api/v1/oauth/from_oauth/`) * For `Oauth Scopes`, we need at least `Manage user data via APIs (api)`and `Perform requests at any time (refresh_token, offline_access)` * Tick `Enable OAuth Settings` and `Enable Authorization and Credentials Flow` * Disable `Require Proof Key for Code Exchange (PKCE)` 5. In the next step, Go to API (Enable OAuth Settings) > Manage Consumer Details to **retrieve your Consumer Key and Consumer Secret**. 1. Add Secoda's IP to Trusted IP Range for OAuth Web Server Flow. If not, see step 6.2 below to relax IP restriction. * You can find the start and end IP addresses for your region [here](https://docs.secoda.co/faq#what-are-the-ip-addresses-for-secoda) ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(16\)%20\(1\).png) 6. Go to Setup > Platform Tools > Apps > Connected Apps > Manage Connected Apps and click Edit next to your App. 1. If you're using Salesforce OAuth flow (**OAuth** tab), set Refresh Token Policy to `Refresh Token is valid until revoked` 2. If you want to relax IP restrictions. Select `Relax IP restrictions` for IP Relaxation. ![](https://secoda-public-media-assets.s3.amazonaws.com/image%20\(12\).png) ### Retrieve your host Your **host** is the url for your Salesforce instance. For example: `https://secoda.my.salesforce.com` ### **Connect Salesforce to Secoda** After retrieving your Salesforce host, Consumer Key and Consumer Secret, the next step is to connect to Secoda: 1. In the Secoda App, select "Add Integration" on the Integrations tab 2. Search for and select "Salesforce", and select the either "Password" or "Oauth" tab 3. Enter your Salesforce host, Consumer Key, and Consumer Secret 4. If you selected "Password" tab, enter your Salesforce account's username and password 5. Click 'Connect' # Salesforce Metadata Extracted List of all the metadata that Secoda pulls from Salesforce Secoda pulls the following metadata from Salesforce: * S Objects * Title * Label * Fields * Title * Description * Formula * Updated At * Dashboard * Title * Description * Last updated timestamp * URL * ApexClass * Reports * Title * Description * URL * Lineage * Salesforce Table <-> Salesforce Table * Salesforce Table <-> Salesforce Report * Salesforce Report <-> Salesforce Dashboard * Salesforce Tables/Reports <-> Jobs from other sources * Salesforce Column <-> Salesforce Column * Preview of S Objects (Optional) # Postgres An overview of the Postgres integration with Secoda {% content-ref url="/pages/HqTtE2yKFsRYyWBZqIxg" %} [Postgres Metadata Extracted](/integrations/databases/postgres-integration/postgres-metadata) {% endcontent-ref %} ## Get Started with Postgres There are three steps to connect Postgres with Postgres: 1. Create a database user 2. Connect Postgres to Secoda 3. Whitelist Secoda IP Address #### **Create a Database User** The username and password you’ve already created for your cluster is your admin password, which you should keep for your own usage. For Secoda, and any other 3rd-parties, it is best to create distinct users. To create a new user, you’ll need to log into the Postgres database directly and run the following SQL commands: ``` -- Create a user named "secoda" that Secoda will use when connecting to your Postgres database. CREATE USER secoda PASSWORD ''; -- Complete this query for any databases you would like Secoda to extract from GRANT CONNECT ON DATABASE TO secoda; -- Complete this query for any schemas you would like Secoda to extract from GRANT USAGE ON SCHEMA TO secoda; GRANT SELECT ON ALL TABLES IN SCHEMA TO secoda; ALTER DEFAULT PRIVILEGES IN SCHEMA GRANT SELECT ON TABLES TO secoda; ``` When connecting to Postgres in Secoda, use the username/password you’ve created here instead of your admin account. #### **Connect Postgres to Secoda** After creating a Postgres user, the next step is to connect Secoda: 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select ‘Postgres’ 3. Enter your Postgres credentials 4. Click 'Connect' ### **Security** VPCs keep servers inaccessible to traffic from the internet. With VPC, you’re able to designate specific web servers access to your servers. In this case, you will be whitelisting the Secoda IPs to read from your data warehouse. Allow Secoda to read into your Postgres database using the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda). # Postgres Metadata Extracted List of all the metadata that Secoda pulls from Postgres ### Metadata pulled Secoda pulls the following metadata from Postgres: * Tables and Views * Name * Description * Last Updated Timestamp * Schema * Database * View Definition (Table Creation Query) * Columns * Name * Description * Type * Foreign Key * Primary Key * Column Profile * Min * Max * Median * STD Deviation * Value Distribution * Statistic Value Count * Percent Filled * Unique * Lineage * Postgres Table <-> Postgres Table * Postgres View <-> Postgres Table * Postgres View <-> Postgres Column * Postgres Column <-> Postgres Column * Postgres Table/View <-> Tables from other sources * Postgres Table/View <-> Dashboards from other sources * Postgres Table/View <-> Jobs from other sources * Preview of first 50 rows (Optional) ### Metadata pushed If enabled, Secoda pushes the following metadata to Postgres: * Tables * Description * Columns * Description # MongoDB An overview of the MongoDB integration with Secoda {% content-ref url="/pages/ecNUtwrvu4DEJ9zqIbeO" %} [MongoDB Metadata Extracted](/integrations/databases/mongodb/data-type-conversion) {% endcontent-ref %} ### Getting started with MongoDB There are 3 steps to get started using MongoDB with Secoda 1. Create a database user 2. Whitelist Secoda IP Address 3. Connect MongoDB to Secoda ### Requirements * The MongoDB cluster must be running version 5.0 or above. Prior MongoDB versions do not support MongoDB's Stable API. You can read more about Stable API in the [MongoDB documentation](https://www.mongodb.com/docs/manual/reference/stable-api/) ### Creating a database user If you are using cloud MongoDB (Atlas), you may need to create a new database user to connect to Secoda. To do that, repeat the following steps. 1. Log into your Atlas account 2. On the sidebar go to `Security -> Database Access` and click on `ADD NEW DATABASE USER` on the top right ![](https://secoda-public-media-assets.s3.amazonaws.com/5c4fc0e4-ea34-4fc5-a862-a54b8b6f3043.png) 3. Use the password authentication method and save both the user name and password. The role given to the database user can be `Only read any database` under `built-in role` but it may need to be updated at a later date when Secoda comes out with new features ![](https://secoda-public-media-assets.s3.amazonaws.com/01d7bc51-e61e-4fa3-bc23-b2c1dc36a5ee.png) ### Whitelist Secoda IP Address Once you have created the new database user, you need to add Secoda's IP address to the allowlist. To do that, repeat the following steps. 1. Log into your Atlas Account 2. On the side, navigate to `Security -> Network Access` and click on `ADD IP ADDRESS` on the top right ![](https://secoda-public-media-assets.s3.amazonaws.com/5efdc668-7d49-48fb-9286-e2c96e75cc30.png) 3. Add the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda). ### Connect MongoDB to Secoda To connect to MongoDB to Secoda, repeat the following steps. 1. In the Secoda App, select "Add Integration" on the Integrations tab 2. Click on `MongoDB` 3. Enter the URI, cluster name, and the team the integration will be associated with ![](https://secoda-public-media-assets.s3.amazonaws.com/c95da3dc-78e8-4774-a404-8e827982e0b2.png) 1. The URI can be found by navigating to `Deployment -> Database` on Atlas and clicking on the cluster you are trying to connect to ![](https://secoda-public-media-assets.s3.amazonaws.com/77ab7a8c-417c-465c-a19d-82f6cd8ff8ce.png) 2. Click on `Driver` to see a sample URI of the cluster ![](https://secoda-public-media-assets.s3.amazonaws.com/90dedcff-64af-4a13-8660-4037bb387d31.png) 3. Copy and paste the `connection string` to the URI field, replacing `` and `` with the username and password that was used to create the database user in step 1 ![](https://secoda-public-media-assets.s3.amazonaws.com/39361fa5-1bea-4858-a5f2-57fc13ab064d.png) 4. Once successfully connected, choose the databases and collections you want to extract to Secoda ![](https://secoda-public-media-assets.s3.amazonaws.com/b583ad6e-8328-402f-bc74-7245fe8456b3.png) 5. Run the initial extraction # MongoDB Metadata Extracted List of all the metadata that Secoda pulls MongoDB Secoda pulls the following metadata from MongoDB: * Clusters * Name * Databases * Name * Collections * Name * Fields * Name * Type # Azure Cosmos DB An overview of the Azure Cosmos DB integration with Secoda {% content-ref url="/pages/2WXQzTIEMXBgTKZmvFlO" %} [Azure Cosmos DB Metadata Extracted](/integrations/databases/azure-cosmos-db/metadata-extracted) {% endcontent-ref %} ### Getting Started with Azure Cosmos DB To integrate Azure Cosmos DB with Secoda, follow these three steps: 1. Retrieve your Cosmos DB Credentials 2. Whitelist Secoda IP Addresses 3. Connect Azure Cosmos DB to Secoda #### 1. Retrieve your Cosmos DB Credentials If using the NoSQL Cosmos DB: * Navigate to the Azure portal. * Go to your Cosmos DB account. * Under 'Settings', select 'Keys'. * Create a read-only key. If using the Tables Cosmos DB: * Navigate to the Azure portal. * Go to your Cosmos DB account. * Under 'Connecting Strings', select "Primary Connecting String" #### 2. Whitelist Secoda IP Address * In Azure Cosmos DB, go to 'Networking'. * Either set public network access to `all networks` or under the `select network` add the [Secoda IP address](/faq#what-are-the-ip-addresses-for-secoda) to the firewall whitelist. #### 3. Connect Azure Cosmos DB to Secoda * Visit [Secoda's Integrations page](https://app.secoda.co/integrations). * Click "New Integration". * Search for "Azure Cosmos DB" and select it. * Select your cosmos DB type: "Tables" or "NoSQL" * Enter your Cosmos DB credentials. * Click "Test Connection". * Once successful, click "Submit". * Choose the data you want to import into Secoda. * Click "Run Initial Extraction". # Azure Cosmos DB Metadata Extracted List of all the metadata that Secoda pulls from Azure Cosmos DB Secoda pulls the following metadata from Azure Cosmos DB: #### NoSQL Cosmos DB * Databases * Name * Containers * Name * Stored Procedures * Columns * Name #### Tables Cosmos DB * Tables * Name * Columns * Name # SingleStore Getting Started with SingleStore To integrate SingleStore with Secoda, follow these three steps: 1. Create a SingleStore database user 2. Whitelist Secoda IP Addresses 3. Connect SingleStore to Secoda {% hint style="info" %} Please ensure the workspace in Singlestore is not suspended. Secoda is unable to connect to a suspended workspace. To prevent your Singlestore workspace from suspending due to inactivity, disable the auto-suspend setting in Singlestore. {% endhint %} **Create a Database User** The username and password you’ve already created for your cluster is your admin password, which you should keep for your own usage. For Secoda, and any other 3rd-parties, it is best to create distinct users. To create a new user, you’ll need to log into the SingleStore database directly and run the following SQL commands. You ``` -- Create a user named "secoda" with appropriate privileges that Secoda will use when connecting to your SingleStore database. CREATE USER 'secoda'@'%' IDENTIFIED BY '' GRANT SELECT, PROCESS, SHOW METADATA ON *.* TO 'secoda'@'%'; ``` When connecting to SingleStore in Secoda, use the username/password you’ve created here instead of your admin account. **Retrieve the host domain** To retrieve the host domain and port number, sign into SingleStore portal and navigate on the sidebar to `Cloud > Group > Workspaces`. ![](https://secoda-public-media-assets.s3.amazonaws.com/a97af923-076b-4e03-b2cb-fab280af89d7.png) Click `Connect` on the workspace you want to connect ![](https://secoda-public-media-assets.s3.amazonaws.com/2d56193b-daf6-4ad7-a2c3-33b1879bd964.png) Then you can navigate to `SQL IDE` tab to see the host and port. ![](https://secoda-public-media-assets.s3.amazonaws.com/1653ed33-3c2c-4670-8bff-f1964635b506.png) **Connect SingleStore to Secoda** After creating a SingleStore user, the next step is to connect Secoda: 1. In the Secoda App, select ‘Add Integration’ on the Integrations tab 2. Search for and select SingleStore 3. Enter your SingleStore credentials (host domain, port, username, password) 4. Click 'Connect' #### **Security** VPCs keep servers inaccessible to traffic from the internet. With VPC, you’re able to designate specific web servers access to your servers. In this case, you will be whitelisting the Secoda IPs to read from your data warehouse. Allow Secoda to read into your SingleStore database using the [Secoda IP address](https://docs.secoda.co/faq#what-are-the-ip-addresses-for-secoda). # SingleStore Metadata Extracted List of all the metadata that Secoda pulls from SingleStore Secoda pulls the following metadata from SingleStore: * Tables and Views * Name * Description * Schema * Size * Amount of rows * View Definition (Table Creation Query) * Columns * Name * Description * Type * Comments * Popularity * Lineage * SingleStore Table <--> SingleStore Table * SingleStore Column <--> SingleStore Column * SingleStore View <--> SingleStore Table * Preview of first 50 rows (Optional) # DynamoDB An overview of the DynamoDB integration with Secoda ## Getting Started with DynamoDB There are 3 steps to get started using DynamoDB with Secoda: 1. Create an IAM User or AWS Role 2. Get AWS credentials 3. Connect DynamoDB to Secoda ### Create an IAM User or AWS Role **Create a Custom Permissions Policy** 1. Navigate to the 'Policies' page in IAM and 'Create Policy' 2. Create the following custom permissions policy. You may modify "Resource" to only allow access to certain tables ``` { "Statement": [ { "Action": [ "dynamodb:ListTables", "dynamodb:DescribeTable", "dynamodb:Scan" ], "Effect": "Allow", "Resource": "*" } ], "Version": "2012-10-17" } ``` **Option 1: Create a new AWS IAM user** 3. Navigate to the 'Users' page in IAM and 'Create User' 4. In the Set Permissions tab select 'Attach policies directly'. Attach the custom permissions policy to the user. 5. Navigate to the newly created user and click 'Create Access Key' to gain Programmatic Access **Option 2: Create a new AWS Role** 1. Navigate to the 'Roles' page in IAM and 'Create Role' 2. In the 'Select trusted entity' page, click 'AWS account' and add the following account ID: 482836992928 3. Click on 'Require External ID' and copy the randomly generated value from Secoda in the DynamoDB connection page 4. In the Add Permissions tab attach your custom permissions policy to the role. ### Get AWS Credentials #### Access Key * AWS Access Key ID * AWS Secret Access Key * AWS Session Token * AWS Region where your DynamoDB tables are located #### Role * ARN Role * AWS Region where your DynamoDB tables are located ### Connect DynamoDB to Secoda 1. In the Secoda App, select 'Add Integration' on the Integrations tab 2. Search for and select DynamoDB 3. Enter your AWS credentials 4. Click 'Test connection' - if successful, you'll be prompted to run your initial sync # DynamoDB Metadata Extracted List of all the metadata that Secoda pulls from DynamoDB Secoda pulls the following metadata from DynamoDB: * Table * Name * Attribute * Name * Type * First 1000 items Since DynamoDB does not utilize databases or schemas, all tables and attributes are assigned to a default database and schema, named 'dynamodb' and 'default' respectively. # Data visualization tools A data visualization tool is software or a platform that enables users to create visual representations of data to convey insights, trends, patterns, and relationships effectively. These tools allow users to transform raw data into visually appealing charts, graphs, maps, and interactive dashboards, making complex information more understandable and accessible. These tools are often categorized as BI or Analytics tools. Secoda currently integrates with the following Data Visualization tools: {% content-ref url="/pages/PA9wgyuwsB3jd26RbDAs" %} [Amplitude](/integrations/data-visualization-tools/amplitude-integration) {% endcontent-ref %} {% content-ref url="/pages/uns4M1u28Z7fOwm6WRgQ" %} [Looker Studio](/integrations/data-visualization-tools/google-data-studio) {% endcontent-ref %} {% content-ref url="/pages/KQaGippufcbSTHpbOIM1" %} [Looker](/integrations/data-visualization-tools/looker-integration) {% endcontent-ref %} {% content-ref url="/pages/S3Rizj7h1dDucaA6nTHo" %} [Metabase](/integrations/data-visualization-tools/metabase) {% endcontent-ref %} {% content-ref url="/pages/mOaIXtsTyzk6bc0OOVMg" %} [Mixpanel](/integrations/data-visualization-tools/mixpanel) {% endcontent-ref %} {% content-ref url="/pages/uRmemIdNiLp8O5fsE6I7" %} [Mode](/integrations/data-visualization-tools/mode) {% endcontent-ref %} {% content-ref url="/pages/Brk0GpGUy0IOUWIbZzKH" %} [Power BI](/integrations/data-visualization-tools/power-bi) {% endcontent-ref %} {% content-ref url="/pages/3lY6yIgXtB20m7eMADL1" %} [QuickSight](/integrations/data-visualization-tools/quicksight-integration) {% endcontent-ref %} {% content-ref url="/pages/mVZPKaEXgg9xciFmrH49" %} [Retool](/integrations/data-visualization-tools/retool-integration) {% endcontent-ref %} {% content-ref url="/pages/GaIILuA4D82KBrxkZwaw" %} [Redash](/integrations/data-visualization-tools/redash) {% endcontent-ref %} {% content-ref url="/pages/5iTpWmSOHbmdPt9PJz1N" %} [Sigma](/integrations/data-visualization-tools/sigma-integration) {% endcontent-ref %} {% content-ref url="/pages/L0gvmdvrc6KoREcyFim4" %} [Tableau](/integrations/data-visualization-tools/tableau-integration) {% endcontent-ref %} {% content-ref url="/pages/Hf4qy9CgjyhMbAo8NgMK" %} [ThoughtSpot](/integrations/data-visualization-tools/thoughtspot) {% endcontent-ref %} {% content-ref url="/pages/UPR5UDQ7AKkEFQCdDSJf" %} [Hashboard](/integrations/data-visualization-tools/hashboard) {% endcontent-ref %} {% hint style="info" %} Don't see an integration for a tool you use? Message us on Slack or email us at and we'll add it to the roadmap. {% endhint %} # Amplitude An overview of the Amplitude integration with Secoda {% content-ref url="/pages/AZHnZEsF7AagbAFEiiZ0" %} [Amplitude Metadata Extracted](/integrations/data-visualization-tools/amplitude-integration/metadata-extracted) {% endcontent-ref %} ## Getting Started with Amplitude There are two steps to get started using Amplitude with Secoda: 1. Find your API Key and an API Secret (`Secret Key`) on Amplitude 2. Connect Amplitude to Secoda with your API Key and API Secret (`Secret Key`) ### Generate an API Key 1. Login to Amplitude and head to the Settings/Project/`Project Name` section. 2. Copy your API Key and API Secret (`Secret Key`). ### Connect to Amplitude 1. In Secoda, head to the **Integrations** page and click **New Integration** 2. Select **Amplitude** 3. Paste the API Key and API Secret (`Secret Key`) 4. Head to the **Sync** **History** tab on the side bar and click **Run sync** {% hint style="info" %} Currently only one environment gets extracted from the integration. [This documentation ](https://www.docs.developers.amplitude.com/analytics/apis/taxonomy-api/#authorization)from Amplitude explains how you can control which environment gets extracted {% endhint %} # Amplitude Metadata Extracted List of all the metadata that Secoda pulls from Amplitude If your data stack includes Amplitude, then you may want to consider integrating Secoda with your Amplitude instance. We've created a space for your event-based data to be represented within the Catalog. You can create Event resources in Secoda for each event that you track and add metadata like descriptions, related resources, owners, etc. Check out Amplitude's documentation on managing event types and properties: ### What does Secoda extract from Amplitude? * Events * Event Name * Event Type * Event Category * Event Description * Event Properties # Looker An overview of the Looker integration with Secoda {% content-ref url="/pages/jSujP7vq3ZX1F8XnuehA" %} [Looker Metadata Extracted](/integrations/data-visualization-tools/looker-integration/looker-metadata) {% endcontent-ref %} There are three steps to connect Looker with Secoda: 1. Retrieve your Looker Client ID and Client Secret 2. Retrieve your Looker host 3. Connect Looker to Secoda 4. Looker Lineage (Optional) #### **Retrieve you Looker Client ID and Client Secret** Create API3 credentials on the [Users page](https://docs.looker.com/admin-options/settings/users) in the Admin section of your Looker instance. If you’re not a Looker admin, ask your Looker admin to create the API3 credentials for you. To create an API3 credential, click **Edit** on a user and then under **API3 Keys** click **Edit Keys**. ![](https://downloads.intercomcdn.com/i/o/378332385/8e16211840f3aa4d3a3aade6/Screen+Shot+2021-08-19+at+10.45.42+PM.png) Create a key, and then save the **Client ID** and **Client Secret** #### **Retrieve your host** Your **host** is the url of your Looker instance, for example `https://company.cloud.looker.com` Sometimes an admin will have specified a custom API url. If that's the case, reach out to your admin and ask them to provide that url. #### **Connect Looker to Secoda** After retrieving your Looker Client ID, Client Secret, and host , the next step is to connect to Secoda: 1. In the Secoda App, select **Add Integration** on the Integrations tab 2. Search for and select Looker 3. Enter your Looker Client ID, Client Secret, and host you retrieved above 4. Click 'Connect' #### Connect Looker Lineage (Optional) To get lineage between Looker and your data warehouse, Secoda will need access your GitHub repo where Looker records changes and manages file versions. Secoda will then automatically detect LookML projects connected to your Looker instance. Navigate to the `Project` tab on the Looker Integration Page and you will see your LookML projects. For each project you want to connect, click the toggle beside the project name. Once the key is generate, you can select your LookML project and click **Copy public key** and head to your LookML repo in GitHub.
#### GitHub Once in your GitHub repo, click on **Settings > Deploy keys** on the sidebar. A new key can be added by clicking on **Add deploy key** button in the top right corner.
Set a title for your new key and then paste the key copied from Secoda into the **Key** text field.
#### GitLab Once in your GitLab repo, click on **Settings > Deploy keys** on the sidebar.
Set a title for your new key and then paste the key copied from Secoda into the **Key** text field. Click **Add key**. You **DO NOT** need to provide write access Once the key has been added to GitHub or GitLab go back to your Looker integration in the Secoda App and go to the **Sync History** tab and click on **Run Sync** in the top right corner to start the sync process. Note: You **DO NOT** need to start the sync process if you would rather wait until the next scheduled sync. --- [Next Page](/llms-full.txt/1)