Connect to Databricks

This document provides instructions on how to configure a Databricks connection with Sigma. 

Summary of Content

Requirements
Configure Databricks
Configure Sigma

Requirements

• Admin privileges in your Sigma organization. Learn about account types.

Configure Databricks

Complete the steps below in Databricks before you add a Databricks connection to Sigma.

1. Create a Databricks SQL endpoint if one doesn't already exist. Learn more about Databricks SQL.
   Note: Databricks SQL endpoints are only available for Databricks accounts on their Premium plan or above.
   
2. We recommend that you disable Auto stop on your Databricks endpoint. In the Databricks SQL menu, go to SQL Warehouses and select the endpoint to use in Sigma.
   
3. Click Edit in the more ••• menu and turn off Auto stop.
   Note: If Auto stop is enabled for your Databricks SQL endpoint, then your first query will be slow or timeout when the SQL endpoint is in a suspended state.  
   
   
   
4. Go to the SQL Admin Console to grant privileges to Databrick users or groups that’ll be added to Sigma. For most use cases, SELECT privileges on tables and USAGE privileges on schemas/catalogs are sufficient.
   
   Note: You need to grant SELECT privileges for each table in a schema, even if the user has USAGE privileges on the schema. Additionally, tables in the catalog hive_metastore require READ_METADATA privileges.
   To enable the use of writeback features, such as Materialization, CSV Upload, and other forthcoming write features, you’ll also need to grant MODIFY privileges on a write schema.
   
   Unity Catalog is Databricks’s newest catalog and is currently in public preview. Eventually, it will replace Hive Metastore. If Unity Catalog is enabled, Sigma can access catalogs, schemas, and tables managed by Unity Catalog. 

Configure Sigma

1. To add a new connection, open your Admin Portal by selecting Administration in the user menu at the top right of your screen.
   
   
2. Select the Connections from the left-hand panel.
3. Click Create Connection.
   
   
4. Choose Databricks.
    
   
   
   
5. To find information for the Connection Credentials section, go to Databricks > Databricks SQL menu > SQL Warehouses. 
6. Select the endpoint to use in Sigma and click Connection details.
7. Enter the Server Hostname found in Databricks, in the Host field in Sigma.
8. Enter the HTTP path found in Databricks, in the SQL endpoint field in Sigma . 
9. Generate a token in Databricks, using these instructions, and enter the value in the Access token field in Sigma. 
10. Click Create in Sigma.