Configure storage integration for file upload columns (Beta)

🚩
This documentation describes one or more public beta features that are in development. Beta features are subject to quick, iterative changes; therefore the current user experience in the Sigma service can differ from the information provided in this page.
This page should not be considered official published documentation until Sigma removes this notice and the beta flag on the corresponding feature(s) in the Sigma service. For the full beta feature disclaimer, see Beta features.

Add a storage integration for file uploads to support file columns in input tables and file upload controls.

This document explains how to configure a storage integration with AWS or GCP.

📘
Much of the storage integration configuration involves completing steps within your cloud storage platform. Because these workflows are maintained and updated by a third party, the steps detailed in this document may differ from the cloud storage platform's current UI and terminology.

System and user requirements

The ability to configure a storage integration for a file upload column requires the following:

You must be assigned the Admin account type in Sigma.
You must be granted administrative permissions (or have the ability to create and manage a storage bucket) in one of the following cloud providers:
- Amazon Web Services (AWS) (S3-compatible providers are not supported)
- Google Cloud (GCP)
In the cloud provier, you must also be granted permissions required to create and manage core security policies (e.g., IAM roles, ARN definitions, or trust policies).

Configure a storage integration to upload files to AWS

To configure a storage integration that allows Sigma to upload files to AWS, you must complete the following procedures:

Create an S3 bucket and IAM policy in AWS to enable secure file uploads.
Create a custom IAM role in AWS to grant permissions to the S3 bucket.
Add an AWS integration in Sigma to enable file upload columns in input tables.
Update the custom IAM role in AWS to grant the IAM user access to the S3 bucket.
Enable CORS for the S3 bucket in AWS to allow cross-origin requests.

Create an S3 Bucket and IAM Policy in AWS

In AWS, create an S3 bucket and an IAM Policy to allow bucket access. See the AWS documentation on Creating a general purpose bucket and Creating IAM policies.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetBucketLocation",
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::{{customer_s3_bucket_name}}"
        },
        {
            "Action": "s3:ListBucket",
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::{{customer_s3_bucket_name}}",
            "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "{{prefix}}/*"
                    ]
                }
            }
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:PutObjectTagging"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::{{customer_s3_bucket_name}}/{{prefix}}/*"
        }
    ]
}

Create a custom IAM role in AWS

In AWS, create a custom IAM role before adding an AWS integration in Sigma (the integration requires AWS credentials). See the AWS documentation on Creating an IAM role.

While creating the IAM role, ensure that your configurations match these requirements for the integration with Sigma:

Select AWS Account as the trusted entity type.
When prompted for an Account ID, you should use your AWS account ID as a temporary value. After you add an AWS integration in Sigma, you must update the IAM role to modify the trusted relationship and grant access to Snowflake.
When creating the role, ensure you select Require external ID.
When prompted for an external ID, enter a placeholder value (for example, 0000). Sigma generates an external ID when you add an AWS integration, after which you must update the IAM role.
When selecting permissions, use the IAM policy you just created.

Add an AWS integration in Sigma

You can now add an AWS integration in Sigma.

In Sigma, go to Administration > Account > General Settings.
In the Storage Integration > External storage integration section, click Add.
In the Add storage integration modal, provide the AWS credentials.
1. In the Provider section, select AWS S3.
2. In the AWS IAM role ARN field, enter the Role ARN value obtained when you created the IAM role.
3. In the Bucket name field, enter the S3 destination folder path that includes the bucket and folder path prefix specified in the IAM policy.
4. In the Path prefix field, enter any folder path prefix.
Click Save, then record the AWS IAM user ARN and AWS external role ARN displayed in the integration details.

Update the custom IAM role trust in AWS

In AWS, edit the trust policy document using the ARN values recorded after you created the integration in Sigma. See the AWS documentation on Editing the trust relationship for an existing role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": <aws_iam_user_arn>
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": <aws_external_role_arn>
                }
            }
        }
    ]
}

Enable CORS for the S3 bucket in AWS

In AWS, enable cross-origin resource sharing (CORS) for the S3 bucket. See the AWS documentation on Configuring cross-origin resource sharing (CORS).


    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "PUT",
            "POST"
        ],
        "AllowedOrigins": [
            "https://app.sigmacomputing.com"
        ],
        "ExposeHeaders": [
            "Access-Control-Allow-Origin"
        ]
    }

Configure a storage integration to upload files to GCP

To configure a storage integration that allows Sigma to upload files to GCP, you must complete the following procedures:

Configure a target bucket in GCP to store uploaded files.
Configure a service account in GCP to mediate access to the bucket.
Add a GCP integration in Sigma to enable file upload columns in input tables.
Grant impersonation access to the service account in GCP to allow Sigma to access the bucket.
Enable CORS for the target bucket in GCP to allow cross-origin requests.

Configure a target bucket in GCP

In GCP, create a dedicated target bucket for file uploads. You can select storage options as you see fit, but Standard storage is recommended. See the GCP documentation on creating buckets.

Configure a service account in GCP

In GCP, configure a service account with Storage Object Admin permissions to mediate access to the bucket. See the GCP documentation on creating service accounts.

Add a GCP integration in Sigma

You can now add a GCP integration in Sigma.

In Sigma, go to Administration > Account > General Settings.
In the Storage Integration > External storage integration section, click Add.
In the Add storage integration modal, provide the GCP credentials.
1. In the Provider section, select Google Cloud Storage.
2. In the Service account field, enter the service account ID.
3. In the Bucket name field, enter the name of the GCP bucket.
Click Save.

Grant impersonation access to the service account in GCP

In GCP, grant impersonation access to the service account with the Service Account Token Creator role. See the GCP documentation on service account impersonation and managing access to service accounts.

Enable CORs for the target bucket in GCP

In GCP, enable cross-origin resource sharing (CORS) for the target bucket. See the GCP documentation on setting up CORS.

[
    {
      "origin": ["https://app.sigmacomputing.com"],
      "method": ["GET", "POST", "PUT"],
      "responseHeader": ["*"],
      "maxAgeSeconds": 3600
    }
]

Updated about 12 hours ago