Dataset Materialization API

The following article covers Sigma’s developer API for dataset materialization.
For Sigma’s baseline dataset API, visit Dataset API.

Summary of Content

Endpoints
The Materialization Instance Object
      Attributes
The Materialization History Object
       Attributes
List a Dataset's Materialization History
      Permissions
      Parameters
       cURL Request
       Response
       Example
Initiate a Materialization
       Permissions 
       Parameters
      cURL Request
      Response
      Example
Related Resources

 

Endpoints

GET /v2/datasets/{datasetId}/materialization

      Returns a list of materializations for the dataset. Learn more
      Try it in Swagger

POST /v2/datasets/{datasetId}/materialization

      Initiates a new materialization of the specified dataset. Learn more

      Try it in Swagger

The Materialization Instance Object

When a dataset’s materialization history is requested, one or more materialization objects may be included in the list of entries. In progress materializations are not included.

Attributes

error object | null

A Json error, if hit.

Response object structure:

{
   code: string,
   errorCode?: string | null,
   message: string | null
}

 

finishedAt string 

The timestamp at which the materialization completed.

runtimeSecs number | null

The materialization’s runtime, in seconds.

numBytes number | null

The size of the materialized table, in bytes.

numRows number | null

The number of table rows materialized.

status string | null

The materialization’s status: “success” | “failed” | “marked” | “swept”.

Note: “marked” and “swept” statuses refer to the cleanup of old materialized tables.

{
   "error": null,
   "finishedAt": "2021-10-07T23:11:56.986Z",
   "runtimeSecs": 2,
   "numBytes": 471552,
   "numRows": 12172,
   "status": "success"
},

 

The Materialization History Object

Returned in response to a request to list a dataset’s materialization history.

Attributes

entries Array<Materialization Instance>

A list of materialization instances matching the requested parameters.

Default number of entries per page: 50

hasMore boolean

True if additional materialization instances are available.

total number

That total number of materialization instances, including those on other pages.

nextPage string | null

The next page in the list of materializations. Used for pagination.

 {
    "entries": [
     <Materialization Instance>,
     <Materialization Instance>,
     ...
  ],
  "hasMore": true,
  "total": 93,
  "nextPage": "50"
  }

 

List a Dataset's Materialization History

Returns a list of materializations for a given dataset. Pagination may be required to access all available instances.

GET /v2/datasets/{datasetId}/materialization

Try it in Swagger

 

Permissions 

• You will only have access to the datasets available to the user account associated with the API token.
• The user account associated with the API token must have access to materializations. By default, materializations are only available to Admins and Creators.
  Note: If you encounter a permission error for the given endpoint, check with your organization Admin to verify your account type and permissions associated with your API token.
  
  

Parameters

page string [optional]

An identifier used for pagination.

limit number [optional]

The number of materialization entries to be returned. 

cURL Request

curl --location --request GET 'https://api.sigmacomputing.com/v2/datasets/{datasetId}/materialization?page={page}&limit={limit} \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \

 

Response

Returns Array<Dataset>.

{
    "entries": [
       <Dataset>,
       <Dataset>,
       ...
  ],
  "hasMore": boolean,
  "total": number,
  "nextPage": string | null
}

 

Example

Example Request:

curl --location --request GET 'https://api.sigmacomputing.com/v2/datasets/4337a3bb-d7a3-46e9-b929-ba264864e0ab/materialization?page=10&limit=25 \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \

Example Response:

{
  "entries": [
    {
      "error": null,
      "finishedAt": "2021-10-08T01:25:23.966Z",
      "runtimeSecs": 1,
      "numBytes": 266240,
      "numRows": 6999,
      "status": "success"
    },
    {
      "error": null,
      "finishedAt": "2021-10-07T23:11:56.986Z",
      "runtimeSecs": 2,
      "numBytes": 471563,
      "numRows": 12172,
      "status": "success"
    },
  ],
  "hasMore": false,
  "total": 2,
  "nextPage": null
}

 

Initiate a Materialization

Initiates a new materialization of the specified dataset.

POST /v2/datasets/{datasetId}/materialization

Try it in Swagger

Permissions 

• The user account associated with the API token must have permission to run a materialization on the specified dataset.
  Note: If you encounter a permission error for the given endpoint, check with your organization Admin to verify your account type and permissions associated with your API token.
  
  

Parameters

None, however as this is a POST request it is necessary to send an empty JSON object in the body 

cURL Request

curl --location --request POST 'https://api.sigmacomputing.com/v2/datasets/{datasetId}/materialization \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--d '{}'

 

Response

Returns an object containing a taskId (a unique string value identifying the materialization task).

{
  "taskId": string
}

 

Example

Example Request:

curl --location --request POST 'https://api.sigmacomputing.com/v2/datasets/4337a3bb-d7a3-46e9-b929-ba264864e0ab/materialization' \
--header 'Authorization: Bearer {access_token}' \
--header 'Content-Type: application/json' \
--d '{}'

Example Response:

{
  "taskId": "add91d11-c685-44f7-a647-903923badbb3"
}

 

Related Resources

API Documentation

Get Started with Sigma's API
Create an API Token and Client Id
Identify Unique Ids in Sigma

Product Documentation

Materialization