AdminAudit and usageUsage overview

AI usage dashboard

View as Markdown

The AI usage dashboard displays AI token usage across the entire organization for all relevant product surfaces that use AI, such as Sigma agents, Sigma Assistant, and the formula assistant.

For example, if you want to evaluate changes that you make to agent instructions or the configured AI provider, use the AI usage dashboard to review the resulting effects on usage and token consumption. If you want to evaluate changes that you make to the AI context for data models configured for Sigma Assistant or the configured AI provider, use the Assistant: Viewer tab to review the resulting effects on usage and token consumption.

User and system requirements

Review AI usage dashboard

The AI usage dashboard includes the following information. You can filter by product and date range, or choose to filter by user, sentiment, model used, agent, or team name:

SectionDescription
KPI chartsDetails about total AI token usage for the organization, active users, total requests sent to AI providers, and average response time in seconds.
TokensDetails about token consumption over time, top users consuming tokens, tokens used by product, and tokens used by agent.
EngagementDetails about active users over time, total requests sent to AI providers by product, requests sent over time, total users by agent, and active users by product.
ConversationsDetails about recent conversations in all products. View details about the full conversation, including user input and AI output, user name, input and output tokens, total turns and duration of the conversation, model used and product that the conversation took place in.
ModelsDetails about requests by model, tokens by model, average response time by model, and unique users by model. Sigma chooses the model based on your configured AI provider.

You can also use natural language to ask questions about the data using the AI usage analyst agent. No tokens are consumed by the AI usage analyst unless you interact with the agent.

If you want to customize the AI usage dashboard, click Add to workspace to open the dashboard for editing as a new workbook in your Documents folder.

Set up alerts for token consumption

If you want to monitor token consumption across your organization, you can set up a conditional alert on a customized version of the AI usage dashboard.

For example, send a notification when total token usage crosses a threshold of 10,000 tokens per day.

You can alert on any data available in the dashboard, such as token consumption by product, or customize the data view to show different time granularity.

  1. On the AI usage dashboard, click Add to workspace to open the dashboard as a new workbook in your Documents folder.

  2. To notify on total tokens consumed, select the Tokens KPI chart.

  3. For the KPI chart, click More to open the element menu and select Alert when.

    The Schedule exports modal opens.

  4. In this example, set up an email notification. For Recipient, enter one or more email addresses to be sent an email when token consumption exceeds the threshold.

  5. For Subject, enter a subject for the email. For example, “Sigma AI usage over threshold”. Optionally use dynamic text to specify a formula and display the threshold value in the subject. For example, use the formula: Max([Total tokens consumed 🔑/Total conversation tokens]).

  6. For Message, enter a message for the email. Optionally use dynamic text to specify a formula and display the threshold details in the message. For example, “Token consumption for Sigma AI-powered features is over the alert threshold. Review the usage patterns.”, and optional formulas like the following:

    • To show the total tokens consumed, use a formula like: Max([TOKENS/Tokens]).
    • To show how many of those tokens are used by Sigma agents, use a formula like: If([TOKENS/Product]="Agent", Max([TOKENS/Tokens]), 0).

  7. In the Attachments section, change the attachment to provide relevant details. For example, choose a CSV of the AI Usage Table element to more easily review product usage details, or choose a PNG of the KPI chart to make the token consumption more visible.

  8. For Frequency, select the frequency at which you want the alert to trigger. Because the alert is conditional, the frequency is how often the conditions are evaluated before sending an email. In this example, you want to evaluate token consumption per day, so set a frequency of daily.

    Because you set up an alert, Send is set to If a condition is met.

  9. In the Condition section, configure the threshold that triggers the alert:

    1. Leave the defaults for Send to If a condition is met and Data element to the New Overview: TOKENS element.
    2. Set Check if to Any value.
    3. For In column, select TOKENS.
    4. For Is, select Greater than.
    5. For Value, enter the token threshold you want to be notified at. For example, 10000.

  10. Turn on the Stop notifying after 1 occurrences in a day toggle to avoid getting multiple alerts for the same activity.

  11. Filter the table to the last 24 hours of data:

    1. Select the checkbox for Customize control values, then select time-filter, the date range control.
    2. Modify the selection to Last 24 hours.

    If you want to alert on more specific token consumption, choose another control to customize:

    • To alert on token consumption for Sigma agents, select the Product control, then specify a product. For example, Agent.
    • To alert on token consumption for a particular team, select the team-control, then specify a team name.
    • To alert on token consumption for a particular user, select the user-name-control, then specify a user name.

  12. Click Create to save the alert.

For more options, see Schedule a conditional export or alert. Test your export by sending it to yourself to confirm that the data looks as you expect.

You can use alerts to monitor consumption, but if possible, set spending limits or budgets in your AI provider.

If you use Snowflake as your AI provider, consider creating a custom budget to cap and track AI spending. See Custom budgets in the Snowflake documentation. For other AI providers, manage or limit consumption using the controls available in that provider.

AI usage table in connection browser

If you are granted access to the relevant schema, you can also explore the AI usage data in the relevant table in the connection browser.

AI usage is stored in a table called AI_USAGE_<ORGANIZATION_NAME> in the configured location.

Column nameDescription
Trace Created At UtcTimestamp when the interaction was started, in UTC.
Trace IdUnique identifier for each turn in a conversation, per user. For example, if a user asks a question and sees a response, that is one turn and is grouped with one Trace ID.
Span IdUnique identifier for each call made by the LLM. For example, an API call, a tool call, or an MCP call each results in a different Span ID as part of a chat session.
Parent IdUnique identifier of the Span Id that caused the activity.
Span TypeType of usage. generation indicates a response, span indicates a call made by an LLM, tool indicates a tool call performed by an agent, such as an action or MCP server, or score indicates user feedback about an agent response.
Trace NameType of usage associated with the Span Type. For example, chat for a Sigma agent interaction in a chat associated with a Span Type of generation, or database_execute_query for a tool call performed by a Sigma agent for a Span Type of tool.
Agent NameIf a Sigma agent, the name of the Sigma agent, otherwise null.
Agent IdUnique identifier of the Sigma agent.
User NameFull name of the user.
Session IdUnique identifier for associated conversations or interactions with an AI feature. For example, one chat session with a Sigma agent.
Ai ProviderType of AI provider configured for the organization.
ModelLLM model used. Should match the supported model for the configured AI provider.
Tokens InputTotal tokens consumed for the input (question).
Tokens OutputTotal tokens consumed for the output (response).
TagsNot yet used. Array of strings for filtering.
Tool CallsDetails about tool calls used for generation. If Span Type is generation, JSON array of results from a previous tool call, otherwise null.
InputInput for the LLM. If Span Type is tool, JSON array of tool call details. For example, for a Trace Name of database_execute_query, includes the SQL statement run for the tool call and the title shown to the user in the chat (if applicable) during the tool call. If Span Type is custom-agent, includes a string of the message sent to the agent. If Span Type is generation, includes all details sent to the LLM.
OutputOutput from the LLM. If Span Type is tool, JSON object with the name, output, role, and toolCallId of the tool call performed. If Span Type is custom-agent, includes a string of the message sent by the agent. If Span Type is generation, includes a JSON object with the organizationId, sigmaRequestId, and system.
MetadataJSON object with metadata about the AI usage. For usage associated with a Sigma agent, includes metadata like the agentId, agentName, and agentToolConfig. For usage associated with Sigma Assistant, includes a JSON object with the organizationId, sigmaRequestId, and system.