AdminAudit and usageUsage overview

AI usage dashboard

View as Markdown

The AI usage dashboard displays AI token usage across the entire organization for all relevant product surfaces that use AI, such as Sigma agents and Sigma Assistant.

For example, if you want to evaluate changes that you make to agent instructions or the configured AI provider, use the AI usage dashboard to review the resulting effects on usage and token consumption. If you want to evaluate changes that you make to the AI context for data models configured for Sigma Assistant or the configured AI provider, use the Assistant: Viewer tab to review the resulting effects on usage and token consumption.

User and system requirements

Review AI usage dashboard

The AI usage dashboard includes the following tabs:

Tab nameDescription
OverviewDetails about AI token usage for the organization, such as usage by day by product surface, token usage per day, and total input and output tokens by product surface. Filter by user, product surface, sentiment of the conversation, or date.
AgentsDetails about agent usage for the organization, such as token usage per agent, agent popularity, token consumption per day, and requests per day. Filter by user, agent name, conversation sentiment, or date.
Assistant: ViewerDetails about usage of Assistant in analyze mode, such as requests per day, token consumption per day, top data sources used to answer questions, total input and output tokens consumed, and total users. Filter by user, conversation sentiment, or date.
Assistant: BuilderDetails about usage of Assistant in build or plan mode, such as requests per day, token consumption per day, total input and output tokens consumed, and total users. Filter by user, conversation sentiment, or date.

If you want to customize the AI usage dashboard, click Add to workspace to open the dashboard for editing as a new workbook in your Documents folder.

Set up alerts for token consumption

If you want to monitor token consumption across your organization, you can set up a conditional alert on a customized version of the AI usage dashboard.

For example, send a notification when total token usage crosses a threshold of 10,000 tokens per day.

You can alert on any data available in the dashboard, such as token consumption by product, or customize the data view to show different time granularity.

  1. On the AI usage dashboard, click Add to workspace to open the dashboard as a new workbook in your Documents folder.

  2. To notify on total tokens consumed, select the Total tokens consumed KPI chart on the Overview tab.

  3. For the KPI chart, click More to open the element menu and select Alert when.

    The Schedule exports modal opens.

  4. In this example, set up an email notification. For Recipient, enter one or more email addresses to be sent an email when token consumption exceeds the threshold.

  5. For Subject, enter a subject for the email. For example, “Sigma AI usage over threshold”. Optionally use dynamic text to specify a formula and display the threshold value in the subject. For example, use the formula: Max([Total tokens consumed 🔑/Total conversation tokens]).

  6. For Message, enter a message for the email. Optionally use dynamic text to specify a formula and display the threshold details in the message. For example, “Token consumption for Sigma AI-powered features is over the alert threshold. Review the usage patterns.”, and optional formulas like the following:

    • To show the total tokens consumed, use a formula like: Max([Total tokens consumed 🔑/Total conversation tokens]).
    • To show how many of those tokens are used by Sigma agents, use a formula like: If([Total tokens consumed 🔑/Product]="Agent", Max([Total tokens consumed 🔑/Total conversation tokens]), "N/A").

  7. In the Attachments section, change the attachment to provide relevant details. For example, choose a CSV of the AI Usage Table element to more easily review product usage details, or choose a PNG of the KPI chart to make the token consumption more visible.

  8. For Frequency, select the frequency at which you want the alert to trigger. Because the alert is conditional, the frequency is how often the conditions are evaluated before sending an email. In this example, you want to evaluate token consumption per day, so set a frequency of daily.

    Because you set up an alert, Send is set to If a condition is met.

  9. In the Condition section, configure the threshold that triggers the alert:

    1. Leave the defaults for Send to If a condition is met and Data element to the Overview: Token usage by day element.
    2. Set Check if to Any value.
    3. For In column, select Total tokens consumed.
    4. For Is, select Greater than.
    5. For Value, enter the token threshold you want to be notified at. For example, 10000.

  10. Turn on the Stop notifying after 1 occurrences in a day toggle to avoid getting multiple alerts for the same activity.

  11. Filter the table to the last 24 hours of data:

    1. Select the checkbox for Customize control values, then select New-Control-37, the date control on the Overview tab.
    2. Modify the selection to Last 24 hours.

    If you want to alert on token consumption for Sigma agents specifically, choose another control to customize and select New-Control-35, the list values control that filters Product on the Overview tab, then specify a product. For example, Agent.

  12. Click Create to save the alert.

For more options, see Schedule a conditional export or alert. Test your export by sending it to yourself to confirm that the data looks as you expect.

You can use alerts to monitor consumption, but if possible, set spending limits or budgets in your AI provider.

If you use Snowflake as your AI provider, consider creating a custom budget to cap and track AI spending. See Custom budgets in the Snowflake documentation. For other AI providers, manage or limit consumption using the controls available in that provider.

AI usage table in connection browser

If you are granted access to the relevant schema, you can also explore the AI usage data in the relevant table in the connection browser.

AI usage is stored in a table called AI_USAGE_<ORGANIZATION_NAME> in the configured location.

Column nameDescription
Trace Created At UtcTimestamp when the interaction was started, in UTC.
Trace IdUnique identifier for each turn in a conversation, per user. For example, if a user asks a question and sees a response, that is one turn and is grouped with one Trace ID.
Span IdUnique identifier for each call made by the LLM. For example, an API call, a tool call, or an MCP call each results in a different Span ID as part of a chat session.
Parent IdUnique identifier of the Span Id that caused the activity.
Span TypeType of usage. generation indicates a response, span indicates a call made by an LLM, tool indicates a tool call performed by an agent, such as an action or MCP server, or score indicates user feedback about an agent response.
Trace NameType of usage associated with the Span Type. For example, chat for a Sigma agent interaction in a chat associated with a Span Type of generation, or database_execute_query for a tool call performed by a Sigma agent for a Span Type of tool.
Agent NameIf a Sigma agent, the name of the Sigma agent, otherwise null.
Agent IdUnique identifier of the Sigma agent.
User NameFull name of the user.
Session IdUnique identifier for associated conversations or interactions with an AI feature. For example, one chat session with a Sigma agent.
Ai ProviderType of AI provider configured for the organization.
ModelLLM model used. Should match the supported model for the configured AI provider.
Tokens InputTotal tokens consumed for the input (question).
Tokens OutputTotal tokens consumed for the output (response).
TagsNot yet used. Array of strings for filtering.
Tool CallsDetails about tool calls used for generation. If Span Type is generation, JSON array of results from a previous tool call, otherwise null.
InputInput for the LLM. If Span Type is tool, JSON array of tool call details. For example, for a Trace Name of database_execute_query, includes the SQL statement run for the tool call and the title shown to the user in the chat (if applicable) during the tool call. If Span Type is custom-agent, includes a string of the message sent to the agent. If Span Type is generation, includes all details sent to the LLM.
OutputOutput from the LLM. If Span Type is tool, JSON object with the name, output, role, and toolCallId of the tool call performed. If Span Type is custom-agent, includes a string of the message sent by the agent. If Span Type is generation, includes a JSON object with the organizationId, sigmaRequestId, and system.
MetadataJSON object with metadata about the AI usage. For usage associated with a Sigma agent, includes metadata like the agentId, agentName, and agentToolConfig. For usage associated with Sigma Assistant, includes a JSON object with the organizationId, sigmaRequestId, and system.