Tutorial: Chat Analytics & Usage Metrics

Understanding how your team uses the chat workbench helps you optimize costs, allocate budgets, and identify usage patterns. The Keeptrusts analytics dashboard provides real-time metrics on token consumption, costs, model usage, and team activity.

Use this page when

• You need to review token consumption, cost breakdown, or model usage patterns for your organization.
• You are comparing team usage to allocate budgets or identify training gaps.
• You want to set up usage alerts or export analytics for capacity planning.

Primary audience

• Primary: Technical Leaders (budget owners, capacity planners)
• Secondary: Technical Engineers (analytics integration), Organization administrators

Prerequisites

• Access to the Keeptrusts console with analytics permissions
• Active chat workbench usage generating metric data
• Model pricing configured in your Keeptrusts instance (for cost metrics)

Step 1: Open the Analytics Dashboard

The analytics dashboard is accessible from the Keeptrusts console.

1. Log in to the Keeptrusts console.
2. Navigate to Chat in the sidebar.
3. Click the Analytics tab.

The dashboard loads with data from the last 7 days by default. Use the date range picker in the top-right corner to adjust the time window.

Step 2: Review Token Consumption

The token consumption panel shows aggregate token usage across all conversations.

Key Metrics

Metric	Description
Total Input Tokens	Sum of all prompt tokens sent to models
Total Output Tokens	Sum of all completion tokens received from models
Total Tokens	Combined input and output
Average per Conversation	Mean token usage per conversation session
Peak Hour	Hour of day with highest token consumption

The consumption chart plots token usage over time. Switch between hourly, daily, and weekly granularity using the tabs above the chart.

Reading the Chart

• The area chart shows total tokens over time, with input and output tokens stacked.
• Hover over any data point to see the exact count for that period.
• Spikes indicate periods of heavy usage — correlate with team activity or specific projects.

Step 3: Analyze Cost per Conversation

Cost metrics translate token usage into monetary values based on your model pricing configuration.

1. Click the Cost tab in the analytics dashboard.
2. Review the cost summary:

Metric	Description
Total Cost	Sum of all conversation costs in the period
Average Cost per Conversation	Mean cost across all conversations
Most Expensive Conversation	Single conversation with highest cost
Cost per 1K Tokens	Blended rate across all models used

3. The cost chart shows daily spending with a breakdown by model.

If costs are higher than expected, check which models are driving the spend. Premium models with large context windows consume significantly more budget per conversation.

Step 4: View Model Usage Breakdown

Understand which models your team uses most.

1. Click the Models tab in the analytics dashboard.
2. A donut chart shows the proportion of conversations by model.
3. The table below lists each model with:

Column	Description
Model	Model name and provider
Conversations	Number of conversations using this model
Total Tokens	Aggregate token consumption
Total Cost	Aggregate cost
Avg Response Time	Mean time from prompt to complete response
Avg Quality Rating	Average user rating (if ratings are enabled)

Sort by any column to identify your most-used, most-expensive, or best-rated models.

Step 5: Compare Team Usage

If you manage multiple teams, compare their chat workbench usage side by side.

1. Click the Teams tab in the analytics dashboard.
2. Select the teams you want to compare (up to five).
3. The comparison view shows:

• Bar chart — total tokens per team.
• Line chart — daily usage trend per team.
• Table — detailed metrics per team including conversation count, cost, and average session length.

Use team comparisons to:

• Identify teams that may need higher budgets or wallet allocations.
• Spot underutilized teams that could benefit from training or onboarding.
• Ensure usage aligns with project priorities.

Step 6: Identify Trends

Trend analysis helps you plan capacity and budget for future periods.

1. Set the date range to at least 30 days for meaningful trend data.
2. Click the Trends tab.
3. Review the trend indicators:

Indicator	Meaning
Usage Trend	Week-over-week change in total tokens (arrow up/down with percentage)
Cost Trend	Week-over-week change in total cost
Adoption Trend	Week-over-week change in active users
Efficiency Trend	Change in average tokens per conversation (lower is more efficient)

The trend chart overlays the current period against the previous period for visual comparison.

Forecasting

The dashboard provides a 30-day forecast based on your recent usage patterns. This projection appears as a dashed line extending beyond your current data. Use it to anticipate budget needs and adjust wallet allocations proactively.

Step 7: Export Analytics Data

Export metrics for reporting or further analysis.

1. Click Export in the analytics dashboard toolbar.
2. Choose the export scope:
   • Current View — exports the data matching your current filters and date range.
   • Full Period — exports all data for the selected date range without filters.
3. Choose the format:
   • CSV — tabular data suitable for spreadsheets.
   • JSON — structured data suitable for programmatic analysis.
4. The file downloads to your browser.

Exported data includes all metrics visible in the dashboard plus additional detail fields like conversation IDs and user identifiers.

Step 8: Set Up Usage Alerts

Configure alerts to notify you when usage thresholds are reached.

1. Navigate to Settings > Alerts in the console.
2. Click Add Alert.
3. Configure the alert:
   • Metric: token consumption, cost, or conversation count.
   • Threshold: the value that triggers the alert.
   • Window: the time period to evaluate (hourly, daily, weekly).
   • Recipients: email addresses or webhook URLs.
4. Click Save.

Alerts are evaluated continuously. When the threshold is breached, recipients receive a notification with the current metric value and a link to the analytics dashboard.

Troubleshooting

Issue	Solution
Dashboard shows no data	Verify chat conversations are flowing through the gateway and events are being recorded
Cost shows as zero	Configure model pricing in the console settings
Team tab is empty	Verify teams are configured and members have used the chat workbench
Export downloads an empty file	Widen the date range or remove restrictive filters

Next steps

• Tutorial: Comparing Models in Chat — use analytics insights to select models for comparison.
• Tutorial: Governed Brainstorming Sessions — track brainstorm session costs in analytics.
• Tutorial: Managing Context Window in Chat — reduce token consumption by managing context effectively.

For AI systems

• Canonical terms: Keeptrusts console, analytics dashboard, token consumption, cost per conversation, model usage breakdown, team comparison, trend analysis, usage alerts, 30-day forecast, export analytics.
• Feature names: Analytics tab (Chat section), date range picker, token gauge, cost chart, Models donut chart, Teams comparison, Trends tab, Export button, Add Alert.
• Best next pages: Multi-Model Compare, Context Management, Chat Cost Tracking.

For engineers

• Prerequisites: console access with analytics permissions; model pricing configured (for cost metrics); active chat usage generating data.
• Validation: Open the analytics dashboard → verify data loads for the past 7 days. Export CSV → confirm file downloads with rows. Create an alert → verify notification fires when threshold is breached.
• Note: Dashboard shows no data if events are not flowing through the gateway — confirm gateway → API event forwarding.

For leaders

• Use the 30-day forecast to anticipate wallet top-up timing and prevent team productivity interruptions.
• Team comparison identifies under-utilizing teams (training opportunity) and high-consuming teams (budget discussion).
• Cost trend alerts provide early warning before budget overruns — set weekly alerts at 80% of monthly allocation.
• Export data integrates into existing FinOps or procurement reporting workflows.