Tutorial: Customizing the Overview Dashboard

This tutorial shows you how to customize the Keeptrusts console Overview Dashboard. You will learn to configure date ranges, understand key metrics widgets, drill down into event data, and set up dashboard alerts for proactive monitoring.

Use this page when

• You want to customize which metrics widgets appear on your Overview Dashboard.
• You need to configure date ranges, drill-down views, or auto-refresh for real-time monitoring.
• You are setting up dashboard-level alerts for policy violations, spend spikes, or gateway health.
• You want to understand what each default widget (Total Events, Policy Decisions, Cost, Gateway Health) shows.

Primary audience

• Primary: Platform engineers and on-call operators who use the dashboard for daily AI governance monitoring
• Secondary: Technical leaders reviewing aggregated metrics; compliance officers tracking policy decision trends

Prerequisites

• Logged in to the Keeptrusts console (see First Login & Console Setup)
• At least one active configuration with gateway traffic generating events
• Admin or Editor role for dashboard customization

Step 1: Open the Overview Dashboard

1. Click Overview in the left sidebar (or click the Keeptrusts logo in the top-left corner).
2. The dashboard loads with default widgets showing your organization's AI governance metrics.

Step 2: Configure the Date Range

The date range selector controls the time window for all dashboard metrics.

1. Locate the date range picker in the top-right area of the dashboard.
2. Click the date range dropdown to see preset options:
   • Last 24 hours — Real-time monitoring
   • Last 7 days — Weekly overview (default)
   • Last 30 days — Monthly summary
   • Last 90 days — Quarterly trends
   • Custom range — Select specific start and end dates
3. Select your preferred range.
4. All widgets and charts update automatically to reflect the selected time window.

Use Last 24 hours when investigating a recent incident or verifying a newly deployed configuration. Use Last 30 days for regular governance reviews and reporting.

Step 3: Understand the Key Metrics Widgets

The dashboard includes several metric widgets displayed as cards at the top of the page.

Total Events

• What it shows: The total count of LLM requests processed through all gateways in the selected time range.
• Trend indicator: An arrow showing the percentage change compared to the previous equivalent period.
• Click to drill down: Opens the Events page filtered to the selected date range.

Blocked Requests

• What it shows: The count of requests blocked by policy enforcement.
• Breakdown: Hover to see a tooltip with blocked counts by policy type (content filter, cost limit, prompt injection).
• Click to drill down: Opens the Events page filtered to status=blocked.

Active Escalations

• What it shows: The count of pending escalations awaiting human review.
• Severity breakdown: Color-coded badges show counts by severity (critical, high, medium, low).
• Click to drill down: Opens the Escalations page.

Cost Summary

• What it shows: Total LLM spend across all teams for the selected period.
• Breakdown: Displays top-spending teams and models in a mini chart.
• Click to drill down: Opens the Usage page.

Step 4: Explore Dashboard Charts

Below the metric cards, the dashboard displays interactive charts.

Events over time

A line chart showing event volume over the selected date range.

1. Hover over any data point to see the exact count and timestamp.
2. The chart displays separate lines for allowed, blocked, and escalated events.
3. Click any data point to navigate to the Events page filtered to that specific time window.

Policy decision distribution

A donut chart showing the breakdown of policy decisions.

1. Segments represent each decision type: allowed, blocked, redacted, escalated, flagged.
2. Click a segment to filter the Events page by that decision type.

Top triggered policies

A horizontal bar chart showing which policies triggered most frequently.

1. Each bar represents a policy rule with its trigger count.
2. Click a bar to open the Events page filtered to that specific policy.

Cost by team

A stacked bar chart showing LLM spend per team over time.

1. Each color represents a different team.
2. Hover to see the exact cost for each team on a given day.
3. Click a bar segment to navigate to the Usage page for that team or scope.

Step 5: Drill Down into Events

From any dashboard widget or chart, you can drill down to the underlying events.

1. Click the Total Events metric card.

2. The Events page opens with the dashboard's date range pre-applied.

3. Use the Events page filters to narrow down further:

   • Policy — Filter by a specific policy name
   • Status — Filter by decision (allowed, blocked, escalated)
   • Provider — Filter by LLM provider
   • Model — Filter by specific model
   • Consumer Group — Filter by team or application

4. Click any event row to open the full event detail view with request/response data.

Step 6: Set Up Dashboard Alerts

Dashboard alerts notify you when key metrics cross defined thresholds.

Create a new alert

1. Click the Alerts icon (bell icon) in the top-right corner of the dashboard.
2. Click Create Alert.
3. Fill in the alert configuration:
   • Alert Name — A descriptive name (e.g., "High block rate alert")
   • Metric — Select the metric to monitor:
     • Total events
     • Blocked requests
     • Block rate (percentage)
     • Active escalations
     • Total cost
   • Condition — Choose the trigger condition:
     • Greater than — Triggers when the metric exceeds the threshold
     • Less than — Triggers when the metric drops below the threshold
     • Percentage change — Triggers on a significant change from the previous period
   • Threshold — Enter the numeric threshold value
   • Time Window — The evaluation window (1 hour, 6 hours, 24 hours)
   • Notification Channel — Where to send the alert:
     • Console notification (in-app bell icon)
     • Email
     • Webhook (for Slack, Teams, PagerDuty integrations)
4. Click Save Alert.

Example alerts

Alert Name	Metric	Condition	Threshold	Window
High block rate	Block rate	Greater than	25%	1 hour
Cost spike	Total cost	Percentage change	50% increase	24 hours
Escalation backlog	Active escalations	Greater than	20	6 hours
Low traffic	Total events	Less than	100	24 hours

Manage existing alerts

1. Click the Alerts icon to view all configured alerts.
2. Each alert shows its status (active, triggered, snoozed).
3. Click an alert to edit its configuration.
4. Use the toggle to enable or disable an alert without deleting it.
5. Click the trash icon to delete an alert permanently.

Step 7: Refresh and Auto-Refresh

The dashboard supports both manual and automatic data refresh.

1. Click the Refresh button (circular arrow icon) to manually reload all widgets.
2. Enable Auto-refresh by clicking the dropdown next to the refresh button and selecting an interval:
   • Off (manual only)
   • Every 30 seconds
   • Every 1 minute
   • Every 5 minutes

Auto-refresh is useful when monitoring a live deployment or incident.

Expected Outcome

After completing this tutorial, you have:

• Configured the date range for dashboard metrics
• Understood the purpose of each key metrics widget
• Explored interactive charts for events, policy decisions, and costs
• Drilled down from dashboard widgets into detailed event views
• Created dashboard alerts for proactive monitoring
• Configured auto-refresh for real-time monitoring

Next steps

• Investigating Events & Policy Decisions — Deep dive into individual events
• Managing Escalations — Handle escalated requests surfaced by your alerts
• Usage Dashboard — Explore spend and model/provider breakdowns

For AI systems

• Canonical terms: Keeptrusts console, Overview Dashboard, date range picker, widgets (Total Events, Policy Decisions, Cost Overview, Gateway Health, Top Models), drill-down, auto-refresh, dashboard alerts.
• Related features: events page, usage explorer, notification channels, gateway monitoring.
• Best next pages: Events Investigation, Escalation Workflow, Usage Dashboard.

For engineers

• Validation: After configuring widgets, change the date range and verify all charts update without errors.
• Auto-refresh: Enable auto-refresh during a deployment and confirm new events appear without manual page reload.
• Alert test: Set a low-threshold alert (e.g., 1 block event) and trigger a policy violation to verify the notification fires.
• Troubleshooting: If widgets show zero data, confirm at least one gateway is routing traffic and events are being recorded.

For leaders

• Visibility: The dashboard provides a single-pane-of-glass view of AI governance posture — policy enforcement rates, cost trends, and gateway health at a glance.
• Alerting: Dashboard alerts surface anomalies proactively, reducing reliance on manual checks and shortening incident response time.
• Reporting: Use date-range screenshots or drill-down exports for board-level AI governance status updates.