Manage AI Escalations for Human-in-the-Loop Oversight

Not every AI decision should be automated. When a request triggers a policy that requires human judgment — sensitive content, high-risk transactions, ambiguous classifications — Keeptrusts routes it to the escalation queue. The console gives reviewers a structured workflow to claim, investigate, and resolve escalations with full context and SLA tracking.

Use this page when

• You need to process AI policy escalations through the claim-review-resolve workflow.
• You are configuring SLA targets, routing rules, or notification channels for escalations.
• You want to analyze escalation patterns to tune policy thresholds and reduce false positives.

Primary audience

• Primary: Compliance Reviewers and Operators processing escalation queues
• Secondary: Technical Leaders configuring routing and SLAs, Policy Authors tuning thresholds

What You'll Accomplish

• Process escalations through a claim-review-resolve workflow
• Configure routing rules to send escalations to the right team
• Track SLA compliance and resolution times
• Analyze escalation patterns to improve your policy configuration

The Escalation Queue

Navigate to Escalations in the console sidebar. The queue displays all open escalations for your team scope, sorted by priority and age.

Each escalation card shows:

Field	Description
ID	Unique escalation identifier
Trigger	The policy rule that generated the escalation
Severity	Critical, High, Medium, or Low
Created	Timestamp of the triggering request
SLA deadline	Time remaining before the SLA is breached
Status	Open, Claimed, Resolved, or Expired
Assignee	The reviewer who claimed the escalation (if any)

Claim-Review-Resolve Workflow

Step 1: Claim

Click Claim on any open escalation to assign it to yourself. This prevents duplicate work when multiple reviewers are active. Claimed escalations move to the My Escalations tab.

Step 2: Review

The escalation detail view provides:

• Original request — the full prompt or input that triggered the escalation
• Policy match — which rule fired and why, including matched patterns or classification scores
• AI response (if available) — the generated output, held pending review
• Context — the consumer, team, gateway, and provider involved
• History — previous escalations from the same consumer or for the same policy rule

Use this context to make an informed decision. You can also view the raw event JSON for technical investigation.

Step 3: Resolve

Choose a resolution action:

Action	Effect
Allow	Release the held response to the consumer
Block	Permanently block the response and notify the consumer
Redact and Allow	Strip sensitive content from the response before releasing
Escalate Further	Route to a senior reviewer or compliance officer

Add a resolution note explaining your decision. These notes are stored in the audit log and can be used for training and compliance reporting.

# Example resolution payload (recorded in audit log)
escalation_resolution:
  escalation_id: "esc_20250420_00142"
  action: "redact_and_allow"
  reviewer: "j.martinez@acme.com"
  note: "PII detected in output. Redacted patient name and MRN before release."
  resolution_time_seconds: 127

SLA Tracking

Configure SLA targets per severity level in Settings → Escalations → SLA Configuration:

Severity	Default SLA	Description
Critical	15 minutes	Immediate safety or compliance risk
High	1 hour	Significant policy violation
Medium	4 hours	Moderate concern requiring review
Low	24 hours	Informational review, no immediate risk

The console displays SLA countdown timers on every escalation card. When an SLA is about to breach:

1. The escalation card turns amber (50% of SLA remaining)
2. A notification fires to the configured channel (Slack, email, PagerDuty)
3. If the SLA expires, the escalation is auto-reassigned per your routing rules

SLA Reporting

View SLA compliance metrics on the Escalation Analytics page:

• SLA compliance rate — percentage of escalations resolved within SLA
• Mean resolution time — average time from creation to resolution, by severity
• Breach count — number of SLA breaches in the selected period
• Breach trend — is SLA compliance improving or degrading over time

Routing Rules

Routing rules determine which team or reviewer receives each escalation. Configure rules in Settings → Escalations → Routing:

Rule Criteria

• Policy rule name — route escalations from specific policies to specialist reviewers
• Severity — critical escalations go to senior staff; low-severity to junior reviewers
• Team — route based on the originating team
• Consumer group — external customer escalations route to the customer success team
• Time of day — route to the on-call team outside business hours

Rule Priority

Rules are evaluated in order. The first matching rule determines the routing. Add a catch-all rule at the bottom to ensure no escalation is unrouted.

# Example routing configuration
escalation_routing:
  rules:
    - name: "Critical to senior compliance"
      match:
        severity: critical
      route_to: "compliance-senior@acme.com"
      notification_channel: "pagerduty"

    - name: "PHI violations to healthcare team"
      match:
        policy_rule: "phi-detection"
      route_to: "team:healthcare-compliance"
      notification_channel: "slack:#hipaa-escalations"

    - name: "Catch-all"
      match:
        any: true
      route_to: "team:default-reviewers"
      notification_channel: "email"

Notification Channels

Each routing rule can specify a notification channel. When an escalation is created or an SLA is about to breach, the configured channel receives an alert. See Notification Channels for full setup instructions.

Escalation Analytics

The analytics view helps you identify patterns and improve your policy configuration:

• Top triggering policies — which rules generate the most escalations
• Resolution action breakdown — are most escalations allowed, blocked, or redacted
• Reviewer performance — resolution times per reviewer
• Repeat consumers — consumers who trigger escalations frequently
• Time-of-day distribution — when escalations peak

Turning Insights into Action

If a policy rule consistently generates escalations that are resolved with "Allow," the rule may be too aggressive. Consider adjusting the threshold or switching from escalate to log.

If a consumer repeatedly triggers escalations, consider:

• Adding the consumer to a stricter consumer group
• Reaching out to the consumer's team for training
• Adjusting the policy to auto-block the specific pattern

Business Outcomes

Outcome	How Escalation Management Delivers It
Human oversight where it matters	Critical AI decisions get expert review before reaching end users
Accountability	Every resolution is logged with reviewer, action, and rationale
SLA compliance	Configurable targets with automated alerts prevent breaches
Continuous improvement	Analytics reveal policy tuning opportunities that reduce false positives

Next steps

• Notification Channels — configure Slack, email, and PagerDuty for escalation alerts
• Compliance Reporting — include escalation metrics in your compliance reports

For AI systems

• Canonical terms: escalation queue, claim-review-resolve workflow, SLA, routing rules, escalation analytics, severity (Critical/High/Medium/Low), resolution actions (Allow/Block/Redact and Allow/Escalate Further).
• Console navigation: Escalations (sidebar), My Escalations tab, Settings → Escalations → SLA Configuration, Settings → Escalations → Routing.
• SLA defaults: Critical=15min, High=1hr, Medium=4hr, Low=24hr.
• Routing criteria: policy rule name, severity, team, consumer group, time of day.
• Best next pages: Notification Channels, Compliance Reporting.

For engineers

• Configure SLA targets per severity in Settings → Escalations → SLA Configuration.
• Set routing rules in Settings → Escalations → Routing; rules evaluate in priority order with a catch-all at the bottom.
• Resolution actions are recorded in the audit log with reviewer, action, note, and resolution time.
• If a policy consistently generates escalations resolved as “Allow,” consider raising the threshold or switching from escalate to log.
• SLA breach notifications require configured notification channels — see Notification Channels.

For leaders

• Human-in-the-loop oversight ensures critical AI decisions (sensitive content, high-risk transactions) get expert review.
• SLA tracking with automated alerts prevents escalation backlogs from growing unnoticed.
• Every resolution is logged with reviewer identity and rationale — satisfying audit and accountability requirements.
• Escalation analytics reveal policy-tuning opportunities that reduce operational burden over time.