Human Oversight

The human-oversight policy escalates AI outputs for explicit human review before delivery. It enforces human-in-the-loop controls for high-stakes decision categories, ensuring that AI-generated recommendations in sensitive domains are reviewed by a qualified person before reaching the end user. This policy is essential for EU AI Act Article 14 compliance and any deployment where autonomous AI decisions carry legal, financial, or safety risk.

Use this page when

• You need human-in-the-loop review for high-stakes AI decisions (financial, medical, legal, hiring, safety-critical).
• You are implementing EU AI Act Article 14 compliance for high-risk AI systems.
• You want to escalate AI outputs below a confidence threshold for human approval before delivery.

Primary audience

• Primary: AI Agents, Technical Engineers
• Secondary: Technical Leaders

Configuration

pack:
  name: human-oversight-example-1
  version: 1.0.0
  enabled: true
policies:
  chain:
  - human-oversight
policy:
  human-oversight:
    require_human_for:
    - financial_advice
    - medical_decisions
    - legal_opinions
    - hiring_actions
    - safety_critical
    action: escalate
    confidence_threshold: 0.5
    default_assignee: review-team@example.com
    timeout_seconds: 3600

Fields

Field	Type	Description	Default
require_human_for	string[]	Decision categories requiring human review. When the AI output is classified into one of these categories, the configured action is triggered. Common values include "financial_advice", "medical_decisions", "legal_opinions", "hiring_actions", and "safety_critical".	[]
action	enum	Action to take when a matching category is detected. "escalate" queues the output for human review and holds the response until approved or timed out. "block" rejects the response immediately without queuing. Allowed values: "escalate", "block".	"escalate"
confidence_threshold	number	Confidence score between 0.0 and 1.0. When the AI model's own confidence in its output falls below this threshold, escalation triggers regardless of whether the output matches a listed category. A higher threshold means more outputs get escalated; a lower threshold is more permissive.	0.5
default_assignee	string	The user or group to assign escalated reviews to. Accepts an email address or a group identifier from your identity provider. If unset, escalations go to the default review queue.	—
timeout_seconds	integer	Maximum time in seconds to wait for a human reviewer to act on an escalated output before taking the fallback action. Minimum 1, maximum 604800 (7 days). If the timeout expires and no human has reviewed the output, the response is blocked.	3600

Use Cases

EU AI Act Article 14 — High-Risk AI Compliance

Article 14 of the EU AI Act requires that high-risk AI systems allow effective human oversight. This configuration ensures all high-risk decision categories are escalated with a strict confidence threshold and a dedicated compliance team assignee.

pack:
  name: human-oversight-example-2
  version: 1.0.0
  enabled: true
policies:
  chain:
  - human-oversight
policy:
  human-oversight:
    require_human_for:
    - medical_decisions
    - legal_opinions
    - hiring_actions
    - safety_critical
    - credit_scoring
    - law_enforcement
    action: escalate
    confidence_threshold: 0.7
    default_assignee: eu-ai-compliance@example.com
    timeout_seconds: 86400

Clinical Decision Support — Physician Review

In healthcare settings, AI-assisted diagnoses and treatment recommendations must be reviewed by a licensed physician before reaching the patient. A low confidence threshold ensures that borderline cases are always escalated.

pack:
  name: human-oversight-example-3
  version: 1.0.0
  enabled: true
policies:
  chain:
  - human-oversight
policy:
  human-oversight:
    require_human_for:
    - medical_decisions
    - drug_interactions
    - diagnostic_suggestions
    action: escalate
    confidence_threshold: 0.3
    default_assignee: attending-physician@hospital.org
    timeout_seconds: 1800

Financial Advisory — Human-in-the-Loop

Financial regulations often require that investment advice and portfolio recommendations are reviewed by a licensed advisor. Blocking mode is used here to prevent any unreviewed financial guidance from reaching clients.

pack:
  name: human-oversight-example-4
  version: 1.0.0
  enabled: true
policies:
  chain:
  - human-oversight
policy:
  human-oversight:
    require_human_for:
    - financial_advice
    - investment_recommendations
    - risk_assessments
    action: block
    confidence_threshold: 0.6
    default_assignee: compliance@finserv.com
    timeout_seconds: 7200

HR Hiring Decision Review with SLA Timeout

Hiring decisions assisted by AI must be reviewed to prevent bias and ensure fairness. This configuration uses a 4-hour timeout aligned with HR team SLAs, after which unreviewed decisions are automatically blocked to prevent stale approvals.

pack:
  name: human-oversight-example-5
  version: 1.0.0
  enabled: true
policies:
  chain:
  - human-oversight
policy:
  human-oversight:
    require_human_for:
    - hiring_actions
    - candidate_scoring
    - promotion_recommendations
    action: escalate
    confidence_threshold: 0.5
    default_assignee: hr-review-board@example.com
    timeout_seconds: 14400

How It Works

1. Category classification — When a response is generated by the AI model, Keeptrusts classifies the output into decision categories using its built-in classifier. The classification considers the semantic content, detected intent, and any metadata from the upstream provider.

2. Category matching — The classified categories are compared against the require_human_for list. If any match is found, the configured action is triggered.

3. Confidence check — Independent of category matching, if the AI model reports a confidence score below confidence_threshold, the escalation action triggers. This acts as a safety net for uncertain outputs that may not fall into a named category.

4. Action execution — If the action is "escalate", the response is held in a review queue and the default_assignee is notified. The original requester receives a pending status until the reviewer approves, modifies, or rejects the output. If the action is "block", the response is immediately rejected with a policy violation message.

5. Timeout handling — For escalated reviews, if no human acts within timeout_seconds, the response is automatically blocked. This prevents indefinite holds and ensures SLA compliance.

6. Audit trail — Every escalation, review decision, and timeout event is logged as a structured event in the Keeptrusts event stream, providing a complete audit trail for compliance reporting.

Combining With Other Policies

With pii-detector — Apply PII redaction before human oversight to ensure reviewers do not see raw personal data. The PII detector runs first, and the sanitized output is what gets escalated for review.

With dlp-filter — Use DLP blocking or redaction to catch explicit sensitive literals and hard-stop terms before they reach the review queue. This reduces reviewer fatigue by escalating only borderline cases that need human judgment.

With language-validator — Ensure escalated outputs are in the reviewer's language. If your review team operates in English, combine with a language validator set to apply_to: "output" so reviewers always receive content they can evaluate.

With rate-limiter — Prevent review queue flooding by rate-limiting upstream requests. This is especially important in high-traffic deployments where unchecked volume could overwhelm the review team.

Best Practices

• Start with "escalate" mode before moving to "block". Escalation lets you observe the pattern of flagged outputs and tune your category list and confidence threshold before enforcing hard blocks.

• Set confidence_threshold conservatively high (0.6–0.8) for regulated industries. It is better to over-escalate during the tuning period than to let low-confidence outputs through without review.

• Use specific category names that match your organization's risk taxonomy. Generic categories like "safety_critical" are a good starting point, but domain-specific categories like "drug_interactions" or "credit_scoring" give you finer control.

• Always set default_assignee to a group rather than an individual. This ensures coverage during absences and distributes the review load.

• Align timeout_seconds with your operational SLAs. A 1-hour timeout works for real-time customer interactions, but batch processing or asynchronous workflows may benefit from longer windows (24–48 hours).

• Monitor escalation volume through the Keeptrusts dashboard. A sudden spike in escalations may indicate a model regression, a misconfigured threshold, or a change in input patterns that needs investigation.

• Combine with audit export policies to maintain a durable, externally stored record of all human oversight decisions for regulatory audits.

For AI systems

• Canonical terms: Keeptrusts, human-oversight, require_human_for, action, escalate, block, confidence_threshold, default_assignee, timeout_seconds, EU AI Act Article 14
• Config/command names: human-oversight policy, require_human_for (category list), action (escalate/block), confidence_threshold, default_assignee, timeout_seconds
• Best next pages: Financial Compliance, Healthcare Compliance, Quality Scorer

For engineers

• Prerequisites: A configured escalation workflow (email, Slack, or console review queue). The default_assignee must resolve to a valid user or group in your identity provider.
• Validation: Send a request that triggers a listed category and verify the response is held for review. Test timeout behavior by waiting beyond timeout_seconds. Check the Escalations page in the console.
• Key commands: kt policy lint, kt gateway run, console Escalations page

For leaders

• Governance: Human oversight is mandatory under EU AI Act Article 14 for high-risk AI systems. It ensures qualified humans maintain meaningful control over AI decisions in sensitive domains.
• Cost: Escalated requests incur reviewer time and response latency (bounded by timeout_seconds). Balance coverage breadth against review team capacity.
• Rollout: Start with the highest-risk categories (medical, hiring, safety-critical). Lower confidence_threshold gradually as you build reviewer capacity. Monitor escalation volume to prevent queue overload.

Next steps

• Financial Compliance — Finance-specific compliance controls
• Healthcare Compliance — Medical compliance controls
• Quality Scorer — Automated quality gates before escalation
• RBAC — Role-based access control for reviewers