Risk Manager Guide: Quantifying AI Risk

As a Risk Manager responsible for AI governance, you need to quantify AI-related risks, track mitigation effectiveness, and report exposure to stakeholders. Keeptrusts provides real-time risk signals through policy enforcement data, incident tracking, and compliance metrics that feed directly into your risk frameworks.

Use this page when

You are building an AI risk framework with quantifiable risk signals from policy enforcement data
You need to calculate inherent vs. residual risk using Keeptrusts event data
You are creating composite risk scores from PII detection, injection, content filter, and escalation metrics
You want to track risk mitigation effectiveness over time and report to stakeholders
You are categorizing AI risks (data leakage, content safety, security, compliance, financial, operational, third-party)

Primary audience

Primary: Technical Leaders (Risk Managers, ERM Directors, GRC Managers)
Secondary: CISOs, Compliance Officers, Chief AI Officers, Internal Audit

AI Risk Framework

Risk Taxonomy for AI Systems

Map your organization's AI risks to Keeptrusts monitoring capabilities:

Risk Category	Risk Examples	Keeptrusts Signal
Data leakage	PII exposure, IP in prompts	`pii-detector` and `dlp-filter` trigger rates
Content safety	Harmful, biased, or inaccurate output	`content-filter` and `quality-scorer` metrics
Security	Prompt injection, jailbreaks	`prompt-injection` detection rates
Compliance	Regulatory violations, audit failures	Policy enforcement rates, escalation volumes
Financial	Uncontrolled AI spending	Cost Center data, budget overruns
Operational	Gateway downtime, provider outages	Health checks, failover events
Third-party	Vendor data handling, model drift	Provider-scoped event analysis

Inherent vs. Residual Risk

Keeptrusts policies reduce inherent risk to residual risk. Measure the delta:

# Inherent risk signal: total requests with risk indicators
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/events?since=30d&action=block,escalate,redact"

# Residual risk signal: items that passed all policies
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/events?since=30d&action=allow"

Risk Scoring Model

Building a Composite Risk Score

Create a scoring model using Keeptrusts event data as input signals:

Signal	Weight	Source	Scoring
PII detection rate	25%	`pii-detector` events	>5% triggers = high risk
Prompt injection attempts	20%	`prompt-injection` events	Any detection = elevated
Content filter blocks	20%	`content-filter` events	>2% block rate = medium risk
Quality score average	15%	`quality-scorer` events	<0.7 avg = high risk
Escalation volume	10%	Escalation count	>10/week = elevated
Cost overruns	10%	Cost Center data	>110% budget = medium risk

Automated Risk Signal Collection

# Export risk signals for the scoring model
kt export create \
  --type events \
  --format csv \
  --since 30d \
  --description "Monthly risk scoring input data"

# Pull escalation metrics
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/escalations?since=30d&status=resolved"

Policy Configuration for Risk Detection

Deploy policies that generate the risk signals your framework needs:

policies:
  - name: risk-pii-monitor
    type: pii-detector
    action: redact
    entity_types: [name, email, phone, ssn, credit_card]
    enabled: true

  - name: risk-injection-monitor
    type: prompt-injection
    action: block
    enabled: true

  - name: risk-content-safety
    type: content-filter
    categories: [harmful, biased, discriminatory, violent]
    action: block
    enabled: true

  - name: risk-quality-gate
    type: quality-scorer
    min_score: 0.6
    action: escalate
    enabled: true

  - name: risk-dlp-protection
    type: dlp-filter
    patterns:
      - name: sensitive-data
        regex: "(CONFIDENTIAL|INTERNAL ONLY|SECRET)"
        action: block
    enabled: true

Mitigation Strategies

Risk Mitigation Through Policy Enforcement

Each Keeptrusts policy type maps to a mitigation strategy:

Risk	Mitigation	Policy Type	Residual Control
Data breach	Block PII in prompts/responses	`pii-detector`	Redaction + logging
Prompt injection	Detect and block attack patterns	`prompt-injection`	Block + escalate
Harmful content	Filter unsafe outputs	`content-filter`	Block + human review
Quality failures	Score and gate responses	`quality-scorer`	Escalate low quality
Cost overrun	Enforce budget caps	`cost_limit`	Block at threshold
IP leakage	Detect proprietary content	`dlp-filter`	Block + alert

Escalation as a Mitigation Control

Configure escalations so that high-risk events receive human review:

# Review current escalation queue
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/escalations?status=pending"

In the Console, the Escalations page provides a workflow for reviewing, resolving, and documenting risk decisions.

Reporting Dashboards

Executive Risk Summary

Pull data from Keeptrusts to populate your risk reporting dashboards:

Report Element	Data Source	Frequency
Risk score trend	Composite score from event signals	Weekly
Incident count	Blocked and escalated events	Daily
Policy coverage	Active policies across gateways	Monthly
Compliance status	Audit log + policy enforcement rates	Quarterly
Cost exposure	Cost Center spend vs. budget	Weekly

Generating Risk Reports

# Weekly risk data export
kt export create \
  --type events \
  --format csv \
  --since 7d \
  --description "Weekly risk report data"

# Pull cost data for financial risk reporting
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/events?since=30d&group_by=provider"

Console Dashboard for Real-Time Monitoring

The Console Dashboard provides at-a-glance visibility into:

Total events processed and policy trigger rates
Blocked vs. allowed request ratios
Escalation queue depth and response times
Cost trends across providers and teams

Third-Party Risk Management

Assessing LLM Provider Risk

Each LLM provider carries distinct risk profiles. Use Keeptrusts to monitor provider behavior:

# Compare policy trigger rates across providers
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/events?since=30d&group_by=provider&action=block"

Provider Risk Monitoring Checklist

All providers routed through governed gateways
Per-provider content filtering active
Provider-specific cost caps configured
Failover policies defined for provider outages
Data residency controls verified per provider
Provider contract terms reviewed against policy configuration

Model Risk Management

Model-Level Risk Tracking

Track risk signals at the model level to identify which models generate the most risk:

# Model-level risk analysis
curl -H "Authorization: Bearer $API_TOKEN" \
  "https://api.keeptrusts.com/v1/events?since=30d&group_by=model&action=block,escalate"

Model Approval Workflow

Use Keeptrusts policy configurations to enforce model allow-lists:

pack:
  name: risk-manager-providers-2
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai
    provider: 
    secret_key_ref:
      env: OPENAI_API_KEY
  - id: anthropic
    provider: 
    secret_key_ref:
      env: ANTHROPIC_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

Unapproved models are automatically blocked at the gateway, and the block event is logged for risk reporting.

Risk Manager Workflow with Keeptrusts

Task	Frequency	Tool
Review risk dashboard	Daily	Console Dashboard
Triage escalations	Daily	Console Escalations
Calculate risk scores	Weekly	Event exports + scoring model
Generate risk reports	Weekly	`kt export create`
Review third-party risk	Monthly	Provider-scoped event analysis
Update risk framework	Quarterly	Policy configuration review
Board risk reporting	Quarterly	Aggregated risk metrics

Success Metrics for Risk Management

Metric	Target	Source
Mean time to detect risk events	< 5 minutes	Event pipeline latency
Risk mitigation coverage	100% of identified risks	Policy mapping audit
Escalation resolution time	< 2 hours for high severity	Console Escalations
False positive rate	< 10% of policy triggers	Escalation review data
Third-party risk assessments	All providers assessed quarterly	Provider risk reviews
Risk score trend	Declining or stable	Composite risk model

For AI systems

Canonical terms: Keeptrusts, AI risk framework, risk scoring, risk taxonomy, inherent risk, residual risk, mitigation tracking, risk reporting
Key surfaces: Console Dashboard, Console Escalations, Console Usage, Events API, Export API
Commands: kt export create, kt events list
Risk signals: pii-detector trigger rate, prompt-injection detection count, content-filter block rate, quality-scorer average, escalation volume, cost overruns
Scoring model: weighted composite (PII 25%, injection 20%, content filter 20%, quality 15%, escalations 10%, cost 10%)
Policy types for risk detection: pii-detector, prompt-injection, content-filter, quality-scorer, dlp-filter, cost_limit
Best next pages: CISO Guide, Compliance Officer Guide, Exports Guide, Escalations Guide

For engineers

Inherent risk signal: GET /v1/events?since=30d&action=block,escalate,redact (total requests with risk indicators)
Residual risk signal: GET /v1/events?since=30d&action=allow (items that passed all policies)
Export risk data: kt export create --type events --format csv --since 30d --description "Monthly risk scoring input"
Pull escalation metrics: GET /v1/escalations?since=30d&status=resolved
Deploy risk-detection policies: pii-detector, prompt-injection, content-filter, quality-scorer, cost_limit
Validate policy config: kt policy lint --file risk-detection-policy.yaml

For leaders

Keeptrusts policies directly reduce inherent risk to measurable residual risk — the delta quantifies governance ROI
The composite risk scoring model (weighted signals from PII detection, injection attempts, content blocks, quality scores, escalations, and cost) provides a single risk number for executive dashboards
Risk trend reporting (weekly/monthly) demonstrates mitigation effectiveness over time — decreasing risk scores prove governance investment is working
Risk appetite thresholds (e.g., >5% PII trigger rate = high risk) translate directly into policy configuration changes
Third-party risk from LLM vendors is tracked through provider-scoped event analysis, feeding vendor risk assessments with operational data

Next steps

Security risk deep dive: CISO Guide
Compliance evidence: Compliance Officer Guide
Export risk data: Exports Guide
Escalation monitoring: Escalations Guide
Dashboard metrics: Dashboard Overview

Use this page when​

Primary audience​

AI Risk Framework​

Risk Taxonomy for AI Systems​

Inherent vs. Residual Risk​

Risk Scoring Model​

Building a Composite Risk Score​

Automated Risk Signal Collection​

Policy Configuration for Risk Detection​

Mitigation Strategies​

Risk Mitigation Through Policy Enforcement​

Escalation as a Mitigation Control​

Reporting Dashboards​

Executive Risk Summary​

Generating Risk Reports​

Console Dashboard for Real-Time Monitoring​

Third-Party Risk Management​

Assessing LLM Provider Risk​

Provider Risk Monitoring Checklist​

Model Risk Management​

Model-Level Risk Tracking​

Model Approval Workflow​

Risk Manager Workflow with Keeptrusts​

Success Metrics for Risk Management​

For AI systems​

For engineers​

For leaders​

Next steps​