AI Usage SIEM Streaming

Keeptrusts can stream a full-fidelity record of every governed AI interaction — including the request, the response, governance decisions, data-leakage findings, and financial metadata — to your organization's SIEM or log-management platform. Security and compliance teams use these records to investigate exposure, exfiltration, and policy violations with complete, correlated evidence.

Supported destinations

Destination kind	Target system	Auth method
splunk	Splunk HTTP Event Collector (HEC)	HEC token
elastic	Elasticsearch bulk API	API key or Basic auth
datadog	Datadog logs-intake endpoint	DD-API-KEY header
generic_https_json	Any HTTPS JSON collector	Configurable (Bearer, API key, or custom header)

All destinations require HTTPS endpoints. Non-HTTPS transports are out of scope. Every destination endpoint is validated against Keeptrusts' SSRF-protection policy at configuration time and on each delivery attempt.

info

These destination kinds are distinct from the removed legacy Trail subscription values (splunk_hec, datadog_logs, s3, syslog). The SIEM streaming feature is a fresh implementation with owned delivery, not a revival of the prior inert surface.

Configuring a destination

Prerequisites

• You must have the AI Usage Stream Write permission (AI_USAGE_STREAM_WRITE) on the organization.
• Your destination must accept HTTPS ingestion.

Console setup

1. Navigate to Settings → Integrations → AI Usage SIEM Streaming.
2. Click Add Destination.
3. Select the destination kind (splunk, elastic, datadog, or generic_https_json).
4. Enter the endpoint URL. The URL is validated against the SSRF policy immediately — private, loopback, and link-local addresses are rejected.
5. Enter the destination credential (HEC token, API key, or custom auth header). The credential is encrypted at rest using the platform's KMS envelope and is never stored in plaintext.
6. Choose the redaction mode (see Redaction modes).
7. Optionally configure event filters to stream only a subset of records.
8. Save the destination.

Per-destination setup

Splunk HEC

1. In Splunk, create or use an existing HTTP Event Collector token.
2. Note the HEC endpoint URL, typically https://your-splunk:8088/services/collector/event.
3. In Keeptrusts, create a destination with kind splunk, the HEC endpoint URL, and the HEC token as the credential.
4. Keeptrusts sends batched JSON events with Authorization: Splunk <token>.

Elasticsearch

1. In Elasticsearch, create an API key with write access to the target index.
2. Note the bulk API endpoint, typically https://your-elastic:9200/_bulk.
3. In Keeptrusts, create a destination with kind elastic, the bulk endpoint URL, and the API key (or Basic auth credentials) as the credential.
4. Keeptrusts sends NDJSON batches with Authorization: ApiKey <key> or Authorization: Basic <credentials>.

Datadog

1. In Datadog, create or use an existing API key.
2. The logs-intake endpoint is typically https://http-intake.logs.datadoghq.com/api/v2/logs (US) or the equivalent for your Datadog site.
3. In Keeptrusts, create a destination with kind datadog, the logs-intake URL, and the API key as the credential.
4. Keeptrusts sends JSON batches with the DD-API-KEY header.

Generic HTTPS JSON collector

1. Note your collector's HTTPS ingestion endpoint.
2. Determine the auth method: Bearer token, API key header, or a custom header name and value.
3. In Keeptrusts, create a destination with kind generic_https_json, the endpoint URL, and the auth credential.
4. Keeptrusts sends JSON array batches with the configured auth header.

Redaction modes

Each destination has an independent redaction mode that controls how request/response content is delivered.

Mode	Behavior	Default
Redacted	Sensitive spans identified by data-leakage detectors are replaced with redaction markers. The delivered record includes redacted content, structured detector findings, and SHA-256 hashes of both raw and redacted content.	✅ Yes
Raw	Full-fidelity request and response bodies are transmitted unredacted. All detector findings and hashes are still included.	No

Choosing a redaction mode

• Redacted (default) is recommended for most deployments. It provides complete data-leakage visibility — every detector finding, category, entity type, confidence, and masked location evidence — without transmitting the actual sensitive content to the SIEM. Investigators can see exactly what was detected, which policies matched, and what actions were taken.

• Raw is available for organizations that need full-fidelity content in their SIEM for forensic analysis. Enabling raw mode requires an explicit data-handling acknowledgement per destination. Raw mode is not available in air-gap or zero-egress deployment profiles.

caution

Enabling raw mode transmits unredacted request and response bodies to the external destination. Ensure your SIEM's access controls, retention policies, and data-handling agreements are appropriate for the content your organization processes through Keeptrusts.

AI Usage Record schema

Every governed AI interaction produces exactly one AI Usage Record. The record contains the following groups:

Envelope

Field	Description
schema_version	Record schema version
record_id	Unique record identifier (UUID)
org_id	Organization identifier
sequence	Per-organization monotonic sequence number
captured_at	Server-owned capture timestamp
source	Capture source (gateway or api)

Identity and tenancy

Field	Description
gateway_id	Gateway that processed the interaction
agent_id	Agent identifier
subject_token_id	Token used for the request
actor_id / actor_type	Actor identity and type
session_id	Session identifier
correlation_id	Cross-system correlation ID
request_id	Unique request identifier

Interaction

Field	Description
request_family	Request family (Chat, Responses, Messages, WebSocket, MCP)
transport	Transport type (buffered, sse, websocket, mcp)
provider / model	AI provider and model used
upstream_host	Upstream host (host only, no credentials)
streamed	Whether the response was streamed
started_at / completed_at	Start and completion timestamps
latency_ms	Total latency in milliseconds
outcome	Governance outcome (allow, block, flag, error)
http_status	Upstream HTTP status code

Content

Field	Description
request_body / response_body	Bounded request and reassembled response content
request_bytes / response_bytes	Content byte counts
request_sha256_raw / response_sha256_raw	SHA-256 of raw content
request_sha256_redacted / response_sha256_redacted	SHA-256 of redacted content
request_truncated / response_truncated	Whether content was truncated

Governance

Field	Description
matched_policies	Policies that matched the interaction
decisions	Governance decisions applied
applied_mutations	Mutations applied to the request/response
applied_redactions	Redactions applied
output_stage_evaluations	Output-stage evaluation results
human_oversight / escalation / flagged_review	Oversight references

Data leakage

Field	Description
detectors[]	Array of detector findings (see below)
redaction_mode	Active redaction mode for this record
secrets_detected	Whether credential/secret patterns were detected
attachments[]	Attachment metadata (type, size, sha256, scanned)
potential_data_leakage	Boolean roll-up: was any data leakage detected?

Each entry in detectors[] includes:

Field	Description
detector	Detector name
category	Detection category
entity_type	Entity type detected
match_count	Number of matches
confidence	Detection confidence
matched_policy_ids	Policies that triggered
action	Action taken (redacted, blocked, allowed)
evidence_locations	Masked location evidence (offsets and hashes only)
sample_hash	Non-reversible hash of the matched content

Detector families include PII, HIPAA/PHI, DLP custom rules, student privacy, and the platform's credential/secret-pattern and high-entropy scanners. An empty detectors[] with scanned: true records a confirmed clean scan.

Financial

Field	Description
prompt_tokens / completion_tokens / cached_input_tokens / total_tokens	Token counts
cost / currency	Interaction cost
wallet_transaction_id / budget_id / reservation_request_id / spend_log_id	Financial references

Integrity

Field	Description
canonical_sha256	RFC-8785 canonical hash of the record
trail_event_id	Correlation to the governance trail
dispatch_attempt_id	Delivery attempt identifier

Event filters

Each destination can be configured with event filters to stream only a subset of records. Filters are evaluated server-side before delivery. Available filter dimensions include:

• Request family — stream only specific families (e.g., Chat only)
• Governance outcome — stream only blocked or flagged interactions
• Data leakage detected — stream only records where potential_data_leakage is true

Filters are combined with AND logic. If no filters are configured, all records are streamed to the destination.

Delivery guarantees

• At-least-once delivery: Every record is delivered at least once to each active destination. Duplicate deliveries carry the same deterministic idempotency key, allowing your SIEM to deduplicate.
• Per-destination ordering: Records are delivered in server-owned sequence order per destination.
• Durable delivery: Records are enqueued in the same database transaction as persistence. A record is never reported as captured without a durable delivery commitment.
• Poison-record isolation: A record that repeatedly fails delivery is quarantined rather than blocking subsequent records.

Monitoring delivery health

The delivery health view in the console shows per-destination status:

• Delivery lag — how far behind the delivery worker is from the latest captured record
• Last delivery outcome — the result of the most recent delivery attempt
• Retry/quarantine counts — how many records are pending retry or have been quarantined
• Attempt history — recent delivery attempt results

Use the Admin console for platform-wide visibility across all tenant destinations.

Gateway policy configuration

The gateway ai_usage_streaming policy stanza controls capture behavior on the data plane. It does not control redaction or destination routing, which are managed in the console.

policy-config.yaml

ai_usage_streaming:
  enabled: true
  body_capture_max_bytes: 1048576  # 1 MiB per request/response
  mandatory: false                 # true = fail-closed if capture enqueue fails

Field	Description	Default
enabled	Enable AI usage record capture	false
body_capture_max_bytes	Maximum bytes captured per request/response body	1048576 (1 MiB)
mandatory	Fail-closed: deny the AI request if durable capture enqueue fails	false

warning

Setting mandatory: true causes the gateway to return a 503 error and prevent the AI request from proceeding if the durable capture enqueue fails. Use this only when audit completeness is a hard compliance requirement.

Air-gap and self-hosted deployments

In air-gap or self-hosted zero-egress deployments:

• SIEM streaming is disabled by default (KEEPTRUSTS_SIEM_WORKER_ENABLED=false).
• When enabled, only in-environment destinations are allowed. The SSRF policy rejects endpoints outside the deployment's internal network.
• Raw redaction mode is refused in zero-egress profiles.
• If no active destination matches, records remain persisted locally with no external egress.
• The zero-external-egress assertion continues to pass regardless of whether the SIEM worker is present in the compose topology.