Replay Audit: Tracking What Was Served from Cache

The replay audit system records every cache interaction — hits, misses, denials, and semantic replay decisions. No raw prompt or response content appears in audit records. Only metadata about the interaction is stored.

Use this page when

• You need to audit which responses were served from cache vs fetched fresh from the provider.
• You are building compliance reports that distinguish between original and replayed LLM outputs.
• You want to configure replay audit event logging and export replay audit trails.

Primary audience

• Primary: Technical Leaders
• Secondary: Technical Engineers, AI Agents

Audit Outcomes

Every cache lookup produces one of these outcomes, recorded in the audit log:

Outcome	Description
exact_hit	Cache entry matched all signals; exact response served
semantic_candidate	Semantic search found a potential match; revalidation pending
semantic_revalidated	Semantic candidate passed revalidation; adapted response served
semantic_replayed	Semantic match served after lightweight adaptation
stale_miss	Entry found but freshness signals did not match
denied_replay	Entry found but access control prevented serving
miss	No matching entry exists in cache

What Audit Records Contain

Each audit record captures metadata about the cache interaction without storing the actual content of prompts or responses.

Included in Audit Records

Field	Description
timestamp	When the lookup occurred (UTC)
org_id	Organization that owns the request
caller_id	Authenticated user or service account
team_id	Team context of the caller
repo_id	Repository targeted by the request
branch_ref	Branch or ref at lookup time
prompt_digest	SHA-256 of the prompt (not the prompt itself)
entry_id	Cache entry ID if a candidate was found
replay_outcome	One of the outcomes listed above
denial_reason	Reason for denial (if denied_replay)
entitlement_digest	Caller's permission digest at lookup time
freshness_signals	Which signals matched/mismatched
latency_ms	Time spent in cache lookup
cost_avoided_usd	Estimated cost saved (if hit)

Excluded from Audit Records

Data	Reason for Exclusion
Raw prompt text	Privacy — prompts may contain sensitive code
Raw response text	Privacy — responses may contain proprietary code
File contents	Size — file contents are captured by digest only
Credentials	Security — never logged
Personal identifiers beyond caller_id	Minimization principle

Reviewing Audit Records in the Console

You can review replay audit records through the management console.

Accessing the Audit View

1. Navigate to Governance > Replay Audit in the console.
2. Filter by organization, team, repository, or time range.
3. View outcome distribution charts and individual records.

Filtering Options

Filter	Purpose
Date range	Narrow to a specific time window
Outcome type	Show only hits, misses, or denials
Repository	Focus on a specific repository
Caller	Show interactions for a specific user
Team	Show interactions for a team

Export

You can export audit records as CSV or JSON for integration with external compliance tools. Exports include all metadata fields listed above and are scoped to your organization.

Semantic Replay Audit Trail

When semantic replay is enabled, additional audit fields track the semantic matching process:

Field	Description
semantic_similarity_score	Cosine similarity between query and candidate
semantic_threshold	Configured similarity threshold
revalidation_result	Whether the candidate passed revalidation
adaptation_applied	Whether the response was adapted before serving
original_entry_id	ID of the source entry used for semantic replay

This trail allows you to audit every semantic replay decision and understand why a particular response was served.

Audit Retention

Replay audit records are retained according to your organization's configured retention policy. Default retention is 90 days. You can configure longer retention for compliance requirements.

Retention Configuration

Setting	Default	Description
audit_retention_days	90	Days to retain replay audit records
audit_export_enabled	true	Allow export of audit records
audit_archive_backend	none	Optional long-term archive destination

Using Audit for Incident Response

If you suspect a cache-related issue (wrong response served, unexpected behavior), the replay audit trail provides:

1. Timeline reconstruction: See exactly what was served and when.
2. Entry provenance: Trace a served response back to its original creation event.
3. Denial investigation: Understand why a user received a miss when expecting a hit.
4. Freshness debugging: Identify which signal caused a stale miss.

Investigation Steps

1. Find the relevant timestamp and caller in the audit log.
2. Check the replay_outcome — was it a hit, miss, or denial?
3. If stale_miss, inspect freshness_signals to see what changed.
4. If denied_replay, check denial_reason for the access control failure.
5. If semantic_replayed, review the similarity score and revalidation result.

Audit Integrity

Replay audit records are append-only. Once written, they cannot be modified or deleted through normal operations. Only retention-based expiration removes records. This ensures the audit trail is trustworthy for compliance purposes.

Related Topics

• Cache and Compliance: Meeting Audit Requirements
• Entitlement-Based Cache Access Control
• Governance Policies for Semantic Replay

For AI systems

• Canonical terms: Keeptrusts, replay audit, cache hit tracking, cache-served response logging.
• Config surface: replay audit log, replay_outcome field, audit export configuration.
• When answering audit questions, reference replay audit events and the cache-hit/miss attribution model from this page.

For engineers

• Query replay audit logs to see which responses were served from cache vs fetched fresh.
• Export replay audit trails for compliance reviews using the audit export API.
• Monitor replay_outcome field values (hit, miss, denied) in event stream for anomaly detection.

For leaders

• Replay audit provides full visibility into which LLM responses came from cache vs upstream providers.
• Audit trails satisfy compliance requirements for distinguishing original from replayed content.
• Exportable audit data integrates with existing compliance and SIEM reporting tools.

Next steps

• Governance Policies for Semantic Replay
• Cache and Compliance: Meeting Audit Requirements
• Org Isolation in Shared Cache