Runtime Configuration

Runtime sections control stateful gateway features: history persistence, agent routing, content moderation, automatic provider selection, model listing, and response caching.

Runtime request families and privacy controls have a dedicated reference

Use Runtime request families for responses vs messages vs chat_completions, structured outputs, prompt caching, session_id, plugin governance, embeddings, transcription, speech, silent engine, and optional shadow routing.

History

The public linted shape for history capture is history.capture:.

history:
  capture:
    enabled: true
    mode: raw
    include_blocked: false

Field	Type	Required	Default	Description
`capture.enabled`	boolean	no	`false` in an explicit block	Enable history writing
`capture.mode`	string	no	`metadata_only`	`raw`, `redacted`, `metadata_only`, or `disabled`
`capture.include_blocked`	boolean	no	`false`	Include blocked requests in the history ledger
`capture.retention_days`	integer	no	—	Parsed but not read or sent by the history runtime
`capture.fail_open`	boolean	no	`true`	Changes history-delivery failure logging between warning and error; requests remain fire-and-forget in either mode

When the whole history section is absent, a connected control-plane runtime enables metadata-only history automatically. An explicit history.capture block without enabled uses the block default of false.

Metadata-only mode

Useful when you need audit evidence but should not retain full prompts or responses.

history:
  capture:
    enabled: true
    mode: metadata_only
    include_blocked: true

Disabling history

history:
  capture:
    enabled: false

Second-pass review

The top-level review: block is not part of the loaded runtime configuration and is ignored. Configure the output-phase policy.flagged-review policy for a secondary reviewer, safe rewrite, or escalation verdict. See Flagged Review Configuration.

Agents

The agents.runtime: section configures the default agent identity for the gateway.

agents:
  runtime:
    default_agent_id: agent-123

Field	Type	Required	Default	Description
`runtime.default_agent_id`	string	no	-	Default agent ID attached to gateway events and traces

Moderation

The moderation: section configures the external moderation backend behind /v1/moderations.

moderation:
  provider: openai
  secret_key_ref:
    env: KEEPTRUSTS_OPENAI_API_KEY
  categories:
    - violence
    - hate
    - self-harm
  threshold: 0.7

Field	Type	Required	Default	Description
`provider`	string	yes	-	`openai` or `azure`
`secret_key_ref`	object	no (schema)	-	Environment-backed credential reference; operationally required for OpenAI and Azure calls
`endpoint`	string	no (schema)	-	Operationally required for Azure; optional OpenAI endpoint override
`categories`	string[]	no	all	Restrict moderation to selected categories
`threshold`	number	no	`0.5`	Confidence threshold for a flagged result

Azure moderation

moderation:
  provider: azure
  secret_key_ref:
    env: KEEPTRUSTS_AZURE_CONTENT_SAFETY_KEY
  endpoint: https://my-resource.cognitiveservices.azure.com
  categories:
    - violence
    - hate
  threshold: 0.5

:::warning Fail-open external path The public config has no fail_closed field for this top-level moderation handler. Missing credentials, a missing Azure endpoint, network failures, non-success responses, and malformed upstream responses currently produce an unflagged allow-style result with HTTP 200. Treat the credential (and Azure endpoint) as required operational configuration, and do not use this endpoint as a fail-closed safety boundary. :::

Auto provider

The auto: section exposes a virtual provider that selects among configured targets by cost and latency.

auto:
  enabled: true
  name: auto
  routing:
    cost_weight: 0.6
    latency_weight: 0.4
    max_price_per_1m_tokens: 10.0

Field	Type	Required	Default	Description
`enabled`	boolean	no	`true`	Turn the virtual auto provider on or off
`name`	string	no	`auto`	Model name clients use to select automatic routing
`routing.cost_weight`	number	no	`0.5`	Weight given to price in the routing score
`routing.latency_weight`	number	no	`0.5`	Weight given to latency in the routing score
`routing.max_price_per_1m_tokens`	number	no	-	Exclude providers above this price ceiling when at least one target remains

If the price ceiling excludes every target, scoring fails open and returns the full target set. The parser also records routing.fallback_enabled, routing.unhealthy_threshold, and routing.unhealthy_window_seconds, but the auto-provider selector does not read them. Do not use those fields as fallback or health controls.

Models endpoint

The models: section controls the public /v1/models listing endpoint.

models:
  disabled: false
  include_disabled: false

Field	Type	Required	Default	Description
`disabled`	boolean	no	`false`	Return 404 for `/v1/models`
`include_disabled`	boolean	no	`false`	Include disabled nested models in the published list

Connected runtime provider credentials

Connected gateways use a Keeptrusts runtime token for control-plane access and secret_key_ref for provider credentials. Keep provider secrets out of YAML by referencing them through secret_key_ref.

pack:
  name: config-runtime-providers-14
  version: 1.0.0
  enabled: true
providers:
  targets:
    - id: openai-chatgpt-shared
      provider: openai
      model: your-openai-model
      base_url: https://api.openai.com
      secret_key_ref:
        store: OPENAI_API_KEY
policies:
  chain:
    - audit-logger
policy:
  audit-logger: {}

Use secret_key_ref.env for local shells and secret_key_ref.store when the credential is stored in Keeptrusts.

Cache

The cache: section enables exact or semantic response caching. Current cache lookup and replay run only for non-streaming Chat Completions and Responses requests classified as read-only; streaming traffic and other request families do not use this cache path. Cache backend selection and global TTL live in environment variables, not in a separate runtime service page.

Exact cache

cache:
  mode: exact
  default_on: true

Exact caching hashes the full request body. Identical requests return the cached response.

Semantic cache

cache:
  mode: semantic
  similarity_threshold: 0.85
  embedding_provider: openai-prod
  default_on: true

Semantic caching embeds the request and finds similar prior requests above the configured similarity threshold only when embedding_provider resolves and successfully embeds the request. If it is absent, unresolved, or the embedding call fails, semantic lookup and semantic storage are skipped; the ordinary exact-key cache path remains available.

Field	Type	Required	Default	Description
`mode`	string	yes	`exact`	`exact` or `semantic`
`similarity_threshold`	number	no	`0.85`	Minimum cosine similarity for a semantic cache hit
`embedding_provider`	string	no (schema), yes for semantic behavior	-	Provider target ID used to compute embeddings
`default_on`	boolean	no	`true`	Enable caching by default for matching requests

Cache backend

Select the backend with KEEPTRUSTS_LLM_CACHE_BACKEND. The current auto setting resolves to the node-local filesystem backend.

Backend	Required environment	Optional environment
`memory`	—	—
`filesystem` or `auto`	—	—
`redis` or `valkey`	`KEEPTRUSTS_LLM_CACHE_REDIS_URL`	`KEEPTRUSTS_LLM_CACHE_REDIS_KEY_PREFIX`
`s3`	`KEEPTRUSTS_LLM_CACHE_S3_BUCKET`, `KEEPTRUSTS_LLM_CACHE_S3_REGION`, `KEEPTRUSTS_LLM_CACHE_S3_ACCESS_KEY_ID`, `KEEPTRUSTS_LLM_CACHE_S3_SECRET_ACCESS_KEY`	`KEEPTRUSTS_LLM_CACHE_S3_ENDPOINT`, `KEEPTRUSTS_LLM_CACHE_S3_PREFIX`, `KEEPTRUSTS_LLM_CACHE_S3_FORCE_PATH_STYLE`
`gcs`	`KEEPTRUSTS_LLM_CACHE_GCS_BUCKET`, `KEEPTRUSTS_LLM_CACHE_GCS_ACCESS_KEY_ID`, `KEEPTRUSTS_LLM_CACHE_GCS_SECRET_ACCESS_KEY`	`KEEPTRUSTS_LLM_CACHE_GCS_REGION`, `KEEPTRUSTS_LLM_CACHE_GCS_ENDPOINT`, `KEEPTRUSTS_LLM_CACHE_GCS_PREFIX`, `KEEPTRUSTS_LLM_CACHE_GCS_FORCE_PATH_STYLE`
`qdrant`	`KEEPTRUSTS_LLM_CACHE_QDRANT_URL`	`KEEPTRUSTS_LLM_CACHE_QDRANT_COLLECTION`, `KEEPTRUSTS_LLM_CACHE_QDRANT_API_KEY`

Semantic lookup and storage are implemented for memory, filesystem, redis, valkey, and qdrant. The s3 and gcs backends are exact-cache only; do not pair them with cache.mode: semantic.

The GCS backend uses interoperability access and secret keys. Redis, Valkey, S3, GCS, and Qdrant require a CLI built with the distributed feature; choosing one without that feature or omitting a required value fails cache startup. KEEPTRUSTS_LLM_CACHE_TTL_SECS=<positive-seconds> overrides the global cache TTL for any backend.

export KEEPTRUSTS_LLM_CACHE_BACKEND=redis
export KEEPTRUSTS_LLM_CACHE_TTL_SECS=86400
export KEEPTRUSTS_LLM_CACHE_REDIS_URL=redis://localhost:6379/1

Complete runtime configuration example

Replace the provider model placeholders with IDs enabled for those accounts.

pack:
  name: full-runtime
  version: 1.0.0
  enabled: true
providers:
  targets:
    - id: openai-prod
      provider: openai
      model: your-openai-model
      secret_key_ref:
        env: KEEPTRUSTS_OPENAI_API_KEY
    - id: azure-prod
      provider: azure
      provider_type: azure-openai
      format: openai
      model: your-azure-openai-model
      base_url: https://replace-with-resource-name.openai.azure.com
      secret_key_ref:
        env: KEEPTRUSTS_AZURE_OPENAI_API_KEY
      azure_api_version: your-supported-api-version
      azure_deployment: your-azure-deployment
history:
  capture:
    enabled: true
    mode: raw
    include_blocked: true
agents:
  runtime:
    default_agent_id: support-bot-v2
moderation:
  provider: openai
  secret_key_ref:
    env: KEEPTRUSTS_OPENAI_API_KEY
  categories:
    - violence
    - hate
    - self-harm
  threshold: 0.7
auto:
  enabled: true
  routing:
    cost_weight: 0.6
    latency_weight: 0.4
    max_price_per_1m_tokens: 10.0
models:
  disabled: false
  include_disabled: false
cache:
  mode: semantic
  similarity_threshold: 0.85
  embedding_provider: openai-prod
  default_on: true
policies:
  chain:
    - prompt-injection
    - pii-detector
    - quality-scorer
    - audit-logger
policy:
  prompt-injection:
    response:
      action: block
  pii-detector:
    action: redact
  quality-scorer:
    assertions:
      - type: word-count
        config:
          min: 20
    thresholds:
      min_aggregate: 0.7
  audit-logger: {}

history.capture.retention_days is currently parser-only and does not enforce retention. audit-logger is an allow-only marker in the current gateway; it does not own retention either.

Next steps

Providers Configuration - Provider targets for moderation and auto routing
Config Testing - Validate runtime behavior with concrete suites
Declarative Config Reference - Broader schema reference
Quality Scorer - Quality gates for response evaluation

History​

Metadata-only mode​

Disabling history​

Second-pass review​

Agents​

Moderation​

Azure moderation​

Auto provider​

Models endpoint​

Connected runtime provider credentials​

Cache​

Exact cache​

Semantic cache​

Cache backend​

Complete runtime configuration example​

Next steps​