WebSocket Gateway

Keeptrusts supports proxying WebSocket connections between clients and LLM providers. This enables policy enforcement on real-time, bidirectional AI communication channels used by streaming chat UIs, voice assistants, and collaborative editing tools.

Use this page when

• You need to proxy WebSocket connections through the Keeptrusts gateway with policy enforcement.
• You are configuring the OpenAI Realtime API, voice assistants, or other bidirectional AI channels.
• You need WebSocket-specific settings: transport, ping_interval_secs, idle_timeout_secs, or rate limiting.

Primary audience

• Primary: AI Agents, Technical Engineers
• Secondary: Technical Leaders

Configuration

pack:
  name: websocket-proxy-providers-1
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai-realtime
    provider: openai
    model: gpt-4o-realtime-preview
    base_url: wss://api.openai.com/v1/realtime
    secret_key_ref:
      env: OPENAI_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

Fields

Field	Description	Default
transport	Set to websocket for WS connections	http
base_url	Upstream WebSocket URL (wss:// or ws://)	Required
ping_interval_secs	Keep-alive ping interval	30
max_message_size_bytes	Maximum single message size	1048576 (1MB)
idle_timeout_secs	Close idle connections after	300

How It Works

1. Client connects to Keeptrusts gateway via WebSocket
2. Gateway establishes upstream WebSocket to the LLM provider
3. Each message in both directions passes through the policy chain
4. Input policies apply to client → provider messages
5. Output policies apply to provider → client messages
6. Connection-level policies (rate limiting, session budget) apply across the session

Client ←→ Keeptrusts Gateway ←→ LLM Provider
  ws://       Policy Chain       wss://
              ↕
         prompt-injection
         pii-detector
         safety-filter
         audit-logger

Policy Enforcement

All standard Keeptrusts policies apply to WebSocket messages:

pack:
  name: realtime-gateway
  version: 0.1.0
  enabled: true
policies:
  chain:
  - prompt-injection
  - pii-detector
  - safety-filter
  - audit-logger
providers:
  targets:
  - id: openai-realtime
    provider: openai
    model: gpt-4o-realtime-preview
    base_url: wss://api.openai.com/v1/realtime
    secret_key_ref:
      env: OPENAI_API_KEY

Use Cases

Real-Time Chat with Safety

policies:
  chain:
    - prompt-injection
    - pii-detector
    - safety-filter
    - bot-detector
    - audit-logger

policy:
  prompt-injection:
    threshold: 0.8
    action: "block"

  safety-filter:
    categories: ["hate", "violence", "sexual"]
    action: "block"

Voice Assistant Governance

For OpenAI Realtime API or similar voice-to-text-to-LLM pipelines:

providers:
  targets:
  - id: voice-assistant
    provider: openai
    model: gpt-4o-realtime-preview
    base_url: wss://api.openai.com/v1/realtime
    secret_key_ref:
      env: OPENAI_API_KEY
policy:
  pii-detector:
    action: redact
    detect_patterns:
    - name
    - ssn
    - credit_card
    - phone
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true
pack:
  name: websocket-proxy-example-4
  version: 1.0.0
  enabled: true
policies:
  chain:
  - pii-detector
  - audit-logger

Rate Limiting WebSocket Sessions

providers:
  rate_limit:
    per_connection:
      messages_per_minute: 30
    per_ip:
      connections: 5

Connection Management

Health Monitoring

Keeptrusts monitors WebSocket health with configurable pings:

pack:
  name: websocket-proxy-providers-6
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: ws-target
    provider: openai
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

Circuit Breaker

If the upstream WebSocket fails repeatedly, the circuit breaker prevents reconnection storms:

providers:
  circuit_breaker:
    failure_threshold: 5
    recovery_timeout_secs: 30

After 5 consecutive failures, the gateway stops attempting reconnections for 30 seconds before retrying.

For AI systems

• Canonical terms: WebSocket gateway, WebSocket proxy, transport: websocket, bidirectional policy enforcement.
• Config keys: providers.targets[].transport (set to websocket), providers.targets[].base_url (wss:// or ws://), ping_interval_secs, max_message_size_bytes, idle_timeout_secs, providers.rate_limit.per_connection, providers.rate_limit.per_ip.
• Policy flow: input policies on client→provider messages, output policies on provider→client messages.
• Circuit breaker: Applies to WebSocket reconnection to prevent reconnection storms.
• Use cases: real-time chat, voice assistants (OpenAI Realtime API), collaborative editing.
• Related pages: kt gateway run, Streaming & SSE, Multi-Provider Fallback.

For engineers

• Prerequisites: A provider target with transport: websocket and a valid wss:// base URL. The gateway handles the WebSocket upgrade automatically.
• Validate: Connect a WebSocket client (e.g., websocat wss://localhost:8080/v1/realtime) and send a message. Confirm policies are applied by triggering a known-blocked input.
• Configuration: Set ping_interval_secs to match your provider's keep-alive requirements (OpenAI Realtime uses 30s). Set idle_timeout_secs based on your session length expectations.
• Rate limiting: per_connection.messages_per_minute prevents individual sessions from overwhelming the provider. per_ip.connections limits concurrent WebSocket sessions per client IP.
• Troubleshooting: If connections drop unexpectedly, increase idle_timeout_secs or verify network intermediaries (load balancers, proxies) support WebSocket upgrades. If circuit breaker opens, check provider connectivity.

For leaders

• WebSocket support extends AI governance to real-time, bidirectional use cases (voice assistants, live chat, collaborative tools) without a separate enforcement layer.
• All standard policies (PII detection, safety filtering, audit logging) apply to every WebSocket message — same compliance guarantees as HTTP-based AI traffic.
• Rate limiting per-connection and per-IP prevents abuse in consumer-facing real-time applications.
• Session-level audit logging captures the full interaction for compliance, even for long-lived connections.

Next steps

• Streaming & SSE — Unidirectional real-time streaming
• kt gateway run — Gateway startup and provider config
• Multi-Provider Fallback — Failover for WebSocket providers
• CLI overview