Skip to main content

Encryption Architecture: Protecting AI Traffic and Secrets at Rest

Encryption Architecture: Protecting AI Traffic and Secrets at Rest

Encryption is necessary in AI systems, but it is not sufficient. It protects traffic in transit and secrets at rest, yet it does not classify prompts, redact identifiers, or guarantee compliant provider routing. A sound Keeptrusts encryption architecture therefore treats encryption as one layer in a larger governance boundary, not as a substitute for policy.

Use this page when​

  • You need a practical encryption architecture for AI traffic, provider credentials, and control-plane secrets.
  • You want to separate transport protection, session boundaries, and secret storage from content governance.
  • You need implementation details that match documented Keeptrusts behavior.

Primary audience​

  • Primary: Technical Engineers
  • Secondary: Technical Leaders, AI Agents

The problem​

Security teams often hear "the data is encrypted" and assume the hard work is done. In AI systems, that phrase usually hides three different questions.

First, is traffic protected in transit between caller, gateway, control plane, and provider? Second, are stored secrets protected at rest? Third, are bearer tokens and upstream credentials exposed to the wrong execution surface, especially the browser?

If you cannot answer those separately, your encryption architecture is probably incomplete.

There is also a category error that shows up in AI deployments: using encryption language to cover a data minimization gap. A prompt that contains raw PANs, SSNs, or internal credentials is still risky even if it moves over TLS. Encryption reduces interception risk. It does not reduce the sensitivity of the payload itself. That is why PII Detector, DLP Filter, and Data Routing Policy still matter.

The solution​

Think about the architecture in four layers.

1. Encrypt transport

The gateway should terminate TLS with a defined minimum protocol version and strong cipher suites. Where service identity matters, use mTLS between internal services. This reduces interception and impersonation risk on the network path.

2. Keep authority server-side

For console flows, the browser should not hold the upstream API bearer token. Keeptrusts uses server-side BFF routes so the browser works with session state rather than raw upstream credentials. Mutating console requests use the x-keeptrusts-csrf-token header to protect request integrity.

3. Reference secrets instead of embedding them

Provider credentials belong in secret_key_ref.env or secret_key_ref.store, not inline YAML. That reduces credential sprawl and makes rotation operationally realistic.

4. Encrypt stored secrets at rest

Keeptrusts documents AES-GCM-SIV encryption at rest for stored secrets. That matters because a database compromise should not expose secret values in plaintext. Encryption at rest is especially important for managed secret records and secret-like control-plane data.

Implementation​

Start with transport protection on the gateway.

gateway:
  tls:
    enabled: true
    cert_file: /etc/kt/tls/server.crt
    key_file: /etc/kt/tls/server.key
    min_version: "1.2"
    cipher_suites:
      - TLS_AES_256_GCM_SHA384
      - TLS_AES_128_GCM_SHA256
      - TLS_CHACHA20_POLY1305_SHA256

Then keep provider credentials out of the file body and reference them instead.

pack:
  name: encrypted-ai-boundary
  version: "1.0.0"
  enabled: true

providers:
  targets:
    - id: primary-zdr
      provider: openai
      model: gpt-5.4-mini-mini
      secret_key_ref:
        store: OPENAI_API_KEY
      data_policy:
        zero_data_retention: true
        training_opt_out: true
        retention_days: 0
        in_memory_only: true
        sanitized: true
        accepts_tokenized_input: true
        allow_internet_egress: false
        local_only_processing: true

policies:
  chain:
    - pii-detector
    - dlp-filter
    - data-routing-policy

policy:
  pii-detector:
    action: redact
    pci_mode: true
    redaction:
      marker_format: label
      include_metadata: true

  dlp-filter:
    detect_patterns:
      - 'AKIA[0-9A-Z]{16}'
      - 'ghp_[0-9A-Za-z]{36}'
    blocked_terms:
      - restricted merger room
    action: block
    fuzzy_matching: true
    max_distance: 1
    sensitivity_level: high

  data-routing-policy:
    require_zero_data_retention: true
    require_no_training: true
    max_retention_days: 0
    on_no_compliant_provider: block
    log_provider_selection: true

This example is deliberate about the boundary between encryption and governance.

TLS protects the network path. secret_key_ref.store avoids inline secrets and enables managed rotation. pii-detector and dlp-filter reduce the sensitivity of the payload itself. data-routing-policy ensures the prompt, once sanitized, only reaches a provider target whose declared handling guarantees match the workload.

If you rotate a managed secret, the operational flow should be simple and non-disruptive:

kt gateway reload
kt health --verbose

The control-plane side of that flow matters as much as the command. The point is not just to swap a value. The point is to rotate a credential without scattering it across application repos or forcing downtime into the workflow.

This is also where Implement Zero-Trust AI with Defense-in-Depth Policies becomes the correct companion read. Encryption protects the pipe and the secret store. Zero trust protects the request boundary. Use Prevent Sensitive Data Leaks in AI Requests to keep raw identifiers out of the encrypted pipe in the first place.

Finally, do not ignore execution economics. If the encrypted and compliant path uses a smaller provider pool, budget for it explicitly. Spend & Wallets is relevant because Keeptrusts reserves cost before dispatch and holds requests when an eligible wallet cannot fund them. That is better than letting operators improvise a less protected route during an incident.

Results and impact​

The first result is cleaner separation of concerns. Network teams own TLS posture. Platform teams own secret references and rotation. Governance teams own redaction and routing policy. Each layer is testable without pretending it replaces the others.

The second result is lower credential exposure. When the browser never sees the upstream bearer token and providers are referenced through secret_key_ref.env or secret_key_ref.store, you reduce the number of places a credential can leak.

The third result is better breach containment. Encryption at rest for stored secrets reduces the value of database theft, while TLS reduces passive interception risk. Neither guarantees safe prompts, but both materially strengthen the baseline.

Key takeaways​

  • Encryption is necessary, but it does not replace prompt sanitation or provider governance.
  • Use TLS on the gateway, and use mTLS where internal service identity requires it.
  • Keep upstream authority server-side; do not expose bearer tokens to the browser.
  • Reference provider credentials with secret_key_ref.env or secret_key_ref.store instead of embedding them in config files.
  • Pair encrypted transport and encrypted secret storage with PII Detector, DLP Filter, and Data Routing Policy.

Next steps​