Multi-Tenant Gateway Fleet Management

Large organizations run dozens of AI gateways — one per team, environment, or regulatory boundary. This guide covers deploying isolated gateway instances, propagating configuration centrally, monitoring fleet health, and managing gateways through the console.

Use this page when

You need to deploy isolated per-team gateways or a central multi-tenant gateway
You are choosing between per-team, central, or hybrid gateway topologies
You want to monitor fleet health, propagate config centrally, and manage gateways through the console

Primary audience

Primary: Technical Engineers
Secondary: AI Agents, Technical Leaders

Gateway Deployment Models

Per-Team Gateways

Each team operates its own kt gateway instance with a dedicated policy config. This provides hard isolation between tenant workloads:

# team-alpha/policy-config.yaml
pack:
  name: team-alpha-prod
  version: 0.1.0
  enabled: true
  description: "Team Alpha production gateway"

policies:
  chain:
    - pii-detector
    - audit-logger

policy:
  pii-detector:
    action: redact
  audit-logger:
    retention_days: 365

providers:
  targets:
    - id: openai-primary
      provider: openai
      model: gpt-4o-mini
      base_url: https://api.openai.com
      secret_key_ref:
        env: TEAM_ALPHA_OPENAI_KEY

Start with the CLI:

export KEEPTRUSTS_API_URL="https://keeptrusts-api.internal:8080"
export KEEPTRUSTS_GATEWAY_TOKEN="$TEAM_ALPHA_GATEWAY_KEY"

kt gateway run \
  --listen 0.0.0.0:41002 \
  --policy-config team-alpha/policy-config.yaml

Hosted Gateway Mode

A single gateway serves multiple tenants, routing requests based on the gateway key presented. The control-plane API resolves the correct policy chain per key:

export KEEPTRUSTS_API_URL="https://keeptrusts-api.internal:8080"
export KEEPTRUSTS_GATEWAY_TOKEN="$SHARED_GATEWAY_KEY"

kt gateway run \
  --listen 0.0.0.0:41002

In hosted gateway mode, each consumer application uses its own gateway key (kt_gk_...). The gateway fetches the associated policy config and provider credentials from the API at request time.

Fleet Topology with Docker Compose

Deploy isolated gateways as separate services:

# docker-compose.fleet.yaml
services:
  gateway-alpha:
    image: keeptrusts/gateway:latest
    command: ["gateway", "run", "--listen", "0.0.0.0:41002", "--policy-config", "/etc/keeptrusts/policy-config.yaml"]
    environment:
      KEEPTRUSTS_API_URL: http://keeptrusts-api:8080
      KEEPTRUSTS_GATEWAY_TOKEN: ${TEAM_ALPHA_GW_KEY}
      TEAM_ALPHA_OPENAI_KEY: ${TEAM_ALPHA_OPENAI_KEY}
    volumes:
      - ./team-alpha/policy-config.yaml:/etc/keeptrusts/policy-config.yaml:ro
    ports:
      - "41002:41002"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:41002/health"]
      interval: 15s
      timeout: 5s
      retries: 3

  gateway-beta:
    image: keeptrusts/gateway:latest
    command: ["gateway", "run", "--listen", "0.0.0.0:41002", "--policy-config", "/etc/keeptrusts/policy-config.yaml"]
    environment:
      KEEPTRUSTS_API_URL: http://keeptrusts-api:8080
      KEEPTRUSTS_GATEWAY_TOKEN: ${TEAM_BETA_GW_KEY}
      TEAM_BETA_OPENAI_KEY: ${TEAM_BETA_OPENAI_KEY}
    volumes:
      - ./team-beta/policy-config.yaml:/etc/keeptrusts/policy-config.yaml:ro
    ports:
      - "41003:41002"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:41002/health"]
      interval: 15s
      timeout: 5s
      retries: 3

Configuration Propagation

Git-Linked Config Sync

The API polls linked Git repositories for config changes and pushes reloads to deployed gateways. Link a repo through the API:

curl -X POST https://keeptrusts-api.internal:8080/v1/git-repos \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "repo_url": "https://github.com/org/ai-policies.git",
    "branch": "main",
    "path": "team-alpha/policy-config.yaml",
    "auto_create_configuration": true,
    "poll_interval_seconds": 60
  }'

When a commit lands on main, the API detects the diff, creates a new configuration version, and signals the gateway to reload.

Environment-Scoped Configs

Prefer one explicit config per environment, and use config variables only for secrets that should resolve at runtime:

# team-alpha/prod/policy-config.yaml
pack:
  name: team-alpha-prod
  version: 0.1.0
  enabled: true

policies:
  chain:
    - spend-limit
    - audit-logger

policy:
  spend-limit:
    max_daily_usd: 500
  audit-logger:
    retention_days: 365

providers:
  targets:
    - id: openai-primary
      provider: openai
      model: gpt-4o-mini
      base_url: https://api.openai.com
      secret_key_ref:
        store: OPENAI_API_KEY

Set the environment-specific secret value through the API:

# Production
curl -X PUT https://keeptrusts-api.internal:8080/v1/config-variables/OPENAI_API_KEY \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"value": "sk-prod-..."}'

# Staging
curl -X PUT https://keeptrusts-api.internal:8080/v1/config-variables/OPENAI_API_KEY \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"value": "sk-staging-..."}'

Fleet Health Monitoring

Health Probes

Every gateway exposes a /health endpoint. Aggregate fleet health with a simple probe script:

#!/usr/bin/env bash
set -euo pipefail

GATEWAYS=("gateway-alpha:41002" "gateway-beta:41003" "gateway-gamma:41004")

for gw in "${GATEWAYS[@]}"; do
  status=$(curl -sf "http://${gw}/health" | jq -r '.status' 2>/dev/null || echo "unreachable")
  echo "${gw}: ${status}"
done

Event Stream Monitoring

Gateway decision events flow to the API via POST /v1/events. Monitor event volume per gateway to detect anomalies:

# Recent events for a specific gateway
curl -s "https://keeptrusts-api.internal:8080/v1/events?gateway_name=team-alpha-prod&limit=10" \
  -H "Authorization: Bearer $API_TOKEN" | jq '.events[] | {timestamp, decision, model}'

Console Gateway Management

The management console provides a gateway list view under Settings → Gateways. Platform engineers can:

View all registered gateways with last-seen timestamps
Inspect the active policy configuration version per gateway
Trigger manual config reloads
Rotate gateway keys without downtime

Use the console to audit which teams are running which policy versions and identify gateways that have fallen behind on config updates.

Scaling Considerations

Topology	Best For	Trade-off
Per-team gateway	Hard tenant isolation, independent scaling	More infrastructure to manage
Hosted gateway	Simplified ops, shared infrastructure	Noisy-neighbor risk, single point of failure
Hybrid (shared + dedicated)	Critical teams get isolation, others share	Config complexity

For hosted gateway mode at scale, deploy multiple gateway replicas behind a load balancer and use the API's config-variable system to ensure consistent policy resolution across replicas.

For AI systems

Canonical terms: per-team gateway, hosted gateway, fleet management, gateway key (kt_gk_...), config propagation, fleet health
CLI commands: kt gateway run --listen <host:port> --policy-config <path>, kt gateway run --listen <host:port>
Hosted gateway: one shared gateway deployment serves multiple tenants, API resolves policy chain per gateway key at request time
Console path: Gateways page shows registered instances, active policy versions, last-seen timestamps
Related pages: Kubernetes Deployment, Config-as-Code, Capacity Management

For engineers

Per-team: each gateway has its own policy-config.yaml and gateway key; deploy as separate Docker Compose services or K8s Deployments
Hosted: set KEEPTRUSTS_API_URL and KEEPTRUSTS_GATEWAY_TOKEN before kt gateway run; consumers present their own kt_gk_... key and the API resolves the correct policy config per key
Deploy per-team gateways in Docker Compose by duplicating the gateway service with different config mounts and port mappings
Use health checks on /health (port 41002) for each gateway instance to detect fleet members that are down
Console Gateways page shows fleet status, active config versions, and supports manual config reload triggers
Validate: confirm each team's gateway key routes through the correct policy chain by sending a test request and checking the event

For leaders

Per-team gateways provide hard isolation (no noisy-neighbor risk) but cost more infrastructure
Hosted gateway simplifies ops but introduces shared-infrastructure risk and a single point of failure
Hybrid approach: critical teams get dedicated gateways, others share the hosted gateway
Fleet visibility through the console enables governance teams to audit which teams run which policy versions
Gateway key rotation is possible without downtime — issue a new key, update the consumer, revoke the old one

Next steps

Deploy on Kubernetes with Kubernetes Deployment Helm charts and HPA
Store and promote configs with Config-as-Code workflows
Set team budgets with Capacity Management wallet allocations

Use this page when​

Primary audience​

Gateway Deployment Models​

Per-Team Gateways​

Hosted Gateway Mode​

Fleet Topology with Docker Compose​

Configuration Propagation​

Git-Linked Config Sync​

Environment-Scoped Configs​

Fleet Health Monitoring​

Health Probes​

Event Stream Monitoring​

Console Gateway Management​

Scaling Considerations​

For AI systems​

For engineers​

For leaders​

Next steps​