Deploy the Gateway on Kubernetes

Running the Keeptrusts gateway on Kubernetes gives you horizontal scaling, self-healing, and seamless integration with your existing cluster infrastructure. This guide covers Helm chart patterns, pod configuration, autoscaling, and service mesh integration.

Use this page when

• You are deploying the Keeptrusts gateway on an existing Kubernetes cluster.
• You need Helm chart values, Deployment manifests, ConfigMaps, and Secrets for the gateway.
• You want horizontal pod autoscaling (HPA) based on CPU/memory for gateway pods.
• You need health probes, rolling update strategy, or service mesh sidecar patterns.

Primary audience

• Primary: Technical Engineers
• Secondary: AI Agents, Technical Leaders

Architecture overview

Ingress / Load Balancer
  → Kubernetes Service (ClusterIP or LoadBalancer)
    → Keeptrusts Gateway Pods (Deployment, replicas: 2+)
      → kt gateway run (port 41002)
      → Sidecar: OTel collector (optional, port 4317)
    → ConfigMap: policy-config.yaml
    → Secret: API key, provider keys
  → Keeptrusts API (external or in-cluster)

Prerequisites

• Kubernetes cluster (1.26+)
• kubectl and helm configured
• Keeptrusts API key
• Container image: keeptrusts/gateway:<version>

Helm chart deployment

values.yaml

replicaCount: 2

image:
  repository: keeptrusts/gateway
  tag: "latest"
  pullPolicy: IfNotPresent

service:
  type: ClusterIP
  port: 41002

config:
  apiUrl: "https://api.keeptrusts.com"
  listen: "0.0.0.0:41002"

secrets:
  apiKeySecretName: keeptrusts-api-key
  apiKeySecretKey: api-key

resources:
  requests:
    cpu: 250m
    memory: 256Mi
  limits:
    cpu: "1"
    memory: 512Mi

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilization: 70
  targetMemoryUtilization: 80

policyConfig: |
  version: "1"
  policies:
    - name: content-safety
      action: block
      rules:
        - type: content-filter
          categories: [hate, violence, self-harm]

Deployment manifest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keeptrusts-gateway
  labels:
    app: keeptrusts-gateway
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: keeptrusts-gateway
  template:
    metadata:
      labels:
        app: keeptrusts-gateway
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
    spec:
      containers:
        - name: gateway
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          ports:
            - containerPort: 41002
              name: gateway
              protocol: TCP
          env:
            - name: KEEPTRUSTS_API_URL
              value: {{ .Values.config.apiUrl }}
            - name: KEEPTRUSTS_GATEWAY_TOKEN
              valueFrom:
                secretKeyRef:
                  name: {{ .Values.secrets.apiKeySecretName }}
                  key: {{ .Values.secrets.apiKeySecretKey }}
          volumeMounts:
            - name: policy-config
              mountPath: /etc/keeptrusts/
              readOnly: true
          args:
            - gateway
            - run
            - --listen
            - {{ .Values.config.listen | quote }}
            - --policy-config
            - /etc/keeptrusts/policy-config.yaml
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          livenessProbe:
            httpGet:
              path: /health
              port: gateway
            initialDelaySeconds: 10
            periodSeconds: 15
            timeoutSeconds: 5
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /health
              port: gateway
            initialDelaySeconds: 5
            periodSeconds: 10
            timeoutSeconds: 3
            failureThreshold: 2
          startupProbe:
            httpGet:
              path: /health
              port: gateway
            initialDelaySeconds: 3
            periodSeconds: 5
            failureThreshold: 10
      volumes:
        - name: policy-config
          configMap:
            name: keeptrusts-policy-config

Health probes

The gateway exposes a /health endpoint on its configured port. Configure three probe types:

Probe	Purpose	Recommended Settings
Startup	Wait for gateway initialization	initialDelaySeconds: 3, failureThreshold: 10
Readiness	Accept traffic only when ready	periodSeconds: 10, failureThreshold: 2
Liveness	Restart if unresponsive	periodSeconds: 15, failureThreshold: 3

Horizontal Pod Autoscaling

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: keeptrusts-gateway
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: keeptrusts-gateway
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120

OTel collector sidecar

Add an OpenTelemetry collector sidecar for trace and metric forwarding:

- name: otel-collector
  image: otel/opentelemetry-collector-contrib:latest
  ports:
    - containerPort: 4317
      name: otlp-grpc
  volumeMounts:
    - name: otel-config
      mountPath: /etc/otel/
  args:
    - --config=/etc/otel/collector-config.yaml
  resources:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 200m
      memory: 128Mi

Service mesh integration

Istio

Add sidecar injection and traffic policies:

metadata:
  annotations:
    sidecar.istio.io/inject: "true"
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: keeptrusts-gateway
spec:
  host: keeptrusts-gateway
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        h2UpgradePolicy: DEFAULT
        maxRequestsPerConnection: 1000
    outlierDetection:
      consecutive5xxErrors: 3
      interval: 30s
      baseEjectionTime: 30s

Secrets management

Create the API key secret:

kubectl create secret generic keeptrusts-api-key \
  --from-literal=api-key="$KEEPTRUSTS_GATEWAY_TOKEN" \
  --namespace keeptrusts

For production, use an external secrets operator (AWS Secrets Manager, HashiCorp Vault, etc.):

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: keeptrusts-api-key
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets
    kind: ClusterSecretStore
  target:
    name: keeptrusts-api-key
  data:
    - secretKey: api-key
      remoteRef:
        key: keeptrusts/gateway-api-key

Rolling updates

Trigger a config reload by updating the ConfigMap and restarting pods:

# Update the policy config
kubectl create configmap keeptrusts-policy-config \
  --from-file=policy-config.yaml=./policy-config.yaml \
  --namespace keeptrusts \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart deployment to pick up new config
kubectl rollout restart deployment/keeptrusts-gateway -n keeptrusts

# Monitor the rollout
kubectl rollout status deployment/keeptrusts-gateway -n keeptrusts

Troubleshooting

Issue	Cause	Fix
Pods in CrashLoopBackOff	Missing API key secret or invalid config	Check kubectl logs and verify secret exists
Readiness probe failing	Gateway not yet connected to API	Increase initialDelaySeconds on readiness probe
HPA not scaling	Metrics server not installed	Install metrics-server in the cluster
Config changes not applied	ConfigMap updated but pods not restarted	Use kubectl rollout restart or annotation checksums

For AI systems

• Canonical terms: Keeptrusts gateway, Kubernetes Deployment, Helm chart, ConfigMap, HPA, readiness probe, liveness probe, OTel sidecar, ExternalSecret.
• Key config: image: keeptrusts/gateway:<version>, port 41002, ConfigMap policy-config.yaml, Secret keeptrusts-api-key, HPA target 70% CPU.
• CLI commands: helm install, kubectl rollout restart, kubectl rollout status, kt policy lint.
• Best next pages: Terraform IaC, Datadog observability, AWS deployment.

For engineers

• Prerequisites: Kubernetes 1.26+ cluster, kubectl and helm configured, container image accessible, Keeptrusts API key.
• Validate: kubectl get pods -n keeptrusts shows Running pods, readiness probe passing on /health, curl <service>:41002/v1/models returns model list.
• Sizing: Start at 250m CPU / 256Mi memory requests; set limits at 1 CPU / 512Mi. HPA scales 2–10 replicas.
• Config reload: Update ConfigMap then kubectl rollout restart deployment/keeptrusts-gateway to pick up new policy.

For leaders

• Horizontal scaling: HPA ensures gateway capacity grows with demand without manual intervention.
• Self-healing: Kubernetes restarts unhealthy pods automatically, improving availability.
• Cost efficiency: Right-size resource requests and use cluster autoscaler to match capacity to actual load.
• Secret management: Use External Secrets Operator (Vault, AWS Secrets Manager) for production-grade credential rotation.

Next steps

• Manage with Terraform for declarative cluster management
• Monitor with Datadog for Kubernetes-native observability
• Deploy on AWS for ECS/Fargate alternatives