Security Hardening the AI Gateway

The Keeptrusts AI gateway sits in the critical path of all LLM traffic. A compromised gateway can leak sensitive prompts, bypass policy enforcement, or exfiltrate data. This guide covers hardening the gateway and supporting infrastructure against common attack vectors.

Use this page when

• You are hardening Keeptrusts containers for production with non-root execution and read-only filesystems.
• You need to configure Docker security options (no-new-privileges, capability dropping, seccomp profiles).
• You are implementing network policies to restrict container-to-container communication.
• You want to run CIS Docker Benchmark checks against your deployment.

Primary audience

• Primary: Technical Engineers
• Secondary: AI Agents, Technical Leaders

Container Security

Non-Root Execution

Run all Keeptrusts containers as a non-root user:

# Dockerfile — gateway
FROM rust:1.91-slim AS builder
WORKDIR /build
COPY . .
RUN cargo build --release --bin kt

FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
    && rm -rf /var/lib/apt/lists/* \
    && groupadd -r keeptrusts && useradd -r -g keeptrusts -d /home/keeptrusts -s /sbin/nologin keeptrusts

COPY --from=builder /build/target/release/kt /usr/local/bin/kt
USER keeptrusts
EXPOSE 41002
ENTRYPOINT ["kt"]

Verify non-root execution:

docker exec keeptrusts-gateway id
# uid=999(keeptrusts) gid=999(keeptrusts) groups=999(keeptrusts)

docker exec keeptrusts-gateway whoami
# keeptrusts

Read-Only Root Filesystem

Prevent runtime filesystem modifications:

# docker-compose.yml
services:
  keeptrusts-gateway:
    image: keeptrusts/gateway:latest
    read_only: true
    tmpfs:
      - /tmp:noexec,nosuid,size=64m
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL

Resource Limits

Prevent resource exhaustion attacks:

services:
  keeptrusts-gateway:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 256M
    ulimits:
      nofile:
        soft: 65536
        hard: 65536

Minimal Base Image

Use distroless or slim images to reduce the attack surface:

# Multi-stage build with distroless
FROM gcr.io/distroless/cc-debian12
COPY --from=builder /build/target/release/kt /usr/local/bin/kt
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
USER nonroot
EXPOSE 41002
ENTRYPOINT ["kt"]

Network Policies

Docker Network Isolation

# docker-compose.yml — network segmentation
services:
  keeptrusts-gateway:
    networks:
      - frontend
      - backend
  keeptrusts-api:
    networks:
      - backend
      - database
  postgres:
    networks:
      - database

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true
  database:
    driver: bridge
    internal: true

Kubernetes NetworkPolicy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: keeptrusts-gateway-policy
  namespace: keeptrusts
spec:
  podSelector:
    matchLabels:
      app: keeptrusts-gateway
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: application
      ports:
        - port: 41002
          protocol: TCP
  egress:
    # Allow access to API
    - to:
        - podSelector:
            matchLabels:
              app: keeptrusts-api
      ports:
        - port: 8080
    # Allow access to LLM providers (HTTPS)
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
      ports:
        - port: 443
          protocol: TCP
    # Allow DNS
    - to: []
      ports:
        - port: 53
          protocol: UDP
        - port: 53
          protocol: TCP

Secret Management

Environment Variable Best Practices

Never pass secrets as CLI arguments — they appear in ps output:

# BAD — visible in process list
kt gateway run --gateway-token kt_gw_secret_token

# GOOD — environment variable
export KEEPTRUSTS_GATEWAY_TOKEN="kt_gw_secret_token"
kt gateway run --policy-config policy-config.yaml

Docker Secrets

# docker-compose.yml
services:
  keeptrusts-api:
    secrets:
      - db_password
      - jwt_secret
      - encryption_key
    environment:
      - DATABASE_URL=postgres://keeptrusts:file:/run/secrets/db_password@db:5432/keeptrusts
      - KEEPTRUSTS_JWT_SECRET_FILE=/run/secrets/jwt_secret
      # Export KEEPTRUSTS_SECRET_ENCRYPTION_KEY from /run/secrets/encryption_key in your entrypoint before starting the API

secrets:
  db_password:
    file: ./secrets/db_password.txt
  jwt_secret:
    file: ./secrets/jwt_secret.txt
  encryption_key:
    file: ./secrets/encryption_key.txt

Kubernetes Secrets

apiVersion: v1
kind: Secret
metadata:
  name: keeptrusts-secrets
  namespace: keeptrusts
type: Opaque
stringData:
  database-url: "postgres://keeptrusts:password@db:5432/keeptrusts"
  jwt-secret: "your-jwt-secret"
  encryption-key: "your-encryption-key"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: keeptrusts-api
spec:
  template:
    spec:
      containers:
        - name: api
          envFrom:
            - secretRef:
                name: keeptrusts-secrets

HashiCorp Vault Integration

# Read secrets from Vault at startup
export DATABASE_URL=$(vault kv get -field=url secret/keeptrusts/database)
export KEEPTRUSTS_JWT_SECRET=$(vault kv get -field=secret secret/keeptrusts/jwt)
export KEEPTRUSTS_SECRET_ENCRYPTION_KEY=$(vault kv get -field=key secret/keeptrusts/encryption)
kt gateway run --policy-config policy-config.yaml

Host Security

Kernel Hardening (sysctl)

# /etc/sysctl.d/99-keeptrusts.conf
# Prevent IP spoofing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Disable source routing
net.ipv4.conf.all.accept_source_route = 0

# Enable SYN cookies
net.ipv4.tcp_syncookies = 1

# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

# Log suspicious packets
net.ipv4.conf.all.log_martians = 1

File Permissions

# Lock down config and certificate directories
chmod 700 /etc/keeptrusts
chmod 600 /etc/keeptrusts/certs/*
chmod 600 /etc/keeptrusts/policy-config.yaml
chown -R keeptrusts:keeptrusts /etc/keeptrusts

AppArmor Profile

# /etc/apparmor.d/keeptrusts-gateway
#include <tunables/global>

profile keeptrusts-gateway /usr/local/bin/kt {
  #include <abstractions/base>
  #include <abstractions/nameservice>

  network inet stream,
  network inet6 stream,

  /etc/keeptrusts/** r,
  /usr/local/bin/kt mr,
  /tmp/** rw,

  deny /proc/*/environ r,
  deny /etc/shadow r,
}

Image Scanning

# Scan container images for vulnerabilities
trivy image keeptrusts/gateway:latest
trivy image keeptrusts/api:latest

# In CI pipeline
trivy image --exit-code 1 --severity HIGH,CRITICAL keeptrusts/gateway:latest

Audit and Compliance

CIS Docker Benchmark

Run the CIS Docker benchmark tool:

docker run --net host --pid host --userns host --cap-add audit_control \
  -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \
  -v /etc:/etc:ro \
  -v /usr/bin/containerd:/usr/bin/containerd:ro \
  -v /usr/bin/runc:/usr/bin/runc:ro \
  -v /usr/lib/systemd:/usr/lib/systemd:ro \
  -v /var/lib:/var/lib:ro \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  --label docker_bench_security \
  docker/docker-bench-security

Security Checklist

# Verify non-root
docker exec keeptrusts-gateway id | grep -v "uid=0"

# Verify read-only filesystem
docker exec keeptrusts-gateway touch /test 2>&1 | grep -i "read-only"

# Verify no capabilities
docker inspect keeptrusts-gateway --format '{{.HostConfig.CapDrop}}'

# Verify no new privileges
docker inspect keeptrusts-gateway --format '{{.HostConfig.SecurityOpt}}'

Next steps

• TLS/SSL Configuration — encrypt traffic in transit
• Network Configuration — firewall and network segmentation
• Air-Gapped Deployment — maximum isolation deployment

For AI systems

• Canonical terms: Keeptrusts security hardening, container security, non-root execution, read-only filesystem, capability dropping, no-new-privileges, secret management, CIS Docker Benchmark.
• Key config/commands: USER keeptrusts in Dockerfile; read_only: true + tmpfs mounts; cap_drop: [ALL]; security_opt: ["no-new-privileges:true"]; Docker Bench Security scan; docker exec keeptrusts-gateway id (verify non-root).
• Best next pages: TLS/SSL Configuration, Network Configuration, Air-Gapped Deployment.

For engineers

• Prerequisites: Multi-stage Dockerfile builds; understanding of Linux capabilities and seccomp profiles.
• Run all containers as non-root user keeptrusts (uid 999); mount root filesystem read-only with tmpfs for /tmp only.
• Validate with: docker exec keeptrusts-gateway id | grep -v "uid=0"; docker exec keeptrusts-gateway touch /test 2>&1 | grep -i "read-only"; docker inspect --format '{{.HostConfig.CapDrop}}'.
• Run Docker Bench Security (docker/docker-bench-security) to identify remaining hardening gaps.

For leaders

• The gateway sits in the critical path of all LLM traffic — a compromised gateway can bypass all policy enforcement.
• Non-root + read-only filesystem eliminates most container escape and privilege escalation attack vectors.
• CIS Docker Benchmark compliance is often required for SOC 2 Type II and ISO 27001 audits.
• Secret management (no secrets in images or environment variables at rest) prevents credential exposure if containers are compromised.