Security Engineering for AI Pipelines

AI pipelines introduce unique attack surfaces — prompt injection, data exfiltration through model outputs, credential exposure in configs, and unauthorized model access. This guide covers defense-in-depth strategies across the Keeptrusts platform.

Use this page when

You are configuring TLS termination and mTLS between services and the gateway
You need to set up secret rotation for provider API keys in the gateway config
You are implementing RBAC via console Teams to control who can access which models
You want to add DLP policies or prompt injection defense to your policy chain

Primary audience

Primary: Technical Engineers
Secondary: AI Agents, Technical Leaders

Security Architecture

TLS Configuration

Gateway TLS Termination

gateway:
  tls:
    enabled: true
    cert_file: /etc/kt/tls/server.crt
    key_file: /etc/kt/tls/server.key
    # Minimum TLS version
    min_version: "1.2"
    # Preferred cipher suites
    cipher_suites:
      - TLS_AES_256_GCM_SHA384
      - TLS_AES_128_GCM_SHA256
      - TLS_CHACHA20_POLY1305_SHA256

Upstream TLS Verification

Always verify provider TLS certificates:

pack:
  name: security-engineering-providers-2
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai
    provider: 
    base_url: https://api.openai.com/v1
    secret_key_ref:
      env: OPENAI_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

mTLS Between Services

Certificate Setup

Gateway mTLS Configuration

gateway:
  mtls:
    enabled: true
    # Gateway's client certificate for connecting to the API
    client_cert: /etc/kt/mtls/gateway.crt
    client_key: /etc/kt/mtls/gateway.key
    # CA to verify the API's certificate
    ca_cert: /etc/kt/mtls/ca.crt

Kubernetes TLS with cert-manager

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kt-gateway-cert
spec:
  secretName: kt-gateway-tls
  issuerRef:
    name: internal-ca
    kind: ClusterIssuer
  commonName: kt-gateway.ai-platform.svc.cluster.local
  dnsNames:
    - kt-gateway
    - kt-gateway.ai-platform
    - kt-gateway.ai-platform.svc.cluster.local
  duration: 720h    # 30 days
  renewBefore: 168h  # Renew 7 days before expiry

Secret Management

API Key Storage

Never store API keys in plain text configs. Use environment variables or the Keeptrusts config variable system:

pack:
  name: security-engineering-providers-5
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai
    provider: 
    secret_key_ref:
      store: OPENAI_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

Secret Rotation

The Keeptrusts API encrypts secrets at rest with AES-GCM-SIV. Rotate secrets without downtime:

# 1. Set the new key in the console (Settings → Config Variables)
#    or via the API
curl -X PUT https://api.keeptrusts.com/v1/config-variables/OPENAI_API_KEY \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -d '{"value": "sk-new-key-value..."}'

# 2. Reload gateway configuration
kt gateway reload

# 3. Verify the new key works
kt health --verbose

Secret Rotation Flow

RBAC via Console Teams

Role Hierarchy

Team-Based Access Control

Organize access by team to enforce least privilege:

Organization: Acme Corp
├── Team: Platform Engineering
│   ├── Role: Admin
│   ├── Access: All gateways, all configs, all events
│   └── Members: platform-eng@acme.com
├── Team: ML Engineering
│   ├── Role: Member
│   ├── Access: ml-gateway, ml-configs, team events
│   └── Members: ml-eng@acme.com
└── Team: Customer Support
    ├── Role: Viewer
    ├── Access: support-gateway (read-only), team events
    └── Members: support@acme.com

Gateway Key Scoping

Issue scoped gateway keys per team or application:

# Create a gateway key scoped to specific policies
# Via console: Settings → Gateway Keys → Create

# The gateway key restricts which policies and models are accessible
# kt_gk_... tokens are short-lived and rotatable

DLP Policies

Data Loss Prevention Configuration

Prevent sensitive data from reaching LLM providers:

policies:
  - name: pii-input-filter
    type: input_filter
    action: redact
    patterns:
      - type: email
        replacement: "[EMAIL REDACTED]"
      - type: phone
        replacement: "[PHONE REDACTED]"
      - type: ssn
        replacement: "[SSN REDACTED]"
      - type: credit_card
        replacement: "[CC REDACTED]"
      - type: regex
        pattern: '\b[A-Z]{2}\d{6,10}\b'
        replacement: "[ACCOUNT REDACTED]"

  - name: pii-output-filter
    type: output_filter
    action: redact
    patterns:
      - type: email
      - type: phone
      - type: ssn

DLP Flow

Content Classification Policies

policies:
  - name: content-safety
    type: content_filter
    action: block
    categories:
      - hate_speech
      - violence
      - self_harm
      - sexual_content
    threshold: medium

  - name: topic-restriction
    type: content_filter
    action: block
    blocked_topics:
      - "competitor pricing"
      - "internal roadmap"
      - "employee compensation"

Prompt Injection Defense

Multi-Layer Defense

Policy Configuration

policies:
  - name: prompt-injection-detection
    type: input_filter
    action: block
    detection:
      # Heuristic patterns
      patterns:
        - "ignore previous instructions"
        - "ignore all prior"
        - "disregard the above"
        - "new instructions:"
        - "system prompt:"
      # ML-based classifier
      classifier:
        enabled: true
        threshold: 0.85
        model: keeptrusts/injection-detector

  - name: output-validation
    type: output_filter
    action: escalate
    detection:
      # Detect if the model leaked system prompt content
      system_prompt_leak: true
      # Detect unexpected format changes
      format_deviation: true

Escalation on Detection

When prompt injection is detected, the gateway can escalate:

policies:
  - name: injection-escalation
    type: input_filter
    action: escalate
    escalation:
      # Notify via the console
      console_notification: true
      # Send webhook
      webhook: https://hooks.example.com/security
      # Block the request while escalating
      block_pending: true

Network Security

Network Segmentation

# Kubernetes NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: kt-gateway-policy
spec:
  podSelector:
    matchLabels:
      app: kt-gateway
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              role: ai-consumer
      ports:
        - port: 41002
  egress:
    # Allow connections to LLM providers
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
      ports:
        - port: 443
    # Allow connections to the control-plane API
    - to:
        - podSelector:
            matchLabels:
              app: kt-api
      ports:
        - port: 8080

Security Audit Checklist

Pre-Production

Ongoing

Review blocked requests weekly for policy tuning
Rotate API keys quarterly
Rotate gateway keys monthly
Review escalation queue for injection attempts
Audit team membership and role assignments
Update DLP patterns for new data types
Test prompt injection defenses with adversarial inputs

Next steps

Architecture Patterns for AI-Governed Systems — secure deployment topologies
Observability for AI-Governed Systems — monitor security events
Resilience Engineering for AI Services — security under failure

For AI systems

Canonical terms: TLS 1.2+, mTLS, gateway.tls, secret_key_ref, AES-GCM-SIV encryption at rest, RBAC, console Teams, DLP policies, prompt injection detection, pii_detection, content_filter, gateway keys, zero trust
Key configuration: gateway.tls.min_version: "1.2", providers[].tls.verify: true, secret_key_ref.store (config variables), Teams-based access control
Best next pages: Architecture Patterns, CI/CD Pipeline Integration, Incident Response

For engineers

Enable TLS: set gateway.tls.enabled: true with cert/key files; enforce TLS 1.2+ minimum and strong cipher suites
mTLS between services: issue certs from shared CA, configure client_ca_file on gateway for mutual authentication
Secret rotation: use secret_key_ref.store (config variables via API) instead of secret_key_ref.env for rotatable credentials
Prompt injection defense: add a type: prompt_injection_detection policy in the input chain; tuned threshold reduces false positives
DLP: combine pii_detection (action: redact) and content_filter (action: block) to prevent data exfiltration through model outputs

For leaders

Defense-in-depth: TLS for transport, mTLS for service identity, RBAC for access control, DLP for data loss prevention — multiple independent layers
Gateway keys replace raw provider API keys, eliminating credential sprawl and enabling instant revocation
Prompt injection defense is the AI-specific attack vector that traditional WAFs cannot detect — the gateway addresses this at the request layer
AES-GCM-SIV encryption at rest for stored secrets means a database breach does not expose provider credentials

Use this page when​

Primary audience​

Security Architecture​

TLS Configuration​

Gateway TLS Termination​

Upstream TLS Verification​

mTLS Between Services​

Certificate Setup​

Gateway mTLS Configuration​

Kubernetes TLS with cert-manager​

Secret Management​

API Key Storage​

Secret Rotation​

Secret Rotation Flow​

RBAC via Console Teams​

Role Hierarchy​

Team-Based Access Control​

Gateway Key Scoping​

DLP Policies​

Data Loss Prevention Configuration​

DLP Flow​

Content Classification Policies​

Prompt Injection Defense​

Multi-Layer Defense​

Policy Configuration​

Escalation on Detection​

Network Security​

Network Segmentation​

Security Audit Checklist​

Pre-Production​

Ongoing​

Next steps​

For AI systems​

For engineers​

For leaders​