Consumer Groups

Consumer groups let you partition traffic by team, application, or environment so that each segment of your user base operates under a distinct set of policies, rate limits, and upstream providers — all from a single gateway instance. Instead of running separate gateway processes for each team or environment, you declare named groups, assign API keys to them, and configure per-group behaviour in one policy config file.

Use this page when

• You need the exact command, config, API, or integration details for Consumer Groups.
• You are wiring automation or AI retrieval and need canonical names, examples, and constraints.
• If you want a guided rollout instead of a reference page, use the linked workflow pages in Next steps.

Typical use cases:

• Environment isolation — internal-dev group uses a cheap, fast provider; production-app group enforces strict compliance policies and routes to a higher-quality provider.
• Team segmentation — a finance team's API keys automatically apply MNP-I and audit-logging policies that other teams never see.
• Partner tiers — partner integrations get a narrower policy scope and lower rate limits than internal services.
• Anonymous / unauthenticated traffic — a default group catches all requests that don't match any named key assignment.

---

Primary audience

• Primary: AI Agents, Technical Engineers
• Secondary: Technical Leaders

Defining a Consumer Group

Consumer groups are declared under the top-level consumer_groups key. Each entry is a ConsumerGroupRule object.

pack:
  name: consumer-groups-providers-1
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: groq-llama33
    provider: groq:chat:llama-3.3-70b-versatile
    secret_key_ref:
      env: GROQ_API_KEY
  - id: openai-gpt4o
    provider: openai:chat:gpt-4o
    secret_key_ref:
      env: OPENAI_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

ConsumerGroupRule field reference

Field	Type	Required	Description
name	string	yes	Unique name for this group. Used in logs, metrics, and the console.
description	string	no	Human-readable label displayed in the Keeptrusts console.
api_keys	list	yes	API keys that belong to this group. Each entry is either a literal key or an env:<VAR> reference.
rate_limit	object	no	Per-group rate limits. Overrides global rate_limits for matched keys.
rate_limit.rpm	integer	no	Maximum requests per minute for the group.
rate_limit.tpm	integer	no	Maximum tokens per minute for the group.
rate_limit.max_parallel_requests	integer	no	Maximum simultaneous in-flight requests for the group.
chain	list of strings	no	Ordered list of policy IDs to apply. Overrides the global default_chain for this group.
upstream	string	no	Provider target ID to route requests from this group to. Overrides the global default_upstream.

API key formats

consumer_groups:
  - name: example
    api_keys:
      - env:MY_API_KEY          # preferred: resolved from environment variable at startup
      - env:SERVICE_KEY_2       # multiple env vars are supported
      - kt-literal-key-abc123   # literal keys are supported but not recommended for production

Literal API key values in policy-config.yaml are supported but not recommended. Use env:<VAR> references so that key rotation doesn't require modifying the config file.

---

Per-Group Policy Chains

Each consumer group can specify a distinct chain — an ordered list of policy IDs that are applied to every request from that group. Policies are executed in the listed order; if any policy blocks the request, processing stops and the rejection response is returned immediately.

policies:
  - id: pii-redaction
    type: redact
    patterns:
      - type: regex
        pattern: '\b\d{3}-\d{2}-\d{4}\b'   # US SSN
        replacement: "[SSN REDACTED]"
      - type: regex
        pattern: '\b\d{16}\b'               # 16-digit card numbers
        replacement: "[CARD REDACTED]"

  - id: mnpi-filter
    type: block
    keywords:
      - earnings before announcement
      - material non-public
      - insider information
    action: block
    message: "Request blocked: potential MNPI content detected."

  - id: financial-disclaimer
    type: append
    placement: system
    text: |
      This is an AI assistant for authorized personnel. Do not share
      confidential financial data. All interactions are logged.

  - id: audit-logger
    type: log
    include_request: true
    include_response: true
    destination: api   # forward to Keeptrusts control-plane API

  - id: basic-safety-filter
    type: block
    categories:
      - violence
      - hate_speech
      - self_harm

consumer_groups:
  - name: finance-team
    description: "Finance analysts — MNPI filter + full audit trail"
    api_keys:
      - env:FINANCE_TEAM_KEY
    chain:
      - financial-disclaimer   # inject system disclaimer first
      - pii-redaction          # strip PII from prompts
      - mnpi-filter            # block MNPI-adjacent content
      - audit-logger           # log everything to the control plane
    upstream: openai-gpt4o

  - name: engineering
    description: "Internal eng tools — safety filter only"
    api_keys:
      - env:ENG_TEAM_KEY
    chain:
      - basic-safety-filter
    upstream: groq-llama33

Chain execution order

Policies in a chain execute left-to-right. Design your chains so that cheap, broad filters come first (blocking obviously-bad content early) and expensive operations such as embedding-based semantic filters come last.

Request → [financial-disclaimer] → [pii-redaction] → [mnpi-filter] → [audit-logger] → Upstream

If mnpi-filter blocks the request, audit-logger is not executed. If you need unconditional audit logging (including blocked requests), move the logger to the front of the chain.

---

Per-Group Upstream Override

Each consumer group can route to a different provider target via the upstream field. This lets you use a cheap, fast provider for internal development without risking unexpected costs, while routing customer-facing traffic to a premium provider.

pack:
  name: consumer-groups-providers-4
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: groq-llama33
    provider: groq:chat:llama-3.3-70b-versatile
    secret_key_ref:
      env: GROQ_API_KEY
  - id: openai-gpt4o
    provider: openai:chat:gpt-4o
    secret_key_ref:
      env: OPENAI_API_KEY
  - id: openai-gpt4o-mini
    provider: openai:chat:gpt-4o-mini
    secret_key_ref:
      env: OPENAI_API_KEY
  - id: anthropic-claude-haiku
    provider: anthropic:chat:claude-3-haiku-20240307
    secret_key_ref:
      env: ANTHROPIC_API_KEY
policies:
  chain:
  - audit-logger
policy:
  audit-logger:
    immutable: true
    retention_days: 365
    log_all_access: true

The upstream field in a consumer group always overrides the global default_upstream. If the upstream target is configured as part of a model group, the model group routing strategy (weighted round-robin, lowest-latency, etc.) still applies.

---

Multiple Groups

A complete example showing four groups with varying access levels:

consumer_groups:
  - name: admin
    description: "Trusted emergency operators — no rate limits, audit-only chain"
    api_keys:
      - env:ADMIN_API_KEY
    rate_limit:
      rpm: 10000
      tpm: 50000000
      max_parallel_requests: 500
    chain:
      - audit-logger        # log admin actions but don't restrict them
    upstream: openai-gpt4o

  - name: internal
    description: "Internal engineering and product teams"
    api_keys:
      - env:ENG_KEY
      - env:PRODUCT_KEY
      - env:DATA_TEAM_KEY
    rate_limit:
      rpm: 300
      tpm: 1000000
      max_parallel_requests: 30
    chain:
      - basic-safety-filter
    upstream: groq-llama33

  - name: partner
    description: "External partner integrations — restricted scope"
    api_keys:
      - env:PARTNER_A_KEY
      - env:PARTNER_B_KEY
    rate_limit:
      rpm: 60
      tpm: 100000
      max_parallel_requests: 5
    chain:
      - partner-scope-filter
      - pii-redaction
      - audit-logger
    upstream: openai-gpt4o-mini

  - name: anonymous
    description: "Unauthenticated or unrecognised keys — strictest defaults"
    api_keys: []              # empty = catch-all default group
    rate_limit:
      rpm: 10
      tpm: 20000
      max_parallel_requests: 2
    chain:
      - content-safety
      - pii-redaction

First-match wins

When a request arrives, Keeptrusts evaluates the consumer_groups list from top to bottom. The first group whose api_keys contains the request's API key wins. List more-specific groups above less-specific ones.

consumer_groups:
  - name: admin          # checked first
    api_keys:
      - env:ADMIN_KEY
  - name: internal       # checked second
    api_keys:
      - env:ENG_KEY
  - name: anonymous      # catch-all — must be last
    api_keys: []

The group with api_keys: [] matches every request that hasn't been claimed by an earlier group. Place it last in the list, otherwise it will shadow all subsequent group definitions.

---

Default Group

The default group is the group with an empty api_keys list. It catches requests that arrive with an API key that isn't listed in any named group, as well as (if configured) unauthenticated requests.

consumer_groups:
  - name: production-app
    api_keys:
      - env:PROD_KEY
    chain:
      - pii-redaction
      - content-safety
    upstream: openai-gpt4o

  - name: default
    description: "Fallback for unrecognised keys"
    api_keys: []
    rate_limit:
      rpm: 5
      tpm: 5000
      max_parallel_requests: 1
    chain:
      - content-safety
      - audit-logger
    upstream: openai-gpt4o-mini

If no group matches and there is no default group defined, Keeptrusts falls back to the globally configured default_chain and default_upstream.

---

Combining with Rate Limiting

Consumer group rate_limit values take precedence over the global rate_limits block for any key that belongs to the group. Other rate-limit tiers (per_user, per_team, ip) continue to apply independently.

rate_limits:
  per_key:
    rpm: 60             # global default for all keys
    tpm: 100000
    max_parallel_requests: 10
  global:
    rpm: 5000
    tpm: 10000000

consumer_groups:
  - name: admin
    api_keys:
      - env:ADMIN_KEY
    rate_limit:
      rpm: 10000        # overrides the global per_key.rpm of 60
      tpm: 50000000     # overrides the global per_key.tpm of 100000
      max_parallel_requests: 500

  - name: batch-jobs
    api_keys:
      - env:BATCH_KEY
    rate_limit:
      rpm: 10           # more restrictive than the global default
      tpm: 500000
      max_parallel_requests: 3

In this example, a request from ADMIN_KEY is allowed up to 10 000 RPM, while a request from BATCH_KEY is capped at 10 RPM. All other keys fall back to the global 60 RPM per-key limit. The global ceiling (rpm: 5000) still applies across all traffic regardless of consumer group overrides.

---

Monitoring Consumer Groups

Keeptrusts adds a kt-consumer-group header to upstream requests and response logs so you can slice metrics by group.

Every structured log line and event forwarded to the control-plane API includes a consumer_group field:

{
  "timestamp": "2026-03-27T14:22:01.342Z",
  "consumer_group": "finance-team",
  "api_key_hash": "sha256:a3f1...",
  "model": "gpt-4o",
  "upstream": "openai-gpt4o",
  "policies_applied": ["financial-disclaimer", "pii-redaction", "mnpi-filter", "audit-logger"],
  "decision": "allow",
  "latency_ms": 842,
  "prompt_tokens": 312,
  "completion_tokens": 156
}

Use the Keeptrusts console Events view and filter by consumer_group to see per-group traffic, latency, token usage, and policy hit rates.

---

Best Practices

1. Use env:<VAR> for all API key references. Literal keys in config files are a security risk. Reference environment variables so you can rotate keys without touching the policy config.

2. Place more-specific groups first. Consumer groups are evaluated top-to-bottom. If you have an admin group, it should come before internal, which should come before default.

3. Always define a default group. An api_keys: [] catch-all prevents unrecognised keys from silently bypassing policy enforcement. Give the default group the strictest rate limits and most conservative policy chain.

4. Keep chains short and targeted. Each additional policy in a chain adds latency. Profile your chain with the /v1/events metrics and remove policies that never trigger for a given group.

5. Pair upstream overrides with cost budgets. If a group is allowed to route to an expensive model, set tight tpm limits so an accidental infinite loop or runaway agent doesn't exceed your billing budget.

6. Audit group memberships periodically. Rotate API keys that belong to privileged groups (e.g., admin) on a regular schedule. Consider setting a short TTL on partner keys and using env:<VAR> references so expiry is enforced at the OS / secrets-manager level without a gateway restart.

For AI systems

• Canonical terms: Keeptrusts Consumer Groups, ConsumerGroupRule, per-group policy chain, per-group rate limit, per-group upstream.
• Config keys: consumer_groups[].name, consumer_groups[].api_keys (env:<VAR> or literal), consumer_groups[].rate_limit (rpm, tpm, max_parallel_requests), consumer_groups[].chain, consumer_groups[].upstream, consumer_groups[].cache_enabled.
• Matching logic: first-match-wins top-to-bottom; api_keys: [] is the catch-all default group.
• Event field: consumer_group on every structured log line forwarded to the control-plane API.
• Upstream header: kt-consumer-group injected on upstream requests.
• Best next pages: Rate Limiting, Custom Routes, Semantic Caching.

For engineers

• Prerequisites: API keys assigned to groups via env:<VAR> references (avoid literal keys in production).
• Validation: run kt config lint to verify all upstream references resolve to declared provider targets.
• Testing: send a request with a known group key and confirm the expected consumer_group field appears in Events.
• Always define a default group (api_keys: []) as the last entry to prevent unrecognised keys from bypassing policy enforcement.
• Chain order matters: cheap broad filters first, expensive semantic filters last.
• Combine with namespace_from_consumer_group: true in cache config to isolate cached responses per group.

For leaders

• Multi-tenancy without multiple gateways: consumer groups let you serve partners, internal teams, and public traffic from a single gateway instance with distinct policies and cost controls.
• Cost allocation: per-group tpm limits prevent any single team or partner from consuming the entire provider budget.
• Compliance: assign regulated teams (finance, healthcare) a chain that includes mandatory audit logging and PII redaction without affecting other teams.
• Rotation: use env:<VAR> key references so API key rotation is a secrets-manager operation, not a config-file deployment.

Next steps

• Rate Limiting — per-group rate limit overrides and distributed limiting
• Custom Routes — combine path-based routing with consumer group policy chains
• Semantic Caching — per-group cache namespace isolation
• Provider Routing — how upstream overrides interact with routing strategies