Cache Sharing Across Gateways in an Agent Group

When multiple gateways belong to the same agent gateway group, they produce identical cache keys for the same request. This page explains how cache key material is constructed, what enables sharing, and what enforces isolation.

Use this page when

• You need to understand how cache keys are constructed for cross-gateway sharing within an agent gateway group.
• You are verifying cache isolation across orgs, entitlements, data residency, or policy versions.
• You want to understand cache behavior during rolling policy deployments.

Primary audience

• Primary: Technical Engineers
• Secondary: AI Agents, Technical Leaders

Cache Key Construction

The org-shared cache key includes the following material:

Field	Included	Purpose
org_id	Yes	Hard security boundary
agent_id	Yes	Logical agent identity
agent_gateway_group_id	Yes	Group-level sharing boundary
codebase_id	Yes	Repository/project context
policy_digest	Yes	Active policy version hash
model_id	Yes	Target model and version
entitlement_tags	Yes	Data-residency and access entitlements
request_content_hash	Yes	Semantic content of the request
gateway_id	No	Excluded — runtime placement only

Because gateway_id is excluded from the key, two gateways in the same group produce the same cache key for the same request when all other fields match.

How Two Gateways Share a Cache Entry

Consider an agent with three gateways (A, B, C) in the same group:

1. Gateway A receives a request and computes the cache key.
2. No cache hit exists. Gateway A forwards to the upstream LLM.
3. Gateway A writes the response to the shared cache under that key.
4. Gateway B receives an identical request and computes the same cache key.
5. Gateway B finds the cache entry written by Gateway A and returns it.

Gateway B never contacts Gateway A directly. The shared cache backend (control-plane metadata + payload store) mediates all access.

Cross-Group Isolation

Different agent gateway groups produce different cache keys even if the underlying request is identical. The agent_gateway_group_id is part of the key material, so:

• Group X cannot read cache entries written by Group Y.
• Two agents with overlapping gateways but different groups maintain separate caches.
• Reassigning a gateway to a different group immediately isolates its cache access.

Security Controls (SEC-008)

Cache sharing is subject to strict security controls. No cache entry is ever shared when any of the following conditions apply:

No Sharing Across Org Boundaries

The org_id is always part of the cache key. Two organizations never share cache entries, even if they use the same agent template or model. This is a hard, non-configurable boundary.

No Sharing With Incompatible Entitlements

Entitlement tags encode data-access permissions. If two requests have different entitlement tags, they produce different cache keys. This prevents a request with restricted entitlements from accessing cached results produced under broader permissions.

Examples of entitlement incompatibility:

• Request A has entitlement: [pii-allowed], Request B has entitlement: [pii-blocked]
• Request A has tier: enterprise, Request B has tier: standard

No Sharing Across Incompatible Data Residency Tags

Data residency tags constrain where data may be stored and processed. Cache entries tagged with a specific residency requirement are only accessible to requests with compatible residency tags.

• A cache entry tagged residency: eu-west is not accessible to a request tagged residency: us-east.
• A cache entry with no residency restriction is accessible to all residency contexts within the same org and group.

No Sharing With Incompatible Policy Digests

The policy digest is a hash of the active policy version. If two gateways in the same group run different policy versions (for example, during a rolling deployment), they produce different cache keys. This ensures cached results always reflect the policy that was active when the response was generated.

What Happens During Policy Rollouts

During a rolling policy update within a gateway group:

1. Gateway A updates to policy v2 (new digest).
2. Gateway B still runs policy v1 (old digest).
3. Gateway A writes cache entries with the v2 digest.
4. Gateway B continues reading and writing with the v1 digest.
5. Once Gateway B updates to v2, it begins reading Gateway A's cached entries.

There is a brief period of reduced cache hit rate during rollouts. This is intentional — it prevents stale policy results from being served.

Cache Entry Metadata

Each cache entry stores metadata that enables the security checks above:

{
  "org_id": "org_abc123",
  "agent_gateway_group_id": "agg_def456",
  "policy_digest": "sha256:9f8e7d...",
  "entitlement_tags": ["pii-blocked", "tier-standard"],
  "residency_tags": ["eu-west"],
  "model_id": "openai/gpt-4o",
  "created_at": "2026-04-30T12:00:00Z",
  "created_by_gateway_id": "gw_789",
  "ttl_seconds": 3600
}

Note that created_by_gateway_id is stored for audit purposes but is not part of the cache key. You can trace which gateway originally produced a cached entry without that information affecting cache hits.

Verifying Cache Sharing Behavior

You can verify that cross-gateway sharing works correctly:

1. Send an identical request through two different gateways in the same group.
2. Check the response headers for x-keeptrusts-cache: hit on the second request.
3. Inspect the cache entry metadata to confirm created_by_gateway_id differs from the serving gateway.

Edge Cases

Gateway Temporarily Removed From Group

If you remove a gateway from a group, it immediately loses access to that group's shared cache. Re-adding it restores access. No cache entries are deleted — only access is revoked.

Gateway in Multiple Groups

A single gateway cannot belong to multiple agent gateway groups simultaneously. This prevents ambiguity in cache key generation.

Empty Group

A group with zero members has no active cache writers or readers. Existing cache entries remain in storage and become accessible again when members are added.

Next steps

• Configuring Gateway Groups — how to set up groups and assign roles
• Distributed Cache Architecture — where cache data physically lives
• Gateway Failover Without Cache Loss — failover behavior

For AI systems

• Canonical terms: Keeptrusts, agent gateway group, cache key construction, cross-gateway cache sharing, cache isolation, SEC-008, entitlement gates, data residency tags, policy digest, org boundary.
• Feature/config names: org_id, agent_id, agent_gateway_group_id, codebase_id, policy_digest, model_id, entitlement_tags, request_content_hash, gateway_id (excluded from key), created_by_gateway_id (audit only), x-keeptrusts-cache: hit.
• Best next pages: Configuring Gateway Groups, Distributed Cache Architecture, Gateway Failover Without Cache Loss.

For engineers

• Verify cross-gateway sharing: send an identical request through two gateways in the same group; check x-keeptrusts-cache: hit on the second response.
• Inspect cache entry metadata to confirm created_by_gateway_id differs from the serving gateway.
• During rolling policy updates, expect a brief hit rate dip — this is intentional while gateways run different policy digests.
• A gateway removed from a group immediately loses shared cache access; re-adding restores it without data loss.

For leaders

• Cache sharing across gateways eliminates redundant provider calls when multiple gateways serve the same agent — cost savings scale with gateway count.
• Hard security boundaries (org, entitlement, residency) are non-configurable and enforced at the cache key level — no data leaks between tenants.
• Policy digest isolation ensures cached results always reflect the correct active policy, even during rolling deployments.
• Compliance: entitlement tags and data residency tags provide auditable proof that cache entries respect access and geographic constraints.