Skip to main content

Recommended Policies for IDE Traffic

IDE assistants send code context frequently and with low tolerance for latency. A strong baseline keeps the chain short, protects sensitive text before it leaves the editor, and adds only the runtime features you can validate day to day.

Start with this baseline​

This configuration keeps IDE traffic predictable while covering the most common risks.

pack:
  name: ide-baseline
  version: 1.0.0
  enabled: true

providers:
  targets:
    - id: openai-ide
      provider: openai
      model: gpt-5.4-mini
      secret_key_ref:
        env: KEEPTRUSTS_OPENAI_API_KEY

user_rate_limit:
  max_requests: 30
  window_seconds: 60
  header_names:
    - x-user-id

cache:
  mode: exact
  default_on: true

policies:
  chain:
    - prompt-injection
    - pii-detector
    - audit-logger

policy:
  prompt-injection:
    attack_patterns:
      - ignore.*previous.*instructions
      - reveal.*system.*prompt
    response:
      action: block

  pii-detector:
    action: redact
    pci_mode: true
    redaction:
      marker_format: label

  audit-logger: {}

Why this baseline works​

Keep the request chain short​

Inline completions are latency-sensitive. Start with:

  • prompt-injection to reject obvious jailbreak and instruction-override attempts
  • pii-detector to redact sensitive identifiers in prompts, pasted logs, comments, and literals
  • audit-logger to mark audit logging active in the policy result. It does not configure persistence or retention by itself; review the resulting evidence with kt events tail or GET /v1/events?since=10m, then use Trail and capture-enabled History as applicable.

Add heavier buffered-output controls only when you have verified that the extra latency is acceptable for the editor workflow you care about.

Use user rate limits for runaway loops​

IDE extensions can retry aggressively or fire repeated completion calls while a developer types. user_rate_limit gives requests that carry a configured, non-empty identity header a stable per-user ceiling without forcing those requests into the same shared bucket. Requests without that header bypass the per-user limiter, so require or inject it at a trusted boundary.

Use Rate Limits Configuration when you need a larger window, the Chat Completions token limiter, or the distributed global request counter across multiple gateway instances.

Use response caching only when repetition is high​

The public runtime cache is configured under top-level cache:. It can help only when the IDE sends non-streaming Chat Completions or Responses requests that the gateway classifies as read-only; streaming completion traffic bypasses the cache. Where that boundary fits the client, exact caching is the easiest mode to reason about because it requires an identical request body.

Use Runtime Configuration for semantic cache options, backend selection, and cache environment variables.

When to add more than the baseline​

Add more controls only for a clear product reason:

  • Add dlp-filter when you need broader proprietary-data pattern checks than pii-detector.
  • Add data-routing-policy when provider retention or training metadata affects where IDE traffic may go.
  • Add quality-scorer only after measuring whether buffered output checks are acceptable for your completion latency budget.
  • Add route-specific chains with Routes and Consumer Groups when Chat Completions and Responses need different policy enforcement. Routes do not select upstream providers.

Validate before rollout​

Run the config through the same public validation flow you use for any other gateway traffic:

kt policy lint
kt policy test --json
kt gateway run --agent ide-gateway --policy-config policy-config.yaml
kt events tail --since 10m --follow

While the gateway is running, trigger completions from the IDE and confirm:

  • requests appear in the event tail
  • the expected token identity or user header is present
  • prompt-injection or redaction verdicts occur when you test known bad input
  • latency stays acceptable for the editor workflow

What not to do​

  • Do not add speculative or unpublished policy kinds just because they sound useful for IDEs.
  • Do not embed provider secrets directly in YAML when secret_key_ref can reference them safely.
  • Do not assume chat-oriented policy chains are appropriate for inline completion traffic without measuring the latency impact.

Next steps​