Embedding Detector

The embedding-detector policy performs similarity-based detection over message content. In the default local mode it uses a built-in bag-of-words cosine-similarity method. In external mode it can call an embedding endpoint, but only when the CLI is built with the embedding-external feature; otherwise it falls back to the local backend.

Configuration

pack:
  name: embedding-detector-example
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector

policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.75
    timeout_ms: 20
    action: block
    categories:
      - label: trade_secret
        reference_text: proprietary manufacturing process internal formula secret recipe
      - label: competitive_intel
        reference_text: competitor pricing strategy acquisition target market positioning

Fields

Field	Type	Description	Default
backend	local | external	Backend selection. external only performs HTTP calls when the CLI is built with --features embedding-external.	local
endpoint	string	External embedding URL. Only used for backend: external.	http://localhost:8080/embed
model	string	External embedding model name. Only used for backend: external.	all-MiniLM-L6-v2
api_key	string	Optional bearer token sent to the external endpoint.	unset
similarity_threshold	number	Minimum cosine similarity required to trigger.	0.70
timeout_ms	integer	External backend timeout in milliseconds.	20
action	redact | block	block stops the request. redact returns a redact verdict for the gateway redaction path.	redact
categories	array	Additional categories appended to the built-in sensitive categories.	[]
categories[].label	string	Stable label for the category.	required
categories[].reference_text	string	Reference text compared against each segment of the input.	required

Built-in sensitive categories

Even when categories is empty, the detector starts with built-in references for:

• social security numbers
• credit cards / payment cards
• medical-record identifiers
• biometric data mentions
• bank-account style identifiers
• prompt-injection language
• export-controlled technical data

External backend note

If you configure backend: external:

• build the CLI with cargo build --features embedding-external
• provide endpoint and, if needed, model and api_key
• expect timeout or transport failures to fall back to the local backend

Use Cases

Local-only semantic screening

pack:
  name: local-similarity-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector
    - audit-logger

policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.8
    action: block

Feature-gated external endpoint

pack:
  name: external-embedding-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector

policy:
  embedding-detector:
    backend: external
    endpoint: https://embeddings.example.com/v1/embeddings
    model: text-embedding-3-small
    timeout_ms: 200
    similarity_threshold: 0.8
    action: block

How It Works

1. The detector splits the input into sentence-like segments.
2. In local mode it compares each segment against the built-in and custom reference texts with a local cosine-similarity calculation.
3. In external mode it requests embeddings for the segment and each reference text, subject to timeout_ms.
4. If the external call fails or times out, the detector falls back to the local path.
5. The first matching segment determines the policy reason code and verdict.

Best Practices

• Treat categories as an extension mechanism; they do not replace the built-in defaults.
• Use higher thresholds (0.8+) first to reduce false positives.
• Reserve external mode for cases where you control the build flags and the endpoint.
• Use block when you need a deterministic outcome rather than a generic redact verdict.

Next steps

• DLP Filter — Exact term and regex controls
• External Moderation — Third-party moderation providers
• Prompt Injection Detection — Dedicated injection policy
• Legal Privilege — Response-phase privilege controls