Skip to main content

Document Analyzer

The document-analyzer policy inspects documents embedded in the request JSON under keys such as documents, attachments, input_documents, and files. It can block oversized or disallowed MIME types and optionally sanitize extracted document text for a fixed set of risky code patterns.

Configuration​

pack:
  name: document-analyzer-example
  version: 1.0.0
  enabled: true

policies:
  chain:
    - document-analyzer
    - prompt-injection
    - pii-detector

policy:
  document-analyzer:
    enabled: true
    sanitize_code: true
    max_document_bytes: 524288
    allowed_mime_types:
      - application/pdf
      - text/plain
      - application/vnd.openxmlformats-officedocument.wordprocessingml.document

Fields​

FieldTypeDescriptionDefault
enabledbooleanDisable the analyzer without removing it from the chain.true
sanitize_codebooleanApply the built-in code-sanitization rules to extracted document text.true
max_document_bytesintegerBlock a document when its raw payload exceeds this byte size.524288
allowed_mime_typesstring[]Optional MIME allowlist. Empty means all MIME types are accepted.[]

Supported request shapes​

The analyzer looks for arrays under these request keys:

  • documents
  • attachments
  • input_documents
  • files

For each item it reads:

  • name (optional, defaults to inline-document)
  • mime_type or content_type
  • text for inline text documents, or content_base64 / data for encoded bytes

Extraction behavior​

  • application/pdf → heuristic PDF text extraction
  • DOCX MIME types containing wordprocessingml.document → text from word/document.xml
  • Everything else → lossy UTF-8 text conversion of the raw bytes

Code sanitization behavior​

When sanitize_code is true, extracted text is scanned for the built-in patterns from the gateway code-sanitizer helper:

  • rm -rf
  • drop table
  • 169.254.169.254
  • curl http://localhost / curl https://localhost
  • chmod 777
  • recursive aws s3 cp

Matches are replaced with [redacted code pattern] in the extracted text fragments that downstream policies see.

Use Cases​

MIME and size guardrail​

pack:
  name: upload-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - document-analyzer
    - audit-logger

policy:
  document-analyzer:
    max_document_bytes: 1048576
    allowed_mime_types:
      - application/pdf
      - text/plain
      - image/png
      - image/jpeg

Document text into downstream privacy checks​

pack:
  name: doc-plus-pii
  version: 1.0.0
  enabled: true

policies:
  chain:
    - document-analyzer
    - pii-detector
    - dlp-filter

policy:
  document-analyzer:
    sanitize_code: true
  pii-detector:
    action: redact
  dlp-filter:
    blocked_terms:
      - Project Titan
    action: block

How It Works​

  1. The analyzer reads document entries from the structured request payload.
  2. It blocks immediately if any document exceeds max_document_bytes.
  3. It blocks immediately if allowed_mime_types is non-empty and a document MIME type is missing from the allowlist.
  4. It extracts text from each accepted document and records findings such as source name, MIME type, extracted character count, and detected code-block count.
  5. It adds the extracted text fragments to the policy details so later policies can inspect them.

Best Practices​

  • Keep the MIME allowlist small and explicit in production.
  • Leave sanitize_code enabled unless you have a strong reason to preserve raw risky code text.
  • Pair this policy with prompt-injection, pii-detector, or dlp-filter when document text should be governed the same way as message text.
  • Do not assume this policy fetches remote URLs or parses multipart uploads outside the JSON body shapes listed above.

Next steps​