Tool Validation

The tool-validation policy controls which tools an AI agent is allowed to call and validates tool-call arguments against JSON Schema definitions. It supports strict allowlisting, per-tool schema enforcement, optional LLM-based semantic validation, and a permissive monitoring mode for gradual rollout.

Use this page when

• You need to control which tools an AI agent is allowed to call via strict allowlisting.
• You are defining JSON Schema validation for tool-call arguments to catch malformed or malicious inputs.
• You want LLM-based semantic validation to detect structurally valid but intentionally misused tool calls.

When a tool call is denied or fails validation, the gateway blocks execution, returns a policy-violation response to the model, and logs the event with the specific validation failure.

Primary audience

• Primary: AI Agents, Technical Engineers
• Secondary: Technical Leaders

Configuration

pack:
  name: tool-validation-example-1
  version: 1.0.0
  enabled: true
policies:
  chain:
  - tool-validation
policy:
  tool-validation:
    declared_tools:
    - web_search
    - database_query
    - file_reader
    - code_execution
    schemas:
      database_query:
        type: object
        properties:
          table:
            type: string
            enum:
            - users
            - orders
            - products
          filters:
            type: object
        required:
        - table
      file_reader:
        type: object
        properties:
          path:
            type: string
            maxLength: 255
        required:
        - path
      code_execution:
        type: object
        properties:
          language:
            type: string
            enum:
            - python
            - javascript
          code:
            type: string
            maxLength: 5000
        required:
        - language
        - code
    allow_undeclared: false
    semantic_validation:
      enabled: true
      endpoint: https://api.openai.com/v1/chat/completions
      model: gpt-4o-mini
      secret_key_ref:
        env: OPENAI_API_KEY
      timeout_ms: 3000

Fields

Top-level

Property	Type	Default	Description
declared_tools	string[]	[]	Allowlisted tool names. When non-empty, only tools in this list are permitted. Any tool call whose name is not in the list is denied unless allow_undeclared is true. An empty list with allow_undeclared: false blocks all tool calls.
schemas	object	{}	Map of tool names to JSON Schema objects. Each key is the exact tool name; each value is a standard JSON Schema (draft 2020-12 compatible) that the tool-call arguments must validate against. If a tool appears in declared_tools but not in schemas, the tool is allowed without argument validation.
allow_undeclared	boolean	false	When true, tool calls to tools not listed in declared_tools are allowed but logged with a tool_undeclared warning event. When false, undeclared tool calls are blocked. This is useful for gradual rollout: start permissive, monitor which tools agents use, then lock down declared_tools.
semantic_validation	object	—	Optional LLM-based semantic validation configuration. When enabled, tool calls that pass schema validation are additionally sent to an LLM to detect semantic misuse — cases where the arguments are structurally valid but the intent is malicious or nonsensical.

semantic_validation object

Property	Type	Format	Constraint	Default	Description
enabled	boolean	—	—	false	Enables or disables semantic validation. When false, the entire semantic_validation block is ignored even if other fields are set.
endpoint	string	URI	—	—	OpenAI-compatible chat completions endpoint URL. The gateway sends a chat completion request with the tool name, arguments, and a system prompt asking the model to evaluate whether the tool call is appropriate. Required when enabled is true.
model	string	—	—	—	Model identifier to use for semantic validation (e.g., gpt-4o-mini, claude-3-haiku-20240307). Smaller, faster models are preferred to minimize latency. Required when enabled is true.
secret_key_ref	string	—	—	—	Name of the environment variable holding the API key for the semantic validation endpoint. The gateway reads this at startup and includes it as a Bearer token. Required when enabled is true.
timeout_ms	integer	—	>= 1	3000	Maximum time in milliseconds to wait for the semantic validation LLM response. If the LLM does not respond within this window, the tool call is allowed (fail-open for semantic validation, since schema and allowlist checks have already passed).

Use Cases

1. Strict tool allowlist

Only permit specific tools — any undeclared tool call is blocked:

pack:
  name: tool-validation-example-2
  version: 1.0.0
  enabled: true
policies:
  chain:
  - tool-validation
policy:
  tool-validation:
    declared_tools:
    - web_search
    - calculator
    - get_weather
    allow_undeclared: false

If the agent attempts to call file_delete or execute_command, the call is blocked immediately.

2. JSON Schema validation per tool

Enforce argument types, required fields, and value constraints:

pack:
  name: tool-validation-example-3
  version: 1.0.0
  enabled: true
policies:
  chain:
  - tool-validation
policy:
  tool-validation:
    declared_tools:
    - create_user
    - send_email
    schemas:
      create_user:
        type: object
        properties:
          name:
            type: string
            minLength: 1
            maxLength: 100
          email:
            type: string
            format: email
          role:
            type: string
            enum:
            - viewer
            - editor
            - admin
        required:
        - name
        - email
        - role
        additionalProperties: false
      send_email:
        type: object
        properties:
          to:
            type: string
            format: email
          subject:
            type: string
            maxLength: 200
          body:
            type: string
            maxLength: 10000
        required:
        - to
        - subject
        - body
        additionalProperties: false

A call to create_user with role: "superadmin" is rejected because superadmin is not in the allowed enum. A call missing the email field is rejected for a missing required property.

3. Semantic validation with LLM

Detect misuse that passes schema validation — like using a search_users tool to enumerate and exfiltrate all user data:

pack:
  name: tool-validation-example-4
  version: 1.0.0
  enabled: true
policies:
  chain:
  - tool-validation
policy:
  tool-validation:
    declared_tools:
    - search_users
    - update_profile
    - send_notification
    schemas:
      search_users:
        type: object
        properties:
          query:
            type: string
            maxLength: 200
          limit:
            type: integer
            minimum: 1
            maximum: 50
        required:
        - query
    semantic_validation:
      enabled: true
      endpoint: https://api.openai.com/v1/chat/completions
      model: gpt-4o-mini
      secret_key_ref:
        env: OPENAI_API_KEY
      timeout_ms: 3000

A call like search_users({ query: "*", limit: 50 }) passes schema validation but semantic validation may flag it as a data exfiltration attempt.

4. Permissive mode with logging

Allow all tool calls but log undeclared ones for monitoring during initial deployment:

pack:
  name: tool-validation-example-5
  version: 1.0.0
  enabled: true
policies:
  chain:
  - tool-validation
policy:
  tool-validation:
    declared_tools:
    - web_search
    - calculator
    allow_undeclared: true

Calls to web_search and calculator proceed normally. Calls to any other tool also proceed but generate a tool_undeclared warning event. Review events to discover which tools agents actually call, then update declared_tools to the final allowlist.

5. Full agent governance stack

Combine tool-validation with tool-security, tool-budget, and agent-firewall for comprehensive agent governance:

pack:
  name: tool-validation-example-6
  version: 1.0.0
  enabled: true
policies:
  chain:
  - agent-firewall
  - tool-validation
  - tool-security
  - tool-budget
policy:
  agent-firewall:
    mode: enforce
  tool-validation:
    declared_tools:
    - web_search
    - code_execution
    - file_read
    - database_query
    schemas:
      code_execution:
        type: object
        properties:
          language:
            type: string
            enum:
            - python
            - javascript
          code:
            type: string
            maxLength: 5000
        required:
        - language
        - code
        additionalProperties: false
      database_query:
        type: object
        properties:
          sql:
            type: string
            maxLength: 1000
          params:
            type: array
            items:
              type: string
            maxItems: 10
        required:
        - sql
    allow_undeclared: false
    semantic_validation:
      enabled: true
      endpoint: https://api.openai.com/v1/chat/completions
      model: gpt-4o-mini
      secret_key_ref:
        env: OPENAI_API_KEY
      timeout_ms: 5000
  tool-security:
    analysis_mode: local
  tool-budget:
    budgets:
      web_search:
        max_tokens: 50000
      code_execution:
        max_tokens: 30000
        max_cost_usd: 1.0
      database_query:
        max_tokens: 20000
        max_cost_usd: 0.25

Evaluation order: agent-firewall → tool-validation (allowlist + schema + semantic) → tool-security → tool-budget → execution.

How It Works

1. Allowlist check — When a tool call is detected, the gateway first checks whether the tool name appears in declared_tools. If the list is non-empty and the tool is absent:

   • If allow_undeclared is false: the call is blocked with reason tool_not_declared.
   • If allow_undeclared is true: the call is allowed but a tool_undeclared warning event is logged.

2. Schema validation — If the tool has an entry in schemas, the tool-call arguments are validated against the JSON Schema. Validation failures (type mismatches, missing required fields, enum violations, pattern failures, constraint violations) block the call with reason tool_schema_invalid and include the specific validation error message.

3. Semantic validation — If semantic_validation.enabled is true and the tool call passed schema validation, the gateway sends a chat completion request to the configured endpoint. The request contains the tool name, arguments, and a system prompt instructing the model to evaluate whether the call is appropriate. If the model responds with a denial verdict, the call is blocked with reason tool_semantic_violation. If the model times out (exceeds timeout_ms), the call is allowed (fail-open) since it already passed structural validation.

4. Pass-through — If all checks pass, the tool call proceeds to the next policy in the chain (e.g., tool-security, tool-budget) or directly to execution.

5. Event logging — Every validation outcome (allowed, blocked, warned) is logged as a decision event with metadata including the tool name, validation stage that produced the result, and the specific failure reason if blocked.

Combining With Other Policies

Combined with	Effect
tool-security	Validation ensures correct structure; security scans for injection in valid arguments. Validation runs first.
tool-budget	Budget accounting only applies to calls that pass validation, so rejected calls don't consume budget.
agent-firewall	The agent firewall provides broad behavioral and intent-level controls; tool-validation provides per-tool structural and semantic controls. The firewall runs first.
content-filter	Content filtering applies to model outputs; tool-validation applies to tool-call arguments within model outputs. Both can run on the same response.
rate-limiter	Rate limiting caps request frequency; tool-validation caps which tools can be called and with what arguments. Orthogonal protections.

Recommended evaluation order: agent-firewall → tool-validation → tool-security → tool-budget → tool execution → content-filter.

Best Practices

• Declare every tool. Start with allow_undeclared: true to discover which tools your agents use, then switch to false and lock down the list. An empty declared_tools with allow_undeclared: false blocks all tool calls.
• Use additionalProperties: false in schemas. This prevents agents from passing unexpected arguments that could be exploited by prompt injection attacks that add hidden fields.
• Keep schemas strict but realistic. Use enum for fields with known values, maxLength for strings, minimum/maximum for numbers, and pattern for structured identifiers. Avoid over-constraining fields that legitimately vary.
• Use semantic validation for high-risk tools. Tools that modify data, send communications, or access sensitive resources benefit from LLM-based intent checking. Low-risk read-only tools may not need it.
• Choose a fast model for semantic validation. Semantic validation adds latency to every tool call. Use a small, fast model like gpt-4o-mini and keep timeout_ms low (2000–5000ms).
• Fail-open for semantic validation is intentional. Schema and allowlist checks are deterministic and fail-closed. Semantic validation is probabilistic and fails open to avoid blocking legitimate calls due to LLM errors or timeouts.
• Monitor tool_undeclared events. These events reveal tools your agents call that you haven't explicitly approved. Either add them to declared_tools or investigate why the agent is calling them.
• Version your schemas alongside your tools. When a tool's argument structure changes, update the schema in your policy config at the same time. Stale schemas cause false rejections.

For AI systems

• Canonical terms: Keeptrusts, tool-validation, declared_tools, schemas, allow_undeclared, semantic_validation, JSON Schema, tool allowlist
• Config/command names: tool-validation policy, declared_tools, schemas.<tool> (JSON Schema), allow_undeclared, semantic_validation.enabled, semantic_validation.endpoint, semantic_validation.model
• Best next pages: Tool Security, Tool Budget, RBAC

For engineers

• Prerequisites: The exact tool/function names your agent can call. JSON Schema definitions for each tool's expected arguments. For semantic validation: an OpenAI-compatible chat completions endpoint.
• Validation: Call an undeclared tool and verify denial. Send malformed arguments to a schema-validated tool and verify rejection. Enable semantic validation and test with structurally valid but semantically malicious arguments.
• Key commands: kt policy lint, kt gateway run, kt events tail

For leaders

• Governance: Tool validation enforces least-privilege at the tool layer — agents can only call explicitly permitted tools with validated arguments. This prevents tool abuse and limits blast radius.
• Cost: Schema validation is local and free. Semantic validation adds per-tool-call LLM cost (use a small model like gpt-4o-mini to minimize). The governance value justifies the cost for high-risk tools.
• Rollout: Start with allow_undeclared: true to monitor which tools agents call. Then lock down declared_tools and add schemas. Enable semantic_validation last for highest-risk tools.

Next steps

• Tool Security — Injection and traversal detection
• Tool Budget — Per-tool resource limits
• RBAC — Role-based tool permissions
• Agent Firewall — Broad intent-level tool blocking