Skip to main content

Bias Monitor

The bias-monitor policy runs in the output phase. threshold is the only field that changes current runtime behavior; the schema also accepts action and protected_characteristics, but the evaluator does not use them.

Phase and verdicts​

  • Phase: output
  • Possible verdicts: allow, escalate

Configuration​

pack:
  name: bias-monitor-example
  version: 1.0.0
  enabled: true
policies:
  chain:
    - bias-monitor
policy:
  bias-monitor:
    threshold: 0.85

Supported fields​

FieldTypeDefaultNotes
thresholdnumber0.85The gateway escalates when the computed bias score is greater than or equal to this value.
protected_characteristicsstring[][]Accepted by the schema but not used by the current heuristic.
actionblock | escalateescalateAccepted by the schema; the current evaluator always escalates a hit and never blocks.

How it works​

The policy uses a narrow heuristic:

  1. It inspects the request for HR-oriented context such as hire, promotion, or performance review.
  2. It inspects the response for protected-characteristic language (race, gender, religion, disability, age) and explicit age references.
  3. When both the HR context and the protected-characteristic signal are present, it assigns a score of 0.95; otherwise it assigns 0.0.
  4. If the score meets or exceeds threshold, the policy returns escalate; otherwise it returns allow.

Behavior notes​

  • The policy always escalates when triggered; it does not block.
  • protected_characteristics is not used by this policy today.
  • action is not used by this policy today.
  • The supported heuristic is currently HR-focused rather than a general-purpose fairness classifier.

Example scenario​

policy:
  bias-monitor:
    threshold: 0.90

With this config, a hiring-related prompt and a response containing protected-characteristic language can escalate; unrelated outputs continue to allow.

Best practices​

  • Use human-oversight after bias-monitor if your escalation flow requires reviewer assignment.
  • Keep expectations narrow: the current logic is best suited to HR-style outputs rather than universal fairness scoring.
  • Validate threshold choices with representative hiring or promotion prompts before relying on escalation volume.

Next steps​