Managing Policy Changes

Policy edits can affect customer traffic immediately. This workflow turns a draft policy-config.yaml into a verified canary rollout with explicit stop and rollback decisions. It does not assume that saving YAML, passing a local test, or uploading a version proves live behavior.

Before you begin

Identify the exact gateway, agent, environment, provider targets, and request families in scope.
Keep the last known-good YAML, configuration version, and digest available.
Name the rollout owner, reviewer, observer, and person authorized to roll back.
Prepare at least one expected-allow case and one expected-block or expected-redaction case. Add routing and escalation cases when those behaviors change.
Use the same provider-secret environment and secret-store access that the target gateway service will receive.
Have a gateway runtime token for KEEPTRUSTS_API_TOKEN and a separate application token for KEEPTRUSTS_REQUEST_TOKEN. Do not use a provider credential as either token.

1. Define success and rollback gates

Write the expected result before editing:

Which requests should be allowed, blocked, redacted, or escalated?
Which provider and model should receive each routed request?
What change in block rate, error rate, latency, or review volume is acceptable?
How long will the canary be observed before expansion?
Which result requires an immediate rollback, and which result only pauses expansion for investigation?

Capture baseline request IDs and event JSON from the current version with kt events tail --since <window> --json or GET /v1/events?since=<window>. Use History only when capture is enabled and the session content is necessary; Events and request IDs remain the runtime decision evidence.

2. Validate the exact draft

From the policy pack directory, run the checks in this order:

kt config validate --file policy-config.yaml --json
kt policy lint --mode runtime --file policy-config.yaml
kt policy test --pack-dir . --json
kt gateway check --config policy-config.yaml --verbose

Require every command to exit successfully. They answer different questions:

Check	What it proves	What it does not prove
`config validate`	The declarative document parses and passes structural validation	Provider credentials or live policy outcomes
`policy lint`	The file is a valid runtime policy document without reported lint errors	That its expected allows and blocks are correct
`policy test`	The JSON golden tests and inline `testing.suites[]` cases in this pack match their declared outcomes	Live provider routing or application connectivity
`gateway check`	The config resolves with the current environment, provider credentials, routing, and policy chain	A provider inference, deployment, or real application request

Run gateway check with the target service's environment. A check from an interactive shell can pass while an installed service is not ready because the service receives different variables or secret-store access.

3. Exercise representative traffic

Start a non-production or canary gateway bound to a real agent:

export KEEPTRUSTS_API_URL="https://api.keeptrusts.com"
export KEEPTRUSTS_API_TOKEN="kt_your_gateway_runtime_token"

kt gateway run \
  --agent policy-canary \
  --policy-config policy-config.yaml \
  --listen 127.0.0.1:41002

In another terminal, use an application token and a model declared in the draft. Replace the example request ID with a unique 32-character lowercase hexadecimal value for each case:

export KEEPTRUSTS_REQUEST_TOKEN="kt_your_application_token"
export KEEPTRUSTS_TEST_MODEL="gpt-5.4-mini"
export KEEPTRUSTS_TEST_REQUEST_ID="0123456789abcdef0123456789abcdef"

curl -i http://127.0.0.1:41002/v1/chat/completions \
  -H "Authorization: Bearer ${KEEPTRUSTS_REQUEST_TOKEN}" \
  -H "X-Request-Id: ${KEEPTRUSTS_TEST_REQUEST_ID}" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"${KEEPTRUSTS_TEST_MODEL}\",
    \"messages\": [
      {\"role\": \"user\", \"content\": \"Run the reviewed policy canary case.\"}
    ]
  }"

Preserve the response status, body, and X-Request-Id header. Then query the affected region and match the normalized request ID:

kt events tail --since 10m --json

For each case, verify the Event's final verdict and reason code, configuration version, gateway or agent attribution, and provider and model when an upstream call occurred. Also test an invalid application token and confirm it is rejected before an upstream provider call.

If no matching Event arrives, stop the rollout check and verify region selection, gateway control-plane connectivity, and evidence support for the request family. An empty History view is not proof that the request did not occur.

4. Review the rollout gate

Do not deploy until all of these are true:

The draft is narrower than or equal to the approved scope.
Expected-allow, expected-block, redaction, routing, and escalation cases pass where applicable.
The provider metadata used for retention or route selection matches the intended posture.
A reviewer has approved the version, digest, target gateways, observation window, and rollback criteria.
The last known-good config is retrievable and the rollback owner is present.
The team can observe Gateways, runtime Events through kt events or the Events API, affected Inbox queues, and application health during the window.

Roll out to one representative canary before expanding. Saving a version in Configurations is not deployment, and a completed upload is not proof that the runtime applied the reviewed content.

5. Verify the deployed runtime

kt gateway check reports the reviewed local version and digest. Copy those values and require the running canary to match them:

export EXPECTED_CONFIG_VERSION="reviewed-version"
export EXPECTED_CONFIG_SHA256="reviewed-sha256"

kt gateway config \
  --gateway-url http://127.0.0.1:41002 \
  --expect-version "${EXPECTED_CONFIG_VERSION}" \
  --expect-sha256 "${EXPECTED_CONFIG_SHA256}"

Add --api-token when the runtime protects its admin endpoint. Then repeat the representative request cases and Event correlation against the deployed canary. Compare verdicts, routing, errors, latency, and human-review volume with the baseline for the full observation window before adding targets.

Use each surface for its owning proof:

Gateways: runtime health and applied version.
kt events or GET /v1/events with required since, plus request IDs: request-time decisions and routing.
Inbox: human-review volume and outcomes.
Trail: audit chronology for the rollout and follow-up actions.
History: captured session context only when capture is enabled.

6. Pause or roll back explicitly

Pause expansion when the active version or digest cannot be verified, evidence is missing, or an operational metric is outside its review band. Investigate before deciding whether the new policy caused the problem.

Roll back when you confirm any pre-agreed trigger, such as:

an unsafe request is allowed or a required business flow is blocked;
routing selects the wrong provider, model, or retention posture;
the canary develops sustained errors or latency beyond the approved limit;
escalation or human-review volume exceeds the staffed capacity; or
the deployed version or digest differs from the approved artifact.

Take these actions:

Stop adding targets and preserve the affected request IDs, event JSON, active version, and digest.
Restore the last known-good YAML through the same reviewed deployment workflow. In Configurations, save it as a new recovery version and deploy only to the affected targets.
For a named local gateway previously changed with kt gateway reload, use its stored rollback target:
```
kt gateway revert \
  --name keeptrusts-proxy \
  --gateway-url http://127.0.0.1:41002
```
Re-run kt gateway config with the known-good version and digest, then send the representative requests again and confirm them through the same kt events or Events API query.
Record the trigger, decision, restored version, request IDs, and verification evidence in the change record.

There is no universal automatic policy rollback. kt gateway reload attempts to restore the previous config only when its immediate activation verification fails and a rollback target is available. A regression discovered after a successful reload, a console rollout, or a multi-gateway deployment still requires the explicit recovery workflow above.

Before you begin​

1. Define success and rollback gates​

2. Validate the exact draft​

3. Exercise representative traffic​

4. Review the rollout gate​

5. Verify the deployed runtime​

6. Pause or roll back explicitly​

Next steps​