Skip to main content

Gateway Response Cache

Use kt cache to inspect the response cache selected by the current KEEPTRUSTS_LLM_CACHE_* environment. The command opens the configured backend directly; it does not query a running gateway.

Command summary​

CommandResult
kt cache statsShow the selected backend and whether caching is enabled. The filesystem backend also reports entries, size, hit/miss counts, evictions, and warm-up state.
kt cache list --top 20List the most recently accessed filesystem entries.
kt cache list --top 20 --by-sizeList the largest filesystem entries.
kt cache inspect <key>Read metadata for one exact cache key.
kt cache clearConfirm interactively, then clear the selected backend.
kt cache clear --yesClear without the prompt; use only in an intentional maintenance workflow.

list is filesystem-specific. stats prints a backend summary for every backend, but detailed entry statistics are available only for the filesystem backend. An empty list on another backend is not evidence that the remote cache has no entries.

Inspect the active cache​

Run commands with the same environment used by the gateway:

export KEEPTRUSTS_LLM_CACHE_BACKEND=filesystem
export KEEPTRUSTS_LLM_CACHE_DIR="$HOME/.keeptrusts/cache"

kt cache stats
kt cache list --top 10 --by-size

The filesystem backend defaults to a 500 MB maximum when KEEPTRUSTS_LLM_CACHE_MAX_BYTES is not set. Treat the command output as the source of truth for the environment you actually invoked.

To inspect an exact key:

kt cache inspect '<exact-cache-key>'

The inspection output contains metadata such as status, body size, stored time, key version, header count, and content type when present. It does not print the cached response body. list truncates long keys for display, so a truncated row is not valid input to inspect; obtain the complete key from the approved cache diagnostic source.

Clear cache entries safely​

Clearing affects the backend selected by the command's environment:

kt cache stats
kt cache clear
kt cache stats

For remote backends, a Cache cleared. message and zero exit status mean that the CLI dispatched the clear operation; they do not prove that every remote delete succeeded. The Redis, Valkey, object-store, and Qdrant adapters can log individual clear failures without returning them to the command. Verify the result with approved backend telemetry and a representative cache miss.

Before clearing a shared Redis, Valkey, object-store, or Qdrant backend, confirm that the configured URL, bucket, prefix, collection, and credentials identify the intended environment. --yes removes the interactive guard; it does not add a dry run or restrict the operation to one gateway.

Use a cache buster for a controlled logical rollover when deleting shared entries is not the desired operation. Replace release-current with a new deployment-specific value for each rollover:

export KEEPTRUSTS_LLM_CACHE_BUSTER="release-current"

The gateway incorporates the buster into cache-key behavior. Changing it makes older entries stop matching without asserting that the old backend objects were deleted.

Configuration checklist​

The cache runtime supports memory, filesystem, Redis, Valkey, S3, GCS, and Qdrant backends. Configure the selected backend through the matching KEEPTRUSTS_LLM_CACHE_* variables, then:

  1. Run kt cache stats in the gateway environment.
  2. Start or restart the gateway with that same environment.
  3. Send a representative request twice when your policy permits caching.
  4. Inspect gateway response metadata and cache metrics rather than inferring a hit from latency alone.
  5. Re-run kt cache stats for filesystem counters.

Do not cache sensitive or regulated content unless your retention, encryption, and access policies explicitly allow it.

Troubleshooting​

SymptomCheck
stats shows an unexpected backendCompare the shell environment with the gateway service environment.
Filesystem list is emptyConfirm the directory, cache enablement, request eligibility, TTL, and cache buster.
Remote backend has no detailed statisticsThis CLI view exposes detailed entry statistics only for the filesystem backend; use the backend's approved operational telemetry.
Remote clear prints success but entries remainInspect CLI and backend logs, confirm connectivity and credentials, then verify the selected prefix, bucket, or collection. The command exit status alone is not proof of remote deletion.
A repeated request missesCheck model, request fields, route, policy eligibility, TTL, key version, and cache buster.
inspect cannot find a keyConfirm the complete, untruncated key and the same cache backend and environment.

Next steps​