Setting Up Cache Warmers

Cache warmers pre-generate engineering cache entries so agents receive instant cache hits instead of waiting for on-demand computation. You deploy warmers as dedicated worker binaries that run independently from the main API, providing performance isolation and horizontal scalability.

Use this page when

• You are deploying the worker_cache_warmer binary for the first time (Docker Compose or Kubernetes).
• You need to verify warmer health, monitor job lifecycle, or troubleshoot stalled warming jobs.
• You want to understand the architecture: how triggers enqueue jobs, workers claim them, and artifacts reach the cache store.

Primary audience

• Primary: AI Agents, Technical Engineers
• Secondary: Technical Leaders

Architecture Overview

The worker_cache_warmer binary runs as a separate process from the main Keeptrusts API. This separation ensures that intensive cache generation tasks (embedding computation, dependency graph analysis, symbol indexing) do not compete with API request handling for CPU and memory resources.

┌─────────────────┐     ┌──────────────────────┐     ┌─────────────┐
│  Warmer Trigger │────▶│  Warmer Job Queue     │────▶│  worker_    │
│  (commit, miss) │     │  (PostgreSQL)         │     │  cache_     │
└─────────────────┘     └──────────────────────┘     │  warmer     │
                                                      └──────┬──────┘
                                                             │
                                                      ┌──────▼──────┐
                                                      │  Cache Store │
                                                      └─────────────┘

Deploying the Warmer Binary

Docker Compose

Add the warmer service to your Docker Compose configuration:

services:
  keeptrusts-cache-warmer:
    image: keeptrusts/api:latest
    command: ["/usr/local/bin/worker_cache_warmer"]
    environment:
      DATABASE_URL: "postgres://keeptrusts:keeptrusts@postgres:5432/keeptrusts"
      KEEPTRUSTS_CACHE_WARMER_CONCURRENCY: "4"
      KEEPTRUSTS_CACHE_WARMER_POLL_INTERVAL_SECS: "5"
    depends_on:
      - postgres
    deploy:
      replicas: 1

Kubernetes

Deploy as a separate Deployment resource:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keeptrusts-cache-warmer
spec:
  replicas: 2
  selector:
    matchLabels:
      app: keeptrusts-cache-warmer
  template:
    spec:
      containers:
        - name: warmer
          image: keeptrusts/api:latest
          command: ["/usr/local/bin/worker_cache_warmer"]
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: keeptrusts-db
                  key: url
            - name: KEEPTRUSTS_CACHE_WARMER_CONCURRENCY
              value: "4"

Verifying the Warmer Is Running

Check Logs

The warmer emits structured log entries on startup and during job processing:

# Docker Compose
docker compose logs keeptrusts-cache-warmer --tail=20

# Expected output
# INFO worker_cache_warmer: starting cache warmer worker concurrency=4
# INFO worker_cache_warmer: polling for warmer jobs
# INFO worker_cache_warmer: claimed job job_id=abc123 artifact_type=repo_map

Check the Console

Navigate to Settings → Engineering Cache → Warmers to see:

• Active warmer instances and their last heartbeat.
• Current job queue depth.
• Jobs in progress with elapsed time.

Health Endpoint

The warmer exposes a health endpoint on port 8081 (configurable via KEEPTRUSTS_CACHE_WARMER_HEALTH_PORT):

curl http://localhost:8081/health
# {"status":"healthy","active_jobs":2,"queue_depth":15}

Warmer Job Lifecycle

Every warmer job progresses through a defined lifecycle:

1. Queued

A trigger (repo connection, commit, or repeated miss) creates a warmer job record in the database with status queued. The job includes the target repository, artifact type, and any relevant parameters.

2. Claimed

A warmer worker polls for queued jobs and claims one using an advisory lock. The status transitions to claimed and the worker ID is recorded. Other workers skip claimed jobs.

3. Running

The worker begins generating the cache artifact. Status transitions to running. The worker periodically updates a heartbeat timestamp. If the heartbeat goes stale (worker crash), the job is released back to queued for retry.

4. Completed

The worker successfully generates the artifact and writes it to the cache store. Status transitions to completed. The job record is retained for audit purposes.

5. Failed

If generation fails after retries, the job transitions to failed with an error message. Failed jobs appear in the console for investigation. The warmer does not retry failed jobs automatically — you can manually re-queue them from the console.

queued → claimed → running → completed
                          └→ failed

Console UI for Warmer Status

The warmer status page in the console provides real-time visibility:

Queue Overview

• Total queued: Number of jobs waiting for a worker.
• In progress: Number of jobs currently being processed.
• Completed (24h): Jobs successfully completed in the last 24 hours.
• Failed (24h): Jobs that failed in the last 24 hours.

Job Details

Click any job to see:

• Repository and branch targeted.
• Artifact type being generated.
• Trigger source (connect, commit, or miss).
• Time spent in each lifecycle phase.
• Error details for failed jobs.

Warmer Instances

View all active warmer processes:

• Instance ID and hostname.
• Last heartbeat timestamp.
• Current concurrency utilization.
• Jobs processed since startup.

Configuration Reference

Variable	Default	Description
KEEPTRUSTS_CACHE_WARMER_CONCURRENCY	4	Max concurrent jobs per worker
KEEPTRUSTS_CACHE_WARMER_POLL_INTERVAL_SECS	5	Seconds between queue polls
KEEPTRUSTS_CACHE_WARMER_HEARTBEAT_SECS	30	Heartbeat interval for running jobs
KEEPTRUSTS_CACHE_WARMER_STALE_THRESHOLD_SECS	120	Time before a stale job is reclaimed
KEEPTRUSTS_CACHE_WARMER_HEALTH_PORT	8081	Health endpoint port
KEEPTRUSTS_CACHE_WARMER_MAX_RETRIES	3	Max retries before marking failed

Scaling Considerations

You can run multiple warmer instances for horizontal scaling. Each instance independently polls the job queue and claims work. The advisory lock mechanism prevents duplicate processing. See Tuning Cache Warmer Concurrency for detailed scaling guidance.

Next steps

• Warmer Triggers: Connect, Commit, and Miss — Understand what enqueues warmer jobs.
• Tuning Cache Warmer Concurrency — Optimize throughput for your workload.
• Cache Invalidation Strategies — Manage entry freshness alongside warming.

For AI systems

• Canonical terms: Keeptrusts, worker_cache_warmer, cache warmer, warmer job queue, advisory lock, job lifecycle (queued → claimed → running → completed/failed).
• Config keys: KEEPTRUSTS_CACHE_WARMER_CONCURRENCY, KEEPTRUSTS_CACHE_WARMER_POLL_INTERVAL_SECS, KEEPTRUSTS_CACHE_WARMER_HEARTBEAT_SECS, KEEPTRUSTS_CACHE_WARMER_STALE_THRESHOLD_SECS, KEEPTRUSTS_CACHE_WARMER_HEALTH_PORT, KEEPTRUSTS_CACHE_WARMER_MAX_RETRIES.
• Best next pages: Warmer Triggers, Tuning Cache Warmer Concurrency, Cache Invalidation Strategies.

For engineers

• Deploy as a separate process from the main API using Docker Compose (replicas: 1) or Kubernetes Deployment.
• The binary is /usr/local/bin/worker_cache_warmer in the keeptrusts/api image.
• Health endpoint: GET http://localhost:8081/health returns {"status":"healthy","active_jobs":N,"queue_depth":M}.
• Check logs for INFO worker_cache_warmer: starting cache warmer worker concurrency=4 on startup.
• Monitor in console: Settings → Engineering Cache → Warmers shows instances, queue depth, job details.
• Failed jobs do not auto-retry — investigate and manually re-queue from the console.

For leaders

• Cache warmers ensure agents get instant cache hits instead of cold-start latency on first interactions.
• Runs independently from the API — intensive generation tasks cannot degrade API request handling.
• Horizontally scalable: add more warmer instances to reduce queue drain time during repository onboarding.
• A typical medium-sized repo (50K–200K LOC) completes all initial warming within 5–15 minutes.