Backup & Recovery Procedures

The Keeptrusts platform stores governance events, configuration, user accounts, and audit data in PostgreSQL. Gateway policy configs can be version-controlled in Git. This guide covers backup strategies for each data tier, recovery procedures, and RTO/RPO planning.

Use this page when

• You need to implement backup strategies for Keeptrusts PostgreSQL data (events, configs, users, audit log).
• You are planning RTO/RPO targets for your Keeptrusts deployment.
• You need recovery procedures for database restoration, point-in-time recovery, or disaster recovery.
• You want to set up automated backup verification and retention policies.

Primary audience

• Primary: Technical Engineers
• Secondary: AI Agents, Technical Leaders

Data Tiers

Data Tier	Storage	Criticality	Backup Method
Configuration	PostgreSQL + Git	High	pg_dump + Git
User/team/auth	PostgreSQL	High	pg_dump
Governance events	PostgreSQL	Medium	pg_dump + event export
Audit log	PostgreSQL	High (compliance)	pg_dump
Export artifacts	Local/S3	Low	S3 replication or rsync
Policy configs	YAML files	High	Git version control

PostgreSQL Backup

Full Backup with pg_dump

#!/usr/bin/env bash
# backup-keeptrusts-db.sh
set -euo pipefail

BACKUP_DIR="/var/backups/keeptrusts"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_HOST="db.internal"
DB_NAME="keeptrusts"
DB_USER="keeptrusts"
RETENTION_DAYS=30

mkdir -p "$BACKUP_DIR"

# Custom format — supports parallel restore
pg_dump -h "$DB_HOST" -U "$DB_USER" -d "$DB_NAME" \
  --format=custom \
  --compress=9 \
  --file="$BACKUP_DIR/keeptrusts_${TIMESTAMP}.dump"

# Verify backup
pg_restore --list "$BACKUP_DIR/keeptrusts_${TIMESTAMP}.dump" > /dev/null 2>&1
echo "Backup verified: keeptrusts_${TIMESTAMP}.dump"

# Prune old backups
find "$BACKUP_DIR" -name "keeptrusts_*.dump" -mtime +"$RETENTION_DAYS" -delete

Scheduled Backups with cron

# /etc/cron.d/keeptrusts-backup
# Daily full backup at 02:00 UTC
0 2 * * * postgres /usr/local/bin/backup-keeptrusts-db.sh >> /var/log/keeptrusts-backup.log 2>&1

# Hourly WAL archiving checkpoint
0 * * * * postgres psql -c "SELECT pg_switch_wal();" >> /var/log/keeptrusts-wal.log 2>&1

Continuous Archiving (WAL)

For point-in-time recovery (PITR):

# postgresql.conf
wal_level = replica
archive_mode = on
archive_command = 'cp %p /var/backups/keeptrusts/wal/%f'
archive_timeout = 300

# Verify WAL archiving is working
psql -c "SELECT * FROM pg_stat_archiver;"

Docker-Based Backup

When PostgreSQL runs in Docker:

# Backup from Docker container
docker exec keeptrusts-postgres pg_dump \
  -U keeptrusts -d keeptrusts \
  --format=custom --compress=9 \
  > "/var/backups/keeptrusts/keeptrusts_$(date +%Y%m%d_%H%M%S).dump"

# Or use docker compose
docker compose exec postgres pg_dump \
  -U keeptrusts -d keeptrusts \
  --format=custom --compress=9 \
  > backup.dump

Event Data Export

Bulk Event Export via API

Export governance events for long-term archival:

# Create an export job via the API
curl -X POST http://api.internal:8080/v1/exports \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "events",
    "format": "csv",
    "filters": {
      "from": "2025-01-01T00:00:00Z",
      "to": "2025-12-31T23:59:59Z"
    }
  }'

# Check export job status
curl http://api.internal:8080/v1/exports/{job_id} \
  -H "Authorization: Bearer $API_TOKEN"

# Download completed export
curl -O http://api.internal:8080/v1/exports/{job_id}/artifact \
  -H "Authorization: Bearer $API_TOKEN"

Automated Export Script

#!/usr/bin/env bash
# export-events-monthly.sh
set -euo pipefail

API_URL="http://api.internal:8080"
EXPORT_DIR="/var/backups/keeptrusts/events"
MONTH=$(date -d "last month" +%Y-%m)
FROM="${MONTH}-01T00:00:00Z"
TO=$(date -d "$FROM +1 month" +%Y-%m-01T00:00:00Z)

mkdir -p "$EXPORT_DIR"

JOB_ID=$(curl -s -X POST "$API_URL/v1/exports" \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"type\":\"events\",\"format\":\"csv\",\"filters\":{\"from\":\"$FROM\",\"to\":\"$TO\"}}" \
  | jq -r '.id')

echo "Export job created: $JOB_ID"

# Poll for completion
while true; do
  STATUS=$(curl -s "$API_URL/v1/exports/$JOB_ID" \
    -H "Authorization: Bearer $API_TOKEN" | jq -r '.status')
  if [[ "$STATUS" == "completed" ]]; then break; fi
  if [[ "$STATUS" == "failed" ]]; then echo "Export failed" >&2; exit 1; fi
  sleep 10
done

curl -o "$EXPORT_DIR/events_${MONTH}.csv" \
  "$API_URL/v1/exports/$JOB_ID/artifact" \
  -H "Authorization: Bearer $API_TOKEN"

echo "Events exported: $EXPORT_DIR/events_${MONTH}.csv"

Configuration Backup via Git

Git-Linked Configuration

Keeptrusts supports Git-backed policy configurations. When enabled, the API syncs configurations from a linked Git repository:

# Link a Git repository for config backup/sync
curl -X POST http://api.internal:8080/v1/git-repos \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://github.com/org/ai-governance-configs.git",
    "branch": "main",
    "path": "policies/",
    "auto_create_configuration": true
  }'

Manual Config Export

# Export current gateway configuration
kt config export --output policy-config-backup.yaml

# Verify exported config
kt policy lint --file policy-config-backup.yaml

Config Versioning Strategy

ai-governance-configs/
├── production/
│   ├── gateway-primary.yaml
│   └── gateway-secondary.yaml
├── staging/
│   └── gateway.yaml
└── README.md

Commit every configuration change to Git for audit trail and rollback capability.

Recovery Procedures

Full Database Restore

# Stop the API server first
systemctl stop keeptrusts-api

# Restore from custom-format dump
pg_restore -h db.internal -U keeptrusts -d keeptrusts \
  --clean --if-exists \
  /var/backups/keeptrusts/keeptrusts_20250420_020000.dump

# Restart API — migrations auto-apply
systemctl start keeptrusts-api

Point-in-Time Recovery

# 1. Stop PostgreSQL
systemctl stop postgresql

# 2. Clear data directory
rm -rf /var/lib/postgresql/16/main/*

# 3. Restore base backup
pg_basebackup -h backup-server -D /var/lib/postgresql/16/main/

# 4. Configure recovery target
cat > /var/lib/postgresql/16/main/recovery.signal << EOF
EOF

cat >> /var/lib/postgresql/16/main/postgresql.auto.conf << EOF
restore_command = 'cp /var/backups/keeptrusts/wal/%f %p'
recovery_target_time = '2025-04-20 14:30:00 UTC'
recovery_target_action = 'promote'
EOF

# 5. Start PostgreSQL — it replays WAL to the target time
systemctl start postgresql

Gateway Recovery

The gateway is stateless — recovery means redeploying with the correct configuration:

# Pull latest configuration from Git
git clone https://github.com/org/ai-governance-configs.git
cd ai-governance-configs

# Start gateway with recovered config
kt gateway run --policy-config production/gateway-primary.yaml

Docker Compose Recovery

# Restore database backup into Docker volume
docker compose down
docker volume rm keeptrusts_pgdata

docker compose up -d postgres
docker compose exec -T postgres pg_restore \
  -U keeptrusts -d keeptrusts --clean --if-exists \
  < /var/backups/keeptrusts/keeptrusts_latest.dump

docker compose up -d

RTO/RPO Planning

Scenario	RPO	RTO	Strategy
Daily backup, no WAL	24 hours	1–2 hours	Scheduled pg_dump
WAL archiving + daily base	Minutes	30 min–1 hour	PITR with WAL replay
Streaming replication	Seconds	Minutes	Hot standby failover
Multi-region replication	Seconds	Minutes	Cross-region standby

Recommended Setup by Scale

Scale	Backup Strategy	Recovery Strategy
Small (< 100 users)	Daily pg_dump + Git configs	Restore from dump
Medium (100–1000 users)	WAL archiving + daily base	PITR
Large (> 1000 users)	Streaming replication	Automated failover

Backup Verification

#!/usr/bin/env bash
# verify-backup.sh — monthly backup restoration test
set -euo pipefail

LATEST_BACKUP=$(ls -t /var/backups/keeptrusts/keeptrusts_*.dump | head -1)

# Restore to a temporary database
createdb -h db.internal -U keeptrusts keeptrusts_verify
pg_restore -h db.internal -U keeptrusts -d keeptrusts_verify "$LATEST_BACKUP"

# Run basic integrity checks
psql -h db.internal -U keeptrusts -d keeptrusts_verify -c \
  "SELECT count(*) FROM events; SELECT count(*) FROM configurations;"

# Clean up
dropdb -h db.internal -U keeptrusts keeptrusts_verify

echo "Backup verification passed: $LATEST_BACKUP"

Next steps

• Security Hardening — secure your backup storage
• Upgrade & Maintenance — pre-upgrade backup checklist
• Monitoring Infrastructure — alert on backup failures

For AI systems

• Canonical terms: Keeptrusts backup, PostgreSQL pg_dump, disaster recovery, RTO/RPO planning, event data export, config backup, point-in-time recovery, WAL archiving.
• Key config/commands: pg_dump -h $DB_HOST -U keeptrusts -d keeptrusts --format=custom --compress=9; pg_restore for recovery; kt events export for event data backup; Git-based policy config versioning; backup verification script with createdb + pg_restore + integrity checks.
• Best next pages: Security Hardening, Upgrade & Maintenance, Monitoring Infrastructure.

For engineers

• Prerequisites: pg_dump / pg_restore access to Keeptrusts PostgreSQL; backup storage with appropriate retention; cron for scheduled backups.
• Configure daily pg_dump with custom format (supports parallel restore), 30-day retention, and monthly verification via restore-to-temporary-database.
• Validate with: pg_restore --list backup.dump > /dev/null 2>&1 for backup integrity; monthly restore test to keeptrusts_verify database; SELECT count(*) FROM events; after restore.
• Policy configs should also be Git-versioned as a secondary recovery path independent of database backups.

For leaders

• Governance audit data (events, escalations) is compliance-critical — loss could mean inability to satisfy regulatory examination requests.
• Define RTO/RPO based on compliance requirements: audit data may require RPO < 1 hour; configuration can tolerate RPO = last Git commit.
• Backup verification testing (monthly restore) prevents discovering backup failures during an actual incident.
• Storage costs scale with event volume and retention period — plan 100+ GB for high-throughput deployments.