Backup strategy for self-hosted HelloJohn — PostgreSQL dumps, point-in-time restore, and disaster recovery procedures.

Backup & Recovery

HelloJohn's state lives entirely in PostgreSQL (and optionally Redis). This page covers how to back up, restore, and recover from failure.

What Needs to Be Backed Up

Component	Contains	Backup Required
PostgreSQL	All users, sessions, orgs, MFA factors, config	✅ Yes — critical
Redis	Rate limit counters, session cache	⬜ No — ephemeral, rebuilds automatically
Signing keys	Ed25519 private key (`HELLOJOHN_JWT_SIGNING_KEY`)	✅ Yes — store in secrets manager
Environment variables	All configuration	✅ Yes — store in secrets manager

Redis data is ephemeral — losing it only causes users to be rate-limited as if they're new visitors. It does not affect authentication or session validity.

PostgreSQL Backup

Manual Dump

# Full database dump (compressed)
pg_dump \
  --dbname="postgresql://user:password@host:5432/hellojohn" \
  --format=custom \
  --compress=9 \
  --file="hellojohn_$(date +%Y%m%d_%H%M%S).dump"

Restore from Dump

# Create target database first
createdb --host=host --username=user hellojohn_restore

# Restore
pg_restore \
  --dbname="postgresql://user:password@host:5432/hellojohn_restore" \
  --verbose \
  hellojohn_20240120_103000.dump

Automated Daily Backup (cron)

# /etc/cron.d/hellojohn-backup
0 2 * * * postgres pg_dump \
  --dbname="$DATABASE_URL" \
  --format=custom \
  --compress=9 \
  --file="/backups/hellojohn_$(date +\%Y\%m\%d).dump" \
  && find /backups -name "hellojohn_*.dump" -mtime +30 -delete

This runs at 2 AM daily and retains 30 days of backups.

Docker Compose Backup

If running with Docker Compose, exec into the PostgreSQL container:

# Backup
docker compose exec postgres pg_dump \
  -U hellojohn \
  -d hellojohn \
  --format=custom \
  --compress=9 \
  > ./backups/hellojohn_$(date +%Y%m%d).dump

# Restore
docker compose exec -T postgres pg_restore \
  -U hellojohn \
  -d hellojohn \
  --clean \
  < ./backups/hellojohn_20240120.dump

Managed Database Backups

Most managed PostgreSQL providers include automatic backups:

Neon

Automatic backups every 24 hours
Point-in-time restore (PITR) available on paid plans
Restore via Neon console → Branches → Restore

Supabase

Daily backups on Pro plan and above
PITR available on Team plan
Restore via Dashboard → Database → Backups

Amazon RDS

# Create manual snapshot
aws rds create-db-snapshot \
  --db-instance-identifier hellojohn-db \
  --db-snapshot-identifier hellojohn-$(date +%Y%m%d)

Enable automated backups:

aws rds modify-db-instance \
  --db-instance-identifier hellojohn-db \
  --backup-retention-period 30 \
  --preferred-backup-window "02:00-03:00"

Point-in-Time Recovery (PITR)

For self-managed PostgreSQL, enable WAL archiving for PITR:

# postgresql.conf
wal_level = replica
archive_mode = on
archive_command = 'aws s3 cp %p s3://your-bucket/wal/%f'
restore_command = 'aws s3 cp s3://your-bucket/wal/%f %p'

Restore to a specific timestamp:

# In recovery.conf (PostgreSQL < 12) or postgresql.conf (PostgreSQL 12+)
recovery_target_time = '2024-01-20 10:30:00 UTC'
recovery_target_action = 'promote'

Backup Verification

Regularly verify backups are restorable:

#!/bin/bash
# verify-backup.sh — run weekly

BACKUP_FILE="hellojohn_$(date +%Y%m%d).dump"
TEST_DB="hellojohn_verify_$(date +%s)"

echo "Creating test database: $TEST_DB"
createdb "$TEST_DB"

echo "Restoring backup..."
pg_restore \
  --dbname="postgresql://user:password@host:5432/$TEST_DB" \
  "$BACKUP_FILE"

echo "Verifying row counts..."
psql "postgresql://user:password@host:5432/$TEST_DB" \
  -c "SELECT COUNT(*) AS users FROM hj_user;"
psql "postgresql://user:password@host:5432/$TEST_DB" \
  -c "SELECT COUNT(*) AS sessions FROM hj_session;"

echo "Dropping test database..."
dropdb "$TEST_DB"

echo "Backup verification complete ✅"

Signing Key Backup

Your Ed25519 signing key must be backed up securely — if lost, all existing JWTs become unverifiable.

Store it in a secrets manager:

# AWS Secrets Manager
aws secretsmanager create-secret \
  --name hellojohn/jwt-signing-key \
  --secret-string "$(cat signing_key.pem)"

# Retrieve
aws secretsmanager get-secret-value \
  --secret-id hellojohn/jwt-signing-key \
  --query SecretString \
  --output text

Never store signing keys in:

Git repositories
Plain .env files on the server without encryption
Unencrypted S3 buckets

Disaster Recovery Procedure

Scenario: Database host failure

Provision new PostgreSQL instance

Restore from latest backup:

pg_restore --dbname="$NEW_DATABASE_URL" hellojohn_latest.dump

Update HELLOJOHN_DATABASE_URL environment variable
Restart HelloJohn instances
Verify health: curl https://auth.example.com/health

RPO (Recovery Point Objective): Time since last backup (with daily backups: up to 24h of data loss) RTO (Recovery Time Objective): ~15-30 minutes (restore + restart)

For RPO < 1 minute, use managed PITR (Neon, Supabase, RDS) or PostgreSQL streaming replication.

Scenario: Signing key lost

If the signing key is lost, all existing sessions become invalid:

Generate a new Ed25519 key pair
Deploy new key in HELLOJOHN_JWT_SIGNING_KEY
Restart HelloJohn
All users will be prompted to sign in again (sessions expire naturally)

This is why signing keys must be stored in a secrets manager.

Recommended Backup Policy

Retention	Frequency	Storage
7 daily backups	Every 24h	Local + S3
4 weekly backups	Every Sunday	S3 (different region)
3 monthly backups	1st of month	S3 Glacier

Backup & Recovery

On this page