From 8acca41ca17be29247aa4e044a9c2a7d7850def6 Mon Sep 17 00:00:00 2001 From: weeihan Date: Fri, 24 Jul 2026 06:35:15 +0800 Subject: [PATCH] =?UTF-8?q?docs:=20phase=209=20=E2=80=94=20update=20vps-cr?= =?UTF-8?q?on.md,=20add=20cutover=20runbook?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Sonnet 4.6 --- docs/phase-9-cutover.md | 86 +++++++++++++++++++++++++++++++++++++++++ docs/vps-cron.md | 61 ++++++++++++++++++++++++----- 2 files changed, 138 insertions(+), 9 deletions(-) create mode 100644 docs/phase-9-cutover.md diff --git a/docs/phase-9-cutover.md b/docs/phase-9-cutover.md new file mode 100644 index 0000000..14b9ea5 --- /dev/null +++ b/docs/phase-9-cutover.md @@ -0,0 +1,86 @@ +# Phase 9 — Cutover Runbook + +## Pre-flight checklist (before cutting traffic) + +Run from the project root on your local machine. + +- [ ] `bash db/01-load-schema.sh` — schema + RLS loaded on new server (14 tables) +- [ ] `SUPABASE_DB_URL="..." bash db/02-dump-and-restore.sh` — all public data restored +- [ ] `SUPABASE_DB_URL="..." bash db/03-migrate-passwords.sh` — passwords migrated, 0 NULL rows +- [ ] `SUPABASE_URL="..." SUPABASE_SERVICE_ROLE_KEY="..." bash db/04-migrate-evidence.sh` — evidence files on server +- [ ] `SUPABASE_DB_URL="..." bash db/05-verify.sh` — row counts match, embeddings present +- [ ] `/var/www/ims/.env` populated on server (see `docs/new-server-setup.md` section I) +- [ ] `bash deploy.sh` — Next.js standalone build deployed, `systemctl status ims` shows Active +- [ ] nginx config live at `ims.setia.com.my` (see `docs/new-server-setup.md` sections E–F) +- [ ] TLS cert issued (`sudo certbot --nginx -d ims.setia.com.my`) +- [ ] `curl -I https://ims.setia.com.my/` returns HTTP/2 200 +- [ ] Cron jobs registered (see `docs/vps-cron.md`) +- [ ] Daily backup cron registered (see `docs/new-server-setup.md` section K) + +## DNS cutover + +Point `ims.setia.com.my` A record to the new server's public IP. +TTL: set to 60s before cutover, restore to 3600s after. + +## Smoke tests (run immediately after cutover) + +Test as each role. All must pass before telling users to switch. + +### Reporter role +- [ ] Log in with a migrated user's real password — succeeds +- [ ] Submit new incident with photo evidence — reference number generated, evidence stored +- [ ] Open own incident detail — evidence renders (HMAC-signed URL serves file) + +### Supervisor role +- [ ] Log in +- [ ] View site incidents list — all historical incidents present +- [ ] Triage an incident — triage saved, audit_log row written with correct `changed_by` + +### HSE role +- [ ] Log in +- [ ] Run AI triage suggestion — response received (Anthropic key working) +- [ ] Run Similar Incidents — embeddings returning results +- [ ] Open dashboard — stats render, JKKP8 export downloads CSV +- [ ] AI risk flags panel loads + +### Admin role +- [ ] Log in +- [ ] Create new user — user created, no Supabase call +- [ ] Delete user — user removed (auth + profile) + +### CAPA owner role +- [ ] Log in +- [ ] View assigned CAPAs +- [ ] Mark a CAPA in-progress — status updates + +### Password reset flow +- [ ] Request reset for own email — Brevo email arrives +- [ ] Complete reset — new password accepted + +### Cron (manual trigger) +- [ ] CAPA escalation endpoint returns `{"ok":true}` +- [ ] Effectiveness recheck endpoint returns `{"ok":true}` + +## RLS spot-check (security) + +```bash +# As app_user role with a reporter's user_id set: +sudo -u postgres psql -d ims +SET app.user_id = ''; +SELECT * FROM incidents WHERE reported_by != ''; +-- Expected: 0 rows (RLS blocks cross-user read) +``` + +## Post-cutover + +- [ ] Monitor `/var/log/ims-cron.log` for first cron run (08:00 MYT next day) +- [ ] Monitor `sudo journalctl -u ims -f` for runtime errors +- [ ] After 48h stable: take snapshot of new server DB (`pg_dump ims > /var/lib/ims/backups/pre-supabase-decommission.sql`) +- [ ] Decommission Supabase project only after snapshot confirmed + +## Rollback plan + +If cutover fails within the first hour: +1. Repoint DNS A record back to the old VPS IP (`64.176.82.100`) +2. Supabase project remains live throughout — no data risk +3. Investigate failure before re-attempting diff --git a/docs/vps-cron.md b/docs/vps-cron.md index 4f7dc70..1878e1d 100644 --- a/docs/vps-cron.md +++ b/docs/vps-cron.md @@ -1,22 +1,65 @@ -# VPS Cron Jobs +# VPS Cron Jobs — ims.setia.com.my -## CAPA escalation +App binds to `127.0.0.1:3003`. All cron calls hit that directly (no nginx proxy overhead). +`CRON_SECRET` is in `/var/www/ims/.env`. -Runs daily at 00:00 UTC (08:00 MYT). Checks all open CAPAs against 4 thresholds -(`warning_3d`, `due_today`, `overdue_3d`, `overdue_7d`) and sends email via Resend. +## CAPA Overdue Escalation + +Runs daily at 08:00 MYT (00:00 UTC). Checks all open CAPAs against 4 thresholds +(`warning_3d`, `due_today`, `overdue_3d`, `overdue_7d`) and sends email via Brevo. Each threshold fires once per CAPA, tracked in `notifications_log.status`. -**Install (SSH into VPS):** +**Install (SSH into server, then):** ```bash crontab -e -# Add: -0 0 * * * curl -s -H "Authorization: Bearer $(grep CRON_SECRET /ims/.env.local | cut -d= -f2)" http://localhost:3000/ims/api/cron/capa-escalation >> /var/log/ims-cron.log 2>&1 +``` +Add: +``` +0 0 * * * CRON_SECRET=$(grep ^CRON_SECRET /var/www/ims/.env | cut -d= -f2) curl -s -H "Authorization: Bearer $CRON_SECRET" http://127.0.0.1:3003/api/cron/capa-escalation >> /var/log/ims-cron.log 2>&1 ``` **Manual test:** ```bash -curl -s -H "Authorization: Bearer $CRON_SECRET" http://localhost:3000/ims/api/cron/capa-escalation +CRON_SECRET=$(grep ^CRON_SECRET /var/www/ims/.env | cut -d= -f2) +curl -s -H "Authorization: Bearer $CRON_SECRET" http://127.0.0.1:3003/api/cron/capa-escalation # Expected: {"ok":true,"notified":N} ``` -**Log:** `/var/log/ims-cron.log` +--- + +## CAPA Effectiveness Recheck + +Runs daily at 08:30 MYT (00:30 UTC). Re-evaluates CAPAs marked `pending_verification` +past their recheck date and sends reminder email to the assigned verifier via Brevo. + +**Install:** +Add to crontab (same `crontab -e` session): +``` +30 0 * * * CRON_SECRET=$(grep ^CRON_SECRET /var/www/ims/.env | cut -d= -f2) curl -s -H "Authorization: Bearer $CRON_SECRET" http://127.0.0.1:3003/api/cron/effectiveness-recheck >> /var/log/ims-cron.log 2>&1 +``` + +**Manual test:** +```bash +CRON_SECRET=$(grep ^CRON_SECRET /var/www/ims/.env | cut -d= -f2) +curl -s -H "Authorization: Bearer $CRON_SECRET" http://127.0.0.1:3003/api/cron/effectiveness-recheck +# Expected: {"ok":true} +``` + +--- + +## Log Rotation + +Prevent `/var/log/ims-cron.log` from growing unbounded: +```bash +sudo nano /etc/logrotate.d/ims-cron +``` +Paste: +``` +/var/log/ims-cron.log { + daily + rotate 30 + compress + missingok + notifempty +} +```