87 lines
3.4 KiB
Markdown
87 lines
3.4 KiB
Markdown
# Phase 9 — Cutover Runbook
|
||
|
||
## Pre-flight checklist (before cutting traffic)
|
||
|
||
Run from the project root on your local machine.
|
||
|
||
- [ ] `bash db/01-load-schema.sh` — schema + RLS loaded on new server (14 tables)
|
||
- [ ] `SUPABASE_DB_URL="..." bash db/02-dump-and-restore.sh` — all public data restored
|
||
- [ ] `SUPABASE_DB_URL="..." bash db/03-migrate-passwords.sh` — passwords migrated, 0 NULL rows
|
||
- [ ] `SUPABASE_URL="..." SUPABASE_SERVICE_ROLE_KEY="..." bash db/04-migrate-evidence.sh` — evidence files on server
|
||
- [ ] `SUPABASE_DB_URL="..." bash db/05-verify.sh` — row counts match, embeddings present
|
||
- [ ] `/var/www/ims/.env` populated on server (see `docs/new-server-setup.md` section I)
|
||
- [ ] `bash deploy.sh` — Next.js standalone build deployed, `systemctl status ims` shows Active
|
||
- [ ] nginx config live at `ims.setia.com.my` (see `docs/new-server-setup.md` sections E–F)
|
||
- [ ] TLS cert issued (`sudo certbot --nginx -d ims.setia.com.my`)
|
||
- [ ] `curl -I https://ims.setia.com.my/` returns HTTP/2 200
|
||
- [ ] Cron jobs registered (see `docs/vps-cron.md`)
|
||
- [ ] Daily backup cron registered (see `docs/new-server-setup.md` section K)
|
||
|
||
## DNS cutover
|
||
|
||
Point `ims.setia.com.my` A record to the new server's public IP.
|
||
TTL: set to 60s before cutover, restore to 3600s after.
|
||
|
||
## Smoke tests (run immediately after cutover)
|
||
|
||
Test as each role. All must pass before telling users to switch.
|
||
|
||
### Reporter role
|
||
- [ ] Log in with a migrated user's real password — succeeds
|
||
- [ ] Submit new incident with photo evidence — reference number generated, evidence stored
|
||
- [ ] Open own incident detail — evidence renders (HMAC-signed URL serves file)
|
||
|
||
### Supervisor role
|
||
- [ ] Log in
|
||
- [ ] View site incidents list — all historical incidents present
|
||
- [ ] Triage an incident — triage saved, audit_log row written with correct `changed_by`
|
||
|
||
### HSE role
|
||
- [ ] Log in
|
||
- [ ] Run AI triage suggestion — response received (Anthropic key working)
|
||
- [ ] Run Similar Incidents — embeddings returning results
|
||
- [ ] Open dashboard — stats render, JKKP8 export downloads CSV
|
||
- [ ] AI risk flags panel loads
|
||
|
||
### Admin role
|
||
- [ ] Log in
|
||
- [ ] Create new user — user created, no Supabase call
|
||
- [ ] Delete user — user removed (auth + profile)
|
||
|
||
### CAPA owner role
|
||
- [ ] Log in
|
||
- [ ] View assigned CAPAs
|
||
- [ ] Mark a CAPA in-progress — status updates
|
||
|
||
### Password reset flow
|
||
- [ ] Request reset for own email — Brevo email arrives
|
||
- [ ] Complete reset — new password accepted
|
||
|
||
### Cron (manual trigger)
|
||
- [ ] CAPA escalation endpoint returns `{"ok":true}`
|
||
- [ ] Effectiveness recheck endpoint returns `{"ok":true}`
|
||
|
||
## RLS spot-check (security)
|
||
|
||
```bash
|
||
# As app_user role with a reporter's user_id set:
|
||
sudo -u postgres psql -d ims
|
||
SET app.user_id = '<reporter-A-uuid>';
|
||
SELECT * FROM incidents WHERE reported_by != '<reporter-A-uuid>';
|
||
-- Expected: 0 rows (RLS blocks cross-user read)
|
||
```
|
||
|
||
## Post-cutover
|
||
|
||
- [ ] Monitor `/var/log/ims-cron.log` for first cron run (08:00 MYT next day)
|
||
- [ ] Monitor `sudo journalctl -u ims -f` for runtime errors
|
||
- [ ] After 48h stable: take snapshot of new server DB (`pg_dump ims > /var/lib/ims/backups/pre-supabase-decommission.sql`)
|
||
- [ ] Decommission Supabase project only after snapshot confirmed
|
||
|
||
## Rollback plan
|
||
|
||
If cutover fails within the first hour:
|
||
1. Repoint DNS A record back to the old VPS IP (`64.176.82.100`)
|
||
2. Supabase project remains live throughout — no data risk
|
||
3. Investigate failure before re-attempting
|