diff --git a/docs/superpowers/specs/2026-09-30-supabase-to-local-migration-design.md b/docs/superpowers/specs/2026-09-30-supabase-to-local-migration-design.md new file mode 100644 index 0000000..3f67b07 --- /dev/null +++ b/docs/superpowers/specs/2026-09-30-supabase-to-local-migration-design.md @@ -0,0 +1,187 @@ +# Supabase → Local Postgres Migration Design + +**Date:** 2026-09-30 +**Status:** Approved +**Goal:** Migrate from Supabase cloud to self-hosted local postgres, reduce cloud dependency + +## Overview + +IMS currently uses Supabase cloud postgres as primary database. Migration to local postgres (127.0.0.1:5433/ims) will enable full self-hosting and eliminate cloud dependency. + +**Data scope:** 30 users, 500 incidents, ~100MB +**Downtime:** Planned maintenance window acceptable +**Services affected:** Database only (no auth, storage, or real-time services to migrate) + +## Architecture + +### Current State +- **Primary DB:** Supabase postgres (nkcfjbgappslicotwopl.supabase.co) +- **Local postgres:** Configured in DATABASE_URL but empty, not used +- **App config:** .env.production has both DATABASE_URL and SUPABASE_* vars +- **Deployment:** /var/www/ims on production server + +### Target State +- **Primary DB:** Local postgres (127.0.0.1:5433/ims) +- **Supabase:** Fully decommissioned +- **App config:** DATABASE_URL only, all SUPABASE_* vars removed +- **Data retention:** Local backups for disaster recovery + +## Migration Plan + +### Phase 1: Pre-Migration Setup +1. Announce maintenance window to all users (24-48hr notice) +2. Create backup of Supabase database (pg_dump via Supabase console or API) +3. Verify local postgres is running and accessible +4. Verify schema exists in local postgres (or run migrations) +5. Stop IMS app: `ssh setia@175.136.239.121 "pm2 stop wsims"` + +### Phase 2: Data Export & Import +1. Export from Supabase: + ```bash + # Use Supabase console or pg_dump with connection string + pg_dump > /tmp/supabase-backup.sql + ``` +2. Transfer dump to production server +3. Import to local postgres: + ```bash + ssh setia@175.136.239.121 "psql -h 127.0.0.1 -p 5433 -U app_admin -d ims -f /tmp/supabase-backup.sql" + ``` +4. Verify import completed without errors + +**Estimated duration:** 5-15 minutes for 100MB database + +### Phase 3: Data Validation +Run validation checks on production server: +1. **Row count verification:** Compare table row counts before/after + - users, incidents, investigations, capa_actions, evidence_files, etc. +2. **Integrity checks:** Verify foreign key constraints + ```sql + SELECT COUNT(*) FROM users; -- Should match Supabase count + SELECT COUNT(*) FROM incidents WHERE reported_by NOT IN (SELECT id FROM users); -- No orphans + ``` +3. **Password hash validation:** Verify password_hash column contains valid bcryptjs hashes ($2a$ or $2b$ prefix) +4. **Recent data spot check:** Verify last 5-10 incidents, users, CAPA records are present +5. **Timestamp validation:** Verify created_at, updated_at timestamps are intact + +### Phase 4: Application Cutover +1. Update production .env.production: + - Remove all `SUPABASE_*` variables + - Keep `DATABASE_URL` pointing to local postgres +2. Restart IMS app: + ```bash + ssh setia@175.136.239.121 "pm2 restart wsims" + ``` +3. Monitor app logs for connection errors +4. Manual test: + - Login as admin@ims.local + - View incidents dashboard + - Create a test incident + - Verify email notifications work +5. Check app health endpoint or dashboard + +### Phase 5: Post-Migration +1. Keep Supabase backup for 7 days (in case emergency rollback needed) +2. Monitor local postgres performance (disk usage, query latency) +3. Update documentation: + - CLAUDE.md: Remove Supabase references, add local postgres setup + - memory/ims-database-setup.md: Mark as complete +4. After 30 days with no issues, delete Supabase project + +## Rollback Plan + +If critical issues discovered during cutover: +1. Stop app +2. Restore Supabase connection in .env.production +3. Restart app to revert to Supabase +4. Investigate issue in local postgres +5. Re-attempt migration after fixes + +**Rollback window:** 7 days (Supabase backup retained) + +## Implementation Details + +### Prerequisites +- Supabase connection credentials (for pg_dump) +- Local postgres credentials (app_user, app_admin) +- Access to production server (ssh -p 9321) +- PM2 running on production + +### Validation Queries +```sql +-- Count check +SELECT 'users' as table_name, COUNT(*) as count FROM users +UNION ALL +SELECT 'incidents', COUNT(*) FROM incidents +UNION ALL +SELECT 'investigations', COUNT(*) FROM investigations +UNION ALL +SELECT 'capa_actions', COUNT(*) FROM capa_actions +UNION ALL +SELECT 'evidence_files', COUNT(*) FROM evidence_files; + +-- Orphan check +SELECT COUNT(*) as orphan_users FROM incidents WHERE reported_by NOT IN (SELECT id FROM users); +SELECT COUNT(*) as orphan_incidents FROM investigations WHERE incident_id NOT IN (SELECT id FROM incidents); + +-- Recent data check +SELECT id, email, created_at FROM users ORDER BY created_at DESC LIMIT 5; +SELECT id, title, created_at FROM incidents ORDER BY created_at DESC LIMIT 5; +``` + +## Success Criteria + +- [ ] All data successfully imported to local postgres +- [ ] Row counts match Supabase +- [ ] No orphaned foreign keys +- [ ] Password hashes intact +- [ ] App connects to local postgres without errors +- [ ] Login works (test with admin@ims.local) +- [ ] Recent incidents visible and editable +- [ ] No error logs during cutover window +- [ ] Performance acceptable on local postgres + +## Timeline + +- **Preparation:** 1-2 hours (backup, schema verification, announce) +- **Migration:** 15-30 minutes (export, import, validation) +- **Testing:** 30 minutes (health checks, manual testing) +- **Total downtime:** 1-2 hours + +## Risks & Mitigations + +| Risk | Mitigation | +|------|-----------| +| Data corruption during transfer | Backup Supabase before migration, validate checksums | +| Foreign key constraint violations | Run orphan checks post-import | +| App connection fails | Have Supabase rollback ready, test connection strings | +| Local postgres out of disk space | Check disk before import, provision sufficient space | +| Password hashes not recognized | Verify bcryptjs format ($2a$/$2b$), test login before declaring success | + +## Configuration Changes + +### .env.production (Production Server) + +**Remove:** +``` +NEXT_PUBLIC_SUPABASE_URL=... +NEXT_PUBLIC_SUPABASE_ANON_KEY=... +SUPABASE_SERVICE_ROLE_KEY=... +``` + +**Keep:** +``` +DATABASE_URL=postgres://app_user:145edf53b9fdca6b189ccf6260422809355b0ce258178ecdeba206f825dcd144@127.0.0.1:5433/ims +DATABASE_URL_ADMIN=postgres://app_admin:7b7c201d2461c8868a7427f81182aa36e1df4eabc0520d0621a5692dee11feeb@127.0.0.1:5433/ims +``` + +## Documentation Updates + +1. **CLAUDE.md:** Remove Supabase references, document local postgres requirement +2. **memory/ims-database-setup.md:** Update from "needs setup" to "migrated from Supabase" +3. **New doc:** /docs/database/local-postgres-setup.md (backup/restore procedures) + +## Future Considerations + +- Automated backups of local postgres (daily dump to `/var/backups/ims/`) +- Monitoring for disk space on /var/www/ims database volume +- Consider replication to secondary postgres for HA (future phase)