docs: Supabase to local postgres migration design spec
Migration plan for moving from Supabase cloud to self-hosted local postgres. Scope: 30 users, 500 incidents, ~100MB database. Downtime: Planned maintenance window. Approach: Direct pg_dump/restore with validation. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,187 @@
|
|||||||
|
# Supabase → Local Postgres Migration Design
|
||||||
|
|
||||||
|
**Date:** 2026-09-30
|
||||||
|
**Status:** Approved
|
||||||
|
**Goal:** Migrate from Supabase cloud to self-hosted local postgres, reduce cloud dependency
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
IMS currently uses Supabase cloud postgres as primary database. Migration to local postgres (127.0.0.1:5433/ims) will enable full self-hosting and eliminate cloud dependency.
|
||||||
|
|
||||||
|
**Data scope:** 30 users, 500 incidents, ~100MB
|
||||||
|
**Downtime:** Planned maintenance window acceptable
|
||||||
|
**Services affected:** Database only (no auth, storage, or real-time services to migrate)
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Current State
|
||||||
|
- **Primary DB:** Supabase postgres (nkcfjbgappslicotwopl.supabase.co)
|
||||||
|
- **Local postgres:** Configured in DATABASE_URL but empty, not used
|
||||||
|
- **App config:** .env.production has both DATABASE_URL and SUPABASE_* vars
|
||||||
|
- **Deployment:** /var/www/ims on production server
|
||||||
|
|
||||||
|
### Target State
|
||||||
|
- **Primary DB:** Local postgres (127.0.0.1:5433/ims)
|
||||||
|
- **Supabase:** Fully decommissioned
|
||||||
|
- **App config:** DATABASE_URL only, all SUPABASE_* vars removed
|
||||||
|
- **Data retention:** Local backups for disaster recovery
|
||||||
|
|
||||||
|
## Migration Plan
|
||||||
|
|
||||||
|
### Phase 1: Pre-Migration Setup
|
||||||
|
1. Announce maintenance window to all users (24-48hr notice)
|
||||||
|
2. Create backup of Supabase database (pg_dump via Supabase console or API)
|
||||||
|
3. Verify local postgres is running and accessible
|
||||||
|
4. Verify schema exists in local postgres (or run migrations)
|
||||||
|
5. Stop IMS app: `ssh setia@175.136.239.121 "pm2 stop wsims"`
|
||||||
|
|
||||||
|
### Phase 2: Data Export & Import
|
||||||
|
1. Export from Supabase:
|
||||||
|
```bash
|
||||||
|
# Use Supabase console or pg_dump with connection string
|
||||||
|
pg_dump <supabase-connection-string> > /tmp/supabase-backup.sql
|
||||||
|
```
|
||||||
|
2. Transfer dump to production server
|
||||||
|
3. Import to local postgres:
|
||||||
|
```bash
|
||||||
|
ssh setia@175.136.239.121 "psql -h 127.0.0.1 -p 5433 -U app_admin -d ims -f /tmp/supabase-backup.sql"
|
||||||
|
```
|
||||||
|
4. Verify import completed without errors
|
||||||
|
|
||||||
|
**Estimated duration:** 5-15 minutes for 100MB database
|
||||||
|
|
||||||
|
### Phase 3: Data Validation
|
||||||
|
Run validation checks on production server:
|
||||||
|
1. **Row count verification:** Compare table row counts before/after
|
||||||
|
- users, incidents, investigations, capa_actions, evidence_files, etc.
|
||||||
|
2. **Integrity checks:** Verify foreign key constraints
|
||||||
|
```sql
|
||||||
|
SELECT COUNT(*) FROM users; -- Should match Supabase count
|
||||||
|
SELECT COUNT(*) FROM incidents WHERE reported_by NOT IN (SELECT id FROM users); -- No orphans
|
||||||
|
```
|
||||||
|
3. **Password hash validation:** Verify password_hash column contains valid bcryptjs hashes ($2a$ or $2b$ prefix)
|
||||||
|
4. **Recent data spot check:** Verify last 5-10 incidents, users, CAPA records are present
|
||||||
|
5. **Timestamp validation:** Verify created_at, updated_at timestamps are intact
|
||||||
|
|
||||||
|
### Phase 4: Application Cutover
|
||||||
|
1. Update production .env.production:
|
||||||
|
- Remove all `SUPABASE_*` variables
|
||||||
|
- Keep `DATABASE_URL` pointing to local postgres
|
||||||
|
2. Restart IMS app:
|
||||||
|
```bash
|
||||||
|
ssh setia@175.136.239.121 "pm2 restart wsims"
|
||||||
|
```
|
||||||
|
3. Monitor app logs for connection errors
|
||||||
|
4. Manual test:
|
||||||
|
- Login as admin@ims.local
|
||||||
|
- View incidents dashboard
|
||||||
|
- Create a test incident
|
||||||
|
- Verify email notifications work
|
||||||
|
5. Check app health endpoint or dashboard
|
||||||
|
|
||||||
|
### Phase 5: Post-Migration
|
||||||
|
1. Keep Supabase backup for 7 days (in case emergency rollback needed)
|
||||||
|
2. Monitor local postgres performance (disk usage, query latency)
|
||||||
|
3. Update documentation:
|
||||||
|
- CLAUDE.md: Remove Supabase references, add local postgres setup
|
||||||
|
- memory/ims-database-setup.md: Mark as complete
|
||||||
|
4. After 30 days with no issues, delete Supabase project
|
||||||
|
|
||||||
|
## Rollback Plan
|
||||||
|
|
||||||
|
If critical issues discovered during cutover:
|
||||||
|
1. Stop app
|
||||||
|
2. Restore Supabase connection in .env.production
|
||||||
|
3. Restart app to revert to Supabase
|
||||||
|
4. Investigate issue in local postgres
|
||||||
|
5. Re-attempt migration after fixes
|
||||||
|
|
||||||
|
**Rollback window:** 7 days (Supabase backup retained)
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- Supabase connection credentials (for pg_dump)
|
||||||
|
- Local postgres credentials (app_user, app_admin)
|
||||||
|
- Access to production server (ssh -p 9321)
|
||||||
|
- PM2 running on production
|
||||||
|
|
||||||
|
### Validation Queries
|
||||||
|
```sql
|
||||||
|
-- Count check
|
||||||
|
SELECT 'users' as table_name, COUNT(*) as count FROM users
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'incidents', COUNT(*) FROM incidents
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'investigations', COUNT(*) FROM investigations
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'capa_actions', COUNT(*) FROM capa_actions
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'evidence_files', COUNT(*) FROM evidence_files;
|
||||||
|
|
||||||
|
-- Orphan check
|
||||||
|
SELECT COUNT(*) as orphan_users FROM incidents WHERE reported_by NOT IN (SELECT id FROM users);
|
||||||
|
SELECT COUNT(*) as orphan_incidents FROM investigations WHERE incident_id NOT IN (SELECT id FROM incidents);
|
||||||
|
|
||||||
|
-- Recent data check
|
||||||
|
SELECT id, email, created_at FROM users ORDER BY created_at DESC LIMIT 5;
|
||||||
|
SELECT id, title, created_at FROM incidents ORDER BY created_at DESC LIMIT 5;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
- [ ] All data successfully imported to local postgres
|
||||||
|
- [ ] Row counts match Supabase
|
||||||
|
- [ ] No orphaned foreign keys
|
||||||
|
- [ ] Password hashes intact
|
||||||
|
- [ ] App connects to local postgres without errors
|
||||||
|
- [ ] Login works (test with admin@ims.local)
|
||||||
|
- [ ] Recent incidents visible and editable
|
||||||
|
- [ ] No error logs during cutover window
|
||||||
|
- [ ] Performance acceptable on local postgres
|
||||||
|
|
||||||
|
## Timeline
|
||||||
|
|
||||||
|
- **Preparation:** 1-2 hours (backup, schema verification, announce)
|
||||||
|
- **Migration:** 15-30 minutes (export, import, validation)
|
||||||
|
- **Testing:** 30 minutes (health checks, manual testing)
|
||||||
|
- **Total downtime:** 1-2 hours
|
||||||
|
|
||||||
|
## Risks & Mitigations
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
|------|-----------|
|
||||||
|
| Data corruption during transfer | Backup Supabase before migration, validate checksums |
|
||||||
|
| Foreign key constraint violations | Run orphan checks post-import |
|
||||||
|
| App connection fails | Have Supabase rollback ready, test connection strings |
|
||||||
|
| Local postgres out of disk space | Check disk before import, provision sufficient space |
|
||||||
|
| Password hashes not recognized | Verify bcryptjs format ($2a$/$2b$), test login before declaring success |
|
||||||
|
|
||||||
|
## Configuration Changes
|
||||||
|
|
||||||
|
### .env.production (Production Server)
|
||||||
|
|
||||||
|
**Remove:**
|
||||||
|
```
|
||||||
|
NEXT_PUBLIC_SUPABASE_URL=...
|
||||||
|
NEXT_PUBLIC_SUPABASE_ANON_KEY=...
|
||||||
|
SUPABASE_SERVICE_ROLE_KEY=...
|
||||||
|
```
|
||||||
|
|
||||||
|
**Keep:**
|
||||||
|
```
|
||||||
|
DATABASE_URL=postgres://app_user:145edf53b9fdca6b189ccf6260422809355b0ce258178ecdeba206f825dcd144@127.0.0.1:5433/ims
|
||||||
|
DATABASE_URL_ADMIN=postgres://app_admin:7b7c201d2461c8868a7427f81182aa36e1df4eabc0520d0621a5692dee11feeb@127.0.0.1:5433/ims
|
||||||
|
```
|
||||||
|
|
||||||
|
## Documentation Updates
|
||||||
|
|
||||||
|
1. **CLAUDE.md:** Remove Supabase references, document local postgres requirement
|
||||||
|
2. **memory/ims-database-setup.md:** Update from "needs setup" to "migrated from Supabase"
|
||||||
|
3. **New doc:** /docs/database/local-postgres-setup.md (backup/restore procedures)
|
||||||
|
|
||||||
|
## Future Considerations
|
||||||
|
|
||||||
|
- Automated backups of local postgres (daily dump to `/var/backups/ims/`)
|
||||||
|
- Monitoring for disk space on /var/www/ims database volume
|
||||||
|
- Consider replication to secondary postgres for HA (future phase)
|
||||||
Reference in New Issue
Block a user