# Bulk Incident Import — Design Specification **Date:** 2026-08-16 **Status:** Ready for Implementation **Scope:** Admin-only bulk import of historical incident cases with AI-extracted metadata --- ## Overview **Problem:** Hundreds of historical incident case folders (containing PDFs, images, videos) need to be migrated into IMS as closed incidents for record-keeping and archival. **Solution:** Bulk import system that accepts single-folder or .zip-based uploads, analyzes PDFs via DeepSeek API to extract incident metadata, and creates closed incidents with attached evidence files. **Success Criteria:** - Admin can upload single folder or .zip containing multiple case folders - System auto-extracts metadata from PDFs (description, date, location, type, severity, witnesses) - Admin reviews extracted data, fills required gaps, approves batch - All incidents created as "Closed" status with full evidence file chain - Audit log captures bulk import source for each incident --- ## Architecture ### Components #### 1. Upload Handler (`POST /api/admin/bulk-import`) - Accept multipart/form-data with single folder upload OR .zip file - Extract files, group by folder (each folder = one incident) - Validate: at least one file per folder, PDFs present for analysis - Response: list of folders ready for metadata extraction #### 2. PDF Metadata Analyzer - Read PDF/DOC content (text extraction via file-type detection) - Send text to DeepSeek API with structured extraction prompt - Extract fields: description, date, location (site + zone), incident_type, priority, witnesses - Return structured JSON for each folder's PDFs - Cache results during session (don't re-analyze on retry) **DeepSeek Prompt Template:** ``` Analyze the following incident document(s) and extract: - Description: summary of what happened - Date: incident date (YYYY-MM-DD format) - Location: site name and zone/area - Type: incident category (env/asset/security/fire) - Priority: severity level (low/med/high) - Witnesses: names and contact info if present Document text: [PDF content here] Return JSON format: { "description": "...", "date": "YYYY-MM-DD", "location": { "site": "...", "zone": "..." }, "incident_type": "env|asset|security|fire", "priority": "low|med|high", "witnesses": [{"name": "...", "contact": "..."}, ...] } ``` #### 3. Preview & Approval Table - Display extracted data as editable table - Columns: folder name, description, date, location (site), location (zone), type, priority, # witnesses, # files - Allow inline editing for any field (admin fills gaps) - Mark rows as complete/incomplete (red if required field missing) - "Import All" button only enabled when all rows have description + date + location #### 4. Incident Factory - Create incident from row data using `withUser()` transaction - Set status = "Closed" (not "Open") - Set created_by = current admin user - Store bulk_import_source in audit_log old_value for tracking - Attach all folder files as evidence (PDFs, images, videos) - Generate unique incident reference_no (existing format: SITE-YYYYMM-####) #### 5. Evidence Attachment - For each file in folder: call `uploadEvidenceFile()` (existing helper) - Store in `/var/lib/ims-evidence/[incident_id]/[filename]` - Record in `evidence_files` table (same as regular incidents) - No special staging/approval; all files attached as "verified" evidence --- ## Data Model ### New Table: `bulk_import_sessions` (optional, for audit) ```sql CREATE TABLE bulk_import_sessions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), imported_by UUID NOT NULL REFERENCES users(id), upload_filename TEXT NOT NULL, folder_count INT NOT NULL, incident_count INT NOT NULL, file_count INT NOT NULL, created_at TIMESTAMP DEFAULT now() ); ``` ### Existing Tables Modified - `incidents`: use existing columns (description, created_by, status, etc.) - `evidence_files`: attach files under incident_id (existing behavior) - `audit_log`: new entries for each bulk-imported incident with `action='bulk_import'` --- ## User Flow 1. **Upload:** Admin navigates to `/admin/bulk-import`, uploads single folder or .zip 2. **Analysis:** System extracts files, analyzes PDFs via DeepSeek, shows progress 3. **Preview:** Table displays extracted metadata; admin reviews and fills gaps 4. **Approval:** Admin clicks "Import All" (button disabled if any required field missing) 5. **Creation:** System creates all incidents + attaches files in parallel 6. **Confirmation:** Shows summary (X incidents created, Y total files imported) --- ## API Routes ### `POST /api/admin/bulk-import` — Upload & Analyze Request: ``` multipart/form-data: file: (or individual folder via file picker) ``` Response: ```json { "status": "success", "folders": [ { "id": "folder_hash", "folder_name": "10AUG2026 - SUNIL", "extracted": { "description": "...", "date": "2026-08-10", "location": { "site": "Main WH", "zone": "Zone 7" }, "incident_type": "asset", "priority": "high", "witnesses": [...] }, "files": { "pdfs": 2, "images": 3, "videos": 1 }, "metadata_confidence": 0.92 } ] } ``` ### `POST /api/admin/bulk-import/create` — Batch Create Request: ```json { "folders": [ { "id": "folder_hash", "description": "...", "date": "2026-08-10", "location": { "site": "...", "zone": "..." }, "incident_type": "asset", "priority": "high", "witnesses": [...] } ] } ``` Response: ```json { "status": "success", "created_incidents": [ { "id": "inc-123", "reference_no": "MAIN-202608-0001", "description": "...", "status": "closed", "file_count": 6 } ], "total_files_imported": 42 } ``` --- ## UI Page: `/admin/bulk-import` **Layout:** - Header: "Bulk Import Historical Incidents" - Upload section: folder/zip picker + drag-drop zone - Progress bar during analysis - Preview table (editable) - "Import All" button + confirmation modal - Success summary with incident list **Permissions:** Admin role only (`requireAdmin()`) --- ## Error Handling | Scenario | Behavior | |----------|----------| | No PDFs in folder | Skip folder, warn admin ("No analyzable documents in 'folder_name'") | | DeepSeek API fails | Show error, allow manual entry (fields stay empty) | | Missing required field after manual review | Disable "Import" button, highlight row in red | | File upload too large (>100MB total) | Reject, suggest splitting into multiple uploads | | Folder name conflicts with existing incident | Auto-increment reference_no, notify admin | | Database transaction fails on create | Rollback entire batch, show error with failed folder names | --- ## Testing Strategy **Unit Tests:** - PDF text extraction (mock file reads) - DeepSeek prompt generation and JSON parsing - Incident factory (create incident, attach evidence, audit log) - Reference_no generation under collision scenarios **Integration Tests:** - End-to-end single-folder upload → preview → create - Batch .zip upload with 5+ folders - Missing required fields → error handling - Evidence file attachment verification **Manual Testing:** - Upload real historical case folder - Verify extracted metadata accuracy - Confirm incident + evidence files appear in HSE incidents list - Check audit_log for bulk_import entries --- ## Deployment Notes **Environment:** - `DEEPSEEK_API_KEY` must be set (already configured for risk analysis) - `EVIDENCE_DIR` must exist and be writable (already set) - Supabase storage for temporary zip extraction (use `/tmp` or disk) **Performance:** - DeepSeek analysis runs sequentially per folder (safe for hundreds) - Incident creation (batch) can run in parallel to save time - Estimated time: ~5-10 minutes for 100 folders (depends on PDF sizes + DeepSeek latency) **Compliance:** - Audit log tracks every bulk import (action='bulk_import', old_value=session_id) - Closed status prevents accidental edits (no workflow triggers) - Evidence retention policy applies (5-year minimum per DOSH) --- ## Future Enhancements (out of scope) - Async job queue for very large imports (1000+ folders) - OCR preprocessing for scanned PDFs - Duplicate detection (warn if folder name matches existing incident) - Scheduled imports (batch upload schedule) --- ## Files to Create/Modify **Create:** - `app/(protected)/admin/bulk-import/page.tsx` — admin page - `components/admin/bulk-import-form.tsx` — upload + preview component - `lib/incidents/bulk-import.ts` — metadata extraction + incident factory - `app/api/admin/bulk-import/route.ts` — upload handler - `app/api/admin/bulk-import/create/route.ts` — batch creation endpoint **Modify:** - `app/(protected)/admin/page.tsx` — add link to bulk import - `lib/storage/evidence.ts` — no changes (use existing `uploadEvidenceFile()`) **Optional:** - `supabase/migrations/####_bulk_import_sessions.sql` — audit table (if needed)