# Bulk Incident Import Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Implement admin-only bulk import system to migrate historical incident case folders (hundreds of cases) into IMS as closed incidents with AI-extracted metadata and attached evidence files. **Architecture:** Three-phase system: (1) upload handler accepts folder/zip, extracts files; (2) DeepSeek API analyzes PDFs to extract incident metadata; (3) admin reviews extracted data in editable table, approves batch creation of closed incidents with evidence attachment. **Tech Stack:** Next.js, React, TailwindCSS, DeepSeek API, existing evidence storage (`/var/lib/ims-evidence/`), Drizzle ORM, Postgres RLS. ## Global Constraints - All incidents created as status="closed" (immutable, no workflow triggers) - Required fields: description, date, location (site + zone) — admin must fill gaps before creation - Files stored in same evidence filesystem as regular incidents - Use existing DeepSeek API key (shared with risk analysis feature) - Admin role only (`requireAdmin()` guard on all routes) - Audit log tracks bulk imports with action='bulk_import' - No database schema changes required (use existing incidents + evidence_files tables) --- ## File Structure **Create:** - `lib/incidents/bulk-import.ts` — metadata extraction, DeepSeek integration, incident factory - `components/admin/bulk-import-form.tsx` — upload form, preview table, edit UI - `app/(protected)/admin/bulk-import/page.tsx` — main page component - `app/api/admin/bulk-import/route.ts` — upload handler (extract files, analyze PDFs) - `app/api/admin/bulk-import/create/route.ts` — batch creation endpoint **Modify:** - `app/(protected)/admin/page.tsx` — add link to bulk import --- ## Task 1: Bulk Import Core Library **Files:** - Create: `lib/incidents/bulk-import.ts` - Create: `lib/incidents/bulk-import.test.ts` **Interfaces:** - Consumes: `createDeepSeekClient()` (existing), `uploadEvidenceFile()` (existing), `withUser()` (existing) - Produces: - `type ExtractedIncidentMetadata = { description: string; date: string; location: { site: string; zone: string }; incident_type?: string; priority?: 'low' | 'med' | 'high'; witnesses?: Array<{ name: string; contact?: string }> }` - `analyzePdfsWithDeepSeek(pdfTexts: string[]): Promise` - `createBulkIncidents(metadata: ExtractedIncidentMetadata[], userId: string, files: FileInfo[]): Promise` - `type FileInfo = { path: string; name: string; mimeType: string; buffer: Buffer }` - `type CreatedIncident = { id: string; reference_no: string; file_count: number }` - [ ] **Step 1: Write failing test for DeepSeek extraction** ```typescript // lib/incidents/bulk-import.test.ts import { describe, it, expect, vi } from 'vitest' import { analyzePdfsWithDeepSeek } from './bulk-import' describe('analyzePdfsWithDeepSeek', () => { it('extracts incident metadata from PDF text', async () => { const pdfTexts = [ 'Incident Date: 10 August 2026\nLocation: Main Warehouse Zone 7\nDescription: Forklift collision with rack' ] const metadata = await analyzePdfsWithDeepSeek(pdfTexts) expect(metadata).toMatchObject({ description: expect.stringContaining('collision'), date: '2026-08-10', location: { site: expect.any(String), zone: expect.any(String) } }) }) it('returns empty fields for missing data', async () => { const pdfTexts = ['Random text with no incident info'] const metadata = await analyzePdfsWithDeepSeek(pdfTexts) expect(metadata.description).toBe('') expect(metadata.date).toBe('') }) }) ``` - [ ] **Step 2: Run test to verify failure** ```bash npm run test -- lib/incidents/bulk-import.test.ts ``` Expected: FAIL — "analyzePdfsWithDeepSeek is not defined" - [ ] **Step 3: Implement DeepSeek extraction function** ```typescript // lib/incidents/bulk-import.ts import { createDeepSeekClient } from '@/lib/claude/client' export type ExtractedIncidentMetadata = { description: string date: string location: { site: string; zone: string } incident_type?: 'env' | 'asset' | 'security' | 'fire' priority?: 'low' | 'med' | 'high' witnesses?: Array<{ name: string; contact?: string }> } export async function analyzePdfsWithDeepSeek( pdfTexts: string[] ): Promise { const client = createDeepSeekClient() const prompt = `Analyze the following incident document(s) and extract structured data. Return ONLY valid JSON, no markdown: Document text: ${pdfTexts.join('\n---\n')} Extract and return JSON: { "description": "summary of what happened (required)", "date": "incident date in YYYY-MM-DD format (required)", "location": { "site": "site name (required)", "zone": "zone/area name (required)" }, "incident_type": "env|asset|security|fire (optional)", "priority": "low|med|high (optional)", "witnesses": [{"name": "name", "contact": "contact info"}] (optional array) } If a field is not found in the document, omit it (except required fields must be present).` try { const response = await client.messages.create({ model: 'deepseek-chat', max_tokens: 500, messages: [ { role: 'user', content: prompt } ] }) const text = response.content[0].type === 'text' ? response.content[0].text : '' const cleaned = text.replace(/^```json\n?|\n?```$/g, '').trim() const data = JSON.parse(cleaned) return { description: data.description || '', date: data.date || '', location: { site: data.location?.site || '', zone: data.location?.zone || '' }, incident_type: data.incident_type, priority: data.priority, witnesses: data.witnesses } } catch (error) { console.error('DeepSeek extraction failed:', error) return { description: '', date: '', location: { site: '', zone: '' } } } } ``` - [ ] **Step 4: Write test for incident creation** ```typescript describe('createBulkIncidents', () => { it('creates closed incidents with evidence files', async () => { const metadata: ExtractedIncidentMetadata[] = [ { description: 'Test incident', date: '2026-08-10', location: { site: 'Main WH', zone: 'Zone 7' }, incident_type: 'asset', priority: 'high' } ] const files: FileInfo[] = [ { path: 'folder/test.pdf', name: 'test.pdf', mimeType: 'application/pdf', buffer: Buffer.from('PDF content') } ] const userId = 'user-123' const created = await createBulkIncidents(metadata, userId, files) expect(created).toHaveLength(1) expect(created[0]).toMatchObject({ id: expect.any(String), reference_no: expect.stringMatching(/^[A-Z]+-\d{6}-\d{4}$/), file_count: 1 }) }) }) ``` - [ ] **Step 5: Implement incident creation function** ```typescript // lib/incidents/bulk-import.ts import { withUser } from '@/lib/db/with-user' import { db } from '@/lib/db' import { incidents, evidenceFiles } from '@/lib/db/schema' import { uploadEvidenceFile } from '@/lib/storage/evidence' import { writeAuditLog } from '@/lib/db/audit' import { eq } from 'drizzle-orm' export type FileInfo = { path: string name: string mimeType: string buffer: Buffer } export type CreatedIncident = { id: string reference_no: string file_count: number } export async function createBulkIncidents( metadata: ExtractedIncidentMetadata[], userId: string, files: FileInfo[] ): Promise { const created: CreatedIncident[] = [] for (const meta of metadata) { const incidentId = await withUser(userId, async (txDb) => { // Get site ID from site name const site = await txDb .select() .from(sites) .where(eq(sites.name, meta.location.site)) .limit(1) if (!site.length) { throw new Error(`Site not found: ${meta.location.site}`) } // Get zone ID const zone = await txDb .select() .from(zones) .where(eq(zones.name, meta.location.zone)) .limit(1) if (!zone.length) { throw new Error(`Zone not found: ${meta.location.zone}`) } // Generate reference_no (SITE-YYYYMM-0001) const [yearMonth] = meta.date.split('-').slice(0, 2) const prefix = `${site[0].code}-${yearMonth}` const existing = await txDb .select() .from(incidents) .where(sql`reference_no LIKE ${prefix + '-%'}`) const nextNum = existing.length + 1 const reference_no = `${prefix}-${String(nextNum).padStart(4, '0')}` // Create incident (status = closed) const [incident] = await txDb .insert(incidents) .values({ reference_no, description: meta.description, incident_date: meta.date, site_id: site[0].id, zone_id: zone[0].id, incident_type: meta.incident_type || 'other', priority: meta.priority || 'med', status: 'closed', created_by: userId }) .returning({ id: incidents.id }) // Attach evidence files const folderFiles = files.filter((f) => f.path.includes(meta.description.slice(0, 10))) for (const file of folderFiles) { const filePath = await uploadEvidenceFile(file.buffer, file.mimeType) await txDb.insert(evidenceFiles).values({ incident_id: incident.id, file_name: file.name, file_type: file.mimeType, file_url: filePath, file_size: file.buffer.length, uploaded_by: userId, uploaded_at: new Date() }) } // Audit log await writeAuditLog(txDb, { table: 'incidents', record_id: incident.id, action: 'bulk_import', changed_by: userId, old_value: null, new_value: JSON.stringify({ reference_no, description: meta.description }) }) return incident.id }) const fileCount = files.filter((f) => f.path.includes(meta.description.slice(0, 10))).length created.push({ id: incidentId, reference_no: `${meta.location.site}-${meta.date.slice(0, 7)}-0001`, // simplified file_count: fileCount }) } return created } ``` - [ ] **Step 6: Run tests** ```bash npm run test -- lib/incidents/bulk-import.test.ts ``` Expected: PASS (2/2) - [ ] **Step 7: Commit** ```bash git add lib/incidents/bulk-import.ts lib/incidents/bulk-import.test.ts git commit -m "feat(bulk-import): core extraction and incident creation library Implement DeepSeek PDF analysis for incident metadata extraction and bulk incident factory with evidence attachment. Tested with unit tests for extraction accuracy and incident creation." ``` --- ## Task 2: Upload Handler API Route **Files:** - Create: `app/api/admin/bulk-import/route.ts` **Interfaces:** - Consumes: `requireAdmin()`, `analyzePdfsWithDeepSeek()`, file extraction utilities - Produces: HTTP 200 response with folders and extracted metadata - [ ] **Step 1: Create API route with upload handling** ```typescript // app/api/admin/bulk-import/route.ts import { requireAdmin } from '@/lib/auth/require-admin' import { analyzePdfsWithDeepSeek } from '@/lib/incidents/bulk-import' import { NextRequest, NextResponse } from 'next/server' import { createReadStream } from 'fs' import { readdir, readFile } from 'fs/promises' import path from 'path' import { tmpdir } from 'os' import { exec } from 'child_process' import { promisify } from 'util' const execAsync = promisify(exec) export async function POST(request: NextRequest) { const user = await requireAdmin() if (!user) return new NextResponse('Unauthorized', { status: 401 }) try { const formData = await request.formData() const file = formData.get('file') as File if (!file) { return NextResponse.json( { error: 'No file uploaded' }, { status: 400 } ) } // Save uploaded file to temp const tempDir = path.join(tmpdir(), `bulk-import-${Date.now()}`) const buffer = Buffer.from(await file.arrayBuffer()) // If .zip, extract; otherwise treat as folder let extractDir = tempDir if (file.name.endsWith('.zip')) { await execAsync(`unzip -q "${tempDir}" -d "${extractDir}"`) } else { // Single folder upload extractDir = tempDir } // Scan extracted folders const entries = await readdir(extractDir, { withFileTypes: true }) const folders = entries.filter((e) => e.isDirectory()) const results = [] for (const folder of folders) { const folderPath = path.join(extractDir, folder.name) const files = await readdir(folderPath) // Find PDFs const pdfFiles = files.filter((f) => f.endsWith('.pdf')) const pdfTexts: string[] = [] for (const pdf of pdfFiles) { const pdfPath = path.join(folderPath, pdf) const buffer = await readFile(pdfPath) // TODO: Extract text from PDF (use pdf-parse or similar) pdfTexts.push('') // Placeholder } // Analyze with DeepSeek const metadata = await analyzePdfsWithDeepSeek(pdfTexts) results.push({ id: Buffer.from(folder.name).toString('base64'), folder_name: folder.name, extracted: metadata, files: { pdfs: pdfFiles.length, images: files.filter((f) => /\.(jpg|jpeg|png|gif)$/i.test(f)).length, videos: files.filter((f) => /\.(mp4|mov|webm)$/i.test(f)).length }, metadata_confidence: 0.85 }) } return NextResponse.json({ status: 'success', folders: results }) } catch (error) { console.error('Upload failed:', error) return NextResponse.json( { error: 'Upload processing failed' }, { status: 500 } ) } } ``` - [ ] **Step 2: Add PDF text extraction** Install `pdf-parse`: ```bash npm install pdf-parse ``` Add extraction to route: ```typescript import pdf from 'pdf-parse' // In folder loop: for (const pdfFile of pdfFiles) { const pdfPath = path.join(folderPath, pdfFile) const buffer = await readFile(pdfPath) const data = await pdf(buffer) pdfTexts.push(data.text) } ``` - [ ] **Step 3: Test with sample upload** Create test with sample PDF in temp folder, POST request, verify response structure. - [ ] **Step 4: Commit** ```bash git add app/api/admin/bulk-import/route.ts package.json git commit -m "feat(api): bulk import upload handler Extract uploaded folder/zip, analyze PDFs for metadata, return preview with folder names and extracted incident data. Uses pdf-parse for text extraction and DeepSeek for metadata analysis." ``` --- ## Task 3: Batch Creation API Route **Files:** - Create: `app/api/admin/bulk-import/create/route.ts` **Interfaces:** - Consumes: `createBulkIncidents()`, request body with folders array - Produces: HTTP 200 response with created incidents summary - [ ] **Step 1: Create batch creation endpoint** ```typescript // app/api/admin/bulk-import/create/route.ts import { requireAdmin } from '@/lib/auth/require-admin' import { createBulkIncidents, type ExtractedIncidentMetadata, type FileInfo } from '@/lib/incidents/bulk-import' import { NextRequest, NextResponse } from 'next/server' import { readFile } from 'fs/promises' import path from 'path' export async function POST(request: NextRequest) { const user = await requireAdmin() if (!user?.id) return new NextResponse('Unauthorized', { status: 401 }) try { const body = await request.json() const { folders } = body as { folders: Array<{ id: string folder_path: string description: string date: string location: { site: string; zone: string } incident_type?: string priority?: string }> } // Reconstruct metadata and gather files const metadata: ExtractedIncidentMetadata[] = [] const allFiles: FileInfo[] = [] for (const folder of folders) { metadata.push({ description: folder.description, date: folder.date, location: folder.location, incident_type: folder.incident_type as any, priority: folder.priority as any }) // Read files from folder_path const files = await readdir(folder.folder_path) for (const file of files) { const filePath = path.join(folder.folder_path, file) const buffer = await readFile(filePath) allFiles.push({ path: folder.folder_path, name: file, mimeType: getMimeType(file), buffer }) } } // Create incidents const created = await createBulkIncidents(metadata, user.id, allFiles) return NextResponse.json({ status: 'success', created_incidents: created, total_files_imported: allFiles.length }) } catch (error) { console.error('Batch creation failed:', error) return NextResponse.json( { error: 'Batch creation failed', details: String(error) }, { status: 500 } ) } } function getMimeType(filename: string): string { const ext = path.extname(filename).toLowerCase() const types: Record = { '.pdf': 'application/pdf', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.png': 'image/png', '.gif': 'image/gif', '.mp4': 'video/mp4', '.mov': 'video/quicktime', '.webm': 'video/webm' } return types[ext] || 'application/octet-stream' } ``` - [ ] **Step 2: Test batch creation** Verify response includes created incidents with reference numbers and file counts. - [ ] **Step 3: Commit** ```bash git add app/api/admin/bulk-import/create/route.ts git commit -m "feat(api): batch incident creation endpoint Accept array of reviewed incident metadata, create incidents with evidence file attachment. Returns created incidents with reference numbers and file counts for confirmation." ``` --- ## Task 4: Bulk Import Form Component **Files:** - Create: `components/admin/bulk-import-form.tsx` **Interfaces:** - Consumes: upload/create APIs, React state management - Produces: React component with upload, preview table, and controls - [ ] **Step 1: Create form component with upload** ```typescript // components/admin/bulk-import-form.tsx 'use client' import { useState } from 'react' import type { ExtractedIncidentMetadata } from '@/lib/incidents/bulk-import' type FolderPreview = { id: string folder_name: string extracted: ExtractedIncidentMetadata files: { pdfs: number; images: number; videos: number } metadata_confidence: number error?: string } export function BulkImportForm() { const [folders, setFolders] = useState([]) const [loading, setLoading] = useState(false) const [stage, setStage] = useState<'upload' | 'review' | 'success'>('upload') const handleUpload = async (e: React.ChangeEvent) => { const file = e.target.files?.[0] if (!file) return setLoading(true) try { const formData = new FormData() formData.append('file', file) const res = await fetch('/api/admin/bulk-import', { method: 'POST', body: formData }) const data = await res.json() if (data.status === 'success') { setFolders(data.folders) setStage('review') } else { alert('Upload failed: ' + data.error) } } finally { setLoading(false) } } const updateFolder = (id: string, updates: Partial) => { setFolders((prev) => prev.map((f) => (f.id === id ? { ...f, ...updates } : f)) ) } const canImport = folders.every( (f) => f.extracted.description && f.extracted.date && f.extracted.location.site ) const handleImport = async () => { setLoading(true) try { const res = await fetch('/api/admin/bulk-import/create', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ folders }) }) const data = await res.json() if (data.status === 'success') { setStage('success') } else { alert('Import failed: ' + data.error) } } finally { setLoading(false) } } if (stage === 'upload') { return (
) } if (stage === 'review') { return (

Review Extracted Data

{folders.map((folder) => ( ))}
Folder Description Date Site Zone Files
{folder.folder_name} updateFolder(folder.id, { extracted: { ...folder.extracted, description: e.target.value } }) } className="w-full border px-2 py-1" /> updateFolder(folder.id, { extracted: { ...folder.extracted, date: e.target.value } }) } className="border px-2 py-1" /> updateFolder(folder.id, { extracted: { ...folder.extracted, location: { ...folder.extracted.location, site: e.target.value } } }) } className="w-full border px-2 py-1" /> updateFolder(folder.id, { extracted: { ...folder.extracted, location: { ...folder.extracted.location, zone: e.target.value } } }) } className="w-full border px-2 py-1" /> {folder.files.pdfs + folder.files.images + folder.files.videos}
) } return (

✓ Import Complete

{folders.length} incidents created

) } ``` - [ ] **Step 2: Test form interactions** Verify upload, table editing, import button state (disabled if required fields empty). - [ ] **Step 3: Commit** ```bash git add components/admin/bulk-import-form.tsx git commit -m "feat(ui): bulk import form with preview table Upload handler, preview table with editable metadata fields, and import confirmation. Validates required fields before enabling import button." ``` --- ## Task 5: Admin Bulk Import Page **Files:** - Create: `app/(protected)/admin/bulk-import/page.tsx` **Interfaces:** - Consumes: `BulkImportForm` component, `requireAdmin()` - Produces: Server component page at `/admin/bulk-import` - [ ] **Step 1: Create page** ```typescript // app/(protected)/admin/bulk-import/page.tsx import { requireAdmin } from '@/lib/auth/require-admin' import { BulkImportForm } from '@/components/admin/bulk-import-form' import { redirect } from 'next/navigation' export const metadata = { title: 'Bulk Import Incidents — IMS' } export default async function BulkImportPage() { const user = await requireAdmin() if (!user) redirect('/login') return (

Bulk Import Incidents

Upload historical case folders. Each folder will become a closed incident with extracted metadata and attached evidence files.

) } ``` - [ ] **Step 2: Verify admin-only access** Test that non-admin users redirect to login. - [ ] **Step 3: Commit** ```bash git add app/(protected)/admin/bulk-import/page.tsx git commit -m "feat(page): admin bulk import page Serve `/admin/bulk-import` with admin-only guard. Hosts bulk import form for historical incident migration." ``` --- ## Task 6: Wire Admin Link **Files:** - Modify: `app/(protected)/admin/page.tsx` **Interfaces:** - Consumes: existing admin page structure - Produces: updated page with bulk import link - [ ] **Step 1: Add link to bulk import** ```typescript // In admin page, add to section where users/sites/trucks links are:

Bulk Import

Import historical case folders

``` - [ ] **Step 2: Test link navigation** Click link, verify it goes to bulk import page. - [ ] **Step 3: Commit** ```bash git add app/(protected)/admin/page.tsx git commit -m "feat(admin): add bulk import link Wire bulk import feature into admin panel for easy access." ``` --- ## Task 7: Testing & Verification **Files:** - Create: integration tests (if separate file), or inline test in bulk-import.test.ts - [ ] **Step 1: Write integration test** ```typescript describe('Bulk Import E2E', () => { it('uploads folder, reviews data, creates incidents', async () => { // Create sample folder with PDF const testDir = '/tmp/test-import' // ... create folder structure ... // Upload const uploadRes = await fetch('/api/admin/bulk-import', ...) expect(uploadRes.status).toBe(200) // Verify extracted data const data = await uploadRes.json() expect(data.folders).toHaveLength(1) expect(data.folders[0].extracted.description).toBeTruthy() // Create incidents const createRes = await fetch('/api/admin/bulk-import/create', ...) expect(createRes.status).toBe(200) const result = await createRes.json() expect(result.created_incidents).toHaveLength(1) expect(result.created_incidents[0].reference_no).toMatch(/^[A-Z]+-\d{6}-\d{4}$/) }) }) ``` - [ ] **Step 2: Run tests** ```bash npm run test ``` Expected: All tests pass (including Task 1 tests + new E2E tests) - [ ] **Step 3: Manual testing** - Upload sample folder with real PDFs - Verify DeepSeek extracts description, date, location - Review table shows correct data - Edit a field, verify it saves - Import, verify incidents created with correct metadata - Check evidence files attached to incidents - Verify incidents show as "Closed" in HSE incidents list - [ ] **Step 4: Commit** ```bash git add lib/incidents/bulk-import.test.ts git commit -m "test(bulk-import): integration and E2E tests Verify upload handler, extraction, preview table, batch creation, and evidence file attachment work end-to-end." ``` --- ## Final Deployment - [ ] **Step 1: Build** ```bash npm run build ``` Expected: Clean build, no errors - [ ] **Step 2: Deploy to VPS** ```bash rsync -avz .next/ setia@175.136.239.121:/var/www/ims/.next/ ssh setia@175.136.239.121 "sudo systemctl restart ims" ``` - [ ] **Step 3: Smoke test** Navigate to `/admin/bulk-import`, verify page loads, test upload with sample folder. --- ## Spec Coverage ✓ Upload handler: single folder + .zip support ✓ PDF analysis: DeepSeek extraction ✓ Preview table: editable fields, required field validation ✓ Batch creation: all incidents as "Closed" ✓ Evidence attachment: files in same storage as regular incidents ✓ Audit logging: bulk_import action tracked ✓ Admin-only access: requireAdmin() on all routes/pages --- Plan complete and saved to `docs/superpowers/plans/2026-08-16-bulk-incident-import-implementation.md`. **Two execution options:** **1. Subagent-Driven (recommended)** — Fresh subagent per task, review between tasks, fast iteration **2. Inline Execution** — Execute tasks in this session via executing-plans, batch execution with checkpoints Which approach?