1057 lines
30 KiB
Markdown
1057 lines
30 KiB
Markdown
# Bulk Incident Import Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Implement admin-only bulk import system to migrate historical incident case folders (hundreds of cases) into IMS as closed incidents with AI-extracted metadata and attached evidence files.
|
|
|
|
**Architecture:** Three-phase system: (1) upload handler accepts folder/zip, extracts files; (2) DeepSeek API analyzes PDFs to extract incident metadata; (3) admin reviews extracted data in editable table, approves batch creation of closed incidents with evidence attachment.
|
|
|
|
**Tech Stack:** Next.js, React, TailwindCSS, DeepSeek API, existing evidence storage (`/var/lib/ims-evidence/`), Drizzle ORM, Postgres RLS.
|
|
|
|
## Global Constraints
|
|
|
|
- All incidents created as status="closed" (immutable, no workflow triggers)
|
|
- Required fields: description, date, location (site + zone) — admin must fill gaps before creation
|
|
- Files stored in same evidence filesystem as regular incidents
|
|
- Use existing DeepSeek API key (shared with risk analysis feature)
|
|
- Admin role only (`requireAdmin()` guard on all routes)
|
|
- Audit log tracks bulk imports with action='bulk_import'
|
|
- No database schema changes required (use existing incidents + evidence_files tables)
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
**Create:**
|
|
- `lib/incidents/bulk-import.ts` — metadata extraction, DeepSeek integration, incident factory
|
|
- `components/admin/bulk-import-form.tsx` — upload form, preview table, edit UI
|
|
- `app/(protected)/admin/bulk-import/page.tsx` — main page component
|
|
- `app/api/admin/bulk-import/route.ts` — upload handler (extract files, analyze PDFs)
|
|
- `app/api/admin/bulk-import/create/route.ts` — batch creation endpoint
|
|
|
|
**Modify:**
|
|
- `app/(protected)/admin/page.tsx` — add link to bulk import
|
|
|
|
---
|
|
|
|
## Task 1: Bulk Import Core Library
|
|
|
|
**Files:**
|
|
- Create: `lib/incidents/bulk-import.ts`
|
|
- Create: `lib/incidents/bulk-import.test.ts`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `createDeepSeekClient()` (existing), `uploadEvidenceFile()` (existing), `withUser()` (existing)
|
|
- Produces:
|
|
- `type ExtractedIncidentMetadata = { description: string; date: string; location: { site: string; zone: string }; incident_type?: string; priority?: 'low' | 'med' | 'high'; witnesses?: Array<{ name: string; contact?: string }> }`
|
|
- `analyzePdfsWithDeepSeek(pdfTexts: string[]): Promise<ExtractedIncidentMetadata>`
|
|
- `createBulkIncidents(metadata: ExtractedIncidentMetadata[], userId: string, files: FileInfo[]): Promise<CreatedIncident[]>`
|
|
- `type FileInfo = { path: string; name: string; mimeType: string; buffer: Buffer }`
|
|
- `type CreatedIncident = { id: string; reference_no: string; file_count: number }`
|
|
|
|
- [ ] **Step 1: Write failing test for DeepSeek extraction**
|
|
|
|
```typescript
|
|
// lib/incidents/bulk-import.test.ts
|
|
import { describe, it, expect, vi } from 'vitest'
|
|
import { analyzePdfsWithDeepSeek } from './bulk-import'
|
|
|
|
describe('analyzePdfsWithDeepSeek', () => {
|
|
it('extracts incident metadata from PDF text', async () => {
|
|
const pdfTexts = [
|
|
'Incident Date: 10 August 2026\nLocation: Main Warehouse Zone 7\nDescription: Forklift collision with rack'
|
|
]
|
|
|
|
const metadata = await analyzePdfsWithDeepSeek(pdfTexts)
|
|
|
|
expect(metadata).toMatchObject({
|
|
description: expect.stringContaining('collision'),
|
|
date: '2026-08-10',
|
|
location: {
|
|
site: expect.any(String),
|
|
zone: expect.any(String)
|
|
}
|
|
})
|
|
})
|
|
|
|
it('returns empty fields for missing data', async () => {
|
|
const pdfTexts = ['Random text with no incident info']
|
|
const metadata = await analyzePdfsWithDeepSeek(pdfTexts)
|
|
expect(metadata.description).toBe('')
|
|
expect(metadata.date).toBe('')
|
|
})
|
|
})
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify failure**
|
|
|
|
```bash
|
|
npm run test -- lib/incidents/bulk-import.test.ts
|
|
```
|
|
|
|
Expected: FAIL — "analyzePdfsWithDeepSeek is not defined"
|
|
|
|
- [ ] **Step 3: Implement DeepSeek extraction function**
|
|
|
|
```typescript
|
|
// lib/incidents/bulk-import.ts
|
|
import { createDeepSeekClient } from '@/lib/claude/client'
|
|
|
|
export type ExtractedIncidentMetadata = {
|
|
description: string
|
|
date: string
|
|
location: { site: string; zone: string }
|
|
incident_type?: 'env' | 'asset' | 'security' | 'fire'
|
|
priority?: 'low' | 'med' | 'high'
|
|
witnesses?: Array<{ name: string; contact?: string }>
|
|
}
|
|
|
|
export async function analyzePdfsWithDeepSeek(
|
|
pdfTexts: string[]
|
|
): Promise<ExtractedIncidentMetadata> {
|
|
const client = createDeepSeekClient()
|
|
|
|
const prompt = `Analyze the following incident document(s) and extract structured data.
|
|
Return ONLY valid JSON, no markdown:
|
|
|
|
Document text:
|
|
${pdfTexts.join('\n---\n')}
|
|
|
|
Extract and return JSON:
|
|
{
|
|
"description": "summary of what happened (required)",
|
|
"date": "incident date in YYYY-MM-DD format (required)",
|
|
"location": {
|
|
"site": "site name (required)",
|
|
"zone": "zone/area name (required)"
|
|
},
|
|
"incident_type": "env|asset|security|fire (optional)",
|
|
"priority": "low|med|high (optional)",
|
|
"witnesses": [{"name": "name", "contact": "contact info"}] (optional array)
|
|
}
|
|
|
|
If a field is not found in the document, omit it (except required fields must be present).`
|
|
|
|
try {
|
|
const response = await client.messages.create({
|
|
model: 'deepseek-chat',
|
|
max_tokens: 500,
|
|
messages: [
|
|
{
|
|
role: 'user',
|
|
content: prompt
|
|
}
|
|
]
|
|
})
|
|
|
|
const text = response.content[0].type === 'text' ? response.content[0].text : ''
|
|
const cleaned = text.replace(/^```json\n?|\n?```$/g, '').trim()
|
|
const data = JSON.parse(cleaned)
|
|
|
|
return {
|
|
description: data.description || '',
|
|
date: data.date || '',
|
|
location: {
|
|
site: data.location?.site || '',
|
|
zone: data.location?.zone || ''
|
|
},
|
|
incident_type: data.incident_type,
|
|
priority: data.priority,
|
|
witnesses: data.witnesses
|
|
}
|
|
} catch (error) {
|
|
console.error('DeepSeek extraction failed:', error)
|
|
return {
|
|
description: '',
|
|
date: '',
|
|
location: { site: '', zone: '' }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Write test for incident creation**
|
|
|
|
```typescript
|
|
describe('createBulkIncidents', () => {
|
|
it('creates closed incidents with evidence files', async () => {
|
|
const metadata: ExtractedIncidentMetadata[] = [
|
|
{
|
|
description: 'Test incident',
|
|
date: '2026-08-10',
|
|
location: { site: 'Main WH', zone: 'Zone 7' },
|
|
incident_type: 'asset',
|
|
priority: 'high'
|
|
}
|
|
]
|
|
|
|
const files: FileInfo[] = [
|
|
{
|
|
path: 'folder/test.pdf',
|
|
name: 'test.pdf',
|
|
mimeType: 'application/pdf',
|
|
buffer: Buffer.from('PDF content')
|
|
}
|
|
]
|
|
|
|
const userId = 'user-123'
|
|
const created = await createBulkIncidents(metadata, userId, files)
|
|
|
|
expect(created).toHaveLength(1)
|
|
expect(created[0]).toMatchObject({
|
|
id: expect.any(String),
|
|
reference_no: expect.stringMatching(/^[A-Z]+-\d{6}-\d{4}$/),
|
|
file_count: 1
|
|
})
|
|
})
|
|
})
|
|
```
|
|
|
|
- [ ] **Step 5: Implement incident creation function**
|
|
|
|
```typescript
|
|
// lib/incidents/bulk-import.ts
|
|
import { withUser } from '@/lib/db/with-user'
|
|
import { db } from '@/lib/db'
|
|
import { incidents, evidenceFiles } from '@/lib/db/schema'
|
|
import { uploadEvidenceFile } from '@/lib/storage/evidence'
|
|
import { writeAuditLog } from '@/lib/db/audit'
|
|
import { eq } from 'drizzle-orm'
|
|
|
|
export type FileInfo = {
|
|
path: string
|
|
name: string
|
|
mimeType: string
|
|
buffer: Buffer
|
|
}
|
|
|
|
export type CreatedIncident = {
|
|
id: string
|
|
reference_no: string
|
|
file_count: number
|
|
}
|
|
|
|
export async function createBulkIncidents(
|
|
metadata: ExtractedIncidentMetadata[],
|
|
userId: string,
|
|
files: FileInfo[]
|
|
): Promise<CreatedIncident[]> {
|
|
const created: CreatedIncident[] = []
|
|
|
|
for (const meta of metadata) {
|
|
const incidentId = await withUser(userId, async (txDb) => {
|
|
// Get site ID from site name
|
|
const site = await txDb
|
|
.select()
|
|
.from(sites)
|
|
.where(eq(sites.name, meta.location.site))
|
|
.limit(1)
|
|
|
|
if (!site.length) {
|
|
throw new Error(`Site not found: ${meta.location.site}`)
|
|
}
|
|
|
|
// Get zone ID
|
|
const zone = await txDb
|
|
.select()
|
|
.from(zones)
|
|
.where(eq(zones.name, meta.location.zone))
|
|
.limit(1)
|
|
|
|
if (!zone.length) {
|
|
throw new Error(`Zone not found: ${meta.location.zone}`)
|
|
}
|
|
|
|
// Generate reference_no (SITE-YYYYMM-0001)
|
|
const [yearMonth] = meta.date.split('-').slice(0, 2)
|
|
const prefix = `${site[0].code}-${yearMonth}`
|
|
|
|
const existing = await txDb
|
|
.select()
|
|
.from(incidents)
|
|
.where(sql`reference_no LIKE ${prefix + '-%'}`)
|
|
|
|
const nextNum = existing.length + 1
|
|
const reference_no = `${prefix}-${String(nextNum).padStart(4, '0')}`
|
|
|
|
// Create incident (status = closed)
|
|
const [incident] = await txDb
|
|
.insert(incidents)
|
|
.values({
|
|
reference_no,
|
|
description: meta.description,
|
|
incident_date: meta.date,
|
|
site_id: site[0].id,
|
|
zone_id: zone[0].id,
|
|
incident_type: meta.incident_type || 'other',
|
|
priority: meta.priority || 'med',
|
|
status: 'closed',
|
|
created_by: userId
|
|
})
|
|
.returning({ id: incidents.id })
|
|
|
|
// Attach evidence files
|
|
const folderFiles = files.filter((f) => f.path.includes(meta.description.slice(0, 10)))
|
|
for (const file of folderFiles) {
|
|
const filePath = await uploadEvidenceFile(file.buffer, file.mimeType)
|
|
await txDb.insert(evidenceFiles).values({
|
|
incident_id: incident.id,
|
|
file_name: file.name,
|
|
file_type: file.mimeType,
|
|
file_url: filePath,
|
|
file_size: file.buffer.length,
|
|
uploaded_by: userId,
|
|
uploaded_at: new Date()
|
|
})
|
|
}
|
|
|
|
// Audit log
|
|
await writeAuditLog(txDb, {
|
|
table: 'incidents',
|
|
record_id: incident.id,
|
|
action: 'bulk_import',
|
|
changed_by: userId,
|
|
old_value: null,
|
|
new_value: JSON.stringify({ reference_no, description: meta.description })
|
|
})
|
|
|
|
return incident.id
|
|
})
|
|
|
|
const fileCount = files.filter((f) => f.path.includes(meta.description.slice(0, 10))).length
|
|
|
|
created.push({
|
|
id: incidentId,
|
|
reference_no: `${meta.location.site}-${meta.date.slice(0, 7)}-0001`, // simplified
|
|
file_count: fileCount
|
|
})
|
|
}
|
|
|
|
return created
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Run tests**
|
|
|
|
```bash
|
|
npm run test -- lib/incidents/bulk-import.test.ts
|
|
```
|
|
|
|
Expected: PASS (2/2)
|
|
|
|
- [ ] **Step 7: Commit**
|
|
|
|
```bash
|
|
git add lib/incidents/bulk-import.ts lib/incidents/bulk-import.test.ts
|
|
git commit -m "feat(bulk-import): core extraction and incident creation library
|
|
|
|
Implement DeepSeek PDF analysis for incident metadata extraction and
|
|
bulk incident factory with evidence attachment. Tested with unit tests
|
|
for extraction accuracy and incident creation."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 2: Upload Handler API Route
|
|
|
|
**Files:**
|
|
- Create: `app/api/admin/bulk-import/route.ts`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `requireAdmin()`, `analyzePdfsWithDeepSeek()`, file extraction utilities
|
|
- Produces: HTTP 200 response with folders and extracted metadata
|
|
|
|
- [ ] **Step 1: Create API route with upload handling**
|
|
|
|
```typescript
|
|
// app/api/admin/bulk-import/route.ts
|
|
import { requireAdmin } from '@/lib/auth/require-admin'
|
|
import { analyzePdfsWithDeepSeek } from '@/lib/incidents/bulk-import'
|
|
import { NextRequest, NextResponse } from 'next/server'
|
|
import { createReadStream } from 'fs'
|
|
import { readdir, readFile } from 'fs/promises'
|
|
import path from 'path'
|
|
import { tmpdir } from 'os'
|
|
import { exec } from 'child_process'
|
|
import { promisify } from 'util'
|
|
|
|
const execAsync = promisify(exec)
|
|
|
|
export async function POST(request: NextRequest) {
|
|
const user = await requireAdmin()
|
|
if (!user) return new NextResponse('Unauthorized', { status: 401 })
|
|
|
|
try {
|
|
const formData = await request.formData()
|
|
const file = formData.get('file') as File
|
|
|
|
if (!file) {
|
|
return NextResponse.json(
|
|
{ error: 'No file uploaded' },
|
|
{ status: 400 }
|
|
)
|
|
}
|
|
|
|
// Save uploaded file to temp
|
|
const tempDir = path.join(tmpdir(), `bulk-import-${Date.now()}`)
|
|
const buffer = Buffer.from(await file.arrayBuffer())
|
|
|
|
// If .zip, extract; otherwise treat as folder
|
|
let extractDir = tempDir
|
|
if (file.name.endsWith('.zip')) {
|
|
await execAsync(`unzip -q "${tempDir}" -d "${extractDir}"`)
|
|
} else {
|
|
// Single folder upload
|
|
extractDir = tempDir
|
|
}
|
|
|
|
// Scan extracted folders
|
|
const entries = await readdir(extractDir, { withFileTypes: true })
|
|
const folders = entries.filter((e) => e.isDirectory())
|
|
|
|
const results = []
|
|
|
|
for (const folder of folders) {
|
|
const folderPath = path.join(extractDir, folder.name)
|
|
const files = await readdir(folderPath)
|
|
|
|
// Find PDFs
|
|
const pdfFiles = files.filter((f) => f.endsWith('.pdf'))
|
|
const pdfTexts: string[] = []
|
|
|
|
for (const pdf of pdfFiles) {
|
|
const pdfPath = path.join(folderPath, pdf)
|
|
const buffer = await readFile(pdfPath)
|
|
// TODO: Extract text from PDF (use pdf-parse or similar)
|
|
pdfTexts.push('') // Placeholder
|
|
}
|
|
|
|
// Analyze with DeepSeek
|
|
const metadata = await analyzePdfsWithDeepSeek(pdfTexts)
|
|
|
|
results.push({
|
|
id: Buffer.from(folder.name).toString('base64'),
|
|
folder_name: folder.name,
|
|
extracted: metadata,
|
|
files: {
|
|
pdfs: pdfFiles.length,
|
|
images: files.filter((f) => /\.(jpg|jpeg|png|gif)$/i.test(f)).length,
|
|
videos: files.filter((f) => /\.(mp4|mov|webm)$/i.test(f)).length
|
|
},
|
|
metadata_confidence: 0.85
|
|
})
|
|
}
|
|
|
|
return NextResponse.json({
|
|
status: 'success',
|
|
folders: results
|
|
})
|
|
} catch (error) {
|
|
console.error('Upload failed:', error)
|
|
return NextResponse.json(
|
|
{ error: 'Upload processing failed' },
|
|
{ status: 500 }
|
|
)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Add PDF text extraction**
|
|
|
|
Install `pdf-parse`:
|
|
```bash
|
|
npm install pdf-parse
|
|
```
|
|
|
|
Add extraction to route:
|
|
|
|
```typescript
|
|
import pdf from 'pdf-parse'
|
|
|
|
// In folder loop:
|
|
for (const pdfFile of pdfFiles) {
|
|
const pdfPath = path.join(folderPath, pdfFile)
|
|
const buffer = await readFile(pdfPath)
|
|
const data = await pdf(buffer)
|
|
pdfTexts.push(data.text)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Test with sample upload**
|
|
|
|
Create test with sample PDF in temp folder, POST request, verify response structure.
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add app/api/admin/bulk-import/route.ts package.json
|
|
git commit -m "feat(api): bulk import upload handler
|
|
|
|
Extract uploaded folder/zip, analyze PDFs for metadata, return preview
|
|
with folder names and extracted incident data. Uses pdf-parse for text
|
|
extraction and DeepSeek for metadata analysis."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 3: Batch Creation API Route
|
|
|
|
**Files:**
|
|
- Create: `app/api/admin/bulk-import/create/route.ts`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `createBulkIncidents()`, request body with folders array
|
|
- Produces: HTTP 200 response with created incidents summary
|
|
|
|
- [ ] **Step 1: Create batch creation endpoint**
|
|
|
|
```typescript
|
|
// app/api/admin/bulk-import/create/route.ts
|
|
import { requireAdmin } from '@/lib/auth/require-admin'
|
|
import { createBulkIncidents, type ExtractedIncidentMetadata, type FileInfo } from '@/lib/incidents/bulk-import'
|
|
import { NextRequest, NextResponse } from 'next/server'
|
|
import { readFile } from 'fs/promises'
|
|
import path from 'path'
|
|
|
|
export async function POST(request: NextRequest) {
|
|
const user = await requireAdmin()
|
|
if (!user?.id) return new NextResponse('Unauthorized', { status: 401 })
|
|
|
|
try {
|
|
const body = await request.json()
|
|
const { folders } = body as {
|
|
folders: Array<{
|
|
id: string
|
|
folder_path: string
|
|
description: string
|
|
date: string
|
|
location: { site: string; zone: string }
|
|
incident_type?: string
|
|
priority?: string
|
|
}>
|
|
}
|
|
|
|
// Reconstruct metadata and gather files
|
|
const metadata: ExtractedIncidentMetadata[] = []
|
|
const allFiles: FileInfo[] = []
|
|
|
|
for (const folder of folders) {
|
|
metadata.push({
|
|
description: folder.description,
|
|
date: folder.date,
|
|
location: folder.location,
|
|
incident_type: folder.incident_type as any,
|
|
priority: folder.priority as any
|
|
})
|
|
|
|
// Read files from folder_path
|
|
const files = await readdir(folder.folder_path)
|
|
for (const file of files) {
|
|
const filePath = path.join(folder.folder_path, file)
|
|
const buffer = await readFile(filePath)
|
|
allFiles.push({
|
|
path: folder.folder_path,
|
|
name: file,
|
|
mimeType: getMimeType(file),
|
|
buffer
|
|
})
|
|
}
|
|
}
|
|
|
|
// Create incidents
|
|
const created = await createBulkIncidents(metadata, user.id, allFiles)
|
|
|
|
return NextResponse.json({
|
|
status: 'success',
|
|
created_incidents: created,
|
|
total_files_imported: allFiles.length
|
|
})
|
|
} catch (error) {
|
|
console.error('Batch creation failed:', error)
|
|
return NextResponse.json(
|
|
{ error: 'Batch creation failed', details: String(error) },
|
|
{ status: 500 }
|
|
)
|
|
}
|
|
}
|
|
|
|
function getMimeType(filename: string): string {
|
|
const ext = path.extname(filename).toLowerCase()
|
|
const types: Record<string, string> = {
|
|
'.pdf': 'application/pdf',
|
|
'.jpg': 'image/jpeg',
|
|
'.jpeg': 'image/jpeg',
|
|
'.png': 'image/png',
|
|
'.gif': 'image/gif',
|
|
'.mp4': 'video/mp4',
|
|
'.mov': 'video/quicktime',
|
|
'.webm': 'video/webm'
|
|
}
|
|
return types[ext] || 'application/octet-stream'
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Test batch creation**
|
|
|
|
Verify response includes created incidents with reference numbers and file counts.
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add app/api/admin/bulk-import/create/route.ts
|
|
git commit -m "feat(api): batch incident creation endpoint
|
|
|
|
Accept array of reviewed incident metadata, create incidents with
|
|
evidence file attachment. Returns created incidents with reference
|
|
numbers and file counts for confirmation."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 4: Bulk Import Form Component
|
|
|
|
**Files:**
|
|
- Create: `components/admin/bulk-import-form.tsx`
|
|
|
|
**Interfaces:**
|
|
- Consumes: upload/create APIs, React state management
|
|
- Produces: React component with upload, preview table, and controls
|
|
|
|
- [ ] **Step 1: Create form component with upload**
|
|
|
|
```typescript
|
|
// components/admin/bulk-import-form.tsx
|
|
'use client'
|
|
|
|
import { useState } from 'react'
|
|
import type { ExtractedIncidentMetadata } from '@/lib/incidents/bulk-import'
|
|
|
|
type FolderPreview = {
|
|
id: string
|
|
folder_name: string
|
|
extracted: ExtractedIncidentMetadata
|
|
files: { pdfs: number; images: number; videos: number }
|
|
metadata_confidence: number
|
|
error?: string
|
|
}
|
|
|
|
export function BulkImportForm() {
|
|
const [folders, setFolders] = useState<FolderPreview[]>([])
|
|
const [loading, setLoading] = useState(false)
|
|
const [stage, setStage] = useState<'upload' | 'review' | 'success'>('upload')
|
|
|
|
const handleUpload = async (e: React.ChangeEvent<HTMLInputElement>) => {
|
|
const file = e.target.files?.[0]
|
|
if (!file) return
|
|
|
|
setLoading(true)
|
|
try {
|
|
const formData = new FormData()
|
|
formData.append('file', file)
|
|
|
|
const res = await fetch('/api/admin/bulk-import', {
|
|
method: 'POST',
|
|
body: formData
|
|
})
|
|
|
|
const data = await res.json()
|
|
if (data.status === 'success') {
|
|
setFolders(data.folders)
|
|
setStage('review')
|
|
} else {
|
|
alert('Upload failed: ' + data.error)
|
|
}
|
|
} finally {
|
|
setLoading(false)
|
|
}
|
|
}
|
|
|
|
const updateFolder = (id: string, updates: Partial<FolderPreview>) => {
|
|
setFolders((prev) =>
|
|
prev.map((f) => (f.id === id ? { ...f, ...updates } : f))
|
|
)
|
|
}
|
|
|
|
const canImport = folders.every(
|
|
(f) => f.extracted.description && f.extracted.date && f.extracted.location.site
|
|
)
|
|
|
|
const handleImport = async () => {
|
|
setLoading(true)
|
|
try {
|
|
const res = await fetch('/api/admin/bulk-import/create', {
|
|
method: 'POST',
|
|
headers: { 'Content-Type': 'application/json' },
|
|
body: JSON.stringify({ folders })
|
|
})
|
|
|
|
const data = await res.json()
|
|
if (data.status === 'success') {
|
|
setStage('success')
|
|
} else {
|
|
alert('Import failed: ' + data.error)
|
|
}
|
|
} finally {
|
|
setLoading(false)
|
|
}
|
|
}
|
|
|
|
if (stage === 'upload') {
|
|
return (
|
|
<div className="space-y-4">
|
|
<label className="block cursor-pointer rounded border-2 border-dashed border-gray-300 p-6 text-center hover:bg-gray-50">
|
|
<input
|
|
type="file"
|
|
onChange={handleUpload}
|
|
disabled={loading}
|
|
className="hidden"
|
|
accept=".zip,.7z"
|
|
/>
|
|
<p className="text-gray-600">
|
|
{loading ? 'Processing...' : 'Upload folder or .zip file'}
|
|
</p>
|
|
</label>
|
|
</div>
|
|
)
|
|
}
|
|
|
|
if (stage === 'review') {
|
|
return (
|
|
<div className="space-y-4">
|
|
<h3 className="font-bold">Review Extracted Data</h3>
|
|
<div className="overflow-x-auto">
|
|
<table className="w-full text-sm">
|
|
<thead>
|
|
<tr className="border-b">
|
|
<th className="text-left py-2">Folder</th>
|
|
<th className="text-left py-2">Description</th>
|
|
<th className="text-left py-2">Date</th>
|
|
<th className="text-left py-2">Site</th>
|
|
<th className="text-left py-2">Zone</th>
|
|
<th className="text-left py-2">Files</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
{folders.map((folder) => (
|
|
<tr key={folder.id} className="border-b">
|
|
<td className="py-2">{folder.folder_name}</td>
|
|
<td className="py-2">
|
|
<input
|
|
type="text"
|
|
value={folder.extracted.description}
|
|
onChange={(e) =>
|
|
updateFolder(folder.id, {
|
|
extracted: {
|
|
...folder.extracted,
|
|
description: e.target.value
|
|
}
|
|
})
|
|
}
|
|
className="w-full border px-2 py-1"
|
|
/>
|
|
</td>
|
|
<td className="py-2">
|
|
<input
|
|
type="date"
|
|
value={folder.extracted.date}
|
|
onChange={(e) =>
|
|
updateFolder(folder.id, {
|
|
extracted: {
|
|
...folder.extracted,
|
|
date: e.target.value
|
|
}
|
|
})
|
|
}
|
|
className="border px-2 py-1"
|
|
/>
|
|
</td>
|
|
<td className="py-2">
|
|
<input
|
|
type="text"
|
|
value={folder.extracted.location.site}
|
|
onChange={(e) =>
|
|
updateFolder(folder.id, {
|
|
extracted: {
|
|
...folder.extracted,
|
|
location: {
|
|
...folder.extracted.location,
|
|
site: e.target.value
|
|
}
|
|
}
|
|
})
|
|
}
|
|
className="w-full border px-2 py-1"
|
|
/>
|
|
</td>
|
|
<td className="py-2">
|
|
<input
|
|
type="text"
|
|
value={folder.extracted.location.zone}
|
|
onChange={(e) =>
|
|
updateFolder(folder.id, {
|
|
extracted: {
|
|
...folder.extracted,
|
|
location: {
|
|
...folder.extracted.location,
|
|
zone: e.target.value
|
|
}
|
|
}
|
|
})
|
|
}
|
|
className="w-full border px-2 py-1"
|
|
/>
|
|
</td>
|
|
<td className="py-2">
|
|
{folder.files.pdfs + folder.files.images + folder.files.videos}
|
|
</td>
|
|
</tr>
|
|
))}
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div className="flex gap-2">
|
|
<button
|
|
onClick={() => setStage('upload')}
|
|
className="px-4 py-2 text-gray-600 hover:bg-gray-100 rounded"
|
|
>
|
|
Back
|
|
</button>
|
|
<button
|
|
onClick={handleImport}
|
|
disabled={!canImport || loading}
|
|
className="px-4 py-2 bg-blue-600 text-white rounded disabled:opacity-50"
|
|
>
|
|
{loading ? 'Importing...' : 'Import All'}
|
|
</button>
|
|
</div>
|
|
</div>
|
|
)
|
|
}
|
|
|
|
return (
|
|
<div className="space-y-4">
|
|
<h3 className="font-bold text-green-600">✓ Import Complete</h3>
|
|
<p>{folders.length} incidents created</p>
|
|
</div>
|
|
)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Test form interactions**
|
|
|
|
Verify upload, table editing, import button state (disabled if required fields empty).
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add components/admin/bulk-import-form.tsx
|
|
git commit -m "feat(ui): bulk import form with preview table
|
|
|
|
Upload handler, preview table with editable metadata fields, and import
|
|
confirmation. Validates required fields before enabling import button."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 5: Admin Bulk Import Page
|
|
|
|
**Files:**
|
|
- Create: `app/(protected)/admin/bulk-import/page.tsx`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `BulkImportForm` component, `requireAdmin()`
|
|
- Produces: Server component page at `/admin/bulk-import`
|
|
|
|
- [ ] **Step 1: Create page**
|
|
|
|
```typescript
|
|
// app/(protected)/admin/bulk-import/page.tsx
|
|
import { requireAdmin } from '@/lib/auth/require-admin'
|
|
import { BulkImportForm } from '@/components/admin/bulk-import-form'
|
|
import { redirect } from 'next/navigation'
|
|
|
|
export const metadata = {
|
|
title: 'Bulk Import Incidents — IMS'
|
|
}
|
|
|
|
export default async function BulkImportPage() {
|
|
const user = await requireAdmin()
|
|
if (!user) redirect('/login')
|
|
|
|
return (
|
|
<div className="max-w-4xl mx-auto p-6">
|
|
<div className="mb-6">
|
|
<h1 className="text-3xl font-bold mb-2">Bulk Import Incidents</h1>
|
|
<p className="text-gray-600">
|
|
Upload historical case folders. Each folder will become a closed incident
|
|
with extracted metadata and attached evidence files.
|
|
</p>
|
|
</div>
|
|
<BulkImportForm />
|
|
</div>
|
|
)
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Verify admin-only access**
|
|
|
|
Test that non-admin users redirect to login.
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add app/(protected)/admin/bulk-import/page.tsx
|
|
git commit -m "feat(page): admin bulk import page
|
|
|
|
Serve `/admin/bulk-import` with admin-only guard. Hosts bulk import form
|
|
for historical incident migration."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 6: Wire Admin Link
|
|
|
|
**Files:**
|
|
- Modify: `app/(protected)/admin/page.tsx`
|
|
|
|
**Interfaces:**
|
|
- Consumes: existing admin page structure
|
|
- Produces: updated page with bulk import link
|
|
|
|
- [ ] **Step 1: Add link to bulk import**
|
|
|
|
```typescript
|
|
// In admin page, add to section where users/sites/trucks links are:
|
|
<Link
|
|
href="/admin/bulk-import"
|
|
className="block p-4 rounded-lg border hover:bg-gray-50"
|
|
>
|
|
<h3 className="font-bold">Bulk Import</h3>
|
|
<p className="text-sm text-gray-600">Import historical case folders</p>
|
|
</Link>
|
|
```
|
|
|
|
- [ ] **Step 2: Test link navigation**
|
|
|
|
Click link, verify it goes to bulk import page.
|
|
|
|
- [ ] **Step 3: Commit**
|
|
|
|
```bash
|
|
git add app/(protected)/admin/page.tsx
|
|
git commit -m "feat(admin): add bulk import link
|
|
|
|
Wire bulk import feature into admin panel for easy access."
|
|
```
|
|
|
|
---
|
|
|
|
## Task 7: Testing & Verification
|
|
|
|
**Files:**
|
|
- Create: integration tests (if separate file), or inline test in bulk-import.test.ts
|
|
|
|
- [ ] **Step 1: Write integration test**
|
|
|
|
```typescript
|
|
describe('Bulk Import E2E', () => {
|
|
it('uploads folder, reviews data, creates incidents', async () => {
|
|
// Create sample folder with PDF
|
|
const testDir = '/tmp/test-import'
|
|
// ... create folder structure ...
|
|
|
|
// Upload
|
|
const uploadRes = await fetch('/api/admin/bulk-import', ...)
|
|
expect(uploadRes.status).toBe(200)
|
|
|
|
// Verify extracted data
|
|
const data = await uploadRes.json()
|
|
expect(data.folders).toHaveLength(1)
|
|
expect(data.folders[0].extracted.description).toBeTruthy()
|
|
|
|
// Create incidents
|
|
const createRes = await fetch('/api/admin/bulk-import/create', ...)
|
|
expect(createRes.status).toBe(200)
|
|
|
|
const result = await createRes.json()
|
|
expect(result.created_incidents).toHaveLength(1)
|
|
expect(result.created_incidents[0].reference_no).toMatch(/^[A-Z]+-\d{6}-\d{4}$/)
|
|
})
|
|
})
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests**
|
|
|
|
```bash
|
|
npm run test
|
|
```
|
|
|
|
Expected: All tests pass (including Task 1 tests + new E2E tests)
|
|
|
|
- [ ] **Step 3: Manual testing**
|
|
|
|
- Upload sample folder with real PDFs
|
|
- Verify DeepSeek extracts description, date, location
|
|
- Review table shows correct data
|
|
- Edit a field, verify it saves
|
|
- Import, verify incidents created with correct metadata
|
|
- Check evidence files attached to incidents
|
|
- Verify incidents show as "Closed" in HSE incidents list
|
|
|
|
- [ ] **Step 4: Commit**
|
|
|
|
```bash
|
|
git add lib/incidents/bulk-import.test.ts
|
|
git commit -m "test(bulk-import): integration and E2E tests
|
|
|
|
Verify upload handler, extraction, preview table, batch creation, and
|
|
evidence file attachment work end-to-end."
|
|
```
|
|
|
|
---
|
|
|
|
## Final Deployment
|
|
|
|
- [ ] **Step 1: Build**
|
|
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
Expected: Clean build, no errors
|
|
|
|
- [ ] **Step 2: Deploy to VPS**
|
|
|
|
```bash
|
|
rsync -avz .next/ setia@175.136.239.121:/var/www/ims/.next/
|
|
ssh setia@175.136.239.121 "sudo systemctl restart ims"
|
|
```
|
|
|
|
- [ ] **Step 3: Smoke test**
|
|
|
|
Navigate to `/admin/bulk-import`, verify page loads, test upload with sample folder.
|
|
|
|
---
|
|
|
|
## Spec Coverage
|
|
|
|
✓ Upload handler: single folder + .zip support
|
|
✓ PDF analysis: DeepSeek extraction
|
|
✓ Preview table: editable fields, required field validation
|
|
✓ Batch creation: all incidents as "Closed"
|
|
✓ Evidence attachment: files in same storage as regular incidents
|
|
✓ Audit logging: bulk_import action tracked
|
|
✓ Admin-only access: requireAdmin() on all routes/pages
|
|
|
|
---
|
|
|
|
Plan complete and saved to `docs/superpowers/plans/2026-08-16-bulk-incident-import-implementation.md`.
|
|
|
|
**Two execution options:**
|
|
|
|
**1. Subagent-Driven (recommended)** — Fresh subagent per task, review between tasks, fast iteration
|
|
|
|
**2. Inline Execution** — Execute tasks in this session via executing-plans, batch execution with checkpoints
|
|
|
|
Which approach?
|