docs: add implementation plan for force rescan feature

2026-02-26 10:12:08 -05:00
parent b7e0e7ffbb
commit 56c80fbe14
1 changed files with 417 additions and 0 deletions
@@ -0,0 +1,417 @@
+# Implementation Plan: Force Rescan Feature
+
+## Overview
+
+Add a `force` parameter to the library scan endpoint that allows re-processing existing media items. Currently, the scanner skips files that already exist in the database (based on file path). The force flag will bypass this check and re-process all files.
+
+## Current Behavior
+
+When scanning a library:
+1. `ScanLibrary` handler receives scan request
+2. Worker enqueues a `JobTypeScan` job
+3. `processScanJob` creates a `MediaScanner` and calls `ScanFolders()`
+4. For each file, `processMediaFile()` checks if file exists in DB
+5. If exists with same size → **skip** (return `false, nil`)
+6. If exists with different size → **update**
+7. If doesn't exist → **create new**
+
+## Proposed Changes
+
+### 1. Backend: Add `force` parameter support
+
+#### File: `internal/handlers/scanner.go`
+
+**Change 1.1** - Add `Force` field to `ScanLibraryRequest` struct (line ~89-92):
+
+```go
+type ScanLibraryRequest struct {
+    FolderPaths []string `json:"folder_paths,omitempty"`
+    LibraryID   string   `json:"library_id,omitempty"`
+    Force       bool     `json:"force,omitempty"`  // NEW: Force rescan of existing files
+}
+```
+
+**Change 1.2** - Pass `force` to job params in `ScanLibrary` function (line ~153):
+
+```go
+job := &services.Job{
+    ID:   jobID,
+    Type: services.JobTypeScan,
+    Params: map[string]interface{}{
+        "library_id": req.LibraryID,
+        "folders":    folderPaths,
+        "admin_id":   userUUID.String(),
+        "db":         h.db,
+        "force":      req.Force,  // NEW
+    },
+    // ... rest unchanged
+}
+```
+
+#### File: `internal/services/worker.go`
+
+**Change 2.1** - Extract `force` param in `processScanJob` function (after line ~195):
+
+```go
+// Existing code:
+db, ok := job.Params["db"].(*database.Queries)
+if !ok {
+    return nil, fmt.Errorf("database queries required")
+}
+
+// NEW: Extract force parameter
+force := false
+if forceVal, ok := job.Params["force"].(bool); ok {
+    force = forceVal
+}
+```
+
+**Change 2.2** - Pass force to MediaScanner (after line ~220):
+
+```go
+scanner.SetAdminID(adminUUID)
+
+// NEW: Set force flag
+scanner.SetForce(force)  // NEW
+```
+
+#### File: `internal/services/media_scanner.go`
+
+**Change 3.1** - Add `forceRescan` field to MediaScanner struct (find struct definition):
+
+```go
+type MediaScanner struct {
+    db           *database.Queries
+    folders      []string
+    adminID      pgtype.UUID
+    job          *Job
+    watcher      *fsnotify.Watcher
+    totalFiles   int
+    newItems     int
+    errors       int
+    forceRescan  bool  // NEW: Force re-scan of existing files
+}
+```
+
+**Change 3.2** - Add `SetForce` method (anywhere in file, after existing setters):
+
+```go
+func (s *MediaScanner) SetForce(force bool) {
+    s.forceRescan = force
+}
+```
+
+**Change 3.3** - Modify `processMediaFile` to respect force flag (line ~356-366):
+
+Current code:
+```go
+// Check if media item already exists in database
+existingItem, err := s.getMediaItemByFilePath(ctx, path)
+if err == nil {
+    fmt.Printf("Media item already exists in database: %s (size: %d vs %d)\n", path, existingItem.FileSize.Int64, info.Size())
+    // Media item exists, check if file has changed (by size)
+    if existingItem.FileSize.Int64 != info.Size() {
+        fmt.Printf("File size changed, updating media item: %s\n", path)
+        _ = s.updateMediaItem(ctx, existingItem.ID, path, info)
+        return false, nil
+    }
+    fmt.Printf("Media item already exists with same size, skipping: %s\n", path)
+    return false, nil  // <-- THIS IS WHERE WE SKIP
+}
+```
+
+New code:
+```go
+// Check if media item already exists in database
+existingItem, err := s.getMediaItemByFilePath(ctx, path)
+if err == nil {
+    fmt.Printf("Media item already exists in database: %s (size: %d vs %d)\n", path, existingItem.FileSize.Int64, info.Size())
+    
+    // If force rescan is enabled, always re-process
+    if s.forceRescan {
+        fmt.Printf("Force rescan enabled, re-processing existing media item: %s\n", path)
+        // Force update: delete existing and re-create
+        if err := s.db.DeleteMediaItem(ctx, existingItem.ID); err != nil {
+            fmt.Printf("Warning: failed to delete existing media item: %v\n", err)
+        }
+        // Continue to create new entry below
+    } else {
+        // Normal behavior: check if file has changed (by size)
+        if existingItem.FileSize.Int64 != info.Size() {
+            fmt.Printf("File size changed, updating media item: %s\n", path)
+            _ = s.updateMediaItem(ctx, existingItem.ID, path, info)
+            return false, nil
+        }
+        fmt.Printf("Media item already exists with same size, skipping: %s\n", path)
+        return false, nil
+    }
+}
+```
+
+### 2. Frontend: Update button and API call
+
+#### File: `web/src/admin.ts`
+
+**Change 4.1** - Update `scanAllLibraries` function to send `force: true` (line ~129):
+
+Current code:
+```typescript
+const scanResp = await fetch(`/api/libraries/${lib.id}/scan`, {
+    method: 'POST',
+    headers: { 'Authorization': `Bearer ${token}` }
+});
+```
+
+New code:
+```typescript
+const scanResp = await fetch(`/api/libraries/${lib.id}/scan`, {
+    method: 'POST',
+    headers: { 
+        'Authorization': `Bearer ${token}`,
+        'Content-Type': 'application/json'
+    },
+    body: JSON.stringify({ force: true })
+});
+```
+
+#### File: `templates/admin.templ`
+
+**Change 5.1** - Update button text and description (line ~55-58):
+
+Current:
+```html
+<button onclick="scanAllLibraries()" class="btn-primary p-4 rounded-lg text-left">
+    <div class="font-medium">Scan Library</div>
+    <div style="color: var(--text-secondary)" class="text-sm">Find new ebooks in your folders</div>
+</button>
+```
+
+New:
+```html
+<button onclick="scanAllLibraries()" class="btn-primary p-4 rounded-lg text-left">
+    <div class="font-medium">Rescan Library</div>
+    <div style="color: var(--text-secondary)" class="text-sm">Re-scan existing files and fix metadata</div>
+</button>
+```
+
+### 3. API Documentation
+
+#### File: `docs/developer/api/scanner/scan_library.md`
+
+**Change 6.1** - Update documentation to reflect force parameter (line ~15):
+
+Current:
+```markdown
+| force | boolean | No | Force rescan of existing files (default: false) |
+```
+
+This is already documented. Ensure the description is accurate:
+```markdown
+| force | boolean | No | Force rescan of existing files. When true, re-processes all files in library regardless of whether they already exist in database (default: false) |
+```
+
+### 4. Tests
+
+#### Unit Tests: `internal/services/worker_test.go`
+
+**Change 7.1** - Add test case for force parameter in existing scan job tests:
+
+```go
+func TestWorker_ProcessJob_ScanJob_ForceRescan(t *testing.T) {
+    // Setup test with existing media item in database
+    // Create job with force: true
+    // Verify media item is re-processed
+}
+```
+
+#### Integration Tests: `cmd/server/tests/scanner_integration_test.go`
+
+**Change 7.2** - Add integration tests covering three contexts (per guidelines line 140):
+
+Follow existing pattern from `scanner_integration_test.go`:
+
+```go
+func TestScanLibrary_ForceFlag(t *testing.T) {
+    s := setupTestServer(t)
+    defer s.TearDown()
+    
+    // Test 1: No user context (unauthenticated) - expect 401
+    // Test 2: Regular user context - expect 403 (admin only)
+    // Test 3: Admin context with force=true - expect 200, verify re-scan
+    // Test 4: Admin context with force=false - expect 200, verify skip
+}
+```
+
+#### Bruno OpenCollection YAML Tests
+
+**Change 7.3** - Add force parameter test case to existing Bruno test:
+
+File: `bruno/scanner/Scan Media Items.yml`
+
+Add a new test request or modify existing to include:
+```yaml
+body: {
+  "force": true
+}
+```
+
+### 5. Frontend: Add Watch Status Display
+
+Replace the "Settings" card on /admin page with a Watch Status display.
+
+#### File: `templates/admin.templ`
+
+**Change 8.1** - Replace Settings card with Watch Status (lines 38-47):
+
+Current:
+```html
+<div class="card p-6 rounded-lg border" style="background-color: var(--bg-secondary); border-color: var(--border)">
+    <div class="flex items-center space-x-3">
+        <div class="text-3xl">⚙️</div>
+        <div>
+            <h3 class="font-semibold" style="color: var(--text-primary)">Settings</h3>
+            <p style="color: var(--text-secondary)" class="text-sm">Configure your preferences</p>
+        </div>
+    </div>
+    <a href="/profile" class="mt-4 inline-block text-sm btn-secondary px-3 py-1 rounded">Manage Settings</a>
+</div>
+```
+
+New:
+```html
+<div class="card p-6 rounded-lg border" style="background-color: var(--bg-secondary); border-color: var(--border)">
+    <div class="flex items-center space-x-3">
+        <div class="text-3xl">👁️</div>
+        <div>
+            <h3 class="font-semibold" style="color: var(--text-primary)">Scan Watch Status</h3>
+            <p style="color: var(--text-secondary)" class="text-sm">Auto-detecting new files</p>
+        </div>
+    </div>
+    <div id="watch-status" class="mt-4 text-sm" style="color: var(--text-secondary)">
+        <span class="inline-block w-2 h-2 rounded-full bg-green-500 mr-2"></span>
+        Watching <span id="watch-count">0</span> libraries
+    </div>
+</div>
+```
+
+#### File: `web/src/admin.ts`
+
+**Change 8.2** - Add function to fetch and display watch status:
+
+```typescript
+async function loadWatchStatus() {
+    const token = localStorage.getItem('token');
+    if (!token) return;
+    
+    try {
+        const response = await fetch('/api/scanner/watch/status', {
+            headers: { 'Authorization': `Bearer ${token}` }
+        });
+        if (response.ok) {
+            const data = await response.json();
+            const countEl = document.getElementById('watch-count');
+            if (countEl) {
+                countEl.textContent = data.total_watching?.toString() || '0';
+            }
+        }
+    } catch (error) {
+        console.error('Failed to load watch status:', error);
+    }
+}
+
+function logout(): void {
+    localStorage.removeItem('token');
+    localStorage.removeItem('user');
+    window.location.href = '/';
+}
+
+// Initialize on page load
+document.addEventListener('DOMContentLoaded', function() {
+    loadWatchStatus();
+});
+```
+
+**Change 8.3** - Move inline script from admin.templ to admin.ts:
+
+Current admin.templ has inline script (lines 121-131):
+```html
+<script>
+    function logout() {
+        localStorage.removeItem('token');
+        localStorage.removeItem('user');
+        window.location.href = '/';
+    }
+
+    document.addEventListener('DOMContentLoaded', function() {
+        loadTheme();
+    });
+</script>
+```
+
+Remove the entire `<script>` block from `admin.templ`. The logout function and DOMContentLoaded are now in admin.ts.
+
+**Change 8.4** - Export functions to window in admin.ts:
+
+Add at the end of admin.ts (with other window exports):
+```typescript
+(window as any).loadWatchStatus = loadWatchStatus;
+(window as any).logout = logout;
+```
+
+#### Generated File: `templates/admin_templ.go`
+
+After editing `admin.templ`, rebuild the generated file:
+```bash
+templ generate templates
+```
+
+## Backward Compatibility
+
+- **Default behavior unchanged**: `force: false` maintains current skip-if-exists behavior
+- **Mobile apps**: Existing API consumers won't break (they just won't send `force` param)
+- **Breaking change**: None
+
+## Verification Steps
+
+After implementation:
+
+1. **Build verification**:
+   ```bash
+   go build ./...
+   ```
+
+2. **Run tests**:
+   ```bash
+   go test ./... -v
+   ```
+
+3. **Manual verification**:
+   - Create library with existing books
+   - Click "Rescan Library" button
+   - Verify metadata is re-extracted (check file hash, cover image, etc.)
+
+## Git Commit Structure
+
+1. **Backend: Add force parameter to scanner handler**
+   - `internal/handlers/scanner.go` - Add Force field to request struct and job params
+
+2. **Backend: Add force parameter to worker and media scanner**
+   - `internal/services/worker.go` - Extract force param
+   - `internal/services/media_scanner.go` - Add forceRescan field and SetForce method
+
+3. **Frontend: Update scan button to use force rescan**
+   - `web/src/admin.ts` - Send force: true in request body
+   - `templates/admin.templ` - Update button text to "Rescan Library"
+
+4. **Frontend: Add Watch Status display on admin page**
+   - `templates/admin.templ` - Replace Settings card with Watch Status
+   - `web/src/admin.ts` - Add loadWatchStatus function
+
+5. **Tests: Add unit and integration tests for force rescan**
+   - Add test cases to `internal/services/worker_test.go`
+   - Add integration tests to `cmd/server/tests/scanner_integration_test.go`
+   - Update `bruno/scanner/Scan Media Items.yml` with force parameter test
+
+6. **Docs: Update API documentation**
+   - Update `docs/developer/api/scanner/scan_library.md`
+   - Rebuild generated templates: `templ generate templates`