bookhoard/force-rescan-plan.md

Implementation Plan: Force Rescan Feature

Overview

Add a force parameter to the library scan endpoint that allows re-processing existing media items. Currently, the scanner skips files that already exist in the database (based on file path). The force flag will bypass this check and re-process all files.

Current Behavior

When scanning a library:

1. ScanLibrary handler receives scan request
2. Worker enqueues a JobTypeScan job
3. processScanJob creates a MediaScanner and calls ScanFolders()
4. For each file, processMediaFile() checks if file exists in DB
5. If exists with same size → skip (return false, nil)
6. If exists with different size → update
7. If doesn't exist → create new

Proposed Changes

1. Backend: Add force parameter support

File: internal/handlers/scanner.go

Change 1.1 - Add Force field to ScanLibraryRequest struct (line ~89-92):

type ScanLibraryRequest struct {
    FolderPaths []string `json:"folder_paths,omitempty"`
    LibraryID   string   `json:"library_id,omitempty"`
    Force       bool     `json:"force,omitempty"`  // NEW: Force rescan of existing files
}

Change 1.2 - Pass force to job params in ScanLibrary function (line ~153):

job := &services.Job{
    ID:   jobID,
    Type: services.JobTypeScan,
    Params: map[string]interface{}{
        "library_id": req.LibraryID,
        "folders":    folderPaths,
        "admin_id":   userUUID.String(),
        "db":         h.db,
        "force":      req.Force,  // NEW
    },
    // ... rest unchanged
}

File: internal/services/worker.go

Change 2.1 - Extract force param in processScanJob function (after line ~195):

// Existing code:
db, ok := job.Params["db"].(*database.Queries)
if !ok {
    return nil, fmt.Errorf("database queries required")
}

// NEW: Extract force parameter
force := false
if forceVal, ok := job.Params["force"].(bool); ok {
    force = forceVal
}

Change 2.2 - Pass force to MediaScanner (after line ~220):

scanner.SetAdminID(adminUUID)

// NEW: Set force flag
scanner.SetForce(force)  // NEW

File: internal/services/media_scanner.go

Change 3.1 - Add forceRescan field to MediaScanner struct (find struct definition):

type MediaScanner struct {
    db           *database.Queries
    folders      []string
    adminID      pgtype.UUID
    job          *Job
    watcher      *fsnotify.Watcher
    totalFiles   int
    newItems     int
    errors       int
    forceRescan  bool  // NEW: Force re-scan of existing files
}

Change 3.2 - Add SetForce method (anywhere in file, after existing setters):

func (s *MediaScanner) SetForce(force bool) {
    s.forceRescan = force
}

Change 3.3 - Modify processMediaFile to respect force flag (line ~356-366):

Current code:

// Check if media item already exists in database
existingItem, err := s.getMediaItemByFilePath(ctx, path)
if err == nil {
    fmt.Printf("Media item already exists in database: %s (size: %d vs %d)\n", path, existingItem.FileSize.Int64, info.Size())
    // Media item exists, check if file has changed (by size)
    if existingItem.FileSize.Int64 != info.Size() {
        fmt.Printf("File size changed, updating media item: %s\n", path)
        _ = s.updateMediaItem(ctx, existingItem.ID, path, info)
        return false, nil
    }
    fmt.Printf("Media item already exists with same size, skipping: %s\n", path)
    return false, nil  // <-- THIS IS WHERE WE SKIP
}

New code:

// Check if media item already exists in database
existingItem, err := s.getMediaItemByFilePath(ctx, path)
if err == nil {
    fmt.Printf("Media item already exists in database: %s (size: %d vs %d)\n", path, existingItem.FileSize.Int64, info.Size())
    
    // If force rescan is enabled, always re-process
    if s.forceRescan {
        fmt.Printf("Force rescan enabled, re-processing existing media item: %s\n", path)
        // Force update: delete existing and re-create
        if err := s.db.DeleteMediaItem(ctx, existingItem.ID); err != nil {
            fmt.Printf("Warning: failed to delete existing media item: %v\n", err)
        }
        // Continue to create new entry below
    } else {
        // Normal behavior: check if file has changed (by size)
        if existingItem.FileSize.Int64 != info.Size() {
            fmt.Printf("File size changed, updating media item: %s\n", path)
            _ = s.updateMediaItem(ctx, existingItem.ID, path, info)
            return false, nil
        }
        fmt.Printf("Media item already exists with same size, skipping: %s\n", path)
        return false, nil
    }
}

2. Frontend: Update button and API call

File: web/src/admin.ts

Change 4.1 - Update scanAllLibraries function to send force: true (line ~129):

Current code:

const scanResp = await fetch(`/api/libraries/${lib.id}/scan`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${token}` }
});

New code:

const scanResp = await fetch(`/api/libraries/${lib.id}/scan`, {
    method: 'POST',
    headers: { 
        'Authorization': `Bearer ${token}`,
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({ force: true })
});

File: templates/admin.templ

Change 5.1 - Update button text and description (line ~55-58):

Current:

<button onclick="scanAllLibraries()" class="btn-primary p-4 rounded-lg text-left">
    <div class="font-medium">Scan Library</div>
    <div style="color: var(--text-secondary)" class="text-sm">Find new ebooks in your folders</div>
</button>

New:

<button onclick="scanAllLibraries()" class="btn-primary p-4 rounded-lg text-left">
    <div class="font-medium">Rescan Library</div>
    <div style="color: var(--text-secondary)" class="text-sm">Re-scan existing files and fix metadata</div>
</button>

3. API Documentation

File: docs/developer/api/scanner/scan_library.md

Change 6.1 - Update documentation to reflect force parameter (line ~15):

Current:

| force | boolean | No | Force rescan of existing files (default: false) |

This is already documented. Ensure the description is accurate:

| force | boolean | No | Force rescan of existing files. When true, re-processes all files in library regardless of whether they already exist in database (default: false) |

4. Tests

Unit Tests: internal/services/worker_test.go

Change 7.1 - Add test case for force parameter in existing scan job tests:

func TestWorker_ProcessJob_ScanJob_ForceRescan(t *testing.T) {
    // Setup test with existing media item in database
    // Create job with force: true
    // Verify media item is re-processed
}

Integration Tests: cmd/server/tests/scanner_integration_test.go

Change 7.2 - Add integration tests covering three contexts (per guidelines line 140):

Follow existing pattern from scanner_integration_test.go:

func TestScanLibrary_ForceFlag(t *testing.T) {
    s := setupTestServer(t)
    defer s.TearDown()
    
    // Test 1: No user context (unauthenticated) - expect 401
    // Test 2: Regular user context - expect 403 (admin only)
    // Test 3: Admin context with force=true - expect 200, verify re-scan
    // Test 4: Admin context with force=false - expect 200, verify skip
}

Bruno OpenCollection YAML Tests

Change 7.3 - Add force parameter test case to existing Bruno test:

File: bruno/scanner/Scan Media Items.yml

Add a new test request or modify existing to include:

body: {
  "force": true
}

5. Frontend: Add Watch Status Display

Replace the "Settings" card on /admin page with a Watch Status display.

File: templates/admin.templ

Change 8.1 - Replace Settings card with Watch Status (lines 38-47):

Current:

<div class="card p-6 rounded-lg border" style="background-color: var(--bg-secondary); border-color: var(--border)">
    <div class="flex items-center space-x-3">
        <div class="text-3xl">⚙️</div>
        <div>
            <h3 class="font-semibold" style="color: var(--text-primary)">Settings</h3>
            <p style="color: var(--text-secondary)" class="text-sm">Configure your preferences</p>
        </div>
    </div>
    <a href="/profile" class="mt-4 inline-block text-sm btn-secondary px-3 py-1 rounded">Manage Settings</a>
</div>

New:

<div class="card p-6 rounded-lg border" style="background-color: var(--bg-secondary); border-color: var(--border)">
    <div class="flex items-center space-x-3">
        <div class="text-3xl">👁️</div>
        <div>
            <h3 class="font-semibold" style="color: var(--text-primary)">Scan Watch Status</h3>
            <p style="color: var(--text-secondary)" class="text-sm">Auto-detecting new files</p>
        </div>
    </div>
    <div id="watch-status" class="mt-4 text-sm" style="color: var(--text-secondary)">
        <span class="inline-block w-2 h-2 rounded-full bg-green-500 mr-2"></span>
        Watching <span id="watch-count">0</span> libraries
    </div>
</div>

File: web/src/admin.ts

Change 8.2 - Add function to fetch and display watch status:

async function loadWatchStatus() {
    const token = localStorage.getItem('token');
    if (!token) return;
    
    try {
        const response = await fetch('/api/scanner/watch/status', {
            headers: { 'Authorization': `Bearer ${token}` }
        });
        if (response.ok) {
            const data = await response.json();
            const countEl = document.getElementById('watch-count');
            if (countEl) {
                countEl.textContent = data.total_watching?.toString() || '0';
            }
        }
    } catch (error) {
        console.error('Failed to load watch status:', error);
    }
}

function logout(): void {
    localStorage.removeItem('token');
    localStorage.removeItem('user');
    window.location.href = '/';
}

// Initialize on page load
document.addEventListener('DOMContentLoaded', function() {
    loadWatchStatus();
});

Change 8.3 - Move inline script from admin.templ to admin.ts:

Current admin.templ has inline script (lines 121-131):

<script>
    function logout() {
        localStorage.removeItem('token');
        localStorage.removeItem('user');
        window.location.href = '/';
    }

    document.addEventListener('DOMContentLoaded', function() {
        loadTheme();
    });
</script>

Remove the entire <script> block from admin.templ. The logout function and DOMContentLoaded are now in admin.ts.

Change 8.4 - Export functions to window in admin.ts:

Add at the end of admin.ts (with other window exports):

(window as any).loadWatchStatus = loadWatchStatus;
(window as any).logout = logout;

Generated File: templates/admin_templ.go

After editing admin.templ, rebuild the generated file:

templ generate templates

Backward Compatibility

• Default behavior unchanged: force: false maintains current skip-if-exists behavior
• Mobile apps: Existing API consumers won't break (they just won't send force param)
• Breaking change: None

Verification Steps

After implementation:

1. Build verification:

   go build ./...
   

2. Run tests:

   go test ./... -v
   

3. Manual verification:

   • Create library with existing books
   • Click "Rescan Library" button
   • Verify metadata is re-extracted (check file hash, cover image, etc.)

Git Commit Structure

1. Backend: Add force parameter to scanner handler

   • internal/handlers/scanner.go - Add Force field to request struct and job params

2. Backend: Add force parameter to worker and media scanner

   • internal/services/worker.go - Extract force param
   • internal/services/media_scanner.go - Add forceRescan field and SetForce method

3. Frontend: Update scan button to use force rescan

   • web/src/admin.ts - Send force: true in request body
   • templates/admin.templ - Update button text to "Rescan Library"

4. Frontend: Add Watch Status display on admin page

   • templates/admin.templ - Replace Settings card with Watch Status
   • web/src/admin.ts - Add loadWatchStatus function

5. Tests: Add unit and integration tests for force rescan

   • Add test cases to internal/services/worker_test.go
   • Add integration tests to cmd/server/tests/scanner_integration_test.go
   • Update bruno/scanner/Scan Media Items.yml with force parameter test

6. Docs: Update API documentation

   • Update docs/developer/api/scanner/scan_library.md
   • Rebuild generated templates: templ generate templates