Files
bookhoard/SUMMARY.md
T
john-okeefe d802236874 scanner: fix library isolation, file mtime, force rescan, and deletion handling
Fix 1 - File modification time for created_at:
- Get file.ModTime() in processMediaFile and pass to CreateMediaItem
- Modified SQL INSERT to include created_at column

Fix 2 - Force rescan UPDATE instead of DELETE+INSERT:
- Changed force rescan logic to call updateMediaItem instead of delete + create
- Preserves created_at timestamp on force rescan

Fix 3 - GetMediaItemByFilePath filters by library_id:
- Added library_id to WHERE clause in SQL query
- Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader)
- Added SetLibraryID method to MediaScanner
- Updated handler to call SetLibraryID for watch mode

Fix 4 - File deletion handling with persistent logging:
- Added fsnotify.Remove handler in WatchChanges
- Added orphan cleanup in ScanFolders after scan completes
- Created scanner_logger.go with daily log rotation (7 days)
- Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log
- Individual deletes with enhanced safety logging

Note: Integration tests can now safely scan /app/uploads because
GetMediaItemByFilePath now filters by library_id, preventing
cross-library interference.
2026-02-26 16:39:42 -05:00

4.2 KiB

Bookhoard Scanner Issues - Summary and Fixes

Issues Identified

Issue 1: Scanner uses scan time instead of file modification time

Problem: When a new book is inserted into the database, created_at uses the database default (NOW()) which is the scan time, not when the file was actually added to the folder.

Example:

  • File Pride and Prejudice.epub was added to folder: Feb 2, 2026
  • Database shows created_at: Feb 26 17:51:19 (scan time)

Impact: "Recently Added" section doesn't reflect actual file modification dates.


Issue 2: Force rescan creates new entries instead of updating

Problem: When force rescan is enabled, the scanner DELETES the existing media item and re-INSERTs it, which creates a new created_at timestamp.

Location: internal/services/media_scanner.go lines 366-372

Current code:

if s.forceRescan {
    // Force update: delete existing and re-create
    if err := s.db.DeleteMediaItem(ctx, existingItem.ID); err != nil {
        fmt.Printf("Warning: failed to delete existing media item: %v\n", err)
    }
    // Continue to create new entry below
}

Impact: Force rescan resets created_at to scan time.


Issue 3: Scanner checks file_path globally, not per-library

Problem: GetMediaItemByFilePath query doesn't filter by library_id:

SELECT * FROM media_items WHERE file_path = $1;

Impact: When tests scan /app/uploads in their test library, they find and modify media items from YOUR manually created library because they share the same file path. This is why all your books show today's timestamps - tests constantly re-scan your uploads folder.


Issue 4: Watch mode doesn't handle file deletions

Problem: The watcher only handles fsnotify.Create and fsnotify.Write events, but not fsnotify.Remove.

Location: internal/services/media_scanner.go - WatchChanges() function (lines 1427-1467)

Impact: When a file is deleted from the OS, the database entry remains.


Issue 5: Rescan doesn't handle deleted books

Problem: When doing a full rescan, the scanner doesn't check for files that were deleted from the filesystem.

Impact: Orphaned media items remain in the database for files that no longer exist.


Fix 1: Use file modification time for created_at

File: internal/services/media_scanner.go

Change: Get file's actual modification time using os.Stat() and pass it to the INSERT query instead of relying on the database default.


Fix 2: Fix force rescan to use UPDATE instead of DELETE + INSERT

File: internal/services/media_scanner.go - lines 366-372

Change: Instead of deleting and re-creating, use an UPDATE query that preserves created_at and only updates updated_at.


Fix 3: Fix GetMediaItemByFilePath to filter by library_id

File: internal/database/queries/queries.sql - GetMediaItemByFilePath

Change: Add AND library_id = $2 to the query, and update the scanner to pass the library_id when checking for existing items.


Fix 4: Add file deletion handling to watch mode

File: internal/services/media_scanner.go - WatchChanges() function

Change: Add handler for fsnotify.Remove events to hard delete media items from the database when files are deleted from the OS.


Fix 5: Add deleted book handling to rescan

File: internal/services/media_scanner.go - ScanFolders() function

Change: After scanning the filesystem, compare the results against the database. Any media items whose files no longer exist should be hard deleted from the database.


Finding: The carousel order is actually correct. The database shows Beowulf as the newest book (created_at: 2026-02-26 17:51:21.977928+00), so it correctly appears first.

If you want oldest books on the left instead: Change ORDER BY created_at DESC to ORDER BY created_at ASC in internal/database/queries/queries.sql - GetRecentlyAddedItems query.


File Deletion Strategy

Decision: Hard delete (completely remove from database) for deleted files.

Note on file logs: Could be added later with a separate logging table if needed. Would require a mount point in Docker Compose.