john-okeefe
d802236874
Fix 1 - File modification time for created_at: - Get file.ModTime() in processMediaFile and pass to CreateMediaItem - Modified SQL INSERT to include created_at column Fix 2 - Force rescan UPDATE instead of DELETE+INSERT: - Changed force rescan logic to call updateMediaItem instead of delete + create - Preserves created_at timestamp on force rescan Fix 3 - GetMediaItemByFilePath filters by library_id: - Added library_id to WHERE clause in SQL query - Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader) - Added SetLibraryID method to MediaScanner - Updated handler to call SetLibraryID for watch mode Fix 4 - File deletion handling with persistent logging: - Added fsnotify.Remove handler in WatchChanges - Added orphan cleanup in ScanFolders after scan completes - Created scanner_logger.go with daily log rotation (7 days) - Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log - Individual deletes with enhanced safety logging Note: Integration tests can now safely scan /app/uploads because GetMediaItemByFilePath now filters by library_id, preventing cross-library interference.
118 lines
4.2 KiB
Markdown
118 lines
4.2 KiB
Markdown
# Bookhoard Scanner Issues - Summary and Fixes
|
|
|
|
## Issues Identified
|
|
|
|
### Issue 1: Scanner uses scan time instead of file modification time
|
|
**Problem:** When a new book is inserted into the database, `created_at` uses the database default (`NOW()`) which is the scan time, not when the file was actually added to the folder.
|
|
|
|
**Example:**
|
|
- File `Pride and Prejudice.epub` was added to folder: Feb 2, 2026
|
|
- Database shows `created_at`: Feb 26 17:51:19 (scan time)
|
|
|
|
**Impact:** "Recently Added" section doesn't reflect actual file modification dates.
|
|
|
|
---
|
|
|
|
### Issue 2: Force rescan creates new entries instead of updating
|
|
**Problem:** When force rescan is enabled, the scanner DELETES the existing media item and re-INSERTs it, which creates a new `created_at` timestamp.
|
|
|
|
**Location:** `internal/services/media_scanner.go` lines 366-372
|
|
|
|
**Current code:**
|
|
```go
|
|
if s.forceRescan {
|
|
// Force update: delete existing and re-create
|
|
if err := s.db.DeleteMediaItem(ctx, existingItem.ID); err != nil {
|
|
fmt.Printf("Warning: failed to delete existing media item: %v\n", err)
|
|
}
|
|
// Continue to create new entry below
|
|
}
|
|
```
|
|
|
|
**Impact:** Force rescan resets `created_at` to scan time.
|
|
|
|
---
|
|
|
|
### Issue 3: Scanner checks file_path globally, not per-library
|
|
**Problem:** `GetMediaItemByFilePath` query doesn't filter by `library_id`:
|
|
|
|
```sql
|
|
SELECT * FROM media_items WHERE file_path = $1;
|
|
```
|
|
|
|
**Impact:** When tests scan `/app/uploads` in their test library, they find and modify media items from YOUR manually created library because they share the same file path. This is why all your books show today's timestamps - tests constantly re-scan your uploads folder.
|
|
|
|
---
|
|
|
|
### Issue 4: Watch mode doesn't handle file deletions
|
|
**Problem:** The watcher only handles `fsnotify.Create` and `fsnotify.Write` events, but not `fsnotify.Remove`.
|
|
|
|
**Location:** `internal/services/media_scanner.go` - `WatchChanges()` function (lines 1427-1467)
|
|
|
|
**Impact:** When a file is deleted from the OS, the database entry remains.
|
|
|
|
---
|
|
|
|
### Issue 5: Rescan doesn't handle deleted books
|
|
**Problem:** When doing a full rescan, the scanner doesn't check for files that were deleted from the filesystem.
|
|
|
|
**Impact:** Orphaned media items remain in the database for files that no longer exist.
|
|
|
|
---
|
|
|
|
## Recommended Fixes
|
|
|
|
### Fix 1: Use file modification time for created_at
|
|
|
|
**File:** `internal/services/media_scanner.go`
|
|
|
|
**Change:** Get file's actual modification time using `os.Stat()` and pass it to the INSERT query instead of relying on the database default.
|
|
|
|
---
|
|
|
|
### Fix 2: Fix force rescan to use UPDATE instead of DELETE + INSERT
|
|
|
|
**File:** `internal/services/media_scanner.go` - lines 366-372
|
|
|
|
**Change:** Instead of deleting and re-creating, use an UPDATE query that preserves `created_at` and only updates `updated_at`.
|
|
|
|
---
|
|
|
|
### Fix 3: Fix GetMediaItemByFilePath to filter by library_id
|
|
|
|
**File:** `internal/database/queries/queries.sql` - `GetMediaItemByFilePath`
|
|
|
|
**Change:** Add `AND library_id = $2` to the query, and update the scanner to pass the library_id when checking for existing items.
|
|
|
|
---
|
|
|
|
### Fix 4: Add file deletion handling to watch mode
|
|
|
|
**File:** `internal/services/media_scanner.go` - `WatchChanges()` function
|
|
|
|
**Change:** Add handler for `fsnotify.Remove` events to hard delete media items from the database when files are deleted from the OS.
|
|
|
|
---
|
|
|
|
### Fix 5: Add deleted book handling to rescan
|
|
|
|
**File:** `internal/services/media_scanner.go` - `ScanFolders()` function
|
|
|
|
**Change:** After scanning the filesystem, compare the results against the database. Any media items whose files no longer exist should be hard deleted from the database.
|
|
|
|
---
|
|
|
|
## Dashboard Carousel Ordering
|
|
|
|
**Finding:** The carousel order is actually correct. The database shows Beowulf as the newest book (created_at: 2026-02-26 17:51:21.977928+00), so it correctly appears first.
|
|
|
|
**If you want oldest books on the left instead:** Change `ORDER BY created_at DESC` to `ORDER BY created_at ASC` in `internal/database/queries/queries.sql` - `GetRecentlyAddedItems` query.
|
|
|
|
---
|
|
|
|
## File Deletion Strategy
|
|
|
|
**Decision:** Hard delete (completely remove from database) for deleted files.
|
|
|
|
**Note on file logs:** Could be added later with a separate logging table if needed. Would require a mount point in Docker Compose.
|