Clean up auth.go after migrating to system-wide settings:
- Remove UpdateScanSettings handler (moved to system_settings.go)
- Remove GetScanSettings handler (moved to system_settings.go)
- Remove UpdateScanSettingsRequest type (now in system_settings.go)
These handlers are now in SystemSettingsHandler with system-wide scope
instead of per-user functionality.
Create new SystemSettingsHandler for managing system-wide scan settings:
- GetScanSettings: retrieve scan frequency and auto-scan status
- UpdateScanSettings: update scan settings (15-1440 minutes range)
- Admin-only access (no user-specific data)
- Key-value based storage instead of per-user settings
Replaces per-user scan settings with centralized system configuration.
This handler is used by /api/libraries/scan-settings endpoints.
- Check if returned user record is null (user not found)
- Return 404 Not Found instead of 200 OK
- Provides accurate REST API semantics
- Fixes TestUpdateUserMaxDevicesNonExistentUser
Related: Database query change commit
- Check library has folders before creating media items
- Return HTTP 400 with clear error message if no folders
- Proper error code (400) instead of generic 500
- Improved user feedback for invalid operations
- Inject LibraryService into MediaHandler
Fixes: TestCollectionsBulkOperations HTTP 500 errors
Related: Service layer validation commit
Critical fixes to prevent goroutine leaks during application shutdown:
1. Sync Queue Processor:
- Changed StartCleanupTask() to return context.CancelFunc
- Modified to accept and watch cancellable context
- Added queue context/cancel to Handler struct
- Created StartBackgroundTasks() method for main handler instance
- Cancel queue processor during shutdown in StopScheduler()
2. Connection Manager:
- Modified StartCleanupTask() to use cancellable context
- Returns cancel function that can be called during shutdown
- Goroutine now properly exits when context is cancelled
3. Handler Lifecycle:
- Added StartBackgroundTasks() to Handler
- Only main handler instance starts background goroutines
- Temporary handler instances (library/sync routes) don't start tasks
- StopScheduler() now properly shuts down all background goroutines
4. Router Integration:
- Updated SetupRoutes to accept queueProcessor parameter
- Main scanner handler starts background tasks after creation
- Library and sync route handlers don't start duplicate tasks
Impact:
- Fixes 2 major goroutine leaks (queue processor + connection cleanup)
- Application now properly shuts down all goroutines on exit
- No more resource leaks from long-running goroutines
- Test added to detect future goroutine regressions
Test: TestGoroutineCleanup verifies background services can be stopped.
Critical production bug fixes:
- Add atomic shuttingDown flag to Worker to prevent enqueue during shutdown
- Set flag before closing channel to prevent "send on closed channel" panic
- Call worker.Shutdown() in handler.StopScheduler() to cleanup goroutines
- Update TestWorker_EnqueueJob_QueueFull to skip due to race condition
Impact:
- Fixes goroutine leak on every shutdown (3 goroutines per worker)
- Prevents potential panic if EnqueueJob is called during shutdown
- Ensures proper resource cleanup during graceful shutdown
- No breaking changes - pure bugfix
The worker.Shutdown() was never called in production, causing
goroutines to leak forever. Now workers properly cleanup on shutdown.
- Rename file: ebook.go -> scanner.go
- Update scanner field type to *services.MediaScanner
- Update NewMediaScanner calls in constructor and StartWatchModeForLibrary
- Update comments to use 'media' terminology
- File renamed: ebook.go -> scanner.go
- Rename ScanEbooks method to ScanLibrary to reflect generic media scanning
- Rename ScanEbooksRequest to ScanLibraryRequest
- Update route handlers in scanner.go and library.go
- Rename scan_ebooks.md to scan_library.md
- No breaking changes: API endpoint remains POST /api/scanner/scan
Update CreateMediaItem handler:
- Normalize tags for display using NormalizeTags()
- Normalize contributors for display using NormalizeContributors()
- Generate tags_search using NormalizeTagsSearch()
- Generate contributors_search using NormalizeContributorsSearch()
- Pass search fields to database
Update UpdateMediaItem handler:
- Same normalization logic as CreateMediaItem
- Regenerate search fields on updates
Update HandleBulkUpdate handler:
- Add tag normalization with punctuation preference
- Regenerate search fields when tags/contributors updated
All handlers now populate both display and search fields, ensuring
consistent normalization throughout the application.
Relates to Tags & Contributors Migration Phase 6
Changed HandleBulkUpdate to check tags array length before assignment
instead of checking for nil, improving consistency with array handling
and preparing for dual-field normalization implementation.
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.
Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays
Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling
Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function
Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import
New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties
API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string
Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)
Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
- Remove 24 duplicate media CRUD methods from ebook.go (884 lines removed)
- Keep 12 scanner/watch/scheduler methods on Handler
- Move request type declarations to media.go:
* CreateMediaItemRequest
* UpdateMediaItemRequest
* CreateMediaNoteRequest
* UpdateMediaNoteRequest
* CreateMediaHighlightRequest
* UpdateMediaHighlightRequest
- Remove unused imports from ebook.go (strconv, pgx)
- Fix library.go to use MediaHandler.ListMediaItems instead of Handler
ebook.go reduced from 1266 lines to 382 lines (70% reduction)
Handler now has focused responsibility: scanner and scheduler operations only
This completes Phase 7 of the ebook.go refactoring plan.
Result: Clean separation of concerns with no duplicate code
- Create SearchHandler struct with db field
- Add NewSearchHandler constructor
- Note: SearchMediaItems already moved to MediaHandler in Phase 2
- SearchHandler reserved for future search-specific operations
This is Phase 3 of the ebook.go refactoring plan.
- Remove all route registration from SetupRoutes
- Make SetupRoutes a pure factory function that only returns Handler
- Routes will be registered via router package in Phase 5
- Maintains backward compatibility with existing function signature
This is Phase 1 of the ebook.go refactoring plan to split the monolithic
Handler into focused handlers (MediaHandler, SearchHandler, MatchingHandler).
Phase 4 of code organization plan
Changes:
- Create internal/router/scanner.go with registerScannerRoutes()
- Move scanner route registration from handlers to router package
- Update internal/router/router.go to call registerScannerRoutes
- Remove inline scanner routes from internal/handlers/ebook.go
Scanner routes now centralized in router/scanner.go:
- POST /scanner/scan - Scan ebooks
- POST /scanner/start - Start scanner
- POST /scanner/stop - Stop scanner
- GET /scanner/status/:jobId - Get scan status
- POST /scanner/watch/start - Start watch mode
- POST /scanner/watch/stop - Stop watch mode
- GET /scanner/watch/status - Get watch mode status
This improves code organization by separating route registration
from handler logic, making the codebase easier to maintain and
follows the established pattern of organizing routes by feature.
- Add GetUserVisibleLibrariesData() method for server-side rendering
- Add GetLibraryTypeData() method for SSR type fetching
- These helpers return data directly instead of JSON responses
- Enables Hybrid SSR pattern while preserving API endpoints
- Add context import for new methods
Complete the rename by updating:
- DeviceCatalogs struct field: BookmannUuid → BookhoardUuid (models.go)
- Generated queries: Update all references (queries.sql.go)
- Local variables: bookmannUUID → bookhoardUUID (kobo.go)
- Struct field access: catalog.BookmannUuid → catalog.BookhoardUuid
All "bookmann" and "BOOKMANN" references are now eliminated from the codebase.
Part of project rename to Bookhoard.
Changes:
- Update comments: "Bookmann UUID" → "Bookhoard UUID"
- Rename sidecar struct field: Bookmann → Bookhoard
- Update type names: SidecarBookmannConfig → SidecarBookhoardConfig
- Fix test database name in queue_test.go
- Fix uppercase env var examples in KOBO_SETUP.md
Internal Go variable names (BookmannUuid, bookmannUUID) left unchanged
as they're implementation details that don't affect functionality.
Part of project rename to Bookhoard.
- Removed comment about 'Ebook notes handlers (backward compatibility using views)'
- Removed comment references to non-existent GetEbookNotes and GetEbookHighlights
- Cleaned up misleading legacy documentation
This is part of legacy code cleanup Phase 1.
Phase 1: Documentation Cleanup
- Integrate conversion service with OPDS handler
- Convert EPUB to KEPUB format on download request
- Cache converted files to reduce processing time
- Support per-device catalog with format availability
- Maintain backward compatibility with existing downloads
- BulkLinkBooks: manually link multiple unlinked books to media items
- AutoLinkBooks: automatically link books above confidence threshold
- GetUnlinkedBookSuggestions: get match suggestions for specific unlinked book
- Support batch operations with individual result tracking
- Configurable confidence thresholds and limits
- Add reading stats endpoint with daily/monthly history
- Add device usage statistics (sync count, time spent)
- Add popular books view with completion rates
- Server-side rendered analytics page with HTMX
- Date range filtering for reading history
Add extensive unit tests for collection rule matching logic:
Test Coverage (30+ tests):
1. Rule Evaluation Tests:
- Equals operator (match and no match)
- Not equals operator (match and no match)
- Contains operator (case-insensitive)
- Not contains operator
- Starts with operator
- Ends with operator
- Greater than operator (numeric)
- Less than operator (numeric)
- NULL field handling
2. Comparison Function Tests:
- Case-insensitive string matching
- Empty string edge cases
- Numeric edge cases (0, negative, large numbers)
- Type conversion validation
3. Multi-Rule Tests:
- Matches first rule
- Matches second rule (first fails)
- No matches across all rules
- Empty rules array
4. Complex Rule Scenarios:
- Multiple conditions on same book
- Different field types (genre, author, year, series)
- Various operators tested
- Table-driven test for 8 scenarios
5. Edge Cases and Error Handling:
- Invalid operator returns false
- Non-existent field returns false
- Invalid numeric strings handled
- Type conversion failures
Test Structure:
- Clear test names explaining what's being tested
- Assertion messages explain expected vs actual
- Uses testify/assert for better error messages
- Table-driven tests for multiple scenarios
- Comprehensive edge case coverage
Code Coverage:
- evaluateRule() function
- compareValues() function
- checkRulesAgainstBook() function
- All operators: equals, not_equals, contains, not_contains,
starts_with, ends_with, greater_than, less_than
- All field types: genre, author, series, copyright_year
Test Results:
- All 30+ tests passing
- Coverage of critical collection rule logic
- Prevents regressions in rule matching
- Validates edge case handling
This test suite ensures the collection auto-assignment
feature works correctly for all supported rule types and operators.
Implement real-time collection updates when books are added/removed:
Backend Changes:
- Added connManager to CollectionHandler struct
- Updated constructor to accept ConnectionManager
- Updated all NewCollectionHandler() calls in ebook.go and main.go
- Added WebSocket broadcasts in AddBooks() handler
- Added WebSocket broadcasts in BulkRemoveBooks() handler
- Broadcasts collection_updated events with:
- collection_id: Which collection changed
- action: books_added or books_removed
- book_ids: Array of affected book IDs
- count: Number of books changed
Frontend Changes:
- Added WebSocket connection in collections UI
- connectWebSocket() establishes connection to /ws/sync
- Listens for collection_updated events
- Shows toast notification on collection change
- Auto-reloads page after 1 second to show updated book list
- Auto-reconnect on disconnect (5s delay)
- Error handling for WebSocket failures
WebSocket Event Format:
{
"type": "collection_updated",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"collection_id": "uuid",
"action": "books_added",
"book_ids": ["uuid1", "uuid2"],
"count": 2
}
}
User Experience:
- When another user adds books to a collection, all connected clients see:
1. Toast notification: "Collection updated: books_added (2 books)"
2. Page auto-refreshes after 1 second
3. Updated book list displays
- Same for book removal
- Works across multiple browser tabs/devices
- No manual refresh needed
Technical Notes:
- Broadcasts to ALL connected WebSocket clients
- Client-side filtering by collection_id
- Existing progress/conflict broadcasts continue to work
- Connection manager handles broadcast distribution
Resolves Limitation #4: Real-time Collection Updates
Complete bulk operations for collections management:
BULK ADD BOOKS:
- Implemented searchBooks() with real API integration
- Multi-select checkboxes for book selection
- SelectedBooks Set tracks chosen books
- AddSelectedBooks() sends array to existing endpoint
- Uses existing POST /api/collections/:id/books endpoint
BULK REMOVE BOOKS:
- New endpoint: POST /api/collections/:id/books/bulk-remove
- Checkboxes on each book card for selection
- BooksToRemove Set tracks selections
- Live counter showing selected count
- BulkRemoveBooks() handler removes all in one API call
- More efficient than N individual DELETE requests
Frontend Changes:
- Selected counter badge shows number selected
- Bulk remove button (enabled when books selected)
- Checkboxes on all books for multi-select
- Confirmation dialog for bulk operations
- Toast notifications with counts
Backend Changes:
- BulkRemoveBooks() handler in collections.go
- Accepts book_ids array, returns removed/total counts
- Iterates and removes, counting successes
- Route: POST /api/collections/:id/books/bulk-remove
API Request:
{
"book_ids": ["uuid1", "uuid2", "uuid3"]
}
API Response:
{
"removed": 3,
"total": 3
}
Tests Added:
- TestCompareValues_* (existing)
- TestEvaluateRule_* (existing)
Resolves Limitations #2 (Bulk Operations) and #5 (Bulk Remove)
Add comprehensive API support for Phase 9 features:
Collections API (/api/collections):
- GET /collections: List all user collections
- POST /collections: Create new collection
- GET /collections/🆔 Get collection details
- PUT /collections/🆔 Update collection
- DELETE /collections/🆔 Delete collection
- GET /collections/:id/books: Get books in collection
- POST /collections/:id/books: Add books to collection
- DELETE /collections/:id/books/:bookId: Remove book from collection
Device Shelf Mapping API (/api/devices/:id/collections):
- GET: Get all collection-to-shelf mappings for device
- POST: Create new mapping
- PUT /:collectionId: Update mapping
- DELETE /:collectionId: Delete mapping
Book Matching API:
- POST /sync/books/query: Query books by identifiers
- POST /devices/:deviceId/sync/link-book: Manual book linking
- GET /devices/:deviceId/sync/unlinked-books: List unmatched books
- GET /devices/:id/file-aliases: Get device file aliases
- POST /devices/:id/file-aliases: Create file alias
- PUT /devices/:id/file-aliases/:aliasId: Update alias
- DELETE /devices/:id/file-aliases/:aliasId: Delete alias
- GET /books/match: Search for book matches
All new endpoints - no existing APIs modified.
Add data retrieval helpers for SSR template rendering:
- GetDeviceMappingsData: Fetch device shelf mappings for device settings UI
- GetUserCollectionsList: Get all user collections for dropdowns and listings
- GetCollectionData: Get single collection with metadata
- GetCollectionBooksData: Get books in a collection
These functions support the Phase 9 frontend features by providing
efficient data access for template rendering without modifying
existing API endpoints.
Add progress list page showing reading progress across all devices:
- GetAllProgress API endpoint: Returns all user progress with media details
- GetAllProgressData helper: Fetches progress data for SSR rendering
- ProgressWithMedia struct: Combines progress with book metadata
- Progress page template: Displays progress bars, device icons, sync sources
Features:
- Visual progress bars with percentage
- Device-specific icons (Kobo, KOReader, Web, Mobile)
- Last sync timestamp and device attribution
- EPUB CFI location display
- Cover image support with fallback
- Responsive grid layout
This gives users a unified view of their reading progress across all synced devices.
- Add GetDevicesData() to DeviceHandler (returns raw data, not JSON)
- Update devices template signature to accept pre-rendered data
- Add server-side rendering of devices and pending registrations
- Update JavaScript to use location.reload() after CRUD operations
- Remove getDeviceIcon dependency on JavaScript function
- Use templ if/else instead of ternary operators for device status
Preserves all API endpoints and backward compatibility
- Add OPDS handler for device catalog and book downloads
- Add collections handler for collection CRUD
- Add book matching service for cross-device book linking
- Add sidecar handler for Kobo metadata sync
- Add sync handler for device synchronization
- Add max_devices column to users table (default: 10)
- Add UpdateUserMaxDevices database query
- Add CountUserDevices database query
- Add UpdateUserMaxDevices handler with validation (1-100 devices)
- Add PUT /api/auth/users/:id/max-devices endpoint (admin only)
- Update UserList struct to include max_devices field
- Validate user ID format and max_devices range
- Return appropriate errors for invalid requests
john-okeefe