Commit Graph
11 Commits
Author SHA1 Message Date
john-okeefe 9b3d8cc949 feat: implement collection library filter with WebSocket improvements and test coverage
This commit adds comprehensive functionality for filtering collections by library,
improves WebSocket real-time updates with user activity detection, and adds
extensive test coverage.

## Core Features

### Collection Library Filter
- Added library_id parameter to media-items search API
- Collections can now be filtered by specific library
- Toggle UI component for enabling/disabling library filter
- Default state is "checked" when library_id is present
- Consistent behavior across partial and fuzzy search modes

### WebSocket Auto-Reload Mitigation
- Added user activity detection to prevent disruptive page reloads
- Checks if user is actively typing in INPUT/TEXTAREA/SELECT elements
- Skips auto-reload when user is interacting with form elements
- Toast notifications still show for awareness
- Prevents data loss during editing operations

## Implementation Changes

### Backend
- internal/database/queries.sql.go: Added library filter support to search queries
- internal/handlers/media.go: Enhanced search with library_id parameter validation
- internal/handlers/collections.go: Updated collection handlers with library filtering
- internal/sync/websocket.go: Improved broadcast mechanism with user-scoped updates
- internal/router/frontend.go: Pass libraryID to collection templates

### Frontend
- templates/collections.templ: Added library filter toggle UI component
- web/src/collections.ts: TypeScript implementation with WebSocket integration
- templates/collections_templ.go: Generated template code

### Testing
- cmd/server/tests/search_test.go: Added TestCollectionSearchLibraryFilter
- cmd/server/tests/websocket_test.go: Added TestWebSocketUserScopedBroadcast
- New helper functions for creating libraries and media items via API
- Comprehensive test coverage for library filtering and user-scoped broadcasts

## API Documentation Updates

### Bruno Tests (Comprehensive Documentation)
- bruno/collections/*: Added detailed API documentation for all collection endpoints
- bruno/devices/*: Added device management and sync API documentation
- bruno/devices/kobo/api.yml: Kobo-specific sync protocol docs
- bruno/devices/koreader/api.yml: KOReader-specific sync protocol docs
- bruno/opds/*: Added OPDS feed and download endpoint documentation
- bruno/library/browse-folders.yml: Library folder browsing API docs

### New Bruno Tests
- bruno/media-items/Search All Libraries.yml: Test search without library filter
- bruno/media-items/Search Specific Library.yml: Test search with library filter
- bruno/media-items/Search Invalid Library ID.yml: Test error handling

## Documentation

- docs/developer/api/media-items/search_media_items.md: Updated with library_id parameter
- IMPLEMENTATION_COLLECTION_FIX.md: Comprehensive implementation guide with test scenarios

## Testing

### Integration Tests
- Library filter tests verify correct filtering across multiple libraries
- Invalid library_id tests ensure proper error handling
- WebSocket tests verify user-scoped broadcast behavior
- User A no longer receives User B's collection updates

### Manual Testing Scenarios
- Open collection in multiple tabs - updates propagate correctly
- Type in search box while another tab adds books - no disruptive reload
- Add/remove books from collection - toast notifications appear
- Toggle library filter - results update dynamically

## Technical Details

- WebSocket broadcasts are now user-scoped for privacy
- Active element detection uses tagName and contenteditable attributes
- Library ID validation uses UUID format checking
- Progressive enhancement maintained - page works without JavaScript
- All changes follow PROJECT_GUIDELINES.md conventions
- TypeScript only for frontend logic
- TailwindCSS only for styling
- Procedural programming style throughout

## Breaking Changes

None - all changes are additive and backward compatible.
2026-03-04 22:37:47 -05:00
john-okeefe 001647cbbe Fix goroutine leaks in sync queue processor and connection manager
Critical fixes to prevent goroutine leaks during application shutdown:

1. Sync Queue Processor:
   - Changed StartCleanupTask() to return context.CancelFunc
   - Modified to accept and watch cancellable context
   - Added queue context/cancel to Handler struct
   - Created StartBackgroundTasks() method for main handler instance
   - Cancel queue processor during shutdown in StopScheduler()

2. Connection Manager:
   - Modified StartCleanupTask() to use cancellable context
   - Returns cancel function that can be called during shutdown
   - Goroutine now properly exits when context is cancelled

3. Handler Lifecycle:
   - Added StartBackgroundTasks() to Handler
   - Only main handler instance starts background goroutines
   - Temporary handler instances (library/sync routes) don't start tasks
   - StopScheduler() now properly shuts down all background goroutines

4. Router Integration:
   - Updated SetupRoutes to accept queueProcessor parameter
   - Main scanner handler starts background tasks after creation
   - Library and sync route handlers don't start duplicate tasks

Impact:
- Fixes 2 major goroutine leaks (queue processor + connection cleanup)
- Application now properly shuts down all goroutines on exit
- No more resource leaks from long-running goroutines
- Test added to detect future goroutine regressions

Test: TestGoroutineCleanup verifies background services can be stopped.
2026-02-09 13:12:31 -05:00
john-okeefe 91b09c6d40 Fix sync package unit test failures
- TestCalculateNextRetry: allow small negative delay for attempt 0
  (immediate retry causes timing-based test flakiness)
- TestPriorityConstants: change assertions from int32 to int
  (constants are untyped int, not int32)
- TestOfflineDetector_*: Move integration tests to cmd/server/tests/

These tests were failing due to type mismatches and timing issues.
All are now fixed and passing.
2026-02-09 10:45:34 -05:00
john-okeefe 5b9f21a592 Final cleanup: Update remaining comments and variable names
Changes:
- Update comments: "Bookmann UUID" → "Bookhoard UUID"
- Rename sidecar struct field: Bookmann → Bookhoard
- Update type names: SidecarBookmannConfig → SidecarBookhoardConfig
- Fix test database name in queue_test.go
- Fix uppercase env var examples in KOBO_SETUP.md

Internal Go variable names (BookmannUuid, bookmannUUID) left unchanged
as they're implementation details that don't affect functionality.

Part of project rename to Bookhoard.
2026-02-01 16:24:58 -05:00
john-okeefe 00a083b60b Rename backend code references: Bookmann → Bookhoard
Backend changes:
- Update import paths: bookmann/internal → bookhoard/internal
- Rename struct fields: BookmannUUID → BookhoardUUID
- Update handler function names: mapContentIdToBookmannUUID → mapContentIdToBookhoardUUID
- Update HTTP response headers: X-Bookmann-* → X-Bookhoard-*
- Update service and middleware references
- Update main.go imports and references

This is part 2 of the project rename to Bookhoard.
2026-02-01 16:11:54 -05:00
john-okeefe 11ee4901a5 Add offline detection and recovery system (Phase 6)
- Implement OfflineDetector with 5-minute online threshold
- Add automatic device scanning (2-minute intervals)
- Add offline mode enforcement (disable sync)
- Add reconnection handling with priority item processing
- Add force reconnect API endpoint
- Add comprehensive offline detection tests
- Handle device offline/reconnected events
2026-01-31 13:06:05 -05:00
john-okeefe b9de7c48d5 Add sync queue system with exponential backoff retry logic (Phase 6)
- Implement SyncQueueProcessor with 5-second polling interval
- Add priority-based queuing (1-10 scale)
- Add exponential backoff retry logic (1m, 5m, 15m, 1h, 24h)
- Add stuck item detection (> 1 hour in processing state)
- Add batch processing (50 items per cycle)
- Add comprehensive test suite (15+ test cases)
- Test enqueue/dequeue, priority ordering, retry logic, concurrent operations
2026-01-31 13:06:00 -05:00
john-okeefe 2d2d643873 Add sync conflict detection and resolution system
Implement conflict detection for concurrent reading progress updates from different devices. Adds conflict management endpoints for listing, viewing, and resolving conflicts.

- Add ConflictHandler with CRUD endpoints for conflict management
- Implement automatic conflict detection in KOReader progress updates
- Add WebSocket broadcast for real-time conflict notifications
- Add database query for listing user conflicts by status
- Add integration tests and Bruno API test collection
2026-01-31 11:45:52 -05:00
john-okeefe 9bfe14bb38 feat: add WebSocket connection manager infrastructure
Add ConnectionManager for real-time WebSocket communication:
- Message types for progress updates, annotations, conflicts
- Broadcast message structure with source device tracking
- Device connection tracking with user and device metadata
- Automatic broadcast loop with concurrent message delivery
- Connection management (add, remove, get by ID/user)
- Stale connection cleanup (2-minute timeout)
- Connection statistics by device type
- Background cleanup task runs every minute

This implements the core WebSocket infrastructure needed for
Week 9 of the Universal Sync Implementation Guide.
2026-01-30 21:46:53 -05:00
john-okeefe a2424bcf74 Phase 1 Week 4: Testing & Validation
- Create comprehensive unit tests for sync package
- format_test.go: 60+ tests for format detection
  - EPUB format detection (mimetype, extension, uppercase)
  - MOBI/AZW3/FB2/TXT reflowable formats
  - PDF/DJVU fixed layout formats
  - CBZ/CBR/CB7/CBT comic archive formats
  - Unknown format handling
  - MimeType lookup tests
  - IsReflowable/HasFixedLayout/IsComicArchive helpers
- progress_test.go: 45+ tests for progress conversion
  - PageToPercentage/PercentageToPage (with clamping)
  - CharacterToPercentage/PercentageToCharacter
  - ConvertProgress between format groups
  - MergeProgress with 'max progress wins' strategy
  - FormatProgressForDisplay for UI rendering
  - Round-trip conversion tests
  - Edge cases (very small/large values, floating point precision)
- All tests pass successfully
- Test coverage: format detection, progress conversion, display formatting
- Validates Phase 1 implementation quality
2026-01-30 16:13:10 -05:00
john-okeefe bb3c32c59f Phase 1 Week 2: Format detection and progress conversion engine
- Add internal/sync package with format detection
- FormatGroup types: reflowable, fixed_layout, comic_archive
- DetectFormatGroup() function based on mimetype and file extension
- MimeType mappings for common ebook formats
- Progress conversion engine with:
  - ConvertProgress() between format groups
  - Extract percentage from various progress formats
  - PageToPercentage / PercentageToPage helpers
  - CharacterToPercentage / PercentageToCharacter helpers
  - MergeProgress() with 'max progress wins' strategy
  - FormatProgressForDisplay() for UI rendering
- Add sqlc queries for format detection and progress updates
- BulkUpdateFormatGroups query for auto-format detection
- GetUniversalProgress query with all location references
- UpdateUniversalProgress query with device sync metadata
- ReadingHistory queries for session tracking
2026-01-30 16:07:00 -05:00