Add real-time synchronization for collection book removal:
- Extract user ID from context for targeted broadcasts
- Broadcast 'collection_updated' message to user's other devices
- Includes collection_id, action, and book_id in message payload
This ensures that when a user removes a book from a collection,
all their connected devices (browser tabs, mobile apps, etc.)
receive real-time updates via WebSocket.
Consistent with existing AddBooks and BulkRemoveBooks operations
which already use BroadcastToUser for synchronization.
This commit adds comprehensive functionality for filtering collections by library,
improves WebSocket real-time updates with user activity detection, and adds
extensive test coverage.
## Core Features
### Collection Library Filter
- Added library_id parameter to media-items search API
- Collections can now be filtered by specific library
- Toggle UI component for enabling/disabling library filter
- Default state is "checked" when library_id is present
- Consistent behavior across partial and fuzzy search modes
### WebSocket Auto-Reload Mitigation
- Added user activity detection to prevent disruptive page reloads
- Checks if user is actively typing in INPUT/TEXTAREA/SELECT elements
- Skips auto-reload when user is interacting with form elements
- Toast notifications still show for awareness
- Prevents data loss during editing operations
## Implementation Changes
### Backend
- internal/database/queries.sql.go: Added library filter support to search queries
- internal/handlers/media.go: Enhanced search with library_id parameter validation
- internal/handlers/collections.go: Updated collection handlers with library filtering
- internal/sync/websocket.go: Improved broadcast mechanism with user-scoped updates
- internal/router/frontend.go: Pass libraryID to collection templates
### Frontend
- templates/collections.templ: Added library filter toggle UI component
- web/src/collections.ts: TypeScript implementation with WebSocket integration
- templates/collections_templ.go: Generated template code
### Testing
- cmd/server/tests/search_test.go: Added TestCollectionSearchLibraryFilter
- cmd/server/tests/websocket_test.go: Added TestWebSocketUserScopedBroadcast
- New helper functions for creating libraries and media items via API
- Comprehensive test coverage for library filtering and user-scoped broadcasts
## API Documentation Updates
### Bruno Tests (Comprehensive Documentation)
- bruno/collections/*: Added detailed API documentation for all collection endpoints
- bruno/devices/*: Added device management and sync API documentation
- bruno/devices/kobo/api.yml: Kobo-specific sync protocol docs
- bruno/devices/koreader/api.yml: KOReader-specific sync protocol docs
- bruno/opds/*: Added OPDS feed and download endpoint documentation
- bruno/library/browse-folders.yml: Library folder browsing API docs
### New Bruno Tests
- bruno/media-items/Search All Libraries.yml: Test search without library filter
- bruno/media-items/Search Specific Library.yml: Test search with library filter
- bruno/media-items/Search Invalid Library ID.yml: Test error handling
## Documentation
- docs/developer/api/media-items/search_media_items.md: Updated with library_id parameter
- IMPLEMENTATION_COLLECTION_FIX.md: Comprehensive implementation guide with test scenarios
## Testing
### Integration Tests
- Library filter tests verify correct filtering across multiple libraries
- Invalid library_id tests ensure proper error handling
- WebSocket tests verify user-scoped broadcast behavior
- User A no longer receives User B's collection updates
### Manual Testing Scenarios
- Open collection in multiple tabs - updates propagate correctly
- Type in search box while another tab adds books - no disruptive reload
- Add/remove books from collection - toast notifications appear
- Toggle library filter - results update dynamically
## Technical Details
- WebSocket broadcasts are now user-scoped for privacy
- Active element detection uses tagName and contenteditable attributes
- Library ID validation uses UUID format checking
- Progressive enhancement maintained - page works without JavaScript
- All changes follow PROJECT_GUIDELINES.md conventions
- TypeScript only for frontend logic
- TailwindCSS only for styling
- Procedural programming style throughout
## Breaking Changes
None - all changes are additive and backward compatible.
Add form:"" tags to CreateCollectionRequest and UpdateCollectionRequest
structs to enable proper form data binding with Echo's c.Bind().
This change aligns with the pattern used in auth handlers where both
form:"" and json:"" tags are present, allowing the same request structs
to work with both JSON payloads (API) and form data (HTMX).
Changes:
- Add form:"name", form:"description", form:"color", form:"icon",
form:"auto_assign_rules", and form:"view_settings" tags to both
CreateCollectionRequest and UpdateCollectionRequest
Additionally, add HTMX redirect support to CreateCollection and
UpdateCollection handlers:
- Add HX-Redirect header for HTMX requests after successful create/update
- Add HTML redirect response to DeleteCollection for HTMX requests
(follows pattern from auth.go: inline script with window.location.href)
This ensures HTMX form submissions properly redirect to /collections
after successful operations, while maintaining API compatibility for
JSON requests.
Add comprehensive collection detail page that works for both system collections
(continue-reading, recently-added, not-started) and user collections.
Backend changes:
- Add new /collections/:id route in internal/router/frontend.go
- Fetches collection using GetCollection with UUID parameter
- Determines collection type from QueryType field
- Resolves library_id for system collections
- Converts database.MediaItems to handlers.BookInfo for display
- Renders CollectionDetail template with collection and books data
- Update SectionData struct in internal/handlers/collections.go
- Add CollectionID string field for view all links
- Update BuildSections() in internal/handlers/dashboard.go
- Pass CollectionID to SectionData for proper link generation
- Simplify getViewAllURL() in internal/handlers/dashboard.go
- Return /collections/{collectionID} instead of /section/{type}
- Works uniformly for both system and user collections
Frontend changes:
- Fix CollectionDetail template in templates/collections.templ
- Fix broken div nesting causing compilation error
- Add null check for CoverImagePath to prevent broken images
- Update aspect ratio to modern aspect-[3/4] syntax
- Use responsive widths (w-16 sm:w-20) for mobile/desktop
- Improve card layout with horizontal flex structure
- Add placeholder image fallback for books without covers
- Remove erroneous renderBooks() function call
This change aligns with the backend update where system collections are
now pre-made user collections in the database with query_type fields.
All collections can now use the same CollectionDetail template for a
consistent viewing experience.
- Implement event queue with 3-second debouncing for file system events
- Add configurable polling fallback (default 3 min) via SCAN_POLL_INTERVAL_MINUTES
- Add SyncFilesystemWithDatabase to detect orphaned DB entries and new files
- Integrate utils.ResolveMediaURL for consistent media file path resolution
- Add COOKIE_SECURE env var with SameSite=LaxMode for session cookies
- Update media handler to properly decode URL paths for file serving
- Refactor scanner initialization to accept poll interval configuration
- Add libraryService dependency to CollectionHandler and OPDSHandler for centralized path resolution
- Create internal/utils/mediaurl.go with ResolveMediaURL() function as single source of truth
- Update GetMediaItem and ListMediaItems handlers to return resolved URLs in API responses
- Update collection handlers (GetCollection, TestRules, PreviewCollection) to use resolved cover URLs
- Update progress handler (GetAllProgress) to use resolved cover URLs
- Add library_id to GetCollectionItems SQL query to enable URL resolution
- Refactor media scanner to store relative paths instead of absolute filesystem paths
- Add ResolveMediaPath() to LibraryService for resolving relative paths to absolute paths
- Add ServeFile endpoint at /uploads/library-:id/* for authenticated file serving
- Add MimeTypes map to library_service.go for consistent MIME type handling
- Update DownloadBook handler to use resolved filesystem paths
- Add getRelativePath() helper to MediaScanner for converting absolute to relative paths
- Use strings.EqualFold for case-insensitive path comparisons in zip extraction
This change enables the application to work with relative paths stored in the
database, making it portable across different server environments while
maintaining backward compatibility with existing absolute paths.
Implement DeleteUser, ResetUserPassword, and UpdateUserAdmin handlers.
Update collections handler to check soft-deleted users. Update dashboard
service to exclude deleted users from statistics.
The PreviewCollection endpoint was missing authentication verification,
allowing unauthenticated access to the preview functionality. Added
check for user in context, returning 401 Unauthorized if missing.
Add preview endpoint for custom section builder and rule evaluation:
Handler Implementation (internal/handlers/collections.go):
- PreviewCollection method: Evaluates filter rules and returns matching items without saving
* Accepts library_id, rules array, manual_book_ids array, and limit
* Evaluates rules against all library items using collectionService.EvaluateRules
* Adds manually selected books to results
* Deduplicates manual books (avoids adding same book twice)
* Applies limit (default: 20, max: 100)
* Returns array of BookInfo with matching items
- Helper function: mediaItemsToListMediaItemsRow
* Converts database.MediaItems to database.ListMediaItemsRow
* Required for EvaluateRules which expects ListMediaItemsRow type
Route Registration (internal/router/collections.go):
- POST /api/collections/preview
- Protected by JWT middleware
- Part of collections API group
Why This Endpoint is Necessary:
- Allows users to see what books match their filter rules BEFORE saving
- Avoids creating incorrect collections
- Enables testing different rule combinations quickly
- Reuses existing service logic (collectionService.EvaluateRules)
- Client-side preview would require downloading entire library (10,000+ books)
- Would duplicate 500+ lines of rule evaluation logic in TypeScript
- Would create maintenance nightmare keeping Go and TypeScript in sync
Bruno Test (bruno/collections/preview-collection.bru):
- Tests POST /api/collections/preview endpoint
- Validates status 200 response
- Validates items array in response
- Example request with genre filter rule
This endpoint is required for both the web UI Custom Section Builder and future mobile apps.
Add dashboard API endpoints with handler layer:
Step 1: Add SectionData to collections.go
- SectionData struct represents dashboard section (carousel of books)
- Used by: Dashboard handler, Templates (SSR), API JSON responses
- Shared type from collections.go (no duplicate definitions)
- Fields: ID, IsSystem, Title, Description, Icon, Items, ViewAllURL, Priority
Step 2: Create dashboard.go handler
- DashboardHandler struct with injected database and dashboard service
- GetSections: Returns dashboard sections as JSON (mobile apps, web UI TypeScript, plugins)
* Validates library_id parameter
* Fetches user dashboard preferences
* Configurable limit (default 20, max 100)
* Calls service layer for business logic
* Converts service types to handler types for JSON serialization
- UpdatePreferences: Saves dashboard preferences
* Validates library_id
* Upserts user dashboard preferences
- RestoreSystemCollection: Resets system collection to defaults
* Validates collection_name against allowed system collections
* Deletes user's copy (system collection reappears automatically)
- BuildSections: Converts service DashboardSection to handler SectionData
* Converts database.MediaItems to handlers.BookInfo
* Uses shared types from collections.go
- getViewAllURL: Maps system collections to their view-all URLs
- Reuses existing textToString helper from collections.go
Architecture Compliance:
- Generic API handler for reuse by SSR, mobile, plugins
- Uses shared types from collections.go (SectionData, BookInfo)
- IsSystem bool matches database field (no string conversion)
- Single service method returns structured data (simpler, less bugs)
- Handler just converts types (no matching logic needed)
- Reusable by mobile apps, web UI, plugins
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.
Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays
Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling
Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function
Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import
New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties
API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string
Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)
Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
Implement real-time collection updates when books are added/removed:
Backend Changes:
- Added connManager to CollectionHandler struct
- Updated constructor to accept ConnectionManager
- Updated all NewCollectionHandler() calls in ebook.go and main.go
- Added WebSocket broadcasts in AddBooks() handler
- Added WebSocket broadcasts in BulkRemoveBooks() handler
- Broadcasts collection_updated events with:
- collection_id: Which collection changed
- action: books_added or books_removed
- book_ids: Array of affected book IDs
- count: Number of books changed
Frontend Changes:
- Added WebSocket connection in collections UI
- connectWebSocket() establishes connection to /ws/sync
- Listens for collection_updated events
- Shows toast notification on collection change
- Auto-reloads page after 1 second to show updated book list
- Auto-reconnect on disconnect (5s delay)
- Error handling for WebSocket failures
WebSocket Event Format:
{
"type": "collection_updated",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"collection_id": "uuid",
"action": "books_added",
"book_ids": ["uuid1", "uuid2"],
"count": 2
}
}
User Experience:
- When another user adds books to a collection, all connected clients see:
1. Toast notification: "Collection updated: books_added (2 books)"
2. Page auto-refreshes after 1 second
3. Updated book list displays
- Same for book removal
- Works across multiple browser tabs/devices
- No manual refresh needed
Technical Notes:
- Broadcasts to ALL connected WebSocket clients
- Client-side filtering by collection_id
- Existing progress/conflict broadcasts continue to work
- Connection manager handles broadcast distribution
Resolves Limitation #4: Real-time Collection Updates
Complete bulk operations for collections management:
BULK ADD BOOKS:
- Implemented searchBooks() with real API integration
- Multi-select checkboxes for book selection
- SelectedBooks Set tracks chosen books
- AddSelectedBooks() sends array to existing endpoint
- Uses existing POST /api/collections/:id/books endpoint
BULK REMOVE BOOKS:
- New endpoint: POST /api/collections/:id/books/bulk-remove
- Checkboxes on each book card for selection
- BooksToRemove Set tracks selections
- Live counter showing selected count
- BulkRemoveBooks() handler removes all in one API call
- More efficient than N individual DELETE requests
Frontend Changes:
- Selected counter badge shows number selected
- Bulk remove button (enabled when books selected)
- Checkboxes on all books for multi-select
- Confirmation dialog for bulk operations
- Toast notifications with counts
Backend Changes:
- BulkRemoveBooks() handler in collections.go
- Accepts book_ids array, returns removed/total counts
- Iterates and removes, counting successes
- Route: POST /api/collections/:id/books/bulk-remove
API Request:
{
"book_ids": ["uuid1", "uuid2", "uuid3"]
}
API Response:
{
"removed": 3,
"total": 3
}
Tests Added:
- TestCompareValues_* (existing)
- TestEvaluateRule_* (existing)
Resolves Limitations #2 (Bulk Operations) and #5 (Bulk Remove)
Add data retrieval helpers for SSR template rendering:
- GetDeviceMappingsData: Fetch device shelf mappings for device settings UI
- GetUserCollectionsList: Get all user collections for dropdowns and listings
- GetCollectionData: Get single collection with metadata
- GetCollectionBooksData: Get books in a collection
These functions support the Phase 9 frontend features by providing
efficient data access for template rendering without modifying
existing API endpoints.
- Add OPDS handler for device catalog and book downloads
- Add collections handler for collection CRUD
- Add book matching service for cross-device book linking
- Add sidecar handler for Kobo metadata sync
- Add sync handler for device synchronization
john-okeefe