- Update WebSocket connection URL in admin.ts to use new /ws/sync endpoint
- Update API documentation in bruno collection to reflect new WebSocket route
- Standardizes WebSocket routing under /ws/ path prefix for better API organization
This change improves API structure consistency and makes WebSocket endpoints
more discoverable and manageable under a dedicated path hierarchy.
This commit adds comprehensive functionality for filtering collections by library,
improves WebSocket real-time updates with user activity detection, and adds
extensive test coverage.
## Core Features
### Collection Library Filter
- Added library_id parameter to media-items search API
- Collections can now be filtered by specific library
- Toggle UI component for enabling/disabling library filter
- Default state is "checked" when library_id is present
- Consistent behavior across partial and fuzzy search modes
### WebSocket Auto-Reload Mitigation
- Added user activity detection to prevent disruptive page reloads
- Checks if user is actively typing in INPUT/TEXTAREA/SELECT elements
- Skips auto-reload when user is interacting with form elements
- Toast notifications still show for awareness
- Prevents data loss during editing operations
## Implementation Changes
### Backend
- internal/database/queries.sql.go: Added library filter support to search queries
- internal/handlers/media.go: Enhanced search with library_id parameter validation
- internal/handlers/collections.go: Updated collection handlers with library filtering
- internal/sync/websocket.go: Improved broadcast mechanism with user-scoped updates
- internal/router/frontend.go: Pass libraryID to collection templates
### Frontend
- templates/collections.templ: Added library filter toggle UI component
- web/src/collections.ts: TypeScript implementation with WebSocket integration
- templates/collections_templ.go: Generated template code
### Testing
- cmd/server/tests/search_test.go: Added TestCollectionSearchLibraryFilter
- cmd/server/tests/websocket_test.go: Added TestWebSocketUserScopedBroadcast
- New helper functions for creating libraries and media items via API
- Comprehensive test coverage for library filtering and user-scoped broadcasts
## API Documentation Updates
### Bruno Tests (Comprehensive Documentation)
- bruno/collections/*: Added detailed API documentation for all collection endpoints
- bruno/devices/*: Added device management and sync API documentation
- bruno/devices/kobo/api.yml: Kobo-specific sync protocol docs
- bruno/devices/koreader/api.yml: KOReader-specific sync protocol docs
- bruno/opds/*: Added OPDS feed and download endpoint documentation
- bruno/library/browse-folders.yml: Library folder browsing API docs
### New Bruno Tests
- bruno/media-items/Search All Libraries.yml: Test search without library filter
- bruno/media-items/Search Specific Library.yml: Test search with library filter
- bruno/media-items/Search Invalid Library ID.yml: Test error handling
## Documentation
- docs/developer/api/media-items/search_media_items.md: Updated with library_id parameter
- IMPLEMENTATION_COLLECTION_FIX.md: Comprehensive implementation guide with test scenarios
## Testing
### Integration Tests
- Library filter tests verify correct filtering across multiple libraries
- Invalid library_id tests ensure proper error handling
- WebSocket tests verify user-scoped broadcast behavior
- User A no longer receives User B's collection updates
### Manual Testing Scenarios
- Open collection in multiple tabs - updates propagate correctly
- Type in search box while another tab adds books - no disruptive reload
- Add/remove books from collection - toast notifications appear
- Toggle library filter - results update dynamically
## Technical Details
- WebSocket broadcasts are now user-scoped for privacy
- Active element detection uses tagName and contenteditable attributes
- Library ID validation uses UUID format checking
- Progressive enhancement maintained - page works without JavaScript
- All changes follow PROJECT_GUIDELINES.md conventions
- TypeScript only for frontend logic
- TailwindCSS only for styling
- Procedural programming style throughout
## Breaking Changes
None - all changes are additive and backward compatible.
- Update API field from scan_frequency_minutes to scan_poll_interval_seconds
- Update database schema default value key
- Update Bruno API collection requests and documentation
- Update OpenAPI documentation examples and field descriptions
- Update endpoint path in Get Scan Settings Bruno collection file
- Update endpoint path in Update Scan Settings Bruno collection file
- Update endpoint path in API reference documentation
This corrects a typo in the API path where 'library' was singular instead of plural.
Fix 1 - File modification time for created_at:
- Get file.ModTime() in processMediaFile and pass to CreateMediaItem
- Modified SQL INSERT to include created_at column
Fix 2 - Force rescan UPDATE instead of DELETE+INSERT:
- Changed force rescan logic to call updateMediaItem instead of delete + create
- Preserves created_at timestamp on force rescan
Fix 3 - GetMediaItemByFilePath filters by library_id:
- Added library_id to WHERE clause in SQL query
- Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader)
- Added SetLibraryID method to MediaScanner
- Updated handler to call SetLibraryID for watch mode
Fix 4 - File deletion handling with persistent logging:
- Added fsnotify.Remove handler in WatchChanges
- Added orphan cleanup in ScanFolders after scan completes
- Created scanner_logger.go with daily log rotation (7 days)
- Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log
- Individual deletes with enhanced safety logging
Note: Integration tests can now safely scan /app/uploads because
GetMediaItemByFilePath now filters by library_id, preventing
cross-library interference.
Update Bruno environment configuration with current library and job IDs
to support API testing workflows.
Changes:
- Update library_id to current active library
- Update job_id to recent scan job
- Add collection_id variable for collection testing
Variables Updated:
- library_id: 551ac19c-896a-4406-b479-353fc489b295
- job_id: 450094c6-a8f1-4125-aea7-77df80223877
- collection_id: 11e8d915-ef95-4257-98b3-b8eeacfd59b9 (new)
Impact:
- Bruno requests reference current test data
- Supports testing scan functionality
- Enables collection endpoint testing
- Maintains test environment consistency
File: bruno/environments/Bookhoard.yml
Fix critical bug where library_id variable was not quoted in the request
body, causing invalid JSON that was rejected by the API parser.
Changes:
- Quote library_id variable: {{library_id}} → "{{library_id}}"
- Add explicit Content-Type header
- Update request body format to use proper YAML multi-line string
Root Cause:
- Unquoted UUID in JSON: {"library_id": 551ac19c-896a-...}
- Produced invalid JSON that failed c.Bind() in scanner handler
- Handler returned 400 "invalid request" error
Impact:
- Bruno requests for scanning now work correctly
- API scan endpoint accepts request successfully
- Enables proper testing of scan functionality via Bruno
Testing:
- Verified scan request completes successfully
- Job ID returned correctly
- Scanner processes files as expected
File: bruno/scanner/Scan Media Items.yml
- Create bruno/library/browse-folders.yml
- Single manual test case for admin context
- Unit/integration tests cover all scenarios (no auth, user, admin)
- Bruno provides quick manual API contract verification
Fixes: Issue 2 (manual testing)
Update test email domain in Bruno API collection files:
- Login User.yml
- Register User.yml
- Update Profile.yml
Ensures API tests use the dedicated test domain and won't conflict
with real user data when developers run tests.
Add collections for Delete User, Reset User Password, and Update User
endpoints. Remove obsolete collections for individual profile update
operations. Update Update Profile collection to reflect new consolidated
API structure.
Phase 12 - Bruno API Tests Standardization
Convert all custom section builder tests to .yml format:
- preview-collection.bru → preview-collection.yml
- create-custom-section-manual.bru → create-custom-section-manual.yml
- create-custom-section-missing-fields.bru → create-custom-section-missing-fields.yml
- create-custom-section-rules.bru → create-custom-section-rules.yml
All Bruno tests now consistently use .yml extension across the project.
Tests cover:
- Preview collection with filter rules
- Create collection with manual book selection
- Validation of required fields
- Multiple filter rules with operators
Part of Carousel Dashboard Plan completion
Phase 10.5.1: Add /custom-section frontend route
- Added route handler in internal/router/frontend.go
- Fetches user libraries and renders custom section builder template
Phase 10.5.2: Create custom section builder template
- Created templates/custom_section.templ with full UI
- Includes section details form, filter rules builder, manual book selection
- Live preview functionality with preview container
- Form actions for save/cancel
Phase 10.5.3: Create custom-section-builder TypeScript
- Created web/src/custom-section-builder.ts with 13+ filter fields
- Filter fields: title, author, genre, series, progress, rating, date_added, last_read, publisher, language, format, tags, narrators
- Procedural/imperative style (no OOP) as per guidelines
- Rule builder with AND/OR logic support
- Book search and multi-select functionality
- Live preview via /api/collections/preview endpoint
- Form validation and submission to /api/collections
Phase 10.5.4: Build TypeScript modules
- Compiled custom-section-builder.ts to web/static/custom-section-builder.js
- Verified successful compilation with no errors
- All existing TypeScript modules continue to compile
Phase 10.5.5: Add Bruno tests for custom section creation
- create-custom-section-rules.bru: Test creating section with filter rules
- create-custom-section-manual.bru: Test creating section with manual book selection
- create-custom-section-missing-fields.bru: Test error handling for missing required fields
Phase 10.6: Build Verification
- ✅ TypeScript modules compile successfully
- ✅ Templates generate successfully
- ✅ Go build succeeds with no compilation errors
- ✅ All build artifacts verified (dashboard.js, custom-section-builder.js, dashboard_templ.go, custom_section_templ.go)
This completes the Custom Section Builder feature, allowing users to create
personalized dashboard sections with flexible filter rules or manual book selection.
Add preview endpoint for custom section builder and rule evaluation:
Handler Implementation (internal/handlers/collections.go):
- PreviewCollection method: Evaluates filter rules and returns matching items without saving
* Accepts library_id, rules array, manual_book_ids array, and limit
* Evaluates rules against all library items using collectionService.EvaluateRules
* Adds manually selected books to results
* Deduplicates manual books (avoids adding same book twice)
* Applies limit (default: 20, max: 100)
* Returns array of BookInfo with matching items
- Helper function: mediaItemsToListMediaItemsRow
* Converts database.MediaItems to database.ListMediaItemsRow
* Required for EvaluateRules which expects ListMediaItemsRow type
Route Registration (internal/router/collections.go):
- POST /api/collections/preview
- Protected by JWT middleware
- Part of collections API group
Why This Endpoint is Necessary:
- Allows users to see what books match their filter rules BEFORE saving
- Avoids creating incorrect collections
- Enables testing different rule combinations quickly
- Reuses existing service logic (collectionService.EvaluateRules)
- Client-side preview would require downloading entire library (10,000+ books)
- Would duplicate 500+ lines of rule evaluation logic in TypeScript
- Would create maintenance nightmare keeping Go and TypeScript in sync
Bruno Test (bruno/collections/preview-collection.bru):
- Tests POST /api/collections/preview endpoint
- Validates status 200 response
- Validates items array in response
- Example request with genre filter rule
This endpoint is required for both the web UI Custom Section Builder and future mobile apps.
Update and create Bruno API tests to reflect new unified collections architecture:
Updated Tests:
1. get-dashboard-sections.yml
- Updated response structure documentation
- Changed from type field to is_system boolean
- Changed from id to media_item_id in items
- Added priority field documentation
- Updated example response to show unified collections structure
- Added test for sections array in response
2. update-preferences.yml
- Changed HTTP method from POST to PUT (matching handler implementation)
- Updated request body field names:
* hidden_sections → hidden_collections
* section_order → collection_order
- Updated documentation with new field names
- Updated example request with valid system collection names
New Tests:
3. restore-system-collection.yml
- Tests POST /api/dashboard/restore-system-collection endpoint
- Validates collection_name against allowed system collections
- Tests success response with message
- Documents valid collection names:
* continue-reading
* recently-added
* recently-read
* not-started
Test Coverage:
- GET /api/dashboard/sections: Returns all dashboard sections
- PUT /api/dashboard/preferences: Updates user preferences
- POST /api/dashboard/restore-system-collection: Resets system collection
All tests follow Bruno YAML format with proper authentication via auth: inherit.
Add after-response scripts to automatically save access and refresh tokens to Bruno environment variables after successful authentication. This eliminates manual token copying during testing.
Changes:
- Refresh Token.yml: Add script to save tokens from refresh response
- Register User.yml: Add script to save tokens from registration response
- Move all files from bruno-yaml/* to bruno/*
- Maintains existing directory structure within categories
- Updates bruno/user/auth files with OAuth2 refresh token flow
- Updates bruno/user/profile files for user profile management
- Adds bruno/dashboard/ directory with dashboard API tests
- Preserves all existing test scenarios and OpenCollection YAML format
- No functional changes - file reorganization only
- Remove 'auth: inherit' from POST request configuration
- Remove post-response script that set token environment variable
- Clean up documentation formatting
This simplifies the login request configuration as authentication
will now be handled via HTTP-only cookies instead of bearer tokens.
- Add IMPLEMENTATION_EXACT.md with exact code changes for all phases
- Update IMPLEMENTATION_PLAN.md with clarifications on two-field approach:
- device_identifier: Serial number (Kobo) or UUID (KOReader)
- auth_token: Auto-generated API key for authentication
- Resolve all user questions with ✅ marked decisions
- Add verification steps for documentation accuracy
- Document Kobo vs KOReader registration workflow differences
- Add SQL query for token regeneration (UpdateDeviceAuthToken)
- Include TypeScript device management code
- Add Bruno API test files for all new endpoints
- Update Kobo setup documentation for URL path token approach
- Fix TestUpdateDevice: Use correct JSON field name and handle float64 type
- Fix TestRejectDeviceRegistration: Expect message response instead of boolean
- Update approve device docs: Add missing response fields
- Update reject device docs: Correct message text and format
- Update Bruno API: Fix example response for reject endpoint
Both integration tests now pass while maintaining API consistency.
System Settings Bruno Tests (new):
- bruno/system/get-scan-settings.bru
- bruno/system/update-scan-settings.bru
- Test GET endpoint for retrieving system scan settings
- Test PUT endpoint for updating system scan settings
- Include admin authentication requirements
- Document response structures
List Users Bruno Test (update):
- bruno/user/admin/List Users.bru
- Add max_devices to response documentation
- Add device_count to response documentation
- Update feature descriptions
These Bruno tests provide API contract verification for the new
system settings endpoints and document the enhanced user list response.
- Add test_create_media_item_no_folders test case
- Document 400 error for libraries without folders
- Add prerequisites section explaining folder requirement
- Include setup workflow example
- Link to Add Library Folder Bruno request
Related: Handler validation and service layer changes
- Convert all existing .bru files from JSON to Bruno DSL format
- Remove obsolete files (conflicts/api.bru, kobo/Kobo Initialization.bru)
- Update auth configuration to use 'inherit' instead of explicit bearer tokens
- Add comprehensive documentation to all test files
- Improve test scripts with proper assertions and error handling
- Rename ebook_id environment variable to media_item_id
- Update scanner endpoint example paths and documentation
- Update library endpoint documentation
- Keep library type 'ebooks' where appropriate (valid type value)
Update Update Media Item.bru to document normalization behavior:
- Note that tags and contributors are auto-normalized (same as Create)
- Document response includes updated search fields
Relates to Tags & Contributors Migration documentation updates
Update Bulk Update Books.bru to demonstrate tag and contributor updates:
- Add tags update example with normalized output
- Add contributors update example
- Shows punctuation preference behavior
Demonstrates that tags and contributors are automatically normalized
when updated via bulk operations, ensuring consistency across the database.
Relates to Tags & Contributors Migration documentation updates
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.
Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays
Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling
Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function
Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import
New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties
API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string
Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)
Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
- Update Login User.bru to use testuser@example.com
- Update Register User.bru to use testuser@example.com
- Update Register Admin User.bru to use maxdevices@example.com
- Standardize password to Test@Pass123! across all tests
- Add TEST_DATA.md documenting shared test credentials
- Update bruno.json with documentation reference
- Add comments linking to TEST_DATA.md for cross-reference
This alignment makes it easier to verify test failures between
Bruno API tests and Go integration tests using identical credentials.
- Fix device registration API test parameters
- Update admin user registration test with proper fields
- Ensure API tests match current endpoint behavior
- Improve API documentation accuracy
- Update PROJECT_GUIDELINES.md to reflect current architecture (Hybrid SSR)
- Integrate service layer and SSR rules into existing sections
- Update README.md paths to match new docs structure (docs/developer/api, docs/user/devices)
- Remove redundant README.md files from bruno/ directories
- Update bruno/collection.bru documentation to current API standard
- Fix architectural pattern description from API-driven to Hybrid SSR
Documentation updates:
- All docs/ files: Update project references
- Bruno API collection: Update collection name and tests
- Device setup guides: Update all examples
- Implementation plan: Update database schema examples
- README files: Update project references
Part of project rename to Bookhoard.
Bruno API test updates:
- Update test assertions: X-Bookmann-* → X-Bookhoard-*
- Update variable names and references in test scripts
- Update documentation and comments in API tests
This is part 4 of the project rename to Bookhoard.
Sync-Kobo:
- Auto link books endpoint tests
- Bulk link books endpoint tests
- Get unlinked book suggestions endpoint tests
OPDS:
- Download book with on-the-fly KEPUB conversion tests
All tests cover no user, user, and admin contexts
Add comprehensive Bruno tests for Phase 9 limitations features:
Test Collection Rules.bru:
- Test rule: genre equals "Science Fiction"
- Test rule: author contains "Asimov"
- Test rule: copyright_year greater than 2000
- Test rule: non-existent genre (expects 0 matches)
- Test validation: empty rules array (expects 400)
Bulk Remove Books.bru:
- Setup: Create test collection
- Add multiple books to collection
- Test 1: Bulk remove all books
- Test 2: Bulk remove with some invalid IDs
- Test 3: Empty list validation
- Test 4: Single book removal (bulk should work for 1 too)
- Cleanup: Delete test collection
Test Coverage:
- Rule evaluation API endpoint
- Bulk remove API endpoint
- Request validation
- Response structure verification
- Edge cases and error handling
Bruno Test Format:
- JSON request bodies
- Status code assertions
- Response structure validation
- Setup/teardown for integration tests
These tests ensure the new Phase 9 API endpoints work correctly
and maintain backward compatibility.
- Update User Max Devices - Complete API documentation
- Update User Max Devices - Success test case
- Update User Max Devices - Invalid Zero test case
- Update User Max Devices - Exceeds Maximum test case
- Update User Max Devices - Missing ID test case
- Include validation rules and example payloads
- Document all status codes and error responses
Implement conflict detection for concurrent reading progress updates from different devices. Adds conflict management endpoints for listing, viewing, and resolving conflicts.
- Add ConflictHandler with CRUD endpoints for conflict management
- Implement automatic conflict detection in KOReader progress updates
- Add WebSocket broadcast for real-time conflict notifications
- Add database query for listing user conflicts by status
- Add integration tests and Bruno API test collection
- Add Kobo sync handler with markup, bookmark, analytics, and initialization endpoints
- Add Kobo integration tests and Bruno API test collection
- Move device approve/reject routes from public to protected routes
- Enhance test infrastructure with DATABASE_URL support and helper functions
- Fix device GetDevice handler nil pointer handling
- Clean up test reports and session files
Add Bruno v3.0 request for testing WebSocket endpoint:
- WebSocket connection type
- JWT token authentication via query parameter
- Ping message body for heartbeat testing
- Assertions for successful WebSocket upgrade (101 status)
Provides API documentation and testing capability for
WebSocket connection functionality per project standards.
john-okeefe