Fix critical security issue where admin users could delete other users'
saved filters due to incorrect error handling in DELETE query.
Database Schema Changes:
- Change DeleteSavedFilter from :exec to :one (queries.sql:1747-1750)
- Add RETURNING * to return deleted row for proper error detection
- Regenerate querier.go and queries.sql.go with updated signature
Service Layer (internal/services/filters.go):
- Update DeleteSavedFilter to capture returned row (using _ to discard)
- Properly propagate pgx.ErrNoRows when no rows are deleted
- Error wrapping preserves original error for handler detection
Handler Layer (internal/handlers/filters.go):
- Add errors.Is() check for pgx.ErrNoRows (line 148)
- Return 404 Not Found when filter doesn't exist or belongs to different user
- Return 500 Internal Server Error for other database errors
- Add "errors" import (line 8)
Security Fix Details:
Before: Admin could delete user's filter → 204 No Content (SUCCESS)
After: Admin tries to delete user's filter → 404 Not Found (DENIED)
The DELETE query uses WHERE id = @id AND user_id = @user_id, which matches
0 rows when attempting to delete another user's filter. The old :exec query
didn't return row count, so 0 affected rows looked like success. The new :one
query with RETURNING * returns pgx.ErrNoRows when no rows match, allowing
the handler to return proper 404 error.
Test Impact:
- TestSavedFilters/User_cannot_access_another_user's_filter now passes
- All 6 integration tests pass with proper user isolation enforcement
Pattern Consistency:
- Matches DeleteLibraryFolder pattern (line 99 in queries.sql)
- Uses same error handling as media handlers (errors.Is + pgx.ErrNoRows)
- Follows user-scoping pattern used throughout codebase
Related: Saved filters implementation user isolation
Security: Prevents unauthorized deletion of user data
Add database schema and SQL queries for generic saved filters system
that allows users to save custom filter presets for any resource type.
Database Schema:
- Add saved_filters table with user_id, name, resource_type, filters (JSONB)
- Create composite index on (user_id, resource_type) for efficient lookups
- Create index on (user_id, name) for future name search feature
- Add update_updated_at_column() trigger to auto-update timestamps
- Make trigger creation idempotent with DROP TRIGGER IF EXISTS
SQL Queries (5 new queries):
- GetSavedFilters: List all filters for user + resource type
- GetSavedFilterByID: Retrieve single filter by ID
- CreateSavedFilter: Create new saved filter
- UpdateSavedFilter: Update filter name/criteria
- DeleteSavedFilter: Remove saved filter
Design Decisions:
- Generic resource_type field supports any resource (media-items, collections, devices)
- JSONB filters field allows flexible schema without migrations
- User-scoped via JWT (user_id foreign key with CASCADE delete)
- Automatic updated_at timestamp via database trigger
Generated Code:
- database.SavedFilters model (10 fields including JSONB filters)
- All 5 CRUD query functions with proper parameter types
- pgtype.UUID wrappers for UUID parameters
Part of: Saved Filters Implementation (Phase 1: Database)
Related: #saved-filters-feature
- Change library_id parameter from interface{} to pgtype.UUID
- Add explicit UUID type casting in SQL queries
- Fix SearchMediaItemsParams to use strongly-typed UUID
- Prevents potential type assertion errors and improves type safety
- Ensures proper NULL handling for optional library_id filter
This commit adds comprehensive functionality for filtering collections by library,
improves WebSocket real-time updates with user activity detection, and adds
extensive test coverage.
## Core Features
### Collection Library Filter
- Added library_id parameter to media-items search API
- Collections can now be filtered by specific library
- Toggle UI component for enabling/disabling library filter
- Default state is "checked" when library_id is present
- Consistent behavior across partial and fuzzy search modes
### WebSocket Auto-Reload Mitigation
- Added user activity detection to prevent disruptive page reloads
- Checks if user is actively typing in INPUT/TEXTAREA/SELECT elements
- Skips auto-reload when user is interacting with form elements
- Toast notifications still show for awareness
- Prevents data loss during editing operations
## Implementation Changes
### Backend
- internal/database/queries.sql.go: Added library filter support to search queries
- internal/handlers/media.go: Enhanced search with library_id parameter validation
- internal/handlers/collections.go: Updated collection handlers with library filtering
- internal/sync/websocket.go: Improved broadcast mechanism with user-scoped updates
- internal/router/frontend.go: Pass libraryID to collection templates
### Frontend
- templates/collections.templ: Added library filter toggle UI component
- web/src/collections.ts: TypeScript implementation with WebSocket integration
- templates/collections_templ.go: Generated template code
### Testing
- cmd/server/tests/search_test.go: Added TestCollectionSearchLibraryFilter
- cmd/server/tests/websocket_test.go: Added TestWebSocketUserScopedBroadcast
- New helper functions for creating libraries and media items via API
- Comprehensive test coverage for library filtering and user-scoped broadcasts
## API Documentation Updates
### Bruno Tests (Comprehensive Documentation)
- bruno/collections/*: Added detailed API documentation for all collection endpoints
- bruno/devices/*: Added device management and sync API documentation
- bruno/devices/kobo/api.yml: Kobo-specific sync protocol docs
- bruno/devices/koreader/api.yml: KOReader-specific sync protocol docs
- bruno/opds/*: Added OPDS feed and download endpoint documentation
- bruno/library/browse-folders.yml: Library folder browsing API docs
### New Bruno Tests
- bruno/media-items/Search All Libraries.yml: Test search without library filter
- bruno/media-items/Search Specific Library.yml: Test search with library filter
- bruno/media-items/Search Invalid Library ID.yml: Test error handling
## Documentation
- docs/developer/api/media-items/search_media_items.md: Updated with library_id parameter
- IMPLEMENTATION_COLLECTION_FIX.md: Comprehensive implementation guide with test scenarios
## Testing
### Integration Tests
- Library filter tests verify correct filtering across multiple libraries
- Invalid library_id tests ensure proper error handling
- WebSocket tests verify user-scoped broadcast behavior
- User A no longer receives User B's collection updates
### Manual Testing Scenarios
- Open collection in multiple tabs - updates propagate correctly
- Type in search box while another tab adds books - no disruptive reload
- Add/remove books from collection - toast notifications appear
- Toggle library filter - results update dynamically
## Technical Details
- WebSocket broadcasts are now user-scoped for privacy
- Active element detection uses tagName and contenteditable attributes
- Library ID validation uses UUID format checking
- Progressive enhancement maintained - page works without JavaScript
- All changes follow PROJECT_GUIDELINES.md conventions
- TypeScript only for frontend logic
- TailwindCSS only for styling
- Procedural programming style throughout
## Breaking Changes
None - all changes are additive and backward compatible.
Change library ordering in dropdown from DESC to ASC to display
libraries in creation order (oldest first).
Database changes in internal/database/queries/queries.sql:
- Modify GetUserLibraries query ORDER BY clause
- Change from ORDER BY l.created_at DESC to ASC
- Displays oldest libraries first in dropdown
This provides a more intuitive ordering where users see their
first-created libraries at the top of the list.
- Add libraryService dependency to CollectionHandler and OPDSHandler for centralized path resolution
- Create internal/utils/mediaurl.go with ResolveMediaURL() function as single source of truth
- Update GetMediaItem and ListMediaItems handlers to return resolved URLs in API responses
- Update collection handlers (GetCollection, TestRules, PreviewCollection) to use resolved cover URLs
- Update progress handler (GetAllProgress) to use resolved cover URLs
- Add library_id to GetCollectionItems SQL query to enable URL resolution
- Refactor media scanner to store relative paths instead of absolute filesystem paths
- Add ResolveMediaPath() to LibraryService for resolving relative paths to absolute paths
- Add ServeFile endpoint at /uploads/library-:id/* for authenticated file serving
- Add MimeTypes map to library_service.go for consistent MIME type handling
- Update DownloadBook handler to use resolved filesystem paths
- Add getRelativePath() helper to MediaScanner for converting absolute to relative paths
- Use strings.EqualFold for case-insensitive path comparisons in zip extraction
This change enables the application to work with relative paths stored in the
database, making it portable across different server environments while
maintaining backward compatibility with existing absolute paths.
Fix 1 - File modification time for created_at:
- Get file.ModTime() in processMediaFile and pass to CreateMediaItem
- Modified SQL INSERT to include created_at column
Fix 2 - Force rescan UPDATE instead of DELETE+INSERT:
- Changed force rescan logic to call updateMediaItem instead of delete + create
- Preserves created_at timestamp on force rescan
Fix 3 - GetMediaItemByFilePath filters by library_id:
- Added library_id to WHERE clause in SQL query
- Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader)
- Added SetLibraryID method to MediaScanner
- Updated handler to call SetLibraryID for watch mode
Fix 4 - File deletion handling with persistent logging:
- Added fsnotify.Remove handler in WatchChanges
- Added orphan cleanup in ScanFolders after scan completes
- Created scanner_logger.go with daily log rotation (7 days)
- Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log
- Individual deletes with enhanced safety logging
Note: Integration tests can now safely scan /app/uploads because
GetMediaItemByFilePath now filters by library_id, preventing
cross-library interference.
Add delete_user, reset_user_password, and update_user endpoints to replace
individual update operations. Update database schema to include deleted_at
column for soft deletion. Add DeleteUser, ResetUserPassword, and
UpdateUserAdmin queries. Update Querier with new methods for user management.
Add support for Carousel-style dashboard with unified collections architecture:
Database Schema Changes:
- Add user_dashboard_preferences table:
- hidden_collections: TEXT[] for managing section visibility
- collection_order: TEXT[] for custom ordering
- items_per_section: INT for limiting items per section
- Update collections table:
- user_id: Make nullable to support system-owned collections (NULL = system)
- show_on_dashboard: BOOLEAN for controlling visibility
- query_type: TEXT for different query types (continue-reading, recently-added, etc.)
- priority: INT for display order (lower = higher priority)
- is_system_collection: BOOLEAN for flagging system defaults
- Update collection_items table:
- Add excluded BOOLEAN for user overrides of auto-assigned items
Indexes:
- idx_collections_dashboard: (user_id, show_on_dashboard, priority) WHERE show_on_dashboard = true
- idx_dashboard_prefs_user_library: (user_id, library_id)
- idx_collection_items_excluded: (collection_id, excluded) WHERE excluded = true
System Collections (pre-seeded defaults):
- continue-reading: Books with 0 < progress < 1
- recently-added: Newly added items to library
- recently-read: Books with progress >= 1
- not-started: Books with progress = 0 or no record
This implements Phase 1 of the Carousel-style dashboard redesign plan.
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods
Generated via: cd internal/database && sqlc generate
- Remove CreateEbookNote, UpdateEbookNote, DeleteEbookNote queries
- These were marked as backward compatibility but never used
- API uses CreateMediaNote, UpdateMediaNote, DeleteMediaNote instead
- Remove misleading backward compatibility comments
- Regenerate sqlc code
Schema changes:
- Add tags_search TEXT[] column for case-insensitive, punctuation-free search
- Add contributors_search TEXT[] column for case-insensitive, punctuation-free search
- Create GIN indexes for fast array searches on both search fields
Query updates:
- CreateMediaItem: Include tags_search and contributors_search parameters
- UpdateMediaItem: Include tags_search and contributors_search parameters
- SearchMediaItems: Search against tags_search instead of tags
- SearchMediaItems: Search against contributors_search instead of contributors
- SearchMediaItemsFuzzy: Use tags_search and contributors_search for fuzzy matching
- Update ranking and priority logic to use search fields
Benefits:
- Case-insensitive search: "acme corp" finds "ACME CORP."
- Punctuation-agnostic search: "oreilly" finds "O'Reilly Media"
- Better UX: Users don't need to match exact casing or punctuation
- Improved performance with dedicated GIN indexes
Relates to Tags & Contributors Migration Phases 5 & 8
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.
Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays
Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling
Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function
Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import
New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties
API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string
Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)
Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
Complete the rename by updating:
- DeviceCatalogs struct field: BookmannUuid → BookhoardUuid (models.go)
- Generated queries: Update all references (queries.sql.go)
- Local variables: bookmannUUID → bookhoardUUID (kobo.go)
- Struct field access: catalog.BookmannUuid → catalog.BookhoardUuid
All "bookmann" and "BOOKMANN" references are now eliminated from the codebase.
Part of project rename to Bookhoard.
Changes:
- Update comments: "Bookmann UUID" → "Bookhoard UUID"
- Rename sidecar struct field: Bookmann → Bookhoard
- Update type names: SidecarBookmannConfig → SidecarBookhoardConfig
- Fix test database name in queue_test.go
- Fix uppercase env var examples in KOBO_SETUP.md
Internal Go variable names (BookmannUuid, bookmannUUID) left unchanged
as they're implementation details that don't affect functionality.
Part of project rename to Bookhoard.
Add analytics queries:
- GetUserReadingHistory: detailed reading history with device info
- GetUserDeviceUsage: device usage statistics (sync count, time spent)
- GetPopularBooks: most read books with completion rates
Add book matching queries:
- GetUnlinkedBookByID: fetch single unlinked book
- DeleteUnlinkedBook: remove resolved unlinked book
- ListUnresolvedUnlinkedBooks: paginated list of unresolved books
- Add bulk_update_progress_from_koreader() function for batch processing
- Handles progress, annotations, and conflict detection
- Returns success/failure status for each book
- Supports device matching by UUID, file path, or title/author
- Implements automatic conflict detection for concurrent syncs
- Part of Phase 3 KOReader Integration implementation
- Add internal/sync package with format detection
- FormatGroup types: reflowable, fixed_layout, comic_archive
- DetectFormatGroup() function based on mimetype and file extension
- MimeType mappings for common ebook formats
- Progress conversion engine with:
- ConvertProgress() between format groups
- Extract percentage from various progress formats
- PageToPercentage / PercentageToPage helpers
- CharacterToPercentage / PercentageToCharacter helpers
- MergeProgress() with 'max progress wins' strategy
- FormatProgressForDisplay() for UI rendering
- Add sqlc queries for format detection and progress updates
- BulkUpdateFormatGroups query for auto-format detection
- GetUniversalProgress query with all location references
- UpdateUniversalProgress query with device sync metadata
- ReadingHistory queries for session tracking
- Add SearchMediaItems query with ILIKE partial matching
- Add SearchMediaItemsFuzzy query with word_similarity()
- Use sqlc.narg() for named parameters (search_pattern, search_query)
- Rank results by relevance: title > author > series > tags
- Fuzzy threshold set to 0.3 for word_similarity
- Generated Go models with proper parameter types
- Increase ISBN column from VARCHAR(13) to VARCHAR(17) to support ISBN-13 with hyphens
- Add normalize_isbn() database function to automatically remove hyphens and spaces
- Create trigger to auto-normalize ISBNs on INSERT/UPDATE operations
- Update all Ebook and MediaItem queries to use ISBN normalization
- Add GetEbookLibraryID query to check for existing ebook libraries
- Add graceful error handling when no ebook library exists
- Return helpful error message: 'no ebook library found. Please create an ebook library first'
- Create comprehensive tests for ISBN normalization and library selection
- Add Bruno test files for various ISBN formats and error scenarios
- Update documentation with ISBN normalization details
- Regenerate queries.sql.go with refresh token queries
- Update models.go with RefreshTokens type
- Update querier.go with new query methods
- Update db.go with generated code
- Add MediaNotes and MediaHighlights model structs with pgx v5 types
- Add EbookNotes and EbookHighlights for backward compatibility
- Add complete CRUD SQL queries for notes and highlights
- Add database connection pool function using pgx v5
- Generate sqlc code for new annotation functionality
- Add library_types table with ebooks, comics, manga types
- Add libraries table for multiple library support
- Add library_folders table for multi-folder libraries
- Add library_visibility table for user access control
- Add media_items table replacing ebooks for broader media support
- Create backward compatibility views for existing API
- Implement library service with type validation and file extension handling
- Support modular extension for future media types
Manga type includes cbz/cbr archives as requested
- Update CreateUser SQL query to accept role parameter
- Regenerate SQLC code to include Role field in CreateUserParams
- Support for explicit role assignment during user registration
- Add role column to users table with admin/user constraint
- Add added_by_admin_id column to ebooks table for tracking
- Add constraint to ensure only admins can manage folders
- Update all SQL queries to include role field
- Regenerate database models with new schema
- Move migrations/ to database/schema/ for clarity on database schema definitions
- Move sqlc.yaml to internal/database/ to group with database code
- Move static/ to cmd/server/static/ to co-locate with server
- Update all configuration files and documentation
- Follow Go project conventions for better organization
- Update CreateUser query to explicitly select only existing columns
- Update all user SELECT queries to explicitly select columns to avoid scan_frequency_minutes column issues
- Add GetUserForLogin query that includes password_hash for authentication
- Update login handler to use GetUserForLogin instead of GetUserByEmailOrUsername
- This prevents errors when migration hasn't been applied yet, allowing user registration to work
- Add admin dashboard at /admin route consolidating all user settings
- Move user preferences (username, email, password, theme) from separate page
- Add ebook library preferences section with folder management
- Add scan settings (frequency and auto-scan toggle) with database persistence
- Create database migration for scan_frequency_minutes and auto_scan_enabled columns
- Add API endpoints for scan settings management:
- PUT /api/library/scan-settings - Update scan preferences
- GET /api/library/scan-settings - Get current scan settings
- Update dashboard navigation to link to admin dashboard
- Remove old preferences.html template (functionality moved to admin)
- Create Bruno API testing files for library endpoints
- Add real-time folder loading and management in admin interface
- Implement scan settings persistence and retrieval from database
- Add /preferences route and preferences.html template for user settings
- Implement username, email, password, and theme update functionality
- Add account deletion feature with confirmation
- Add navigation link to preferences from dashboard
- Create API endpoints:
- PUT /api/user/username - Update username
- PUT /api/user/email - Update email address
- PUT /api/user/password - Change password with verification
- DELETE /api/user/account - Delete user account
- Add database queries for user updates and account deletion
- Create Bruno API testing files for all user preference endpoints
- Add proper validation, error handling, and security checks
- Rename project from 'bookmann' to 'shelf'
- Move all backend/ contents to root level (flatten structure)
- Update Go module name from 'bookmann' to 'shelf'
- Update all import paths to use new 'shelf' module
- Update Dockerfile to work without backend/ subdirectory
- Update docker-compose.yml to use new structure and rename containers
- Update .gitignore for new file paths
- Update README.md with new project name and structure
- Regenerate database code with new module imports
john-okeefe