Commit Graph
79 Commits
Author SHA1 Message Date
john-okeefe 051ee287c7 feat(backend): add EPUB CFI and percentage to reading progress
Enhance reading progress tracking to support EPUB-specific location data:
- Add epubcfi field to store EPUB Canonical Fragment Identifier
- Add percentage field for normalized position across formats
- Update UpdateReadingProgress API handler to accept new fields
- Modify database queries to persist additional progress metadata

This enables precise position tracking in reflowable EPUB content
where page numbers are insufficient for accurate bookmarking.
2026-04-09 21:16:53 -04:00
john-okeefe cf8c690474 Add library_type_name column to media_items with auto-population trigger
- Add library_type_name VARCHAR(50) column to media_items table in schema.sql
- Create PostgreSQL trigger 'set_library_type_name_on_insert' that automatically
  populates library_type_name by joining libraries table with library_types on insert
- Add library_type_name parameter to CreateMediaItem SQL query
- Update Go models (models.go) to include LibraryTypeName field
- Add UpdateMediaItemChapterMetadata query method to querier.go
- Regenerate queries.sql.go with sqlc

This allows media items to store their library type (e.g., 'Books', 'Comics', 'Manga')
at the database level, enabling filtering and display without needing additional joins.
2026-04-04 22:49:07 -04:00
john-okeefe e8544ac8f4 feat: add UpdateMediaItemChapterMetadata query
- Add SQL query to update chapter metadata in media_items table
- Enables caching of detected chapter structures
2026-04-03 17:20:18 -04:00
john-okeefe 74575d9a86 feat: add reader service, handler, and router 2026-04-03 16:53:37 -04:00
john-okeefe 0667cad8a9 feat: add reader infrastructure - Phase 0 database schema and queries
Implement Phase 0 prerequisites for reader functionality including
database schema, SQL queries, and frontend dependencies.

## Database Schema (5 New Tables + 1 Column Addition)

### New Tables Added:
1. **panel_data** - Comic/manga panel detection results
   - Stores detected panel boundaries (x, y, width, height)
   - Supports grid, ML, and manual detection methods
   - JSONB storage for flexible panel structures

2. **reading_speed** - User reading speed statistics
   - Tracks pages per minute and total reading time
   - Per-user per-media-item tracking
   - Enables progress estimation and analytics

3. **dictionary_cache** - Offline dictionary word definitions
   - Caches external dictionary lookups
   - Reduces API calls and improves performance
   - Supports offline reading functionality

4. **reader_settings** - User reader preferences (per-user)
   - Stores typography, theme, and display settings
   - JSONB storage for flexible configuration
   - Per-user customization (fonts, margins, themes)

5. **media_bookmarks** - Enhanced bookmarks with chapter/CFI support
   - Unified bookmarking for ebooks, comics, manga, PDFs
   - Supports page_number, chapter_number, and epubcfi_position
   - Includes notes field for annotations
   - Unique constraint on (media_item_id, user_id, title)

### Column Addition:
- **media_items.chapter_metadata** (JSONB) - Caches detected chapter structure
  - Stores TOC/chapter detection results
  - Prevents re-parsing files on every read
  - Populated by ReaderService.DetectChapters()

## Database Queries (12 New Queries)

Added queries for all reader functionality:
- Panel data: GetPanelData, UpsertPanelData
- Reading speed: GetReadingSpeed, CreateReadingSpeed, UpdateReadingSpeed
- Dictionary: GetDictionaryEntry, CreateDictionaryEntry, UpdateDictionaryAccessed
- Settings: GetReaderSettings, UpsertReaderSettings
- Bookmarks: GetMediaBookmarks, CreateMediaBookmark, DeleteMediaBookmark, UpdateMediaBookmark

## Frontend Dependencies

Added to package.json:
- jszip@^3.10.1 - EPUB/comic archive parsing (client-side)
- pdfjs-dist@^3.11.174 - PDF rendering library (Mozilla PDF.js)

## Generated Code

Ran `sqlc generate` to regenerate:
- models.go - Go structs for new tables (55 lines added)
- querier.go - Database query methods (14 lines added)
- queries.sql.go - Compiled SQL queries (504 lines added)

## Implementation Status

Phase 0 prerequisites now complete:
 Database schema (5 tables + 1 column)
 SQL queries (12 queries)
 Frontend dependencies (2 packages)
 Generated Go code (sqlc)
 Database recreated with new schema

Ready for Phase 1: Infrastructure & Basic Reader implementation.

Related to: Universal web reader for ebooks, comics, manga, PDFs
2026-04-02 21:01:45 -04:00
john-okeefe dd81dc08a2 db: update SQL queries and regenerate models for comic metadata
Phase 1 implementation: Update CreateMediaItem query to support 14 new comic metadata fields.

Changes:
- Add 14 new columns to CreateMediaItem INSERT statement
- Regenerate sqlc models with new fields
- CommunityRating now maps to pgtype.Float8 (was pgtype.Numeric)
- All new comic and universal metadata fields included

New fields supported:
- Reading direction: manga_type, reading_direction
- Universal: series_count, volume, imprint, age_rating, web_url
- Comic-specific: story_arc, is_black_and_white, metadata_notes,
  community_rating, alternate_info, scan_information, summary

Generated models verified:
- MediaItems struct includes all 14 new fields
- CreateMediaItemParams has correct parameter count (42 total)
- CommunityRating is pgtype.Float8 (not pgtype.Numeric)

Relates to: Phase 4.1-4.2 database layer implementation
2026-03-29 21:12:20 -04:00
john-okeefe 9e3e0ff931 db: regenerate sqlc models with comic metadata fields
Auto-generated by sqlc from updated schema.sql
- MediaItems struct now includes all 14 new comic metadata fields
- CreateMediaItemParams struct updated with new parameters
- All SELECT queries now include new columns in RETURNING clauses
- Properly typed with pgtype.Text, pgtype.Int4, pgtype.Bool, pgtype.Numeric, []byte

New Fields in MediaItems:
- MangaType pgtype.Text
- ReadingDirection pgtype.Text
- SeriesCount pgtype.Int4
- Volume pgtype.Int4
- Imprint pgtype.Text
- AgeRating pgtype.Text
- WebUrl pgtype.Text
- StoryArc pgtype.Text
- IsBlackAndWhite pgtype.Bool
- MetadataNotes pgtype.Text
- CommunityRating pgtype.Numeric
- AlternateInfo []byte (JSONB)
- ScanInformation pgtype.Text
- Summary pgtype.Text

Part of Phase 1: Database Schema Changes
Implementation: IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
2026-03-29 19:11:51 -04:00
john-okeefe 0e11c9263c fix: add unique constraint for saved filter names and fix search sort ordering
Database changes:
- Add unique index on saved_filters(user_id, name, resource_type)
  Prevents duplicate filter names while allowing same name across
  different users or different resource types

Search functionality fix:
- Remove DISTINCT ON (mi.id) from SearchMediaItemsUnified query
- Remove mi.id from ORDER BY clause (was required by DISTINCT ON)
- This allows user-selected sort field to be primary sort criteria
- Previously results were always sorted by ID first, making sort
  dropdown ineffective
- Relevance score and title remain as fallback sorts

This fixes the sort dropdown functionality on the bookshelf page
where changing the sort option appeared to have no effect.
2026-03-28 00:46:03 -04:00
john-okeefe 36ae781765 fix: implement proper 3-state boolean logic for has_cover filter
Fixed the SearchMediaItemsUnified query to properly handle the has_cover
parameter in three states:
- NULL (not specified): Show all books
- TRUE: Show only books with cover images
- FALSE: Show only books without cover images

Changes:
- Added explicit boolean casting (::bool) to sqlc.narg('has_cover')
  to resolve PostgreSQL type inference error (SQLSTATE 42P08)
- Replaced single AND condition with OR'd logic to handle all three
  states without mutual exclusion
- Used IS NULL check to detect when parameter is not specified
- Used IS TRUE/IS FALSE to explicitly check boolean states

The previous implementation had mutually exclusive AND conditions that
prevented any records from matching when has_cover was explicitly set
to TRUE or FALSE, causing the filter to block all searches.

This fix resolves the issue where searches were returning 0 results
regardless of other filter parameters when has_cover was included in
the query.
2026-03-27 18:07:41 -04:00
john-okeefe be31cc88f1 feat: enhance search with date-prioritized year filtering and true exact matching
Improve media item search functionality with two key enhancements:

1. Date-prioritized year filtering:
   - Prioritize date_published over copyright_year for year range queries
   - Fall back to copyright_year when date_published is NULL
   - Extract year from date_published timestamp for comparison

2. True exact search matching:
   - Replace ILIKE pattern matching with exact equality for quoted queries
   - Use search_query directly instead of wildcard pattern for exact matches
   - Remove SearchPattern parameter and related wildcard logic
   - Add COALESCE handling for author/series NULL values in exact matches

These changes make year filtering more accurate with published dates
and provide genuine exact matching when users wrap queries in quotes.

Refs internal/database/queries/queries.sql:475, internal/services/search.go:62
2026-03-26 15:35:31 -04:00
john-okeefe fcc8b38c52 chore: remove queries.sql.go.backup file
Remove outdated backup file that is no longer needed.
The generated Go code is maintained in queries.sql.go.
2026-03-25 21:00:26 -04:00
john-okeefe 0f70f74f12 fix: correct tag alias references in SearchTagsValues query
- Change all tag.value references to tag in SearchTagsValues query
- Fix PostgreSQL error: "column tag.value does not exist"
- CROSS JOIN LATERAL unnest() creates alias 'tag', not 'tag.value'
- Updates SELECT, WHERE, GROUP BY, and ORDER BY clauses
- Regenerate Go code with sqlc generate

When using CROSS JOIN LATERAL unnest(mi.tags_search) AS tag,
PostgreSQL creates 'tag' as the column alias, not 'tag.value'.
This fix aligns all references to use just 'tag', matching the
actual column name created by the LATERAL join.

Resolves tags autocomplete SQLSTATE 42703 error.

Relates to TestTagsFilter tags autocomplete test
2026-03-25 20:57:14 -04:00
john-okeefe c4ebbd990c fix: resolve tags autocomplete SQL error with CROSS JOIN LATERAL
- Fix SearchTagsValues query to use CROSS JOIN LATERAL instead of unnest() in WHERE clause
- PostgreSQL error: "set-returning functions are not allowed in WHERE"
- Change from direct unnest() calls to a proper lateral join pattern
- References: tag.value instead of repeated unnest(mi.tags_search) calls
- Regenerate Go code with sqlc generate

This fixes the tags autocomplete functionality which was failing with
SQLSTATE 0A000 error. The CROSS JOIN LATERAL approach properly expands
the tags array before filtering, allowing set-returning functions to
work correctly in the query.

Relates to TestTagsFilter tags autocomplete test
2026-03-25 20:51:21 -04:00
john-okeefe 00840c2fe1 feat: add fuzzy tags_filter to search query
- Add tags_filter parameter to SearchMediaItemsUnified
- Add EXISTS clause with word_similarity() for fuzzy tag matching
- Add tag similarity scoring to ORDER BY clause (GREATEST function)
- Add SearchTagsValues query for autocomplete with ::TEXT cast
- Keep genre_filter for backward compatibility
- Regenerate Go code with sqlc generate

This enables filtering books by tags (from Calibre) instead of genre,
which is always NULL for imported books. Uses fuzzy matching consistent
with author/series filters, with best matches sorted first.

Relates to IMPLEMENTATION_TAGS_FILTER.md Phase 1
2026-03-25 20:38:08 -04:00
john-okeefe 5571a47830 fix: eliminate duplicate search results from library visibility LEFT JOIN
Problem:
The search API was returning duplicate media items when searching across
libraries. For example, searching for "Harry" with 2 books would return
4-8 results instead of 2, depending on how many users had library visibility
entries.

Root Cause:
The SearchMediaItemsUnified query uses a LEFT JOIN with library_visibility:

  LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = $1

When multiple library_visibility entries exist for the same library
(e.g., one per user during testing), the LEFT JOIN can create duplicate
rows for each media_item. The query didn't have a DISTINCT clause to
eliminate these duplicates.

Solution:
Added DISTINCT ON (mi.id) clause with mi.id as the first ORDER BY expression:

  SELECT DISTINCT ON (mi.id) mi.*, ...
  FROM media_items mi
  ...
  ORDER BY mi.id, <other_sort_criteria>

This ensures that even if the LEFT JOIN produces multiple rows per
media_item, only one row per mi.id is returned, preserving the first
occurrence based on the relevance sorting.

Impact:
- Search results now correctly return unique media items
- Test TestCollectionSearchLibraryFilter will pass after database cleanup
- No API changes required - this is purely a query optimization

Note: After deploying this change, residual test data should be cleaned up
with: docker-compose down -v && docker-compose up -d

Files changed:
- internal/database/queries/queries.sql: Added DISTINCT ON clause
- internal/database/queries.sql.go: Regenerated from sqlc
2026-03-24 20:25:13 -04:00
john-okeefe fe8a65af84 feat: enable cross-library search in unified search query
Updates SearchMediaItemsUnified query to support searching across all
libraries when library_id parameter is not provided. Changes SQL from
requiring library_id to checking for NULL:

  AND (sqlc.narg('library_id')::uuid IS NULL
      OR mi.library_id = sqlc.narg('library_id')::uuid)

The explicit ::uuid cast ensures PostgreSQL handles type inference
correctly when comparing UUID columns with nullable parameters.

Regenerates Go database code including queries.sql.go and querier.go
to reflect the updated SQL schema.

This enables the /api/media-items/search endpoint to search all libraries
by omitting the library_id query parameter, matching the behavior of
the OPDS search endpoint.
2026-03-24 16:47:23 -04:00
john-okeefe 43a6d843a3 feat: add unified search SQL queries with fuzzy filters
- Add SearchMediaItemsUnified query combining search + filters
- Add 4 field value search queries (author, genre, series, language) for autocomplete
- Support fuzzy text matching via pg_trgm (threshold: 0.3 similarity)
- Support exact match with quotes detection for search queries
- Add sort parameter support (title ASC/DESC, author ASC/DESC, created_at ASC/DESC, page_count ASC/DESC)
- Primary sort by relevance score when searching, secondary by user-specified sort
- Combine search query with all filter types in single optimized query
- Uses 4 separate simple queries instead of 1 complex query due to sqlc v1.30.0 limitation with CASE in GROUP BY

This consolidates the deprecated /filtered and /search endpoints into one unified endpoint.
2026-03-23 22:37:37 -04:00
john-okeefe 1ee96a502e gen: regenerate database code with new search queries
Run sqlc generate to create Go code for new search queries:

Added methods to Querier interface:
- SearchMediaItemsUnified - Main unified search with fuzzy/exact matching
- SearchAuthorValues - Author field autocomplete
- SearchGenreValues - Genre field autocomplete
- SearchSeriesValues - Series field autocomplete
- SearchLanguageValues - Language field autocomplete

Generated parameter structs and row types for all new queries.
All queries include proper library visibility checks.
2026-03-22 20:35:06 -04:00
john-okeefe 26c81c8793 feat: add unified search queries with fuzzy matching
Add comprehensive search queries supporting both fuzzy and exact matching:

1. SearchMediaItemsUnified - Main search query with:
   - Fuzzy matching on author, series, genre, language filters
   - Fuzzy search on title, author, series, tags, contributors
   - Exact matching with quotes (is_exact_search flag)
   - Year range and boolean filters
   - Relevance-based ordering using word_similarity scores

2. Field-specific autocomplete queries:
   - SearchAuthorValues, SearchGenreValues, SearchSeriesValues, SearchLanguageValues
   - Each returns distinct values with counts and similarity scores
   - Threshold of 0.3 for word_similarity filter
   - Ordered by relevance (score DESC, count DESC)

Note: Using 4 separate field value queries instead of 1 complex query
due to sqlc v1.30.0 limitation with CASE expressions in GROUP BY clauses.
2026-03-22 20:35:04 -04:00
john-okeefe 85964ec932 fix(api): enforce user isolation on saved filters delete operation
Fix critical security issue where admin users could delete other users'
saved filters due to incorrect error handling in DELETE query.

Database Schema Changes:
- Change DeleteSavedFilter from :exec to :one (queries.sql:1747-1750)
- Add RETURNING * to return deleted row for proper error detection
- Regenerate querier.go and queries.sql.go with updated signature

Service Layer (internal/services/filters.go):
- Update DeleteSavedFilter to capture returned row (using _ to discard)
- Properly propagate pgx.ErrNoRows when no rows are deleted
- Error wrapping preserves original error for handler detection

Handler Layer (internal/handlers/filters.go):
- Add errors.Is() check for pgx.ErrNoRows (line 148)
- Return 404 Not Found when filter doesn't exist or belongs to different user
- Return 500 Internal Server Error for other database errors
- Add "errors" import (line 8)

Security Fix Details:
Before: Admin could delete user's filter → 204 No Content (SUCCESS)
After:  Admin tries to delete user's filter → 404 Not Found (DENIED)

The DELETE query uses WHERE id = @id AND user_id = @user_id, which matches
0 rows when attempting to delete another user's filter. The old :exec query
didn't return row count, so 0 affected rows looked like success. The new :one
query with RETURNING * returns pgx.ErrNoRows when no rows match, allowing
the handler to return proper 404 error.

Test Impact:
- TestSavedFilters/User_cannot_access_another_user's_filter now passes
- All 6 integration tests pass with proper user isolation enforcement

Pattern Consistency:
- Matches DeleteLibraryFolder pattern (line 99 in queries.sql)
- Uses same error handling as media handlers (errors.Is + pgx.ErrNoRows)
- Follows user-scoping pattern used throughout codebase

Related: Saved filters implementation user isolation
Security: Prevents unauthorized deletion of user data
2026-03-21 01:24:15 -04:00
john-okeefe e17a96123f feat(db): add saved_filters table and CRUD operations
Add database schema and SQL queries for generic saved filters system
that allows users to save custom filter presets for any resource type.

Database Schema:
- Add saved_filters table with user_id, name, resource_type, filters (JSONB)
- Create composite index on (user_id, resource_type) for efficient lookups
- Create index on (user_id, name) for future name search feature
- Add update_updated_at_column() trigger to auto-update timestamps
- Make trigger creation idempotent with DROP TRIGGER IF EXISTS

SQL Queries (5 new queries):
- GetSavedFilters: List all filters for user + resource type
- GetSavedFilterByID: Retrieve single filter by ID
- CreateSavedFilter: Create new saved filter
- UpdateSavedFilter: Update filter name/criteria
- DeleteSavedFilter: Remove saved filter

Design Decisions:
- Generic resource_type field supports any resource (media-items, collections, devices)
- JSONB filters field allows flexible schema without migrations
- User-scoped via JWT (user_id foreign key with CASCADE delete)
- Automatic updated_at timestamp via database trigger

Generated Code:
- database.SavedFilters model (10 fields including JSONB filters)
- All 5 CRUD query functions with proper parameter types
- pgtype.UUID wrappers for UUID parameters

Part of: Saved Filters Implementation (Phase 1: Database)
Related: #saved-filters-feature
2026-03-21 00:16:05 -04:00
john-okeefe 79690751c8 fix: improve type safety in media item search queries
- Change library_id parameter from interface{} to pgtype.UUID
- Add explicit UUID type casting in SQL queries
- Fix SearchMediaItemsParams to use strongly-typed UUID
- Prevents potential type assertion errors and improves type safety
- Ensures proper NULL handling for optional library_id filter
2026-03-06 01:52:33 -05:00
john-okeefe 9b3d8cc949 feat: implement collection library filter with WebSocket improvements and test coverage
This commit adds comprehensive functionality for filtering collections by library,
improves WebSocket real-time updates with user activity detection, and adds
extensive test coverage.

## Core Features

### Collection Library Filter
- Added library_id parameter to media-items search API
- Collections can now be filtered by specific library
- Toggle UI component for enabling/disabling library filter
- Default state is "checked" when library_id is present
- Consistent behavior across partial and fuzzy search modes

### WebSocket Auto-Reload Mitigation
- Added user activity detection to prevent disruptive page reloads
- Checks if user is actively typing in INPUT/TEXTAREA/SELECT elements
- Skips auto-reload when user is interacting with form elements
- Toast notifications still show for awareness
- Prevents data loss during editing operations

## Implementation Changes

### Backend
- internal/database/queries.sql.go: Added library filter support to search queries
- internal/handlers/media.go: Enhanced search with library_id parameter validation
- internal/handlers/collections.go: Updated collection handlers with library filtering
- internal/sync/websocket.go: Improved broadcast mechanism with user-scoped updates
- internal/router/frontend.go: Pass libraryID to collection templates

### Frontend
- templates/collections.templ: Added library filter toggle UI component
- web/src/collections.ts: TypeScript implementation with WebSocket integration
- templates/collections_templ.go: Generated template code

### Testing
- cmd/server/tests/search_test.go: Added TestCollectionSearchLibraryFilter
- cmd/server/tests/websocket_test.go: Added TestWebSocketUserScopedBroadcast
- New helper functions for creating libraries and media items via API
- Comprehensive test coverage for library filtering and user-scoped broadcasts

## API Documentation Updates

### Bruno Tests (Comprehensive Documentation)
- bruno/collections/*: Added detailed API documentation for all collection endpoints
- bruno/devices/*: Added device management and sync API documentation
- bruno/devices/kobo/api.yml: Kobo-specific sync protocol docs
- bruno/devices/koreader/api.yml: KOReader-specific sync protocol docs
- bruno/opds/*: Added OPDS feed and download endpoint documentation
- bruno/library/browse-folders.yml: Library folder browsing API docs

### New Bruno Tests
- bruno/media-items/Search All Libraries.yml: Test search without library filter
- bruno/media-items/Search Specific Library.yml: Test search with library filter
- bruno/media-items/Search Invalid Library ID.yml: Test error handling

## Documentation

- docs/developer/api/media-items/search_media_items.md: Updated with library_id parameter
- IMPLEMENTATION_COLLECTION_FIX.md: Comprehensive implementation guide with test scenarios

## Testing

### Integration Tests
- Library filter tests verify correct filtering across multiple libraries
- Invalid library_id tests ensure proper error handling
- WebSocket tests verify user-scoped broadcast behavior
- User A no longer receives User B's collection updates

### Manual Testing Scenarios
- Open collection in multiple tabs - updates propagate correctly
- Type in search box while another tab adds books - no disruptive reload
- Add/remove books from collection - toast notifications appear
- Toggle library filter - results update dynamically

## Technical Details

- WebSocket broadcasts are now user-scoped for privacy
- Active element detection uses tagName and contenteditable attributes
- Library ID validation uses UUID format checking
- Progressive enhancement maintained - page works without JavaScript
- All changes follow PROJECT_GUIDELINES.md conventions
- TypeScript only for frontend logic
- TailwindCSS only for styling
- Procedural programming style throughout

## Breaking Changes

None - all changes are additive and backward compatible.
2026-03-04 22:37:47 -05:00
john-okeefe eb2da1e05b fix: Change library ordering to oldest-first
Change library ordering in dropdown from DESC to ASC to display
libraries in creation order (oldest first).

Database changes in internal/database/queries/queries.sql:
- Modify GetUserLibraries query ORDER BY clause
  - Change from ORDER BY l.created_at DESC to ASC
  - Displays oldest libraries first in dropdown

This provides a more intuitive ordering where users see their
first-created libraries at the top of the list.
2026-03-01 00:29:27 -05:00
john-okeefe 209e9f2a3c feat: implement relative path storage and URL resolution for media files
- Add libraryService dependency to CollectionHandler and OPDSHandler for centralized path resolution
- Create internal/utils/mediaurl.go with ResolveMediaURL() function as single source of truth
- Update GetMediaItem and ListMediaItems handlers to return resolved URLs in API responses
- Update collection handlers (GetCollection, TestRules, PreviewCollection) to use resolved cover URLs
- Update progress handler (GetAllProgress) to use resolved cover URLs
- Add library_id to GetCollectionItems SQL query to enable URL resolution
- Refactor media scanner to store relative paths instead of absolute filesystem paths
- Add ResolveMediaPath() to LibraryService for resolving relative paths to absolute paths
- Add ServeFile endpoint at /uploads/library-:id/* for authenticated file serving
- Add MimeTypes map to library_service.go for consistent MIME type handling
- Update DownloadBook handler to use resolved filesystem paths
- Add getRelativePath() helper to MediaScanner for converting absolute to relative paths
- Use strings.EqualFold for case-insensitive path comparisons in zip extraction

This change enables the application to work with relative paths stored in the
database, making it portable across different server environments while
maintaining backward compatibility with existing absolute paths.
2026-02-27 16:51:44 -05:00
john-okeefe d802236874 scanner: fix library isolation, file mtime, force rescan, and deletion handling
Fix 1 - File modification time for created_at:
- Get file.ModTime() in processMediaFile and pass to CreateMediaItem
- Modified SQL INSERT to include created_at column

Fix 2 - Force rescan UPDATE instead of DELETE+INSERT:
- Changed force rescan logic to call updateMediaItem instead of delete + create
- Preserves created_at timestamp on force rescan

Fix 3 - GetMediaItemByFilePath filters by library_id:
- Added library_id to WHERE clause in SQL query
- Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader)
- Added SetLibraryID method to MediaScanner
- Updated handler to call SetLibraryID for watch mode

Fix 4 - File deletion handling with persistent logging:
- Added fsnotify.Remove handler in WatchChanges
- Added orphan cleanup in ScanFolders after scan completes
- Created scanner_logger.go with daily log rotation (7 days)
- Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log
- Individual deletes with enhanced safety logging

Note: Integration tests can now safely scan /app/uploads because
GetMediaItemByFilePath now filters by library_id, preventing
cross-library interference.
2026-02-26 16:39:42 -05:00
john-okeefe e3a3aa124f feat(api): consolidate user profile update endpoints
Add delete_user, reset_user_password, and update_user endpoints to replace
individual update operations. Update database schema to include deleted_at
column for soft deletion. Add DeleteUser, ResetUserPassword, and
UpdateUserAdmin queries. Update Querier with new methods for user management.
2026-02-22 01:57:42 -05:00
john-okeefe dd1e56d2f7 chore(dashboard): regenerate database code from Phase 3 queries
Run sqlc generate to create Go code for dashboard queries:
- GetDashboardPreferences / UpsertDashboardPreferences / UpdateDashboardPreferences
- GetSystemCollectionsForDashboard / GetUserCollectionsForDashboard
- DeleteUserSystemCollection
- GetContinueReadingItems / GetRecentlyAddedItems / GetRecentlyReadItems / GetNotStartedItems
- GetCollectionItemsForDashboard / GetLibraryItems

Auto-generated from queries.sql changes.
2026-02-19 20:55:59 -05:00
john-okeefe 1f80f6acfd feat(dashboard): add Phase 3 database queries for Carousel-style dashboard
Add SQL queries for dashboard functionality and system collections:

Dashboard Preferences Queries:
- GetDashboardPreferences: Fetch user preferences for a library
- UpsertDashboardPreferences: Create or update user dashboard preferences
- UpdateDashboardPreferences: Update existing preferences

Dashboard Collections Queries:
- GetSystemCollectionsForDashboard: Fetch system collections (user_id IS NULL)
- GetUserCollectionsForDashboard: Fetch user collections marked for dashboard
- DeleteUserSystemCollection: Delete user's copy of a system collection

System Collection Smart Queries:
- GetContinueReadingItems: Books with 0 < progress < 1
- GetRecentlyAddedItems: Newly added items to library
- GetRecentlyReadItems: Books with progress >= 1
- GetNotStartedItems: Books with progress = 0 or no record

Collection Management Queries:
- GetCollectionItemsForDashboard: Fetch collection items with excluded flag
- GetLibraryItems: Fetch all items in a library

These queries support the unified collections architecture where system
defaults and user-created sections are both collections with user_id
NULL for system-owned and NOT NULL for user-created.
2026-02-19 20:55:54 -05:00
john-okeefe 3af2fb0ba4 schema(dashboard): implement Phase 1 unified collections architecture
Add support for Carousel-style dashboard with unified collections architecture:

Database Schema Changes:
- Add user_dashboard_preferences table:
  - hidden_collections: TEXT[] for managing section visibility
  - collection_order: TEXT[] for custom ordering
  - items_per_section: INT for limiting items per section
- Update collections table:
  - user_id: Make nullable to support system-owned collections (NULL = system)
  - show_on_dashboard: BOOLEAN for controlling visibility
  - query_type: TEXT for different query types (continue-reading, recently-added, etc.)
  - priority: INT for display order (lower = higher priority)
  - is_system_collection: BOOLEAN for flagging system defaults
- Update collection_items table:
  - Add excluded BOOLEAN for user overrides of auto-assigned items

Indexes:
- idx_collections_dashboard: (user_id, show_on_dashboard, priority) WHERE show_on_dashboard = true
- idx_dashboard_prefs_user_library: (user_id, library_id)
- idx_collection_items_excluded: (collection_id, excluded) WHERE excluded = true

System Collections (pre-seeded defaults):
- continue-reading: Books with 0 < progress < 1
- recently-added: Newly added items to library
- recently-read: Books with progress >= 1
- not-started: Books with progress = 0 or no record

This implements Phase 1 of the Carousel-style dashboard redesign plan.
2026-02-19 20:52:21 -05:00
john-okeefe 557f057621 fix(db): cast status to varchar in sync queue update for proper enum comparison 2026-02-14 21:37:38 -05:00
john-okeefe 2706ae52c1 refactor: remove Phase X terminology from source code comments
Remove planning document phase references from code comments:

app_test.go:
- Remove Phase 5 references from 8 test function comments

querier.go & queries.sql.go:
- Remove Phase 1, 2, 3, 4, 6 references from section headers
- Clean up week numbers (Weeks 5-6, Week 3-4, etc.)

queries.sql:
- Remove Phase 4 references from Kobo queries

kobo.go:
- Remove Phase 6 references from ContentId mapping comments

progress.go:
- Remove Phase 1 reference from route comment

media_scanner.go & media_scanner_library_type_test.go:
- Remove Phase 2 references from library type scanning comments

schema.sql:
- Remove Phase 1, 2, 3, 4, 5, 7 references from table/section comments
- Clean up: Format Detection, Progress Tracking, Device Registry,
  Sync Queue, Conflict Resolution, Reading History, Indexes, etc.

test_helpers.go:
- Remove Phase 6 reference from handler setup comment

These phase numbers were from internal planning documents and have no
meaning in the codebase. Removing them makes the code self-documenting.
2026-02-13 21:50:29 -05:00
john-okeefe 8321149957 test: add Bruno API test collections for device authentication
- Device token regeneration tests (success, forbidden, not found, unauthorized)
- OPDS authentication tests (Bearer token, query token)
- Kobo sync tests with token authentication
- Test various authentication methods and error cases
2026-02-13 12:12:17 -05:00
john-okeefe 1f65933653 fix: handle schema.sql file path in containerized environment
- Try multiple locations for schema.sql file
- Support both local dev and containerized deployment paths
- Add informative logging when schema is loaded
- Prevent runtime.Caller issues in containers

Locations checked:
- database/schema/schema.sql (working directory)
- /app/database/schema/schema.sql (container)
- ../database/schema/schema.sql (relative)
- ../../database/schema/schema.sql (relative)

This fixes the 'no such file or directory' error in production containers.
2026-02-10 16:52:12 -05:00
john-okeefe 9d1a1e0228 feat: add schema initialization logic (Phase 2)
- Create internal/database/schema.go with full initialization logic
- Parse table names from schema.sql using regex (handles both formats)
- Execute schema in atomic transaction
- Verify all expected tables exist
- Verify all critical functions exist (6 functions)
- PostgreSQL advisory locking with 30-second timeout
- Self-healing from partial/corrupted state
- Load schema.sql from filesystem at runtime

Features:
- Defensive regex handles IF NOT EXISTS and legacy CREATE TABLE
- Lock timeout prevents indefinite hangs
- Function verification ensures sync operations work
- Clear error messages with debug hints
2026-02-10 16:48:28 -05:00
john-okeefe b506046be0 chore(db): regenerate database code from sqlc
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods

Generated via: cd internal/database && sqlc generate
2026-02-09 20:09:58 -05:00
john-okeefe 363e747cb3 feat(db): add system settings queries and enhance user queries
System Settings Migration:
- Add GetSystemSetting query for single setting retrieval
- Add UpdateSystemSetting query for updating settings
- Add GetAllSystemSettings query for all settings
- Remove UpdateScanSettings and GetScanSettings (per-user queries)

User Query Enhancement:
- Add max_devices field to GetUser query
- Add device_count computed field to GetUser query
- Add max_devices field to ListUsers query
- Add device_count computed field to ListUsers query

These changes support:
1. System-wide scan settings instead of per-user settings
2. Users can now see their device count and limits
3. Admins can monitor device usage across all users
2026-02-09 20:09:48 -05:00
john-okeefe eed1ef37dc Change UpdateUserMaxDevices to return updated user record
- Change query from :exec to :one with RETURNING *
- Allows handler to detect when user doesn't exist
- Follows pattern established by UpdateMediaItem
- Required for 404 response on non-existent user

Related: Fix for TestUpdateUserMaxDevicesNonExistentUser
2026-02-09 15:46:57 -05:00
john-okeefe 701815c1bc docs: update documentation for media scanner naming
- Update development.md with new file names (scanner.go, media_scanner.go)
- Update API reference: "Book/ebook operations" -> "Media item operations"
- Remove historical migration comments from schema:
  - Simplify media_items table comment
  - Remove backward compatibility comments for reading_progress and media_ratings
  - Remove note about backward compatibility views
- Remove historical comment from queries.sql about ebook folders
2026-02-08 14:55:06 -05:00
john-okeefe 9e166c9670 chore(database): remove unused EbookNote backward compatibility functions
- Remove CreateEbookNote, UpdateEbookNote, DeleteEbookNote queries
- These were marked as backward compatibility but never used
- API uses CreateMediaNote, UpdateMediaNote, DeleteMediaNote instead
- Remove misleading backward compatibility comments
- Regenerate sqlc code
2026-02-08 14:28:08 -05:00
john-okeefe c606a7ffcc feat: add tags_search and contributors_search fields to database
Schema changes:
- Add tags_search TEXT[] column for case-insensitive, punctuation-free search
- Add contributors_search TEXT[] column for case-insensitive, punctuation-free search
- Create GIN indexes for fast array searches on both search fields

Query updates:
- CreateMediaItem: Include tags_search and contributors_search parameters
- UpdateMediaItem: Include tags_search and contributors_search parameters
- SearchMediaItems: Search against tags_search instead of tags
- SearchMediaItems: Search against contributors_search instead of contributors
- SearchMediaItemsFuzzy: Use tags_search and contributors_search for fuzzy matching
- Update ranking and priority logic to use search fields

Benefits:
- Case-insensitive search: "acme corp" finds "ACME CORP."
- Punctuation-agnostic search: "oreilly" finds "O'Reilly Media"
- Better UX: Users don't need to match exact casing or punctuation
- Improved performance with dedicated GIN indexes

Relates to Tags & Contributors Migration Phases 5 & 8
2026-02-08 11:05:18 -05:00
john-okeefe 516cec5a7f feat: migrate tags and contributors from TEXT to TEXT[] arrays
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.

Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays

Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling

Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function

Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import

New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties

API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string

Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)

Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
2026-02-07 22:53:12 -05:00
john-okeefe 6592db2c65 Update Go struct field names: BookmannUuid → BookhoardUuid
Complete the rename by updating:
- DeviceCatalogs struct field: BookmannUuid → BookhoardUuid (models.go)
- Generated queries: Update all references (queries.sql.go)
- Local variables: bookmannUUID → bookhoardUUID (kobo.go)
- Struct field access: catalog.BookmannUuid → catalog.BookhoardUuid

All "bookmann" and "BOOKMANN" references are now eliminated from the codebase.

Part of project rename to Bookhoard.
2026-02-01 16:26:59 -05:00
john-okeefe 5b9f21a592 Final cleanup: Update remaining comments and variable names
Changes:
- Update comments: "Bookmann UUID" → "Bookhoard UUID"
- Rename sidecar struct field: Bookmann → Bookhoard
- Update type names: SidecarBookmannConfig → SidecarBookhoardConfig
- Fix test database name in queue_test.go
- Fix uppercase env var examples in KOBO_SETUP.md

Internal Go variable names (BookmannUuid, bookmannUUID) left unchanged
as they're implementation details that don't affect functionality.

Part of project rename to Bookhoard.
2026-02-01 16:24:58 -05:00
john-okeefe 153b330dd1 Update database schema and queries: Bookmann → Bookhoard
Database changes:
- schema.sql: Update column name bookmann_uuid → bookhoard_uuid
- schema.sql: Update index names and example URLs
- queries.sql: Update all SQL queries to use bookhoard_uuid
- Update example configuration values

Part of project rename to Bookhoard.
2026-02-01 16:21:03 -05:00
john-okeefe d43e0105eb Rename project module and database references: Bookmann → Bookhoard
Core infrastructure changes:
- Update Go module name: bookmann → bookhoard
- Rename database schema references and comments
- Update database column names: bookmann_uuid → bookhoard_uuid
- Rename SQL query functions: GetDeviceCatalogByBookmannUUID → GetDeviceCatalogByBookhoardUUID
- Update configuration defaults

This is part 1 of the project rename to Bookhoard.
2026-02-01 16:11:47 -05:00
john-okeefe f23b6d35e9 feat(database): add analytics and book matching queries
Add analytics queries:
- GetUserReadingHistory: detailed reading history with device info
- GetUserDeviceUsage: device usage statistics (sync count, time spent)
- GetPopularBooks: most read books with completion rates

Add book matching queries:
- GetUnlinkedBookByID: fetch single unlinked book
- DeleteUnlinkedBook: remove resolved unlinked book
- ListUnresolvedUnlinkedBooks: paginated list of unresolved books
2026-02-01 12:15:33 -05:00
john-okeefe 63008001c6 feat(db): Add kobo_shelves and device catalog tables
- Add kobo_shelves table for Kobo device shelves management
- Add device_shelf_mappings for collection<->device shelf mappings
- Update device_catalogs with better ContentId tracking
- Add indexes for performance
2026-01-31 22:32:04 -05:00
john-okeefe 030d30a225 Add queue management API and database queries
- Add 7 new database queries for queue management
- GetStuckSyncQueueItems - Detect stuck items
- GetSyncQueueStats - Queue statistics
- GetNextRetryTime - Exponential backoff calc
- ListAllSyncQueueItems - Admin view
- IncrementSyncQueueAttempts - Retry counter
- Add queue handler with 7 REST endpoints
- GET /api/queue/devices/:id/stats - Queue statistics
- GET /api/queue/devices/:id/items - List device queue
- POST /api/queue/items/:id/retry - Retry failed item
- DELETE /api/queue/items/:id - Delete queue item
- DELETE /api/queue/devices/:id/clear - Clear device queue
- GET /api/queue/items - List all items (admin)
- Add full user/admin access control
2026-01-31 13:06:12 -05:00
john-okeefe 2d2d643873 Add sync conflict detection and resolution system
Implement conflict detection for concurrent reading progress updates from different devices. Adds conflict management endpoints for listing, viewing, and resolving conflicts.

- Add ConflictHandler with CRUD endpoints for conflict management
- Implement automatic conflict detection in KOReader progress updates
- Add WebSocket broadcast for real-time conflict notifications
- Add database query for listing user conflicts by status
- Add integration tests and Bruno API test collection
2026-01-31 11:45:52 -05:00