Commit Graph
29 Commits
Author SHA1 Message Date
john-okeefe d0460885ff feat(db): add imported_at column to media_items for accurate "Recently Added" sorting
The created_at column stores file modification time (intentional for preserving
original metadata), but this makes 'Recently Added' sorting unreliable for
imported files. Add imported_at column that records the actual database insert
timestamp.

Changes:
- Add imported_at TIMESTAMPTZ column to media_items (nullable)
- Update GetRecentlyAddedItems to sort by imported_at DESC NULLS LAST first
- Add ReassignLibraries and ReassignMediaItems queries for admin deletion handover
- Add SyncLibraryTypeExtensions query for startup extension sync
- Update all media_items SELECT queries to include imported_at column
2026-05-16 19:30:27 -04:00
john-okeefe caf50ade31 Add timezone support to database schema and queries
- Add timezone column (VARCHAR(50) DEFAULT 'UTC') to users table
- Add default_timezone row to system_settings seed data
- Add idx_users_timezone index for user timezone lookups
- Add UpdateUserTimezone and GetSystemTimezone queries
- Regenerate sqlc code (models, querier, queries.sql.go)
- Reuse existing UpdateSystemSetting for system timezone updates
  instead of creating a redundant UpdateSystemTimezone query
2026-04-27 21:30:37 -04:00
john-okeefe 059955be72 chore(db): Regenerate database code from processing issues queries
- Add ProcessingIssues model struct
- Update Querier interface with processing issues methods
- Add generated query implementations for CreateProcessingIssue, ListProcessingIssuesByLibrary, GetProcessingIssueStats, ResolveProcessingIssue, DeleteProcessingIssue
- Add GetLibraryWithType query for fetching library with type information
2026-04-12 20:43:46 -04:00
john-okeefe cf8c690474 Add library_type_name column to media_items with auto-population trigger
- Add library_type_name VARCHAR(50) column to media_items table in schema.sql
- Create PostgreSQL trigger 'set_library_type_name_on_insert' that automatically
  populates library_type_name by joining libraries table with library_types on insert
- Add library_type_name parameter to CreateMediaItem SQL query
- Update Go models (models.go) to include LibraryTypeName field
- Add UpdateMediaItemChapterMetadata query method to querier.go
- Regenerate queries.sql.go with sqlc

This allows media items to store their library type (e.g., 'Books', 'Comics', 'Manga')
at the database level, enabling filtering and display without needing additional joins.
2026-04-04 22:49:07 -04:00
john-okeefe 74575d9a86 feat: add reader service, handler, and router 2026-04-03 16:53:37 -04:00
john-okeefe 0667cad8a9 feat: add reader infrastructure - Phase 0 database schema and queries
Implement Phase 0 prerequisites for reader functionality including
database schema, SQL queries, and frontend dependencies.

## Database Schema (5 New Tables + 1 Column Addition)

### New Tables Added:
1. **panel_data** - Comic/manga panel detection results
   - Stores detected panel boundaries (x, y, width, height)
   - Supports grid, ML, and manual detection methods
   - JSONB storage for flexible panel structures

2. **reading_speed** - User reading speed statistics
   - Tracks pages per minute and total reading time
   - Per-user per-media-item tracking
   - Enables progress estimation and analytics

3. **dictionary_cache** - Offline dictionary word definitions
   - Caches external dictionary lookups
   - Reduces API calls and improves performance
   - Supports offline reading functionality

4. **reader_settings** - User reader preferences (per-user)
   - Stores typography, theme, and display settings
   - JSONB storage for flexible configuration
   - Per-user customization (fonts, margins, themes)

5. **media_bookmarks** - Enhanced bookmarks with chapter/CFI support
   - Unified bookmarking for ebooks, comics, manga, PDFs
   - Supports page_number, chapter_number, and epubcfi_position
   - Includes notes field for annotations
   - Unique constraint on (media_item_id, user_id, title)

### Column Addition:
- **media_items.chapter_metadata** (JSONB) - Caches detected chapter structure
  - Stores TOC/chapter detection results
  - Prevents re-parsing files on every read
  - Populated by ReaderService.DetectChapters()

## Database Queries (12 New Queries)

Added queries for all reader functionality:
- Panel data: GetPanelData, UpsertPanelData
- Reading speed: GetReadingSpeed, CreateReadingSpeed, UpdateReadingSpeed
- Dictionary: GetDictionaryEntry, CreateDictionaryEntry, UpdateDictionaryAccessed
- Settings: GetReaderSettings, UpsertReaderSettings
- Bookmarks: GetMediaBookmarks, CreateMediaBookmark, DeleteMediaBookmark, UpdateMediaBookmark

## Frontend Dependencies

Added to package.json:
- jszip@^3.10.1 - EPUB/comic archive parsing (client-side)
- pdfjs-dist@^3.11.174 - PDF rendering library (Mozilla PDF.js)

## Generated Code

Ran `sqlc generate` to regenerate:
- models.go - Go structs for new tables (55 lines added)
- querier.go - Database query methods (14 lines added)
- queries.sql.go - Compiled SQL queries (504 lines added)

## Implementation Status

Phase 0 prerequisites now complete:
 Database schema (5 tables + 1 column)
 SQL queries (12 queries)
 Frontend dependencies (2 packages)
 Generated Go code (sqlc)
 Database recreated with new schema

Ready for Phase 1: Infrastructure & Basic Reader implementation.

Related to: Universal web reader for ebooks, comics, manga, PDFs
2026-04-02 21:01:45 -04:00
john-okeefe dd81dc08a2 db: update SQL queries and regenerate models for comic metadata
Phase 1 implementation: Update CreateMediaItem query to support 14 new comic metadata fields.

Changes:
- Add 14 new columns to CreateMediaItem INSERT statement
- Regenerate sqlc models with new fields
- CommunityRating now maps to pgtype.Float8 (was pgtype.Numeric)
- All new comic and universal metadata fields included

New fields supported:
- Reading direction: manga_type, reading_direction
- Universal: series_count, volume, imprint, age_rating, web_url
- Comic-specific: story_arc, is_black_and_white, metadata_notes,
  community_rating, alternate_info, scan_information, summary

Generated models verified:
- MediaItems struct includes all 14 new fields
- CreateMediaItemParams has correct parameter count (42 total)
- CommunityRating is pgtype.Float8 (not pgtype.Numeric)

Relates to: Phase 4.1-4.2 database layer implementation
2026-03-29 21:12:20 -04:00
john-okeefe 9e3e0ff931 db: regenerate sqlc models with comic metadata fields
Auto-generated by sqlc from updated schema.sql
- MediaItems struct now includes all 14 new comic metadata fields
- CreateMediaItemParams struct updated with new parameters
- All SELECT queries now include new columns in RETURNING clauses
- Properly typed with pgtype.Text, pgtype.Int4, pgtype.Bool, pgtype.Numeric, []byte

New Fields in MediaItems:
- MangaType pgtype.Text
- ReadingDirection pgtype.Text
- SeriesCount pgtype.Int4
- Volume pgtype.Int4
- Imprint pgtype.Text
- AgeRating pgtype.Text
- WebUrl pgtype.Text
- StoryArc pgtype.Text
- IsBlackAndWhite pgtype.Bool
- MetadataNotes pgtype.Text
- CommunityRating pgtype.Numeric
- AlternateInfo []byte (JSONB)
- ScanInformation pgtype.Text
- Summary pgtype.Text

Part of Phase 1: Database Schema Changes
Implementation: IMPLEMENTATION_PLAN_MERGE_METADATA_READING_DIRECTION.md
2026-03-29 19:11:51 -04:00
john-okeefe e17a96123f feat(db): add saved_filters table and CRUD operations
Add database schema and SQL queries for generic saved filters system
that allows users to save custom filter presets for any resource type.

Database Schema:
- Add saved_filters table with user_id, name, resource_type, filters (JSONB)
- Create composite index on (user_id, resource_type) for efficient lookups
- Create index on (user_id, name) for future name search feature
- Add update_updated_at_column() trigger to auto-update timestamps
- Make trigger creation idempotent with DROP TRIGGER IF EXISTS

SQL Queries (5 new queries):
- GetSavedFilters: List all filters for user + resource type
- GetSavedFilterByID: Retrieve single filter by ID
- CreateSavedFilter: Create new saved filter
- UpdateSavedFilter: Update filter name/criteria
- DeleteSavedFilter: Remove saved filter

Design Decisions:
- Generic resource_type field supports any resource (media-items, collections, devices)
- JSONB filters field allows flexible schema without migrations
- User-scoped via JWT (user_id foreign key with CASCADE delete)
- Automatic updated_at timestamp via database trigger

Generated Code:
- database.SavedFilters model (10 fields including JSONB filters)
- All 5 CRUD query functions with proper parameter types
- pgtype.UUID wrappers for UUID parameters

Part of: Saved Filters Implementation (Phase 1: Database)
Related: #saved-filters-feature
2026-03-21 00:16:05 -04:00
john-okeefe 3af2fb0ba4 schema(dashboard): implement Phase 1 unified collections architecture
Add support for Carousel-style dashboard with unified collections architecture:

Database Schema Changes:
- Add user_dashboard_preferences table:
  - hidden_collections: TEXT[] for managing section visibility
  - collection_order: TEXT[] for custom ordering
  - items_per_section: INT for limiting items per section
- Update collections table:
  - user_id: Make nullable to support system-owned collections (NULL = system)
  - show_on_dashboard: BOOLEAN for controlling visibility
  - query_type: TEXT for different query types (continue-reading, recently-added, etc.)
  - priority: INT for display order (lower = higher priority)
  - is_system_collection: BOOLEAN for flagging system defaults
- Update collection_items table:
  - Add excluded BOOLEAN for user overrides of auto-assigned items

Indexes:
- idx_collections_dashboard: (user_id, show_on_dashboard, priority) WHERE show_on_dashboard = true
- idx_dashboard_prefs_user_library: (user_id, library_id)
- idx_collection_items_excluded: (collection_id, excluded) WHERE excluded = true

System Collections (pre-seeded defaults):
- continue-reading: Books with 0 < progress < 1
- recently-added: Newly added items to library
- recently-read: Books with progress >= 1
- not-started: Books with progress = 0 or no record

This implements Phase 1 of the Carousel-style dashboard redesign plan.
2026-02-19 20:52:21 -05:00
john-okeefe b506046be0 chore(db): regenerate database code from sqlc
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods

Generated via: cd internal/database && sqlc generate
2026-02-09 20:09:58 -05:00
john-okeefe c606a7ffcc feat: add tags_search and contributors_search fields to database
Schema changes:
- Add tags_search TEXT[] column for case-insensitive, punctuation-free search
- Add contributors_search TEXT[] column for case-insensitive, punctuation-free search
- Create GIN indexes for fast array searches on both search fields

Query updates:
- CreateMediaItem: Include tags_search and contributors_search parameters
- UpdateMediaItem: Include tags_search and contributors_search parameters
- SearchMediaItems: Search against tags_search instead of tags
- SearchMediaItems: Search against contributors_search instead of contributors
- SearchMediaItemsFuzzy: Use tags_search and contributors_search for fuzzy matching
- Update ranking and priority logic to use search fields

Benefits:
- Case-insensitive search: "acme corp" finds "ACME CORP."
- Punctuation-agnostic search: "oreilly" finds "O'Reilly Media"
- Better UX: Users don't need to match exact casing or punctuation
- Improved performance with dedicated GIN indexes

Relates to Tags & Contributors Migration Phases 5 & 8
2026-02-08 11:05:18 -05:00
john-okeefe 516cec5a7f feat: migrate tags and contributors from TEXT to TEXT[] arrays
Convert tags and contributors columns from comma-separated strings to PostgreSQL
TEXT[] arrays for better data normalization and query performance.

Database Changes:
- schema.sql: Change tags/contributors from TEXT to TEXT[]
- schema.sql: Add GIN indexes for fast array searches
- queries.sql: Update search queries to use ANY() operator
- queries.sql: Update fuzzy search with unnest() for arrays

Generated Code (sqlc):
- models.go: Auto-generated with []string types for tags/contributors
- queries.sql.go: Auto-generated with proper array handling

Handler Changes:
- media.go: Update request structs to use []string for tags/contributors
- media.go: Remove pgtype.Text wrapping, use direct array assignment
- media.go: Add tag normalization in CreateMediaItemHandler
- collections.go: Update tags evaluation to join arrays for comparison
- collections.go: Add strings import for Join() function

Service Changes:
- ebook_scanner.go: Update EbookMetadata struct to use []string
- ebook_scanner.go: Remove string Join(), assign arrays directly
- collection_service.go: Update tags rule evaluation to join arrays
- collection_service.go: Add strings import

New Utilities:
- internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags()
- Normalizes tags by trimming, lowercasing, removing duplicates/empties

API Documentation:
- bruno/media-items/Create Media Item.bru: Update examples to use arrays
- bruno/media-items/Update Media Item.bru: Update examples to use arrays
- Update docs: tags/contributors now array of string

Breaking Change:
- JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"]
- Tests already use array format (no changes needed)

Benefits:
- GIN indexes enable faster array searches
- Normalization prevents data quality issues (case, duplicates)
- Array operations use PostgreSQL native operators (ANY, &&, unnest)
- Better separation of concerns (no string parsing in application)
2026-02-07 22:53:12 -05:00
john-okeefe 6592db2c65 Update Go struct field names: BookmannUuid → BookhoardUuid
Complete the rename by updating:
- DeviceCatalogs struct field: BookmannUuid → BookhoardUuid (models.go)
- Generated queries: Update all references (queries.sql.go)
- Local variables: bookmannUUID → bookhoardUUID (kobo.go)
- Struct field access: catalog.BookmannUuid → catalog.BookhoardUuid

All "bookmann" and "BOOKMANN" references are now eliminated from the codebase.

Part of project rename to Bookhoard.
2026-02-01 16:26:59 -05:00
john-okeefe d43e0105eb Rename project module and database references: Bookmann → Bookhoard
Core infrastructure changes:
- Update Go module name: bookmann → bookhoard
- Rename database schema references and comments
- Update database column names: bookmann_uuid → bookhoard_uuid
- Rename SQL query functions: GetDeviceCatalogByBookmannUUID → GetDeviceCatalogByBookhoardUUID
- Update configuration defaults

This is part 1 of the project rename to Bookhoard.
2026-02-01 16:11:47 -05:00
john-okeefe 63008001c6 feat(db): Add kobo_shelves and device catalog tables
- Add kobo_shelves table for Kobo device shelves management
- Add device_shelf_mappings for collection<->device shelf mappings
- Update device_catalogs with better ContentId tracking
- Add indexes for performance
2026-01-31 22:32:04 -05:00
john-okeefe 030d30a225 Add queue management API and database queries
- Add 7 new database queries for queue management
- GetStuckSyncQueueItems - Detect stuck items
- GetSyncQueueStats - Queue statistics
- GetNextRetryTime - Exponential backoff calc
- ListAllSyncQueueItems - Admin view
- IncrementSyncQueueAttempts - Retry counter
- Add queue handler with 7 REST endpoints
- GET /api/queue/devices/:id/stats - Queue statistics
- GET /api/queue/devices/:id/items - List device queue
- POST /api/queue/items/:id/retry - Retry failed item
- DELETE /api/queue/items/:id - Delete queue item
- DELETE /api/queue/devices/:id/clear - Clear device queue
- GET /api/queue/items - List all items (admin)
- Add full user/admin access control
2026-01-31 13:06:12 -05:00
john-okeefe db5d51e77e Regenerate database code for UUID token support
- Update models: RefreshTokens.Token now pgtype.UUID
- Update querier: GetRefreshToken/RevokeRefreshToken accept pgtype.UUID
- Regenerate queries.sql.go via sqlc after schema change
2026-01-31 11:44:26 -05:00
john-okeefe ed336c6c41 feat: add Kobo entitlements and shelves models 2026-01-31 00:29:37 -05:00
john-okeefe fa4a9c35bb Phase 1 Week 1: Database schema for universal sync system
- Add format_group columns to media_items table
- Add universal progress tracking to reading_progress (percentage, epubcfi, chapter, etc.)
- Add device sync metadata (last_sync_device, conflict tracking)
- Add location enhancements to media_notes and media_highlights
- Create devices table for device registry
- Create sync_queue table for offline support
- Create sync_conflicts table for conflict resolution
- Create reading_history table for session tracking
- Add 15 new indexes for performance
- Create update_updated_at_column trigger function
- Add SQL helper functions: detect_format_group, convert_progress, detect_conflict, merge_progress

Schema grew from 272 to 598 lines (+326 lines)
Verified with sqlc generate
2026-01-30 16:04:06 -05:00
john-okeefe fc71c2ef76 fix: database queries and schema sync
- Remove references to non-existent columns (language, edition, page_count, etc.)
- Fix CreateMediaItem and UpdateMediaItem queries
- Remove normalize_isbn() function calls (moved to Go code)
- Regenerate database code with sqlc generate
- Add GetLibraryByFolder method to queries

Fixes compiler error: s.db.GetLibraryByFolder undefined
2026-01-30 13:51:44 -05:00
john-okeefe f96044b6c7 refactor: remove ebook system, unify on media-items
Phase 1-3: Database layer cleanup
- Remove 5 backward compatibility VIEWs (ebooks, ebook_ratings, etc.)
- Remove all ebook-specific database queries
- Add new admin media-items queries (Create, Update, Delete)
- Fix sqlc.yaml to point to schema.sql file
- Regenerate database code successfully

Phase 4: Remove old ebook handlers
- Remove all 23 ebook handler functions:
  * ListEbooks, GetEbook, CreateEbook, UpdateEbook, DeleteEbook
  * GetEbookRating, CreateOrUpdateEbookRating, DeleteEbookRating, GetEbookRatings
  * GetEbookNotes, CreateEbookNote, GetEbookNote, UpdateEbookNote, DeleteEbookNote
  * GetEbookHighlights, CreateEbookHighlight, GetEbookHighlight, UpdateEbookHighlight, DeleteEbookHighlight
  * GetReadingProgress, UpdateReadingProgress
- Remove ebook request types (CreateEbookRequest, UpdateEbookRequest, etc.)

Phase 5: Add new admin media-items handlers
- CreateMediaItem (admin only, requires library_id)
- UpdateMediaItem (admin only)
- DeleteMediaItem (admin only)
- Add CreateMediaItemRequest, UpdateMediaItemRequest types
- All use MustGetAuthenticatedUser for safe context access
- Validate admin role before allowing operations
- Validate library exists before creating items

Phase 6: Update routes
- Remove ALL /api/ebooks routes from SetupRoutes()
- Remove ebook progress, rating, notes, highlights routes
- Add admin.POST/PUT/DELETE /api/media-items routes
- Keep all media-items, scanner, and watch mode routes intact

Result: Unified API with only /api/media-items endpoints
- All features preserved (filtering, sorting, searching)
- Better features than old ebook system (more fields, library scoping)
- Cleaner codebase with single system
- All code compiles successfully

Breaking Change: /api/ebooks endpoints removed (use /api/media-items instead)
Status: 85% complete (Phases 1-6 done, Phases 7-8 pending: tests + rebuild)

Tests: Need update (rename Ebooks → MediaItems, update API paths)
Build: Need rebuild with clean cache
2026-01-30 10:03:13 -05:00
john-okeefe 3b0b18770e chore(db): regenerate database code after schema changes
- Regenerate queries.sql.go with refresh token queries
- Update models.go with RefreshTokens type
- Update querier.go with new query methods
- Update db.go with generated code
2026-01-29 09:23:34 -05:00
john-okeefe 29669e2fa3 feat: add database models and queries for annotations
- Add MediaNotes and MediaHighlights model structs with pgx v5 types
- Add EbookNotes and EbookHighlights for backward compatibility
- Add complete CRUD SQL queries for notes and highlights
- Add database connection pool function using pgx v5
- Generate sqlc code for new annotation functionality
2026-01-28 15:42:34 -05:00
john-okeefe dbc3590cad feat: implement library system database schema
- Add library_types table with ebooks, comics, manga types
- Add libraries table for multiple library support
- Add library_folders table for multi-folder libraries
- Add library_visibility table for user access control
- Add media_items table replacing ebooks for broader media support
- Create backward compatibility views for existing API
- Implement library service with type validation and file extension handling
- Support modular extension for future media types

Manga type includes cbz/cbr archives as requested
2026-01-28 11:00:06 -05:00
john-okeefe 0b126202c8 feat: Add role-based access control to database schema
- Add role column to users table with admin/user constraint
- Add added_by_admin_id column to ebooks table for tracking
- Add constraint to ensure only admins can manage folders
- Update all SQL queries to include role field
- Regenerate database models with new schema
2026-01-26 16:54:57 -05:00
john-okeefe f4e8c0d983 Add user names support: add first_name/last_name to users table and regenerate database queries 2026-01-23 21:27:08 -05:00
john-okeefe 11621462f2 feat: create admin dashboard with user preferences and library settings
- Add admin dashboard at /admin route consolidating all user settings
- Move user preferences (username, email, password, theme) from separate page
- Add ebook library preferences section with folder management
- Add scan settings (frequency and auto-scan toggle) with database persistence
- Create database migration for scan_frequency_minutes and auto_scan_enabled columns
- Add API endpoints for scan settings management:
  - PUT /api/library/scan-settings - Update scan preferences
  - GET /api/library/scan-settings - Get current scan settings
- Update dashboard navigation to link to admin dashboard
- Remove old preferences.html template (functionality moved to admin)
- Create Bruno API testing files for library endpoints
- Add real-time folder loading and management in admin interface
- Implement scan settings persistence and retrieval from database
2026-01-23 09:38:40 -05:00
john-okeefe 4318f8624b refactor: restructure project from bookmann to shelf
- Rename project from 'bookmann' to 'shelf'
- Move all backend/ contents to root level (flatten structure)
- Update Go module name from 'bookmann' to 'shelf'
- Update all import paths to use new 'shelf' module
- Update Dockerfile to work without backend/ subdirectory
- Update docker-compose.yml to use new structure and rename containers
- Update .gitignore for new file paths
- Update README.md with new project name and structure
- Regenerate database code with new module imports
2026-01-23 09:08:04 -05:00