Commit Graph
57 Commits
Author SHA1 Message Date
john-okeefe b1eda696f1 Revert "fix(bookmarks): upsert on title conflict so position upgrades don't 500"
This reverts commit 27b3dcb69f.
2026-08-30 21:01:59 -04:00
john-okeefe 27b3dcb69f fix(bookmarks): upsert on title conflict so position upgrades don't 500
Bookmark dedup is keyed on hash(title + position bucket), but the table
also enforces UNIQUE(media_item_id, user_id, title). When a client re-
saves the same bookmark title with a changed position form - e.g. the
Android app upgrading a percentage-only row to an EPUB CFI, or a web and
app bookmark landing on the same 'Bookmark at 44%' title - the dedup-key
lookup misses and the INSERT violates the title constraint, returning
HTTP 500 and failing the sync.

A title collision on the same (user, item) is by definition the same
bookmark slot, so take the LWW semantics all the way: ON CONFLICT DO
UPDATE replaces position/cfi_position/page/chapter/percentage, refreshes
dedup_key and timestamps, merges device_sync_data, and - matching
UpdateMediaBookmarkForSync - clears deleted/deleted_at so a re-create
resurrects a tombstoned title slot instead of leaving an invisible row
holding it.

Device sync flows are unaffected: KOReader/Kobo pushes that carry their
own dedup-key echoes never reach the INSERT, and same-key saves still go
through applyBookmarkLWW with its tombstone freshness checks.
2026-08-30 20:57:15 -04:00
john-okeefe acbb6c7981 feat(db): queries for the deleted-annotation history
ListDeletedAnnotationsForBook unions tombstoned highlights, notes, and
bookmarks for a user+book regardless of the sync TTL cutoff (the history
must show everything still restorable, not just recent deletes), with
display text, secondary text, color, and both timestamps.

Restore queries clear deleted/deleted_at (lossless — the row was soft-
deleted, never removed) and are scoped to the owning user and media item
so a restore can never touch another user's annotation.

Purge queries hard-delete an already-tombstoned row: the user-driven
counterpart of the TTL maintenance sweep, for explicit 'delete
permanently' actions from the history.

All six write queries are :execrows so callers can distinguish 'restored'
from 'nothing matched' without a follow-up read.
2026-08-22 13:16:31 -04:00
john-okeefe ba95cc3e8b fix(reader): stabilize chrome panels, bookmarks end-to-end, dead UI removal
Phase 0 of the reader redesign:

- Panels no longer render under the top/bottom bars: sidebars get
  measured insets (same resize/safe-area mechanism as the viewport);
  panel max-height now derives from the bounded sidebar instead of a
  100vh guess; right-side border targets the actual sidebar.
- Bookmarks work end-to-end for the first time: REST CRUD under
  /api/media-items/:id/bookmarks (create/delete route through
  AnnotationService for dedup/LWW/tombstones), fix UpdateMediaBookmark
  referencing nonexistent updated_at column, frontend posts to the
  real API with per-format position (CFI vs page), live list with
  jump + delete instead of SSR-only snapshot.
- Fix chapter matching in progress saves: boundaries were compared by
  a nonexistent tocItem property, so chapter was never persisted.
- Remove dead UI: Navigator panel stub, empty dictionary popup shell,
  unwired Chrome Behavior select; purge 160 stale build artifacts.
- Reader chrome now follows the user's app theme instead of hardcoded
  theme-tokyo-night.
2026-08-14 09:05:33 -04:00
john-okeefe 0c39e04e4a feat(db): hash_conflicts table and backfill/conflict queries
Content duplicates (same library + file_sha256 at different paths,
e.g. the same book imported twice under two names on a preexisting
database) cannot be auto-collapsed the way path duplicates were:
keeping both copies may be intentional. Surface them for an explicit
admin decision instead.

Schema:
- new hash_conflicts table keyed (library_id, file_sha256) with a
  status/resolution lifecycle: 'pending' until an admin resolves via
  'keep_all' or 'kept:<uuid>' (which copy was kept after merging)
- resolution is VARCHAR(50) - 'kept:<uuid>' is 41 chars; include a
  widening ALTER for databases created with the initial 30-char width
- resolution/resolved_by/resolved_at record who decided what and when

Queries:
- ListMediaItemsMissingHash: items imported before hashing existed
  (file_sha256 IS NULL), ordered oldest-first for the backfill pass
- FindHashConflictGroups: the content-duplicate group detection
  (GROUP BY library_id, file_sha256 HAVING COUNT(*) > 1)
- ListMediaItemsBySHA256AndLibrary: full membership of one group
- CreateHashConflict: upsert with DO NOTHING so already-tracked groups
  are untouched - critical behavior: a group an admin resolved as
  'keep both' is never re-flagged by later sweeps
- ListPendingHashConflicts: admin listing with library name and live
  item counts (items may have been deleted since flagging)
- GetHashConflict / ResolveHashConflict: lifecycle
- GetMediaItemUsageCounts: per-item progress/highlight/bookmark/note/
  collection counts so the admin can make an informed keep choice
- ReparentMediaItemChildren: sqlc binding for the existing
  reparent_media_item_children() migration function, used to merge a
  losing copy's child rows into the kept copy
2026-08-14 08:52:05 -04:00
john-okeefe 9b171a0060 fix(scanner): prevent duplicate media item imports
A read-then-write race in processMediaFile allowed the same file to be
imported twice: two concurrent scan jobs (startup scan, fsnotify dirty-
directory scan, periodic backup poll, or a manual scan each run on
separate worker goroutines with separate MediaScanner instances) could
both SELECT 'not found' and both INSERT. There was no transaction, no
row lock, no unique constraint on (library_id, file_path), and no
ON CONFLICT clause, so nothing stopped the double insert. Observed in
production as two identical 'Head First SQL' rows created in the same
second (same sha256, size, path, library).

Database enforcement:
- schema.sql: add UNIQUE(library_id, file_path) constraint, guarded so
  re-runs don't error
- schema.sql: add self-healing migration that runs on every startup -
  dedup_media_items_by_path() collapses existing path-duplicates and
  reparent_media_item_children() moves all child rows (progress,
  highlights, bookmarks, notes, collections, formats, aliases, kobo
  entitlements, etc.) onto a survivor before deleting losers, so the
  constraint applies cleanly on already-duplicated servers without
  losing reading history. Survivor picks the row with the most user
  data, ties broken by lowest id
- CreateMediaItem: upsert via ON CONFLICT (library_id, file_path) DO
  UPDATE so concurrent inserts collapse to one row and return it
- CreateMediaItemFormat: upsert via ON CONFLICT (media_item_id,
  format_type), closing the same race on format rows

Application-level guards:
- media_scanner processMediaFile: after computing the file hash, check
  GetMediaItemBySHA256AndLibrary (new query) and treat the file as
  existing when identical content is already in the library under a
  different path (content dedup, library-scoped so multi-library
  setups still work)

Ops tooling:
- scripts/dedup_media_items.sql: standalone idempotent maintenance
  script with a dry-run report (path + content duplicate groups, child
  row counts) and transactional cleanup, for servers that prefer to
  dedup manually before upgrading

Verified against the live database: the duplicate pair was collapsed
(reading_progress preserved on the survivor), schema.sql re-runs are a
no-op, and the constraint is in place with 62 unique books remaining.
2026-08-14 08:18:36 -04:00
john-okeefe bc47450653 feat(db): typed tunable system settings + SettingsRegistry
Add a typed, cached registry over the system_settings table so that
values which used to be hardcoded Go literals can be changed at runtime.

Schema (database/schema/schema.sql):
- Extend system_settings with setting_type, min_value, max_value,
  requires_restart, and category columns (all ADD COLUMN IF NOT EXISTS,
  nullable for backward compat with the original three rows).
- Seed rows for every tunable: session duration, password rules,
  login lockout, auth/device rate limits, OPDS page size, tombstone TTL,
  conversion cache TTL, sync queue interval/batch, and worker pool
  size/cap. Seed values equal the previous hardcoded literals, so
  behavior is unchanged on upgrade. ON CONFLICT DO NOTHING preserves
  any admin-modified values.

Queries (queries.sql):
- Add UpsertSystemSetting (RETURNING *) so new keys without a seed row
  can still be written through the API.
- Add GetSystemSettingFull + GetAllSystemSettingsFull returning the
  full typed row.
- Refactor CleanupExpiredRefreshTokens to take the retention window as
  a parameter (make_interval(secs => $1)) instead of the INTERVAL '7
  days' literal, so it can follow a configurable session duration.

Registry (internal/database/settings_registry.go):
- SettingsRegistry holds an in-memory cache of all known settings,
  populated by Load at startup and refreshed by Reload on writes.
- Typed domain getters (SessionDuration, PasswordRules, DeviceRateLimits,
  TombstoneTTL, OpdsPageSize, ConversionCacheTTL, SyncQueueConfig,
  WorkerPoolConfig, LoginLockout, AuthRateLimit, ...) with compiled-in
  fallback defaults and min/max clamping, so a corrupt or missing row
  can never break the app.
- SettingDefaults is the single source of truth for keys, types, bounds,
  and human descriptions; All() exposes metadata + current values for
  the admin UI/API.

The registry lives in the database package (rather than its own
internal/settings package) because a quirk in this custom go1.26.5
toolchain prevented the large handlers package from importing any
newly-created package; every consumer already imports database.

Tests: settings_registry_test.go covers default validity per type,
int clamping at both bounds, garbage-value fallback, and unknown-key
lookup.
2026-08-10 08:00:52 -04:00
john-okeefe 114a4574b0 feat(db): add query to count media items per visible library
Add GetVisibleLibraryMediaCounts, which returns the media item count for
each library visible to a given user in a single GROUP BY query over
media_items. It mirrors the visibility logic in GetUserVisibleLibraries
(libraries default to visible unless an explicit false row exists) so
counts can be resolved in one round-trip instead of N per-library
lookups.

Regenerated sqlc bindings (querier.go, queries.sql.go).
2026-07-30 11:41:05 -04:00
john-okeefe 2a15effc3e feat(db): add annotation sync schema, queries, and tombstone support
Add migration columns to media_highlights, media_notes, and media_bookmarks
for cross-device annotation sync:

- dedup_key: SHA-1 of normalized selection text + bucketed position, used
  as the stable cross-device identity for annotations
- last_modified_at / last_modified_source: edit clock for LWW resolution
  and cross-source conflict detection
- deleted / deleted_at: sticky tombstone columns for delete-wins semantics
  with a 30-day TTL before rows are physically purged
- note_text on highlights: stores attached notes from KOReader entries that
  have both selected text and a user note
- Location columns on bookmarks (cfi_position, percentage_location,
  epubcfi_location, chapter_reference, paragraph_reference)
- device_sync_data JSONB on all three tables: stores per-device native
  identifiers (e.g. KOReader pos0/datetime, Kobo bookmark_id) so each
  device can locate and manipulate its own copy of an annotation

Add partial unique indexes on (user_id, media_item_id, dedup_key) where
deleted = FALSE to enforce one active annotation per dedup key.

Add tombstone purge indexes on (deleted, deleted_at) for efficient GC.

New queries:
- GetByDedupKey for all three tables (returns active or most-recent tombstone)
- CreateFull / UpdateForSync for all three tables (populate sync columns)
- TombstoneByDedupKey / TombstoneByID for all three tables
- PurgeExpired* for all three tables (GC past TTL)
- GetActiveAnnotationsForBook (filtered union of highlights + notes)
- GetTombstonedAnnotationsForBook (union of all 3 deleted within TTL)
- GetMediaBookmarks with deleted filter
- GetMediaBookmark (singular) with deleted filter
- Added deleted=FALSE filter to GetMediaHighlights, GetAnnotationsForBook
- CreateAutoResolvedSyncConflict (INSERT with resolution_status='auto_resolved')
2026-07-29 14:48:30 -04:00
john-okeefe 980aaee0d9 refactor(setup): derive setup-complete status from admin user count
Setup completion was previously tracked by a manually-flipped setup_complete row in system_settings, written via a JWT-protected PUT /api/setup/complete endpoint. This meant any admin user created outside the setup wizard (future CLI, seed scripts, direct DB inserts) would not flip the switch, leaving the app stuck redirecting to /setup.

The trigger is now derived from real data: setup is complete iff at least one admin user exists. This is self-correcting regardless of how users are created, and re-engages setup automatically if all admins are ever removed.

Changes:
- Add internal/setupstatus package with IsSetupComplete() (queries CountAdmins, 10s in-memory cache, fails open on DB error) and Invalidate() to clear the cache. Uses an AdminCounter interface to avoid importing the database package.
- Add CountAdmins sqlc query (SELECT COUNT(*) FROM users WHERE role = 'admin') and regenerate.
- Rewire router/setup.go isSetupComplete() to delegate to setupstatus; drop the old setup_complete setting read, cache vars, and the PUT /api/setup/complete route.
- Call setupstatus.Invalidate() in the auth handler after CreateUser, UpdateUserRole, and DeleteUser so the cache reflects admin-count changes immediately.
- Align first-user promotion in Register to key off !adminExists instead of len(users) == 0, so the two checks cannot diverge.
- Remove the now-dead SetSetupComplete/GetSetupStatus handlers.
- Drop the setup_complete seed row from schema.sql.
- Remove the apiPut('/setup/complete') call from the setup wizard finishSetup(); the admin account created in submitAdmin already marks setup complete server-side.
2026-07-29 11:08:18 -04:00
john-okeefe c82f20c3f2 feat(db): add context_text column to reading_progress
Stores surrounding text (~100 chars) at the reader's current position.
Used as fallback for CFI resolution when converting between epubcfi
and CREngine XPointer formats.

Updates:
- schema.sql: add context_text TEXT column, update stored procedure
- queries.sql: add context_text to GetUniversalProgress and
  UpdateUniversalProgress queries
- Regenerate sqlc Go code (models.go, querier.go, queries.sql.go)
2026-06-02 19:44:31 -04:00
john-okeefe 0af940c7b3 chore(database): regenerate with sqlc v1.31.1
Regenerated database code after sqlc version upgrade from v1.30.0 to
v1.31.1. No functional changes — only the version header in generated
files was updated.

Files: db.go, models.go, querier.go, queries.sql.go
2026-05-24 20:23:08 -04:00
john-okeefe e7a4f0f758 refactor(sql): use sqlc.narg() pattern for optional library_id in all library-filtered queries
Convert 12 SQL queries to use sqlc.narg('library_id') instead of
direct @library_id parameters. This allows passing a NULL/invalid
pgtype.UUID to mean "no library filter" (i.e., All Libraries),
making the SQL layer correctly handle the optional filter via:
  (sqlc.narg('library_id')::uuid IS NULL
   OR mi.library_id = sqlc.narg('library_id')::uuid)

Also remove the library_id filter from GetSeriesBooks entirely —
a series is a series regardless of library.

Queries affected:
- GetDashboardSections, GetRecentlyAdded, GetInProgress
- GetHighestRated, GetMostRead, GetAbandonedBooks
- GetLeastRead, GetBooksByTag, GetCollectionItemsForDashboard
- SearchMediaItemsUnified, GetSeriesCardsData

Generated code (queries.sql.go, querier.go) regenerated via sqlc.
2026-05-18 17:52:13 -04:00
john-okeefe d0460885ff feat(db): add imported_at column to media_items for accurate "Recently Added" sorting
The created_at column stores file modification time (intentional for preserving
original metadata), but this makes 'Recently Added' sorting unreliable for
imported files. Add imported_at column that records the actual database insert
timestamp.

Changes:
- Add imported_at TIMESTAMPTZ column to media_items (nullable)
- Update GetRecentlyAddedItems to sort by imported_at DESC NULLS LAST first
- Add ReassignLibraries and ReassignMediaItems queries for admin deletion handover
- Add SyncLibraryTypeExtensions query for startup extension sync
- Update all media_items SELECT queries to include imported_at column
2026-05-16 19:30:27 -04:00
john-okeefe fa38ca0d1b feat(db): add GetBooksByTag query for tag detail page
Uses $2 = ANY(tags) to match against the tags text[] column with GIN
index support. sqlc generates a single string Column2 param (not []string).
2026-05-10 16:11:46 -04:00
john-okeefe d48404e800 feat(series): add SQL queries for series browsing and continue-series
Add five new sqlc queries to support the series browse page and
continue-series dashboard collection:

- GetDistinctSeries: list unique series with book counts, sorted by
  most recent entry, with pagination
- GetDistinctSeriesCount: total distinct series count for pagination
- GetSeriesCovers: fetch up to N cover image paths for a series,
  ordered by series_number
- GetSeriesBooks: fetch all books in a series ordered by series_number
- GetContinueSeriesItems: CTE-based query using DISTINCT ON to find
  the next unread book per series for a given user/library, sorted
  by most recent last_read_at
2026-05-08 20:26:39 -04:00
john-okeefe caf50ade31 Add timezone support to database schema and queries
- Add timezone column (VARCHAR(50) DEFAULT 'UTC') to users table
- Add default_timezone row to system_settings seed data
- Add idx_users_timezone index for user timezone lookups
- Add UpdateUserTimezone and GetSystemTimezone queries
- Regenerate sqlc code (models, querier, queries.sql.go)
- Reuse existing UpdateSystemSetting for system timezone updates
  instead of creating a redundant UpdateSystemTimezone query
2026-04-27 21:30:37 -04:00
john-okeefe ad27902790 fix(conflicts): use ListConflictsByUser in DismissAllResolved so resolved conflicts are found
DismissAllResolved was calling ListSyncConflictsByUser which filters to
'unresolved' conflicts only, so it could never find the user_resolved or
bulk_resolved conflicts it was trying to delete. The query always returned
an empty set, making dismiss-all a no-op.

Fix the leading space in three SQL query name annotations (ListConflictsByUser,
ListAllConflictsByUserAndStatus, CheckForProgressConflicts) that prevented
sqlc from generating their Go functions. Regenerate the query code and swap
DismissAllResolved to use ListConflictsByUser (no status filter) — the
existing Go loop already filters by resolution_status.
2026-04-21 21:15:34 -04:00
john-okeefe 059955be72 chore(db): Regenerate database code from processing issues queries
- Add ProcessingIssues model struct
- Update Querier interface with processing issues methods
- Add generated query implementations for CreateProcessingIssue, ListProcessingIssuesByLibrary, GetProcessingIssueStats, ResolveProcessingIssue, DeleteProcessingIssue
- Add GetLibraryWithType query for fetching library with type information
2026-04-12 20:43:46 -04:00
john-okeefe cf8c690474 Add library_type_name column to media_items with auto-population trigger
- Add library_type_name VARCHAR(50) column to media_items table in schema.sql
- Create PostgreSQL trigger 'set_library_type_name_on_insert' that automatically
  populates library_type_name by joining libraries table with library_types on insert
- Add library_type_name parameter to CreateMediaItem SQL query
- Update Go models (models.go) to include LibraryTypeName field
- Add UpdateMediaItemChapterMetadata query method to querier.go
- Regenerate queries.sql.go with sqlc

This allows media items to store their library type (e.g., 'Books', 'Comics', 'Manga')
at the database level, enabling filtering and display without needing additional joins.
2026-04-04 22:49:07 -04:00
john-okeefe 0667cad8a9 feat: add reader infrastructure - Phase 0 database schema and queries
Implement Phase 0 prerequisites for reader functionality including
database schema, SQL queries, and frontend dependencies.

## Database Schema (5 New Tables + 1 Column Addition)

### New Tables Added:
1. **panel_data** - Comic/manga panel detection results
   - Stores detected panel boundaries (x, y, width, height)
   - Supports grid, ML, and manual detection methods
   - JSONB storage for flexible panel structures

2. **reading_speed** - User reading speed statistics
   - Tracks pages per minute and total reading time
   - Per-user per-media-item tracking
   - Enables progress estimation and analytics

3. **dictionary_cache** - Offline dictionary word definitions
   - Caches external dictionary lookups
   - Reduces API calls and improves performance
   - Supports offline reading functionality

4. **reader_settings** - User reader preferences (per-user)
   - Stores typography, theme, and display settings
   - JSONB storage for flexible configuration
   - Per-user customization (fonts, margins, themes)

5. **media_bookmarks** - Enhanced bookmarks with chapter/CFI support
   - Unified bookmarking for ebooks, comics, manga, PDFs
   - Supports page_number, chapter_number, and epubcfi_position
   - Includes notes field for annotations
   - Unique constraint on (media_item_id, user_id, title)

### Column Addition:
- **media_items.chapter_metadata** (JSONB) - Caches detected chapter structure
  - Stores TOC/chapter detection results
  - Prevents re-parsing files on every read
  - Populated by ReaderService.DetectChapters()

## Database Queries (12 New Queries)

Added queries for all reader functionality:
- Panel data: GetPanelData, UpsertPanelData
- Reading speed: GetReadingSpeed, CreateReadingSpeed, UpdateReadingSpeed
- Dictionary: GetDictionaryEntry, CreateDictionaryEntry, UpdateDictionaryAccessed
- Settings: GetReaderSettings, UpsertReaderSettings
- Bookmarks: GetMediaBookmarks, CreateMediaBookmark, DeleteMediaBookmark, UpdateMediaBookmark

## Frontend Dependencies

Added to package.json:
- jszip@^3.10.1 - EPUB/comic archive parsing (client-side)
- pdfjs-dist@^3.11.174 - PDF rendering library (Mozilla PDF.js)

## Generated Code

Ran `sqlc generate` to regenerate:
- models.go - Go structs for new tables (55 lines added)
- querier.go - Database query methods (14 lines added)
- queries.sql.go - Compiled SQL queries (504 lines added)

## Implementation Status

Phase 0 prerequisites now complete:
 Database schema (5 tables + 1 column)
 SQL queries (12 queries)
 Frontend dependencies (2 packages)
 Generated Go code (sqlc)
 Database recreated with new schema

Ready for Phase 1: Infrastructure & Basic Reader implementation.

Related to: Universal web reader for ebooks, comics, manga, PDFs
2026-04-02 21:01:45 -04:00
john-okeefe 00840c2fe1 feat: add fuzzy tags_filter to search query
- Add tags_filter parameter to SearchMediaItemsUnified
- Add EXISTS clause with word_similarity() for fuzzy tag matching
- Add tag similarity scoring to ORDER BY clause (GREATEST function)
- Add SearchTagsValues query for autocomplete with ::TEXT cast
- Keep genre_filter for backward compatibility
- Regenerate Go code with sqlc generate

This enables filtering books by tags (from Calibre) instead of genre,
which is always NULL for imported books. Uses fuzzy matching consistent
with author/series filters, with best matches sorted first.

Relates to IMPLEMENTATION_TAGS_FILTER.md Phase 1
2026-03-25 20:38:08 -04:00
john-okeefe fe8a65af84 feat: enable cross-library search in unified search query
Updates SearchMediaItemsUnified query to support searching across all
libraries when library_id parameter is not provided. Changes SQL from
requiring library_id to checking for NULL:

  AND (sqlc.narg('library_id')::uuid IS NULL
      OR mi.library_id = sqlc.narg('library_id')::uuid)

The explicit ::uuid cast ensures PostgreSQL handles type inference
correctly when comparing UUID columns with nullable parameters.

Regenerates Go database code including queries.sql.go and querier.go
to reflect the updated SQL schema.

This enables the /api/media-items/search endpoint to search all libraries
by omitting the library_id query parameter, matching the behavior of
the OPDS search endpoint.
2026-03-24 16:47:23 -04:00
john-okeefe 43a6d843a3 feat: add unified search SQL queries with fuzzy filters
- Add SearchMediaItemsUnified query combining search + filters
- Add 4 field value search queries (author, genre, series, language) for autocomplete
- Support fuzzy text matching via pg_trgm (threshold: 0.3 similarity)
- Support exact match with quotes detection for search queries
- Add sort parameter support (title ASC/DESC, author ASC/DESC, created_at ASC/DESC, page_count ASC/DESC)
- Primary sort by relevance score when searching, secondary by user-specified sort
- Combine search query with all filter types in single optimized query
- Uses 4 separate simple queries instead of 1 complex query due to sqlc v1.30.0 limitation with CASE in GROUP BY

This consolidates the deprecated /filtered and /search endpoints into one unified endpoint.
2026-03-23 22:37:37 -04:00
john-okeefe 1ee96a502e gen: regenerate database code with new search queries
Run sqlc generate to create Go code for new search queries:

Added methods to Querier interface:
- SearchMediaItemsUnified - Main unified search with fuzzy/exact matching
- SearchAuthorValues - Author field autocomplete
- SearchGenreValues - Genre field autocomplete
- SearchSeriesValues - Series field autocomplete
- SearchLanguageValues - Language field autocomplete

Generated parameter structs and row types for all new queries.
All queries include proper library visibility checks.
2026-03-22 20:35:06 -04:00
john-okeefe 85964ec932 fix(api): enforce user isolation on saved filters delete operation
Fix critical security issue where admin users could delete other users'
saved filters due to incorrect error handling in DELETE query.

Database Schema Changes:
- Change DeleteSavedFilter from :exec to :one (queries.sql:1747-1750)
- Add RETURNING * to return deleted row for proper error detection
- Regenerate querier.go and queries.sql.go with updated signature

Service Layer (internal/services/filters.go):
- Update DeleteSavedFilter to capture returned row (using _ to discard)
- Properly propagate pgx.ErrNoRows when no rows are deleted
- Error wrapping preserves original error for handler detection

Handler Layer (internal/handlers/filters.go):
- Add errors.Is() check for pgx.ErrNoRows (line 148)
- Return 404 Not Found when filter doesn't exist or belongs to different user
- Return 500 Internal Server Error for other database errors
- Add "errors" import (line 8)

Security Fix Details:
Before: Admin could delete user's filter → 204 No Content (SUCCESS)
After:  Admin tries to delete user's filter → 404 Not Found (DENIED)

The DELETE query uses WHERE id = @id AND user_id = @user_id, which matches
0 rows when attempting to delete another user's filter. The old :exec query
didn't return row count, so 0 affected rows looked like success. The new :one
query with RETURNING * returns pgx.ErrNoRows when no rows match, allowing
the handler to return proper 404 error.

Test Impact:
- TestSavedFilters/User_cannot_access_another_user's_filter now passes
- All 6 integration tests pass with proper user isolation enforcement

Pattern Consistency:
- Matches DeleteLibraryFolder pattern (line 99 in queries.sql)
- Uses same error handling as media handlers (errors.Is + pgx.ErrNoRows)
- Follows user-scoping pattern used throughout codebase

Related: Saved filters implementation user isolation
Security: Prevents unauthorized deletion of user data
2026-03-21 01:24:15 -04:00
john-okeefe e17a96123f feat(db): add saved_filters table and CRUD operations
Add database schema and SQL queries for generic saved filters system
that allows users to save custom filter presets for any resource type.

Database Schema:
- Add saved_filters table with user_id, name, resource_type, filters (JSONB)
- Create composite index on (user_id, resource_type) for efficient lookups
- Create index on (user_id, name) for future name search feature
- Add update_updated_at_column() trigger to auto-update timestamps
- Make trigger creation idempotent with DROP TRIGGER IF EXISTS

SQL Queries (5 new queries):
- GetSavedFilters: List all filters for user + resource type
- GetSavedFilterByID: Retrieve single filter by ID
- CreateSavedFilter: Create new saved filter
- UpdateSavedFilter: Update filter name/criteria
- DeleteSavedFilter: Remove saved filter

Design Decisions:
- Generic resource_type field supports any resource (media-items, collections, devices)
- JSONB filters field allows flexible schema without migrations
- User-scoped via JWT (user_id foreign key with CASCADE delete)
- Automatic updated_at timestamp via database trigger

Generated Code:
- database.SavedFilters model (10 fields including JSONB filters)
- All 5 CRUD query functions with proper parameter types
- pgtype.UUID wrappers for UUID parameters

Part of: Saved Filters Implementation (Phase 1: Database)
Related: #saved-filters-feature
2026-03-21 00:16:05 -04:00
john-okeefe d802236874 scanner: fix library isolation, file mtime, force rescan, and deletion handling
Fix 1 - File modification time for created_at:
- Get file.ModTime() in processMediaFile and pass to CreateMediaItem
- Modified SQL INSERT to include created_at column

Fix 2 - Force rescan UPDATE instead of DELETE+INSERT:
- Changed force rescan logic to call updateMediaItem instead of delete + create
- Preserves created_at timestamp on force rescan

Fix 3 - GetMediaItemByFilePath filters by library_id:
- Added library_id to WHERE clause in SQL query
- Created GetMediaItemByFilePathAnyLibrary for cross-library lookups (KOReader)
- Added SetLibraryID method to MediaScanner
- Updated handler to call SetLibraryID for watch mode

Fix 4 - File deletion handling with persistent logging:
- Added fsnotify.Remove handler in WatchChanges
- Added orphan cleanup in ScanFolders after scan completes
- Created scanner_logger.go with daily log rotation (7 days)
- Logs to /app/logs/scanner-deletes-YYYY-MM-DD.log and scanner-errors-YYYY-MM-DD.log
- Individual deletes with enhanced safety logging

Note: Integration tests can now safely scan /app/uploads because
GetMediaItemByFilePath now filters by library_id, preventing
cross-library interference.
2026-02-26 16:39:42 -05:00
john-okeefe e3a3aa124f feat(api): consolidate user profile update endpoints
Add delete_user, reset_user_password, and update_user endpoints to replace
individual update operations. Update database schema to include deleted_at
column for soft deletion. Add DeleteUser, ResetUserPassword, and
UpdateUserAdmin queries. Update Querier with new methods for user management.
2026-02-22 01:57:42 -05:00
john-okeefe dd1e56d2f7 chore(dashboard): regenerate database code from Phase 3 queries
Run sqlc generate to create Go code for dashboard queries:
- GetDashboardPreferences / UpsertDashboardPreferences / UpdateDashboardPreferences
- GetSystemCollectionsForDashboard / GetUserCollectionsForDashboard
- DeleteUserSystemCollection
- GetContinueReadingItems / GetRecentlyAddedItems / GetRecentlyReadItems / GetNotStartedItems
- GetCollectionItemsForDashboard / GetLibraryItems

Auto-generated from queries.sql changes.
2026-02-19 20:55:59 -05:00
john-okeefe 2706ae52c1 refactor: remove Phase X terminology from source code comments
Remove planning document phase references from code comments:

app_test.go:
- Remove Phase 5 references from 8 test function comments

querier.go & queries.sql.go:
- Remove Phase 1, 2, 3, 4, 6 references from section headers
- Clean up week numbers (Weeks 5-6, Week 3-4, etc.)

queries.sql:
- Remove Phase 4 references from Kobo queries

kobo.go:
- Remove Phase 6 references from ContentId mapping comments

progress.go:
- Remove Phase 1 reference from route comment

media_scanner.go & media_scanner_library_type_test.go:
- Remove Phase 2 references from library type scanning comments

schema.sql:
- Remove Phase 1, 2, 3, 4, 5, 7 references from table/section comments
- Clean up: Format Detection, Progress Tracking, Device Registry,
  Sync Queue, Conflict Resolution, Reading History, Indexes, etc.

test_helpers.go:
- Remove Phase 6 reference from handler setup comment

These phase numbers were from internal planning documents and have no
meaning in the codebase. Removing them makes the code self-documenting.
2026-02-13 21:50:29 -05:00
john-okeefe 8321149957 test: add Bruno API test collections for device authentication
- Device token regeneration tests (success, forbidden, not found, unauthorized)
- OPDS authentication tests (Bearer token, query token)
- Kobo sync tests with token authentication
- Test various authentication methods and error cases
2026-02-13 12:12:17 -05:00
john-okeefe b506046be0 chore(db): regenerate database code from sqlc
Auto-generated changes from running 'sqlc generate' after query updates:
- models.go: updated with SystemSettings struct, removed scan fields from Users
- querier.go: updated interface with new system settings methods
- queries.sql.go: regenerated with new query methods

Generated via: cd internal/database && sqlc generate
2026-02-09 20:09:58 -05:00
john-okeefe 9e166c9670 chore(database): remove unused EbookNote backward compatibility functions
- Remove CreateEbookNote, UpdateEbookNote, DeleteEbookNote queries
- These were marked as backward compatibility but never used
- API uses CreateMediaNote, UpdateMediaNote, DeleteMediaNote instead
- Remove misleading backward compatibility comments
- Regenerate sqlc code
2026-02-08 14:28:08 -05:00
john-okeefe 5b9f21a592 Final cleanup: Update remaining comments and variable names
Changes:
- Update comments: "Bookmann UUID" → "Bookhoard UUID"
- Rename sidecar struct field: Bookmann → Bookhoard
- Update type names: SidecarBookmannConfig → SidecarBookhoardConfig
- Fix test database name in queue_test.go
- Fix uppercase env var examples in KOBO_SETUP.md

Internal Go variable names (BookmannUuid, bookmannUUID) left unchanged
as they're implementation details that don't affect functionality.

Part of project rename to Bookhoard.
2026-02-01 16:24:58 -05:00
john-okeefe d43e0105eb Rename project module and database references: Bookmann → Bookhoard
Core infrastructure changes:
- Update Go module name: bookmann → bookhoard
- Rename database schema references and comments
- Update database column names: bookmann_uuid → bookhoard_uuid
- Rename SQL query functions: GetDeviceCatalogByBookmannUUID → GetDeviceCatalogByBookhoardUUID
- Update configuration defaults

This is part 1 of the project rename to Bookhoard.
2026-02-01 16:11:47 -05:00
john-okeefe f23b6d35e9 feat(database): add analytics and book matching queries
Add analytics queries:
- GetUserReadingHistory: detailed reading history with device info
- GetUserDeviceUsage: device usage statistics (sync count, time spent)
- GetPopularBooks: most read books with completion rates

Add book matching queries:
- GetUnlinkedBookByID: fetch single unlinked book
- DeleteUnlinkedBook: remove resolved unlinked book
- ListUnresolvedUnlinkedBooks: paginated list of unresolved books
2026-02-01 12:15:33 -05:00
john-okeefe 63008001c6 feat(db): Add kobo_shelves and device catalog tables
- Add kobo_shelves table for Kobo device shelves management
- Add device_shelf_mappings for collection<->device shelf mappings
- Update device_catalogs with better ContentId tracking
- Add indexes for performance
2026-01-31 22:32:04 -05:00
john-okeefe 030d30a225 Add queue management API and database queries
- Add 7 new database queries for queue management
- GetStuckSyncQueueItems - Detect stuck items
- GetSyncQueueStats - Queue statistics
- GetNextRetryTime - Exponential backoff calc
- ListAllSyncQueueItems - Admin view
- IncrementSyncQueueAttempts - Retry counter
- Add queue handler with 7 REST endpoints
- GET /api/queue/devices/:id/stats - Queue statistics
- GET /api/queue/devices/:id/items - List device queue
- POST /api/queue/items/:id/retry - Retry failed item
- DELETE /api/queue/items/:id - Delete queue item
- DELETE /api/queue/devices/:id/clear - Clear device queue
- GET /api/queue/items - List all items (admin)
- Add full user/admin access control
2026-01-31 13:06:12 -05:00
john-okeefe db5d51e77e Regenerate database code for UUID token support
- Update models: RefreshTokens.Token now pgtype.UUID
- Update querier: GetRefreshToken/RevokeRefreshToken accept pgtype.UUID
- Regenerate queries.sql.go via sqlc after schema change
2026-01-31 11:44:26 -05:00
john-okeefe cd240d3054 feat: add Kobo shelf and entitlement SQL queries 2026-01-31 00:29:38 -05:00
john-okeefe 4c01c0d12e Phase 3 Week 7: Add KOReader bulk sync function to database schema
- Add bulk_update_progress_from_koreader() function for batch processing
- Handles progress, annotations, and conflict detection
- Returns success/failure status for each book
- Supports device matching by UUID, file path, or title/author
- Implements automatic conflict detection for concurrent syncs
- Part of Phase 3 KOReader Integration implementation
2026-01-30 20:54:51 -05:00
john-okeefe 23ad70158c Phase 2 Week 5: Device Registration & Management
Implement device registration and management system for universal sync.

Database Changes:
- Add device queries to queries.sql (CRUD operations, registration, auth)
- Add sync queue management queries
- Add conflict resolution queries
- Regenerate sqlc models with new device-related types

Device Handler (devices.go):
- InitiateRegistration: Start device registration with auth URL and QR code
- CheckRegistrationStatus: Poll for registration approval
- ListDevices: Get all devices for current user
- GetDevice: Get specific device details
- UpdateDevice: Update device settings (name, sync settings, frequency)
- DeleteDevice: Remove device from account
- ApproveDevice: User approves device registration via web
- RejectDevice: Reject pending device registration
- ListPendingRegistrations: Show all pending registrations
- generateDeviceToken: Generate secure Bearer token for devices

Device Authentication Middleware (device_auth.go):
- Authenticate: Validate device Bearer tokens
- RequirePermission: Check device permissions by type
- hasPermission: Define permissions per device type
- UpdateLastSeen: Auto-update device last_seen timestamp

Configuration:
- Add BaseURL field to Config for device setup URLs

API Endpoints:
POST /api/devices/register - Initiate device registration
POST /api/devices/register/status - Check registration status
GET /api/devices/approve/:id - Approve device (web UI)
POST /api/devices/reject/:id - Reject device
GET /api/devices - List user's devices
GET /api/devices/:id - Get device details
PUT /api/devices/:id - Update device settings
DELETE /api/devices/:id - Delete device
GET /api/devices/pending - List pending registrations

Bruno API Collection:
- Initiate Device Registration
- Check Registration Status
- List Devices
- Get Device
- Update Device
- Delete Device

Dependencies:
- github.com/skip2/go-qrcode for QR code generation

Device Types Supported:
- koreader: Calibre-compatible sync
- kobo: Kobo sync protocol
- web: Web interface
- mobile: Mobile apps

Device Permissions:
- sync:progress
- sync:annotations
- sync:metadata
- device:manage (web only)
2026-01-30 16:45:10 -05:00
john-okeefe bb3c32c59f Phase 1 Week 2: Format detection and progress conversion engine
- Add internal/sync package with format detection
- FormatGroup types: reflowable, fixed_layout, comic_archive
- DetectFormatGroup() function based on mimetype and file extension
- MimeType mappings for common ebook formats
- Progress conversion engine with:
  - ConvertProgress() between format groups
  - Extract percentage from various progress formats
  - PageToPercentage / PercentageToPage helpers
  - CharacterToPercentage / PercentageToCharacter helpers
  - MergeProgress() with 'max progress wins' strategy
  - FormatProgressForDisplay() for UI rendering
- Add sqlc queries for format detection and progress updates
- BulkUpdateFormatGroups query for auto-format detection
- GetUniversalProgress query with all location references
- UpdateUniversalProgress query with device sync metadata
- ReadingHistory queries for session tracking
2026-01-30 16:07:00 -05:00
john-okeefe fc71c2ef76 fix: database queries and schema sync
- Remove references to non-existent columns (language, edition, page_count, etc.)
- Fix CreateMediaItem and UpdateMediaItem queries
- Remove normalize_isbn() function calls (moved to Go code)
- Regenerate database code with sqlc generate
- Add GetLibraryByFolder method to queries

Fixes compiler error: s.db.GetLibraryByFolder undefined
2026-01-30 13:51:44 -05:00
john-okeefe 3b2075fc70 Phase 1: Add enhanced database fields and sorting
- Add 9 new fields to media_items table (language, edition, page_count, goodreads_id, openlibrary_id, google_books_id, copyright_year, genre, subjects)
- Add indexes for new fields (language, genre, page_count, copyright_year, series_order, date_published)
- Add ListMediaItemsSorted SQL query for dynamic sorting
- Update ListMediaItems handler to process sort parameter
- Support 16 sorting options (title, author, created_at, date_published, copyright_year, page_count, genre, series)
- Add /api/media-items/filtered endpoint for advanced filtering
- Register new filtered endpoint in routes
2026-01-30 08:32:49 -05:00
john-okeefe 3145f64a66 feat: add search queries with partial matching and fuzzy fallback
- Add SearchMediaItems query with ILIKE partial matching
- Add SearchMediaItemsFuzzy query with word_similarity()
- Use sqlc.narg() for named parameters (search_pattern, search_query)
- Rank results by relevance: title > author > series > tags
- Fuzzy threshold set to 0.3 for word_similarity
- Generated Go models with proper parameter types
2026-01-29 20:20:30 -05:00
john-okeefe 66f1eb11a0 feat(ebooks): add ISBN normalization and graceful library requirement handling
- Increase ISBN column from VARCHAR(13) to VARCHAR(17) to support ISBN-13 with hyphens
- Add normalize_isbn() database function to automatically remove hyphens and spaces
- Create trigger to auto-normalize ISBNs on INSERT/UPDATE operations
- Update all Ebook and MediaItem queries to use ISBN normalization
- Add GetEbookLibraryID query to check for existing ebook libraries
- Add graceful error handling when no ebook library exists
- Return helpful error message: 'no ebook library found. Please create an ebook library first'
- Create comprehensive tests for ISBN normalization and library selection
- Add Bruno test files for various ISBN formats and error scenarios
- Update documentation with ISBN normalization details
2026-01-29 10:52:14 -05:00
john-okeefe 3b0b18770e chore(db): regenerate database code after schema changes
- Regenerate queries.sql.go with refresh token queries
- Update models.go with RefreshTokens type
- Update querier.go with new query methods
- Update db.go with generated code
2026-01-29 09:23:34 -05:00
john-okeefe 8f11219e03 chore: cleanup temporary files and update gitignore
- Remove temporary DOCUMENTATION_UPDATES.md after merging content
- Remove obsolete internal/database/connection.go file
- Update .gitignore to exclude build artifacts
- Clean up generated files and temporary directories
2026-01-28 15:45:10 -05:00