bc91ce68d4bb098a389d9b3383d312034a985285
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 John O'Keefe
|
abb7c84833 |
feat(podcasts): smart episode collections — ALL/ANY match modes + episode rule fields (item 2.4)
The daily briefing needs AND: today's rules engine was OR-only. The
match semantics become a per-collection column
(collections.rule_match_mode, 'any' legacy default | 'all'), flipped on
its own endpoint (PUT /api/collections/:id/match-mode, owner-only —
nothing else on the row changes) and surfaced in the rules page as an
ANY/ALL toggle with matching copy.
Episode rule fields join the vocabulary (plan item 2.4): feed (the
multi-select rides the new `in` operator — comma-separated feed ids),
played (the canonical finished predicate — viewer progress ≥ 0.999,
same as hide_finished), downloaded, published_within (N days), plus
title/description contains. The rules candidate query now carries the
viewer's progress (LEFT JOIN reading_progress) so played/unplayed is
per-user, and TestRules/preview speak the SAME evaluator: the handler's
duplicated field switch is deleted and delegates to
CollectionService.FieldMatches — one vocabulary, one semantics.
Legitimate semantics note: in ANY mode `played:false` matches unplayed
BOOKS too (everything with no progress row) — correct per-rule
behavior, and exactly why the briefing mode exists.
Backward compatibility: every legacy row defaults to 'any' (the old
OR), UpdateCollection keeps the stored mode when the form doesn't send
one, and the evaluator treats absent data honestly (no date → no
match; no progress → unplayed).
Pinned by TestPodcastSmartCollections: the AND-mode briefing
(feed∈{A} ∧ unplayed ∧ <7d = exactly the new episode), OR-mode
widening, owner-only flips, the preview honoring the mode, and
downloaded/contains rules. Also fixes a latent bug the new query
surfaced: UpdateCollectionViewSettings referenced a nonexistent
updated_at column (never exercised until now — the new sibling query
was written without it).
|
||
|
John O'Keefe
|
8631aa00bf |
refactor: remove dead code across server, templ pages, and client modules
Deadcode-verified removals (RTA reachability from main, cross-checked with -test and textual xrefs): Go server: - whole files: middleware/transaction.go, middleware/error_handler.go, middleware/password_validator.go (live registration machinery kept), handlers/sync.go, sync/offline.go - unrouted endpoints: auth UpdateUsername/UpdateEmail, timezone settings handler, OPDS RegisterOPDS, Kobo LibrarySync handler, ReaderHandler.ShowReader, JSON-data twins on collections/library - duplicate book-matching handlers (book_matching.go Handler twins, matching.go MatchingHandler twins), superseded helpers (router.parseUUID, handlers.ValidateDeviceToken, normalizePath, userExists, generateLockID) Templ/client: - never-routed pages: api_explorer.templ, queue.templ and their client modules api-explorer.ts, api-explorer-docs.ts, queue.ts - fully orphaned wrapper modules events.ts, dom.ts - broken clearSyncQueue (called a nonexistent endpoint) and the hidden, never-populated sync-queue section on the devices page - 166+ unused exports demoted to private (knip-verified); ~85 truly dead declarations removed via tsc noUnusedLocals fixpoint - stale 'interactive API Explorer' claims dropped from docs The docs drift-gate package (internal/docs), templ-bound modules (dashboard, stats, header, index, login, profile*, bookshelf, collections, toast-error), sqlc-generated code, and web/static are intentionally untouched. |
||
|
John O'Keefe
|
f94e7472b4 |
feat(filters+ui): per-element author/publisher filters, playlist toggles, Up Next API, reorder UI, dashboard badges
Phases 4 + 5 of the metadata overhaul (plan §4, §5) plus §3 handler wiring. Filters (§4.1): author_filter/publisher_filter now match per-element against the authors[]/publishers[] arrays — a co-authored book matches any one of its authors — with comma-joined params acting as OR-within-facet multi-select (string_to_array + cross unnest). Legacy single-value params unchanged on the wire; series/genre/language filters stay on their varchar columns; narrator filter already per-element. Collections API (§3.3-3.5): collection detail carries viewer + hide_finished into the listing query and exposes up_next (visible- library-enforced reading-queue head); PUT /api/collections/:id/items/order (owner-only, full-order rewrite); UpdateCollectionRequest gains hide_finished/show_up_next ([] string last-value-wins for the hidden-false-sibling checkbox pattern) merged server-side OVER stored view_settings so imported_from_series provenance survives edits; EditCollectionModal renders the toggles from view_settings. Dashboard (§5.5): sections JSON carries is_playlist/complete (dashboard.ts is the AGENTS-documented JSON-render exception; first paint stays templ). Fully-read playlists render '✓ Playlist complete' instead of an empty rail; imported playlists get a 📖 Playlist badge. Frontend: book detail shows subtitle + full authors + Explicit badge; edit modal gains subtitle, comma-separated authors/publishers (client- split to arrays exactly like narrators), step=any series number, explicit checkbox; collection book cards get playlist move ↑/↓ buttons persisting the full order via the reorder endpoint (templ partial re-render on failure — DOM order is client-local, API order is truth). AGENTS.md gotcha #3 amended per plan §3.7: user collections are dashboard sections by default; the show_on_dashboard premise is retired. Verified: full affected integration sweep green (pins widened for up_next/is_playlist/complete — changelog entries in docs/contract_pins.md); unit suite green; go build + templ + vite green. |
||
|
John O'Keefe
|
1dbb0ce406 |
feat(collections): playlist auto-creation, position ordering, hide_finished, Up Next, dashboard eligibility
Phase 2b + 3 of the metadata overhaul (plan §2.4, §3.1-3.7). Auto-playlist collections (§2.4): the scanner turns obviously-playlist sidecar series (explicit '(Reading Order)'/chronological/publication/ in-order/timeline markers only — precision over recall) into curated static collections: upsert by view_settings.imported_from_series then name (rename-safe), position set ONLY on insert (clean integers; 1.5/1a stay unpositioned rather than guessing), membership add-if-missing, never removed or re-ordered by rescans, provenance in description + view_settings, show_on_dashboard true (dashboard-eligible by design), 'scanner.auto_playlist_collections' setting (default on, Settings Registry + schema seed). Ordering (§3.1-3.2): collection listings order position ASC NULLS LAST then added_at ASC (was added_at DESC — walked playlists backwards); PUT /api/collections/:id/items/order applies full-order rewrites (owner-only, service-gated). hide_finished + Up Next (§3.3-3.4): GetCollectionItemsForDashboard gains viewer_user_id + hide_finished (LEFT JOIN reading_progress, COALESCE percentage < 0.999) and viewer_percentage rides the row for card graying; GetCollectionUpNext serves the collection's reading-queue head with the family-lens visible-library array enforced. Dashboard (§3.5): user collections are dashboard-ELIGIBLE BY DEFAULT — the show_on_dashboard gate (never writable for user rows since the column shipped) is gone from GetUserCollectionsForDashboard; hiding stays dashboard-prefs (hidden_collections) territory; hide_finished filtered-everything now emits a Complete section instead of silently vanishing; sections JSON carries is_playlist/complete. Series page ordering hardening (§3.6): GetSeriesBooks tiebreaks by title ASC — NULL/tied numbers previously returned heap (scan) order, which is what the user saw. Frontend custom-section-builder note: the builder has sent show_on_dashboard: true since its creation; the API never bound it. User-row visibility is now intentionally prefs-governed, so the field stays unbound by design (documented in code). Pins widened (docs/contract_pins.md): collection detail + up_next; dashboard sections + is_playlist/complete; new reorder endpoint recorded. Verified: TestPlaylistCollectionsEndToEnd (new, full e2e: sidecar series split, collection auto-creation with position 7, rescan idempotency, owner-only reorder, finished-filter + completion marker + Up Next drain) plus the full affected integration sweep green. |
||
|
John O'Keefe
|
18727fef93 |
refactor(collections): rule evaluators take the real row; both 45-field converter twins deleted
checkRulesAgainstBook/evaluateRule and the service's EvaluateRules existed in two parallel universes: one over database.MediaItems (what every query actually returns) and one over ListMediaItemsRow (what two hand-written 45-field mediaItemsToListMediaItemsRow converters produced — verbatim twins in the handler and the dashboard service, with a URL-resolution side effect baked into a "converter"). The rule fields (genre/series/author/language/publisher/copyright_year/tags) all exist on MediaItems, so the evaluators now take the real row and the converters, their three call-site loops, and the dead ApplyAutoAssignRules entry point (zero callers) are gone. The rules surface is wire-covered by the collections rules scenarios; handlers+services suites and the full pin/H1/H3 battery green. |
||
|
John O'Keefe
|
6457fbaf72 |
feat(stats): reading statistics — session capture wired + /stats page + widget + per-book card
The design parked in the handoff (multi-tier stats off reading_history; bookorbit-style in data, not looks) is in. The real prerequisite work was data: reading_history existed but NOTHING wrote to it — the whole stats feature was sitting on an empty table. - CAPTURE (no client changes, no repo-boundary crossings): every successful progress write through SaveProgress (KOReader + app + web + Kobo + offline queue — the one chokepoint) now records one reading_history session row: span = the interval since the PREVIOUS write instant, pages = positive page delta when both sides know the page index, marker rows for first contact. Spans over 2h are kept as zero-second markers (overnight suspend→resume, offline flushes must not inflate totals) — the honest 'sync cadence, not a stopwatch' caveat stays. device_id only when a REAL device row matches (the web/app progress path historically stuffs the USER id into DeviceID — probe instead of trusting it; FK violation would silently drop every capture). Source preserved in device_metadata. - API: /api/stats/summary?range=7|30|90|year (totals, zero-filled daily arrays, streaks, most-active day/weekday, top books, library breakdown — one call), /api/stats/this-week (widget: rolling 7d vs previous + streak), /api/stats/books/:id (lifetime + recent sessions). All bucketed in the user's timezone; finished ≡ any push ≥0.995; visible-library lens enforced at source. Legacy /api/analytics/* kept untouched. Range windows anchor end-of-day to the START of today (blasting the clock time into tomorrow resolved 'today' a calendar day late — found by the streak test). - /stats page (sidebar Stats between Progress and Devices, previously /analytics was an orphan): range selector, five stat cards, two Chart.js v4 charts (daily minutes bar, pages/day line — chunked via dynamic import so main.js stays untouched), per-library breakdown, most-read list, empty-state explainer. /analytics now 301s to /stats. The orphaned analytics.templ + analytics.ts removed (their containers and the page JSON never matched; the page was astub). - Dashboard 'This Week' widget (server-rendered, suppressed on service failure) and the book detail 'Your Reading Stats' card (lifetime totals of the viewer's own sessions) complete the tiers: per-user default, per-library breakdown inline, per-book on the page, per-system/admin aggregate parked as before. Tests: statistics_test.go (capture semantics at the chokepoint, tz/day buckets, zero-fill, streak math, finished detection, range validation, auth gate, widget, per-book lifetime). Full suite PASS. |
||
|
John O'Keefe
|
0a7fbd6f48 |
feat(collections): smart (dynamic) collections — rules evaluated live at render
Type is DERIVED, not stored: a saved ruleset makes a collection SMART,
no rules = STATIC (stored collection_items links, unchanged behavior).
No schema change — collections.auto_assign_rules JSONB already carried
the config; user's decision 2026-10-14 (handoff: live-at-render, not
refresh-on-scan; create-time Static/Smart toggle; v1 NO hybrid).
- Rule gains an enabled flag (absent = enabled, legacy rules unaffected).
- Rule CRUD: POST/PUT(DELETE) /api/collections/:id/rules[/:ruleId] — the
survivor dead effort of the rules-builder UI finally has real
endpoints (rules were never editable after creation; nothing ever
re-evaluated them). First rule converts a user collection to smart;
removing the last reverts it to static. UpdateCollectionRulesJSON
edits ONLY the rules doc so rule CRUD cannot clobber metadata.
- Live evaluation: candidates from GetVisibleLiveMediaForRules —
visible-library set (family lens enforced at source) + archived IS
NULL AND missing_scan_count = 0 — any enabled rule matches, title
order. Zero refresh hooks by design: always current by construction.
- Membership mutations (add/remove/bulk-add) are 400 on smart — v1 no
hybrid; collection_items.excluded stays the reserved future hook.
- Rules builder page (/collections/:id/rules) first time ever routable;
templ whole-attribute interpolation ("{ expr }" inside a quoted attr
value is literal text, as theme-{ user.Theme } silently proved).
- UI: create modal Static/⚡Smart cards (smart create redirects into the
rules builder), ⚡ badge + real per-card counts on the grid (smart =
live matches, static = stored-link count), smart detail renders
matched books with no hand-pick actions + an Edit Rules button.
- API: GET /:id gains is_smart + live books; GET /: keeps the Android
app's default exclusion, include_auto=true surfaces smart rows with
live counts; TestRules previews against the SAME candidates the live
render uses (visible + live, was raw first-1000).
- FIXED pre-existing regression from
|
||
|
John O'Keefe
|
7ce73213ae |
feat(server): system collections become permanent — soft-delete + in-place restore + query_type-keyed provisioning
The old model hard-deleted system collections and re-created them at the tail, resurrecting stale dedups and breaking ordering. Redesign (user-directed B-final): - schema.sql: collections.deleted_at TIMESTAMPTZ (idempotent ALTER — the whole schema is re-applied on every boot via executeSchema). - queries.sql: every collection list/read query filters deleted_at IS NULL; new GetSystemCollectionByQueryType :one (unfiltered, sees tombstones), TombstoneSystemCollection :exec, ClearCollectionItems :exec, RestoreSystemCollectionRow :one (in-place metadata reset + un-tombstone; collections have no updated_at column — ResetUpdatedAt was a 500 caught by tests). sqlc regenerated. - collection_service.DeleteCollection: system collections are tombstoned (preserving position/ownership); user collections keep the hard delete. No ErrNoRows to the client (delete is idempotent). - dashboard_service.RestoreSystemCollection: resolve by query_type; found → in-place restore (ClearCollectionItems + metadata reset, un-tombstoned); missing → provision fresh. "Want to Read"/etc. stay at their home positions forever. - handlers/collections.CreateCollection: name-taken is 409, not 500. - handlers/auth.CreateDefaultCollectionsForUser: idempotent by (user_id, query_type) — update-in-place when a row for that query type exists (heals renames/tombstones), insert when missing; this also fixes the duplicate-system-collections regression at source. |
||
 john-okeefe
|
e1aef8e85f |
refactor(services): Modernize Go code style in collection and filters services
Apply Go 1.18+ language features and modern style:
internal/services/collection_service.go:
- Use map[string]any instead of map[string]interface{} (Go 1.18+)
- Use range clause with single variable for iteration-only loops
- Replace if-else chains with switch statements for better readability
- Remove explicit type initialization for zero values
internal/services/filters.go:
- Add Err prefix to custom error variable for error naming convention
internal/router/library.go:
- Use cfg.ProcessingIssuesHandler instead of local processingIssuesHandler variable
- Ensures proper dependency injection through router config
These changes follow current Go best practices and improve code readability.
|
||
|
john-okeefe
|
516cec5a7f |
feat: migrate tags and contributors from TEXT to TEXT[] arrays
Convert tags and contributors columns from comma-separated strings to PostgreSQL TEXT[] arrays for better data normalization and query performance. Database Changes: - schema.sql: Change tags/contributors from TEXT to TEXT[] - schema.sql: Add GIN indexes for fast array searches - queries.sql: Update search queries to use ANY() operator - queries.sql: Update fuzzy search with unnest() for arrays Generated Code (sqlc): - models.go: Auto-generated with []string types for tags/contributors - queries.sql.go: Auto-generated with proper array handling Handler Changes: - media.go: Update request structs to use []string for tags/contributors - media.go: Remove pgtype.Text wrapping, use direct array assignment - media.go: Add tag normalization in CreateMediaItemHandler - collections.go: Update tags evaluation to join arrays for comparison - collections.go: Add strings import for Join() function Service Changes: - ebook_scanner.go: Update EbookMetadata struct to use []string - ebook_scanner.go: Remove string Join(), assign arrays directly - collection_service.go: Update tags rule evaluation to join arrays - collection_service.go: Add strings import New Utilities: - internal/utils/tags.go: Create NormalizeTags(), JoinTags(), SplitTags() - Normalizes tags by trimming, lowercasing, removing duplicates/empties API Documentation: - bruno/media-items/Create Media Item.bru: Update examples to use arrays - bruno/media-items/Update Media Item.bru: Update examples to use arrays - Update docs: tags/contributors now array of string Breaking Change: - JSON format changes from "tags": "tag1,tag2" to "tags": ["tag1", "tag2"] - Tests already use array format (no changes needed) Benefits: - GIN indexes enable faster array searches - Normalization prevents data quality issues (case, duplicates) - Array operations use PostgreSQL native operators (ANY, &&, unnest) - Better separation of concerns (no string parsing in application) |
||
|
john-okeefe
|
00a083b60b |
Rename backend code references: Bookmann → Bookhoard
Backend changes: - Update import paths: bookmann/internal → bookhoard/internal - Rename struct fields: BookmannUUID → BookhoardUUID - Update handler function names: mapContentIdToBookmannUUID → mapContentIdToBookhoardUUID - Update HTTP response headers: X-Bookmann-* → X-Bookhoard-* - Update service and middleware references - Update main.go imports and references This is part 2 of the project rename to Bookhoard. |
||
|
john-okeefe
|
964a4583ab |
feat(services): Add book matching and collection services
- Add book matching service for intelligent book deduplication - Add collection service for collection management - Add test files for book matching and collections |